2 Copyright © 2006, Oracle. All rights reserved. Defining Data Warehouse Concepts and Terminology

Copyright © 2006, Oracle. All rights reserved Objectives After completing this lesson, you should be able to do the following: Identify a common, broadly accepted definition of a data warehouse Describe the differences of dependent and independent data marts Identify some of the main warehouse development approaches Define some of the operational properties and common terminology of a data warehouse

Copyright © 2006, Oracle. All rights reserved Data Warehouse: Definition “A data warehouse is a subject-oriented, integrated, non-volatile, and time-variant collection of data in support of management’s decisions.” — W.H. Inmon “An enterprise structured repository of subject-oriented, time-variant, historical data used for information retrieval and decision support. The data warehouse stores atomic and summary data.” — Oracle’s definition of a data warehouse

Copyright © 2006, Oracle. All rights reserved

Copyright © 2006, Oracle. All rights reserved Data Warehouse Properties Integrated Time variant Nonvolatile Subject oriented Data Warehouse

Copyright © 2006, Oracle. All rights reserved Subject Oriented Data is categorized and stored by business subject rather than by application. OLTP applications Equity plans Shares Insurance Loans Savings Data warehouse subject Customer financial information

Copyright © 2006, Oracle. All rights reserved Integrated Data on a given subject is defined and stored once. Data WarehouseOLTP applications Customer Savings Current accounts Loans

Copyright © 2006, Oracle. All rights reserved

Copyright © 2006, Oracle. All rights reserved Data warehouse Time Variant Data is stored as a series of snapshots, each representing a period of time.

Copyright © 2006, Oracle. All rights reserved Nonvolatile Typically, data in the data warehouse is not updated or deleted. Warehouse Read Load Operational Insert, update, delete, or read

Copyright © 2006, Oracle. All rights reserved Changing Warehouse Data Operational databasesWarehouse database First-time load Refresh Purge or archive

Copyright © 2006, Oracle. All rights reserved Data Warehouse Versus OLTP AnalysisProcessesActivities Operational, internal, external Operational, internalData sources Large to very largeSmall to largeSize Subject, timeApplicationData organization Snapshots over time30–60 daysNature of data Primarily read-onlyDMLOperations Seconds to hoursSub-seconds to seconds Response time Data WarehouseOLTPProperty

Copyright © 2006, Oracle. All rights reserved

Copyright © 2006, Oracle. All rights reserved Enterprisewide Data Warehouse Large scale implementation Scopes the entire business Data from all subject areas Developed incrementally Single source of enterprisewide data Synchronized enterprisewide data Single distribution point to dependent data marts

Copyright © 2006, Oracle. All rights reserved Data Warehouses Versus Data Marts MonthsMonths to yearsImplementation time FewManyData source Single-subject, LOBMultipleSubjects DepartmentEnterpriseScope Data martData WarehouseProperty

Copyright © 2006, Oracle. All rights reserved

Copyright © 2006, Oracle. All rights reserved Dependent Data Mart Data Warehouse Data marts Flat files Marketing Sales Finance Marketing Sales Finance HR Operational systems External data Operations data Legacy data External data

Copyright © 2006, Oracle. All rights reserved Independent Data Mart Sales or marketing Flat files Operational systems External data Operations data Legacy data External data

Copyright © 2006, Oracle. All rights reserved Typical Data Warehouse Components Source systems Staging area Presentation area Access tools ODS Operational External Legacy Metadata repository Data marts Data Warehouse

Copyright © 2006, Oracle. All rights reserved

Copyright © 2006, Oracle. All rights reserved Warehouse Development Approaches “Big bang” approach Incremental approach: –Top-down incremental approach –Bottom-up incremental approach

Copyright © 2006, Oracle. All rights reserved “Big Bang” Approach Analyze enterprise requirements. Build enterprise data warehouse. Report in subsets or store in data marts.

Copyright © 2006, Oracle. All rights reserved

Copyright © 2006, Oracle. All rights reserved Top-Down Approach Analyze requirements at the enterprise level. Develop conceptual information model. Identify and prioritize subject areas. Complete a model of selected subject area. Map to available data. Perform a source system analysis. Implement base technical architecture. Establish metadata, extraction, and load processes for the initial subject area. Create and populate the initial subject area data mart within the overall warehouse framework.

Copyright © 2006, Oracle. All rights reserved Bottom-Up Approach Define the scope and coverage of the data warehouse and analyze the source systems within this scope. Define the initial increment based on the political pressure, assumed business benefit, and data volume. Implement base technical architecture and establish metadata, extraction, and load processes as required by increment. Create and populate the initial subject areas within the overall warehouse framework.

Copyright © 2006, Oracle. All rights reserved

Copyright © 2006, Oracle. All rights reserved Incremental Approach to Warehouse Development Multiple iterations Shorter implementations Validation of each phase Strategy Definition Analysis Design Build Production Increment 1 Iterative

Copyright © 2006, Oracle. All rights reserved Data Warehousing Process Components Methodology Architecture Extraction, transformation, and loading (ETL) Implementation Operation and support

Copyright © 2006, Oracle. All rights reserved Methodology Ensures a successful data warehouse Encourages incremental development Provides a staged approach to an enterprisewide warehouse that is: –Safe –Manageable –Proven –Recommended

Copyright © 2006, Oracle. All rights reserved Architecture “Provides the planning, structure, and standardization needed to ensure integration of multiple components, projects, and processes across time.” “Establishes the framework, standards, and procedures for the data warehouse at an enterprise level.” — The Data Warehousing Institute

Copyright © 2006, Oracle. All rights reserved Extraction, Transformation, and Loading (ETL) “Effective data extract, transform, and load (ETL) processes represent the number one success factor for your data warehouse project and can absorb up to 70 percent of the time spent on a typical data warehousing project.” — DM Review SourceTargetStaging area

Copyright © 2006, Oracle. All rights reserved Implementation Data Warehouse Architecture Implementation e.g., Incremental Implementation Increment 1 Increment 2 Increment n...

Copyright © 2006, Oracle. All rights reserved Operation and Support Data access and reporting Refreshing warehouse data Monitoring Responding to change

Copyright © 2006, Oracle. All rights reserved Phases of the Incremental Approach Strategy Definition Analysis Design Build Production Increment 1 Strategy Definition Analysis Design Build Production

Copyright © 2006, Oracle. All rights reserved

Copyright © 2006, Oracle. All rights reserved Strategy Phase Deliverables Business goals and objectives Data warehouse purpose, objectives, and scope Enterprise data warehouse logical model Incremental milestones Source systems data flows Subject area gap analysis

Copyright © 2006, Oracle. All rights reserved Strategy Phase Deliverables Data acquisition strategy Data quality strategy Metadata strategy Data access environment Training strategy

Copyright © 2006, Oracle. All rights reserved Sales History ( SH ) Schema CUSTOMERS rows COUNTRIES 23 rows CHANNELS 5 rows PRODUCTS 72 rows TIMES 1826 rows SALES rows PROMOTIONS rows COSTS rows

Copyright © 2006, Oracle. All rights reserved Introducing the Case Study: Roy Independent School District (RISD) In January 2000, RISD and Oracle representatives met to discuss the details about the RISD Data Warehouse (RISD DW) project: Oracle to develop technical architecture for development, test, and production instance of RISD Data Warehouse Oracle to develop the logical and physical data models RISD responsible for data cleansing ETL process to be designed RISD DW project to support Student Information System (SIS) Reports to be created based on subject areas, and to be integrated with Portal Data access security to be implemented based on user roles

Copyright © 2006, Oracle. All rights reserved

Copyright © 2006, Oracle. All rights reserved Summary In this lesson, you should have learned how to: Identify a common, broadly accepted definition of a data warehouse Describe the differences of dependent and independent data marts Identify some of the main warehouse development approaches Recognize some of the operational properties and common terminology of a data warehouse

Copyright © 2006, Oracle. All rights reserved Practice 2-1: Overview This practice covers the following topics: Answering questions regarding the data warehousing concept and terminology Discussing some of the data warehouse concepts and terminology Discussing the case study to understand the requirements of the system

Copyright © 2006, Oracle. All rights reserved

Copyright © 2006, Oracle. All rights reserved