1. Data Warehouse Architecture Sakthi Angappamudali Data Architect, The Standard @ Oregon State University, Corvallis 16 th May, 2005

2. What is a Data Warehouse? A Data Warehouse is a Subject Oriented Integrated Non-volatile Time variant collection of detailed and summary data used to support the strategic decision making process for the enterprise

3. Characteristics of a Data Warehouse A Data Warehouse has the following characteristics : Purpose - Decision Support Users - Operational personnel, Analysts, Strategists Orientation - Discovery-oriented Integration - Maximum integration Data Quality - Enhanced Data Enrichment - Strategic Volatility - Non-volatile Chronology - Time-variant Granularity - Detailed and summary

4. The Decision Making Roadmap Transaction Systems Decision Support Systems Executive Information Systems Business Planning RUN MANAGEGROW Users Knowledge Brokers Management Operational Functional Current Detailed Dimensional Subject History Summary Analytical Subject History Detailed DataInformation Knowledge VisionActions

5. Design Mapping Design Mapping Source OLTP Systems Raw Detail No/Minimal History Integrated Scrubbed History Summaries Targeted Specialized (OLAP) Data Characteristics System Monitoring Meta Data Extract Scrub Transform Extract Scrub Transform Central Repository Load Index Aggregation Load Index Aggregation Data Warehouse Architected Data Mart Replication Data Set Distribution Replication Data Set Distribution Access & Analysis Resource Scheduling & Distribution Access & Analysis Resource Scheduling & Distribution End User Workstations A Data Warehouse Is A Process

6. Types of Warehousing Solutions Operational Data Store (ODS) –integrated, current, detailed data for operational activities Corporate Information Factory (CIF) –integrated, historic, summary and detailed data for company-wide data analysis Data Mart (DM) –independent, historic, summary data for a small group of business users analyzing a specific business process

7. Operational Source Systems ExtractionSystemsExtractionSystems Operational Data Store Independent Data Mart Data Warehouse Architected Data Mart User Workstations There Are Many Options

8. Solution Choices - ODS Run your business (Operational Data Store) – Tactical –Perform functions not supported in transaction systems –Perform operational reporting (without impacting “real” system) –Data is currently valued (could be real-time as well) –Detail data analysis capabilities –Subject oriented along the lines of the major entities of the corporation –Integrated (physical unification and cohesiveness of the data –Volatile - can be updated as a normal part of processing –Detailed - contains detailed data only

9. Solution Choices - DW Manage your business (Enterprise Warehouse, Data Marts) – Strategic –No business functions are performed (read only) –Aggregations are maintained –Measures and dimensions are defined for slice and dice capabilities –History is maintained for trend analysis –On-line Analytical Processing (OLAP) model

10. A typical E-R Model

11. A typical Star Schema

12. Data Warehouse Systems Operational Transaction Systems Operational Data Store Systems So, When Is Each System Type Used

13. Detailed Data Summary Information + Appropriate Detail Detailed Data + Appropriate Summary Single Function Summary CurrentPoint-in-Time Nearly CurrentPoint-in-Time ContinuallyPeriodically Frequently Periodically Tuned for UpdateTuned for Query Tuned for Production Environment Tuning Not Usually An Issue Operational Source Systems Properties Operational Data Store Data Warehouse Data Mart Contents Timeliness Updated Performance Needs Volatility of Contents Very VolatileNon-VolatileVolatile Non-Volatile What are the differences? May Be Very High Controlled for Performance Moderate Low Amount of Data Accessed

14. Job Schedulers RDBMS Utilities Replication/Distribution Tools Extract//Transform/Load CASE DB Design Repositories Design/Transform/Extract/Aggregate/Monitor/Manage Suites / Environments Database & System Monitors MOLAP/ROLAP Data Mining EIS Data Visualization Metadata Browsers Design Mapping Design Mapping Extract Scrub Transform Extract Scrub Transform Load Index Aggregation Load Index Aggregation Replication Data Set Distribution Replication Data Set Distribution Access & Analysis Resource Scheduling & Distribution Access & Analysis Resource Scheduling & Distribution Meta Data System Monitoring Data Warehouse Tools

15. Data Warehouse Development Methodology W A R E H O U S E P L A N N I N G S T A G E W A R E H O U S E D E V E L O P M E N T S T A G E Methodology and Process Technology Knowledge Team Business Knowledge Team Business Sponsor Maintain Architected Data Mart Deploy Architected Data Mart Develop Architected Data Mart Design Architected Data Mart Define Architected Data Mart Focus Define Architecture Survey User Needs Initiate Project

16. Individual Architected Data Marts Common Logical Subject Area ERD Common Business Dimensions Common Business Rules Common Business Metrics Glossary Sales Distribution Product Marketing Customer Accounts Finance Inventory Vendors An Incremental Approach

17. ArchitectedEnterpriseFoundation Sales Distribution Product Marketing Customer Accounts Finance Inventory Vendors Enterprise Data Warehouse The Eventual Result