DWH-Ahsan Abdullah 1 Data Warehousing Lecture-4 Introduction and Background Virtual University of Pakistan Ahsan Abdullah Assoc. Prof. & Head Center for Agro-Informatics Research FAST National University of Computers & Emerging Sciences, Islamabad

DWH-Ahsan Abdullah 2 Introduction and Background

DWH-Ahsan Abdullah 3 How is it Different?  Starts with a 6x12 availability requirement... but 7x24 usually becomes the goal.  Decision makers typically don’t work 24 hrs a day and 7 days a week. An ATM system does.  Once decision makers start using the DWH, and start reaping the benefits, they start liking it…  Start using the DWH more often, till want it available 100% of the time.

DWH-Ahsan Abdullah 4 How is it Different?  Starts with a 6x12 availability requirement... but 7x24 usually becomes the goal.  For business across the globe, 50% of the world may be sleeping at any one time, but the businesses are up 100% of the time.  100% availability not a trivial task, need to take into account loading strategies, refresh rates etc.

DWH-Ahsan Abdullah 5 How is it Different?  Does not follows the traditional development model Classical SDLC  Requirements gathering  Analysis  Design  Programming  Testing  Integration  Implementation Requirements Program  

DWH-Ahsan Abdullah 6 How is it Different?  Does not follows the traditional development model DWH SDLC (CLDS)  Implement warehouse  Integrate data  Test for biasness  Program w.r.t data  Design DSS system  Analyze results  Understand requirement Requirements Program  DWH

DWH-Ahsan Abdullah 7 Data Warehouse Vs. OLTP OLTP (On Line Transaction Processing) Select tx_date, balance from tx_table Where account_ID = 23876;

DWH-Ahsan Abdullah 8 Data Warehouse Vs. OLTP DWH Select balance, age, sal, gender from customer_table, tx_table Where age between (30 and 40) and Education = ‘graduate’ and CustID.customer_table = Customer_ID.tx_table;

DWH-Ahsan Abdullah 9 Data Warehouse Vs. OLTP OLTPDWH Primary key usedPrimary key NOT used No concept of Primary IndexPrimary index used Few rows returnedMany rows returned May use a single tableUses multiple tables High selectivity of queryLow selectivity of query Indexing on primary key (unique) Indexing on primary index (non-unique)

DWH-Ahsan Abdullah 10 Data Warehouse Vs. OLTP OLTP: OnLine Transaction Processing (MIS or Database System)

DWH-Ahsan Abdullah 11 Comparison of Response Times  On-line analytical processing (OLAP) queries must be executed in a small number of seconds.  Often requires denormalization and/or sampling.  Complex query scripts and large list selections can generally be executed in a small number of minutes.  Sophisticated clustering algorithms (e.g., data mining) can generally be executed in a small number of hours (even for hundreds of thousands of customers).

DWH-Ahsan Abdullah 12 Data Warehouse Server (Tier 1) Data Warehouse Operational Data Bases Semistructured Sources Query/Reporting  Data Marts MOLAP ROLAP Clients (Tier 3) Tools Meta Data Data sources Data (Tier 0)                IT Users   Business Users   Business Users Data Mining  Archived data Analysis  OLAP Servers (Tier 2) Extract Transform Load (ETL)  www data Putting the pieces together