Why Data Warehouse?

Scenario 1 ABC Pvt. Ltd is a company with branches at Mumbai, Delhi, Chennai and Bangalore. The Sales Manager wants quarterly sales report. Each branch has a separate operational system.

Scenario 1 : ABC Pvt Ltd. Mumbai Delhi Chennai Banglore Sales Manager Sales per item type per branch for first quarter.

Solution 1:ABC Pvt Ltd. Extract sales information from each database. Store the information in a common repository at a single site.

Solution 1:ABC Pvt Ltd. Mumbai Delhi Chennai Banglore Data Warehouse Sales Manager Query & Analysis tools Report

Scenario 2 One Stop Shopping Super Market has huge operational database. Whenever Executives wants some report the OLTP system becomes slow and data entry operators have to wait for some time.

Scenario 2 : One Stop Shopping Operational Database Data Entry Operator Management Wait Report

Solution 2 Extract data needed for analysis from operational database. Store it in another system, the data warehouse. Refresh warehouse at regular intervals so that it contains up to date information for analysis. Warehouse will contain data with historical perspective.

Solution 2 Operational database Data Warehouse Extract data Data Entry Operator Data Entry Operator Manager Transaction Report

Scenario 3 Cakes & Cookies is a small, new company. The chairman of this company wants his company to grow. He needs information so that he can make correct decisions.

Solution 3 Improve the quality of data before loading it into the warehouse. Perform data cleaning and transformation before loading the data. Use query analysis tools to support adhoc queries.

Solution 3 Query & Analysis tool Chairman Expansio n Improvemen t sales time Data Warehouse

Summing up? Why do you need a warehouse? Operational systems could not provide strategic information Executive and managers need such strategic information for Making proper decision Formulating business strategies Establishing goals Setting objectives Monitoring results

Why operational data is not capable of producing valuable information? Data is spread across incompatible structures and systems Not only that, improvements in technology had made computing faster, cheaper and available

FAILURES OF PAST DECISION- SUPPORT SYSTEMS OLTP systems

Decision support systems

Operational and informational

What is Data Warehouse?? Is it the only viable solution

Business intelligence at DW

Functional definition of a DW The data warehouse is an informational environment that Provides an integrated and total view of the enterprise Makes the enterprise’s current and historical information easily available for decision making Makes decision-support transactions possible without hindering operational systems Renders the organization’s information consistent Presents a flexible and interactive source of strategic information

Questions???? Describe five differences between operational systems and informational systems A data warehouse in an environment, not a product. Discuss.

)A data warehouse is not a single software or hardware product that provide the strategic information. It is a computing environment where the users can find strategic information, an environment where the users are put directly in touch with the data to make better decisions. It is a user centric environment. The characteristics of this computing environment are, 1. An ideal environment for data analysis and decision support. 2. Fluid, flexible and interactive 3. It is hundreds percent user-driven 4. Provide the ability to discover answer to complex and unpredictable questions

Inmons’s definition A data warehouse is - subject-oriented, - integrated, - time-variant, - nonvolatile collection of data in support of management’s decision making process.

Subject-oriented Data warehouse is organized around subjects such as sales, product, customer. It focuses on modeling and analysis of data for decision makers. Excludes data not useful in decision support process.

Integration Data Warehouse is constructed by integrating multiple heterogeneous sources. Data Preprocessing are applied to ensure consistency. RDBMS Legacy System Data Warehouse Flat File Data Processing Data Transformation

Integration In terms of data. encoding structures. Measurement of attributes. physical attribute. of data naming conventions. Data type format remarks

Time-variant Provides information from historical perspective, e.g. past 5-10 years Every key structure contains either implicitly or explicitly an element of time, i.e., every record has a timestamp. The time-variant nature in a DW Allows for analysis of the past Relates information to the present Enables forecasts for the future

Non-volatile Data once recorded cannot be updated. Data warehouse requires two operations in data accessing Initial loading of data Incremental loading of data load access

Data Granularity In an operational system, data is usually kept at the lowest level of detail. In a DW, data is summarized at different levels. Three data levels in a banking data warehouse Daily DetailMonthly SummaryQuaterly Summary Account Activity DateMonth AmountNo. of transactions Deposit/ WithdrawWithdrawals Deposits Beginning Balance Ending Balance

Operational v/s Information System FeaturesOperationalInformation CharacteristicsOperational processingInformational processing OrientationTransactionAnalysis UserClerk,DBA,database professional Knowledge workers FunctionDay to day operationDecision support Data ContentCurrentHistorical, archived, derived ViewDetailed, flat relationalSummarized, multidimensional DB designApplication orientedSubject oriented Unit of workShort,simple transactionComplex query AccessRead/writeRead only

Operational v/s Information System FeaturesOperationalInformation FocusData inInformation out No. of records accessed tens/ hundredsmillions Number of usersthousandshundreds DB size100MB to GB100 GB to TB UsagePredictable, repetitiveAd hoc, random, heuristic Response TimeSub-secondsSeveral seconds to minutes PriorityHigh performance,high availability High flexibility,end- user autonomy MetricTransaction throughputQuery throughput

Two approaches in designing a DW Top-down approachBottom-up approach Enterprise view of dataNarrow view of data Inherently architectedInherently incremental Single, central storage of dataFaster implementation of manageable parts Centralized rules and controlEach datamart is developed independently Takes longer time to buildComparatively less time than a DW Higher risk to failureLess risk of failure Needs higher level of cross-functional skills Unmanageable interfaces

Bottom Up Approach

Top Down Approach

A Practical Approach-Kimball 1. Plan and Define requirements 2. Create a surrounding architecture 3. Conform and Standardize the data Content 4. Implement Data Warehouse as series of super-mart one at a time.

Individual Architected Data Marts Common Logical Subject Area ERD Common Business Dimensions Common Business Rules Common Business Metrics Glossary Sales Distribution Product Marketing Customer Accounts Finance Inventory Vendors An Incremental Approach

ArchitectedEnterpriseFoundation Sales Distribution Product Marketing Customer Accounts Finance Inventory Vendors Enterprise Data Warehouse The Eventual Result

Data Warehouse: Holds multiple subject areas Holds very detailed information Works to integrate all data sources Does not necessarily use a dimensional model but feeds dimensional models. Data Mart Often holds only one subject area- for example, Finance, or Sales May hold more summarised data (although many hold full detail) Concentrates on integrating information from a given subject area or set of source systems Is built focused on a dimensional model using a star schema.

Data Warehouse verses data marts