1 IS 605/606: Information Systems Technology Focus Evolution of DSS Introduction to Data Warehousing Dr. Boris Jukić

2  PC/4GL Technology end-users able to directly control data and systems ( notion that data can be used for more than operational transactional purposes )  Management Information System MIS ( later also known as Decision Support Systems-DSS ) defined as processing used to drive management decisions ( as opposed to processing exclusively used to drive detailed operational decisions )  single-database-serving-all-purpose paradigm One DBMS supporting both regular transactional processing and Decision Support Evolution of Decision Support Systems 1980’s

3  Extract Processing Extract Program  rummages through a file or database, uses some criteria for selection, and, upon finding qualified data, transports the data over onto another file or database  used to move data used for DDS out of the way of day-to-day transaction processing systems and to give the DSS user the control over that data Evolution of Decision Support Systems 1980’s: continued

4  Proliferation of Extracts Extracts everywhere (and extracts of extracts, and extracts of extracts of extracts, and …)  Productivity Problems In order to write a corporate report  Data must be located  Lots of customized programs must be written Queries for databases are relatively easy, but the programs that create reports must cross every technology that the company has, such as  Older file systems  Third party data Evolution of Decision Support Systems Early 1990’s

5  Inability to go from Data to Information Due to the following:  Data Applications were built to service the needs of current transaction processing Time horizon of data that is collected and kept accessible is typically 6 moths or so Example; customer monthly stamens may be kept in direct access form for about 5-6 months, assuming that typical customer will only try to resolve billing disputes within that time period  The standard transactional database systems were never designed to hold the historical data needed for DSS analysis Evolution of Decision Support Systems Early 1990’s: continue

6  Lack of Credibility of Data E.g. Account activity report done by 2 analysts No set time basis for data (e.g. one analyst extracts data on Sunday evening, another extracts data on Wednesday afternoon) Algorithmic differential (e.g. one analyst extracts data only on large accounts, another extracts data on all accounts) The levels of extraction (each additional level of extraction increases the probability of discrepancy) External Data (e.g. one analyst is bringing in Wall Street Journal data and another is bringing in Business Week data, and they both strip the data identity) No common source of data to begin with (analysts are extracting data from different databases within the company) E.g. Final Result Analyst A Analyst B Activity is up 10%Activity is down 15% Evolution of Decision Support Systems Early 1990’s: continue

7  Approach Change Recognition that there are two fundamentally different kinds of data  OPERATIONAL DATA (PRIMITIVE DATA)  ANALYTICAL DATA (DERIVED DATA, DSS DATA) Data Warehouses emerge as the New DSS Architecture Evolution of Decision Support Systems Late 1990’s - Current

8 Need for Data Warehousing  Integrated, company-wide view of high-quality information.  Separation of operational and analytical systems and data.

9 Operational DataAnalytical Data Data Differences Typical Time-Horizon: Days/MonthsTypical Time-Horizon: Years DetailedSummarized (and/or Detailed) CurrentValues over time (Snapshots) Technical Differences Can be UpdatedRead (and Append) Only Control of Update: Major IssueControl of Update: No Issue Small Amounts used in a ProcessLarge Amounts used in a Process Non-RedundantRedundancy not an Issue High frequency of AccessLow/Modest frequency of Access Purpose Differences For “Clerical Community”For “Managerial Community” Supports Day-to-Day OperationsSupports Managerial Needs Application OrientedSubject Oriented OPERATIONAL vs. ANALYTICAL DATA

10 Application vs. Subject Oriented

11 Application vs. Subject Oriented

12 Application vs. Subject Oriented

13 OPERATIONAL vs. ANALYTICAL DATA Hardware Utilization (Frequency of Access) Operational Data Warehouse

14 Data Warehouse: Definition  Data Warehouse: An enterprise-wide structured repository of subject- oriented, time-variant, historical data used for information retrieval and decision support. The data warehouse stores atomic and summary data. (Bill Inmon, paraphrased by Oracle Data Warehouse Method)

15  Data Warehouse: An enterprise-wide structured repository of subject-oriented, time-variant, historical data used for information retrieval and decision support. The data warehouse stores atomic and summary data. (Oracle Data Warehouse Method)  enterprise-wide refers to the fact that a DW provides a company-wide view of the information it contains Data Warehouse: Definition

16  Data Warehouse: An enterprise-wide structured repository of subject-oriented, time-variant, historical data used for information retrieval and decision support. The data warehouse stores atomic and summary data. (Oracle Data Warehouse Method)  structured repository refers to the fact that a DW is a structured data repository like any other data base Data Warehouse: Definition

17  Data Warehouse: An enterprise-wide structured repository of subject-oriented, time-variant, historical data used for information retrieval and decision support. The data warehouse stores atomic and summary data. (Oracle Data Warehouse Method)  subject-oriented refers to the fundamental difference in the purpose of a traditional Database System and a DW. Traditional Database System is developed in order to support a specific business process (e.g. shipping company order-entry database, dental office appointment management database). DW is developed to analyze a specific subject area (e.g. sales, profit). Data Warehouse: Definition

18  Data Warehouse: An enterprise-wide structured repository of subject-oriented, time-variant, historical data used for information retrieval and decision support. The data warehouse stores atomic and summary data. (Oracle Data Warehouse Method)  time-variant refers to the fact that a DW contains slices of data across different periods of time. With these data slices, the user can view reports from now and in the past. Data Warehouse: Definition

19  Data Warehouse: An enterprise-wide structured repository of subject-oriented, time-variant, historical data used for information retrieval and decision support. The data warehouse stores atomic and summary data. (Oracle Data Warehouse Method)  historical refers to the fact that a DW typically contains several years worth of data (as opposed to days typical time horizon for data in traditional operational databases). In fact, DW often does not contain the latest transactional data. Data Warehouse: Definition

20  Data Warehouse: An enterprise-wide structured repository of subject-oriented, time-variant, historical data used for information retrieval and decision support. The data warehouse stores atomic and summary data. (Oracle Data Warehouse Method)  information retrieval and decision support refers to the fact that a DW is a facility for getting information to answer questions. It is not meant for direct data entry; batch updates are the norm for refreshing data warehouses. Data Warehouse: Definition

21  Data Warehouse: An enterprise-wide structured repository of subject-oriented, time-variant, historical data used for information retrieval and decision support. The data warehouse stores atomic and summary data. (Oracle Data Warehouse Method)  atomic and summary data refers to the fact that a DW, depending on purpose, may contain atomic data, summary data, or both. Data Warehouse: Definition

22 DW is Subject Oriented!  Data is organized around major subject areas of an enterprise, and is therefore useful for an enterprise-wide understanding of those subjects  E.g. a banking operational system keeps independent records of customer savings, loans, and other transactions. A warehouse pulls this independent data together to provide customer financial information (fees, profits, losses …)  Examples of subject areas Customer Financial Information Toll calls made in the telecommunications industry company Airline passenger booking information  Data from operational systems must be transformed so that is consistent and meaningful in the DW

Subject Orientation Applications auto life health casualty Insurance Co. Operational Databases Data Warehouse claim Subject

Subject Orientation Applications auto life health casualty premium claim Subjects Insurance Co. Operational Databases Data Warehouse

25 DW is Integrated!  In many organizations, data resides in diverse independent systems, making it difficult to acquire meaningful information for analysis.  In DW data is completely integrated, even when the underlying sources store data differently  There is no magic wand: the transformation and integration process (which involves ETL – extraction, transformation, and transportation-load) can be time consuming and costly, and it requires commitment from every part of the organization, particularly top-level managers

26 Common Example of a Data Warehouse Purpose  Data warehouse is often designed and implemented to answer these TWO fundamental questions: Who is buying what? When and where are they doing so?  More specific Who [which customer] is buying [buying / using / delivering / shipping / ordering / returning] what [products / services] from where [outlet / store / clinic / branch] on what occasion [when], how [credit card / cash / check / exchange / debit] and why [causation]?

27 Some Uses of a Data Warehouse  Airlines for aircraft deployment, analysis of route profitability, frequent flyer promotions, and maintenance  Banking for promotion of products and services, and customer service  Health care for cost reduction  Investment and insurance companies for customer analysis, risk assessment, and portfolio management  Retail stores for buying pattern analysis, promotions, customer profiling, and pricing  Telecommunications for product and service promotions.

Source: Oracle Corp. 1999