Business Intelligence : a primer Rev April 2012 Introduction & overview The paradigm of BI systems Platforms Appendix Review questions

Our approach to BI BI systems modelling ES taxonomy BI Architecture Enterprise Information Modeling SIRE KPI Identification / mapping HIGO Aggregate Strategic Level (ASL) Plan Exec Mon Dash Rep DSS Ctl Info ES taxonomy GUI Modeling GOA Analytic Information Modeling DFM Rich Semantic Level (RSL) BI systems modelling BI Architecture Software Engineering Interface (SEI) Implementation Level

Business Intelligence: the role within Enterprise Systems Management support Management Information Systems [Planning & Management Control + Business Intelligence ] Front-end systems (Support the life cycle of customers and end products) Back-end systems (Support the cycle of production and delivery) Administrative systems (Finance, HR etc.) Operations support

Acronyms ABC: Activity Base Costing ABM: Activity Based Management BI: Business Intelligence BW: Business Warehouse (synonym of DW) BSC: Balanced Score Card CPM: Corporate Performance Management (synonym of SEM) CRM: Customer Relationship Management CSF: Critical Success Factor DBMS: Data Base Management System DSS: Decision Support System DW: Data Warehouse EIS: Executive Information System EPM: Enterprise Performance Management (synonym of SEM) ERP: Enterprise Resource Planning ERM: Enterprise Resource Management ES: Enterprise System KPI: Key Performance Indicator MBO: Management By Objectives MRP: Manufacturing Resource Management ODS: Operational Data Store OLAP: On Line Analytical Processing OLTP: On Line Transaction Processing SCM: Supply Chain Management SEM: Strategic Enterprise Management

Characteristics of Analytic & Management Information Information is Periodical Output of computation or aggregations Reflects objectives or actual data E.g. data of P& L of an imaginative Car Company come from different transaction processing systems Sales Purchasing Accounting Etc. Therefore, the design of BI / MIS : Is top-own Defines first target data i.e. the variables that BI should process Identifies corresponding source data Defines the process to extract and transform source in target data

The 4-layer paradigm of BI /MIS systems Extraction DATA ENTRY BASI DATI OPERATIVE Transactions Data Bases Tranformation Loading DATA WAREHOUSE Decision support engines (DSS) Presentation / reporting engine (EIS, reporting) Mining & other application engines DATA MART

The 4-layer paradigm of BI /MIS systems BI/MIS applications are based on 4 layers Layer 1 contains source data, typically stored in Transaction Data Base Layer 2 extracts information, and transforms source data into Multi-key & Time-dependent data Layer 3 stores such transformed information Layer 4 processes transformed information according various purposes Support decisions (DSS) E.g. define the sale budget Prepare reports and dashboard (Report) E.g., sales performance Mine stored data (Mining) E.g. identify customer who may churn

Business Intelligence : a primer Rev April 2012 Introduction & overview The paradigm of BI systems Platforms Appendix Review questions

“Jones” case study CONTEXT REQUIREMENTS The Supermarket Chain «Jones» includes 300 shops in 3 regions with 60k items on sale A POS (Point Of Sale) system supports all activities of each shop : item receiving, storing, scrapping, selling Specifically, POS terminals record sales transactions and issue receipts REQUIREMENTS Management want to analyze sales Facts : Sales Measures: amount, quantity, number of tickets Analysis dimensions Date Item Shop Time span : 24 months rolling

Level 1 (source data) «Jones» case study Ticket # 2002a23b11 Store #0021MI Item Des Price Qty Amount #190 Pen 3560 2 7.12 #69 Mat 550 10 5.50 #90 Lib 32000 1 32.00 TOTALE 44.62 Payment Fidelity P. Date 120109 Item Master Data # Item # Store Description Price Qunatity mesuere Stock on hand Stock at the beginning of the day Average forecasted dayly sale Receipt Heading # Store # Ticket Amount Payment Date Receipt detail # Ticket # Item Amount Qty 22

TRANSACTIONS DATABASES Level 2 Extraction includes Select source data Check and clean source data (data cleaning o data cleansing) Staging of extracted data (as needed) Log of extractions Extraction can be Automatic: a batch procedure that runs periodically (e.g. daily, weekly, monthly) Interactive: integrates and fixes automatic data ETL can use intermediate databases Staging Area : where extracted data are temporarily parked (e.g. Data of each individual shop) Operational Data Store (ODS): where granular data are stored and reconciled for future use (e.g. receipt data) DSS Report/ dashboard Mining & other DATA MART DATA WAREHOUSE LOADING DATA ENTRY TRANSFORMATION EXTRACTION TRANSACTIONS DATABASES 23

TRANSACTIONS DATABASES Level 3 Data are stored in Data Warehouse and Data Marts A Data Warehouse is a “subject-oriented, integrated, time-variant (temporal), non volatile collection of summary and detailed data, used to support strategic decision-making process for the enterprise” (Inmon 1996) Data Mart is a smaller warehouse, often a subset or extraction of a warehouse. Warehouse e Mart typically adopt different data schemas DSS Report/ dashboard Mining & other DATA MART DATA WAREHOUSE LOADING DATA ENTRY TRANSFORMATION EXTRACTION TRANSACTIONS DATABASES 25

Level 3 : Data Warehouse Key table 1 Key 1 Attribute 1 Attribute 2 Attribute …. Key table … Key … Attribute 1 Attribute 2 Attribute …. Fact table Key 1 Key 2 Key … Measure 1 Measure 2 Measure …. Key table 2 Key 2 Attribute 1 Attribute 2 Attribute …. The warehouse is typically implemented by relational database, whose schema reflects the corresponding DFM (Dimensional Fact Model). In relational schemas: Fact tables: Store the value of facts (measures) Are identified by multiple keys (K>= 2) Key tables Describe the attributes of dimensions

Level 3: Data Warehouse: star schema Jones case study Shop Shop# Description Shop-class ZIP-code Item Item# Billing-metric Item description Bar-code# Package qty Package-class Supplier-brand Item-class Sales Date# Item# Shop# Sales amount Sales qty Number of receipts Time Date# Week-day Flag work/holyday for local calendar Date in muslim calendar Flag work/holyday for muslim calendar A simple implementation of the DFM is a STAR schema where key tables are implemented only for immediate keys Further analysis / segmentation is obtained by queries on attributes of key tables

Level 3 : Data Warehouse : Snow flake schema Jones Case study Sales Shop ZIP Province-region Area Time Date Holiday Muslim date Chinese date Week-day Item Class Super-class Supplier A full implementation of the DFM requirements implies a snow flake schema with a key table for every hierarchy node

Level 3: design steps The process from extraction up to data warehouse creation is supported by warehouse building tools that are incorporated in most BI platforms 2 Target Data design 3 Mapping of Source Data into Target Data 4 ETL code generation 5 Creation of Data Warehouse 1 Source Data Base Identification The easiest way to understand what a tool can do is to apply it to real-life tasks and problems. This slide illustrates some of the typical tasks performed when designing, building and maintaining a data warehouse with Warehouse Builder. The following slides cover the individual steps in more detail. 6 Data extraction

Level 3: design steps : detail The easiest way to understand what a tool can do is to apply it to real-life tasks and problems. This slide illustrates some of the typical tasks performed when designing, building and maintaining a data warehouse with Warehouse Builder. The following slides cover the individual steps in more detail.

TRANSACTIONS DATABASES Level 3: Data Mart DSS Report/ dashboard Mining & other DATA MART DATA WAREHOUSE LOADING DATA ENTRY TRANSFORMATION Data mart store frequently accessed information From a same warehouse multiple data marts can be created Data marts are typically implemented by hypercube (OLAP technology) EXTRACTION TRANSACTIONS DATABASES 30

From a same warehouse multiple data marts can be created Level 3: Data Mart Customer History Sales Analysis Accounting Marketing From a same warehouse multiple data marts can be created Data Warehouse Shop

Level 3: Data Mart : Hyper-cube : display Columns Pages Facts

Level 3: Data Mart : Hyper-cube : logic Fact An hypercube is a matrix of tables A Fact (e.g. Sales) is identified in a multidimensional space whose axes are Analysis Dimensions (e.g. Shop, Time, Item) An hypercube enables to instantly retrieve complex information e.g. : Sales in last Year (aggregation of Time) by Region (=aggregation of Shops) by Category (= aggregation of Product) Sales Shop Item Time Event Quantity = 20 Amount= 100 Dimension

Level 3: Data Mart : Hyper-cube : logic Shops Item MB21000 MB31000 MB41000 0601 0602 Date Jan Feb Mar Apr Shop Item Month BUDGET MB21000 MB31000 MB41000 0601 0602 Jan Feb 50 55 60 65 45 70 75 ITEM’ SHOP OLAP dimensions = warehouse key MONTH

Level 3: Data Mart : Hyper-cube : logic Dimension Hierarchy Dimensions are arranged in «aggregation hierarchies» (roll-up) Levels of hierarchies are called «dimensional attributes» A multidimensional analysis is performed by navigating trough aggregation levels of dimensions Product Type Category Svelto …. Washing powder Ajax House Cleaning Dash … Soap Palmolive Dairy All Products Bread & Biscuit Food Drinks Tools Hardware Nuts & bolts

Level 3: Data Mart : Hyper-cube : implementation A wise approach to implement multidimensional information is to have an hyper-cube for each measure This easies arithmetic operations and keeps hyper-cubes light FACT TIME Date Item Shop Sales-amount Sales-qty Receipt-number ITEM Shop Prodotto (ch) Prodotto attributi (da def.) PuntoVendita (ch) PuntoVendita attributi (da def.) Tempo (ch) Tempo attributi (da def.) Time Time Time Sales-qty Sales-amount Shop Shop Shop Receipt-number Item Item Item 32

TRANSACTIONS DATABASES Level 4 It processes information for management from various perspectives Define / assess decisions and program (DSS) Present information with a friendly navigation that enables roll up and drill down (EIS & dashboard) Produce structured reports (reporting) Identify trends an pattern in stored information (mining and profiling) DSS Report/ dashboard Mining & other DATA MART DATA WAREHOUSE LOADING DATA ENTRY TRANSFORMATION EXTRACTION TRANSACTIONS DATABASES 30

Leve 4 : reporting Information distribution and privileges handling Format editing Semantic Layer Data Marts Data Bases Data warehouse

Level 4: reporting : semantic layer Purpose: to map data from heterogeneous sources Generally semantic layer includes a set of types e.g.: Dimensions (= warehouse keys) Dimensions attributes ( = key attributes) Measures and Facts

Level 4: reporting : format editing Includes editing functions by which report pages are defined. He content of the report is obtained by dragging an dropping information item from the catalogue of the semantic layer Further activities manage the layout of pages

Level 4: reporting : information distribution

Level 4 : DSS A DSS is a computer based application designed to support semi-structured management decisions by Searching and analyzing information on a collection of sources Compute and assess results (e.g. sensitivity analysis) Typical application fields are: Planning Budgeting Optimization Funding and Investment Decisions ERP / CRM vendors offer DSS suites for corporate planning as Oracle’s EPM and SAP’s BO

Level 4 : DSS : an example (budgeting) The control system produces monthly a financial report and a report with physical performance indicators (KPI) Financial report and KPI report are on 5 dimensions: Time Cost centers Item Sales channel Activity Sales data come from the Sales systems and are stored in a data mart; the same approach is also for sales budget, actual costs and budget costs Data marts are merged in two hyper-cubes, respectively KPI and Financial. Over hyper-cubes a software processes reports on P&L, A&L, Cashflow, KPI

Level 4 : Analysis Engines Data mining applications for research and marketing are designed for Discover in a data base relations and associations previously unknown (“data mining helps end user extract useful business information from large databases” (Berson 1997)). Mining software is a key in marketing to calculate predictive indicators as Churning, Fraud risk, Saving attitude, Economic potential etc. Customer Profiling systems (Analytic CRM)

Business Intelligence : a primer Rev April 2012 Introduction & overview The paradigm of BI systems Platforms Appendix Review questions

BI solutions are offered by all main vendors BI is 5-10% of the ES market Main vendors offer BI products & applications ES vendors Oracle: the largest DB vendor products on Warehousing and applications from vendors acquired (Essbase, Hyperion ) Applications: EPM analogous of SAP’s SEM SAP: the largest ERP vendor Applications: Strategic Enterprise Management (SEM) to support the entire management and analysis life cycle Products : Crystal report, Business Object (founder of reporting paradigm) Microsoft : Office products , SQL server family BI vendors SAS: founder of BI and the largest BI independent vendor, offers a wide range of applications by industry and business area, and specific solutions Microstrategy Open source platforms: e.g. Pentaho

Business Intelligence Platforms : SAS By industry … Education Financial Services Government ….. By solution Analytics Business Analytics Business Intelligence Customer Intelligence Data Management Fraud & Financial Crimes High-Performance Analytics IT & CIO Enablement On Demand Solutions Performance Management Risk Management SAS® 9.3 Supply Chain Intelligence Sustainability Management Featured solutions SAS® 9.3 SAS® Clinical Data Integration SAS® Curriculum Pathways® SAS® Enterprise Guide® SAS® Enterprise Miner™ SAS Fraud Framework for Government SAS® High-Performance Analytics SAS® Inventory Optimization SAS® OnDemand for Academics SAS® Social Media Analytics SAS® Text Analytics SAS® Visual Data Discovery

Business Intelligence : a primer Rev April 2012 Introduction & overview The paradigm of BI systems Platforms Appendix Review questions

Data Warehouse and Data Mart vs Database Conceptual modeling (Rich Semantic Layer) ERA DFM Information type (Master, Event, Analysis) Master + Event Analysis Information organization Normalized (e.g. 3NF) Star or snowflake Hypercube Data schema Relational OLAP or Relational Processing orientation Create + Update Read Typical data operations Insert one individual record or modify one or multiple records Access a vector of records Roll-up, Drill down, Dice Access one ore multiple a vector of records Roll-up, Drill down, Dice Transaction example Enter a customer order Segment customer in Italy with a degree of loyalty >70% by age and region

Business Intelligence : a primer Rev April 2012 Introduction & overview The paradigm of BI systems Platforms Appendix Review questions

Review questions Illustrate the input, process and output of the four layers of BI systems What is an Hyper-cube ? What is a data mart? What is a data warehouse? Compare data warehouse versus classic database in terms of Conceptual modeling (Rich Semantic Layer) Implementation (DB schema) Information type (Master, Event, Analysis) Processing orientation