Presentation on theme: "Developing and Deploying Data Warehouse and Business Intelligence Solutions Kerr-McGee Information Management Group Skye Brannon Jeff BridgwaterSarena."— Presentation transcript:
Developing and Deploying Data Warehouse and Business Intelligence Solutions Kerr-McGee Information Management Group Skye Brannon Jeff BridgwaterSarena Sherrard DW Analyst DW ManagerSr. DW Analyst
Who is Kerr-McGee? Kerr-McGee is an Oklahoma City-based energy and inorganic chemical company with worldwide operations and assets of approximately $10 billion.
Agenda Introduction to DW/BI Concepts Extract, Transform & Load (ETL) Business Intelligence / Reporting A Day in the Life
DW / BI Concepts
Information Management Strategy Structure the systems and data relationships to provide user-friendly customer access to data in order to provide decision-making information.
Adding Value to Data
What is a Data Warehouse? A copy of data from one or more On-line Transaction Processing (OLTP) systems specifically structured for Query, Reporting and Analysis (QRA). Data is typically at a summarized level to limit the size and complexity of the data warehouse Data is usually cleansed and merged to create an “apples to apples” comparisons OLTP Systems Data Warehouse End-User Reporting
The Idea Behind Data Warehousing Extract & Transform Meaningful & Easy Access Data Warehouse POS FINANCIALS INVENTORY Source Systems BILLING ?? Information Data
Metadata Management Project Management & Quality Assurance Source Systems Source Systems External Data Data Transformation Data Transformation Data Extraction Data Cleansing Data Integration Data Improvement Operational Data Store Operational Data Store Data Warehouse Data Warehouse OLAP Query Information Delivery Information Delivery Data Mining Operations & Systems Management Datamarts Enterprise Reporting Framework Architecture
Business Intelligence Information Data Sales Business Activity Marketing Service Customers Manage Inventory Maintain Accounts Product Performance Sales Performance Customer Behavior Marketing Performance Inventory Monitoring Financial Performance Business Intelligence Integrated Meaningful Consistent Validated Easy to Use Leveragable Timely Integrated Meaningful Consistent Validated Easy to Use Leveragable Timely
Issues Multiple Versions of the Truth Different definitions for similar data Multiple Reporting Tools and Reports Impacts Performance of Oracle Financials Corp. Chemical O&G Oracle Financials Passport Domestic Oracle Financials European Oracle Financials Passport Adage Maximo TOLAS Novistar Energy Financials Aberdeen Oracle Financials PREMAS Intl. Systems Aberdeen Data Warehouse O&G Data Warehouse Production Operations Financial Operations Data Warehouses Existing Reporting Systems P2000 DFW Merak Tobin Others InPowerHR Data Warehouse Chemhouse
Corp. Chemical O&G Oracle Financials Passport? Domestic Oracle Financials European Oracle Financials Passport Adage Maximo TOLAS Novistar Energy Aberdeen Oracle Financials PREMAS Intl. Systems Aberdeen Data Warehouse O&G Data Warehouse P2000 DFW Merak Tobin Others “Centralized” Data Warehouse(s) Dashboards & KPI Mgmt Budget & Forecasting Application Peoplesoft Consolidated Analysis & Reporting Solution (Cognos Business Intelligence) Possibly Phased Out or Integrated Possibly Phased Out or Integrated Production Operations Financial Operations Data Warehouses Consolidated Reporting Systems Existing Reporting Systems Benefits Single Version of the Truth Business Intelligence environment leads to Timely Analysis & Reporting Consolidated Tool Enables Multiple Economies of Scale - Massaging & Distribution - Desktop/Maintenance - Transaction System Load’ Chemhouse STANDARDIZE, CONSOLIDATE, MINIMIZE, & SIMPLIFY Phased Out HR Data Warehouse
Manager Planning and management of entire product or project lifecycle; May assist in ETL & BI Interface design and development Data Warehouse Architect – Applies knowledge of technology options, platforms, and design techniques across product and project lifecycle; responsible for design of overall warehouse process ETL Specialist – Analysis and design of extraction, transformation, and loading strategy; development of ETL scripts and procedures Business Intelligence Specialist – Design and development of multidimensional-cubes & reports; performance and tuning of chosen technologies Web Interface Specialist – Design and development of application interface elements; coordinates interfaces between application components Data Warehouse Roles
Plan/Forecast/ Analysis Source Systems Source Systems Data Extract Cleansing and Integration Process Operational Data Store Operational Data Store Data Warehouse Data Warehouse OLAP Query Datamarts Information Analysis Information Analysis Data Mining Data Visualization Data Visualization Global / Dept/ Business Unit Summary and Analysis Global / Dept/ Business Unit Summary and Analysis Metadata Management Executive Information Systems Executive Information Systems External Data Data Transformation Data Transformation Data Extraction and Transformation Applying business rules to turn data into useable information Clean up and standardization of consumers, vendors, products, etc. Integration of disparate internal and external data Can be 70% - 80% of effort Issues - Can be difficult and time consuming to define business rules - Extraction tools automate only the more simple tasks Project Management & Quality Assurance Operations & Systems Management Data Extraction and Transformation
ETL Challenges Warehouse Complexity Destructive vs. Incremental Loads Integration (mergers/acquisitions) –Data integration (consolidation of information) –Process integration (consolidation of processes) Smaller windows of opportunity –Make decision in a shorter period of time due to competitive, global market Global marketplace (DW timing updates) High-profile e-Business initiatives –Satisfying requirements Data Volume Source Inclusion Extract Timing Warehouse Complexity ++=
Challenge to develop efficient, consistent methods of gathering and cleansing heterogeneous data –Capture and load of data from multiple source systems (both internal and external) –Integrates data into a single source –Cross-system mapping to standard identifiers (surrogate keys) –Aggregation for information delivery and BI initiatives ETL - The “Heavy Lifting”
ETL Tools - Only Half the Story Half the story: ETL Tools Extract, Transform, and Load data Transport data between sources and targets Document data element changes (metadata) Administer run-time processes and operations –Scheduling –Error management –Audit logs –Statistics
What is Business Intelligence? Business Intelligence is the transformation of data into information you can use to drive your business. There are a number of vendors that have developed Business Intelligence software. Kerr-McGee uses Cognos.
Source Systems Source Systems Data Extract Cleansing and Integration Process Data Warehouse Data Warehouse Metadata Management Project Management & Quality Assurance External Data Data Transformation Data Transformation Operational Data Store Operational Data Store Business Intelligence Tools Combination of applications and toolsCombination of applications and tools Provide analysis, presentation and reporting facilities for usersProvide analysis, presentation and reporting facilities for users Tailored to meet diverse needs of executives, mgrs, analystsTailored to meet diverse needs of executives, mgrs, analysts Data may reside in ODS, data warehouse or data martData may reside in ODS, data warehouse or data mart IssuesIssues –How do you choose the right tool or tools? Datamarts OLAP Query Plan/Forecast/ Analysis Information Analysis Information Analysis Data Mining Data Visualization Data Visualization Global / Dept/ Business Unit Summary and Analysis Global / Dept/ Business Unit Summary and Analysis Executive Information Systems Executive Information Systems Project Management & Quality Assurance Operations & Systems Management Business Intelligence Tools
Highly Summarized Highly Summarized Moderately Summarized Moderately Summarized Integrated Operational Data Business Users Management Business Analysts Market Researchers Management Business Analysts Market Researchers Executive Categorize Information Needs Financial analysts, product managers, etc.. Senior Management Salespersons, line managers, administrative staff, etc.. Salespersons, line managers, administrative staff, etc..
Information Delivery Mechanisms Operational Trends Web or C/S Wireless Mobile Operational Trends Web or C/S (in millions) Net Revenues $x,xxx Net income xxx x,xxx Earnings per share x.xx Return on net revenues xx% Cash & s/t investments $xxx Total Assets $xxx Shareholder Equity xxx x,xxx Operational Trends Web or C/S Predefined Summaries Predefined Summaries Specialized Algorithms Specialized Algorithms Directed Analysis Ad-hoc Queries Ad-hoc Queries Standardized Operational Reporting Standardized Operational Reporting DataVisualization Delivery Mechanism Considerations Integrated with Operations? Detailed Reporting only? Real-time or based on a Periodic Business Cycle (Financials) Tethered or ‘disconnected’? C/SWebWireless
All things Cognos -Cognos is a vendor. The suite of applications we’ve bought and use from Cognos are: -Access Manager – (Security) -Upfront – (Portal – -PowerPlay – (reports/cubes) -Impromptu – (Web based PDF reports) -NoticeCast – (conditional report notifications) -*Visualizer – (graphic depictions of data warehouse/cube information) *What we will cover.
Terminology Cube - A multidimensional way to analyze information, designed to provide quick answers to the who, what, why, when, and where business questions. Drill Down - Going from a summarized view to a more detailed view of information within the same cube Drill Across - Linking data from One Subject Area to Another (General Ledger to Accounts Payable) Drill Through - Linking to source data using selected filters Powerplay Web - On-Line Analysis Tool for cubes (slice/dice, drill down, drill across & drill through) Newsbox - A web based folder used to store views of data (reports). Every KMBI user has their own personal newsbox.
Cognos - Upfront - Upfront - Portal Management
Cognos - PowerPlay - PowerPlay – web reports/slicing and dicing/data analysis, based on cubes. More Information on Cognos website: website More Information on Cognos website: website
Cognos - Impromptu -Impromptu – printable reports (in PDF) that may/or may not be produced with prompts for filtered information.
Cognos - Visualizer -Visualizer – interactive graphic depictions of data warehouse/cube information
Initial Project meeting should include: –Client - gives input on look and feel, data requirements, timelines –Project Manager – ensures project is feasible within budget and time restraints at the onset and through out the project. –Data Warehouse Architect – ensures all the needed data is in the data warehouse. –Business Intelligence Specialist (cube builder) – ensures all the needed data is in the cube, in the correct format –Web Interface Specialist (visualization builder) – works with cube builder and client to established look and feel, navigation, chart styles, etc.. Developing Visualizations