Presentation on theme: "Kerr-McGee Information Management Group"— Presentation transcript:
1Kerr-McGee Information Management Group Developing and Deploying Data Warehouse and Business Intelligence SolutionsKerr-McGee Information Management GroupSkye Brannon Jeff Bridgwater Sarena SherrardDW Analyst DW Manager Sr. DW Analyst
2Who is Kerr-McGee?Kerr-McGee is an Oklahoma City-based energy and inorganic chemical company with worldwide operations and assets of approximately $10 billion.
3Agenda Introduction to DW/BI Concepts Extract, Transform & Load (ETL) Business Intelligence / ReportingA Day in the Life
8What is a Data Warehouse? A copy of data from one or more On-line Transaction Processing (OLTP) systems specifically structured for Query, Reporting and Analysis (QRA).Data is typically at a summarized level to limit the size and complexity of the data warehouseData is usually cleansed and merged to create an “apples to apples” comparisonsEnd-UserReportingOLTPSystemsDataWarehouse
9The Idea Behind Data Warehousing POSFINANCIALSINVENTORYSource SystemsBILLING?Data WarehouseMeaningful&EasyAccessExtract&TransformDataInformation
12K-M As-Is Corp. Chemical O&G Issues Multiple Versions of the Truth OracleFinancialsPassportInPowerHR DataWarehouseChemicalDomesticOracleFinancialsProductionOperationsPassportFinancialOperationsAdageEuropeanOracleFinancialsDataWarehousesMaximoChemhouseExistingReportingSystemsTOLASO&GO&GDataWarehouseTobinNovistarEnergyFinancialsIssuesMultiple Versions of the TruthDifferent definitions for similar dataMultiple Reporting Tools and ReportsImpacts Performance of Oracle FinancialsDFWPREMASP2000AberdeenOracleFinancialsMerakOthersIntl.SystemsAberdeenDataWarehouse
13STANDARDIZE, CONSOLIDATE, MINIMIZE, & SIMPLIFY Corp.HR DataWarehouseK-M To-BeOracleFinancialsPhased OutPassport?PeoplesoftProductionOperationsBudget &ForecastingApplicationChemical“Centralized”Data Warehouse(s)PassportDomesticOracleFinancialsFinancialOperationsConsolidatedAnalysis & Reporting Solution(CognosBusiness Intelligence)AdageDataWarehousesEuropeanOracleFinancialsMaximoExistingReportingSystemsDashboards& KPI MgmtTOLASChemhouseConsolidatedReportingSystemsO&GNovistarEnergyBenefitsSingle Version of theTruthBusiness Intelligence environment leads to Timely Analysis &ReportingConsolidated ToolEnables MultipleEconomies of Scale- Massaging & Distribution- Desktop/Maintenance- Transaction System Load’TobinDFWO&GDataWarehousePossiblyPhased Out or IntegratedPREMASP2000AberdeenOracleFinancialsAberdeenDataWarehouseMerakPossiblyPhased Out or IntegratedIntl.SystemsOthersSTANDARDIZE, CONSOLIDATE, MINIMIZE, & SIMPLIFY
14Data Warehouse RolesManager Planning and management of entire product or project lifecycle; May assist in ETL & BI Interface design and developmentData Warehouse Architect – Applies knowledge of technology options, platforms, and design techniques across product and project lifecycle; responsible for design of overall warehouse processETL Specialist – Analysis and design of extraction, transformation, and loading strategy; development of ETL scripts and proceduresBusiness Intelligence Specialist – Design and development of multidimensional-cubes & reports; performance and tuning of chosen technologiesWeb Interface Specialist – Design and development of application interface elements; coordinates interfaces between application components
16Project Management & Quality Assurance Operations & Systems Management Data Extraction and TransformationMetadata ManagementData Extraction and TransformationApplying business rules to turn data into useable informationClean up and standardization of consumers, vendors, products, etc.Integration of disparate internal and external dataCan be 70% - 80% of effortIssues- Can be difficult and time consuming to define business rules- Extraction tools automate only the more simple tasksPlan/Forecast/ AnalysisDataExtractCleansingandIntegrationProcessExternal DataOLAPQueryGlobal / Dept/ Business Unit Summary andAnalysisDatamartsDataVisualizationDataMiningExecutiveInformationSystemsDataTransformationOperationalData StoreDataWarehouseSourceSystemsInformationAnalysisProject Management & Quality AssuranceOperations & Systems Management
17+ + = ETL Challenges Data Volume Source Inclusion Extract Timing WarehouseComplexityWarehouse ComplexityDestructive vs. Incremental LoadsIntegration (mergers/acquisitions)Data integration (consolidation of information)Process integration (consolidation of processes)Smaller windows of opportunityMake decision in a shorter period of time due to competitive, global marketGlobal marketplace (DW timing updates)High-profile e-Business initiativesSatisfying requirementsNo ONE correct answer/solution
18ETL - The “Heavy Lifting” Challenge to develop efficient, consistent methods of gathering and cleansing heterogeneous dataCapture and load of data from multiple source systems (both internal and external)Integrates data into a single sourceCross-system mapping to standard identifiers (surrogate keys)Aggregation for information delivery and BI initiatives
19ETL Tools - Only Half the Story Half the story: ETL Tools Extract, Transform, and Load dataTransport data between sources and targetsDocument data element changes (metadata)Administer run-time processes and operationsSchedulingError managementAudit logsStatistics
25What is Business Intelligence? Business Intelligence is the transformation of data into information you can use to drive your business.There are a number of vendors that have developed Business Intelligence software. Kerr-McGee uses Cognos.
26Business Intelligence Tools Metadata ManagementBusiness Intelligence ToolsCombination of applications and toolsProvide analysis, presentation and reporting facilities for usersTailored to meet diverse needs of executives, mgrs, analystsData may reside in ODS, data warehouse or data martIssuesHow do you choose the right tool or tools?Plan/Forecast/ AnalysisDataExtractCleansingandIntegrationProcessExternal DataGlobal / Dept/ Business Unit Summary andAnalysisOLAPQueryDatamartsDataVisualizationDataMiningExecutiveInformationSystemsDataTransformationOperationalData StoreDataWarehouseSourceSystemsInformationAnalysisProject Management & Quality AssuranceProject Management & Quality AssuranceOperations & Systems Management
27Categorize Information Needs HighlySummarizedModeratelyIntegrated Operational DataExecutiveSenior ManagementManagementBusiness AnalystsMarket ResearchersFinancial analysts, product managers, etc..Business UsersDuring Requirements Gathering characterize the information access needs of the target audience(s) for the solution. Basic questions:What business process /activities am I focusing on as part of the scope?What is the data?How do people across an organization want to access, view and manipulate it?What’s the primary audience for the application?Executive:top down view of the businessmonitoring the overall effectiveness of the organization in predictable waysBusiness Analysts:analyze information from a number of different perspectivesanalysis can be aided by ability to view key aspects of a business in a hierarchical fashionProduct: Product Groups, Brands, ProductsGeography: Regions, States, CitiesTime: Year, Quarter, Month, Week, DayBusiness Users:Hard copy reports (e.g. customer lists, sales totals, etc. )Exploratory questions and answers (e.g.. customers in Texas spending > 500 at a store)Salespersons,line managers,administrative staff, etc..
28Information Delivery Mechanisms WirelessMobileOperational TrendsWeb orC/SDelivery Mechanism ConsiderationsIntegrated with Operations?Detailed Reporting only?Real-time or based on a Periodic Business Cycle (Financials)Tethered or ‘disconnected’?C/SWebWirelessPredefinedSummariesDataVisualizationOperational TrendsWeb orC/SSpecializedAlgorithmsDirected AnalysisDetermine appropriate delivery mechanisms to satisfy information access needs.Standardized Reporting:regularly scheduled, monthly, quarterlyparameter driven on demand (e.g.. enter date range, division, etc..)example: promotion reports, contributing zipsAdhoc Queriesunplanned information access to support unexpected business questionsexample: generate a report of all sweater buyers in xxx storesUser Directed Analysisanalysis of key aspects of a business along natural business hierarchies (product, geography, time, etc.. )looking for exceptions & anomalies, the process is largely user drivenexample: J&J product tracking by customer over timeSpecialized Algorithmsuncovering hidden relationships & correlations between different pieces of datafar less user directed, the idea is to use algorithms to help identify key factsexample: discovering correlations between doctors and types of proceduresExecutivedrawing pretty pictures and charts of interesting key high level measures(in millions)19981999200020012002Net Revenues$x,xxxNet incomexxxx,xxxEarnings per sharex.xxReturn on net revenuesxx%Cash & s/t investments$xxxTotal AssetsShareholder EquityOperational TrendsWeb orC/SStandardizedOperationalReportingAd-hocQueries
29B.I. InfrastructureProduction environment consists of 6 load balanced servers within a DMZ. These include;A Web Server (Primary and Secondary)A Reports Server (Primary and Secondary, NT today SUN Enterprise tomorrow)An Application Server (Primary and Secondary)Outside of the DMZ the actual warehouse resides on a cluster of Sun Enterprise 3500 servers attached to a 100GB EMC RAID Array.Keep in mind that duplicate environments for Development, Test and Production were created.
30Access Manager – (Security) Upfront – (Portal – http:/intranet/kmbi) All things CognosCognos is a vendor. The suite of applications we’ve bought and use from Cognos are:Access Manager – (Security)Upfront – (Portal –PowerPlay – (reports/cubes)Impromptu – (Web based PDF reports)NoticeCast – (conditional report notifications)*Visualizer – (graphic depictions of data warehouse/cube information)*What we will cover.
31TerminologyCube - A multidimensional way to analyze information, designed to provide quick answers to the who, what, why, when, and where business questions.Drill Down - Going from a summarized view to a more detailed view of information within the same cubeDrill Across - Linking data from One Subject Area to Another (General Ledger to Accounts Payable)Drill Through - Linking to source data using selected filtersPowerplay Web - On-Line Analysis Tool for cubes (slice/dice, drill down, drill across & drill through)Newsbox -A web based folder used to store views of data (reports). Every KMBI user has their own personal newsbox.
33Cognos - PowerPlayPowerPlay – web reports/slicing and dicing/data analysis, based on cubes.More Information on Cognos website:
34Cognos - ImpromptuImpromptu – printable reports (in PDF) that may/or may not be produced with prompts for filtered information.
35Cognos - VisualizerVisualizer – interactive graphic depictions of data warehouse/cube information
36Developing Visualizations Initial Project meeting should include:Client - gives input on look and feel, data requirements, timelinesProject Manager – ensures project is feasible within budget and time restraints at the onset and through out the project.Data Warehouse Architect – ensures all the needed data is in the data warehouse.Business Intelligence Specialist (cube builder) – ensures all the needed data is in the cube, in the correct formatWeb Interface Specialist (visualization builder) – works with cube builder and client to established look and feel, navigation, chart styles, etc..