InfoSphere Information Server: Trends & Tactics for Improving Data Quality of your Business Intelligence Solution

Session Abstract Many organizations struggle with broader user adoption of Business Intelligence and Performance Management due to a lack of trust in their data, and the inability to deliver the breadth, speed and consolidated information perspective necessary to keep pace with the business. This "how-to" session will discuss how to enhance your existing and planned Cognos initiatives by addressing the need for on-time delivery of trusted information. Specifically, learn how to leverage the IBM InfoSphere product portfolio, as a foundation for Cognos 8 BI, to immediately address your data quality; real-time information integration and data warehousing challenges to drive more business value. 2

Performance Management Challenges Faced How to deliver: quality information from fragmented, disparate systems at volume and velocity required by the business? How to address the diverse needs of everyone in the business with a complete, consistent view of information? How to establish standards, governance, and breakdown barriers to establish an IT-business partnership Business Challenge Information Challenge Process Challenge 3

Increasing Focus on Data Quality Businesses are beginning to realize that data quality issues not only cost them time and money, but also inhibit their ability to address core strategic projects More and more businesses are establishing programs for data quality, to measure and improve the reliability of information Analysts contend that companies with focused data quality programs will find more opportunities to outperform their peers 4

Why Does this Problem Exist? Most enterprises are running distinct sales, services, marketing, manufacturing and financial applications, each with its own master reference data. No one system is the universally agreed-to system of record. Enterprise Application Vendors do not guarantee a complete & accurate integrated view – they point to their dependence on the quality of the raw input data Data quality continues to erode at the point of entry, though it is not a data entry problem 5

Business Drivers for Investment Depend on Data Quality Empowering risk and compliance initiatives with the information they require Optimizing Revenue Opportunities by ensuring effective and efficient interactions with customers, partners, and suppliers Enabling collaborative business processes with consistent and trustworthy information Reducing the total cost of ownership for maintaining consistent information across the enterprise 6

What is the Impact of Poor Data Quality? Lost Sales Opportunity SKU misplaced or hard to find Out of stocks attributed to the store Hard Losses Lost potential for cross-sell and up-sell (staff not trained or available) Reduced store visit frequency Abandoned carts (poor service or excessive queues) Soft Losses 1.5% 1.7% 2-4% 1-3% 1-2% Total 7.2%- 12% Source: GMA/FMI/CIES 2003 (US grocery), ECR Europe 2003, Lineraires.com, California Management Review, IBM case studies, interviews and IBM Institute for Business Value analysis 7

Data Quality is a Subjective Business Standard Data = facts used as a basis for decision making suitable for storage on a computer Quality = the general standard or grade of something Data Quality = a subjective standard used to determine if a set of facts is suitable for a particular business purpose Relevant? Accurate? Valid? Complete? Business Purpose Ultimately, Data Quality = Trust 8

So, What Constitutes Data Quality? Data is standardized Data is fit for purpose (conforms to rules) Each record is unique View of information is complete Records are certified against authoritative sources Lineage is understood Data quality is measured over time 9

What Do You Need to Establish a Data Quality Program? A foundation platform that centralizes quality rules and provides auditable data quality Business-driven, data-centric design environment for data quality rules An ongoing process for data quality A way to measure quality over time Universal deployment of quality rules across all points of entry Data quality ownership and data governance Management sponsorship and a corporate mandate for data quality improvement 10

Common Data Problems Lack of information standards - d ifferent formats & structures across different systems Data surprises in individual fields - d ata misplaced in the database Information buried in free- form fields Data myopia - l ack of consistent identifiers inhibit a single view The redundancy nightmare - d uplicate records with a lack of standards Kate A. Roberts 416 Columbus Ave #2, Boston, Mass Catherine Roberts Four sixteen Columbus APT2, Boston, MA Mrs. K. Roberts 416 Columbus Suite #2, Suffolk County Name Tax ID Telephone J Smith DBA Lime Cons Williams & Co. C/O Bill st Natl Provident HP 15 State St Orlando WING ASSY DRILL 4 HOLE USE 5J868A HEXBOLT 1/4 INCH WING ASSEMBY, USE 5J868-A HEX BOLT.25 - DRILL FOUR HOLES USE 4 5J868A BOLTS (HEX.25) - DRILL HOLES FOR EA ON WING ASSEM RUDER, TAP 6 WHOLES, SECURE W/KL2301 RIVETS (10 CM) RS232 Cable 6' M-F CandS CS ft. Cable Male-F, RS232 #87951 C&SUCH6Male/Female 25 PIN 6 Foot Cable IBM 187 N.Pk. Str. Salem NH I.B.M. Inc. 187 N.Pk. St. Salem NH Int. Bus. Machines 187 No. Park St Salem NH International Bus. M. 187 Park Ave Salem NH Inter-Nation Consults 15 Main Street Andover MA I.B. Manufacturing Park Blvd. Bostno MA

A Platform for Data Quality 12

A Process For Data Quality Establish Data Quality Ownership & Sponsorship Analyze Source Data Measure & Baseline Data Quality Standardize Certify & Enrich Match Link or Survive Re-Measure Report Understanding Data Quality Enforcing Data Quality Standards Monitoring Data Quality 13

Analyzes data structure, Quality Controls for Completeness and Validity of data values Incomplete or Invalid values set by value, range, or reference sources Consistency checks for data formats Removes duplicates Cross-references matching records Survives a single complete record Cleanses and enriches data Understanding and Monitoring Data Quality Enforcing Data Quality Standards Data Quality Capabilities 14

Understanding Data Quality: Data Quality Assessment Methodology Define clear business problem statement Increase revenue by cross selling more effectively our services to all clients Reduce materials costs by negotiating better prices from our suppliers Reduce parts inventory across our manufacturing plants Reduce IT costs and improve service levels by consolidating overlapping applications Over 5 days, our technical experts analyze data that supports your business problem statement IBM and customer map issues to relevant data samples Agree scope of measures and customer provides data sample: e.g., 4 or 5 key tables and 5-10 key columns IBM analyzes the data Column usage and completeness Compliance with business formats Variation in standards Range and outliers Incidence of duplicates Data Quality Analysis Business Subject Matter Expert Data Steward InfoSphere Information Analyzer 15

Understanding Data Quality: Assessment Outcomes Management report and presentation of findings Identify Performance Management project exposures Optional follow-on workshops Regulatory exposures Data Discovery Quantitative results Data completeness and format issues Business rule compliance Data Quality Baseline The DQA sets a shared baseline platform for an ongoing data quality improvement initiative (data governance) or tactical remedial project Case Study: Pharmaceutical company The Tipping Point – unable to get a consolidated view of data. Report accuracy was suspect. The Hurdle – marketing and sales data warehouse contained many data quality issues The Result – using IBM InfoSphere Information Analyzer and IBM InfoSphere QualityStage they reduced development time and their reports now support better targeted marketing

Enforcing Data Quality Standards: Investigation Parsing: Separating multi-valued fields into individual pieces 123 | St. | Virginia | St. Virginia Lexical analysis: Determining business significance of individual pieces Context Sensitive: Identifying various data structures and content Number Street Alpha Street Type Type 123 | St. | Virginia | St. House Street Number Street Name Type 123 | St. Virginia | St. 123St.St. The instructions for handling the data are inherent within the data itself. 17

Enforcing Data Quality Standards: Standardization Input File: Address Line 1Address Line N MILLS AVENUE ORLANDO, FLA W MAIN STR, CUMMING, GA WEST CENTRAL AV TOLEDO OH HEARD AVE AUGUSTA-GA GREENE ST ACCT #1234 AUGUSTA GEORGIA OWENS ROAD SUITE 536 EVANSGA Result File: House # DirStr. NameTypeUnitNo.NYSIISCitySOUNDEXStateZipACCT# 639N MILLSAVEMALORLANDOO645FL W MAINSTMAN CUMMINGC552GA W CENTRALAVECANTRAL TOLEDOT430OH HEARDAVEHAD AUGUSTA A223GA GREENESTGRAN AUGUSTAA223GA OWENSRDSTE 536ON EVANS E152GA Results in strongly typed fixed fielded standardized data 18

Enforcing Data Quality Standards: Matching Clerical review Record linkage Survivorship Append/Fix sources ? Cross-reference = 19

Lessons Learned and Best Practice Recruit an executive sponsor Signals that the initiative is important Assures that funds continue to be available Discourages other business units from implementing conflicting projects Convene a data quality working group Assess and report on quality early in the process May coincide with implementation teams or data warehousing teams Business leads, but IT coordinates and facilitates Strive for consensus Have the business appoint a data quality steward for each business unit For business units with large user populations, several stewards are appropriate 20

Summary Data quality is becoming an increasingly important organizational issue Improving data quality and ensuring information delivery requires a focused programmatic and varied approach At the core of any data quality program is a platform capable of providing auditable data quality assessment services IBM InfoSphere Information Server, InfoSphere Warehouse and Cognos 8 BI delivers informational understanding, ownership and trust 21

How Can IBM Help? Comprehensive platform for data quality assessment, cleansing and on-going monitoring Experience and repeatable process for helping organizations set up data quality programs Domain and industry-specific expertise in establishing repeatable data quality services Data quality assessment offering to report on existing data quality and establish the business value of a data quality program Stop by the Solution Center for demos of InfoSphere with Cognos 8 BI integration Contact your Cognos or IBM InfoSphere representative for more information, or visit: Thank you for your time © Copyright IBM Corporation 2008 All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBMs sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, Cognos, the Cognos logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.