Presentation on theme: "Monitor the Quality of your Master Data"— Presentation transcript:
1 Monitor the Quality of your Master Data THOMAS RAVNMarch 16thth 2010, San Francisco
2 Platon A leading Information Management consulting firm Independent of software vendorsHeadquarter in Copenhagen, Denmark220+ employees in 9 offices300+ customers and 800+ projectsFounded in 1999Employee owned company“Platon received good feedback in our satisfaction survey. Clients cited the following strengths: experience and skill of consultants, business focus and the ability to remain focused on the needs of the client, and a strong methodological approach”Gartner July 2008
3 Key Concepts and Definitions “Master Data Management (MDM) is the structured management of Master Data in terms of definitions, governance, architecture, technology and processes”“Information Management is the discipline of managing and leveraging information in a company as a strategic asset”Information ManagementDataGovernanceMDMDQM“Data Governance is the cross-functional discipline of managing, improving, monitoring, maintaining, and protecting data”“Data Quality Management is the discipline of ensuring high quality data in enterprise systems”
4 Components of an effective MDM approach Formalize business ownership and stewardship around data.MDMBusiness ownership, responsibility, accountabilityCommon definitionsEffective Master Data processesData Quality ManagementProtect, validate and integrate data in IT applicationsIT Change controlEnsure that Master Data is taken into account each and every time a business process or an IT system is changed.To be able to share data you need to share definitions and business rules. Definitions require management, rigor and documentation.Control in which systems Master Data is entered and how it is synchronized across systems.Manage Master Data Repository.Capturing Master data efficiently needs to be built into the business processes. Equally consistent usage of Master Data needs to be ensured across business processes and business functions.Measure and monitor the quality of data
5 Typical Data Problems - 1 NoNameAddressPurchaseIBM187 N.Pk. Str. Salem NH 014568,494.00I.B.M. Inc.187 N.Pk. St. Sarem NH 014563,432.00International Bus. M.187 No. Park St Salem NH 041562,243.00Int. Bus. Machines187 Park Ave Salem NH 041565,900.00Inter-Nation Consults15 Main St. Andover MA 023416,800.00Int. Bus. ConsultantsPO Box 9 Boston MA 0221010,243.00I.B. ManufacturingPark Blvd. Boston MA15,999.00How much did we spend with IBM last year?
6 Typical Data Problems - 2 Is this the same customer?NameStreetZip CodeCityCAFÉ SPORTSCLUB15 3rd Street10001New YorkCAFÉ SPORT KLUB15 Third St..NYCAre these the same products?Description, System 21 L Cappucino - Mathilde CafeFETA W/OLIVES & GARLIC 60G, 45+1000 ML YOG. PEACH/BANANADescription, System 11/1L Mathilde Cafe Ice Cappucino45+ FETA M/OLI+HVIDL 60G, 45+YOGHURT PÆRE/BANAN, 1000ML
7 Typical Problems - 3A common problem is overloading of fields, which is the misuse of a field compared to the intended use. Often because the field the user wanted to use wasn’t available in the applicationSometimes a field might even have been used for different purposes by different parts of the organizationCustomer NoNameFax1234JohnVip Customer3368PeteTel:2345Bob
8 Where Does the Bad Data Come From? State is a required field – regardless of country
10 Lack of ownership and clearly defined responsinility Top 5 Sources of Bad DataLack of ownership and clearly defined responsinilityLack of common definitions for dataLack of control of field usageLack of process controlLack of synchronization between systems
11 What is Good Data Quality? Larry English:Quality exists solely in the eye of a customer of a product or service based on the value they perceiveInformation quality is consistently meeting ‘end customers’ expectations through information and information services, enabling them to perform their jobs effectivelyTo define information quality, one must identify the "customer" of the data - the knowledge worker who requires data to perform his or her jobPlaton definition:Data Quality is the degree to which data meets the defined standards
12 “The Law of Information Creation” “Information producers will create information only to the quality level for which they are trained, measured and held accountable.”Larry English
13 Data Standards & Data Quality It’s all about the Meta Data…Good Meta Data is prequisite to achieve great data quality (inferred from the trained part of the ”Law of Information Creation”)You can only achieve high quality data if you have standards to measure against!
14 Defining Good Data standards For every entity define:For every field define:Definition and keysBusiness descriptionHierarchiesData entry format and conventionsClassification(s)Definition ownerLife cycleStakeholdersBusiness Owner(s)Consider what a user needs to know to produce high quality data
15 Data Standards – An Example ChallengesRelating the data definitions to the process documentationKeeping the definitions up to dateThe same piece of information may be entered in multiple different systems
16 Defining Good Data standards There are two basic approaches to defining your data standardsDefine a system independent Enterprise Information Model and then map attributes to system fields, orDefine data definitions for a system (screen/table) specific view of dataIf you have one primary system where a data entity is used, option 2 is preferableIf you have many different systems where the same data entity is used, option 1 is preferable
17 Garbage In = Garbage Out QualityStandard1 In + QualityStandard2 In Generating GarbageGarbage In = Garbage OutQualityStandard1 In + QualityStandard2 In= Garbage Out
18 Data Quality Monitoring Like most other things, data quality can only be managed properly if it is measured and monitoredA data quality monitoring concept is necessary to ensure that you identifyTrends in data qualityData quality issues before they impact critical business processesAreas where process improvements are needed
19 Data Quality Monitoring For this to work, clearly-defined standards, targets for data quality and follow-up mechanisms are requiredThere is little point in monitoring the quality of your data if no one in the business feels responsible and if clear business rules data have not yet been definedThus a data quality monitoring concept should go hand in hand with a data governance model
20 The Dimensions of Data Quality Do we have all required data?Are all data values within the valid domain for the field?CompletenessValidityDoes data reflect the real world objects or a trusted source?DataQualityTimelinessAccuracyAre data available at the time needed?IntegrityConsistencyAre business rules on field and table relationships met?Are shared data elements synchronized correct across the system landscape?
21 KPI Examples in the different dimensions CompletenessPct of active customer records with an addressValidityPct of active US customers with a phone number of 10 digitsAccuracyPct of active customers with an mailing address that is verified as correct against Dun & BradstreetConsistencyPct. of customer records shared between our CRM system and our ERP system that has identical values for name, address and telephone number.IntegrityPct. of active product records with [type] = “Service” where [weight] = 0, or Pct. of open sales orders that refer to an active customer.TimelinessPct. of supplier records where the time from request of a new record to completion and release of the record is less then 24 hours
22 The Dimensions of Data Quality Business ImpactAccuracyTimelinessConsistencyIntegrityValidityCompletenessDifficulty of Measurement
23 The steps in building a monitoring concept Building a data quality monitoring concept involves the following five basic steps:Identify stakeholdersConduct interviews with stakeholders and selected business usersIdentify data quality candidate KPI’sSelect KPI’s for data quality monitoringFor each KPI, define details
24 Finding Good Data Quality KPI’s To find good data quality KPIs collect business input through interviews with stakeholders (use Interviewing Technique) and a data assessment. The technique Data Profiling contains more details on how to analyze dataCollect business inputBusiness process requirementsData quality pain pointsBusiness IntelligenceBusiness KPIsKPI CandidatesDEFINED KPIsXXXXXXXXXKPIFrqTargetUoMABCXXXXXXXXXXXXXXXXXXPerform a thorough data assessment (profiling) exercise searching for common data quality problems and look for abnormalities
25 Tying Data Quality KPIs to Business Processes It is essential that KPIs are not just made up, so your organization has something to measureDon’t measure data quality because it’s great to have high quality data. Measure it because your business processes depend on itDerive data quality KPIs from business process requirementsStart with a high level business process like procurement (also known as a macro process) and then break it down.
26 Tying Data Quality KPIs to Business Processes Data Entity ScopeDefine the data entities used within the processProcurementMaterial Master DataMacro processVendor Master DataSpend analysisVendor SelectionProcessData quality requirementsDEFINED KPIsNo duplicate vendorsCorrect industry code for vendorsCorrect placement in hierarchy (parent vendor)Correct address for vendorsIs the required data quality aspect meaningful to monitor?KPIFrqTargetUoMABCBusiness Meta DataEx: A vendor record is uniquely defined as an address of a vendor where we place orders, receive shipments from or…..It may be better to improve data validation or perhaps problems are not experiencedBusiness Meta Data is required to define the actual KPIs.
27 Tying Data Quality KPIs to Business Processes Using a simple model like the one illustrated on the previous slide allows you to tie data quality KPIs to business processes and to business stakeholdersThis relationship is critical for the success of the data quality monitoring initiative. Clearly illustrating how poor data quality impacts specific business processes is instrumental in getting the executive support and the business buy inWhen conducting data quality KPI interviews you may encounter KPI suggestions like “measure if there is a valid relationship between gross weight and product type”. Ask why this is important and which process this is important forA particular data quality KPI may be important for multiple different processes. Document the relationship to all relevant processes
28 Defining Data Quality KPI’s Data quality KPIs should express the important characteristics of quality of a particular data elementTypically units of measures are percentages, ratios, or number of occurrencesFor consistency reasons, try to harmonize the measures. If for instance one measure is “number of customers without a postal code” while another is “percentage of customers with a valid VAT-no” a list of measures will look strange, since one measure should be as high as possible, and the other as low as possibleA good simple approach is to define all data quality KPI’s as percentages, with a 100% meaning all records meet the criteria behind this KPIBe careful not to define too many measures, as this will just make the organizational implementation more difficultPay attention to controlling fields (like material type) that may determine rules like whether a specific attribute is required
29 KPI: Customer Fax number correctly formatted Defining HierarchiesUse hierarchical measures where possible, so that measures can be rolled up in regions and countries for instanceIn the below example a KPI related to customer data is broken down in individual countries to allow detailed follow upA concern here is that fields may be used differently in different countries. Given the below data insight, it might make sense to define a separate KPI’s for CA and perhaps ignore MX and USAvg. ValueKPI: Customer Fax number correctly formatted25%Data InsightFax numbers are not required for US customers since all communication is done via .Fax is the primary communication channel with Canadian customers.Only some customers in Mexico have a fax machine.ValueRecsUS Customers5%85,000CA Customers77%19,000MX Customers43%38,000
30 Defining KPI Thresholds Along with each KPI two thresholds should be defined:Lowest acceptable valueWithout specifying the lowest acceptable value (or worst value), it’s difficult to know when to reactIf the measure falls below this threshold action is requiredTarget valueWithout target values, you don’t know when the quality is ok. Remember fit-for-purposeSpecifying a low and target threshold allows for traffic light reporting that provides an easy overviewDefining appropriate thresholds can be difficult as even a single product record with wrong dimensions may cause serious process impact. But without any indication of when to be alerted any form of automated monitoring is difficultTarget Value: 95 %Lowest acceptable value: 80 %
31 Indirect MeasuresConsider critical fields (e.g. weight of a product or customer type) where the correct value is of utmost importance, but it’s close to impossible to define the rules to check if a new value entered is correct….One approach is to measure indirectly by for instance reporting what users have changed these values for which products over the last 24 hours, week or whatever is appropriate in your organization
32 Cross field KPIs and Process KPIs Common KPIs that are not related to a single fieldNumber of new customer records created this weekAverage time from request to completion of a new material recordNumber of materials with a non-unique description (or pct. of materials with a unique description)Number of vendors, where a different payment is defined in different purchasing organizationsNumber of open sales orders referring to an inactive customer
33 Think Prevention!Every possible business rule related to completeness, integrity, consistency and validity should be enforced by the system at the time of data entry.If it isn’t, consider implementing a data input validation rule rather than allowing bad data to be entered and then measure it!However, there are cases, where the business logic of a field is too ambiguous to be enforced by a simple input validation rule.Process (workflow) adjustments may also be the answer.
34 Documentation of KPIs KPI Name: A meaningful name of the KPI that expresses what is being measuredObjective:Why do you measure this? What business processes are impacted if there data is not ok?Dimensions:What data quality dimensions (integrity, validity, etc.) are this KPI related to?Frequency of measure:How often do you wish to report on this KPI? Daily, daily, weekly or monthly?Unit of measure:What is the unit of the KPI? Number of records, pct of records, number of bad values, etc.?Lowest acceptable measure:Threshold that indicates if the data quality aspect the KPI represents is at a minimal acceptable level. The value here must be in the unit of measure of the KPI.Target value:At what value is the KPI considered to represent data quality at a high level?Responsible:The person responsible for the particular KPI.Formula:The tables and fields that are used to analyze and calculate the KPI. This is the functional design formula that forms the basis for the technical implementation.Hierarchies:When reporting on a KPI it is very useful to be able to slice and dice the measure according to different dimensions or hierarchies. For a customer data KPI for instance, good hierarchies would be regions, country, company code and account group.Being able to view the KPI through a hierarchy also makes it easier to follow up with specific groups of business users.Notes and assumptions:If certain assumptions are made about the KPI make sure to document it here
35 Remember!Quality is in the Eye of the beholder!Data quality is defined by our Information CustomersData is not always clean or dirty in itself – it may depend on the viewpoint and a defined standardFocus on what’s important to those that use the data
36 Monitoring Process A simple example Y N Publish KPI Analyze KPIs Low value in KPI?YEvaluate root causePlan corrective actionsImplement ImprovementsN
37 Monitor the Quality of your Master Data Thomas RavnPractice Director, MDME:M:PLATON US INC.5 PENN PLAZA, 23rd FloorNEW YORK NY37