How to Measure Data Quality Data Quality Management System (DQMS) Self Assessment Product Inspection & KPIs Continuous Improvement “Quality is a measurable, manageable business issue” John Guaspari

How to Measure Data Quality Key Elements of DQF Data Quality Management System Self-assessment Model Product Inspection Model Scorecard & KPIs Monitoring & Continuous Improvement Guidelines Lets go into more detail

Data Quality Management System (DQMS) Plan your Data Quality strategy Define goals and objectives Data Quality Management System (DQMS)

How to Measure Data Quality Data Quality Management System (DQMS) A strategic approach that enables enterprise-level governance and management of data quality to generate superior information assets. Based upon best practices and capabilities that support increasing data quality. Organized by: Functional Areas Action Types Flexible design and implementation depending on the unique requirements of an organisation Scalable model that accommodates any size organization large or small

How to Measure Data Quality DQF DQMS - Capabilities “Capability” Ability to perform Capacity, expertise, knowledge, or talent required to perform a task A business capability is the expression of the ability, materials, and expertise an organization needs in order to perform core functions. Quality Data People Process Technology

How to Measure Data Quality DQF DQMS – Functional Areas Capabilities have been organised by Functional Area: Organisational What DQ enterprise level structure is required? What DQ roles and responsibilities must be defined? Policies and standards What DQ guidelines are needed to provide governance and reference? Business processes What critical processes drive day-to-day operations? Systems capabilities What technology is necessary to support the business?

How to Measure Data Quality DQF DQMS – Activity Types Within each functional area, there are four key elements or activity types: Plan: Strategy & Goals Document Policies, Process SOPs, Org, RACI Execute Communication, Training, Change Mgmt Monitor/Control Audits, Reporting Life Cycle, Continuous Improvement

How to Measure Data Quality Example – DQF DQMS Capability Matrix Organisational Capabilities Policies & Standards Business Process System Capabilities Plan Executive sponsorship - Mission & vision; Accountable leadership; Staff roles & skill sets; Data owners & stakeholders; Data governance office. Mission & vision; Goals & objectives; Guiding principles; Success measures; Action plans; Policy & standards management Initial data entry & setup; Ongoing data maintenance; Processes involved in the information's life- cycle Unified data repository; Design & architecture; Workflow, user interface; Data validations; Security, access controls; Revision/change history; External publication; Internal publication Document Governance organisational structure; Roles & responsibilities; Personal objectives; Reporting alignment Mission, goals, principles and success measures; Governance model, decision process; Data definitions & standards; Security & use policy; Audit procedures; Documentation standards; Risk Management; Customer feedback policy Operating procedures; Process flow diagrams; Job aids, work instructions; Performance metrics System requirements; Operating procedures; Performance metrics Execute Education & awareness; Internal communication; Training Education & awareness; Documentation management; Policies & standards management; Data issue management Training; Customer feedback resolution Education & awareness; Performance management; Process issue management; Change management *See note on section 2.4.3 Monitor Organisational capability review; Review of personal objectives Policy & standards review Workflow controls; System validations; Performance reporting on service levels; Performance reporting on data quality; External & internal feedback; Process compliance audits; Product measurements; Review & reporting audit results; Monitor impact of erroneous data Performance reporting on service levels

How to Measure Data Quality Example – DQF DQMS Documentation Functional Area Activity Type Capability Name What – definition Why - rationale Recommendations - hints for implementation Practical Examples Self assessment - Questions linked to capability

How to Measure Data Quality DQF DQMS – Essential Steps Gain top management commitment Appoint responsible managers Start data quality awareness programmes Provide training Create Data Quality Management Processes Develop data quality management system documentation Document controls Implement and execute DQ program Perform Internal data quality audit Conduct management review Conduct conformity assessment (Optional) Perform continual improvement

Identify areas of opportunity Assessment of capabilities and priorities Self Assessment

How to Measure Data Quality Why Self-Assess? Identify areas of improvement as part of a continuous DQ program Engage in trading partner collaboration for process improvement Although the final assessment results are discussed among trading partners, the execution of the self-assessments themselves is always performed by one organisation without interference or involvement from external parties. Measure compliance with global standards or best practices Benchmark DQ practices within organisation or external to the industry

How to Measure Data Quality Self-Assessment Tools The following self-assessment tools are currently available as part of the DQF: Questionnaire The questionnaire contains 73 questions that relate to an organisation’s data quality management capabilities and their deployment level within the organisation. This is the core component of the self-assessment process. Scoring Model Indication of how many of the recommended best practices for a DQMS are in place within a given organisation. Master Data KPIs Performance indicators based on the monitoring and inspection of key GDSN attributes.

How to Measure Data Quality Self-Assessment Tools Questionnaire KPIs, Scoring model

How to Measure Data Quality Self-Assessment – Scope In defining the scope consider the following: How will the results of this assessment support the organisation’s goals? Factors impacting the assessment: Product categories Product life-cycle (e.g. new introductions vs. line items) Brands Specific DQ process and it’s complexity Type of change in the data/Attributes to be assessed Manufacturing facilities/locations Timing Performance goals Greatest areas of market interest Scope must be clearly defined, documented, and well communicated Scope definition will determine the size and complexity of the effort – start small

How to Measure Data Quality Self-Assessment – Considerations A Self-Assessment… Is a snapshot in time Should include a Capabilities Assessment and Process Review at a minimum Is not be the ultimate goal; it should lead to further improvements Needs to be carefully conducted to ensure the results are reliable the DQ Framework and Implementation Guide provide a detailed process flow for the execution of a self-assessment (see chapter 3 in both)* Additional implementation training is available on the DQF

How to Measure Data Quality Self-Assessment – Essential Steps Decide to self-assess Define the scope of the self-assessment Appoint an assessment leader Form an assessment team Educate the organisation about the self-assessment Baseline deployed capabilities Apply the self-assessment questionnaire Consolidate and analyse results Communicate results to the organisation Communicate results to trading partners (optional)

Product Inspection & KPI Model Identify areas of opportunity Assessment of capabilities and priorities Product Inspection & KPI Model

How to Measure Data Quality Product Inspection – What is it? Use of a standardised methodology for the physical inspection of a product to: Verify, objectively, electronic data in comparison with a physical product or global standard Ensure the results of the inspection are consistent and reliable Monitor KPIs developed to give a more granular indication of the quality of an organisation’s data output Track or Audit performance of an organisation and its data quality management system

How to Measure Data Quality Product Inspection – Scope Goals and objectives of inspection Document accuracy of the data sample Snapshot of DQMS process Methodology Sample size Information sources to be compared Attributes to be reviewed Product type Category Location of production Target Market

How to Measure Data Quality Product Inspection – Essential Steps Select inspection body Prepare for inspection Define scope of inspection Identify sample Gather documents for inspectors KPI Model & Scorecard GS1 Package Measurement Rules and related standards Calibrate and Secure measuring equipment Perform inspection Report results Launch appeals procedure Document complaints Apply corrective measures

Identify areas of opportunity Assessment of capabilities and priorities KPI Model & Scorecard “Not everything that can be counted counts." Albert Einstein

How to Measure Data Quality KPI Model – What is it? A list of Key Performance Indicators (KPIs) Key: Measures a critical business processes Supports DQ strategy Performance Indicator: an objective measure Internally focused and may be proprietary Short-term Long-term Based on DQ Dimensions DQ dimensions list is unique to enterprise Accuracy - the degree in which the (electronic) product information stored in a repository is consistent with the physically observable characteristics of the trade item.

How to Measure Data Quality DQF KPI Model DQF KPI model was defined to: Provide trading partners with a neutral, common set of KPIs for data accuracy Cover the most commonly synchronised attributes across all regions Offer a basic structure to validate the effectiveness of data quality management systems deployed within an organisation DQF KPI model includes the following: Overall item accuracy Generic attribute accuracy Dimension and weight accuracy Hierarchy accuracy Active/Orderable Target % is a business decision

Data quality KPI scorecard

How to Measure Data Quality DQF KPI Model – When to use it The DQF KPI model may be used for any of the following scenarios: As a means to report the results of product audits: the KPI model is a good way to offer structured results of product audits as it proposes a logical way to group related attributes. As a benchmark: the KPI model may be used as a reference to compare the accuracy of the data performance of two different entities. To track progress on improvements: the KPI model can be also used to compare the progression of data accuracy within the organisation by striving always to improve the results obtained every time the KPIs are measured.

How to Measure Data Quality DQF KPI Model – How to use it Customise KPI model Select KPI categories relevant to organisation Add or remove attributes to/from the categories Add specific performance targets Conduct product inspection Analyse using the KPI Scorecard Report results

Implement improvements Execute, document and take action Implementation plan

How to Measure Data Quality Product Inspection – Results Communicate results Prioritize actions Identify the data issues causing the most negative impact on the organisation Identify and adjust the processes and capabilities required to correct the negative impact Develop process and capacities to prevent reoccurrence Define realistic timeframes for execution Dependent on the extent and complexity of the changes Communicate Action Plan

How to Measure Data Quality DQF Phased Implementation Organisational Capabilities Policies & Standards Business Process System Capabilities Plan Executive sponsorship - Mission & vision; Accountable leadership; Staff roles & skill sets; Data owners & stakeholders; Data governance office. Mission & vision; Goals & objectives; Guiding principles; Success measures; Action plans; Policy & standards management Initial data entry & setup; Ongoing data maintenance; Processes involved in the information's life-cycle Unified data repository; Design & architecture; Workflow, user interface; Data validations; Security, access controls; Revision/change history; External publication; Internal publication Document Governance organisational structure; Roles & responsibilities; Personal objectives; Reporting alignment Mission, goals, principles and success measures; Governance model, decision process; Data definitions & standards; Security & use policy; Audit procedures; Documentation standards; Risk Management; Customer feedback policy Operating procedures; Process flow diagrams; Job aids, work instructions; Performance metrics System requirements; Operating procedures; Performance metrics Execute Education & awareness; Internal communication; Training Education & awareness; Documentation management; Policies & standards management; Data issue management Training; Customer feedback resolution Education & awareness; Performance management; Process issue management; Change management *See note on section 2.4.3 Monitor Organisational capability review; Review of personal objectives Policy & standards review Workflow controls; System validations; Performance reporting on service levels; Performance reporting on data quality; External & internal feedback; Process compliance audits; Product measurements; Review & reporting audit results; Monitor impact of erroneous data Performance reporting on service levels

Continuous Improvement Monitor changes and plan further improvements Start a cycle of continuous improvement Continuous Improvement

How to Measure Data Quality Monitoring & Continuous Improvement Ensuring DQ is a continuous, dynamic, day-to-day activity. Must be Integrated with the product life-cycle Create, Syndicate, Maintain, Archive, Purge does not end when a product is published to trading partners data must be maintained and updated Periodical audits (both of processes and data) are necessary to monitor progress. Most importantly, it needs to ensure that there is good control and monitoring on performance in order to ensure processes remain in optimal condition. Best Practice - “GDSN Trade Item Implementation Guide” Chapter 11: Item Futurisation and sustain the corrective action That is why ongoing monitoring is the key to creating sustainability and ensuring a continuous level of quality.

How to Measure Data Quality Summary – Elevator Pitch The DQF is adaptable and scalable and may be applied whole or in part – as required by the organisations goals & objectives DQF as part of a continuous cycle: Planning & Strategy Opportunity identification Corrective Action & Implementation Continuous Monitoring DQF Components Elements for Data Quality Management System Model for a self-assessment Model for a product inspection Model KPIs & Scorecard Recommendations for Monitoring & Continuous Improvement

The Data Quality Framework PACKAGE is publically available All you need to use the Framework in one package Includes: The Data Quality Framework v3.0 Implementation Guides (user’s manual!) Automated scorecard for self-assessment Automated scorecard for KPIs Data Quality Introductory Presentation Read me http://www.gs1.org/gdsn/dqf/data_quality_framework