1. TÜV SÜD AG 1 TL 9000 Measurements Auditing the Validation Process Jerry Wilson TL 9000 Program Manager / Lead Auditor TÜV SÜD America, Inc.

2. TÜV SÜD AG 2 Focus There are 4 basic components to the TL 9000 measurements process as shown below. Today we will focus on the Validate step and review a few of the things that are verified during an audit of this process step. Collect AnalyzeReport Data Sources Customer Supplier Internal system

3. TÜV SÜD AG 3 Inputs Simplified Process Flow Inputs Customers Internal Systems Suppliers Collect Validate Analyze Report to : MRS Organization Customers (if required) Report to : MRS Organization Customers (if required) Correct Data Process Outputs N Y

4. TÜV SÜD AG 4 INPUTS The Auditor must gain an understanding of:The Auditor must gain an understanding of: 1. the product categories being reported 2. the inputs What customers or suppliers provide data? What data do they supply? 3. The collection methodologies for each How do they supply it?

5. TÜV SÜD AG 5 INPUTS The Auditor must assess:The Auditor must assess: 4. the status of any ‘N/A’ or ‘Exempt’ and ensure acceptability 5. consistency with scope of registration – Example: All Common and Hardware measurements are collected if the organizations scope is H 6. Review corrective actions if organization has previously been on ‘suspended’ or ‘probation’ status (ref: Alert 06-003A)

6. TÜV SÜD AG 6 PROCESS Look for :Look for : – documented validation process (measurements handbook 3.5.2. a)) responsibilities for review and approval handling of invalid data – access to the source data is available for validation – verification that the output from any software application or spreadsheet has been verified (7.1.C.4 Tools Management) ensure correct calculations and proper use of the counting and exclusion rules Sample calculations, counting, and exclusion rules Ensure that validation personnel are trained and understand the counting rules

7. TÜV SÜD AG 7 PROCESS CHANGE Look for:Look for: – Evidence that when changes are made to software applications / spreadsheets used to consolidate the information that there is verification that the output is acceptable (7.1.C.4 Tools Management) verify correct calculations verify data filters are working correctly (counting rules vs. exclusions)

8. TÜV SÜD AG 8 OUTPUT Look for :Look for : – evidence data validation occurs prior to each reporting period – acceptable DCR’s for each category – sample the summary data and comparing to DCR’s

9. TÜV SÜD AG 9 Indicators Possible Indicators – Management Reviews show consistent poor results – DCR’s indicating zero for NPR / SPR (especially for several consecutive months) may indicate that the linkage between the reporting method and collection system is broken – Large variations in quantities reported Issues related to the date filters or counting rules

10. TÜV SÜD AG 10 Thank You