Quality Audits 102 Objectives “Quality (environmental, safety, food safety, financial, IT, …) audits are required.” Audits can enable quality control and quality assurance. Product or service conformance Meet customer requirements System conformance Process conformance Process capability Continual improvement Share personal experience and Explore recent information from others Quality: The Basic Tool Kit Quality Audits 102, Alex Klein ASQ Illiana section and Calumet College of St. Joseph, January 21, 2011

Review about Audits Audit Requirements: Goals: A systematic, independent and documented process for obtaining records, facts or other information and evaluating it objectively to determine the extent to which policies, procedures or requirements are fulfilled. ISO 19011 Audit Requirements: Planned: audit criteria, scope frequency and methods Select auditors who are objective and impartial; shall not audit own work. Procedure for conducting, recording and reporting Management must take action Goals: Analyze processes Review conformance Confirm information Evaluate risks Quality: The Basic Tool Kit Quality Audits 102, Alex Klein ASQ Illiana section and Calumet College of St. Joseph, January 21, 2011

Quality Control and Quality Assurance Fulfilling quality requirements (ISO 9000:2005) Providing confidence that quality requirements will be fulfilled (ISO 9000:2005) Concepts/ requirements: Quality objectives (ISO 9001, clause 5.4.1) Standardized processes (7.5.1, 7.5.2) Process monitoring and control (7.5, 8.2.3) Statistical process control, SPC (ISO/TS 16949, clauses 8.1.1 and 8.1.2) Measurement system analyses, MSA (TS 7.6.1) Inspection, measurement and testing (8.2.3, 8.2.4) Calibration (7.6) Measurement traceability (7.6.a) Process capability (TS 8.2.3.1) Product or service conformance (8.1.a, 8.2.4) Quality: The Basic Tool Kit Quality Audits 102, Alex Klein ASQ Illiana section and Calumet College of St. Joseph, January 21, 2011

Quality Control and Quality Assurance A process is a sequence of activities or steps which transforms inputs into outputs (ISO 9000:2005). Processes should be standardized, monitored, controlled, measured and continually improved. Every individual participates in at least one process that impacts the product, service or customer. Quality: The Basic Tool Kit Quality Audits 102, Alex Klein ASQ Illiana section and Calumet College of St. Joseph, January 21, 2011

Quality Control and Quality Assurance Annual internal audits of the system, every process and products/ services. Document, record, process and information control systems Manufacturing processes (including order management, R&D, scheduling, purchasing, inspecting, testing, shipping, transportation) Support processes (personnel services, training, maintenance, testing, safety, utilities) and Other processes (control of nonconforming product or service, billing and invoicing, internal audits, CA, PA) that impact customers. Top management processes (resource management, management review) Product coming to and leaving from each processing unit and in storage Quality: The Basic Tool Kit Quality Audits 102, Alex Klein ASQ Illiana section and Calumet College of St. Joseph, January 21, 2011

Supplier  Inputs  Process  Outputs  Customer A process is a set of interrelated or interacting activities which transform inputs into outputs. Supplier  Inputs  Process  Outputs  Customer Materials Information Inquiries, Orders VOC, voice of the customer People Requirements, Specifications Procedure Energy Money Activities/ Steps Product (Material, report, plan, schedule) Service (Inspection, testing, maintenance) Report Plan (business, production, maintenance) Results Quality: The Basic Tool Kit Quality Audits 102, Alex Klein ASQ Illiana section and Calumet College of St. Joseph, January 21, 2011

The SIPOC model may be used to understand processes Supplier Inputs Process Output Customer Inquiry Review inquiry Contract Order Enter order Accepted order Order management Accepted order Manufacturing limits Prepare manufacturing order Manufacturing order Scheduling Materials producer Order for material Procurement Material Production unit #1 Prepare line-ups Line-ups Production units Line-up Production process #1 Product #1 Production unit #2 Production process #2 Product #2 Production unit #3 Product # 2 Product #3 Final production unit Audit team Audit schedule Audit procedure Internal audit Audit report Auditee Product #n Inspect, test Results Packaging unit Packaging specs Package Packaged coil Shipping Packaged product Deliver packaged product Delivery of product Processor or customer Quality: The Basic Tool Kit Quality Audits 102, Alex Klein ASQ Illiana section and Calumet College of St. Joseph, January 21, 2011

Lessons Learned Train auditors about the standard(s) and process approach to auditing; use ISO 19011. Train all auditors annually. Revised standard requirements New interpretations/ understanding about requirements Discuss focus areas (e.g., performance training, calibration) for the current audit cycle. Review feedback from previous audits. Provide aids to auditors: Checklists (e.g., manufacturing, non-manufacturing, maintenance processes Checklists for layered process audits Practice Review findings from previous audits Quality: The Basic Tool Kit Quality Audits 102, Alex Klein ASQ Illiana section and Calumet College of St. Joseph, January 21, 2011

Lessons Learned Include process/ technology experts on audit teams. Be efficient and effective: Time management – audits are an intrusion in the audited area. It may not be necessary to audit what has been audited by Internal Affairs and SOx auditors and financial consultants. Verify that action was taken to correct any findings. Interview small groups of individuals performing the same activity. Observe the evidence for effective implementation of actions taken to correct nonconformances from previous audits Representative/ random sampling – enough to reach comfortable conclusions Quality: The Basic Tool Kit Quality Audits 102, Alex Klein ASQ Illiana section and Calumet College of St. Joseph, January 21, 2011

Lessons Learned Observe measures of performance for each process Not just output provided, also consider, Was process effective and efficient? Allow correction and corrective action Allow RCA and correction of minor nonconformances. Expect formal corrective action for systemic problems. Expect continual improvement for some, not all measures of performance. Remember the organization Exists for its customers Must run the business Quality: The Basic Tool Kit Quality Audits 102, Alex Klein ASQ Illiana section and Calumet College of St. Joseph, January 21, 2011

Lessons Learned Recognize good performance Share information Processes capable Processes improving Share information About successful implementation in another area From benchmarking Offer opportunities for improvement How to audit mature systems Consider the audit team – rotate the members, include different persons. Consider the audited areas – have the systems, processes or people changed since the previous audit? Are measure of performance being met? Is performance improving? Quality: The Basic Tool Kit Quality Audits 102, Alex Klein ASQ Illiana section and Calumet College of St. Joseph, January 21, 2011

Lessons Learned Inform/ train all employees about the benefits of auditing: Understanding about the audit process Employees’ role in the audit Reduce the fear about an audit Enables participation in continual improvement activities. See “Improving the Internal Audit Experience”, Theresa Wasche and Nancy Sciortino, QP, November 2007 Quality: The Basic Tool Kit Quality Audits 102, Alex Klein ASQ Illiana section and Calumet College of St. Joseph, January 21, 2011

Lessons Learned Explore/ benchmark Consult consultants Don’t allow them to dictate What does the standard require? How can we most efficiently and effectively meet requirements? How can the management system enable meeting customer requirements and continual improvement? Read and evaluate Network Stay current with rapidly advancing technology and use it to improve audit efficiency Quality: The Basic Tool Kit Quality Audits 102, Alex Klein ASQ Illiana section and Calumet College of St. Joseph, January 21, 2011

“Word Power, 12-point style guide for the quality professional”, Burjor Mehta, QP (Quality Progress), October 2010 Use words and phrases with positive connotations. Be brief and concise. Use consistent terms. Use consistent formats. Use white space. Be specific. Don’t use jargon or clichés. Use complete words and sentences. Use “I” to ask and “you “ to give. Quality: The Basic Tool Kit Quality Audits 102, Alex Klein ASQ Illiana section and Calumet College of St. Joseph, January 21, 2011

“Brace for Impact, Modified approach improves outcome of internal audits”, James Wells, QP, October 2010 Observe measures of performance (metrics) and determine which are not meeting target performance. Audit processes that support these metrics. Analyze finding for impact on the business (reputation, scrap, rework, wasted time and effort). For many findings a direct or indirect connection may be found to a potential financial risk associated with a poor practice found during the audit: Past due calibration/ daily verification Closure time of a customer complaint Also see “All Ears, Use financial terms to get management to take notice of your quality message”, Peter J. Sherman and James G. Vono, QP, July 2009 Quality: The Basic Tool Kit Quality Audits 102, Alex Klein ASQ Illiana section and Calumet College of St. Joseph, January 21, 2011

“Dealing with DISCONNECT”, Joseph D. Conklin with Michael J “Dealing with DISCONNECT”, Joseph D. Conklin with Michael J. Mazu, QP, October 2010 Raise the awareness about significant nonconformances. Same recommendation made each year: Trained person to back-up lone knowledgeable person Funding needed to improve system Question upper management and external auditors Clearly understood expectations Poorly written findings Quality: The Basic Tool Kit Quality Audits 102, Alex Klein ASQ Illiana section and Calumet College of St. Joseph, January 21, 2011

“Trust, but Verify”, J.P. Russell, QP, November 2009 There are systems audits. There are more product, service and process audits. Combined product, service and process audits = verification audit. Verification should be performed when there are routine changes (suppliers, equipment, process settings, methods, requirements, personnel). This audit helps minimize risk and ensure sustainability. Quality: The Basic Tool Kit Quality Audits 102, Alex Klein ASQ Illiana section and Calumet College of St. Joseph, January 21, 2011

“Think Again, Change your view of audits to improve their effectiveness”, Sonny Sahi, QP, October 2009 Auditors and management must accept nonconformances and present them and receive them as opportunities to investigate how the process or system failed not how the person(s) failed. Quality: The Basic Tool Kit Quality Audits 102, Alex Klein ASQ Illiana section and Calumet College of St. Joseph, January 21, 2011

“Gauge Audit Program Value”, J.P. Russell, QP, June 2009 Effectiveness of a system or process: Achieves desired result consistent with objectives for the product Is capable, efficient and consistent with objectives for the process Effectiveness of audit process and follow-up actions: Counting audits performed, nonconformances cited, closed corrective actions are typical performance measures. Consider: Objectives How? Performance measures Meet customer requirements Obtain feedback Internal satisfaction Meet management system requirements Identify and report CAs that improved performance Upgrade competence of auditors No serious findings from external audits Root cause(s) identified CA effective, no reoccurrences Maintain supplier and outsourcing requirements Verify claimed improvements Quality and delivery met Quality: The Basic Tool Kit Quality Audits 102, Alex Klein ASQ Illiana section and Calumet College of St. Joseph, January 21, 2011

“Gauge Audit Program Value”, J.P. Russell, QP, June 2009 Effectiveness of audit process and follow-up actions: Counting audits performed, nonconformances cited, closed corrective actions are typical. Consider: Objectives How? Performance measures Improve effectiveness and efficiency of management system Prompt audit reporting Findings linked to objectives or customer requirements Report terminology understood Repeat nonconformances Address OFIs = PAs Determine management objectives for lowering expenses Mini audits Reduced cost of audits Implement processes to ensure sustainability Schedule audits based on process changes Train QS resources in areas Validate management system design to meet future needs ? Quality: The Basic Tool Kit Quality Audits 102, Alex Klein ASQ Illiana section and Calumet College of St. Joseph, January 21, 2011

“Can Do, Effective Overseas Food Safety Audits”, Steve Wilson, QP, May 2009 These audits must/ can be successful when considering and planning for Language and social barriers Varying requirements in other countries. Quality: The Basic Tool Kit Quality Audits 102, Alex Klein ASQ Illiana section and Calumet College of St. Joseph, January 21, 2011

“Putting Your Internal Audit System on Steroids”, Mike Micklewright, presented at NIQC, May 2008 Audit for conformance Audit for effectiveness Teach auditors about wastes Overproduction → PA Excess inventory → PA Transportation → PA Delays → PA (If there was objective/ target, then CA) Excess processing → PA Wasted motion → PA Defective product → CA Underutilized employees → PA Quality: The Basic Tool Kit Quality Audits 102, Alex Klein ASQ Illiana section and Calumet College of St. Joseph, January 21, 2011

“Putting Your Internal Audit System on Steroids”, Mike Micklewright, presented at NIQC, May 2008 Teach auditors about Value stream mapping and process mapping RCA Mistake proofing Evaluate auditors: Preparation Auditing Report Relevance of nonconformances Quality: The Basic Tool Kit Quality Audits 102, Alex Klein ASQ Illiana section and Calumet College of St. Joseph, January 21, 2011

“Drill Deep for Strategic Alignment”, Joe Kausek, QP, October 2008 Provide auditors the organization’s: Vision, mission and values Overall business plan Balanced scorecard Process for establishing metrics and reporting performance. Audit for strategic alignment How does process contribute to strategic objectives? How have process owners aligned processes with strategic objectives? Were resources provided for successful accomplishment of the objectives? Were process metrics aligned with metrics for strategic objectives? Is the process contributing to the success of the strategic objective? Quality: The Basic Tool Kit Quality Audits 102, Alex Klein ASQ Illiana section and Calumet College of St. Joseph, January 21, 2011

“10 Auditing Rules”, Joe Kausek, QP, July 2008 Make auditees feel like members of the audit team. Start by asking general, open-ended questions, then use clarifying questions to fill the gaps. Be an active listener. Never let the auditee pick the samples. Always try to identify the financial effects of findings, when possible. Always confirm your findings with the auditee. Don’t look for findings that could never have an impact on the performance of the management system. Provide sufficient information about the requirement, nonconformance, and evidence found. When citing areas of strength, be specific. Feed the auditor. Quality: The Basic Tool Kit Quality Audits 102, Alex Klein ASQ Illiana section and Calumet College of St. Joseph, January 21, 2011

“The Great Debate, Can one process approach apply to QMS and EMS audits?”, Terry A. Mors, QP, April 2008 (And Safety management system and Food Safety management and other management system audits? OHSAS 18001 does require determining effectiveness of the OHSAS management system.) ISO 9001 and ISO/TS 16949 support the process approach. Although an EMS (and SMS) is a system of processes, some adjustments are required due to language, system and standard differences. ISO 14001 (and OHSAS 18001) was not written based on the process approach. ISO 14001 does not require measuring effectiveness of each process. Using inputs and outputs (interaction) terminology can steer organizations toward a process approach. Quality: The Basic Tool Kit Quality Audits 102, Alex Klein ASQ Illiana section and Calumet College of St. Joseph, January 21, 2011

“Sharpen Your Auditing Skills”, Joe Kausek, QP, February 2008 Use checklists to guide planning, auditing and reporting. Identify the process, inputs, procedures, instructions, outputs, recent performance. Prepare an audit planning checklist. Review procedures, diagrams and other information. Select items to verify. Mark up the documents to serve as a checklist. After initial interviews were completed, enter the objective evidence. Identify additional questions or items to be audited. Quality: The Basic Tool Kit Quality Audits 102, Alex Klein ASQ Illiana section and Calumet College of St. Joseph, January 21, 2011

Other resources “The Right Approach, An effective QMS audit requires a process-based strategy”, Sandford Leibsman, QP, September 2009 ISO 9004, Managing for the sustained success of an organization Other periodicals: “Quality” “Quality Digest” The Process Auditing and Techniques Guide, J.P. Russell, second edition, 2010 Websites: ASQ, www.asq.org Quality: The Basic Tool Kit Quality Audits 102, Alex Klein ASQ Illiana section and Calumet College of St. Joseph, January 21, 2011

Your Comments and Questions are desired. Thank you! Quality: The Basic Tool Kit Quality Audits 102, Alex Klein ASQ Illiana section and Calumet College of St. Joseph, January 21, 2011 29