Presentation on theme: "Effective Preparation to Successful Certification"— Presentation transcript:
1 Effective Preparation to Successful Certification Managing Your AuditEffective Preparation to Successful CertificationJohn KukolyBRC Global Standards
2 Agenda Preparing for the audit Managing the audit Non-conformities Corrective ActionRoot Cause Analysis
3 Intent of the Audit Gather evidence of conformance Identification of risks or non-complianceOpportunity for improvementCertification
4 The Audit What Happens Arrival Opening meeting Document review Site reviewInterviewsEvidence documentedClosing meetingCorrective actions to lose NC’sCertification or report
5 Preparing for the Audit Long Term Planning When do you want the auditWho is performing the auditWho will be involvedPractice with employeesUse internal auditing – continual practice through living it
6 5 Best Things to DoAssign responsibility to departments, manage through senior managementTrain and inform everyoneGet good at internal auditingLook for helpOrganize, and anticipate
7 Worst Things to Do “it’s a QC responsibility” “leave something for the auditor”“we’ve been auditor before”Try to misdirect or hideStopping after the auditUnrealistic expectationsLack of preparation
8 Preparing for the Audit Who is Involved All staffQC, QA, productionPest controlMaintenance, purchasing, receiving, shipping,ReceptionSetting senior management expectations
9 Preparing for the Audit Short Term Preparing materialsLast minute effortsRunning the business while the audit happensInforming staffService suppliers
10 Preparing for the Audit Logistics for the Day Opening meetingProtective equipmentTaking care of the auditorRunnersDaily recapClosing meeting
11 The Audit What to offer Gaining clarity on issues Arguing your positionDefending your facilityProviding evidence, documents and materialsFailures during the auditManaging the auditor
12 The Audit Timing of Events Gaining understandingTrainingImplementationPre-assessmentAuditCorrective actionsCommunications
13 During the Audit - DO Anticipate and provide Show off where you do things well or good resultsLead the auditQuestion and ask for explanationExpect breakdowns, let the process take over
14 During the Audit – DON’T Waste time or stallHide, misdirect, show partial informationInterrupt interviews or audit flowExpect a perfect resultArgue every point, or where evidence does not support
15 Dealing with problemsAppealsDebatesDisagreements
16 The Auditor Getting the most out of the auditor Understand their role, needs and positionProvide space and time when neededUnacceptable auditor activities
17 Most common NC’sDocument ControlGlass ControlHygieneChemical ControlCorrective ActionRecall TestDoorsTemporary RepairsWallsRecord ControlHACCP Plan
18 Document Control Version control Consistency across documents Complete information and instructionsTracked changes, with reasonUncontrolled documents
19 Glass Control Relevant complete list Condition Appropriate checks Complete procedure for loss or breakage
20 Correction and Corrective Action Immediate actions to protect product integrityShort term corrective actionCorrective actionRoot cause analysisEveryone knows and uses it!
21 Hygiene - Cleaning Non production areas Deep assessments Procedures Verification and validation
22 Chemical Control Cabinets Unapproved chemicals Secret stashes, uncontrolled materialsOver the counter products
23 Corrective Action Missing for documented non-conformities Incomplete corrective actionsNo corrective action in procedureNot following procedure
24 Recall Test Packaging Once, or twice annually? Unacceptable ranges or resultsFull trace test, finished product, and ingredients
26 Temporary RepairsExtended timesNot trackedCreates new risks
27 Walls Condition Temporary construction becomes permanent Poor choice of materials
28 Record Control Inaccessible Review performed as required? Unidentified errors
29 HACCP PlanMissed risksDiagram not representativeNot up to date
30 Root Cause Analysis Define Root Cause Analysis Improve understanding of the requirement for RCAUnderstand how to undertake Root Cause AnalysisPerform a Root Cause AnalysisIndications from the first year of audits to Issue 6 of the BRC Standard suggests that RCA is not well understood by the food industry.Although well know in other sectors, in the food industry we have been rather slower at introducing this useful tool. Therefore the aim of this training is to:1: Define what RCA is2: Thereby eliminate confusion3: to improve both auditor and food industry understanding4: to understand the importance of RCA5: ensure that we can undertake RCA competently6: practice some RCA and understand how to perform it correctly
31 Definitions Root Cause The underlying cause of the problem which, if successfully addressed, will prevent a recurrence of that problem.Corrective Action (Immediate Corrective Action)Action taken to manage a non-conformity (the non-conformity may originate from any source, e.g. a product incident, site audit or product testing). Corrective action should be completed as soon after detecting the non-conformity as possible.Corrective Action PlanWe need to begin by understanding some of the common terms used in RCA and the specific terminology used by the BRC Food Standard:1: Read definition – could be cause or causes – we will obviously spend most of the course looking at this2: Read definition - It is important once a non-conformity is identified, that corrective action is taken to address the issue. The term ‘immediate corrective action’ is sometimes used in the USA which emphasises that it is the action that should take place as soon after detecting the non-conformity as possible. Corrective action might include – complaint management, incident management, product recall, etc.3: The Food Standard specifically uses the term Corrective Action Plan in relation to the Enrolment programme. Following an audit, a site in the Enrolment programme must submit a Corrective action plan to their certification body within 28 days of the audit. We will not be covering CAPs in the rest of this presentation but it is included here to avoid future confusion.Following an audit, sites in the Enrolment Programme are required to develop a Corrective Action Plan which outlines the non-conformities and the action that has/will be taken to address the non-conformity.
32 Definitions Proposed Action Plan Following root cause analysis, the site must develop a proposed action plan to correct each of the root causes, such that they prevent recurrence of the non-conformity.Preventative ActionRoot cause analysis can identify a cause that indicates other systems or that are susceptible to the same failure. In these situations it is good practice to complete preventative action on the implicated systems before a non-conformity actually occurs.1: Of more importance – a proposed action plan is the output of the RCA. Once a site has identified the root cause of a problem or non-conformity, it must introduce corrective action/controls to correct the root cause and prevent recurrence of the non-conformity. I.e. it describes what will be done and when.Following an audit, the site is required to submit its proposed action plan, complete with timescales for completion of the actions. The proposed action plan must be submitted within 28 days of the audit.2: Preventative action is not mentioned in the food Standard however, on some occasions a RCA can identify a cause which suggests that systems or processes other than the one where the non-conformity occurred, are susceptible to the same failure. In these situations it is good practice to complete preventative action on these implicated systems before a non-conformity actually occurs.3: read definitionCausal Factors are any behaviour, omission, or deficiency in the process that if corrected, eliminated, or avoided probably would have prevented the non-conformity.Causal Factor
33 Good Solutions Most problems are systemic Don’t jump to conclusions Are separate from immediate corrective actionDiagnose and address the illnessEliminate the cause(s)Prevent re-occurrenceREMOVE BULLET POINT 4 AND THE RELEVANT NOTE BELOW AND ADD INFORMATION TO SLIDE 8 SO ALL OF THIS MESSAGE IS IN ONE PLACERCA is therefore a problem solving process for conducting an investigation into an identified incident, problem or non-conformity.1: RCA is looking for the process or system that failed or allowed the non-conformity to occur. It is not a blame game – we are not looking for who but what.2: It is always tempting to assume we know what happened – any hypothesis of this type should be tested for accuracy rather than assumed – don’t jump to conclusions without supporting facts3: RCA is a completely different process to incident management and immediate corrective action, although these activities are often completed in close proximity – but must be considered as separate processes to avoid confusion.AS ABOVE THIS POINT TO GO TO SLIDE 8:4: RCA requires the investigator(s) to look beyond the immediate problem to understand the underlying or fundamental cause of the situation. In this respect it is often compared to the difference between an illness and a symptom; immediate corrective action manages the immediate symptom, but RCA seeks to diagnose and address the illness that causes the symptoms.5:6: The over-arching aim is to introduce action which eliminates the cause and prevents re-occurrence
34 Identifying the cause Symptom Symptom Symptom Cause or causes Go through diagramLike peeling back the layers of an onion until we get to the cause
35 What is the aim of RCA? Identification of the root cause Implementation of suitable actionUltimately, the point of a RCA is to improve the systems such that a repetition of the incident is preventedVerification & monitoring activity demonstrates that the action has been effectiveTHIS SLIDE HAS A LOT OF DUPLICATION – RECOMMEND:DELETE BULLETS AS ALREADY COVEREDMOVE BULLETS 3, 4 & 5 TO SLIDE 7 ‘GOOD SOLUTIONS’AMEND BULLET 3 TO READ: ‘Ultimately, the benefit of RCA is an improvement in systems’What does the Standard expect?Go through slideThe key question: What in the system or process failed such that this problem could occur?
36 Factors in Corrections Stop the problemIdentify the scope of the impactCorrection (clean up the mess)Corrective actionRoot cause analysis
37 Process overview Step 1 Define the Non-Conformity Step 2 Step 3 Step 4 Investigate the Root Cause(s)Step 3Create Proposed Action Plan & Define TimescalesThe first step in the process is to define the problem – this might sound obvious but some immediate corrective actions, especially those involving a product withdrawal or recall, often employ ‘all inclusive’ or ‘scattergun’ approaches to ensure that all product is managed, however, this can mask the precise details of the non-conformity and therefore make RCA more difficult.Go through slideIt should be noted that this is a simplified process flow, which is typical of the majority of situations required by the Standard. However, for a complex non-conformity, such as a serious incident involving a microbiological outbreak, root cause analysis will become proportionally more detailed, require extra activities and can require considerable time and investment.In these situation the use of a RCA team rather than a single individual is recommended – cross-category for all possible activities implicated eg technical, QA, production, etc.Step 4Implement Proposed ActionStep 5Verification & Monitoring of Effectiveness
38 Asking the Right Questions OPENWho?What?Where?Why?When?How?To drill further into specificsPROBINGCLOSEDTo check understandingYou have probably encountered the Question Funnel which highlights that there are different ways of asking questions:Open – get explanation – who, what, where, when, etc?Probing – get as much information as possible – drill down into specificsAuditors will also be familiar with the BRC requirement to use challenging questions (ie asking the site how they know the conclusion/output/critical limit is correct) and this technique is also useful in this context with the investigator clarifying how the information they are provided is known to be correctClosed – usually get Yes/No answer and have limited use – but useful to confirm you have understood the information you have been given
39 The MethodsThere is no single prescribed method of conducting root cause analysis.The choice of root cause methodology may be:Matter of personal choiceCompany policyDependent on the type of non-conformity being investigatedGo through slide
40 SOME RCA Methods The 5 Whys Fishbone (Ishikawa) Diagram Cause and EffectPareto’s PrincipleWe will spend the next part of the course looking at some of the common methods that can be used
41 5 Whys?The investigator keeps asking the question ‘Why?’ until meaningful conclusions are reached.Why?Why?Why?Why?Why?Root CauseIt is important to ensure that the questions continue to be asked until the real cause is identified rather than a partial conclusion.Once you have asked ‘why?’ you need to prove the answer by either gathering data or objective evidence.1: 2: As you probably guessed from the video the investigator keeps asking the question ‘why?’ until meaningful conclusions are reached. It is generally assumed that this will require asking the question at least 5 times (hence the name), but it is important that the questions continue to be asked until the real cause is identified.3: When completing RCA it is important to avoid pure speculation, therefore each answer should be supported by either objective evidence or data gathering. For example, if the answer appears to be that the procedure doesn’t work, then it may be necessary for the investigator (or an experienced operative) to follow the procedure to identify the exact nature of any short-comings, other supporting evidence might include discussions with staff (eg to establish staff awareness or understanding) or conducting and evaluating product tests.
42 Investigation Techniques Interview personnelObserve the processTry the processConduct specific inspectionsReview the system dataSpecific testing if necessaryExplain the options within each bullet along with its value
43 5 Whys?For example:An operator is instructed to perform a simple action ‘weigh out ingredient A’.The operator inadvertently used ingredient B.An immediate reaction = operator error was the cause.This does not establish the reason why the error occurred or prevent it happening in future.The root cause analysis insists that a series of ‘Why?’s are asked.1: 2: Lets consider an example in which an operator is instructed to complete a simple task, ‘weigh out ingredient A’, however the operator inadvertently uses ingredient B.3: An immediate reaction is that this must be an operator error.4: 5: However, whilst this might be true, it does not establish the cause or prevent it re-occurring in the future. The RCA must ask a series of Why? Questions, for example:
44 5 Whys?An operator should have weighed out ingredient A on line 2. However inadvertently used ingredient B. Why is this happening?Why did the operator make the error? Operator unfamiliar with procedure.Operator trained but no supervision or sign off to confirm training was satisfactory.Why Is that?Why was the training not satisfactory? Both ingredients look identical and were not labelled.Why Is that?Labels were removed during the last clean and not replaced.Why Is that?1: Why did the operator weigh out the wrong ingredient = Perhaps the operator was unfamiliar with the procedure.Which naturally leads to the question:2: Why was an operator, unfamiliar with the procedure, asked to complete the task? = Operator was trained but there was no supervision or sign-off3: Why was the training not satisfactory? = On this occasion both the ingredients look identical and were not labelled4: Why werent the ingredients labelled? = The labels wew removed during the last deep clean and not replaced5: Why werent the labels replaced? = Cleaning staff didn’t consider the potential for error and checking the labels were not part of anyone’s dutiesCleaners didn’t consider potential for error & checking labels not part of anyone’s duties.Why Is that?
45 Fishbone or Ishikawa Diagram The next tool originated in the Japanese car industry hence it is sometimes referred to as an Ishikawa diagram but a fishbone diagram is more common name in many parts of the world.
46 Ishikawa This type seeks to understand the possible causes by asking: A second commonly used method of root cause analysis, is the fishbone.This is most useful when the ‘5 whys’ is too basic diagrams.Primary CauseSecondary CauseEquipmentProcessPeopleMaterialsEnvironmentManagementEFFECTThe fish bone name comes from the form the diagram takes as it is constructedThis is most useful when the ‘5 Whys?’ are too basic or where it is useful to trend significant amounts of data. It works by breaking down the various aspects of the problem, to allow the investigator to consider one section at a time and to visualise the relevant information.These constituent sections are not fixed but generally include aspects such as equipment, people, raw materials, processes, manufacturing environment, etcThe investigator then asks why?, when?, who?, where?, etc. in relation to each of the sections to gain information regarding the possible root cause.If planning to use the fishbone it is often useful to prepare a series of prompts to ensure that everything in each section is covered i.e. what you might need to ask or investigate in each section – do this first time and keep the prompts for future use – you wont need all the prompts for every occasion but they are then ready when/if needed. For example:People – training/competency/PPE/IllnessMaterials – Specifications/handling/temperatures/Stock controls/traceability/labellingEnvironment – humidity/lighting/temperature/cleanlinessEquipment – Maintenance/calibration/suitable access e.g. for cleaning/locationProcedures – available/working correctly/up to dateThis type seeks to understand the possible causes by asking:‘What’, ‘When?’, ‘Where?’ ‘Why?’, ‘How?’ and ‘So what?’A possible cause is identified and the consequences are investigated for each of the group categories.
47 ManagementMaterialsProcessChecking the labels didn’t form part of anyone’s duties.Significance of delay & the potential for an error not considered by cleaners.Labels were removed during the previous clean and not replaced.An operator should have weighed out ingredient A. However inadvertently used ingredient B.Operator trained but no sign off.Operator unfamiliar with procedure.SEE NOTE BELOWIf we use the same example as before with the use of the wrong ingredient we find:Go through slidesTHE FOLLOWING HAS GENERATED A LOT OF INTEREST ON BOTH OCCASSIONS THAT THE COURSE HAS RUN – I THINK IT WOULD BE WORTH GENERATING A NEW SLIDE WITH A SIMPLE TABLE ON IT SO THAT THE POINT CAN BE FULLY DISCUSSEDSome sites have indicated that they like the concept of Isikawa diagrams but find the diagram cluttered or unhelpful and is it therefore possible to use the concept without the diagram eg by tabulating and having a column for each of the sections instead of the fishbone diagram – the answer is YES – the important bit is getting to the root cause in a logical manner how that is done can be chosen by the site.Incomplete training procedure for cleaners.Both ingredients looked identical and were not labelled.EnvironmentEquipmentPeople
48 Pareto Principle Not a tool for Root Cause Analysis 80% ProblemsNot a tool for Root Cause AnalysisA tool for determining where to start your analysis20% CausesPareto's Principle is named after the man who first discovered and described the '80:20' phenomenon, Vilfredo Pareto ( ), an Italian economist and sociologist. Pareto was born in Paris, and became Professor of Political Economy at Lausanne, Switzerland in An academic, Pareto was fascinated by social and political statistics and trends, and the mathematical interpretation of socio-economic systems.Vilfredo Pareto first observed the 80/20 principle when researching and analysing wealth and income distribution trends in nineteenth-century England (some people suggest this was Italy; I say England, or Britain), in which, broadly he noted that 20 percent of the people owned 80 percent of the wealth. Beyond this he also noted that this 'predictable imbalance' could be extrapolated (extended) to illustrate that, for example, 10 per cent would have 65 percent of the wealth, and 5 percent of people would own 50 percent of the wealth. Again these other ratios are what Pareto found in this particular study - they are not scientific absolutes that can be transferred reliably to other situations.Pareto then tested his principle (including related numerical correlations) on other countries, and all sorts of other distribution scenarios, by which he was able to confirm that the 80:20 Principle, and similarly imbalanced numerical correlations, could be used reliably as a model to predict and measure and manage all kinds of effects and situations.80 percent of revenue comes from 20 percent of customers80 percent of problems come from 20 percent of causes80 percent of profit comes from 20 percent of the product range80 percent of complaints come from 20 percent of customers80 percent of sales will come from 20 percent of sales people
49 ManagementMaterialsProcessChecking the labels didn’t form part of anyone’s duties.Significance of delay & the potential for an error not considered by cleaners.Labels were removed during the previous clean and not replaced.An operator should have weighed out ingredient A. However inadvertently used ingredient B.Operator trained but no sign off.Operator unfamiliar with procedure.AMEND ANIMATION ON THIS SLIDE – IT SHOULD APPEAR ALL AT ONCE (ON SLIDE 25 ANIMATION IS NEEDED BUT NOT HERE)If we consider the previous exercise as an example – wrong ingredient man:Incomplete training procedure for cleaners.Both ingredients looked identical and were not labelled.EnvironmentEquipmentPeople
50 Proposed Action PlanUpdate training procedure to ensure sign off (& possibly a supervision step).Replace ingredient labels – if practical with ones that cannot be removed.If labels must occasionally be removed, ensure that post cleaning line checks include a check of signage.Ensure an individual is authorised and responsible for post-cleaning line sign off.Ensure cleaners fully understand and are trained in the need to return labelling (and all equipment) in a fully operational state.In this situation our PAP will have a number of aspects as there were a number of causes:Go through slideNB colour coding might be another option to labels
51 Common mistakes Unmanageable Conclusions Root cause analysis should lead to controllable, manageable or adjustable processes.Duplication of the Corrective ActionThe purpose of root cause analysis is to look beyond the immediate non-conformity.Proposed Action Plan Doesn’t PreventRe-occurrenceIt may be necessary to re-visit and identify additional causes and appropriate controls.PeopleThe true root cause establishes what system, policy or process allowed the human error.There are some common mistakes which we need to be aware of – as they can prevent effective RCA and PAP1: Read – However it is often tempting to conclude ‘not enough time or money’, ‘made a mistake’, or ‘staff off sick’, whilst these factors might be true they are not helpful in preventing future occurrence as they are not controllable. The investigator needs to consider ‘what failed?’ Was it a process or system or method that allowed these events to occur? For example, staff sickness should be managed by identifying and training suitable deputies.2: Read – Therefore the PAP should never be a duplication of the immediate corrective action.3: PAP doesn’t prevent re-occurrence of the problem – there are a number of possible reasons- incomplete initial RCA, failing to identify the true root cause- multiple root causes, some of which haven’t been identified or controlled- failure to implement the PAP or train the new procedures to staffIt will be necessary for the investigator to re-visit the analysis to establish why the PAP has not been effective.4: People are rarely the cause – RCA is not a blame game – the RCA must therefore consider what policy or procedure allowed the human error.In some situations PAP will involve aspects of training or re-training but where re-training is genuinely needed, consideration must be given to the format of the training, on the basis that, if the original training was ineffective then it is unlikely that just repeating the same training material will solve the problem.5: The final common mistake is the introduction of extra checks or monitoring. It is preferable that the PAP is not solely an extra check as:- monitoring isn’t solving the cause of the non-conformity- monitoring involves an on going commitment of time and therefore cost.Of course, some monitoring will be needed to ensure the PAP was effective but this shouldn’t be the PAP itselfExtra ChecksExtra checks are often required for verification or monitoring, it is preferable that the proposed action plan is not solely an extra check.
52 What does success look like Proposed action plan That preventsre-occurrence of thenon-conformity is in place.Preventive ActionWhat other systems exist that might have the same root cause present?Proposed Corrections Have been taken.VerificationDemonstratesnon-conformity can no longer occur.AMEND ‘PROPOSED CORRECTIONS’ TO ‘PROPOSED ACTION PLAN IMPLEMENTED’What does success look like?Firstly, that the RCA identifies the cause(s) allows PAP to be taken.That any relevant preventative action on other systems is addressed and completed.That the PAP has been completed and embedded into the normal working practices of the site.Any verification confirms that the non-conformity can no longer occur.
53 RCA and BRC Global Standards Food Standard Issue 6:ClauseRequirement1.1.10Senior management to ensure the root cause of audit non-conformities have been effectively addressed to prevent recurrence.3.7.1Identification of the root cause of non-conforming products and implementation of any necessary corrective action.3.10.1Root cause analysis and associated actions relating to customer complaints.Audit Protocol 9.2 & 9.2.2Following an audit, the root cause of non-conformities shall be identified and an action plan to correct this, including timescales, must be provided to the Certification Body.Consumer Products Standard Issue 32.4.2Review of risk assessment and evaluation of complaints / incidents3.7.4Corrective actions and prevention of re-occurrence3.12.4Analysis of complaint dataPackaging Standard (Issue 4)There are no requirements for root cause analysis in the Packaging Standard, but sites are required to state “corrective action taken (with consideration of root cause)” in the audit report.Go through clauses of the Standards
54 Consider… Could it have been prevented from happening? Could it have been identified earlier?Known’ s and unknowns…
55 Audit report How should RCA appear in the audit report? Explain audit reportGo through slide