Presentation on theme: "PROFICIENCY TESTING A BRIEF OVERVIEW"— Presentation transcript:
1PROFICIENCY TESTING A BRIEF OVERVIEW CPT Anne SterlingLTC Paul Mann
2OUTLINE WHAT IS PT? & WHY DO WE DO IT? WHO DOES IT & HOW OFTEN? HOW IS IT EVALUATED?WHAT DO RESULTS MEAN?WHAT HAPPENS WHEN WE FAIL?HOW TO INVESTIGATE PT
3PROFICIENCY TESTING Is a form of external quality control Is a mechanism to ensure standardized testing across clinical laboratories and evaluate your lab’s performance in comparison to peer groups performance- Uses commercially available materials & evaluations- CAP Surveys
4WHY DO WE DO IT? CLIA 88 CLIP (CCLM) CAP TJC GOOD LABORATORY PRACTICES - Develops confidence in the accuracy and reliability of results
5WHO IT APPLIES TO? CLIA ‘88 - ALL TESTING LOCATIONS PERFORMING CLIP MODERATE OR HIGH COMPLEXITY TESTINGCLIP- ALL TESTING LOCATIONS PERFORMING MODERATE OR HIGH COMPLEXITY TESTING + Waived (Minimal Complexity) labs are required to be enrolled in a PT program if one is commercially available.
6WHO IS EXEMPT? CLIA ‘88 - WAIVED & PPM SITES CLIP LOCAL POLICY - nobodyLOCAL POLICY- Can be more stringent than CLIA/CLIP. This is what TJC/CAP will hold you to.
7WHAT IT APPLIES TO ? CLIA ‘88 - ALL REGULATED ANALYTES CLIP - ALL ANALYTES
8Regulated vs UnregulatedAnalytes Regulated Analytes: Listed in 42 CFR Part 493 Subpart IUnregulated analytes: those not listed in the CFR
9HOW OFTEN?CAP SURVEYS- THREE TIMES A YEAR (MINIMUM OF 5 SAMPLES FOR ALL REGULATED ANALYTES)Unregulated analytes usually have fewer challenges, fewer samples per challengeFOR TESTS WITH NO COMMERCIAL SURVEYS AVAILABLE- Alternate method (Split Testing) will be performed at least every 6 months
11e-Lab SolutionsThe best way to view CAP evaluations is to use e-Lab SolutionsRequest access from your institution’s administrator
12THREE COMPONENTS TO PT PERFORMANCE the actual resultthe target valuethe evaluation interval - or acceptable error - for that specimen.
13THREE DIFFERENT TYPES OF TARGET VALUES method group (“peer group”) meansmeans from another group or an all-results meanvalues derived from an external source (for example, reference laboratory consensus or definitive/reference methods).
14FOUR GENERAL TYPES OF EVALUATION INTERVALS fixed intervals (e.g., + 4 mmol/L)fixed percentages (e.g., + 10% of the target value)a combination of these two (e.g., + 6 mg/dL or 10 % of the target value, whichever is greater)intervals based on the group standard deviation (SD) (e.g., + 2 SD).
15WHAT DO THE RESULTS MEAN? WOW!!IF YOUR IN, YOUR GOOD- GUESS AGAINIF YOUR OUT, YOUR INCOMPETENT- PROBABLY, BUT MAYBE NOT
16CAP Evaluation What Should You Review? Evaluate each analyte and specimen for negative/positive bias, trends or shiftsEvaluate ungraded challenges
18PT Grading Policy Minimum passing score of 80%, 100% for ABO/Rh & Compatibility testingPassing grades on PT surveys with 5 challenges will require 4 correct, 4 challenges will require 3 correct, 3 or fewer challenges will require all to be correct.Challenge FailuresAnalyte Failures
19Proficiency Testing Exception Summary (PTES) Unsatisfactory PT performanceFailure to attain the minimum satisfactory score for an analyte, test, subspecialty, or specialty for a testing event.Unsuccessful PT performance (2 of 3 testing events)Failure to attain the minimum satisfactory score for an analyte, test, subspecialty, or specialty for two consecutive or two of three consecutive testing events.Critical PT performance (3 of 4 testing events)Failure to attain the minimum satisfactory score for an analyte, test, subspecialty, or specialty for three consecutive or three of four consecutive testing events. A laboratory must immediately cease testing for that analyte or the discipline.
23UNSUCCESSFUL STOP TESTING INVESTIGATE & DETERMINE CAUSE OF FAILURE EVALUATE PATIENT RESULTSINITIATE CORRECTIVE ACTIONTRAIN STAFFEVALUATE CORRECTIVE ACTION (verify on 2 separate occasions)DOCUMENTATIONREQUEST APPROVAL TO RESUME TESTING
24Do a Root Cause Analysis A rigorous systematic approach to answering:- What happened & Why?
25How do we do Root Cause Analysis? Said simply, Root Cause Analysis is asking why the problem occurred, and then continuing to ask why that happened until we reach the fundamental process element that failed.
26RCA Goals Find out: What happened? Why did it happen? What do you do to prevent it from happening again?How do we know we made a difference?
27Team Approach to RCAQASupervisorBench TechsDirector
28Involve Techs?Techs know what happens at the microprocess (bench) levelWill give them a sense of accomplishment/contributionInvolve in both the investigation and solution developmentTechs will be the ones that implement the solution
294Ms of a RCA Materials Machine / Equipment Methods Man (Management) Defective Survey MaterialWrong Survey MaterialMachine / EquipmentInstrument MalfunctionMaintenanceCalibrationQuality ControlMethodsSOP written , current, & availableSOP adequateSOP followedMan (Management)TrainingCompetencyResult EntryReview Process
30RCA Methods How do you determine the cause? Examine the original test print outs to ensure accurate entryVerify QC and look at Levy-Jennings graphs to see if there was a problem (drift or shift) that was not caughtRe-test CAP specimen to see if the same result is achievedSplit test with other instruments (other facility)Verify staff competency
31CLASSIFY THE PROBLEMUnacceptable results may be classified as follows:Clerical errorMethodological problemTechnical problemProblem with proficiency testing materialsProblem with evaluation of resultsNo explanation after investigation
33INITIATE CORRECTIVE ACTIONS PROVE ACCURACY & PRECISION OF METHOD- RECALIBRATION-PROCEDURAL UPDATE- SURVEY MATERIAL HANDLING & PROCESSINGRETRAININGNEW METHOD/INSTRUMENT
34EVALUATION OF PATIENT RESULTS Review patient data from the time of the unacceptable PT result, to determine whether the problem could have affected patient careIf so, appropriate follow-up action should be documented
35EVALUATE CORRECTIVE ACTIONS Ensure the validity of patient results by verifying on two consecutive occasions that the corrective action taken has resolved the problemThis maybe done by reanalysis and/or retesting of frozen or additional PT material, purchase of supplemental PT material, or blind, split-sample testing of patient material with another certified laboratory
36DOCUMENTATIONDocument investigation, conclusions, and corrective actions takenMaintain documentation for at least 2 years to include worksheets, instrument tapes, reporting forms, evaluation reports, participant summaries, and documentation of follow-up, as applicable. (5 years for Immunohematology) from the date of event.
38Director ReviewThe laboratory director will review the effectiveness of the corrective actions and, if satisfied, will document his/her recommendation whether to resume testing.
39Approval Process Unsatisfactory: Local Unsuccessful: RMC Critical: CCLM
40Critical 3rd of 4 PT events Failed to address the problem, identify the cause, and correct to avoid future problems
41REQUEST APPROVAL TO RESUME TESTING MEDICAL DIRECTORS APPROVAL- 1 of 2 or 1 of 3 unsatisfactory performanceRMC APPROVAL- 2 of 2 or 2 of 3 unsatisfactory performanceMEDCOM APPROVAL- 3 of 3 or 3 of 4 unsatisfactory performance
42Actions Laboratories Should Take when a PT Result is not Graded Code 11 – unable to analyzeCode 20 – No appropriate target/response cannot be gradedCode 21 – Specimen ProblemCode 22 – Result is outside the method/instrument reportable rangeCode 24 – Incorrect response due to failure to provide a valid response codeCode 25 – Inappropriate use of antimicrobial
43Actions Laboratories Should Take when a PT Result is not Graded cont’d Code 26 – Educational ChallengeCode 27 – Lack of participant or referee consensusCode 28 – Response qualified with a greater than or less than sign; unable to quantitateCode 30 – Scientific committee decisionCode 33 – Specimen determined to be unsatisfactory after contacting the CAPCode 40 – Results for this kit were not received
44Education Challenges Code 26 Phase IIIs there evidence of evaluation and, if indicated, corrective action in response to "unacceptable" results on the proficiency testing reports and results of the alternative performance assessment system?NOTE: The evaluation must document the specific reason(s) for the "unacceptable" result(s) and actions taken to reduce the likelihood of recurrence. This must be done within one month after the program receives its evaluation. In addition, each ungraded challenge, each educational challenge, and each episode of nonparticipation must be reviewed and corrective action instituted as appropriate.
45Actions Laboratories Should Take when a PT Result is not Graded cont’d Code 41 – Results of this kit were received past the due dateCode 42 – No credit assigned due to absence of responseCode 44 – This drug is not included in our test menu. Use of this code counts as a correct response
46CONCLUSIONEVERY EFFORT SHOULD BE MADE TO FIND THE CAUSE(S) OF AN UNACCEPTABLE PT RESULT.ACTIONS TO IMPROVE THE LABORATORY SYSTEM WILL MINIMIZE THE RISK OF RECURRENCE AND POTENTIALLY IMPROVE THE QUALITY OF PATIENT RESULTS.