1. PROFICIENCY TESTING A BRIEF OVERVIEW
CPT Anne Sterling
LTC Paul Mann

2. OUTLINE WHAT IS PT? & WHY DO WE DO IT? WHO DOES IT & HOW OFTEN?
HOW IS IT EVALUATED?
WHAT DO RESULTS MEAN?
WHAT HAPPENS WHEN WE FAIL?
HOW TO INVESTIGATE PT

3. PROFICIENCY TESTING Is a form of external quality control
Is a mechanism to ensure standardized testing across clinical laboratories and evaluate your lab’s performance in comparison to peer groups performance
- Uses commercially available materials & evaluations
- CAP Surveys

4. WHY DO WE DO IT? CLIA 88 CLIP (CCLM) CAP TJC GOOD LABORATORY PRACTICES
- Develops confidence in the accuracy and reliability of results

5. WHO IT APPLIES TO? CLIA ‘88 - ALL TESTING LOCATIONS PERFORMING CLIP
MODERATE OR HIGH COMPLEXITY TESTING
CLIP
- ALL TESTING LOCATIONS PERFORMING MODERATE OR HIGH COMPLEXITY TESTING + Waived (Minimal Complexity) labs are required to be enrolled in a PT program if one is commercially available.

6. WHO IS EXEMPT? CLIA ‘88 - WAIVED & PPM SITES CLIP LOCAL POLICY
- nobody
LOCAL POLICY
- Can be more stringent than CLIA/CLIP. This is what TJC/CAP will hold you to.

7. WHAT IT APPLIES TO ? CLIA ‘88 - ALL REGULATED ANALYTES CLIP
- ALL ANALYTES

8. Regulated vs UnregulatedAnalytes
Regulated Analytes: Listed in 42 CFR Part 493 Subpart I
Unregulated analytes: those not listed in the CFR

9. HOW OFTEN?
CAP SURVEYS
- THREE TIMES A YEAR (MINIMUM OF 5 SAMPLES FOR ALL REGULATED ANALYTES)
Unregulated analytes usually have fewer challenges, fewer samples per challenge
FOR TESTS WITH NO COMMERCIAL SURVEYS AVAILABLE
- Alternate method (Split Testing) will be performed at least every 6 months

10. HOW IS IT EVALUATED?

11. e-Lab Solutions
The best way to view CAP evaluations is to use e-Lab Solutions
Request access from your institution’s administrator

12. THREE COMPONENTS TO PT PERFORMANCE
the actual result
the target value
the evaluation interval - or acceptable error - for that specimen.

13. THREE DIFFERENT TYPES OF TARGET VALUES
method group (“peer group”) means
means from another group or an all-results mean
values derived from an external source (for example, reference laboratory consensus or definitive/reference methods).

14. FOUR GENERAL TYPES OF EVALUATION INTERVALS
fixed intervals (e.g., + 4 mmol/L)
fixed percentages (e.g., + 10% of the target value)
a combination of these two (e.g., + 6 mg/dL or 10 % of the target value, whichever is greater)
intervals based on the group standard deviation (SD) (e.g., + 2 SD).

15. WHAT DO THE RESULTS MEAN?
WOW!!
IF YOUR IN, YOUR GOOD
- GUESS AGAIN
IF YOUR OUT, YOUR INCOMPETENT
- PROBABLY, BUT MAYBE NOT

16. CAP Evaluation What Should You Review?
Evaluate each analyte and specimen for negative/positive bias, trends or shifts
Evaluate ungraded challenges

17. EVALUATE RESULTS QUANTITATIVE QUALITATIVE - MEAN +/- 2 SD
- EVALUATE BIAS
- EVALUATE CV
- EVALUATE TRENDS
QUALITATIVE
- INTENDED RESPONSE
- MAJORITY OF RESULTS

18. PT Grading Policy Minimum passing score of 80%,
100% for ABO/Rh & Compatibility testing
Passing grades on PT surveys with 5 challenges will require 4 correct, 4 challenges will require 3 correct, 3 or fewer challenges will require all to be correct.
Challenge Failures
Analyte Failures

19. Proficiency Testing Exception Summary (PTES)
Unsatisfactory PT performance
Failure to attain the minimum satisfactory score for an analyte, test, subspecialty, or specialty for a testing event.
Unsuccessful PT performance (2 of 3 testing events)
Failure to attain the minimum satisfactory score for an analyte, test, subspecialty, or specialty for two consecutive or two of three consecutive testing events.
Critical PT performance (3 of 4 testing events)
Failure to attain the minimum satisfactory score for an analyte, test, subspecialty, or specialty for three consecutive or three of four consecutive testing events. A laboratory must immediately cease testing for that analyte or the discipline.

20. WHAT HAPPENS WHEN WE FAIL?

21. UNSATISFACTORY BE PROACTIVE INITIATE CORRECTIVE ACTION
- determine why
- prevent 2nd unsatisfactory performance
INITIATE CORRECTIVE ACTION
PREVENT LIMITATION OF SERVICES

22. 2nd PT Failure

23. UNSUCCESSFUL STOP TESTING INVESTIGATE & DETERMINE CAUSE OF FAILURE
EVALUATE PATIENT RESULTS
INITIATE CORRECTIVE ACTION
TRAIN STAFF
EVALUATE CORRECTIVE ACTION (verify on 2 separate occasions)
DOCUMENTATION
REQUEST APPROVAL TO RESUME TESTING

24. Do a Root Cause Analysis
A rigorous systematic approach to answering:
- What happened & Why?

25. How do we do Root Cause Analysis?
Said simply, Root Cause Analysis is asking why the problem occurred, and then continuing to ask why that happened until we reach the fundamental process element that failed.

26. RCA Goals Find out: What happened? Why did it happen?
What do you do to prevent it from happening again?
How do we know we made a difference?

27. Team Approach to RCA QA
Supervisor
Bench Techs
Director

28. Involve Techs?
Techs know what happens at the microprocess (bench) level
Will give them a sense of accomplishment/contribution
Involve in both the investigation and solution development
Techs will be the ones that implement the solution

29. 4Ms of a RCA Materials Machine / Equipment Methods Man (Management)
Defective Survey Material
Wrong Survey Material
Machine / Equipment
Instrument Malfunction
Maintenance
Calibration
Quality Control
Methods
SOP written , current, & available
SOP adequate
SOP followed
Man (Management)
Training
Competency
Result Entry
Review Process

30. RCA Methods How do you determine the cause?
Examine the original test print outs to ensure accurate entry
Verify QC and look at Levy-Jennings graphs to see if there was a problem (drift or shift) that was not caught
Re-test CAP specimen to see if the same result is achieved
Split test with other instruments (other facility)
Verify staff competency

31. CLASSIFY THE PROBLEM
Unacceptable results may be classified as follows:
Clerical error
Methodological problem
Technical problem
Problem with proficiency testing materials
Problem with evaluation of results
No explanation after investigation

32. Reasons for Failure

33. INITIATE CORRECTIVE ACTIONS
PROVE ACCURACY & PRECISION OF METHOD
- RECALIBRATION
-PROCEDURAL UPDATE
- SURVEY MATERIAL HANDLING & PROCESSING
RETRAINING
NEW METHOD/INSTRUMENT

34. EVALUATION OF PATIENT RESULTS
Review patient data from the time of the unacceptable PT result, to determine whether the problem could have affected patient care
If so, appropriate follow-up action should be documented

35. EVALUATE CORRECTIVE ACTIONS
Ensure the validity of patient results by verifying on two consecutive occasions that the corrective action taken has resolved the problem
This maybe done by reanalysis and/or retesting of frozen or additional PT material, purchase of supplemental PT material, or blind, split-sample testing of patient material with another certified laboratory

36. DOCUMENTATION
Document investigation, conclusions, and corrective actions taken
Maintain documentation for at least 2 years to include worksheets, instrument tapes, reporting forms, evaluation reports, participant summaries, and documentation of follow-up, as applicable. (5 years for Immunohematology) from the date of event.

38. Director Review
The laboratory director will review the effectiveness of the corrective actions and, if satisfied, will document his/her recommendation whether to resume testing.

39. Approval Process Unsatisfactory: Local Unsuccessful: RMC
Critical: CCLM

40. Critical 3rd of 4 PT events
Failed to address the problem, identify the cause, and correct to avoid future problems

41. REQUEST APPROVAL TO RESUME TESTING
MEDICAL DIRECTORS APPROVAL
- 1 of 2 or 1 of 3 unsatisfactory performance
RMC APPROVAL
- 2 of 2 or 2 of 3 unsatisfactory performance
MEDCOM APPROVAL
- 3 of 3 or 3 of 4 unsatisfactory performance

42. Actions Laboratories Should Take when a PT Result is not Graded
Code 11 – unable to analyze
Code 20 – No appropriate target/response cannot be graded
Code 21 – Specimen Problem
Code 22 – Result is outside the method/instrument reportable range
Code 24 – Incorrect response due to failure to provide a valid response code
Code 25 – Inappropriate use of antimicrobial

43. Actions Laboratories Should Take when a PT Result is not Graded cont’d
Code 26 – Educational Challenge
Code 27 – Lack of participant or referee consensus
Code 28 – Response qualified with a greater than or less than sign; unable to quantitate
Code 30 – Scientific committee decision
Code 33 – Specimen determined to be unsatisfactory after contacting the CAP
Code 40 – Results for this kit were not received

44. Education Challenges Code 26
Phase II
Is there evidence of evaluation and, if indicated, corrective action in response to "unacceptable" results on the proficiency testing reports and results of the alternative performance assessment system?
NOTE: The evaluation must document the specific reason(s) for the "unacceptable" result(s) and actions taken to reduce the likelihood of recurrence. This must be done within one month after the program receives its evaluation. In addition, each ungraded challenge, each educational challenge, and each episode of nonparticipation must be reviewed and corrective action instituted as appropriate.

45. Actions Laboratories Should Take when a PT Result is not Graded cont’d
Code 41 – Results of this kit were received past the due date
Code 42 – No credit assigned due to absence of response
Code 44 – This drug is not included in our test menu. Use of this code counts as a correct response

46. CONCLUSION
EVERY EFFORT SHOULD BE MADE TO FIND THE CAUSE(S) OF AN UNACCEPTABLE PT RESULT.
ACTIONS TO IMPROVE THE LABORATORY SYSTEM WILL MINIMIZE THE RISK OF RECURRENCE AND POTENTIALLY IMPROVE THE QUALITY OF PATIENT RESULTS.

47. QUESTIONS?