THE VALUE OF CAP’S Q-PROBES & Q-TRACKS Peter J. Howanitz MD Professor, Vice Chair, Laboratory Director Dept. Of Pathology SUNY Downstate Brooklyn, NY 11203, USA Peter.Howanitz@downstate.edu

OVERVIEW Discuss History of Q-Probes & Q-Tracks Demonstrate Results Of Some Studies Discuss Impact On Pathology Improve Patient Care “Skunkworks” For College Of American Pathologists

HOW Q-PROBES & Q-TRACKS HAPPENED 1979-Chairman NYSSPATH QC Committee Workshop For Participants On QC Made Case For Pre & Post Analytical QC Chair Of CAP QC Committee 1982 Filled QA Workshop ASCP/CAP Meeting 1986 Began Pilot Planning 6 Quality Indicators 1987 Began Career 1976, and had two mentors, early in my career One in academics, other in CAP: was one of 2 major CP programs in U.S. In 1982, only QC and PT

TOTAL TESTING PROCESS Howanitz PJ Laboratory Quality Assurance McGraw Hill 1987 p 2

CAP’s Q-PROBES & Q-TRACKS Q-Probes Launched 1989 Peak Participants 1996-1700 Labs Q-Tracks Launched 1999 Both Programs Continue Today 25th Year Anniversary 2014 CAP began QC committee in 1971, and PT program began in 1968. Operated by QPC within CAP. Division of 2 committees in QAS QA and QAS QC in 1989. QAS QC operated until 1999. Lawson et al. Arch Pathol Lab Med. 1997; 121:1000-1008

WHY DEVELOP PROGRAM? Determined Pathology Quality Attributes Teach Laboratory Community QI JCAHO (TJC) Requires QI CLIA’88 Requires QA For All Steps Total Testing Process CAP Accreditation Requires QI Q-Tracks Best Drives Improvement Improve Patient Care No definition of what quality attributes were for a clinical laboratory. JCAHO now called TJC Did workshops with TJC, Participated in CLIA for CAP, and worked with CDCs No work on importance of Preanalytical and Postanalytical testing CLIA’88 is a regulation for clinical laboratories.

ADVANTAGES OF PROGRAMS Provide Educational Tools i.e. Publications Develop Benchmarks Provide “Off Shelf” Products Conserve Participant Resources Partially Fulfill Regulatory Requirements Help Pathologist With Leadership & Management

TYPES OF Q-PROBES STUDIES All Short Term Subscription Studies All Steps In Total Testing Process Extensive List Other Quality Indicators Safety Practices Competency Assessment Good Laboratory Practices Repeat Studies Similar To Snapshot We had discussions about what to do first. Decided to do Q-Probes, and then follow with Q-Tracks. Lots of management issues. Had data collection from within lab as well as a number of questions which accompanied study.

Q-TRACKS Ongoing Studies For Years Limited Number Of Studies Use Q-Probes Benchmarks Submit Data Every Quarter Similar To Movie 18 studies to date. We were unwilling to develop Q-Track until we had experience with the monitor.

HOW PROGRAMS WORK Studies Developed By Committee Field Evaluated Before Made Available Purchased By Participants Directions & Materials Participant Data Collection Data Sent To CAP For Analysis Benchmarks, Participant Data Returned In Critique Educational Tools Available Educational tools include publications, purchase old studies, teleconferences, workshops at National & International Meetings Used data collection worksheets

DATA-COMPLICATIONS OF PHLEBOTOMY Indicator 613 Institutions 10th Percentile 50th Percentile 90th Percentile Median Size Bruise (mm) 4048 Bruises 20.5 11.0 5.0 % Bruised Patients 32.0 16.7 7.1 % Pts Identifying Outstanding Employee 11107 Patients 25.6 46.7 69.8 Median Wait Time (Minutes) 23783 Patients 15.0 6.0 4.0 Early study. Uses percentile ranking with higher percentile means better performance. Data base is large. Phlebotomy is one of the Clinical Laboratory functions that directly influences patient care. Sometimes hard to know what is better performance. Must wait for 10-15 minutes before phlebotomy Howanitz et al. Arch Pathol Lab Med 1991: 115:867-872

BEDSIDE GLUCOSE ENABLERS INCREASED ACCURACY VARIABLE MEDIAN ACCURACY P VALUE Lab Personnel vs RN Responsible For Testing 67 vs 49 .0007 Lab Personnel Perform Testing 65 vs 53 .01 Nursing Personnel Not Performing Testing 63 vs 57 .04 Lab Personnel Performs Training 64 vs 50 .02 Lecture Used In Training Program 63 vs 45 Repeat Training/Performance Review Operators 63 vs 41 .0002 Regular Clinical Lab Result Correlations 63 vs 50 Regularly Compare Proficiency Results 62 vs 50 Participate In Bedside Glucose Proficiency Testing .03 Laboratorian vs RN Collected This Study Results 67 vs 51 Offers opportunity to study variables used to improve. Here 10 enablers studied on their effect to improve performance in 605 institutions with statisticians defining significance. Statiticians provided for each study, attended committee meetings, and help design studies. Some are expected (Lab Personnel vs RN Responsible for testing) Jones et al. Arch Pathol Lab Med 1993;177:1080-1087

SELECTED BENCHMARKS STEP Sample Size Median Benchmark Order Right Test 15,011 Tests 23.0% Anti-HBC Test, No AST, ALT Patient Prepared 18,679 Toxic Levels 24.4% Digoxin Collected > 6 Hrs Dosing Accurate Orders 224,431 Measurements 1.8% Test Ordered, Not Received Lab Patient Identified 451,436 Pts 6.5% Patients Wristband Incorrect Specimen Collection 29,700 Pts 6.0 min Timely Of Collection Specimens Rejected 35,325 Specimens 0.38% CBCs Rejected Results Evaluated 5837 Results 85.0% % Abnormal Results Documented QI Resources 9860 Indicators 40 h/Mo Time To Complete QI An example of benchmarks that were developed. They need to updated from time to time: Many studies in literature about single laboratories improvement-

COMPTENCY ASSESSMENT Howanitz et al Arch Pathol Lab Med:2000;124:195-202.

TROPONIN TURNAROUND TIMES 159 Hospitals, 1352 ED Physicians, 7020 troponin values One of most common studies, CSF , ED x 4, Routine test, outpatient testingUA, OR blood delivery, Early morning rounds collection, blood component preparation, 2 Q-Tracks Outliers and troponin, 6 AP studies. Q-Tracks study resulted in collaboration with Chest physicians. Novis, DA. Arch Pathol Lab Med 2004:128: 158-164

ED TURNAROUND RESULTS TAT of potassium and hemoglobin results from ED patients in 2 studies

CLINICAL LABORATORY ERROR RATES These are 12 of the studies we have conducted over time using Q-Probes. Looks at errors throughout the total testing process. Remember that there are many steps in this process of trying to get the results to a patient and the liklihood of not having an error is rare. Most errors in preanalytical and postanalytical areas. One of most difficult issues is ordering the correct test. Howanitz PJ Arch Pathol Lab Med 2005;129: 1252-1261

27 TURNAROUND TIME STUDIES CSF Analytes ED-(4) Routine Test Stat Test Outliers* Routine Outpatient Tests (2) Biochem Markers AMI* Reporting Positive Blood Cultures Morning Rounds Test Results Available* Blood Component Preperation OR Blood Delivery Urinalysis 27 studies, 10 AP, 14 CP Q-Probes and 3 Q-Tracks Have looked at all the disciplines of the clinical laboratory, heme, chem, TM, and MB. *Also Q-Tracks Studies

Q-TRACKS WRISTBAND ERROR RATES CONTINUOUS IMPROVEMENT Data from the first Q-Tracks study that was published. In US, wristbands are used to identify patients. When we published this, some from Europe commented that it was improper to use wristbands and that patients should not have numbers/ID. We argued it was a patient safety issue. Note that beginning new year with new participants adds a potential issue and rate in this case stays same. Howanitz et al. Arch Pathol Lab Med 2002: 126:809-815.

Q-TRACKS WRISTBAND ERROR RATES CONTINUOUS IMPROVEMENT Have Wristband errors continue to decrease overall over 4 years. We have studied some of these and found that rates continue to decrease even after 10 years!

VARABILES Q-TRACKS STUDY Q-Tracks program is able to monitor one variable that is important. Here for wristband errors, missing wristband was the largest source of errors. Conflicting wristbands include patients that have had their own wristband on Aggregate percentage of types of wristband errors (N=45197) for 2 years. Arch Pathol Lab Med. 2002:126: 809-815

PHYSICIANS’ 8 MOST IMPORTANT CLINICAL LABORATORY SERVICE ASPECTS Chose most important laboratory service characteristic. Other choices accounted for less than 1% of other characteristics. Also different for nurses.

COMMITTEES’ 8 MOST IMPORTANT QUALITY INDICATORS DISCIPLINE TESTING PROCESS Customer Satisfaction Entire Laboratory Entire Process Test Turnaround Times Each Discipline Blood Utilization Transfusion Medicine Preanalytical Patient Identification Blood Culture Contamination Microbiology Specimen Rejection Proficiency Testing Analytical Critical Value Reporting Post Analytical 8 include all major steps in total testing process, and each of the major clinical hospital laboratories. Howanitz PJ. Arch Pathol Lab Med 2005; 129: 1252-1261

Q-PROBES DEMOGRAPHICS AP NUMBER CP NUMBER Q-PROBES STUDIES 52 115 AUTHORS & COAUTHORS 33 50 ARCHIVES PUBLICATIONS 49 75 OTHER PEER REVIEWED PUBLICATIONS 18 21 NON-ARCHIVES CITATIONS ARCHIVES CITATIONS 1355 1609 PARTICIPANTS 15,406 42,663 COUNTRIES 24 CAP TODAY ARTICLES 17 55

PROGRAM ACHIEVEMENTS 17TH CAP CONFERENCE -300 PARTICIPANTS Arch Pathol Lab Med 1990:114:1101-177 Invited To Discuss Q-Probes @ Juran Institute International Meeting Identified 1 or 6 Outstanding Medicine Programs By Healthcare Forum Healthcare Forum J 1993: 36:37-52 Personal Awards, Careers CDC Finalists Best Manuscript 3 Times CAP conference open to public in 1990-Only CAP conference that was profitable. First CAP conference open to the public Juran Institute Conference in Atlanta by Invitation. 600 participants interested in Quality Improvement -1991 Healthcare forum

PROGRAM ACHIEVEMENTS Competency Assessment Program POCT Influence On CAP Accreditation Program Approved For Maintenance Of Certification Cytology Conference CDC Grant Specialty CAP Pathologist Certificate Program Evalumetrics Assessment of best practices for standardized QA activities-7 manuscripts published Awards from CAP, CDC, AACC Two committee members had individual grants for PI Committee members defined as experts, asked for interviews for magazines, newspapers, etc –may not be a good thing-testified in court cases.

41,000 enrollees. Idea sent to CAP leadership, began for POCT personnel and has grown.

THE JOINT COMMISSION REQUIREMENTS Organization Monitors Healthcare Quality Medical Staff Requirements Performance Data On All Physicians Ongoing i.e. Not At 2 Year Reappointment Process Department Specific Requirements Chair Of Department Responsible Med Staff Executive Committee Responsible Credentials Committee Responsible Earlier we talked about CLIA’88 and regulatory requirements. Also regulations from CAP, and from a group assuring quality in healthcare. When we stated Q-Probes, I participated in a number of jointly sponsored workshops between CAP and TJ.

EVALUMETRICS CAP Released 2013 2 Years In Development Software Designed In House Ongoing Professional Practice Evaluation Focused Professional Practice Evaluation Competency Program For Pathologists Over 60 Metrics On Introduction

EVALUMETRICS CP METRICS Metric Title Practice Area Description Laboratory Management TAT Core Timeliness Document Approval PT Peer Review Quality PT Review Transfusion RX Report Review Transfusion Medicine Written Report Review Bone Marrow Aspiration Hematology Properly Performing Procedure Protein Electrophoresis Peer Review Chemical Pathology Interpretation Concordance On Call Reliability General Pathology Available, Respond Promptly

CONCLUSIONS Discussed History of Q-Probes & Q-Tracks Demonstrated Results Of Some Studies Discussed Impact On Pathology Improved Patient Care “Skunkworks” For College Of American Pathologists-Innovation New Programs Questions