Presentation on theme: "Exploring the Medical Laboratory Quality ToolBox-A Michael A Noble MD FRCPC Clinical Microbiology Proficiency Testing program Program Office for Laboratory."— Presentation transcript:
Exploring the Medical Laboratory Quality ToolBox-A Michael A Noble MD FRCPC Clinical Microbiology Proficiency Testing program Program Office for Laboratory Quality Management University of British Columbia Vancouver BC
Management by the Moment
You can try to fix them…
Management by the Moment But they never go away!
So the better approach is… An systemic approach of organization, plan, review, and action gives you the best chance of success.
The Quality Toolbox Lean RISK FMEA Six Sigma Metrics Surveys Statistics The Quality Toolbox
January What are quality tools? ISO 9001: 2000 Six Sigma LEAN A quality tool is a supplement or component part of a quality program that usually will not stand alone but can enhance the total quality system
Tools in the Toolbox Mega-Tools –Lean –Six Sigma –Priority Matrices and Risk Assessment Tools/Techniques –Balanced Scorecards –Brainstorming –Control Charting –Flow Charting –Quality Indicators –Surveys
Quality Indicators are Metrics (measured process information) Determine quality of services. Highlight potential quality concerns, Identify areas that need further study and investigation, and Track changes over time.
A really good, inexpensive reference book Amazon.com $30.00
Keeping Score Using the Right Metrics to Drive World Class Performance 1996 Many organizations spend thousands of hours collecting and interpreting data. However many of these hours are nothing more than wasted time because they analyze the wrong measurements, leading to inaccurate decision making. –Mark Graham Brown.
Characteristics of Good Metrics Good Metrics Measurable objective Achievable contained Interpretable specific Actionable action oriented Balanced full cycle Engaging all levels Timed short and long term
Seven Steps to Successful Indicators Do not start data collection until these are addressed 1.O bjective 2.M ethodology 3.L imits 4.I nterpretation 5.L imitations 6.P resentation 7.A ction plan
Developing Indicators Objective What are you trying to measure? 1.Why am I collecting this information? Be specific Methodology How to capture the data 1.What data needs to be captured 2.Who (or what) to capture the data 3.How often to capture the data 4.Is it achievable (time, resources, revenue)? Limits Can I preset levels for: 1.Acceptable, Concern, Unacceptable, Critical Presentation Graphic or Text Interpretation 1.What does it mean? 2.Does it reflect on YOUR quality? 3.Can I compare it? 4.Can I trend it? Limitations 1.Unintended variables 2.What does it not mean? Action Plan 1.What will I do if it indicates acceptable performance? 2.What will I do if it does not?
Developing Indicators Objective What are you trying to measure. 1.Why am I collecting this information? Be specific Methodology How to capture the data 1.What data needs to be captured 2.Who (or what) to capture the data 3.How often to capture the data 4.Is it achievable (time, resources, revenue)? Limits Can I preset levels for: 1.Acceptable, Concern, Unacceptable, Critical Presentation Graphic or Text Interpretation 1.What does it mean? 2.Does it reflect on YOUR quality? 3.Can I compare it? 4.Can I trend it? Limitations 1.Unintended variables 2.What does it not mean? Action Plan 1.What will I do if it indicates acceptable performance? 2.What will I do if it does not?
Setting Relevant Limits and Ranges Set Objectively Validate by Study Clinical Relevancy Customer Expectation Matched Benchmarks Regulation 60 minutes Relevant or Easy?
Assessing Quality Indicators Importance Potential for Improvement Scientific Acceptability Reliability and Validity Feasibility Implementation and cost Usefulness Comprehensive Having Quality Quality Indicators
IQLM Indicator List 1.Diabetes monitoring (system) 2.Hyperlipidemia screening (system) Test Order Accuracy and Appropriateness (pre-analytic) Patient Identification (pre-analytic) Adequacy and Accuracy of Specimen Information (pre-analytic) Blood Culture Contamination (pre-analytic) Accuracy of point-of-care testing (analytic) Cervical cytology/biopsy correlation (analytic) Critical Values Reporting (post-analytic) Turnaround time (post-analytic) Clinician satisfaction (post-analytic) Clinician follow-up (post-analytic)
CLMA Survey Pre-examination Phase Indicator List Ordered test is appropriate for patient care Patient consent appropriately collected Test utilization by clinician for best patient care Physician written order with every specimen Cost/benefit assessment for laboratory test menu Patient identification and its accuracy Preparation of patient for specimen collection Appropriate specimen container Timing of specimen collection Phlebotomy success Specimen integrity Specimen quantity Specimen transportation Accuracy of specimen identification Condition for specimen storage
CLMA Survey Examination Phase Indicator List Quality Control EQA-external quality assessment Time to first result availability Specimen contamination Laboratory injuries or accidents Competency of testing personnel Vacancy of technical staff
CLMA Survey Post-examination Phase Indicator List Result reporting accuracy Adequacy of information for interpretation of laboratory tests Report delivery turnaround time Consistency of critical values reporting Result interpretation by physician Patients satisfaction with laboratory services Patients satisfaction specifically with phlebotomy services Physicians satisfaction with laboratory services
Limits: Below 2% Interpretation: Meeting accepted limits all the time Limitations Definition may include some true infections and may miss others Presentation: Linear time graph Action plan: Identify and educate blood collector group. If stable for 2 years, then consider dropping from routine Objective: to ensure that blood culture results reflect sepsis. Methodology: Count single bottle positives of common skin flora/ total sets
Quality Indicators and Timing Use an indicator only as long as it provides you with useful information. Dont get tied to your indicators
Limits: Below 2% (?) Interpretation: Wards with inexperienced collectors have problems Limitations Some frail and elder people have very weak veins and may be impossible to collect Presentation: Linear time graph Action plan: Identify and educate blood collector group. Objective: to ensure that blood culture are properly filled. Methodology: Count underfilled bottles / total bottles collected
Caution about patient outcome indicators Theoretically, outcomes best assess quality, but they are the most difficult to measure –too many confusing variables Age, underlying conditions, therapy, circumstance –require high volumes of detailed data –need long collection periods David Hsia Medicare Quality Improvement Bad Apples or Bad Systems? JAMA. 2003;289:
Quality Indicators, Done Well, Will Consume More Time Than You Have FACT: Set Priority Set Limits Drop Non-Productive Activity Target: Dont be an Indicators Glutton
Computer Nonsense Metrics [urine culture] * [glucose] * [INR] [NUPA hr ] * [Telephone minutes ] X100 Just because a computer can calculate a value, doesnt mean that it should.
The BIG SECRET for Quality Indicator Team Engage the folks who do the work, because they know what they do!
Risk management is activity directed towards assessing, mitigating (to an acceptable level) and monitoring of risks to an enterprise. Risk Management helps define PRIORITIES and helps PREVENT ERROR
Tough Decisions 1 A group of 10 laboratories over 300 square miles considers centralizing all tests to a single facility. Improved cost efficiencies Simplified process control Closer oversight Improved utilization management Improved Patient Care Reduced clinical-lab interface Extended pre-examination phase Test delays Impeded Patient care
Tough Decision 2 A laboratory considers laboratory redesign based on LEAN analysis. Improved workflow Improved time efficiencies Solve ergonomic challenges Improved cost efficiencies Improved Patient Care Expensive renovations Construction interruptions Staff retraining Equipment specificity Test menu specificity Impeded Patient care
Tough Decision 3 Laboratory considers implementing a quality management team. Better information for better decisions Better process control Continual Improvement Process Improved Patient Care Staff Reallocation Increased immediate costs Quality Commitment Impeded Patient care
Tough Decision 4 A laboratory proposes the need for an on-site level 3 (increased biosafety level) microbiology laboratory. Improved biosafety Reduced laboratory infections Faster diagnosis Improved Patient Care Construction costs Operating costs Potential Bioterror target Impeded Patient care
Risk and Priority and Prevention are Relative Terms Risk tolerance Risk transfer Risk reduction Risk prevention Risk avoidance Risk aversion COMFORT ACTIONACTION PR IO RI TY
Risk Management Documents for the Medical Laboratory ISO 14971:2007 Medical devices -- Application of risk management to medical devices ISO 20993:2006 Biological evaluation of medical devices -- Guidance on a risk-management process ISO WD TS22367:2007 Medical laboratories -- Reduction of error through risk management and continual improvement
Linking Quality and Risk Managements
Introduction to ISO TS Risk management framework has been described as being of steps, planning for risk, identifying risk and its impacts, developing risk-handling strategies, and monitoring for risk control. These steps are consistent with management requirements as described in ISO 15189:2003 including identification and control of non-conformities, establishment of preventive and corrective actions, performance of internal audit and management review and continual improvement. It is the objective of this technical report to link these in the context of the medical laboratory.
Linking Prevention and Risk ISO TS Assessment of risk of deviations from standard The quality manager should establish and maintain a process for identifying incidents associated with deviations from standards requirements, estimating and evaluating the associated risks to patient care, and controlling these risks and monitoring effectiveness of the control. The process should include a prospective risk assessment method for processes considered as high risk. Assessment of potentially high risk processes may be based upon previous audit, survey, experience, or evidence-based literature on procedures where a failure may lead to significant safety risk to patients.
Linking Prevention and Risk ISO TS (Continued) –The quality manager should identify a team of people to study the selected process. Note 1: The team should have personal knowledge of the process. Note 2: The team should be comprised of people with different levels and types of knowledge. –The team should organize a thorough investigation of the process to include: a) each step of the process b) how each step of the process may fail c) how each failure at each step of the process may affect patient safety d) the rank of severity of each failure mode effect e) the most critical failure mode effects f) potential root causes of the most critical failure mode effects g) procedures to address the potential root causes. –The analysis of this Failure Mode Effects Analysis should form the basis of a Prevention Action Plan.
Severity – Occurrence Grid Numerical Values –Literature Benchmarks –Consensus –Experience Regardless of source, values should be both verified, and validated to fit the FMEA being performed.
HIV Testing Risk Evaluation Canadian versus American Risk Analysis Remote (0.2) Rare (0.4) Common (0.8) Frequent (0.9) Insignificant (0.1) Mild (0.2) Moderate (0.8) Severe (0.9) UNACCEPTABLE RISK ACCEPTABLE RISK ? UNACCEPTABLE RISK UNACCEPTABLE RISK DEBATABLE RISK UNACCEPTABLE RISK UNACCEPTABLE RISK ACCEPTABLE RISK DEBATABLE RISK ACCEPTABLE RISK DEBATABLE RISK DEBATABLE RISK
Quality Approaches Look Alike StrategyEvent Steps Corrective Action Plan PLANMONITOREXAMINE REMEDIATE AND CORRECT CONTROL Preventive Action Plan PLANIDENTIFYEXAMINE DEVELOP STRATEGY MONITOR Risk Management PLANIDENTIFYEXAMINE DEVELOP STRATEGY MONITOR Six Sigma DEFINEMEASUREANALYZEIMPROVECONTROL
Failure Mode Effects Analysis FMEA is a systematic method of studying failure. 1940sUS Armed Forces 1960sAerospace (Apollo Space program) 1970s Ford Motor Company (post-Pinto) 1990s semiconductors, food service, plastics, software, and diagnostics equipment. Documented FMEA is a REQUIREMENT in the automotive, in-vitro diagnostics, and pharmaceutical industries. October 1, 2008
FMEA: Failure Mode Effects Analysis In a process: – where are points that the process is likely to break down? –what is the probable occurrence of the different possible break-downs? –what are the likely severity and consequences of the different possible break-downs? –What is the priority list for addressing the different possible break-downs?
FMEA: Failure Mode Effects Analysis FMEA –Failure Mode Effects Analysis FMECA –Failure Modes Effects and Criticality Analysis FTA –Fault Tree Analysis HACCP –Hazards Analysis and Critical Control Points PDPC –Process Design Program Chart
A Medical Laboratory FMEA Physician Requisition Sample Collection Sample Set-up Sample Analysis Sample Transport
A Medical Laboratory FMEA Physician Requisition Sample Collection Sample Transport Sample Set-up Sample Analysis Good Sample Proper Container Site Identification Proper Packaging Courier competent and available Equipment and stains working properly Workload Time issues Expertise Labelling
A Medical Laboratory FMEA Physician Requisition Sample Collection 1.5 Sample Transport 0.5 Sample Set-up 1.9 Sample Analysis 1.3 Good Sample Proper Container Site Identification Proper Packaging Courier competent and available Equipment and stains working properly Workload Time issues Expertise Labelling 0.9 SEVERITY
A Medical Laboratory FMEA Physician Requisition Sample Collection 0.45 Sample Transport 0.8 Sample Set-up 0.25 Sample Analysis 1.05 Good Sample Proper Container Site Identification Proper Packaging Courier competent and available Equipment and stains working properly Workload Time issues Expertise Labelling 0.1 OCCURENCE
Pre-Examination Grid October 1, 2008 Remote 0.2 Rare 0.5 Common 0.8 Frequent >1 Insignificant (0.2) Mild (0.5) RequisitionTransport Moderate (0.8) Severe (>1) Set-upCollectionAnalysis
A warning about FMEA If done as a structured top-down exercise, many critical break points will be missed. If done as an unstructured bottom-up exercise, the team will get bogged down in minutiae FMEAs should ALWAYS be: Directed Team Efforts Planned Structured
The BIG SECRET for The FMEA Team Engage the folks who do the work, because they know what they do!
Sten Westgard MS August Even trying to implement just one of these programs requires more than tepid enthusiasm. And attempting to graft ISO or Lean or Six Sigma on top of current management practices is only a recipe for failure. None of these are add on programs – half-hearted or part-time implementations will produce only limited, temporary gains, at best. To enjoy enduring success, the organization must be willing modify its core DNA. So if your chief officers aren't willing to stake their careers (and their salaries) on the success of the program, then ISO/Lean/Six Sigma will be the equivalent of a one night stand: brief, exciting, but ultimately disappointing.