1 Linking Outcomes of Care to the ACGME Core Competencies: A Matrix Solution 3:15 pm – 3:25 pm IntroductionBerend Mets, MB, Ph.D., Moderator3:25 pm – 3:55 pm Embedding the Core Competencies Using the MatrixJohn BinghamDirector, Center for Clinical ImprovementVanderbilt University Medical Center Nashville Tennessee3:55 pm – 4:10 pm Question & Answer Session4:10 pm – 4:40 pm Practical Examples of the MatrixDoris Quinn, Ph.D.Assistant Professor, Division of Medical Education4:10 pm – 4:55 pm Question & Answer Session
2 John Bingham, MHA Director Center for Clinical Improvement Linking Outcomes of Care and the ACGME Core Competencies: A Matrix SolutionSAAC/AAPD Annual Meeting Washington, DC November 5, 2005Doris Quinn, PhDAssistant ProfessorDivision of Medical EducationJohn Bingham, MHADirectorCenter for Clinical Improvement
3 Objectives for today:Discuss the Institute of Medicine (IOM) Aims for Improvement and the ACGME Core Competencies.Describe how the Healthcare Matrix helps link outcomes of care to learning the core competencies.Provide examples of how the Healthcare Matrix is used to improve education and the delivery of care.Initiated in July 2001 for all Residencies and initial Board Certification (10 year phase-in process).Applicable to all subsequent Board Maintenance of Certification applications.
4 “Kyros” Events in Healthcare: 19992001200220032004Emerging public reporting of quality measures“Hospital Compare”
5 Extrapolated study results imply that between 44,000-98,000 U. S Extrapolated study results imply that between 44,000-98,000 U. S. hospital patients die each year as a result of medical errors.March 2000
6 And what about today?“Five Years After To Err is Human: What Have We Learned?”Lucian L. Leape, MD; Donald M. Berwick, MD JAMA, May 18, 2005“If the experience of the past 5 years demonstrates anything, it is that neither strong evidence of ongoing serious harm nor the activities, examples, and progress of a courageous minority are sufficient to generate the national commitment needed to rapidly advance patient safety.”
7 Patient Care should be: Efficient, Equitable, Patient-Centered Safe, Timely, Effective,Efficient, Equitable, Patient-Centered(STEEEP)
8 Phase I Phase II Phase III Phase IV 7/2001 6/2002 7/2002 6/2006 7/2006 6/20117/2011BeyondPhase IPhase IIPhase IIIPhase IVDefine specific objectives for residents to demonstrate learning of the competencies.Begin integrating the teaching and learning of competencies into residents’ didactic and clinical experiences.Improve the evaluation processes for all six of the Competencies.Provide aggregated resident performance data for Internal Review Process.Use resident performance data as the basis for improvement.Begin to use external quality measures to verify resident and program performance levels.Identify benchmark programs.Involve community in building knowledge about good GME.
9 “Clinical education simply has not kept pace with or been responsive enough to: shifting patient demographics,changed health system expectations,evolving practice requirements,new information,a focus on improving quality,new technologies.”
10 Emerging public reporting of quality measures “Hospital Compare”Emerging public reporting of quality measuresReporting of CMS Quality Measures tied to Annual “CMS Market Basket Update”November 2004“Recommend to Congress that it adopt pay-for-performance for physicians, hospitals, and home health agencies”Medicare Payment Advisory Commission: March 2005
11 The first Core Competency: Patient Care (Assessing it …and getting ready for physician report cards!)
12 Patients with Needs Met What are you measuring to evaluate the quality of Anesthesia care?Patients with NeedsPatients with Needs MetAccessDiagnosisTreatmentFollow-upAssessmentHow and where are these data reported?How is the information utilized to improve:the education of residents?the quality of care provided?
13 Patient Care should be: Safe, Timely, Effective,Efficient, Equitable, Patient-Centered(STEEEP)
14 Healthcare Matrix: Care of Patient(s) with…. AimsCompetenciesSAFETIMELYEFFECTIVEEFFICIENTEQUITABLEPATIENT-CENTEREDAssessmentPATIENT CARE(Overall Assessment)Yes/NoMEDICAL KNOWLEDGE(What must we know)INTERPERSONAL AND COMMUNICATION SKILLS(What must we say)PROFESSIONALISM(How must we act)SYSTEM-BASED PRACTICE(What is the Process?On whom do we depend and who depends on us)ImprovementPRACTICE-BASED LEARNING AND IMPROVEMENT(What have we learned, what will we improve)
16 PATIENT CARE that is…Safe“Avoiding injuries to patients from the care that is intended to help them”
17 What is our Anesthesia performance for: PATIENT CARE that is…SafeWhat is our Anesthesia performance for:% Patients with appropriate prophylactic antibiotic?% Cardiac Surgical Patients with controlled perioperative serum glucose (200 mg/dL)% Cases with documented Time Out?Intra- or postoperative:Cardiac arrest during hospitalization?PE during hospitalization?DVT during hospitalization?Anesthesia Complications/1000 surgeries?
18 Is Care Safe ? Observed to Expected Mortality: VUMC Goal: Achieve lowest mortality in nationVUMC 2004VUMC 2005Observed to Expected Mortality:53 UHC AMCs with Level I Trauma Centers
19 VUMC Observed to Expected Mortality and Actual Number of Mortalities 2003-2005
20 PATIENT CARE that is…TimelySafe“Reducing waits and sometimes harmful delays for both those who receive and those who give care”
21 What is our Anesthesia performance for: PATIENT CARE that is…SafeTimelyWhat is our Anesthesia performance for:% Patients with Anesthesia Prep Time < 15 Minutes?% Patients with on-time prophylactic antibiotics?% Patients with prophylactic antibiotics? discontinued <24 hours after surgery end time?% cases completed < 15% of scheduled length?% cases with surgical consent before day of surgery?Average time between cases (Gap Time)?Average time between “room ready” and “in room”?
22 Percentage of Surgery Patients Who Received Preventive Antibiotic (s) One Hour Before Incision Top Hospitals: 93%AVERAGE FOR ALL REPORTING HOSPITALS IN THE UNITED STATESAVERAGE FOR ALL REPORTING HOSPITALS IN THE STATE OF TENNESSEEVANDERBILT UNIVERSITY HOSPITAL69% 64% 47% Top Hospitals represents the top 10% of hospitals nationwide(Data displayed are from data reported July-Dec.04)
23 Patients with Needs Met AccessDiagnosisTreatmentFollow-upAssessmentWhat is the infection rate for surgical patients (in total, by procedure, by specialty, by surgeon; by site of surgery) ?Received prophylactic antibiotics?YesNoExceptions by procedure, by specialty, by surgeon; by site of surgery ?Received within one hour prior to surgical incision?%YesNoReceived the appropriate antibiotic?% with Infection% with InfectionNoYesExceptions% with Infection% with Infection
24 Percentage of Surgery Patients Whose Preventive Antibiotics are stopped Within 24 Hours After SurgeryTop Hospitals: 100%AVERAGE FOR ALL REPORTING HOSPITALS IN THE UNITED STATESAVERAGE FOR ALL REPORTING HOSPITALS IN THE STATE OF TENNESSEEVANDERBILT UNIVERSITY HOSPITAL64% 58% 78% Top Hospitals represents the top 10% of hospitals nationwide(Data displayed are from data reported July-Dec.04)
25 PATIENT CARE that is…SafeTimelyEffective“Providing services based on scientific knowledge to all who could benefit and refraining from providing services to those not likely to benefit”
26 What is our Anesthesia performance for: PATIENT CARE that is…SafeTimelyEffectiveWhat is our Anesthesia performance for:% Patients that received preoperative prophylaxis for VTE?% non-cardiac vascular surgery patient receiving beta-blockers during perioperative period% Patients with CAD who received beta blockers during perioperative period?% Patients on a ventilator whose post op orders included elevating bed >= 30 degrees?
28 PATIENT CARE that is…SafeTimelyEffectiveEfficient“Avoiding waste, including waste of equipment, supplies, ideas, and energy”
29 What is our Anesthesia performance (over time) for: PATIENT CARE that is…SafeTimelyEffectiveEfficientWhat is our Anesthesia performance (over time) for:Total cost per case?Supply cost per case?Supply waste per case?OR non-billable time delays due to Anesthesia?Rate of increase in revenue vs. expenses?
30 PATIENT CARE that is…SafeTimelyEffectiveEfficientEquitable“Providing care that does not vary in quality because of personal characteristics such as gender, ethnicity, geographic location, and socio-economic status”
31 AHRQ 2004 National Healthcare Disparities Report Is Care Equitable?AHRQ 2004 National Healthcare Disparities ReportReleased 2/22/2005Blacks:had worse access than whites for about 40% of access measuresreceived poorer quality for about 66% of quality measuresAsians:had worse access than whites for about 33% of access measuresreceived poorer quality than whites for about 10% of quality measuresHispanics:had worse access than non-Hispanic whites for about 90% of access measuresreceived lower quality of care than non-Hispanic whites for 50% of quality measuresPoor people:had worse access for about 80% of access measures than those with high incomesreceived lower quality of care for about 60% of quality measures
32 “Providing care that is respectful of, and responsive to: PATIENT CARE that is…TimelyEfficientEffectiveEquitablePatient CenteredSafe“Providing care that is respectful of, and responsive to:individual patient preferences,needs and values,and ensuring that patient values guide all clinical decisions”
33 Is Care Patient Centered? What are our patients’ perceptions of: HCAPS/CMS Patient Perception SurveysEffective in 2006-Public in 2007What are our patients’ perceptions of:Communications with Nurses?Communications with Doctors?Communications about medications?Nursing services?Pain management?The hospital environment?Adequacy of discharge information?Our system overall?Their willingness to recommend us?
34 What must we know?PATIENT CARE that is…TimelyEfficientEffectiveEquitablePatient CenteredSafeMedical Knowledge“…about established and evolving biomedical, clinical, and cognate sciences, (e.g. epidemiological and social-behavior) and the application of this knowledge to patient care”
35 What must we say?PATIENT CARETimelyEfficientEffectiveEquitablePatient CenteredSafeMedical KnowledgeInterpersonal and Communication Skills“…that result in effective information exchange and teaming with patients, their families, and other health professionals.”
36 How must we behave?PATIENT CARETimelyEfficientEffectiveEquitablePatient CenteredSafeMedical KnowledgeInterpersonal and Communication SkillsProfessionalism“…as manifested through a commitment to carrying out professional responsibilities, adherence to ethical principles, and sensitivity to a diverse patient population.”
37 What is the Process? On whom do we depend? Who depends on us? PATIENT CARESafeTimelyEffectiveEfficientEquitablePatient CenteredMedical KnowledgeInterpersonal and Communication SkillsProfessionalismSystem-Based Practice“…as manifested by actions that demonstrate an awareness of, and responsiveness to, a larger context and system of healthcare and the ability to effectively call on system resources to provide care that is of optimal value.”
38 What have we learned? What will we improve? PATIENT CARETimelyEfficientEffectiveEquitablePatient CenteredSafeMedical KnowledgeInterpersonal and Communication SkillsProfessionalismSystem-Based PracticePractice-Based Learning & Improvement“…involves investigation and evaluation of their own patient care, appraisal and assimilation of scientific evidence, and improvements in patient care.”
39 Linking it all together…. Patients with Needs Met AccessAssessmentDiagnosisTreatmentFollow-up-Medical Knowledge-Interpersonal and Communication Skills-Professionalism-Practice-Based Learning & ImprovementTimelyEfficientEffectiveEquitablePatient CenteredSafe-System-Based PracticeClinicians competent in:Patient Care that is…
44 IOM Anesthesia: One resident’s learning Case presentation preparation before expose to the MatrixIOMACGMESAFETYTIMELINESSEFFECTIVE-NESSEFFICIENCYEQUITA-BILITYPATIENTCENTERED-NESSPATIENT CAREMEDICAL KNOWLEDGE & APPLICATIONXPROFESSIONALISMINTERPERSONAL & COMMUNICATION SKILLSSYSTEMS- & TEAMS-BASED PRACTICEPRACTICE-BASED LEARNING & IMPROVEMENT(Process to Improve)
45 IOM Case presentation after dialogue with faculty using the Matrix. ACGMESAFETYTIMELINESSEFFECTIVE-NESSEFFICIENCYEQUITA-BILITYPATIENTCENTERED-NESSPATIENT CAREMEDICAL KNOWLEDGEXPROFESSIONALISMINTERPERSONAL & COMMUNICATION SKILLSSYSTEMS- & TEAMS-BASED PRACTICEPRACTICE-BASED LEARNING & IMPROVEMENT(Process to Improve)P and P changed for Mom/Child in troubleChanged STAT pages to Anes. From OBClass on care of Mom with DICProcedure outlined for fastest prep for ORAssure Mom aware of what is happening. Communication with father.
46 Patient with Pregnancy and D. I Patient with Pregnancy and D.I.C (Disseminated Intravascular Coagulopathy)Case PresentationIOMACGMESAFETIMELYEFFECTIVEEFFICIENTEQUITABLEPATIENT-CENTEREDAssessment of CarePATIENT CARE(Overall Assessment)NOPatient nearly diedLife saving treatment was delayed for variety of reasonsDelays in treatment impaired effectiveness of therapyResources (blood products, staff time) were not utilized in an efficient manner.Language was a problemPatient was not adequately apprised of her own health problems and did not participate fully in her care decisionsMEDICAL KNOWLEDGE(What must we know)Priorities in hemorrhagic shock are ABC: ensure oxygen delivery, support BP, aggressive IV resuscitation, treat causeHemorrhagic shock is life-threatening emergency: Prompt diagnosis, recognize urgency, initiate therapy, incl. timely transport to OR. Diagnosis was made late. No urgency to treat. Delay in contacting Anesth. Inadequate assistance in transport to ORD.I.C. in pregnancy: Physiology, diagnosis, causes, treatment. Regional v. General Anesth? Post resuscitation pulmonary edema. Hypocalcemia due to massive transfusion. Invasive monitoring indications. Pharmacology of uterotonic drugs.Survival in postpartum hemorrhage requires aggressive IV resuscitation: always consider combining procedures (start 2nd IV while drawing blood sample for transfusion cross match).INTERPERSONAL AND COMMUNICATION SKILLS(What must we say)Safety is jeopardized unless team members are fully apprised of patient’s condition (blood loss following delivery, vital signs, plans for intervention).Orders (blood cross match) must be prioritized and fully implemented in a timely fashion.Effectiveness of life-saving intervention depends on effective communication between team members.Communications of a defensive or argumentative nature are counter-productive to efficient and safe care. The focus should be patient care, with analysis of misunderstandings at a later time.Must communicate patient’s condition and intended interventions (blood transfusion, emergency hysterectomy), and in a way that is understandable and useful to the patient, respecting patient autonomy.PROFESSIONALISM(How must we act)Professional duty to accompany critically ill patient to the OR, to ensure safety, and to expedite therapy.Patient’s ethnic, socio-economic, “service patient” status should have no effect on quality of care.Professional duty to attempt to preserve patient autonomy (make sure patient understands situation and interventions)
57 Improvements From Medicine Residents: Pat Covington RN, ManagerEMR: We can now text message across departments.Use of pt waiting time: Have Kiosk in exam room to fill in review of systems. Questionnaires being sent to pts ahead of time. Those with get questionnaire and can return via .Availability of techs: Modified schedule of techs to improve service. Residents’ schedules were also changed to better utilize staff.Patient visit survey and phone calls will now be done after visit.Patient Letter revised: “Bring old records, come 15 minutes before appt.”
58 Transforming M&M Conferences into Practice-based Learning and Improvement
63 Care of Patient with Femoral Vein Cannulation COMMUNICA-TION SKILLS AIMSCompetenciesSAFETIMELYEFFECTIVEEFFICIENTEQUITABLEPATIENT-CENTEREDAssessmentPATIENT CARE(What must I do)NOPt not always safe as evidence by several adverse eventsYesNoNeed to find/learn best method.Evidence of Ultrasound for dialysis line placement.Not SureHow informed is patient/family? No post procedure instructions.MEDICAL KNOWLEDGE(What must I know) Need additional anatomy lessons for performing this procedure. Need to know what to do with arterial punctures. What to do when patient cannot be still?No guidelines in literature for Fem. Cannulation.HCT not efficient way to monitor bleedingINTERPERSONAL ANDCOMMUNICA-TION SKILLS(What must I say)Nurses need to know when cannula has been pulled in order to have more observationCommunicating use of Niagra cath that other areas have found less favorable.Use of patches used on other specialties for punctures not well known.Better instructions for patient and family.Nephrology M&M 4/2/04
67 Healthcare Matrix: Care of Patient with stroke - occlusion of the ICA of unknown origin. AIMSCompetenciesSAFETIMELYEFFECTIVEEFFICIENTEQUITABLEPATIENT-CENTEREDAssessment of CareI.PATIENT CARE(Overall Assessment)Yes/NoYesNoYes, from VUMC,No for placement.Yes from VUMCb/c of insurance issuesYes – patient was informed and incorporated in decision making processII. AMEDICAL KNOWLEDGE(What must we know)Yes. Everyone on the stroke service was on top of the latest in knowledge. Yes.II. BINTERPERSONAL AND COMMUNICATION SKILLS(What must we say) Yes. Communication between neuro and surgery was clear. Attendings and residents were in contact Yes –phone calls and meetings were used when things couldn’t wait for note in chart Yes and no –comm. Between medical teams was great. Ins issues led to placement problems though Yesyes Yes – always kept in mind patients perspectiveII. CPROFESSIONALISM(How must we act) Yes There was no breakdown in safety due to pro problems Yes – there were never any delays in doing anything for the pt in terms of pro Yes and no – pt was on service for a while,– but not really treated much better than othersII. DSYSTEM-BASED PRACTICE(On whom do we depend and who depends on us) Yes – patient was monitored and kept in system Yes and no– no delays in providing emergent care, but getting rehab was hard. No– consultants used appropriately. Problem was not in Vanderbilt system, but in insurance system Yes – all resources were used according to pts own goals for rehabImprovementIII.PRACTICE-BASED LEARNING AND IMPROVEMENT(What have we learned and what do we improve) Patient safety was maintained at all times. We still don’t know what caused stroke after surgery though. Rehab placement took too long – everyone worked hard, but maybe could have worked harder Care was administered effectively within limits – not much treatment for strokes like this yet A lot of energy and time was used ineffectively trying to place himEveryone worked hard for him because he was there so long and trying hard to rehab. not more than everyone else Pt was very involved in his own care and course and his wishes were always respected.
84 Using the Matrix History Physical Exam Labs Tests Consults Etc. Care of Patient (Matrix)Diagnosis
85 “Closing the Loop” Start with diagnosis as basis for assessment Identify issues of care related to Aims and CompetenciesIdentify lessons learned and improvement neededComplete action plan for improvements with accountabilities and timeline
92 How to Flowchart a Process On-line web site for Improvement education(Dr. Quinn’s current course being redesigned for managers and physicians)
93 On Transformation:“And one should bear in mind that there is nothing more difficult to execute, nor more dubious of success, nor more dangerous to administer than to introduce a new system of things; for he who introduces it has all those who profit from the old system as his enemies, and he has only lukewarm allies in all those who might profit from the new system.”Machiavelli
94 Healthcare Matrix Summary Points: Is a framework for integrating competencies into existing educational activitiesProvides a new mental model for Clinicians analyzing patient careFacilitates use of “resident performance data as the basis for improvement”Encourages use of “external quality measures to verify resident and program performance levels”Initiated in July 2001 for all Residencies and initial Board Certification (10 year phase-in process).Applicable to all subsequent Board Maintenance of Certification applications.
97 Internal Review Questionnaire Core Competencies How does your program provide education that develops patient care practice that is compassionate, appropriate and effective?How effective is that training?1Not effective2Somewhat effective3Moderately Effective4Effective5Very effective
98 Implementation Internal Review Process: Analyze responses to competency questionnaire and discuss with program director; suggest improvements if neededProvide information on competencies and use of MatrixOffer to assist in the integration of competencies in M&M and case conferences, etc.
99 ImplementationIntroduction to Matrix: Program Director or Dept. Chairs invite us to do lecture or Grand Rounds to introduce competencies and Matrix.Using the Matrix:Attend M&M or case conferences as observersNote the discussion on a blank Matrix showing which cells/competencies were discussed and which were omittedSend Matrix to program director and discuss next steps
100 Implementation Residents and the Matrix: Helpful hint: Residents fill in Matrix on their ownBest to let them struggle a little with the competencies as they think about care of their patientGet someone (coach) to review Matrix with themIf the situation/case is difficult, Dept Chair, Program Director and mentors may assist with filling out Matrix and presentationHelpful hint:Find a “coach” to help residents. At the outset, we work with the residents and faculty. Then Chief residents or interested faculty take the lead. Sometimes nurses can be coaches such as in Psychiatry at VU.
101 Phase I Phase II Phase III Phase IV 7/2001 6/2002 7/2002 6/2006 7/2006 6/20117/2011BeyondPhase IPhase IIPhase IIIPhase IVDefine specific objectives for residents to demonstrate learning of the competencies.Begin integrating the teaching and learning of competencies into residents’ didactic and clinical experiences.Improve the evaluation processes for all six of the Competencies.Provide aggregated resident performance data for Internal Review Process.Use resident performance data as the basis for improvement.Begin to use external quality measures to verify resident and program performance levels.Identify benchmark programs.Involve community in building knowledge about good GME.
102 Research Agenda to Validate Matrix (Based on Kirkpatrick, Evaluation of Training, 1994) Does the Matrix provide a useful framework for teaching and evaluating the performance of clinicians around the competencies?Phase I of ACGME :Define objectives for learningBegin integrating the teaching and learning of competencies into didactic and clinical educational experiences
103 Research Agenda to Validate Matrix What are we learning about the care (columns) and education (rows) from completed matrices?Phase II of ACGME:Improve the evaluation processes for all six of the CompetenciesProvide aggregated resident performance data for Internal Review Process
104 Research Agenda to Validate Matrix Are the behaviors of clinicians changing based on their completion of practice-based learning and improvement?Phase III of ACGME:Use resident performance data as the basis for improvementBegin to use external quality measures to verify resident and program performance levels
105 Research Agenda to Validate Matrix Are the processes and outcomes of care improving?Phase III of ACGME:Begin to link clinical quality indicators and patient surveys with educationPhase IV of ACGME:Adapt and adopt generalizable information about emerging models of excellence. Involve community building knowledge about good GME.
106 Learning Core Competencies Evaluation of Evaluation of Residents (2006) Tools (2011)Care of Patient with ……Suggested Tools based on Matrix dataAppropriate tools?AIMSCompetenciesSafeTimelyEffectiveEfficientEquitablePatient-CenteredAssessmentPatient CareMedicalKnowledgeInterpersonal Communication SkillsProfessionalismSystem-based PracticeImprovementPractice-based learning and ImprovementInformation Technology(Dr. Paul Batalden provided the idea for this graph)Based on matrices for a dept or diagnosis, which evaluation tools best fit the need?
107 Learning Core Competencies Evaluation of Evaluation of Residents (2006) Tools (2011)Care of Patient with ……Suggested Tools based on Matrix dataAppropriate tools?AIMSCompetenciesSafeTimelyEffectiveEfficientEquitablePatient-CenteredAssessmentPatient CareMedicalKnowledgeInterpersonal Communication SkillsProfessionalismSystem-based PracticeImprovementPractice-based learning and ImprovementInformation Technology(Dr. Paul Batalden provided the idea for this graph)Are the evaluation tools appropriate and providing useful data?