Presentation on theme: "Preparing the Future Primary Care Workforce Together"— Presentation transcript:
1 Preparing the Future Primary Care Workforce Together Primary Care Faculty Development Initiative (PCFDI)CBME in the Ambulatory SettingNov
2 Outline CBME background Frameworks and outcomes Key concepts and definitionsFrameworks and outcomesWhere we are/where we need to beThe role of milestones and entrustment in the assessment and evaluation of competence
3 Competency versus Competent Competency: an observable ability of a health professional, integrating multiple components such as knowledge, skills, values and attitudes.Competent: demonstrating the required abilities in all domains in a certain context at a defined stage of medical education or practice.Adapted from: The International CBME Collaborators, 20093
4 What does competency-based medical education means to you?
5 Competency-Based Medical Education is an outcomes-based approach to the design, implementation, assessment and evaluation of a medical education program using an organizing framework of competenciesBottom line:CBME = Outcomes-based Medical Education (OBME)So what is the outcome, and what is the framework?The International CMBE Collaborators 2009
6 The Framework: ACGME Competencies Medical knowledgePatient care and procedural skillsInterpersonal and communication skillsPractice-based learning and improvementSystems-based practiceProfessionalism
8 The OutcomeFrenk J, et al. Health professionals for a new century: transforming education to strengthen health systems in an interdependent world. Lancet. 2010
9 What is the outcome?A competent (at a minimum) practitioner aligned with:IOM Six Aims for QualityCMS Triple AimNational Priorities Partnership
10 Individual Physician Readiness: The Gaps Office-based Practice CompetenciesInter-Professional team skillsClinical IT Meaningful Use skillsPopulation management skillsReflective practice and CQI skillsCare CoordinationContinuity of CareLeadership and management skillsSystems thinkingProcedural SkillsThe health care system is complex. The comment is frequently made that it is not due to the physician. It’s the system, the payers, the incentives etc. However, you can look at physician performance. Jay Crossen did just that in recent graduates entering the Kiaser –Perm Health care system in California. This is the data on all trainees and is not specialty specific.Crosson Health Affairs 2011
11 Is CBME/OBME Just a “Fad”? Pet rocksLeisure suitsStreakingDisco musicYugosPokemonTickle me Elmo…probably not…
12 Is CBME/OBME a Paradigm Shift? Thomas Kuhn (1962): “Normal science, the activity in which most scientists inevitably spend almost all of their time, is predicated on the assumption that the scientific community knows what the world is like. Much of the success of the enterprise derives from the community’s willingness to defend that assumption, if necessary at considerable cost”Thomas S. Kuhn. The Structure of Scientific Revolutions. University of Chicago Press. Chicago Pg. 5.
13 Could the Same be True of UME and GME? “Normal medical education, the activity in which most faculty inevitably spend almost all of their time, is predicated on the assumption that the medical educational community knows what the world is like. Much of the success of the enterprise derives from the community’s willingness to defend that assumption, if necessary at considerable cost”Thomas S. Kuhn. The Structure of Scientific Revolutions. University of Chicago Press. Chicago Pg. 5.
14 Is CBME/OBME a Paradigm Shift? Maybe…but perhaps that is not the main point:CBME is yet another stage on what should be the ongoing evolution and improvement of medical educationThe focus on outcomes is worthy of our attention
15 The Transition to Competency Fixed length, variable outcomeStructure/ProcessKnowledge acquisitionSingle subjective measureNorm referenced evaluationEvaluation setting removedEmphasis on summativeCompetency BasedEducationCompetency BasedKnowledge applicationMultiple objective measuresCriterion referencedEvaluation setting: DOEmphasis on formativeVariable length, defined outcomeCaraccio et al 2002
16 The definition of expected outcomes or competencies MilestonesThe definition of expected outcomes or competencies
17 MilestonesA significant point in development that identifies the discrete knowledge, skills, and attitudes expected of learners as they progress through training.Milestones should enable the trainee, program and the certification board to know an individuals trajectory of competency acquisition.
18 Dreyfus & Dreyfus Development Model PGY3Expert/MasterPGY1ProficientMS4CompetentMS3Advanced BeginnerReality is skills develop over time and training. We expect different levels of competence for different skills. For example history takingNoviceTime, Practice, ExperienceDreyfus SE and Dreyfus HL. A 1980Carraccio CL et al. Acad Med 2008;83:761-7
20 “Curricular” Milestone ACGMECompetencyDevelopmental MilestonesInformingACGME CompetenciesApproximateTime FrameTrainee toAchieveStageAssessmentMethods/ToolsClinical skills and reasoningManages patients using clinical skills of interviewing and physical examinationHistorical Data GatheringAcquire accurate and relevant history from the patient in an efficiently customized, prioritized, and hypothesis driven fashionSeek and obtain appropriate, verified, and prioritized data from secondary sources (e.g. family, records, pharmacy)Obtain relevant historical subtleties that inform and prioritize both differential diagnoses and diagnostic plans, including sensitive, complicated, and detailed information that may not often be volunteered by the patient6 months9 months18 monthsStandardized patientDirect ObservationSimulationI specifically have a box around the 18 month milestone in preparation for the next slide…Sub-bullet“Curricular” Milestone
21 Milestones BenefitsProvide the learner with a clear path of progressionThere are no surprisesAllow for rich formative feedback. Learners know where they are and where they need to goDefine specific behaviors that can focus assessment
22 Milestones Criticisms Milestones are too reductionistChecklist = competenceChecking off a milestones list does not equal competent practice in a highly complex health care environmentOperationalize the milestones to develop and apply meaningful assessment and evaluation.
23 Entrustment/Entrustable Professional Activities (EPAs) A framework for work-based assessment?
24 Entrustable Professional Activities EPAs represent the routine professional-life activities of physicians based on their specialty and subspecialtyThe concept of “entrustable” means:‘‘a practitioner has demonstrated the necessary knowledge, skills and attitudes to be trusted to independently perform this activity.’’11Ten Cate O. Acad Med. 2007;82(6):542–547.
25 An Entrustable Professional Activity Part of essential work for a qualified professionalRequires specific knowledge, skill, attitudeAcquired through trainingLeads to recognized outputObservable and measureable, leading to a conclusionReflects the competencies expected…EPA’s together constitute the core of the professionHere is a table that was produced by Olle ten Cate who developed the framework for EPA’s and has been instrumental to the work of our community. I like this example because it provides some concrete description of what an EPA might look like –Part of the essential work for qualified professional – similar to what I just readRequires specific knowledge, skill, attitude – makes sense if it isAcquired though training - not something that you are born with or comes pre-packagedLeads to a recognized output – you know it when you see itObservable and measurable –Reflects the competencies expected (competencies – the things that they are able to do)The Milestones fit into this picture here with the last two bullets – where Milestones link to EPA’s – more on this to come.Also important to note that the EPA’s together constitute the core of the profession – emphasize togetherConsider these criteria when reviewing some examples of what I think are EPA’s.ten Cate et al.Acad Med 2007
26 “Entrustment in Medical Education” Focused assessments around whatfaculty and training programs already“entrust” trainees to do?Reflects the most important outcome of training: a trainee’s readiness to bear professional responsibility”Enables work-based assessment focusing on demonstrating competence in desired outcomes of training.
27 Baystate Ambulatory LMT Model Learners: have Direct supervisionFaculty member sees every patientManagers: have Indirect supervisionFaculty member discretion to see patientTeachers: Oversight from facultyResident discretion to allow patients to leave before preceptingAdapted from Sudeep K. Aulakh & Michael J. Rosenblum. Presented at ICRE 2012, Ottawa.
28 Ambulatory Milestone: Demonstrates patient-centered interviewing using the Invite, Listen, Summarize formatFailure -Frequently does not use these skillsNeeds work -Inconsistently uses these skillsCompetent -Consistently uses Invite & Listen; Summarizes in a reporter fashionProficient -Consistently uses all three skills, Summarizes interpreted informationExpert -Consistently uses all three skills, Summarizes interpreted information in complex casesCompetencies: Interpersonal communication & Patient CareAdapted from Sudeep K. Aulakh & Michael J. Rosenblum. Presented at ICRE 2012, Ottawa.
29 Competencies, Milestones and EPAs CharacteristicCompetenciesMilestonesEPAsGranularityLowModerate to HighLow to ModerateSynthetic/IntegratedModerateHighPracticality (application)Conceptual
30 The Synthesis – Analytic Tension Physicians do not apply each competency independently in caring for patientsAs a result, judging overall performance is a synthetic/integrative assessment activityHowever,You will often have to pull things apart (analysis) to create shared mental models and to provide meaningful and actionable feedback to the resident
31 What has this resident been entrusted to do? Lets watch a video.What has this resident been entrusted to do?If this were your institution, could you attest that the resident had the required competence to provide this care?
32 With Your Neighbor -Pick a goal of training that would meet a national priority for ambulatory-based care.Identify two to three entrustments in training that could serve as a focus for assessment in that competency?What assessment methods would you use?What process will you use to make an entrustment decision?What facilitators and barriers would you face?
33 Judgment and Synthesis: as Guiding Framework and Blueprint The “System”Accreditation:ACGME/RRCInstitution and ProgramResidentsProgram AggregationAssessments within Program:Direct observationsAudit and performance dataMulti-source FBPatient experienceSimulationITExamJudgment and Synthesis:CommitteeNAS MilestonesBoard ReportingNo AggregationFaculty, PDs and othersCertification:BoardMilestone and EPAsas Guiding Framework and Blueprint