Bruce Britton, MD Professor, Medical Student Education Director

Why Weight? A Consistent Evaluation Process in which Assessment Drives Learning
Bruce Britton, MD Professor, Medical Student Education Director Mary McBride Medical Student Education Coordinator Eastern Virginia Medical School Department of Family and Community Medicine Norfolk, VA Handouts MITI3, MIRS, sp cheat sheets, SP faculty Rubric, syllabus/overview, bluecard, NCC curriculum, grade sheet on health literacy, grade sheet on QI, Assignment for Transitions, Shelf Review Grade, Professionalism Rubric, Paper Mid/final Clerkship evaluation for community preceptors, Community site assignment, Journal Club questions, Student comments.

Disclosures This project is supported by the Health Resources and Services Administration (HRSA) of the US Department of Health and Human Services (HHS) under grant number D56HP20782 – Pre-Doctoral Education Advancing Community Health (PEACH) for $2.1 million over 5 years. This information or content and conclusions are those of the author and should not be construed as the official position or policy of nor should any endorsements be inferred by HRSA, HHS, or the US government. PEACHPredoctoral Educational Advancing Community Health (PEACH) is a five-year project focusing on increasing the number of medical school graduates choosing to become primary care physicians who desire to practice in medically underserved communities (MUC). The dual foci of PEACH are to increase medical students’ positive exposures to medically underserved communities (MUC) and to increase their experience and confidence with the Patient Centered Medical Home (PCMH) model of care.

Objectives Review current clerkship assessment structure to reduce inconsistencies Incorporate STFM National Clerkship Curriculum (NCC) goals and objectives with American Council on Graduate Medical Education (ACGME) core competencies into clerkship assessment Integrate clerkship experiences and weighted assessment methods into curriculum

Common Clerkship Assessment Issues
Patient care and procedural skills Community faculty as preceptors – standardized evaluation? Simulated cases – student acceptance? Medical knowledge How to weight? Practice-based learning and improvement How to assess? Interpersonal and communication skills How to incorporate faculty, peers, patient/ family, and other staff? Professionalism How to weight, make objective? Systems-based practice ACGME Core Competencies. Evaluation assessment instruments of choice: fmCases, National Shelf, clerkship specific examination Patient care: who assesses, how do you make is standardized, how do you weight it Communications Skills: where are you going to get a complete picture of students ability to communicate well in all of the domains for physicians: patients, staff, peers, idea: put required evaluation by community faculty on how well they worked with staff at office back in…..NEED HANDOUT Discussion : ask for other areas that there are concerns for objective reproducible assessments that drive learning and accurately evaluate student attainment

EVMS’s Family Medicine Clerkship
What do we want our students to learn? STFM NCC goals and objectives How do we teach what we want our students to learn? Clinical experiences, didactics, simulated cases, flipped learning, peer to peer teaching, and other projects How to do formative and summative assessments to accomplish this? Use of Bloom’s taxonomy and ACGME core competencies, weight based on importance Bottom line: Assessment drives learning Assessment Drive Learning-we all know that students will learn based on how they are evaluated, so there needs to a rationale for what we need and how we assess to drive learning.

Bloom’s taxonomy and ACGME core competencies
Knowledge Medical knowledge Skills Patient care and procedural skills Interpersonal and communication skills Attitudes Practice-based learning and improvement Systems-based practice Behaviors Professionalism Patient Care: Identify, respect, and care about patients' differences, values, preferences, and expressed needs; listen to, clearly inform, communicate with and educate patients; share decision making and management; and continuously advocate disease prevention, wellness, and promotion of healthy lifestyles, including a focus on population health. Medical Knowledge: Established and evolving biomedical, clinical, and cognate (e.g. epidemiological and social behavioral) sciences and the application of knowledge to patient care. Practice-Based Learning and Improvement: Involves investigation and evaluation of one's own patient care, appraisal and assimilation of scientific evidence, and improvements in patient care. Interpersonal and Communication Skills: That result in effective information exchange and teaming with patients, their families and other health professionals. Professionalism: Commitment to carrying out professional responsibilities, adherence to ethical principles and sensitivity to a diverse patient population. Systems-Based Practice: Actions that demonstrate an awareness of and responsiveness to the larger context and system of health care and the ability to effectively call on system resources to provide care that is of optimal value.

Setting the Stage for the Curriculum
Provide meaningful rationale of curriculum and weighting Coordinate with other clerkships to ensure experiences are novel or complementary, not redundant Teach Bloom’s taxonomy and ACGME core competencies to ponder use for evaluation in residencies What kind of physician do you want for your self? Go over key competencies that you would want for your personal physician. Tell the students that we will be teaching them to the STFM NCC with the goal of learning like a family medicine resident-what you need to know to succeed in FM residency. Look at school educational needs for LCME and connect with STFM NCC-to make sure not a rehash. ex.dermatology changed to not be redundant, add skills. Make the percentages understandable with clear evaluation cut offs-especially that there is clear need to pass each component of blooms or acgme core to complete clerkship.

Comments and Discussion

National Clerkship Curriculum
Instructional goals and objectives based on acute, chronic, and preventive care experiences Patient care and procedural skills, medical knowledge, and interpersonal and communication skills assumed in all parts of recommended curriculum Practice-based learning and improvement (PBLI), professionalism (PR), and systems-based practice (SBP) highlighted in recommended experiences Following many of the objectives, we include a designation for the corresponding ACGME competency. These appear as abbreviations in the parentheses for the competencies of problem-based learning and improvement (PBLI), professionalism (PR), and systems-based practice (SBP). We did not highlight the remaining competences of medical knowledge, patient care, and interpersonal communication because we believe that all objectives incorporate these aspects.

NCC Principles of Family Medicine
NCC Curriculum Biopsychosocial model Comprehensive care Contextual care Continuity of care Coordination and complexity of care Experience and Measure Blackboard didactics with quizzes before classroom discussions Flipped standardized patient simulations with faculty and SP evaluations on efficient and thorough history, exam, and management Simulations have been designed so that students have the knowledge of medicine before they see the standardized patients, but they don’t know what are the barriers. Emphasis on highly effective patient communications skills to efficiently understand culture, context, literacy, understanding and motivation to follow plan of care.

Standardized Patient Simulations
Community health center residency clinic setting Flipped cases – Medicine is available before SP simulation Cases designed to emphasize principles Example: Series of 3 cases connected by family, continuity, culture/ context, and multiple layers – accepted as STFM NCC resource

Examples of SP Simulations
Woman with possible scaphoid fracture caused by being pushed by husband Teen pregnancy from same family Follow up visit of possible scaphoid fracture with IBS symptoms Pre read on wrist pain evaluation and IPV screen; video on FOOSH injury Pre read on counseling of teen pregnancy Pre read on abdominal pain and IBS NCC Acute Chronic NCC submission Rubino EVMS

SP Simulations Assessments and Evaluations
Formative assessment tools recorded by standardized patients Formative assessment tools used by faculty (both medical and behavioral) Summative evaluation MIRS: patient centered communication MITI: motivational interviewing ABIM: patient satisfaction Global adherence: readiness Medical history/ exam/ management key teaching points Faculty assessment Post case group discussion and video recorded examples Using above combined SP and faculty Bring copies of IPV cheat sheets and faculty evaluation. EVMS students USMLE CS communications scores mean 1 SD above national average.

SP Simulations Competencies Evaluated
Bloom’s taxonomy ACGME core competencies Knowledge Medical knowledge Skills Patient care and procedural skills Interpersonal and communication skills Attitudes Behaviors

SP Learning Objectives Met
Discuss principles of family medicine care Gather information, formulate differential diagnoses, and propose plans for initial evaluation and management of patients with common presentations Manage follow-up visits with patients having one or more common chronic diseases (multiple SP cases) Develop evidence-based health promotion/ disease prevention plans for patients of any age or gender Demonstrate competency in advanced elicitation of history, communication, physical examination, and critical thinking skills Discuss critical role of family physicians within health care system (referral to ED SP case) Each of the cases can be used to meet NCC spectrum of cases seen in acute/chronic/preventive care.

NCC Student Learning Objective
Discuss critical role of family physicians within any health care system Blackboard quizzes on American health care system and changes due to Affordable Care Act (ACA) Health disparities and effect of primary care Student feedback on 3 populations: urban, rural, and “special” Hospital transitions assignment

Family Medicine in Healthcare System Competencies Evaluated
Bloom’s taxonomy ACGME core competencies Knowledge Medical knowledge Skills Patient care and procedural skills Interpersonal and communication skills Attitudes Practice-based learning and improvement Systems-based practice Behaviors

NCC Learning Objective: Patient Education
Provide patient education tools taking into account literacy and cultural factors Team based health literature development for <5th grade literacy level for student run free clinic and safety net clinics Community faculty evaluation of student teaching patient in practice setting

Patient Education Competencies Evaluated
Bloom’s taxonomy ACGME core competencies Knowledge Medical knowledge Skills Patient care and procedural skills Interpersonal and communication skills Attitudes Practice-based learning and improvement Systems-based practice Behaviors

NCC Learning Objective: Information Gathering and Assessment
Use critical appraisal skills to assess validity of resources Shelf review questions from last year’s family medicine residency in-service training examination Pre read on effective presentation/ teaching skills Team PowerPoint presentation – evaluation based on effective presentation with team based grade Team Journal Club 3 published research paper reviews – evaluation based on effort and depth of answers to questions posed by faculty

Information Gathering Competencies Evaluated
Bloom’s taxonomy ACGME core competencies Knowledge Medical knowledge Skills Patient care and procedural skills Interpersonal and communication skills Attitudes Behaviors

NCC Learning Objective: Team Approach & Quality and Safety
Coordination/ complexity of care, team approach, and quality and safety Team approach to projects with shared grade PCMH QI Project evaluated on originality, research, presentation, and usefulness Hospital transitions assignment evaluated on effort, depth of review, and usefulness

Team & Quality Improvement Competencies Evaluated
Bloom’s taxonomy ACGME core competencies Knowledge Medical knowledge Skills Patient care and procedural skills Interpersonal and communication skills Attitudes Practice-based learning and improvement Systems-based practice Behaviors Professionalism

NCC Learning Objective: Barriers to Access
Describe barriers to access and utilization of health care that stem from personal barriers 3 populations: urban, rural, and “special” Describe barriers that patients encounter to accessing and utilizing health care that stem from particular community HOPES student run free clinic – evaluation as teachers by M1/M2 peers Describe barriers stemming from health care system that affect ability of patients to obtain and use health care Community safety net site assignment – evaluated on effort, depth of written answers to questions

Barriers to Access Competencies Evaluated
Bloom’s taxonomy ACGME core competencies Knowledge Medical knowledge Skills Patient care and procedural skills Interpersonal and communication skills Attitudes Practice-based learning and improvement Systems-based practice Behaviors Professionalism

Professionalism Assumed not earned
Lack of professionalism removes point(s) Point removal anchors provided Examples provided

Professionalism All students in M3 and M4 clinical rotations will be awarded 10 professionalism points for each rotation. Students with professionalism point deductions will receive a maximum final grade as outlined below for the clerkship. Honors = 9-10 points High Pass = 8 points Pass = 7 points Not meeting minimal standards (Fail) = <6 points

Pass/ Fail Assignments
Patient encounter logging Radiology MedU Core radiology cases tied to SP simulation cases Final community faculty evaluation Student evaluation of educational sites and preceptors/ faculty members

Clerkship Evaluations (Non Weighted)
Formative feedback– no points toward final evaluation Self assessment Mid clerkship community faculty Community sites/ staff Peers SP/ faculty simulation cases Review of progress on patient logs and assignment feedback

Why Weight?

Clerkship Assessment Bloom’s taxonomy: knowledge
ACGME competency: medical knowledge NBME subject exam (includes musculoskeletal module) 7th percentile minimum passing score Weight: 25% Mean score of NMBE shelf so far is 72.3.

Clerkship Assessment Bloom’s taxonomy: skills
ACGME competencies: patient care and procedural skills Simulated (standardized patient) cases No points through 6th week 5% for 7th week experience 20% for 8th week experience (final) Weight: 25% Students must get medicine correct to pass; receive honors if done efficiently and comprehensively and with high patient satisfaction and adherence to plan USMLE Step 2 CS communications score mean 1 SD above national average.

Clerkship Assessment Bloom’s taxonomy: knowledge, skills, abilities, and behaviors ACGME competencies: medical knowledge, patient care and procedural skills, interpersonal and communication skills, practice-based learning and improvement, and professionalism Team-based learning 5% team shelf review 10% team health literature project 10% team PCMH QI project Weight: 25%

Clerkship Assessment Bloom’s taxonomy: knowledge, skills, abilities, and behaviors ACGME core competency: medical knowledge, patient care and procedural skills, interpersonal and communication skills, practice-based learning and improvement, systems-based practice, and professionalism Adult learning 10 points each Blackboard quizzes Journal Club discussion Lecture quizzes 20 points each Outpatient site assignments Inpatient transition assignments Converted to 100 point scale Weight: 25%

Final Grade Requirements: Honors
Final average score of 92% or above Subject exam score in 60th percentile or greater Completion of assignments by established deadlines At least 9 points on professionalism score No unexcused absences

Final Grade Requirements: High Pass
Final average score of 87% or above Subject exam score in 40th percentile or greater Completion of assignments by established deadlines At least 8 points on professionalism score

Final Grade Requirements: Pass
Final average score of 75% or above Subject exam score in 7th percentile or greater Completion of assignments by established deadlines At least 7 points on professionalism score

Final Grade Requirements: Incomplete
Final average score of 70% or above Failure or marginal performance in no more than one of the required components of the rotation as determined by faculty Retake of subject exam score in 7th percentile or greater Retake of SP final exam with passing evaluation Completion of assignments by established deadlines At least 7 points on professionalism score

Questions or Comments? Bruce Britton, MD Mary McBride
Professor, Medical Student Education Director Mary McBride Medical Student Education Coordinator

Please evaluate this session at stfm.org/sessionevaluation.

Bruce Britton, MD Professor, Medical Student Education Director

Similar presentations

Presentation on theme: "Bruce Britton, MD Professor, Medical Student Education Director"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Bruce Britton, MD Professor, Medical Student Education Director

Similar presentations

Presentation on theme: "Bruce Britton, MD Professor, Medical Student Education Director"— Presentation transcript:

Similar presentations

About project

Feedback