Presentation is loading. Please wait.

Presentation is loading. Please wait.

Improving the Precision of Community Preceptors' Grading of Students in the Family Medicine Clerkship Presented by Laura Mayans MD, Kari Clouse MD, Scott.

Published byBeryl Turner Modified over 8 years ago

Similar presentations

Presentation on theme: "Improving the Precision of Community Preceptors' Grading of Students in the Family Medicine Clerkship Presented by Laura Mayans MD, Kari Clouse MD, Scott."— Presentation transcript:

1 Improving the Precision of Community Preceptors' Grading of Students in the Family Medicine Clerkship Presented by Laura Mayans MD, Kari Clouse MD, Scott Moser MD

2 Background 8 week required M3 clerkship using volunteer community faculty Clinical evaluation: hybrid normative/competency-based grading form Marked inter-rater variability Poor Spearman correlations (n=132) – Preceptor grade vs. OSCE =.133 – Preceptor grade vs. NBME shelf =.164

3 Purpose Improve the precision of community preceptors' grading of student performance.

4 Methods Subjects: 19 community volunteer preceptors Intervention: Preceptor development workshop at annual regional CME program

5 Workshop Agenda Feedback: + / Δ card Evaluation: clinical grading form SP Videos: – Student presentation to preceptor case 1 – Student / SP encounter case 1 – Student /SP encounter case 2 – Each video followed by + / Δ discussion, grading form vote and further dialogue Wrap-up

6 Demonstration VIDEO CLIP

7 Evaluation Comparison of preceptor grading form votes post video Participant post-workshop survey Pre and post intervention comparison of family medicine clerkship grades Precision correlation of preceptor grade to end-of-clerkship OSCE

8 Results Grading form – Technical difficulties resulted in voting by hand – Generally noted inter-rater variability with good discussion Survey – Average rating of the workshop = 3.57 1-4 scale, highest = 4 – Reoccurring comments: Increased direct observation and use of + / Δ card

9 Conclusions Workshop well-received by participants Standardized, scripted video accurately depicted student performance at border grading margins Improved grading precision remains unknown

10 Limitations Workshop attendees represented only 11.4% of clerkship preceptors Some family medicine preceptors have limited student volume Potential time lapse between intervention and preceptor – student interaction

11 Next Steps Complete follow up on grade comparison and grade form/OSCE precision outcomes Reinforce material during routine academic detailing visit and through creation of a resource website Disseminate videos to other departments and schools for preceptor development

12 Reflections Workshop key elements – Accuracy: Importance of direct observation – Inter-preceptor precision: Value of SP video and discussion Current grading form needs revision to clearer competency-based criteria Need ways to reach more preceptors

13 Acknowledgements Funding - MAITF Grant Nancy Davis, PhD Trisha Melhado, MPH Terry Ast, research assistant Whitney Rowe, MD KUSM–W SP Lab Staff KUSM–W IT Staff

14 1. Alexander E, Osman NY, Walling JL, Mitchell VG. Variation and Imprecision of Clerkship Grading in US Medical Schools. Academic Medicine. 2012; 87(8): 1070-76. 2. Ogden PE, Edward J, Howell M, Via R, Song J. The Effect of Two Different Faculty Development Interventions on Third-year Clerkship Performance Evaluations. Fam Med. 2008; 40(5): 333-38. 3. Christopher S, Weaver MD, Alysius J, et al. A more Explicit Grading Scale Decreases Grade Inflation in a Clinical Clerkship. Academic Emergency Medicine. 2007; 14: 283-286. 4. Plymale MA, French J, Donnelly MB, Iocono J, Pulito AR. Variation in Faculty Evaluation of Clerkship Students Attributable to Surgical Service. J Surgical Education. 2010; 67(3): 179-83. 5. Kogan JR,et al. Opening the black box of clinical skills assessment via observation: a conceptual model. Medical Education. 2011; 45:1048-60. References

15 Contact Information Laura Mayans – lmayans@kumc.edu Kari Clouse – kclouse@kumc.edu Scott Moser – smoser@kumc.edu

Download ppt "Improving the Precision of Community Preceptors' Grading of Students in the Family Medicine Clerkship Presented by Laura Mayans MD, Kari Clouse MD, Scott."

Similar presentations

Empowering tobacco-free coalitions to collect local data on worksite and restaurant smoking policies Mary Michaud, MPP University of Wisconsin-Cooperative.

Empowering tobacco-free coalitions to collect local data on worksite and restaurant smoking policies Mary Michaud, MPP University of Wisconsin-Cooperative.
Evaluating the Reliability and Validity of the Family Conference OSCE Across Multiple Training Sites Jeffrey G. Chipman MD, Constance C. Schmitz PhD, Travis.

Evaluating the Reliability and Validity of the Family Conference OSCE Across Multiple Training Sites Jeffrey G. Chipman MD, Constance C. Schmitz PhD, Travis.
Faculty Development for Ambulatory Care Education LuAnn Wilkerson, Ed.D. UCLA.

Faculty Development for Ambulatory Care Education LuAnn Wilkerson, Ed.D. UCLA.
Paired Observation and Video Editing (POVE) Blending Continuous Peer Feedback and Reflection to Strengthen Communication Training Larry Mauksch, M.Ed,

Paired Observation and Video Editing (POVE) Blending Continuous Peer Feedback and Reflection to Strengthen Communication Training Larry Mauksch, M.Ed,
Professional Development & the Evaluation Process MARIA BERNALBROWARD COLLEGE EDWARD CORNEJOJUNE 9, 2015 PEDRO OLIVEIRA.

Professional Development & the Evaluation Process MARIA BERNALBROWARD COLLEGE EDWARD CORNEJOJUNE 9, 2015 PEDRO OLIVEIRA.
Using Teen Actors to Teach How to Communicate with Adolescents Anisha Abraham, MD, MPH Associate Professor, Department of Pediatrics Chief, Section of.

Using Teen Actors to Teach How to Communicate with Adolescents Anisha Abraham, MD, MPH Associate Professor, Department of Pediatrics Chief, Section of.
Establishing Content Validity of a Novel Written Examination to Assess Medical Students on the Surgery Clerkship A Berlin MD, A Reinert BA, A Swan-Sein.

Establishing Content Validity of a Novel Written Examination to Assess Medical Students on the Surgery Clerkship A Berlin MD, A Reinert BA, A Swan-Sein.
Educating Medical Students about the Care of Patients with Disabilities Kira Zwygart, MD Laurie Woodard, MD University of South Florida College of Medicine.

Educating Medical Students about the Care of Patients with Disabilities Kira Zwygart, MD Laurie Woodard, MD University of South Florida College of Medicine.
Seth D. Goldstein, MD 1, Brenessa Lindeman, MD 1, Jorie Colbert-Getz, PhD 3, Trisha Arbella 1, Robert Dudas, MD 2, Anne Lidor, MD, MPH 1, Bethany Sacks,

Seth D. Goldstein, MD 1, Brenessa Lindeman, MD 1, Jorie Colbert-Getz, PhD 3, Trisha Arbella 1, Robert Dudas, MD 2, Anne Lidor, MD, MPH 1, Bethany Sacks,
Building Research Capacity for Community Organizations: Strategies from the Alliance for Research in Chicagoland Communities Ben Rucker MPH, 1,2 Sherida.

Building Research Capacity for Community Organizations: Strategies from the Alliance for Research in Chicagoland Communities Ben Rucker MPH, 1,2 Sherida.
CBEI Essentials for Residents, Fellows, and Faculty A 10-minute primer on student performance assessment in required clerkships Stanford School of Medicine.

CBEI Essentials for Residents, Fellows, and Faculty A 10-minute primer on student performance assessment in required clerkships Stanford School of Medicine.
NAVIGATING THE PROCESS OF STUDENT LEARNING OUTCOMES: DEVELOPMENT, EVALUATION, AND IMPROVEMENT Shannon M. Sexton, Julia M. Williams, & Timothy Chow Rose-Hulman.

NAVIGATING THE PROCESS OF STUDENT LEARNING OUTCOMES: DEVELOPMENT, EVALUATION, AND IMPROVEMENT Shannon M. Sexton, Julia M. Williams, & Timothy Chow Rose-Hulman.
Impact of the “Asthma Toolbox” for Improving Documentation of Pediatric Asthma Management in an Urban Community Health Center Presenter: Delaney Gracy,

Impact of the “Asthma Toolbox” for Improving Documentation of Pediatric Asthma Management in an Urban Community Health Center Presenter: Delaney Gracy,
Background: As students complete their clerkships throughout their M3 year they gain in clinical experience and confidence, which may translate into improved.

Background: As students complete their clerkships throughout their M3 year they gain in clinical experience and confidence, which may translate into improved.
Disclosure of Financial Conflicts of Interest in Continuing Medical Education Michael D. Jibson, MD, PhD and Jennifer Seibert, MD University of Michigan.

Disclosure of Financial Conflicts of Interest in Continuing Medical Education Michael D. Jibson, MD, PhD and Jennifer Seibert, MD University of Michigan.
In order to improve the Family Medicine obstetrics (OB) rotation orientation and to encourage resident- resident teaching, we developed an orientation.

In order to improve the Family Medicine obstetrics (OB) rotation orientation and to encourage resident- resident teaching, we developed an orientation.
Copyright restrictions may apply Randomized Trial of Teaching Brief Motivational Interviewing to Pediatric Trainees to Promote Healthy Behaviors in Families.

Copyright restrictions may apply Randomized Trial of Teaching Brief Motivational Interviewing to Pediatric Trainees to Promote Healthy Behaviors in Families.
Introduction Aimed at educators from wide-ranging backgrounds and with varying levels of experience, the Teaching and Assessment for Medical Educators.

Introduction Aimed at educators from wide-ranging backgrounds and with varying levels of experience, the Teaching and Assessment for Medical Educators.
CBEI Essentials for residents, fellows, and faculty A 10-minute primer on the Criterion-Based Evaluation Initiative and student performance assessment.

CBEI Essentials for residents, fellows, and faculty A 10-minute primer on the Criterion-Based Evaluation Initiative and student performance assessment.
BackgroundResults GME completed all design, start-up, reviews, assessment tool and inter-program comparisons, statistics PDs completed the ‘field work’

BackgroundResults GME completed all design, start-up, reviews, assessment tool and inter-program comparisons, statistics PDs completed the ‘field work’

Similar presentations

Ads by Google