1. Improving Preceptor Narrative Feedback Comments on
Student Assessments
Suzanne Minor MD; Sarah E Stumbar MD MPH; Marquita Samuels MBA
Florida International University Herbert Wertheim College of Medicine
Background
Methods
Results
Conclusions
This pilot demonstrated that some preceptors, although not all, improved the specificity and originality of their narrative comments with the addition of the new prompt.
Our next steps include petitioning the curriculum committee to add this prompt to the electronic assessment for clerkship students. We also plan to target faculty development towards improving the specificity of narrative comments.
Learners benefit more from narrative, specific feedback than from feedback consisting solely of numerical ratings. This specific feedback is more easily assimilated, and can lead to improved in-training evaluations.
Narrative comments provided by preceptors are also important in justifying high or low scores and grades, for documenting worrisome behavior, and for use in the Medical Student Performance Evaluation.
The quality and quantity of narrative comments can be improved through faculty development about their importance. However, these interventions—including in person visits with faculty, s, and phone conversations-- are often limited by time constraints.
The purpose of our pilot study is to determine if an intervention requiring little devoted time – the addition of a simple prompt to the student clinical assessment – can improve the quality of narrative feedback comments given to our students on the Family Medicine clerkship.
In October 2017, this prompt was added to our Family Medicine clerkship clinical assessment:
“Please comment on one patient encounter the student had that stands out to you (please try to be as descriptive as possible).”
The prompt was added 11/17/2017, and only to our paper assessment, limiting our data to preceptors who use the paper form.
We compared the specificity and quality of responses to the new prompt from 11/2017- to the prior comments received 4/ /2017, when the existing prompts on the assessment were: “Comment on the strengths of this student’s performance” and “Comment on areas that need improvement for this student.”
After individually rating the comments on specificity and genericity, the three authors met to come to agreement in ratings through an interactive process.
A total of 7 faculty regularly complete paper assessments and were included in this pilot study.
Preceptors 1, 2, 3 increased the percent of very specific narrative comments.
Preceptor 1 increased to 60% non- generic comments.
Preceptors 2, 3, 6 increased the percent of partially (from all) generic comments.
Specificity of Each Preceptor’s Comments,
Pre- and Post-Intervention
Genericity of Each Preceptor’s Comments,
pre % (n)
post % (n)
Preceptor NS SS VS 1
90% (9)
10% (1)
0% (0)
20% (1)
60% (3) 2
67% (4)
33% (2)
31% (5)
50% (8)
19% (3) 3
100% (4)
14% (1)
71% (5) 4
50% (3)
64% (9)
36% (5) 5 6
86% (6)
56% (5)
44% (4)
pre % (n)
post % (n)
Preceptor AG PG NG 1
90% (9)
10% (1)
0% (0)
40% (2)
60% (3) 2
83% (5)
17% (1)
38% (6)
62% (10) 3
100% (4)
14% (1)
86% (6) 4
100% (6)
75% (9)
25% (3) 5
50% (3)
33% (2)
67% (4) 6
67% (6)
33% (3)
Bibliography
1. Pelgrim EA, Kramer AW, Mokkink HG, Van der Vleuten CP. Quality of written narrative feedback and reflection in a modified mini-clinical evaluation exercise: an observational study.BMC Med Educ Oct 18;12:97.
2. van der Leeuw RM, Overeem K, Arah OA, Heineman MJ, Lombarts KM. Frequency and determinants of residents' narrative feedback on the teaching performance of faculty: narratives in numbers. Acad Med Sep;88(9):