1. Pediatric Milestones and Evaluations A. White, MD February 7 th, 2014

2. Milestones Observable developmental steps moving from novice physician to expert/master Analogous to Developmental Stages – Crawls, cruises, walks, runs, wins Olympic Gold Organized under the domains of Competency We must report our residents’ milestones to the ACGME every 6 months

4. Pediatric Milestones 51 Pediatric Milestones Only 21 have been written/created to date 165 page document describes the 21 (We have 96 residents) Goal: Create rotation evals that easily mimic/simulate or map to the milestones! (The ACGME does not want the milestones to be used as evaluations)

5. One of the 51 Pediatric Milestones PC 5: Develop and carry out management plans. Develops and carries out management plans based on directives from others, either from the health care organization or the supervising physician. Unable to adjust plans based on individual patient differences or preferences. Communication about the plan is unidirectional from the practitioner to the patient and family. Develops and carries out management plans based on one’s theoretical knowledge and/or directives from others. Can adapt plans to the individual patient, but only within the framework of one’s own theoretical knowledge. Unable to focus on key information, so conclusions are often from arbitrary, poorly prioritized, and time-limited information gathering. Management plans based on the framework of one’s own assumptions and values. Develops and carries out management plans based on both theoretical knowledge and some experience, especially in managing common problems. Follows health care institution directives as a matter of habit and good practice rather than as an externally imposed sanction. Able to more effectively and efficiently focus on key information, but still may be limited by time and convenience. Plans begin to incorporate patients’ assumptions and values through more bidirectional communication. Develops and carries out management plans based most often on experience. Effectively and efficiently focuses on key information to arrive at a plan. Incorporates patients’ assumptions and values through bidirectional communication with little interference from personal biases. Develops and carries out management plans, even for complicated or rare situations, based primarily on experience that puts theoretical knowledge into context. Rapidly focuses on key information to arrive at the plan and augments that with available information or seeks new information as needed. Has insight into one’s own assumptions and values that allow one to filter them out and focus on the patient/family values in a bidirectional conversation about the management plan.

12. Evaluation Observations The general scores on the old Likert scale evaluations seemed about the same as the new Milestone language based evaluations.

13. Evaluation Observations The general scores on the old Likert scale evaluations seemed about the same as the new Milestone language based evaluations. So, we compared the average scores from the “old” evals with the scores from the “new” evals for each specific competency – From equivalent time periods – Equivalent levels of training – Normalized to a 10 point scale

14. Faculty Evaluations MilestoneNew EvalsOld Evals (Likert) Patient Care Skills7.68.0 Professionalism8.58.9 Problem Based Learning7.68.2 Systems Based Practice8.68.1 Communication Skills8.18.3 Medical Knowledge7.58.0 Avg: 8.0 8.2

16. Peer Evaluations MilestoneNew EvalsOld Evals (Likert) PC17.59.3 PC56.89.1 PBLI27.79.3 ICS18.69.3 Prof28.29.6 Prof38.09.7 Prof38.49.3 Avg: 7.9 9.4

18. Preliminary Conclusions Evaluations based on milestone language offer no benefit over Likert scale based evaluations from attendings. Evaluations based on milestone language are more discriminating in peer–peer ratings. Faculty may have a better innate understanding of the developmental stages of resident maturation (ie, milestones) than residents.

19. Limitations Statistical evaluation not yet complete Number of evaluations could be larger Milestone language may have been made too short. No comparison was made between groups (board scores, etc)

20. Pediatrics Plan Modify our evaluations to mimic the basic terminology of the milestones (done!) Map each evaluation question to a specific Milestone (done!) Assign numerical values behind the scenes the each milestone for easy use by the CCC. (done!) Eagerly await the next 30 milestones!