1. Jennifer Kesselheim, MD Nothing to disclose Discussion of off-label drug use: not applicable 56 th ASH Annual Meeting Disclosure Statement

2. Jennifer Kesselheim, MD, EdM ASH Program Directors’ Workshop December 5, 2014

3.  51 sub competencies in pediatrics  15 EPAs  Clinical Competency Committees  Requirement to report to ACGME on 21 milestones every 6 months for every fellow starting in this academic year  How do we optimally support program directors???

4.  Formed to provide guidance and resources to pediatric hematology- oncology (PHO) program directors ◦ Subgroup of the Program Directors’ Committee ◦ Approximately 10 members, 1 fellow

5.  Tasks ◦ Work on subspecialty-specific EPAs ◦ Develop assessment instruments to measure the milestones  Available to any program  Avoid reinventing the wheel ◦ Determine how the sub competencies and EPAs relate to one another ◦ G ather data to measure outcomes of transition

6.  No current plans to require reporting on EPAs  Discussion about EPAs tabled for now ◦ Entrustment is the subject of a research study happening in pediatric subspecialties ◦ More on that below…

7.  Patient Care: 13  5  Medical knowledge: 2  1  Interpersonal Communication: 6  3  Practice-based Learning and Improvement:10  4  Professionalism: 5  1  Systems based Practice: 7  5  Personal and Professional Development: 8  3

8. The Task force identified 5 additional competencies (red) we believed essential to the training of a PHO, raising total number to 26. PC-3Transfers in care PC6Exhibits good clinical judgment PC7Construct a management plan PC8Procedures PC12Role modeling PC13Supervision MK2Applied knowledge PBLI1Self-identify deficiencies PLBI4QI, improve practice mgmt PBLI7Information technology PBLI9Educate patients ICS-1Communicate to patients ICS3Communicate in profession ICS4Lead team ICS5Consult P-conductProfessional conduct P-humanismHumanism in medicine SBP1Health care setting SBP2Med home SBp3Cost/risk-benefit SBP5Team safety/qi SBP6System errors/soln PPD2Coping mechanisms PPD5Trustworthiness PPD6Leadership PPD8Ambiguity

9.  No one wants to fill out an assessment form with 26 items!  Variable relevance of sub competencies ◦ Inpatient vs. outpatient settings ◦ Different PGY levels  Conducted survey of PHO program directors ◦ Significant overlap in rotations and curricula

10.  Distribute competencies among different rotations/ fellowship activities ◦ Inpatient ◦ Outpatient/Consults ◦ Continuity clinic ◦ Teaching conferences ◦ Research ◦ Procedures ◦ 360 evaluations  Allow more advanced competencies to be assessed later  Allow crucial competencies to be assessed frequently

11. InpatientConsult/Outpt ABCDEFPC6 PBLI7 PC7 ICS5 PC3MK2PC12PBLI7PC13PC12P-con ICS1PBLI1PC13PLBI9P-conICS4P-hum PPD6 Continuity Clinic360Conferences ABCDSBP1MK2 SBP5PLBI4 ICS1PBLI1ICS1P-humSPB6PBLI9 ICS3PBLI9ICS3SBP2PPD2ICS3 P-conP-humP-conPPD5 SBP3PPD8PPD6Procedures PPD8PC8 First-Year Evaluation Forms

12. Continuity360Inpatient/Outpatient/Consult ABICS1PC3 PC6 ICS3PC8 PC7PC12SBP1ICS4 PC13MK2SBP5ICS5 P-conP-humSPB6 SBP2SBP3PPD2 PPD6PPD8PPD5 PPD6 ResearchPPD8 PLB1 PLBI7Conferences PBLI9MK2 ICS3PLBI4 P-conPBLI9 PPD2ICS3 Upper-Level Evaluation Forms

13. Evaluation Source:InpatientContinuityConsult/OutptProceduresConferences360ResearchCCC Frequency:qRotationq3 monthqRotationq6 moAs appropriateq6 mo Minimumq2 moq3 mo q6 mo Competency PC3Transfer of careA PC6Clinical judgmentAll X PC7Management plansAll PC8*Procedures X PC12Role modelingC,F PC13*SupervisionC,E MK2EBMBAll X PBLI1Self -knowledgeBB PLBI4QI X PBLI7Information technologyD X PBLI9Educate othersDB X ICS1*Communication: pts/publicAA.C ICS3Communication:professional A,C X ICS4Team leadershipF ICS5Consultant role X P-con Professional ConductEA,CX P-hum*Humanism B.DX SBP1Work in health care system X SBP2Coordinate care D SBP3Cost/risk-benefit analysis C SBP5Interprofessional teams X SBP6ID and solve system errors X PPD2*Healthy coping mechanisms X PPD5Trustworthiness in pt care D X PPD6Leadership skillsF X PPD8Tolerance of ambiguity D X First-Year Evaluation Matrix * Not yet required for ACGME reporting

14. Is reluctant to use information technology. Is unable to prioritize information retrieved. Has basic EBM and EHR skills but is not able to use them with ease and facility. Efficiently retrieves and utilizes information for medical decision making. Habitually uses information for medical decision making for patients and populations. Contributes to development and improvement of information technology for patient care and/or professional learning. Uses information technology to optimize learning and care delivery (PBLI 7)

16.  How could you use the Heme-Onc Milestones as an “item bank” to create meaningful evaluations aligned with the purpose of the curricular experience?

17.  At 2014 ASPHO annual meeting, we led a workshop open to program directors, associate program directors, and teaching faculty

18.  Describe the theory behind EPAs and CBME and the assessment instruments developed by the task force  Plan their own program’s implementation of CBME with guidance from a worksheet and task force members  Become facile with faculty development modules, disseminated during the workshop, for use at home institution

19.  Hour 1: Didactic presentation ◦ Principles of the Next Accreditation System ◦ Structure and function of the CCC ◦ Orientation to new assessment instruments created by task force  Hour 2: Small group work ◦ How will you implement new assessments in your program?  Hour 3: Training the trainer ◦ Strategies for faculty development at home ◦ Review of PowerPoint module created by task force

20.  Pre-test administered to registrants (N=37)  Repeated as post-test (N=38)  Rate agreement with 6 different item stems

23.  Council of Pediatric Subspecialties  How are entrustment decisions being made? ◦ Measure program directors’ assessments of entrustment on each EPA (standard) ◦ Compare to CCC entrustment decisions derived from sub competencies and milestones ◦ Determine which sub competencies most influential  Participation ◦ 8 subspecialties, 15 PHO programs, ~500 fellows ◦ IRB approval at hospital level ◦ Data collection in progress

24.  Mark Atlas, MD  Task Force members  ASPHO leadership and staff ◦ Funded 2 in-person meetings ◦ Allocated time and space during annual meetings ◦ Conveyed fellowship training as major priority