1. Introduction Results Objectives Conclusions Methods Limitations
Critical Care Simulation Curriculum to Assess Pediatric Residents on ACGME Developmental Milestones
Joshua D. Patterson, MD; Matthew A. Borgman, MD;
Heather M. Delaney, MD; Renée I. Matos, MD, MPH
Department of Pediatrics, San Antonio Military Medical Center, Ft Sam Houston, TX 78234
Introduction
Results
In 1999, the Accreditation Council for Graduate Medical Education (ACGME) introduced the six domains of clinical competency to better assess resident progression
In 2009, ACGME began a multiyear process to restructure its system to be based on educational outcomes, which resulted in the development of a consensus set of milestones in the six competencies in each residency training program1
Milestones were organized with a 5-level model beginning with goals for entering training in a new specialty, progressing to graduation goals, and ending at an advanced level2
ACGME put forth 21 milestones in 2012 to better assess pediatric residents; however, subjective assessment of can be difficult and may vary based on patient acuity and volume3
Jan 2013 – Nov 2015: 39 PGY2 residents trained
Milestones addressed: Decision Making (PC4), Management Plans (PC5), Identifying one’s knowledge (PBLI1), Emotional response (ICS2), Respect for others (Prof1), and Coordinating Care (SBP1)
Simulation scores contributed to milestone scores
End-of-year milestones for all residents were assessed between Level 2 to 3.5 based on all PGY2 rotations
The Asthma and Sedation simulations were more likely to correlate with the average milestone score than the PEA or TBI scenarios
20 of 158 (12.7%) scenarios were repeated for failure
Table 1. Simulation Scores
Sim #
Mean Score SD
Failures
R2 Milestone Correlation S1
84.48%
8.21% 3
0.24 S2
69.97%
14.23% 8
0.00 S3
75.52%
8.07% 2
0.37 S4
80.86%
10.61% 7
0.05
Objectives
Our 6-bed PICU (Average daily census = 2.7) has a rotating PGY2 who previously would receive variable lectures and ad hoc simulation scenarios
To standardize learning and assessment, in January 2013 we initiated a standardized curriculum of lectures and high fidelity simulated pediatric scenarios, including objectives and quantifiable assessment tools
This curriculum synchronized with the pediatric resident ICU rotation and aided with milestone calculations and evaluations
Conclusions
Hospital-based simulations may enhance multidisciplinary patient care through accessible, repetitive team training4
An integrated lecture and crisis-based simulation curriculum aids in assessing pediatric residency ACGME milestones
Simulation creates opportunities for objective evaluation during a 4-week rotation in a low-volume pediatric ICU
Pediatric residents demonstrated difficulty with basic PALS skills & knowledge, especially pulseless simulations, resulting in added simulation exposure during ward block
Less clinically challenging simulation scenarios were more likely to correlate with global milestone assessments
Quantifiable simulation assessments may be more sensitive in detecting weaknesses in clinical competency than the more qualitative ACGME milestones assessment
Methods
4 high-fidelity simulations (SimJunior & SimBaby, Laerdal):
S1: Procedural Sedation with associated Complications
S2: Pulseless Electrical Activity due to hyperkalemia
S3: Asthmatic respiratory failure with ventricular tachycardia
S4: Traumatic brain injury and hyponatremic seizures
Objective assessment forms created for each scenario
Same four attending physicians gave standardized lectures and graded simulation (inter-rater reliability, Kappa = 1.0)
After the first 10 residents, the scenarios were considered failures when they scored 2 SD below the mean, and then were repeated. Only initial scores were utilized for results
All residents were certified in PALS prior to training
Limitations
Curriculum developed before milestones
Retrospective analysis
Milestone scores are based on a collective summary of all rotations and evaluations
Limited evaluation of milestones (6 of 21)
Future Direction
Prospectively create scenarios or curriculum to specifically target ACGME milestone assessments
Reference
Swing SR, Clyman SG, Holmboe ES, William RG. Advancing resident assessment in graduate medical education. J Grad Med Educ 2009; 1:278-86
Swing SR, Beeson MS, Carraccio C, Coburn M, et al. Educational milstones development in the first 7 specialties to enter the next accreditation system. J Grad Med Educ 2013; 5:98-106
ACGME, ABP. The Pediatric Milestones Project, July 2015.
Weinstock PH, Kappus LJ, Kleinman ME, et al. Toward a new paradigm in hospital-based pediatric education: The development of an onsite simulator program. Pediatr Crit Care Med 2005; 6:
Figure 2. During the simulation scenarios, both SimJunior© and SimBaby©, pictured here, were used by pediatric residents
The views expressed herein are those of the authors and do not reflect the official policy or position of San Antonio Military Medical Center, the U.S. Army Medical Department, the U.S. Army Office of the Surgeon General, the Department of the Army, Department of the Air Force, or U.S. Government.
Texas Pediatric Society Electronic Poster Contest