Assessing Shoulder Dystocia Simulations for Quality Relieving the stuck shoulders is a cognitive dance of hands countering the inherent and primal response to pull the head. Studies that examined the efficacy of shoulder dystocia maneuvers uniformly agree that the maneuvers must be performed calmly, but quickly, in order to decrease fetal morbidity. (Gherman, Ouzounian, & Goodwin, 1998) The maneuvers lend themselves to repetition and can be practiced via low-fidelity simulation or within the practice and drills of complex emergency events involving high-fidelity scenarios and full multi-disciplinary teams (Fahey, 2008). The potential for the variety of programs and anticipated impending global utilization, justify the need for the standardization of simulation assessment and enhanced external validity. Elizabeth Yznaga, CNM, MSN, sDNP University of San Francisco, Kaiser Permanente San Jose

Quality Shoulder Dystocia Simulations Quality simulations balance internal and external validity. They are designed to demonstrate replicability, integrity and generalizability. Quality can be observed and tracked by process checklists and assessment forms. Participants recognize quality as learner support, excellent feedback, and a coherent reflection of real life events. Prion, 2008 Over the last five years, evidence that simulation can improve patient safety has influenced accrediting agencies such as the US Joint Commission on Accreditation of Healthcare Organizations to recommend simulation of critical events such as the obstetrical emergency of shoulder dystocia. (JCAHO, 2004) In addition recent evidence that shoulder dystocia simulation decreases fetal morbidity, (Draycott, 2008) is expected to promote said simulation for accreditation and medico-legal concerns. As hospitals and educational sites embrace simulation drills and trainings, the increasing variability of design, personnel, and outcomes make external comparisons between simulations difficult.

Inherent structural validity Unique restrictions of time, language, personnel, and actions promote internal and external validity Place Process, People Statistical analysis Treatment of the process Specificity Internal Validity Generalizability Organizational design formulas and protocols for simulations create internal validity within a specified simulation series, by the replication of the scenarios, the consistency of treatments, and by the reliability of the observers’ assessments. The external validity of a simulation is more difficult to establish without a close review of the characteristics that define comparable, operational structures in simulation design. The Jefferies Simulation Design Characteristics (Objectives, Fidelity, Problems Solving, Student Support, and Debriefing, Jeffries, 2007) The SIRC web course Evaluating Simulations by Susan Prion, provides a theoretical basis for exploring the essential characteristics of shoulder dystocia simulations that enhance external validity. External Validity

Quality characteristics Clear objectives: Learning, practice or drill Reliability of the content: Evidenced-based, and standardized Reliability of the experience: Scripted scenario, consistency in treatments ( fetal manikin), clear directions, and endpoints Jefferies (2007) Consistent trainers: Standardized cues and other participant support Documentation: Checklists for process and assessment, audiovisual records decrease bias in assessment Thoughtful Feedback: Respectful, standardized forms with empirical, aesthetic, personal, ethical, and reflection prompts Johns (2004), Jefferies (2007) Evaluation: Promotes improvement and integrity Prion suggests that, assessment feedback given to maintain and improve performance is the important feature and most important focus of simulation. “Assessment is not associated with a grade or any other type of formal measurement, as the goal is to give helpful information that will assist the learner to master the content or skill.” (Prion, 2008).

Tracking quality Increase fidelity with checklists: the setting, participant roles, manikin, and equipment Standardized forms for observation and audiovisual assessments Pre and post testing of content and confidence Electronic simulators are high fidelity by their limited nature The plastic pelvis and doll are low fidelity unless consistent treatments are followed Quality is not conferred by the manikin, but by the scenario

Design for the site Scheduled simulations Birth center and home birth Best control of most factors Education costs Easier to use audio video technology Simple checklists Small teams Multiple responsibilities On shift simulations Hospital sites Flexible participants Informal team building Short programs Shift acuities may degrade the simulation Multidisciplinary teams Complexity = participants Multiple checklists and evaluators

Define the basic tasks Performance check points Identify the problem Call for help: assistants, nurses, pediatricians, neonatal code team Communicate with the staff and the patient and family Call out and perform the maneuvers Document the delivery Feedback

Assessments: measuring quality Track the study or course methods of assessment Study Pretest/ training Posttest/ Written Test in Class Student Self-Report/feedback Clinical Instructor Feedback Clinical Elements Checklist Repeat Demo/ Other Crofts 2006 Baseline test 1-3 wks 185 questions Force data Video recording Key events Actor rated com Track the shoulder dystocia maneuvers Identifies the problem Call for help Rubins Woods Posterior arm MacRobert’s Suprapubic pressure Com/ Doc. Time Track the internal validity of the scenario Clear objectives Fidelity Adherence to scenario Students are supported Reflective feedback Evaluation Scenario 1

Summative Evaluation Simulation Summative Evaluation Scorecard Category Element 1 2 Comments Knowledge and understanding Clear objectives Appropriate level of complexity Skill performance High expectations of everyone Collaboration and teamwork Learner satisfaction Active learners Diverse learning Fidelity for engagement Critical thinking abilities Appropriate debriefing Appropriate level of problem solving Self confidence Appropriate student support Total Total out of 20

Summary Design the simulation to reflect the site, the objectives Maintain internal validity by the use of scripts, checklists and consistent trainers Develop external validity by sharing scripts, checklists and assessments with other entities Design basic simulations for on-site, spontaneous practice