Planning and Designing Scenario-based Simulations Mindy Ju, MD Sandrijn van Schaik MD PhD UCSF Center for Faculty Educators & Kanbar Center for Simulation

Teach for UCSF Mini Certificate in Simulation Teaching http://tiny.ucsf.edu/simplanning

Workshop Objectives Use Kern’s 6 Steps in Curriculum Development to plan a scenario-based simulation session Create objectives for simulation scenarios Design a simulation scenario appropriate for your learners Develop a strategy to evaluate curriculum

Workshop Overview Review Kern’s 6 steps to curriculum development applied to simulation Go through the 6 steps using the example of the UCSF Pediatric Resident Simulation Program Practice using the 6 steps on your own curriculum/scenarios in small groups

Curricular design Six step approach (David Kern)

Curricular design Six step approach (David Kern)

Example Pediatric Resident Simulation Program

Step 1a. Problem identification What is the problem? In pediatrics, emergencies are rare leading to pediatric residents graduating from residency with limited experience leading a team during a crisis situation. Who is affected by this problem? (stakeholders) Pediatric Residents Pediatric Residency Program Attending Pediatricians Patients

Step 1b. General Needs Assessment Current Approach Pediatric residents get exposure to medical emergencies as they arise during their clinical service time The get to lead the team only if fellow or attending allows them to Ideal Approach Pediatric residents would get to lead more medical emergencies in a safe and controlled environment that would allow for adequate feedback

Step 2. Targeted Needs Assessment Who are your targeted learners? UCSF Pediatric Residents at all levels of training What do you know about them and their learning environment? They receive much of their learning in the clinical setting They have some protected educational time They do not receive formal training in crisis resource management and team communication skills They frequently are directly supervised by fellows or attendings

Step 2. Targeted Needs Assessment What do you need to know about your targeted learners? What clinical skills and knowledge should they have at each level of training? How much protected time do they have? How will you gather this information? Survey attending providers in each rotation about what medical cases each level of trainee should be able to management independently for 5 minutes Talk with residency leadership about protected time

Step 3. Define goals and objectives What is the overall purpose of your simulation activity? Objectives What specific and measurable outcomes should a learner expect to achieve by participating in your simulation exercise?

Learning objectives for simulation Specific Define exactly what needs to be learned Measurable Can you assess that the objectives were met? Acceptable Appropriate complexity for learners, relevant to their learning needs Realistic Achievable considering limitations of fidelity, time etc. Time bound At the end of one session, multiple sessions?

Bloom’s taxonomy is a framework to categorize educational objectives and assessments according to complexity, with recall of facts at the bottom and more complex tasks such as analyzing and evaluating at the top.

Example Goal: manage a patient in hypovolemic shock from dehydration Objectives: By the end of this session the first year pediatrics resident should be able to: Verbalize patient status correctly as decompensated hypovolemic shock from dehydration Implement appropriate supportive therapy for decompensated hypovolemic shock from dehydration Demonstrate use of ISBAR communication tool

Breakout Exercise 1: Break up into groups of 3 Use the scenario from Exercise 1 to complete STEP 3 on the worksheet (15 minutes) Share goals and objectives with larger group (15 minutes)

Breakout Exercise 1: Discussion: Critique goals & objectives Questions to consider: Are objectives measurable? Reasonable number of objectives? Is simulation the best strategy? What other strategies should be considered to reach the objectives?

Step 4. Educational Strategies Which simulation method is the right teaching modality for all or part of your needs? Scenario-based using mannequin Partial task trainer Standardized patient Combination?

Step 4. Educational Strategies Integration in the Curriculum Simulation is most successful if related to other educational content and/or clinical experiences Deliberate Practice Practice followed by feedback and reflection, followed by focused repeated practice Mastery Learning Learner progresses through educational units of increasing level of difficulty

Step 4. Educational Strategies Integration in the Curriculum Became integrated in the Peds Residency Curriculum through support from Peds Residency Leadership Drawn from clinical cases Deliberate Practice Each session had multiple scenarios with debriefings in between Mastery Learning N/A

Breakout Exercise 2: Break up into same groups of 3 Use the scenario from Exercise 1 to complete STEP 4 on the worksheet (10 minutes) Share with larger group (10 minutes) What simulation modality will you use? How could you integrate your simulation into an existing curriculum? Incorporate deliberate practice and mastery learning?

Step 5. Implementation Planning the simulation session Preparing the learning content (scenario) Scheduling the session Preparing the instructors Preparing the learners Preparing the environment Preparing the technical staff

Scenario Design Template: standard form for creating scenarios for mannequin-based simulation Goals and objectives Initial state of patient Equipment and props needed What are the possible/desired developments? What are the endpoints? Several templates exist: Kanbar has a word-based template

Scenario Design Why use a template? Helps establish common understanding with other faculty instructors about goals and objectives Helps identify what is needed to reach the objectives Helps with implementation phase Identify necessary resources Ensures staff understands & prepares what you need

Trigger End Scenario design Scenario development states Scenario goes from one state to the next when a certain trigger is achieved (or not achieved) Example: Patient has an unstable arrhythmia. Shock is delivered, patient goes to sinus tachycardia. Baseline state Trigger Intermediate state Resolution state End

Breakout Exercise 3: Break up into same groups of 3 Use a scenario template to create a simulation scenario building on goals and objectives from Exercise 2 (15 minutes) Share with larger group thoughts on the process (5 minutes) Difficulties? Recognition of additional resources? Changes in goal or learning objectives?

Step 5. Implementation Planning the simulation session Preparing the learning content (scenario) Scheduling the session Preparing the instructors Preparing the learners Preparing the environment Preparing the technical staff

Planning the simulation session Scheduling the session When, how long, how often? Preparing the learning content Design the scenario(s) Discussion/learning points for debriefing Debriefing is where most of the learning takes place! Preparing the instructors How are instructors trained? Where will they be, what will they do? Need for confederates?

Planning the simulation session Preparing the learners How are learners oriented to the simulator? What will learners do, how many in each group and how will roles be divided? Preparing the environment What equipment do you need? What props? AV-needs? Where will you pre- and debrief? Preparing the technical staff At Kanbar, staff will help you with much of the planning and preparation Dry run the scenario

STEP 6: Evaluation and feedback   Level Use Individual Program Formative Summative Formative evaluation that is used to help individual improve performance Summative evaluation used to make judgments or decisions about individual Individual can be learner or faculty 2/22/2019 [ADD PRESENTATION TITLE: INSERT TAB > HEADER & FOOTER > NOTES AND HANDOUTS]

STEP 6: Evaluation and feedback   Level Use Individual Program Formative Identification of areas for improvement Specific suggestions for improvement  Summative Certification of performance Grades Promotion  Judgments on success Justification of resources Recruitment of learners/educators Dissemination  Formative evaluation that is used to help individual improve performance Summative evaluation used to make judgments or decisions about individual Individual can be learner or faculty 2/22/2019 [ADD PRESENTATION TITLE: INSERT TAB > HEADER & FOOTER > NOTES AND HANDOUTS]

STEP 6: Evaluation and feedback   Level Use Individual Program Formative Use of team leadership checklist tools Use of clinical skills for each scenario Learners and facilitators fill out evaluations at end of each session Summative N/A Biannual evaluations by learners on value of program Formative evaluation that is used to help individual improve performance Summative evaluation used to make judgments or decisions about individual Individual can be learner or faculty 2/22/2019 [ADD PRESENTATION TITLE: INSERT TAB > HEADER & FOOTER > NOTES AND HANDOUTS]

STEP 6: Evaluation and feedback 2/22/2019 [ADD PRESENTATION TITLE: INSERT TAB > HEADER & FOOTER > NOTES AND HANDOUTS]

Summary Kern’s 6 steps to curriculum development provide a framework for creating effective simulation scenarios and curriculums Remember the SMART mnemonic and Bloom’s Taxonomy to create learning objectives for each scenario Use a template to create the stages of the scenario to help guide facilitators and simulation technicians Remember Kirkpatrick’s pyramid when developing evaluations of your simulation curriculum