1. Planning and Designing Scenario-based Simulations
Mindy Ju, MD
Sandrijn van Schaik MD PhD
UCSF Center for Faculty Educators & Kanbar Center for Simulation

2. Teach for UCSF Mini Certificate in Simulation Teaching

3. Workshop Objectives
Use Kern’s 6 Steps in Curriculum Development to plan a scenario-based simulation session
Create objectives for simulation scenarios
Design a simulation scenario appropriate for your learners
Develop a strategy to evaluate curriculum

4. Workshop Overview
Review Kern’s 6 steps to curriculum development applied to simulation
Go through the 6 steps using the example of the UCSF Pediatric Resident Simulation Program
Practice using the 6 steps on your own curriculum/scenarios in small groups

5. Curricular design
Six step approach (David Kern)

6. Curricular design
Six step approach (David Kern)

7. Example
Pediatric Resident Simulation Program

8. Step 1a. Problem identification
What is the problem?
In pediatrics, emergencies are rare leading to pediatric residents graduating from residency with limited experience leading a team during a crisis situation.
Who is affected by this problem? (stakeholders)
Pediatric Residents
Pediatric Residency Program
Attending Pediatricians
Patients

9. Step 1b. General Needs Assessment
Current Approach
Pediatric residents get exposure to medical emergencies as they arise during their clinical service time
The get to lead the team only if fellow or attending allows them to
Ideal Approach
Pediatric residents would get to lead more medical emergencies in a safe and controlled environment that would allow for adequate feedback

10. Step 2. Targeted Needs Assessment
Who are your targeted learners?
UCSF Pediatric Residents at all levels of training
What do you know about them and their learning environment?
They receive much of their learning in the clinical setting
They have some protected educational time
They do not receive formal training in crisis resource management and team communication skills
They frequently are directly supervised by fellows or attendings

11. Step 2. Targeted Needs Assessment
What do you need to know about your targeted learners?
What clinical skills and knowledge should they have at each level of training?
How much protected time do they have?
How will you gather this information?
Survey attending providers in each rotation about what medical cases each level of trainee should be able to management independently for 5 minutes
Talk with residency leadership about protected time

12. Step 3. Define goals and objectives
What is the overall purpose of your simulation activity?
Objectives
What specific and measurable outcomes should a learner expect to achieve by participating in your simulation exercise?

13. Learning objectives for simulation
Specific
Define exactly what needs to be learned
Measurable
Can you assess that the objectives were met?
Acceptable
Appropriate complexity for learners, relevant to their learning needs
Realistic
Achievable considering limitations of fidelity, time etc.
Time bound
At the end of one session, multiple sessions?

14. Bloom’s taxonomy is a framework to categorize educational objectives and assessments according to complexity, with recall of facts at the bottom and more complex tasks such as analyzing and evaluating at the top.

15. Example Goal: manage a patient in hypovolemic shock from dehydration
Objectives: By the end of this session the first year pediatrics resident should be able to:
Verbalize patient status correctly as decompensated hypovolemic shock from dehydration
Implement appropriate supportive therapy for decompensated hypovolemic shock from dehydration
Demonstrate use of ISBAR communication tool

16. Breakout Exercise 1: Break up into groups of 3
Use the scenario from Exercise 1 to complete STEP 3 on the worksheet (15 minutes)
Share goals and objectives with larger group (15 minutes)

17. Breakout Exercise 1: Discussion: Critique goals & objectives
Questions to consider:
Are objectives measurable?
Reasonable number of objectives?
Is simulation the best strategy?
What other strategies should be considered to reach the objectives?

18. Step 4. Educational Strategies
Which simulation method is the right teaching modality for all or part of your needs?
Scenario-based using mannequin
Partial task trainer
Standardized patient
Combination?

19. Step 4. Educational Strategies
Integration in the Curriculum
Simulation is most successful if related to other educational content and/or clinical experiences
Deliberate Practice
Practice followed by feedback and reflection, followed by focused repeated practice
Mastery Learning
Learner progresses through educational units of increasing level of difficulty

20. Step 4. Educational Strategies
Integration in the Curriculum
Became integrated in the Peds Residency Curriculum through support from Peds Residency Leadership
Drawn from clinical cases
Deliberate Practice
Each session had multiple scenarios with debriefings in between
Mastery Learning
N/A

21. Breakout Exercise 2: Break up into same groups of 3
Use the scenario from Exercise 1 to complete STEP 4 on the worksheet (10 minutes)
Share with larger group (10 minutes)
What simulation modality will you use?
How could you integrate your simulation into an existing curriculum? Incorporate deliberate practice and mastery learning?

22. Step 5. Implementation Planning the simulation session
Preparing the learning content (scenario)
Scheduling the session
Preparing the instructors
Preparing the learners
Preparing the environment
Preparing the technical staff

23. Scenario Design
Template: standard form for creating scenarios for mannequin-based simulation
Goals and objectives
Initial state of patient
Equipment and props needed
What are the possible/desired developments? What are the endpoints?
Several templates exist:
Kanbar has a word-based template

24. Scenario Design Why use a template?
Helps establish common understanding with other faculty instructors about goals and objectives
Helps identify what is needed to reach the objectives
Helps with implementation phase
Identify necessary resources
Ensures staff understands & prepares what you need

25. Trigger End Scenario design Scenario development states
Scenario goes from one state to the next when a certain trigger is achieved (or not achieved)
Example: Patient has an unstable arrhythmia. Shock is delivered, patient goes to sinus tachycardia.
Baseline state
Trigger
Intermediate state
Resolution state
End

26. Breakout Exercise 3: Break up into same groups of 3
Use a scenario template to create a simulation scenario building on goals and objectives from Exercise 2 (15 minutes)
Share with larger group thoughts on the process (5 minutes)
Difficulties?
Recognition of additional resources? Changes in goal or learning objectives?

27. Step 5. Implementation Planning the simulation session
Preparing the learning content (scenario)
Scheduling the session
Preparing the instructors
Preparing the learners
Preparing the environment
Preparing the technical staff

28. Planning the simulation session
Scheduling the session
When, how long, how often?
Preparing the learning content
Design the scenario(s)
Discussion/learning points for debriefing
Debriefing is where most of the learning takes place!
Preparing the instructors
How are instructors trained?
Where will they be, what will they do?
Need for confederates?

29. Planning the simulation session
Preparing the learners
How are learners oriented to the simulator?
What will learners do, how many in each group and how will roles be divided?
Preparing the environment
What equipment do you need? What props?
AV-needs? Where will you pre- and debrief?
Preparing the technical staff
At Kanbar, staff will help you with much of the planning and preparation
Dry run the scenario

30. STEP 6: Evaluation and feedback
Level
Use
Individual
Program
Formative
Summative
Formative evaluation that is used to help individual improve performance
Summative evaluation used to make judgments or decisions about individual
Individual can be learner or faculty
2/22/2019
[ADD PRESENTATION TITLE: INSERT TAB > HEADER & FOOTER > NOTES AND HANDOUTS]

31. STEP 6: Evaluation and feedback
Level
Use
Individual
Program
Formative
Identification of areas for improvement
Specific suggestions for improvement 
Summative
Certification of performance
Grades
Promotion 
Judgments on success
Justification of resources
Recruitment of learners/educators
Dissemination 
Formative evaluation that is used to help individual improve performance
Summative evaluation used to make judgments or decisions about individual
Individual can be learner or faculty
2/22/2019
[ADD PRESENTATION TITLE: INSERT TAB > HEADER & FOOTER > NOTES AND HANDOUTS]

32. STEP 6: Evaluation and feedback
Level
Use
Individual
Program
Formative
Use of team leadership checklist tools
Use of clinical skills for each scenario
Learners and facilitators fill out evaluations at end of each session
Summative
N/A
Biannual evaluations by learners on value of program
Formative evaluation that is used to help individual improve performance
Summative evaluation used to make judgments or decisions about individual
Individual can be learner or faculty
2/22/2019
[ADD PRESENTATION TITLE: INSERT TAB > HEADER & FOOTER > NOTES AND HANDOUTS]

33. STEP 6: Evaluation and feedback
2/22/2019
[ADD PRESENTATION TITLE: INSERT TAB > HEADER & FOOTER > NOTES AND HANDOUTS]

34. Summary
Kern’s 6 steps to curriculum development provide a framework for creating effective simulation scenarios and curriculums
Remember the SMART mnemonic and Bloom’s Taxonomy to create learning objectives for each scenario
Use a template to create the stages of the scenario to help guide facilitators and simulation technicians
Remember Kirkpatrick’s pyramid when developing evaluations of your simulation curriculum