1. Impact of a Structured Skills Lab Curriculum on Operative Judgment Deb DaRosa, David Rogers, Reed Williams, Linnea Hauge, Ken Murayama, Alex Nagle, Keith Millikan, and Gary Dunnington

2. Purpose Develop and test an instructional design template for skills lab curriculum that is: 1.Based on pertinent theories 2.Focuses on judgment & procedural skills 3.Is feasible and portable 4.Includes evaluation component

3. Research Question Is there a difference…. in the judgment and technical skills of residents exposed to a skills lab curriculum based on a structured instructional design template?

4. Team Leaders Curriculum Team David Rogers* Linnea Hauge Ken Murayama Deb DaRosa Evaluation Team Reed Williams* Keith Millikan Alex Nagle Gary Dunnington Deb DaRosa

5. Cognitive Task Analysis Arises from Human Factors Group of analytical techniques designed to capture expert thinking Elstein AS. Decision analysis in surgical education. World J. Surg 1989;13:287-91.

6. CTA Example During the course of a laparoscopic cholecystectomy, the surgery resident variously strays into the lumen of the gallbladder and the parenchyma of the liver.

7. Experimental Design T group RO 1 X O 2 C group RO 3 O 4 InstitutionTC Total A 9 7 16 B1315 28 C10 9 19 Total 3231 63

8. Instructional Design Template Curriculum & Learning Theories Whole-Part Method Objectives Feedback – written/oral Distributed learning episodes Active & learner-centered Repetition and practice Problem & experience-oriented

9. Curriculum: Session 1 Both T and C Groups - Baseline Orientation by Chair or PD Written pretest View MedAscend CD of Lap Chole Question/Answer period Box trainer exercises

10. Written Exam Items Which trocar placement is optimal for a laparoscopic cholecystectomy?

11. Example: Written Test Which retraction of the gallbladder provides optimal exposure of the cystic duct?

12. Example: Written Exam List up to three methods for determining whether the veress needle has penetrated into the abdominal cavity.

13. Curriculum: Session 2 Homework: Pblm centered exercise Assigned readings Pre trial testimony Review designed CD-ROM – 12 critical steps of LC Worksheet to record steps Error brainstorm session Summary

14. Curriculum: Session 3 Recall exercise w/ CD-ROM Review/Discussion Porcine Practice –Think aloud rehearsal –Peer feedback using Porcine checklist

15. Curriculum: Session 4 Both Groups Written post-test Porcine exercise (45 minutes) using think aloud rehearsal Faculty (blinded) direct observation with final feedback – Porcine checklist

16. Measures and Analysis Measurement Instruments Written pre and post-test LC Porcine Evaluation Form LC Evaluation Form for OR Data Analysis Cronbach’s Alpha Coefficients Student’s T-test

17. Results: Reliability of Measures MeasureKnowledgeSkillErrors Pretest.75 Post test.64 LC Porcine.83.90.53 LC OR Eval.74.71.50

18. Results: LC Porcine Items T M (sd) C M (sd) t p Knowledge36(5.8)36(6.9)-.10ns Technical68(21.5)56(26.7)-1.6ns Errors47(52.9)41(48.6)-.33ns

19. Results – OR Evaluation Items T M (sd) C M (sd) t p knowledge34(3.1)32(4.5)-1.31ns Technical25(2.7)23(2.9)-0.10ns Errors4.0(3.7)4.2(3.3)0.12ns

20. Written Pre and Post-test Measure T M (sd) C M (sd) tp Pretest48(11.1)51(9.2)1.16ns Post-test61(6.2)55(8.0)-2.47.02* Pre/Post Difference 11 (0.6)4(10.3)-2.38.02*

21. Study Limitations Small sample size Subject mortality – OR evaluation Insufficient practice time Changes in study personnel Outcome evaluation flaws –Talk aloud –Eubanks items –Direct observation by faculty

22. Conclusions Instructional Design Template Effective in: CTA methodology – blueprint for cohesive curriculum and reliable exams Enhancing intra-operative judgment as assessed on written examination

23. Revised Instructional Design Template: Part 1 Descriptive Information Assumptions about learners Description of skill - Cognitive task analysis - Procedural task analysis Potential errors and complications - Prevention, anticipation, & management strategies

24. Revised Instructional Design Template: Part 2 Instructional Plan Objectives and Teaching plan –Time, resources, sequence, etc Homework/assignments Guidelines for facilitator/learners Practice guidelines –Frequency, intensity, duration

25. Practice, Practice, Practice Ericsson KA, Krampe RT, Tesch-Romer C. The role of deliberate practice in the acquisition of expert performance. Acad Med, 2004.

26. Revised Instructional Design Template: Part 3 Feedback and Evaluation Practice performance standards –Time, conditions, etc. How feedback will be given Pre-test description Post-test description Outcome evaluation criteria

27. Thank You! Foundation of the Association for Surgical Education (CESERT) SIU: Janet Ketchum Rush: Daniel Deziel, MD Northwestern: John Coyle,MD, Tom Powers & Deb Rooney