Presentation is loading. Please wait.

Presentation is loading. Please wait.

Creation of the Simulator Value Index Tool Adapted from workshop on 4.21.14 presented by American College of Surgeons Accreditation Education Institutes,

Similar presentations


Presentation on theme: "Creation of the Simulator Value Index Tool Adapted from workshop on 4.21.14 presented by American College of Surgeons Accreditation Education Institutes,"— Presentation transcript:

1 Creation of the Simulator Value Index Tool Adapted from workshop on presented by American College of Surgeons Accreditation Education Institutes, Technologies & Simulation Committee) Deborah Rooney PhD James Cooke MD Yuri Millo MD David Hananel MEDICAL SCHOOL UNIVERSITY OF MICHIGAN

2 Disclosures o David Hananel, No Disclosures o Yuri Millo, No Disclosures o James Cooke, No Disclosures o Deborah Rooney, No Disclosures

3 Overview of Main Topics o Introduction of project o Overview of 2014 IMSH Survey results o Summary of 2014 ACS Consortium results o Working meeting to refine the AVI algorithm o Apply AVI algorithm in group exercise o Discuss next steps

4 Introduction: How it all started o ACS AEI, Technologies and Simulation Committee o Guidelines for Simulation Development (Millo, George, Seymour and Smith) o University of Michigan o Need to support faculty in sim purchase/decision-making process (Cooke) o Discourse o Definition of “value” o Differences across stakeholder role (institution, administration, clinician, educator, researcher...)

5 Introduction: How it all started o Reached consensus on factors used when considering a simulator purchase o Survey 1 o IMSH general membership, N=2800 o January, 2014 o Workshop 1, n=16 o IMSH, January, 2014 o Survey 2 o ACS AEI Consortium membership, N = 455 o March, 2014 o Workshop 2, n = ? o ACS AEI-March, 2014

6 Introduction: The Instrument o Began with 31-item survey accessed via www (Qualtrics) o 4-point rating scale o (1 = not considered/not important  4= critical to me when I consider a simulator purchase) o 6 Domains o Cost, Impact, Manufacturer, Utility, Assessment, Environment/Ergonomics) o Demographics o Country/Institution o Stakeholder role o Involvement o Follow-up

7 = Grenada 1= Chile 1= Peru 1=Czech Republic 2 2=Singapore 3 = New Zealand total respondents, 72 individuals completed survey approximately 2+% of IMSH membership (2,800), 7 undesignated/16 incomplete IMSH Survey Sample: 67 institutions x 12 Countries

8 = Massachusetts 3 = Rhode Island 1 = New Jersey participants from US IMSH Survey Sample: 44 institutions x 22 States/US

9 46 58% 28 35% 26 33% 20 25% 6 8% 4 5% 1 1% n = 79 1 undesignated IMSH Survey Sample: Institution Affiliation

10 o Cost o Commercial Skills Centers (CSC) rated C1 (Purchase cost) lower than each of the other institutions, p =.001. o Manufacturer o CSCs rated M1 (Reputation of manufacturer) lower than each of the other institutions, p =.001. o Utility o CSCs rated U3 (Ease of data management) and o U11 (portability) lower than each of the other institutions, p =.001. o Ergonomics o Medical Schools rated item E2 (Ergonomic risk factor) much higher thank other institutions), p =.05.  CSCs rated E3 (Ease of ergonomic setup) lower than each of the other institutions, p =.001. IMSH Survey Results: Rating Differences by Institutional Affiliation

11 31 39% 19 24% 7 9% 8 10% 14 18% n = 79 1=undesignated IMSH Survey Sample : Stakeholder Role

12 o Cost o Clinicians rated C2 (Cost of warranty) lower than the other stakeholders, p =.048. o Utility o Clinicians rated U11 (portability of simulator) higher than other stakeholders, p =.037. IMSH Survey Results: Rating Differences by Stakeholder Role

13 37 46% 37 46% 4 5% 2 3% n = 80 IMSH Survey Sample : Involvement in Decision

14 o Although there are no differences across level of involvement, o There are different considerations during simulator purchasing process across; o Country o Institutional affiliation (commercial skills center may have unique needs) o Stakeholder role (Clinicians may have unique needs) o Keeping this in mind, let’s review the top factors considered IMSH Survey Results : Summary

15 AverageFactor (survey item number, item description)Domain Technical stability/reliability of simulatorUtility Customer service Manufacturer Ease use for instructor/administrator Utility Ease of use for learner Utility Relevance of metrics to real life/clinical setting Impact Ease of delivery and installation, orientation to simManufacturer Reproducibility of task/scenario/curriculumAssmnt/Res Purchase cost of simulatorCost Reputation of manufacturerManufacturer ScalabilityImpact Quality of tutoring/feedback from sim to learnersUtility Number of learners impactedImpact Cost of warrantyCost Cost of maintenanceCost Ease of configuration/authoring sim's learning management systemUtility -Physical durabilityUtility The SVI Factors: Top 15+1 Factors Ranked

16 ACS Consortium Survey: Introduction o Identical Survey items, ratings o Added durability of simulator question o 31  32-item survey accessed www (Qualtrics) o 4-point rating scale o (1 = not considered/not important  4= critical to me when I consider a simulator purchase) o 6 Domains o Cost, Impact, Manufacturer, Utility, Assessment, Environment/Ergonomics) o Demographics o Country/Institution o Stakeholder role o Involvement o Follow-up

17 =UK 1=France 1=Italy 1 65 total respondents, 54 individuals completed survey approximately 12% of ACS membership (455), 2 undesignated ACS Survey Sample : 41 institutions x 7 Countries 1=Greece 1=Sweden

18 = Massachusetts 1 = Rhode Island 1=Delaware 1 = Maryland participants from US 47 indicated institution ACS Survey Sample: 36 institutions x 17 States/US 1 1 1

19 37 67% 28 51% 24 44% 16 29% 2 4% 0 0% n = 55 ACS Survey Sample: Institution Affiliation 0 0%

20 9 16% 27 48% 1 <2% 3 <6% 13 23% n = 56 ACS Survey Sample: Stakeholder Role 2 <4% 1 <2%

21 25 45% 29 52% 2 3% n = 56 ACS Survey Sample: Involvement in Decision

22 o Although there are no differences across; o institution o stakeholder role o There are different considerations during simulator purchasing process across; o Level of involvement o (Self-reported “Responsible” folks are more concerned about number of learners impacted and Scalability) ACS Survey Results: Summary

23 But are there differences across IMSH and ACS membership? ACS Survey Results: Summary

24 Survey Results: IMSH v. ACS 4 (C2) 7 (I2)11 (M3)15 (U4) 22 (U11 )

25 o Cost o ACS members rated C2 (Cost of warranty) higher than the IMH members, bias =.40, p =.04. o Impact o ACS members rated I2 (Number of learners) higher than other stakeholders, bias =.53, p =.01. o Utility o ACS members rated U4 (Ease of report generation) higher than the IMH members, bias =.43, p =.02. o ACS members rated U11 (Portability of simulator) higher than other stakeholders, bias =.48, p =.01. Survey Results: Rating Differences by Conference

26 The SVI Factors: Top 15+1 Factors Ranked

27 Applying the SVI Tool o General impressions? What stood out? o What worked well? o What could have gone better? o Any surprises? o Usefulness? How might you use the SVI Tool at your institution? o Please complete the questions on “Feedback” Tab on the SVI Worksheet

28 Thank you: Our Contact Information o Deb Rooney University of Michigan o Jim Cooke University of Michigan o o David Hananel SimPORTAL & CREST University of Minnesota Medical School o o Yuri Millo Millo Group o Olivier Petinaux ACS American College of Surgeon, Division of Education


Download ppt "Creation of the Simulator Value Index Tool Adapted from workshop on 4.21.14 presented by American College of Surgeons Accreditation Education Institutes,"

Similar presentations


Ads by Google