Validation of the UW Virtual Reality TURP Simulator Version 1.0 Presented By: Timothy Kowalewski Robert Sweet, MD Peter Oppenheimer Suzanne Weghorst Jeff Berkley, Ph.D

What is TURP? Transuretheral Resection of the Prostate Transuretheral Resection of the Prostate Gold-standard for treating obstructive urinary symptoms Gold-standard for treating obstructive urinary symptoms Challenging to teach and learn Challenging to teach and learn

Goals of the Project Overall Project Goal: Overall Project Goal: To help streamline urological procedure education with the aid of simulation technologyTo help streamline urological procedure education with the aid of simulation technology Specific Goals of This Study: Specific Goals of This Study: Establish validation status of Version 1.0, UW Establish validation status of Version 1.0, UW TURP Simulator as a training and assessment tool. TURP Simulator as a training and assessment tool.

TURP: a good model for Simulation Objective assessment Objective assessment High risk to patients High risk to patients Apparent training gap Apparent training gap Common problem Common problem National Average Number of TURPS performed by residents Source: ACGME Sweet, et al. Journal of Endourology. October, 2002.

Mainstay operation Mainstay operation Apparent demand Apparent demand Amenable to current technology Amenable to current technology 99%: Validated Simulator Useful in Training 75%: Validated Simulator Useful after Residency …YES 98%: TURP is the Gold Standard of care TURP: a good model for Simulation

Methods Simulator Construction Simulator Construction  Virtual anatomy and visual elements  Virtual anatomy and visual elements  Real-time force / tactile feedback  Real-time force / tactile feedback  O.R. instrumentation  O.R. instrumentation  External physical model  External physical model  Auditory cues  Auditory cues

Methods Simulator Construction Simulator Construction Training Video Training Video  Definition of task and errors  Definition of task and errors  Statement of goal:  Statement of goal: “Efficiently resect as much tissue as possible while avoiding errors “Efficiently resect as much tissue as possible while avoiding errors and minimizing blood loss, amount of irrigant used, coagulation and minimizing blood loss, amount of irrigant used, coagulation current and number of cuts…” current and number of cuts…” Special Thanks to Anthony Gallager, Ph.D

Methods Simulator Construction Simulator Construction Training Video Training Video Pre-Task Questionnaire Pre-Task Questionnaire  Demographics, gender, professional experience, education, ‘video  Demographics, gender, professional experience, education, ‘video game’ experience, TURP-related questions, etc… game’ experience, TURP-related questions, etc…  Stratification of database  Stratification of database

Methods Simulator Construction Simulator Construction Training Video Training Video Pre-Task Questionnaire Pre-Task Questionnaire Pre-Compiled, 5-minute Task Pre-Compiled, 5-minute Task  Three trained, non-medical technicians  Three trained, non-medical technicians  Consistent, pre-determined responses  Consistent, pre-determined responses  Privacy and Anonymity  Privacy and Anonymity

Methods Simulator Construction Simulator Construction Training Video Training Video Pre-Task Questionnaire Pre-Task Questionnaire Pre-Compiled, 5-minute Task Pre-Compiled, 5-minute Task Post-Task Questionnaire Post-Task Questionnaire  Critique of simulator according to its elements  Critique of simulator according to its elements  Open feedback  Open feedback

Methods Simulator Construction Simulator Construction Training Video Training Video Pre-Task Questionnaire Pre-Task Questionnaire Pre-Compiled, 5-minute Task Pre-Compiled, 5-minute Task Post-Task Questionnaire Post-Task Questionnaire AUA Annual Conference AUA Annual Conference  72 Experts  72 Experts  19 Trainees  19 Trainees  19 Novices  19 Novices

Metrics Grams of tissue resected Grams of tissue resected Amount of irrigant used Amount of irrigant used Quantity of ambient blood Quantity of ambient blood Number and severity of bleeding vessels Number and severity of bleeding vessels Instrument position and interaction Instrument position and interaction Number of cuts at tissue Number of cuts at tissue ~ 12 Hz resolution ~ 12 Hz resolution Combinations thereof … Combinations thereof …

Results: Face Validity  Do you feel the UW TURP Simulator would be useful as a training tool?

Results: Face Validity  Would you like to see it implemented into the curriculum of residency programs?

Results: Face Validity  Do you feel it would be useful as a tool for accreditation?

Results: Content Validity Categorical Standardized Likert scale of global acceptability Acceptability threshold Slightly acceptable Moderately acceptable Totally acceptable Slightly unacceptable Moderately unacceptable Totally unacceptable

Results: Novices vs. Experts ( p < ) Result of Levene-conditioned two-tailed T-test quantifies resolving potential of selected metrics to determine differences between novices and experts.

Results: Construct Validity Performance of Errors

Construct/Concurrent Validity (Cross-sectional) Metrics correlations mimicked the operating room. Experts and trainees were more efficient at cutting than novices. (P<.01) Experts were more efficient at coagulating bleeders than trainees who were more efficient than novices. (P<.05, P<.01) Performance among experts did not exhibit “decay” Video game experience influenced performance in novices and experts only.

Conclusions Established Face and Content Validity Established Face and Content Validity Established Construct Validity (X-sectional) Established Construct Validity (X-sectional) Addressed Concurrent Validity (incomplete) Addressed Concurrent Validity (incomplete) NO assessment predictive validity or longitudinal studies NO assessment predictive validity or longitudinal studies For Version For Version

Version 2.0 Predictive and Construct Validation Anatomy ID Module Coagulation Module Cutting Module Advanced Resection Module