R. Sweet 1,2, T. Kowalewski 2, P. Oppenheimer 2, J. Berkley 2, J. Porter 1, R. Satava 3, S. Weghorst 2 1 Department of Urology, University of Washington, Seattle, WA Human Interface Technology Laboratory, Seattle, WA Department of General Surgery, University of Washington, Seattle, WA # Validation of the UW TURP Simulator as a Training Tool CRITERION LEVELS For the five-minute task of version 1.0, we established the benchmark values and criterion levels by calculating the mean grams resected, milliliters of blood loss, milliliters of irrigant used, number of cuts taken, time spent in orientation, time spent cutting, and time spent on coagulation for the accredited urologists group. CONSTRUCT VALIDITY AGE We found that among the board certified urologists, age did not correlate with performance measures, though the older participants tended to spend less time cutting and used the coagulation pedal more times. KEY METRIC CORRELATIONS  Grams resected As in the operating room, the amount of grams resected correlated with the amount of blood lost, the number of cuts and the amount of time devoted to cutting (p<.01). Only among the trainees did the amount of grams resected diminish with the amount of time spent with orientation (p<.05).  Blood loss Blood loss correlated strongly with the amount of time spent coagulating amongst the trainees (p<.01) but not the experts. In both groups, blood loss correlated directly with number of cuts, grams resected (p<.01, P<.05) and time spent cutting (p<.01). Among experts, the amount of blood loss also correlated directly with the subject’s pre-task perception that more TURPS are needed prior to being competent (p<.05).  Irrigant management The amount of fluid used correlated directly with amount of grams resected (p<.05), and inversely with the amount of time devoted to coagulation (p<.05) among the trainees but not the experts. Experts and trainees outperformed novices in all aspects of the simulation task, though more novice subjects and trainees are needed to establish meaningful conclusions. Do the skills acquired to perform TURP exhibit decay? We found no correlation between the number of TURPs performed in the last months-2 years and any of the basic performance measures in the experts. One and two year experience influenced trainee performance, but the number of TURPs performed in the last month did not influence trainee performance. CONCURRENT VALIDITY VIDEO GAME EXPERIENCE  As expected, trainees and experts more recently trained tended to have more video game experience.  Attendings with more video game experience, independent of age, tended to resect more tissue and have more blood loss (p<.05), while video game experience did not influence performance amongst the trainees. METHODS: We collected data on 141 participants. Urologists and trainees of varying levels of experience completed a pre-task questionnaire providing demographic data. They then watched a video instructing them to complete a five-minute resection task as efficiently as possible, attempting to maintain the least amount of blood loss, while conserving irrigant and avoiding operative errors. Metrics which were logged for each subject included grams resected, blood loss, fluid used, time spent with operative field orientation, time spent on coagulation, time spent cutting, number of cuts, number of times coagulation pedal pressed, and operative errors. Operative errors were defined as ureteral orifice injury, externalurinary sphincter injury, capsular perforation, rectal injury, and resection of the dorsal vein complex. They were asked to complete a post-task questionnaire that assessed their impressions of the simulator. Resection styles were correlated with Efficiency and all metrics were stratified with respect to experience level, video game experience and other demographic data. Feedback with regards to usability and acceptability levels was obtained after the simulation task. RESULTS: We addressed content, face, construct and concurrent validity. CONTENT VALIDITY Is TURP an important skill to learn? Ninety-eight percent of the participants believed that TURP is the standard of care for medically refractory symptoms of bladder outlet obstruction (Fig.2). Is there a role for computerized simulation in training TURP? Ninety-nine percent of the participants believed that a validated simulator would be useful in training programs (Fig. 3). Is there a role for computerized simulation of TURP to maintain skills after training? Seventy-six percent of the participants felt that a validated simulator would be useful to maintain skills after residency (Fig. 4). The various components of the simulator were individually rated using a standardized Likert Scale (0 = totally unacceptable, and 6 = totally acceptable) and all means were above the acceptability threshold (Fig. 5) FACE VALIDITY Was version 1.0 of the UW TURP simulator acceptable? OTHER FINDINGS Among the participants who are in the expert group, the mean perception of the average number of TURPS done by residents was (Actual number is 62) Among the participants who are in the expert group, the perception of the average number of TURPS that should be done before being allowed to practice independently is RESECTION STYLES  Qualitatively, we noticed that the younger generation of experts tended to resect until bleeding obscured their view and then focused on coagulation, while the older resectionists tended to attempt to maintain constant hemostasis. INTRODUCTION: We have completed version 1.0 of a prototype training simulator for TURP. Value of such a training tool is evident only through validation protocols. We present our work in progress for validating the UW TURP simulator as a training tool. Goals of initial validation experiment: We brought version 1.0 of the UW TURP simulator to the 2002 AUA in Orlando and performed a preliminary validation experiment. Our goals were to establish face, content, concurrent and some elements of construct validity of the model to simulate TURP. In addition to determining validity, the study was intended to target areas for improvement. UW TURP simulator at ACMI ® booth AUA 2002, Orlando Fig. 1 Fig. 3 Would a validated TURP simulator have utility for use in training programs? Fig. 2 Is TURP the standard of care? Fig. 4 Would a validated TURP simulator have utility for use after residency? ACCEPTABILITY THRESHOLD Fig. 5 Global Acceptability Assessment of Version 1.0 of UW TURP Simulator Totally Acceptable Moderately Acceptable Slightly Acceptable Slightly Unacceptable Moderately Unacceptable Totally Unacceptable 88% felt that it should be implemented into the curriculum of residency programs 93% of participants believed that the UW TURP simulator would be useful as a training tool. 58% felt that it should be used for accreditation.