Stanford Vascular Surgery Simulation-based Endovascular Skills Assessment: The Future of Credentialing? Maureen M. Tedesco, Jimmy J. Pak, E. John Harris, Jr, Thomas M. Krummel, Ronald L. Dalman, Jason T. Lee 22nd Annual Meeting Western Vascular Society September 10, 2007

Vascular Surgery Disclosures Jason T. Lee- educational grant from Cordis Endovascular to study simulation technology. Drs. Dalman, Krummel and Lee: technical grant from Cordis Endovascular in the form of 2 simulators.

Vascular Surgery Background High-fidelity simulation has become important in surgical education. –Laparoscopy –Endoscopy –Cystoscopy Training on simulation improves operating room performance of surgical residents.* * Seymour et al. Annals of Surgery, 2002

Vascular Surgery Background Simulation required during physician training for carotid angioplasty and stenting. Recent applications: –Skills assessment –Technical competency –Board certification American Board of Vascular Medicine American Board of Surgery

Vascular Surgery Purpose Does global performance assessment during endovascular simulation correlate well with self- reported procedural skill and prior experience level?

Vascular Surgery

Methods 17 general surgery residents interviewing for vascular fellowship training Pre-test questionnaire: –# of major index vascular procedures –# of specific endovascular procedures Diagnostic arteriograms Aortic stent-grafts Peripheral angioplasty/stenting Renal stenting Carotid stenting

Vascular Surgery Methods Procedicus Vascular Intervention System Trainer (VIST ® ) simulator: Right Renal angioplasty and stenting (RAS) module

Vascular Surgery Methods Subjects were evaluated by an experienced interventionalist using a global rating scale.

Vascular Surgery Angiogram  advance wire into suprarenal aorta without forming a J or pushing against obstruction  place pigtail catheter into renal angiogram position/wire manipulation  knowledge of renal anatomy/perform angiogram Wire Access  select proper catheter/wire for renal canalization  safely traverse lesion Intervention  select guiding catheter  select appropriate renal stent  deploy renal stent  select proper balloon for renal angioplasty post- stent  perform completion angiogram Global Rating Scale (1-5)

Vascular Surgery Methods VIST ® provided objective measurements: –total procedure time –fluoroscopy time –volume of contrast used (mL) –% of lesion covered –placement accuracy –presence of residual stenosis –# of cine loops used

Vascular Surgery Methods Post test questionnaire: –Grade his/her own performance –Opinion about optimal number of cases

Vascular Surgery Results Low Experience (LE,<20) Moderate Experience (ME, ) TOTAL Subjects Endovascular Cases (range) 11.1 ± 6.8 (4-20) 46.6 ± 22.6 (25-89) 29.9 ± 24.6 (4-89) Open Cases (range) 78.8 ± 38.0 (40-150) 75.0 ± 41.1 (40-150) 76.9 ± 38.2 (40-150)

Vascular Surgery Low Experience (n = 8) Moderate Experience (n = 9) p value Global assessment Total procedure time (sec) NS Fluoroscopy Time NS Contrast used (mL) NS % lesion covered NS *Placement accuracy (mm) NS No residual stenosis (% of group) 75%89%NS Number of Cine loops NS

Vascular Surgery Low Experience (n = 8) Moderate Experience (n = 9) p value Global assessment Total procedure time (sec) NS Fluoroscopy Time NS Contrast used (mL) NS % lesion covered NS Placement accuracy (mm) NS No residual stenosis (% of group) 75%89%NS Number of Cine loops NS

Vascular Surgery Post-test questionnaire: poor correlation between the global assessment score and subjects ’ self assessment score. Results

Vascular Surgery Results Post-test questionnaire: –vascular surgeons = 19.2 ± 14.4 cases –interventional cardiologists = 14.7 ± 14.8 cases –interventional radiologists = 12.3 ± 12.0 cases p = NS

Vascular Surgery Summary Significant difference in a global assessment score between two groups of surgical residents with varying levels of self-reported endovascular experience. Global rating scale was able to discern even minimal differences in experience. No difference between the two study groups with respect to the VIST objective measurements.

Vascular Surgery Limitations Only one “ expert ” observer, no inter- observer variability. Each subject underwent only one session, without the opportunity to practice or learn the equipment. Stress may have played a role in this testing situation.

Vascular Surgery Conclusion Correlation between self-reported case completion and global rating score by an observer. Objective measures provided by the simulator may not be valid to determine endovascular skills. More meaningful criteria to determine how to integrate simulation into skill assessment. Future research is required to determine if simulator-based testing should be incorporated into the credentialing of vascular specialists.

Vascular Surgery Thank you!