Weekly Assigned Reading and Exams Improve ABS Exam Pass Rates PRESENTATION IS LIMITED TO 10 MIN. Five minutes of Q & A follow. Bring this presentation on a jump drive or CD and give it to the audio-visual guys the day before if possible. I’ve already filled out an AV Form specifying that you’re presenting this on Powerpoint. They should have it ready. Christian de Virgilio MD, Tony Chan MD, and Kel Miller MBA Harbor-UCLA Medical Center

Success on ABS Exams Linked to ABSITE Performance Shetler Am J Surg 1982 High correlation between ABSITE and QE ABSITE < 25th percentile in PGY 5 year Wade et al Surgery 1993 Correlation between ABSITE and QE pass Correlation between Chief resident ABSITE and CE pass No specific ABSITE cutoff value Sparked interest in improving ABSITE

Factors Affecting ABSITE Performance Dean et al Am Surg 1984 Programmed reading Hirvela and Becker, Am J Surg 1991 Reading effort Itani et al J Surg Res 1997 Problem based learning conference Individual resident self-study Godellas and Huang, Am J Surg 2001 Conference attendance Amount of study Confidence Segue: “Of note, one reason why programmed reading as opposed to passive learning improves ABSITE performance comes from the field of cognitive science, which has demonstrated that memory is facilitated by active learning and stress - de Kloet Joels - which leads to our experience with educational conferences designed at Harbor-UCLA…” and then on to next slide…

Harbor-UCLA ABSITE Performance Arch Surg 2003 1 year of assigned reading and weekly exams Mean ABSITE increased from 59th to 75th national percentile Am Surg 2005 3 year study ABSITE improvement sustainable Mean ABSITE remained at 75th percentile Unclear whether this would translate to improved ABS exam performance

Objectives Primary: to determine whether weekly assigned reading and examinations, sustained over a 6 year period, would result in an improvement in the combined ABS QE and CE first-time pass rates Secondary: to identify whether ABSITE thresholds were predictive of passing the ABS exams

Methods Period 1: Jan 1997 - Jun 2001 Period 2: July 2001 - Nov 2007 Prior to institution of reading and exam program Period 2: July 2001 - Nov 2007 After assigned reading and weekly exams Main outcome measures First time pass rates on ABS QE, CE and combined QE/CE Secondary outcomes Mean ABSITE national percentiles Number of ABSITE scores < 35th percentile Overall ABSITE thresholds predictive of ABS exam failure

ABSITE Review Conference Instituted July 2001 and has continued for first 7 months of each academic year Program director (PD) assigned chapter in standard textbook weekly 15 question exam created by PD each week Goal: stimulate reading and simulate stress of exam Mandatory conference (PD administered exam) Time limit, adjacent seat partner graded exam PD called on residents to give correct answers Grades recorded, answers reviewed, top scores announced About 28 chapters reviewed each year Content of questions changed yearly Conference in 7th year, now held every other week

Results Median ABSITE Percentile % ABSITE < 35th Percentile ABS QE Pass Rate CE Pass Rate QE/CE Pass Rate Period 1 16 residents 62nd 18% 23/126 81% 67% Period 2 28 residents 79th 7% 12/159 93% 93%† 89% p value < 0.001 0.01 0.4 0.07 †One resident has not yet taken the CE

ABSITE < 35th Percentile 0.57 4/26 (15%) 7/24 (29%) QE/CE Fail 1.0 3/25 (12%) 3/23 (13%) CE 0.1 1/25 (4%) 5/24 (21%) QE P Value ABSITE  35th percentile ABSITE < 35th percentile

ABSITE < 30th Percentile 0.55 4/26 (15%) 7/21 (15%) QE/CE Fail 0.47 3/27 (11%) 3/20 (15%) CE 0.01 1/34 (3%) 5/15 (33%) QE P Value ABSITE  30th percentile ABSITE < 30th percentile

ABSITE < 25th Percentile p Value QE Fail 5/15 (33%) 1/34 (3%) 0.01 CE 3/14 (21%) 3/34 (9%) 0.34 QE/CE 9/17 (53%) 2/31 (6%) 0.001

Discussion How is Cognitive Knowledge Best Acquired? ABSITE course as structured stimulates reading and produces a degree of stress Joels et al Trends Cogn Sci 2006 Stress facilitates learning and memory if: stress is experienced in the context and time of the event to be remembered and stress not too powerful Stress hormones induce focused attention and improve memory of relevant information Ferguson and Warshaw Arch Surg 2006 Web based educational tool (done at home) did not improve ABSITE performance Low level of usage ? too low stress

Conclusions A program of weekly assigned reading and examinations in a conference format can be sustained over several years Improved ABSITE performance Trend towards improved first-time pass rates on combined ABS QE and CE Labor intensive Reason for success of format Motivation to read regularly Active learning with appropriate stress facilitates memory

Conclusions Low ABSITE at any PGY level predicts failure of ABS exams ABSITE < 35th percentile Trends towards failing QE Remediation ABSITE < 30th percentile predicts failing QE ten fold increased risk ABSITE < 25th percentile predicts failing QE and combined QE/CE Residencies seeking to improve ABSITE and ABS exam performance should consider a similar educational program