1. Moving from observation to intervention Developing Interventions to Improve Mammography Interpretation: AIM Study Design sponsored by: American Cancer Society Longaberger® Company National Cancer Institute Breast Stamp Funds Berta Geller, Ed.D. Research Professor University of Vermont

2. Great variability in radiologists’ interpretative accuracy of mammography Few previous studies have tried to improve accuracy BCSC has the ability to examine the effect of an intervention on actual practice through our observational database. What we knew when we started

3. Interpreting mammograms

4. Assessing and Improving Mammography (AIM)  To develop and test two educational interventions designed to improve radiologists’ mammography interpretive performance.  Objective is to reduce inappropriate recall and increase cancer detection

5. Study Design Planned Aug 07 Jun 08 Oct 08 Feb 09 Actual Apr 08-Mar 09 Sep 09-Mar 10 May 10 Sep 10

6. Challenges to studying physicians in the community  Recruiting busy community physicians  Time it takes to complete intervention and evaluation  Dropout of physicians and loss of f/u data over time  Contamination within facilities  Complexities of computer programming  Predicting clinical trends  Use of digitized vs. digital mammography

7. Four study components 1) Determine types of findings in screening mammography that are challenging to radiologists 2) Develop educational content for the two intervention formats 3) Evaluate the acceptability and feasibility of both interventions; and 4) Measure the effect of the two interventions on clinical practice.

8. Learning theory  We used a blended framework:  Bandura’s Social Leaning Theory - Complex relationships exists among knowledge, attitudes, beliefs and environment that promote or inhibit change Supports self-efficacy by enabling radiologists to learn and reinforce what they learn by observing others & increasing confidence using tailored approach  Green and Krueter’s Precede/Proceed Model Identify gaps in performance Enable learning by providing relevant Information and practice Reinforce change by providing feedback on lessons learned (immediately and in comparison to peers and experts)

9. 1. Determine difficult findings  From the test set data, identify cases that were most often answered incorrectly (inappropriate recall and missed cancers) according to expert findings:  Masses  Calcifications  Asymmetries  Architectural Distortions

10. 2. Develop educational content  Selected 40 cases to be used for both educational formats  Live Seminar  Self-paced DVD (had the option to do an additional 21 cases)  Experts developed teaching points for each case  Included feedback to radiologist

11. Live seminar  One 8-hour session  Conducted at two sites  San Francisco, CA (WA, SF, OR, NM)  Hanover, NH (NC, NH, VT)  Co-led by two Experts  Ed Sickles and Larry Bassett or Barbara Monsees  Used a case-based approach  Audience response system to provide comparative feedback  Agreed upon teaching points  Answered questions from the audience

12. Self- paced DVD  Used the same case-based approach as the live seminar  Told if their responses were correct or not  Provided same teaching points as in live seminar  Additional imaging was used to amplify teaching points  Participants could start and stop whenever they wanted  Could do in the convenience of home or office  Could view up to 21 extra cases

13. Screen for both interventions

14. Feedback reports

15. 3. Acceptability and Feasibility 4. Evaluation  After each component we asked for feedback on acceptability and feasibility  Will evaluate whether participation is related to clinical performance  Performance on the pre- and post test sets  Knowledge-based pre- and post tests  Recall rates and other measures from actual clinical practice in the future

16. Next steps  Identify low performing radiologists using criteria developed by this group to recruit into new intervention study.  At the International Cancer Screening Network in June, countries will meet to discuss an international AIM study.  Radiologists from other countries take the same test set to determine if radiologists’ skill or setting improves mammography accuracy.

17. References from the BCSC and FAVOR II  Aiello Bowles EA, Geller BM. Best ways to provide feedback to radiologists on mammography performance. AJR Am J Roentgenol. 2009 Jul;193(1):157-64.  Elmore JG, Aiello Bowles EJ, Geller B, Oster NV, Carney PA, Miglioretti DL, Buist DSM, Kerlikowske K, Sickles EA, Onega T, Rosenberg RD, Yankaskas BC. Radiologists’ Attitudes and Use of Mammography Audit Reports. In press, Academic Radiology.

18. Longaberger Headquarters