1. Responding to User Experience in the Design of Interactive Tutorials Robert Vander Hart and Peg Spinner University of Massachusetts Medical School October Conference for New England Librarians Dartmouth College - October 25, 2001

2. About the Web Site Site aims to teach principles of evidence-based medicine (EBM) Collaborative project between the Lamar Soutter Library and the Department of Family and Community Medicine at UMMS Funded through the Medical School’s Innovations in Medical Education Grant (IMEG) program

3. What is Evidence-Based Medicine? EBM movement began in 1970s (1) EBM integrates best research, clinical expertise, and patient concerns (2) EBM emphasizes keeping up-to-date with the latest research findings

4. Design Principles Give clear goals for your tutorials Know your audience Realize that people generally scan, not read, text on the web (3) Give users opportunity for review Consider implications of linear versus non-linear presentation

5. Goals for Our Tutorials Define a clinical question Translate a clinical question into a searchable question Decide on the best type of study to address the question Perform a literature search

6. Our Audience Medical students –Second-year epidemiology class –Third-year clerkship program Residents –Family practice –Internal medicine –Pediatrics Faculty and community physicians Nursing students also show interest

7. Usability Testing Can be as simple or complex as you wish –Simple: get five volunteers and observe them using your site (4) –Complex: build a laboratory with one-way glass and video cameras Provides immediate and useful feedback in a “non-scientific” manner Ideally should be performed before a site is made public

8. Simple Testing Procedures Three separate testing series 4-5 volunteers for each series Observer form that included –Brief instructions read to the volunteer –Section to record demographic information –Sections for recording observations, start/stop times, volunteer’s comments and actions for each task Open debriefing period if time permitted

9. Test Results and Experiences Results recorded on a spreadsheet Nature of our audience became apparent –No time to spare –No patience to peruse information –Expect information to be presented concisely –These characteristics require a design response Direct observations had greatest impact on our decisions to make revisions (more than surveys or evaluation forms)

10. Before...

11. Actions Taken “Tightened up” wording of instructions Broke up instructions into separate, concise steps with bullets Bolded important words Changed font Provided in-depth help in separate links Included review section for those who skip Help

12. After...

13. Current Status Completed two tutorials Working on a third tutorial May include a PubMed (free MEDLINE) option for literature search section Interest expressed in parallel series of tutorials focusing on pediatrics All new projects will be subject to more usability testing

14. References 1.Levin A. “The Cochrane Collaboration,” Ann Intern Med 2001; 135(4): 310. 2.Sackett DL et al. Evidence-Based Medicine: How to Practice and Teach EBM. 2nd ed. New York: Churchill Livingstone, 2000, 1. 3.Nielsen J. “How Users Read on the Web,” http://www.useit.com/alertbox/9710a.html 4.Nielsen J. “Why You Only Need to Test With 5 Users,” http://www.useit.com/alertbox/20000319.html

15. For Further Information... Contact: Robert.VanderHart@umassmed.edu OR Peg.Spinner@umassmed.edu Web Site: http://library.umassmed.edu/EBM/tutorials