1. Overview of Online CME The Seventh Annual Meeting of the Global Alliance for Medical Education June 23-25, 2002 The McGill Faculty Club

3. Online CME – An Update Review of June 2002 Bernard M. Sklar, M.D., M.S. www.cmelist.com bersklar@netcantina.com

4. Plan of Talk Results of Survey Types of Instruction Physician Use of CME and Online CME Obstacles to Physician Use

5. Master’s Thesis This review is based on a recent update of the database that I created for my master’s thesis, The Current Status of Online Continuing Medical Education (June 2000). Find the thesis online at http://www.cmelist.com/mastersthesis http://www.cmelist.com/mastersthesis The thesis was based on a review of the CME literature and a survey of online CME done in February 2000.

6. How Was the Survey Done? Internet search of multiple search engines using search string “online + continuing + medical +education” Following up leads from those searches Information from ACCME Email from viewers and CME providers

7. Description of the List Each entry shows the name and URL of the site, when I last visited, how many credit hours are available, who awards the credit, the cost per unit, when the educational material was last updated, a description of the site and its contents and links to individual courses found at the site.

8. Extensive Updates I have been maintaining the list for about 5 years The list was updated for my master’s thesis in February 2000, again in August and December 2000, in December 2001 and in June 2002.

9. List of Online CME

10. Database Created from List Based on examining each site, I created an Access database of the 207 sites actively offering CME in June 2002. The DB contains the number of activities, number of hours of instruction, types of instruction, specialty audiences, cost to users, sources of financial support and other parameters.

11. Results of Study I The number of sites and activities is rapidly increasing April 1997 – 13 sites December 1997 – 18 sites August 1998 – 61 sites May 1999 – 69 sites December 1999 – 87 sites

12. Results of Study II February 2000 –96 sites, 1874 activities, 3064 credit hours August 2000 – 135 sites, 3659 activities, 5659 credit hours December 2000 – 150 sites, 3510 activities, 6553 credit hours – Because of overlap, duplication, and miscounting, the “true number” of hours should have been about 5500.

13. Results of Study III December 2001 197 sites 12026 activities 17523 hours

14. Results of Study IV June 2002 209 sites 10952 activities 18266 hours

15. Size of Sites – December 2000 No. of Credit- Hours Number of Sites Number of Hours % of Sites % 0f Hours Greater than 10064082462 50-9912779812 25-49217171411 10-24416812710 5-929193193 < 541100272 Total1506553100

16. Size of Sites – December 2001 No. of Credit- Hours Number of Sites Number of Hours % of Sites % 0f Hours Greater than 1001414587783 50-99959653 25-49291046156 10-2456918285 5-938271192 < 551103261 Total19718266100

17. Size of Sites – June 2002 No. of Credit- Hours Number of Sites Number of Hours % of Sites % 0f Hours Greater than 1001214615580 50-9916111986 25-49311137156 10-24621036306 5-933227161 < 55412926<1 Total20918263100

18. The Largest Sites December 2000 Name of SiteNo. of Activities No. of Hours HealthStream*9071360 Challenger7901 CMEWeb507759 Medscape**539726 ArcMesa63232 Milliman/Robertson** * 6104

19. The Largest Sites December 2001 Name of SiteNo. of ActivitiesNo. of Hours eMedicine CME650010000 CMEWeb9761400 Challenger10897 cmecourses (HS)300500 WEBMD Just in Time 1400350 Medscape CME200300 ArcMesa80293

20. The Largest Sites June 2002 Name of SiteNo. of ActivitiesNo. of Hours eMedicine CME*650010000 CMEWeb*9761400 Challenger14964 TheAnswer.com*5300 cmecourses (HS)*300500 JournalBytes (All Spec)108372 Medscape CME**200300 ArcMesa80293 Radcourses (HS)28128 RSNA Education Exhibits127 Natal U83122 Mypatient.com109

21. Fee Structure by Site Dec 2000 Dominant or Average FeeNumber of Sites% of Sites Free7449 <$5 per hour21 $5 per hour64 $6-9 per hour1611 $10 per hour149 $11-14 per hour64 $15 per hour107 $16-19 per hour53 $20 per hour64 $21-24 per hour21 $25 per hour53 >$25 per hour43

22. Fee Structure by Site Dec 2001 Dominant or Average FeeNumber of Sites% of Sites Free9850 <$5 per hour63 $5 per hour64 $6-9 per hour137 $10 per hour2211 $11-14 per hour95 $15 per hour1910 $16-19 per hour32 $20 per hour53 $21-24 per hour21 $25 per hour105 >$25 per hour42

23. Fee Structure by Site June 2002 Dominant or Average FeeNumber of Sites% of Sites Free10550 <$5 per hour52 $5 per hour63 $6-9 per hour135 $10 per hour2411 $11-14 per hour84 $15 per hour2512 $16-19 per hour41 $20 per hour31 $21-24 per hour1<1 $25 per hour104 >$25 per hour52

24. Hourly Fee Structure Dec 2000 Dominant or Average FeeNo of Hours% of Hours Free158724 <$5 per hour1032 $5 per hour2233 $6-9 per hour118318 $10 per hour105016 $11-14 per hour5248 $15 per hour150523 $16-19 per hour1703 $20 per hour511 $21-24 per hour180 $25 per hour611 >$25 per hour791

25. Hourly Fee Structure Dec 2001 Dominant or Average FeeNo of Hours% of Hours Free197811 <$5 per hour3658 $5 per hour1028559 $6-9 per hour171210 $10 per hour14478 $11-14 per hour4703 $15 per hour8455 $16-19 per hour55<1 $20 per hour87<1 $21-24 per hour9<1 $25 per hour1141 >$25 per hour1551

26. Hourly Fee Structure June 2002 Dominant or Average FeeNo of Hours% of Hours Free176310 <$5 per hour5663 $5 per hour1020356 $6-9 per hour215312 $10 per hour16439 $11-14 per hour4342 $15 per hour10095 $16-19 per hour1371 $20 per hour57<1 $21-24 per hour7<1 $25 per hour115<1 >$25 per hour1771

27. Financial Support Dec 2000 Source of SupportNo. of Sites% of Sites Commercial Companies4332 University/Medical School5440 Government75 Medical/Specialty Association1310 Foundation75 Insurance or Managed Care43 User Fees/Partial or Complete7651

28. Financial Support Dec 2001 Source of SupportNo. of Sites% of Sites Commercial Companies9950 University/Medical School8342 Government116 Medical/Specialty Association4724 Foundation179 Insurance or Managed Care53 User Fees/Partial or Complete9950

29. Financial Support June 2002 Source of SupportNo. of Sites% of Sites Commercial Companies11354 University/Medical School8641 Government157 Medical/Specialty Association5828 Foundation157 User Fees/Partial or Complete10450

30. Sites by Specialty – Primary Care Dec 2000 No. of Sites% of Sites Primary Care Sites9362 including: Family Practice8355 Internal Medicine8053 Pediatrics2215 Obstetrics/Gynecology2215

31. Sites by Specialty – Primary Care Dec 2001 No. of Sites% of Sites Primary Care Sites11458 including: Family Practice9548 Internal Medicine8644 Pediatrics2915 Obstetrics/Gynecology2513 Multiple Specialties63

32. Sites by Specialty – Primary Care June 2002 No. of Sites% of Sites Primary Care Sites12359 including: Family Practice9847 Internal Medicine9144 Pediatrics3718 Obstetrics/Gynecology2914 Multiple Specialties (>6)105

33. Sites by Specialty – Subspecialties Dec 2000 Subspecialty sitesNumber of Sites% of Sites including:11375 Neurology2114 Psychiatry2315 Cardiology2013 Oncology139 Infectious Disease139 Radiology*107 Dermatology75 Gastroenterology75 Pulmonary96 Surgery75 General Interest2617

34. Sites by Specialty – Subspecialties Dec 2001 Subspecialty sitesNumber of Sites% of Sites including:14071 Neurology147 Psychiatry2111 Cardiology1910 Oncology105 Infectious Disease146 Radiology126 Geriatrics126 Pulmonary95 Surgery65 General Interest2617

35. Sites by Specialty – Subspecialties June 2002 Subspecialty sitesNumber of Sites% of Sites 12359 including: Cardiology2311 Psychiatry2010 Oncology178 Neurology157 Radiology147 Infectious Disease136 Urology115 Endocrine105 Pulmonary105 Geriatrics94 Surgery94

36. Sites by Specialty-2001- Other 26 sites (13%) offer subjects of interest to many different specialties; for example, ethics, legal, practice management, genetics, and basic science Many other specialties were included at 5 or fewer sites

37. Sites by Specialty-June 2002- Other 33 sites (16%) offer subjects of interest to many different specialties: for example, ethics, legal, practice management, risk management, tobacco cessation, genetics, basic science Many other specialties are included at 5 or fewer sites

38. Five “Different” Sites I CE Medicus has no CME of its own, but offers access without fee to about 600 activities at five sites (apparently by special arrangement)CE Medicus Digiscript contains many hundreds of audio and video slide lectures recorded at medical meetings. The yearly charge is $400. Some activities offer CME and some do not. The site is searchable by medical topic and by sponsoring organization. You may have to pay an additional fee for CME credit by any given sponsor. Digiscript Doctor’s Guide also has no CME of its own, but offers descriptions of over 600 activities (free and fee) with links to those coursesDoctor’s Guide

39. Five “Different” Sites II Stanford SKOLAR offers credit for performing Internet literature searches on topics of your own interestStanford SKOLAR University of Wisconsin Professional Courses offer credit for courses on non-medical subjects which could be expected to improve your practice or your life.University of Wisconsin Professional Courses

40. Sites I Could Not View There may be a number of proprietary sites, e.g., staff model HMOs, like Kaiser- Permanente, where access to instruction is limited to staff members of that organization. Those sites are not reviewed in this report.

41. Email Reminders June 2002 About 25 sites send out regular email reminders about additions to their lists of activities on request by users: American College of Cardiology, Boston University, Cancer Education, CME Reviews, Cyberounds, Doctor's Guide Webcasts, Ecornell, EMedHome, EMedicine, cmecourses (HealthStream), Journal of Clinical Psychiatry, Medscape, Medinfosource, Medsite, MMWR, mypatient.com, Natal U, PDR.net, Pedsref.org, psychLINK, Psychiatrist.com (NetSociety), Serono, University of Wisconsin, Virtual Lecture Hall, and World Medical Leaders.

42. Types of Instruction-Definitions Text-Only Text-and-Graphics Slides-Only (or Slides and Text) Slide-Audio Slide-Video Question-and-Answer Case-Based Interactive Guideline or Consensus (usually text only) Correspondence Games

43. Types of Instruction – Dec 2000 Text only -- 37 sites; 25% Text-and-graphics – 45 sites; 30 % Slide-audio – 45 sites; 30 % Slide-video – 21 sites; 14 % Guidelines – 5 sites; 3 % Question-and-answer – 6 sites; 4 % Case-based Interactive – 27 sites; 18 % Many sites have more than one type of instruction

44. Types of Instruction – Dec 2001 Text only -- 47 sites; 24% Text-and-graphics – 59 sites; 30 % Slide-audio – 57 sites; 29 % Slide-video – 21 sites; 11% Guidelines – 5 sites; 3 % Question-and-answer – 9 sites; 5 % Case-Based-Interactive – 26 sites; 13 % Correspondence – 3 sites; 2 % Games – 2 sites; 1 % Slides-Only – 4 sites; 2 % Many sites have more than one type of instruction

45. Types of Instruction – June 2002 Text only -- 57 sites; 27% Text-and-graphics – 71 sites; 34 % Slide-audio – 60 sites; 29 % Slide-video – 23 sites; 11% Guidelines – 8 sites; 3 % Question-and-answer – 6 sites; 3 % Case-Based-Interactive – 31 sites; 15 % Correspondence – 2 sites; 2 % Games – 2 sites; 1 % Slides-Only – 4 sites; 2 % Slides-and-Text – 4 sites; 2 % Many sites have more than one type of instruction

46. More about Q&A Instruction Only 6 sites (2%) feature Q&A, BUT the number of hours is relatively large Challenger - 964 hours TheAnswerPage – 300 hours E-core – 27 hours Familypractice.com - 25 hours Total about 1316 hours (7 %)

47. CME Participation by Location Based on ACCME Figures for 2001 Live meetings and conferences account for 76 % of “physician-registrants” Home study CME (“enduring materials”) and journals account for 19.6 % of physician-registrants Online CME accounts for only 4.4 % of physician-registrants

48. Physician Usage of Online CME Physician usage of online CME is increasing, but still accounts for less than 5% of all CME According to ACCME: 1997: 13,115 physician-registrants (0.34%) 1998: 37,879 physician-registrants (1.03%) 1999: 79,536 physician-registrants (1.79%) 2000: 181,922 physician-registrants (3.57%) 2001: 230,055 physician-registrants (4.44%)

49. Why is Online CME Use So Low? I Many physicians still uneasy with computers and Internet* Many physicians unaware of online CME or don’t know how to find it Much live CME, especially at the hospital, is convenient, free and offers collegial interaction

50. Why is Online CME Use So Low? II A series of “gates” for the user to pass through Navigation; Download and install plug-ins Registration hassle Fear of giving out license, DEA, credit card Paying in advance for content you can’t view Get content free, leave without paying Each site has a different procedure and password

51. Why Choose One Online CME Site Over Another? Price Preference for Type of Instruction Email reminders Part of larger medical site Help with CME reporting Recommendation by colleagues, medical group Special arrangements with physician group

52. A Long Term Solution Eventually, CME will be totally integrated with the physician’s daily practice life Systems will be developed which allow a computer program to “know” when a physician is making a mistake or needs additional information The system will present instruction on the spot to help the physician do the right thing

53. Other Problems to Solve Another problem will be to prove that a given CME activity actually improves physician performance. For now, CME providers and evaluating groups will need to settle for some lesser measure, such as the difference in scores between pre-tests and post-tests, or statements by “experts” that the course will correct the deficiency.

54. Opportunities for Research More sophisticated (and expensive) methods of evaluation exist, such as reviewing physician charts or interviewing patients This is a great opportunity for research. There is grant money available for research into effective CME.

55. Conclusions I The number of online CME activities and credits is growing rapidly (but not so rapidly as in the previous periods) Online CME is becoming nicer to look at, with more graphics, lots more audio and video, and a bit more interactive programming

56. Conclusions II The percentage of CME hours earned online is still only about 4.4% Barriers to usage are still major Almost no proof that any kind of standalone CME, whether live, home study or online, and regardless of mode of instruction, is useful in changing physician practice

57. Conclusions III The future lies in the integration of medical practice, quality assessment and user- specific CME The challenges and opportunities are great

58. Important URLs Master’s thesis: www.cmelist.com/mastersthesis/ www.cmelist.com/mastersthesis/ My home page: www.cmelist.comwww.cmelist.com Online CME list: www.cmelist.com/list.htmwww.cmelist.com/list.htm Definitions of types of online CME instruction: www.cmelist.com/Instruction_Types_defin ed.htm www.cmelist.com/Instruction_Types_defin ed.htm