1. Accreditation of Canadian Medical Schools: History and the Current Practices May 2010 Dr. Linda N. Peterson, Assistant Secretary Committee on the Accreditation of Canadian Medical Schools

2. Canadian Medical Schools 17 Medical Schools in Canada 1-6 per province Oldest 1854 Queen’s University School of Medicine Newest 2005 Northern Ontario School of Medicine

3. Undergraduate Medical Education Programs and Distributed Campuses

4. Canadian Medical Schools are Accredited by Two Bodies Accreditation of undergraduate medical education programs leading to the MD degree Committee on Accreditation of Canadian Medical Schools (CACMS) Liaison Committee on Medical Education (LCME)

5. Overview of the Accreditation of Canadian Medical Schools Abraham Flexner 1910

6. Overview of the Accreditation of Canadian Medical Schools Flexner in 1910 (8/155 were Canadian) –Only 5 schools were identified as excellent, how many were Canadian? LCME was created in 1942 and accreditation became regular for Canadian schools after 1952 CACMS created in 1979 to reflect the difference in health care delivery Dual accreditation using a joint process

7. Accreditation A standards-based, peer-review process of continuous quality assurance / improvement of the function, structure, and performance of a medical education program

8. CMA AFMCAMA AAMC Accreditation of Programs leading to the MD degree

9. CACMS and LCME Both Committees meet 3 times a year within a week or two of each other. LCME meets in Washington or Chicago, alternating between cities every year. The two Committees have a joint meeting once every three years.

10. CACMS and the LCME CACMS voting members include 4 AFMC, 4 CMA, 1 LCME (Co-chair LCME), 2 CFMS and 2 FMEQ, and 2 public members LCME voting members include 6 AAMC, 6 AMA, 1 CACMS (CACMS Chair), 2 student members (AAMC and the AMA), and 2 public members

11. CACMS representation on the LCME and its Subcommittees The CACMS Chair is an LCME member Subcommittee on Standards –Chair of CACMS and Assistant CACMS Secretary attend Standards Subcommittee meetings Subcommittee on Policy –Secretary of CACMS attends Policy Subcommittee meetings

12. CACMS-LCME Joint Process Survey team has one LCME member Survey report documents the findings (one single report goes to both Committees) Committees deliberate separately and come to independent accreditation decisions Secretariats reconcile Committees’ decisions A single consolidated letter of accreditation Follow-up actions More on this later in the session

13. Secretariat Staff Each Accreditation Committee has a Secretariat Staff –Former senior members of a Faculty of Medicine with extensive experience in medical education, administration and accreditation

14. Secretariat Staff Roles 1 Secretariat Staff provide support to their respective committees- Survey team training- surveyors, secretaries and chairs Develop educational materials for schools, students and surveyors Organize and implement and participate in survey visits Conduct fact-finding visits, mandated and requested consultation visits

15. Secretariat Staff Roles 2 Secretariat Staff provide support to their respective committees- Document and communicate the decisions of the committees Organize systematic reviews of the standards and policies and procedures of the committees Provide consultative advice to schools on all matters related to accreditation

16. Secretariat Staff CACMS Nick Busing, MD, Secretary, CACMS Linda Peterson, PhD, Assistant Secretary LCME Dan Hunt, MD, MBA, Co-Secretary (AAMC) 2010-2011 Robert Sabalis, PhD, Assistant Secretary (AAMC) Barbara Barzansky, PhD, MHPE, Co-Secretary (AMA) 2009-2010 Dan Winship, MD, Assistant Secretary (AMA)

17. CACMS and the LCME Secretariat can help schools CACMS and LCME members can NOT provide consultation The CACMS and LCME Secretariat do NOT participate in the accreditation meetings/decisions/or provide information on schools The CACMS and LCME Secretariat CAN provide consultation support

18. CACMS/LCME Standards Functions and Structure of a Medical School 129 standards (128 apply to Canadian schools) in 5 sections: Institutional Setting (IS) – 16 standards Education Program Leading to the MD Degree (ED) - 49 standards Medical Students (MS) – 39 standards Faculty (FA) – 13 standards Educational Resources (ER) – 12 standards

19. The CACMS-LCME Full Survey Canadian medical schools accreditation cycle is 8 years Preparation for a full survey begins ~18 months before the on-site visit Three documents produced during a Self-study preceding the onsite visit –Medical Education Database –Institutional Self-study –Student conducted Independent Student Analysis Survey Team reviews the program- Standards based Survey Report Review and Decision by the CACMS and the LCME

20. Accreditation standards are NOT negotiable CACMS/LCME accreditation surveys are SUMMATIVE CACMS/LCME accreditation should be used as leverage for change “Two year rule” CACMS and the LCME expect that any areas of noncompliance should be brought into compliance within two years

21. Three components of preparing for the Full Survey Completing the Medical Education Database Conducting the Institutional Self-Study (ISS) Independent Student Analysis

22. Medical Education Database 1 The database is the mirror image of the standards Items contained in the database are linked to specific standards and are used to judge compliance Let’s look at some examples: –ED-2 –ED-31ED-31 The questions represent the elements of the standard and the data mentioned represent the evidence necessary to evaluate compliance with the standard. A SINGLE BASE YEAR, i.e., the most recently completed academic year

23. Medical Education Database 2 Each section should be completed by the person most knowledgeable about the corresponding topic Care should be take to ensure accuracy and consistency of data across sections of the database –ED-31 on mid-clerkship feedback, the clerkship formsED-31 on mid-clerkship feedback, the clerkship forms Faculty member who is the Self-study coordinator is responsible for ensuring that the database undergoes a comprehensive review to identify missing items or inconsistencies in the reported information. The Self-Study Taskforce should review the database to identify areas of noncompliance

24. Institutional Self-study Main objectives: To certify that a medical education program meets prescribed standards To promote institutional self-evaluation and improvement Brings together representatives of the administration, faculty (>100), student body(  5) and other constituencies to: –Collect and review data about the medical school and its educational program –Identify institutional strengths and issues requiring action –Define strategies to ensure that the strengths are maintained and any problems are addressed.

25. Organizing and Conducting an Institutional Self-study Select the right people to be the Self-study (Faculty member) and logistics (administrative) coordinators –Organized, knowledgeable, respected, committed ISS Taskforce- select “true leaders” who can make things happen and have institutional memory (include alumni, university officials, partners, etc.)

26. Organizing and Conducting an Institutional Self-study Ideally, the database materials and the Self- study document will be written coherently and in the same “voice” throughout Complete an honest and complete self-appraisal A “good survey outcome”: one in which the school’s self-study findings and the survey team’s findings are similar

27. Independent Student Analysis STUDENT-MANAGED QUESTIONNAIRE TO ALL STUDENTS STUDENT COMMITTEE ANALYSIS OF DATA INDEPENDENT STUDENT ANALYSIS: STRENGTHS AND CONCERNS + SUMMARY

28. Independent Student Analysis The student survey covers not only the teaching program but access and responsiveness of the faculty, quality of the facilities including technology, lockers, lounge and study space The survey team uses the Student Analysis and the AAMC Canadian Graduation Questionnaire (CGQ) which includes multiple factors, student support services, financial debt and data on mistreatment Poor student participation in the CGQ (incentives are OK) and/or the student survey may adversely affect the review of the program

29. Timeline for the Independent Student Analysis Process begins almost at the same time as the Self- study Co-chair model of one fourth year and one second year student is helpful for continuity and building corporate knowledge Survey development, implementation, collection and analysis takes time Executive summary and the full report should be ready ~5 months before the full survey

30. Before the Survey Visit Ideally, the Self-study Taskforce has access to the student report several months before the materials have to be submitted to CACMS/LCME survey team Schools can use the Survey Report Writing Guide- these are questions the survey team must answer and the evidence they have to use Faculty corrects as many areas of noncompliance as possible and creates effective plans to address those that remain

31. Self-study materials sent to the Survey Team The Medical Education Database along with the Institutional Self-Study (ISS) and the Independent Student Analysis should be sent to the full survey team about 12 weeks before the survey visit

32. Organizing the Survey Visit Holds are placed on the calendars of all key participants: University chief executive Dean, UGME, Student Affairs, and Admissions Deans and senior leadership Department chairs Heads of affiliated hospitals Course/clerkship directors Career, academic, financial aid and personal counsellors, student health and mental health staff members as appropriate

33. Organizing the Survey Visit 2 In programs with more than one geographically separated campuses, it may be necessary to add members to the survey team and/or another day into the schedule

34. Organizing the Survey Visit 3 Sunday evening to Wednesday morning- full survey Sunday evening to Tuesday morning- limited “Snapshot” in time Administrative (logistics) Self-study Coordinator in Canadian schools works with the survey team secretary to develop the visit schedule Work on the schedule soon after the database is submitted No gifts of any type for the team

35. CACMS-LCME Survey Team 1 team representing CACMS and LCME –For Canadian Schools, team includes: Chair appointed by CACMS Secretary appointed by CACMS Team member appointed by CACMS Team member appointed by the LCME Student (member of CACMS) – often but not always Faculty fellow appointed by CACMS – often but not always

36. Purpose of the Survey Visit Survey team answers questions regarding compliance with standards raised by the database, the Self-study, and the Independent Student Analysis Fill in gaps in information- follow the Survey Report Writing Guide Verify information and impressions from the three documents Have questions answered Collect updated information

37. Survey Visit & Exit Conference The survey team reviews compliance with all 128 standards Meets with university, hospital and faculty leadership, course and clerkship directors, students, department chairs, junior faculty, library and educational resource administrators Exit conferences with the dean and university leadership to review a list of findings (organized by the sections of standards) This list will be provided to the dean The Exit Conference IS NOT THE FINAL WORD

38. After the Survey Visit: The Survey Report Team writes the one single survey report which documents the findings - linked to individual standards The report will begin with the summary of the team’s findings linked to specific standards This initial report is reviewed by the CACMS and the LCME Secretariats After incorporation of Secretariat comments, the report is sent to the dean for identification of any factual errors

39. After the Survey Visit Review the Survey Report The dean reviews the survey report carefully –The final report will become the formal record of the visit and will be used by the CACMS and the LCME as the basis for the accreditation decisions The dean provides timely feedback on any errors of fact in the report to the CACMS and LCME Secretariats If there is a disagreement with the “tone” of the report –Concerns are forwarded via the CACMS and LCME Secretariats –If these are not resolved, the concerns are documented in WRITING- this letter will accompany the report on the CACMS and LCME meeting agendas

40. CACMS and the LCME Separately Decide Accreditation Status CACMS and the LCME deliberate separately and come to independent accreditation decisions –The final survey report is reviewed by 2 reviewers from both Committees. The two reviewers come to consensus on a recommended decision; followed by Committee discussion and a Committee vote Secretariats reconcile Committees’ decisions

41. Possible accreditation decisions Full continuing accreditation for an eight-year term (with or without follow-up from the school) Full continuing accreditation with a reduced term. Continuing accreditation with no fixed term pending outcome of A status report A limited (focused) survey A Secretariat fact-finding survey A Secretariat consultation Place program on Warning of Probation Place program on Probation (appealable) Withdraw accreditation (appealable) Deny accreditation (appealable)

42. CACMS/LCME Consolidated Letter of Accreditation A process for reconciling accreditation decisions has been developed by the two Committees and now covers the full range of actions Secretariats reconcile Committees’ decisions All areas of noncompliance identified by each committee are included The decision about accreditation status is the most conservative and the follow-up actions are the most comprehensive A single consolidated letter of accreditation

43. Citations – 2004-2008 Canadian schools

44. Canadian Top 10 Areas of Noncompliance 2004-2008 1.ED-2 Required clinical experiences and monitoring 2.ED-8 Comparability of educational experiences across instructional sites 3.ED-24Residents’ preparation for their roles in teaching and evaluation 4.ED-30Formative and Summative evaluation 5.ED-31Mid-course and Mid-Clerkship Feedback

45. Canadian Top 10 Areas of noncompliance 2004-2008 6.ED-33 Curriculum Management 7.ED-37Monitoring curriculum content 8.ED-35Systematic review of the curriculum 9.MS-32Student mistreatment 10.ED-1Educational program objectives

46. New AFMC Interim Accreditation Review Process AFMC Deans- Working Group on Accreditation To assist the schools in achieving and maintaining compliance with accreditation standards Checklists and an Interim Survey FORMATIVE

47. AFMC Future of Medical Education in Canada 10 recommendations for the direction of medical education Accreditation is seen as a leverage for change AFMC Deans strongly support measuring school’s achievement of these recommendations – informal moving to a more formal process in the future

48. Resources http://afmc.ca/education-cacms-e.php http://www.lcme.org/pubs.htm –Functions and Structure of a Medical School (Standards for Accreditation of Medical Education Programs leading to the MD degree) –LCME Rules of Procedure –Guide to the Institutional Self-Study –LCME Medical Education Database –Guidelines for the Conduct of Accreditation Survey Visits –The Role of Students in the Accreditation of Medical Education Programs –Survey Report Writing Guide –Accreditation Issues related to Distance Learning –Guidelines for new and developing Medical Schools –Guidelines for Increasing Class Size in Medical Schools Accredited by the LCME or CACMS –Template for reporting a Proposed Class Size Increase –Template for reporting a Proposed new or Expanded Branch Campus to the LCME and CACMS –Defining the scope of LCME Accreditation: Programs, Campus and Tracks –List of LCME-accredited medical schools in the US and Canada –AFMC Future of Medical Education in Canada