1. Authors: Dr Craig M. Campbell Dr Marianne Xhignesse Dr. Ted Toffelmire September 20, 2012

2. Introduction Who we are …. Dr Craig M. CampbellDr Marianne Xhignesse Professor Ted Toffelmire

3. Conflict of Interest Declaration We do not have any: 1. financial relationships with members of pharmaceutical or medical supply companies. 2. research grants funded by industry 3. memberships on advisory boards/committees for any ‘for- profit’ industry Dr Craig M. Campbell and Dr Marianne Xhignesse

4. Conflict of Interest Declaration I do not have any: 1. research grants funded by industry 2. memberships on advisory boards/committees for any ‘for- profit’ industry I do: 1. receive payment from a commercial organization, “in kind” compensation, gifts or other considerations from Fresenius Medical Care Canada as the Senior Director of Medical Affairs (employee). Dr. Ted Toffelmire

5. Objectives By the end of this session, you will be able to: 1. Describe the proposed draft national accreditation standards for CPD provider organizations. 2. Discuss how these proposed national accreditation standards will affect your organization. 3. Provide your feedback and contribute to the development of these standards.

6. Who is involved…. CPD Provider Accreditation Standards Working Group Members 6 CACMERoyal CollegeCMQ / ACCME Penny Davis (Saskatchewan) Connie Switzer (CAG) Ernest Prégent (CMQ) Ford Bursey (Memorial) Nancy Brager (CPA) Murray Kopelow (ACCME) Sue Tallett (CPS)

7. Imperative for Change… Two sets of national standards 1. CACME standards are the basis to review University Offices of CME (or equivalent title) 2. Royal College standards are the basis to review National Specialty Societies and other national or provincial organizations. Note: CMQ uses the CACME standards to review provincial CPD provider organizations within Quebec. 7

8. Imperative for Change… Two sets of national standards Growing concerns regarding: 1. Degree of equivalence between these two sets of standards 2. Complexity required to understand and apply two sets of standards. 3. Concerns (pleas) to simplify the complexity and promote clarity and utility of the standards. Conclusion: Need to establish one national standard that is relevant to all types of CPD provider organization 8

9. Over the past year... CPD Provider Accreditation Standards Working Group 1. Implemented a survey of all accredited CPD providers in March 2012 (response rate 48/128) 2. Held several teleconferences and organized a one day retreat in April 2012 to incorporate all the feedback provided. 3. Reduced the number of standards from 16 to 12!!! 4. Developed a set of compliance criteria, draft questions and documentation requirements. 9

10. Purpose of the Accreditation Standards “Collectively the standards, criteria, and the accreditation process are foundational to establishing a national CPD provider accreditation system that is transparent, accountable and responsive to the profession and the public it is privileged to serve”.

11. Section 1:Purpose and Mission Standard 1.1 Organizational CPD Mission Standard 1.2 Assessing Effectiveness Standard 1.3 Scholarly Activities in Health Education Standard 1.4 Educational Independence

12. Section 1:Purpose and Missio n Standard 1.1 Organizational CPD Mission An accredited CPD provider organization has a formally written CPD Program mission statement that identifies the: target audience(s); overall purpose; and measurable goals.

13. Section 1:Purpose and Mission Standard 1.1 Organizational CPD Mission Evaluation Criteria Non-Compliance There is no written mission statement for the accredited CPD Provider organization’s CPD Program Partial Compliance There is a mission statement that has either not been formally approved or does not include all the required elements Compliance There is a formally approved written mission statement that identifies all required elements

14. Section 1:Purpose and Mission Standard 1.2 Assessing Effectiveness An accredited CPD provider organization implements a process, at least once every accreditation cycle, to: assess the degree to which the CPD Program’s mission statement has been achieved; and identify opportunities and develop plans for improvement.

15. Section 1:Purpose and Mission Standard 1.2 Assessing Effectiveness Evaluation Criteria Non-complianceThere is no process established to assess the achievement of the CPD program’s mission statement Partial complianceThe process to assess the achievement of the CPD program’s mission statement is in development; has not been executed this accreditation cycle’ or does not enable the identification of plans for improvement ComplianceThe process to assess the achievement of the CPD program’s mission statement has been implemented at least once during the current accreditation cycle and has enabled the organization to identify plans for improvement

16. Section 1:Purpose and Mission Standard 1.3 Scholarly Activities in Health Education An accredited CPD provider organization approaches their work in a scholarly manner by integrating evidence of educational effectiveness or innovations into the design, development and implementation of their activities. University offices of CME (or equivalent title) must participate independently or collaboratively in scholarship (sharing innovations through presentations or publications) or original research in medical education, CPD, or healthcare professional education.

17. Section 1:Purpose and Mission Standard 1.3 Scholarly Activities in Health Education: Evaluation Criteria Non-compliance The organization is not able to demonstrate how evidence of educational effectiveness or innovations has been integrated into the design, development, or implementation of their activities University offices of CME have not participated in scholarship or original research in medical education, CPD, or health care professional education Partial compliance Compliance

18. Section 1:Purpose and Mission Standard 1.3 Scholarly Activities in Health Education: Evaluation Criteria Non-compliance Partial compliance The organization is infrequently able to demonstrate how evidence of educational effectiveness or innovations has been integrated into the design, development or implementation of their activities University offices of CME have infrequently participated in scholarship or the development of original research Compliance

19. Section 1:Purpose and Mission Standard 1.3 Scholarly Activities in Health Education: Evaluation Criteria Compliance Partial compliance Compliance The organization is able to consistently demonstrated how evidence of educational effectiveness or innovations has been integrated into the design, development or implementation of their activities University offices of CME have consistently participated in scholarship or the development of original research.

20. Section 1:Purpose and Mission Standard 1.4 Educational Independence An accredited CPD provider organization implements written policies and procedures based on the (most current) CMA Guidelines for Physicians in Interactions with Industry or the Conseil québécois de développement professionnel continu des médecins Code of Ethics for parties involved in Continuing Medical Education addressing: how relationships with other organizations are managed to ensure independence of the planning process from commercial influence; the process(es) by which individuals in positions to contribute to or develop the content of learning activities disclose all real or potential conflicts of interest; and, the process(es) used to manage identified conflicts of interest.

21. Section 1:Purpose and Mission Standard 1.4 Educational Independence Evaluation Criteria Non-compliance There are no written policies and procedures that describe how the provider ensures the independence of the planning process from commercial influence including how conflicts of interest are identified, disclosed, and managed Partial compliance The written policies and procedures (as above)… do not address all required elements, are not based on established current guidelines or are not adhered to. Compliance

22. Section 1:Purpose and Mission Standard 1.4 Educational Independence Evaluation Criteria Non-compliance Partial compliance Compliance There are implemented written policies and procedures that address all required elements, are based on current established guidelines describing how the provider ensures the independence of the planning process from commercial influence, including how conflicts of interest are identified, disclosed and managed.

23. Your Feedback on Section 1 Question 1 : Are the standards included within this section relevant to the roles and responsibilities of accredited CPD provider organizations? a) Yes b) No c) Not Sure 23

24. Your Feedback on Section 1 Question 2a: Within this section, which standard requires revision to clarify its purpose or intent? a) Standard 1.1 b) Standard 1.2 c) None of the above standards d) All of the above standards 24

25. Your Feedback on Section 1 Question 2 b: Within this section, which standard requires revision to clarify its purpose or intent? a) Standard 1.3 b) Standard 1.4 c) None of the above standards d) All of the above standards 25

26. Your Feedback on Section 1 Question 3: Which of the standards included in this section would you benefit most from receiving further education or support? a) Standard 1.1 b) Standard 1.2 c) Standard 1.3 d) Standard 1.4 e) None of these standards 26

27. General Reflections What other comments, questions or concerns would you like to raise about the standards in this section?

28. Section 2: Educational Planning, Implementation and Evaluation Standard 2.1 Assessing Needs Standard 2.2 Development of Learning Objectives Standard 2.3 Educational Content Development Standard 2.4 Promoting Self-Learning and Self- Assessment Standard 2.5 Evaluation Strategies Standard 2.6 External Program Review

29. Section 2: Educational Planning, Implementation and Evaluation Standard 2.1: Assessing Needs An accredited CPD provider organization uses multiple methods to regularly assess perceived and unperceived professional practice needs of members of its target audience(s) across the CanMEDS (or CanMEDS FM) framework to establish the focus and select the format for educational initiatives.

30. Section 2: Educational Planning, Implementation and Evaluation Standard 2.1 Assessing Needs Evaluation Criteria Non-compliance The provider organization does not assess perceived and unperceived needs to establish the focus and select the format for educational initiatives Partial compliance The provider organization uses limited methods or inconsistently assesses perceived and unperceived needs; or does not assess needs across the CanMEDS framework(s); or cannot demonstrate how needs are used to establish the focus and select the format for educational initiatives. Compliance The provider organization uses multiple methods to regularly assess perceive and unperceived needs of its target audience across the CanMEDS framework(s) to establish the focus and select the format for educational initiatives.

31. Section 2: Educational Planning, Implementation and Evaluation Standard 2.2 Development of Learning Objectives An accredited CPD provider organization uses identified professional practice needs to develop learning objectives for the overall activity and individual sessions that are: provided to the learner prior to the activity; written from the learner’s perspective; and consistent with the chosen educational format.

32. Section 2: Educational Planning, Implementation and Evaluation Standard 2.2 Development of Learning Objectives : Evaluation Criteria Non-compliance The CPD provider organization cannot demonstrate how assessed needs are used to develop overall and session specific learning objectives. Partial compliance The CPD provider organization does not consistently use needs to develop overall and session specific learning objectives; or the learning objectives are not provider to the learner prior to the activity; or are not written from the learner’s perspective; or a re mot consistent with the educational format Compliance

33. Section 2: Educational Planning, Implementation and Evaluation Standard 2.2 Development of Learning Objectives : Evaluation Criteria Non-compliance Partial compliance Compliance The CPD provider organization consistently demonstrates how professional practice needs are used to create overall and session specific learning objectives that are provider to the learner prior to the activity, are written from the learner’s perspective and are consistent with the chosen educational format.

34. Section 2: Educational Planning, Implementation and Evaluation Standard 2.3 Educational Content Development An accredited CPD provider organization has a process to develop content for each learning activity which is relevant, balanced, and responsive to identified needs.

35. Section 2: Educational Planning, Implementation and Evaluation Standard 2.3 Educational Content Development: Evaluation Criteria Non-compliance There is no process established to ensure the content developed and presented at learning activities is relevant, balanced or responsive to identified needs Partial compliance The process to ensure the content developed and presented at each learning activity meets established criteria is either in development; or is inconsistently applied Compliance There is a process that consistently ensures the content developed and presented at each learning activity is relevant, balanced and responsive to identified needs.

36. Section 2: Educational Planning, Implementation and Evaluation Standard 2.4 Promoting Self-Learning & Self- Assessment An accredited CPD provider organization consistently implements multiple strategies, services or tools to assist individual members of the target audience to enhance their self-assessment, reflection, and self- learning skills. Examples include.....

37. Section 2: Educational Planning, Implementation and Evaluation Standard 2.4 Promoting Self-Learning & Self- Assessment Examples include: Developing and answering questions stimulated by their practice or from participation in group learning activities; Identifying areas for future learning based on assessments (including self-assessments) of knowledge, competence or performance; Developing a continuing professional development plan; and, Documenting outcomes from participation in learning activities.

38. Section 2: Educational Planning, Implementation and Evaluation Standard 2.4 Promoting Self-Learning and Self- Assessment: Evaluation Criteria Non-compliance There are no strategies, services or tools to assist individual members of the target audience to enhance their self-assessment, reflection and self-learning skills Partial compliance The strategies, services or tools to assist individual members of the target audience to enhance their self-assessment, reflection, and self-learning skills are either in development; are limited in scope; or are inconsistently provided Compliance The CPD provider organization has consistently implemented multiple strategies, services, and tools to assist individual members of their target audience to enhance their self- assessment, reflection and self-learning skills and competencies.

39. Section 2: Educational Planning, Implementation and Evaluation Standard 2.5 Evaluation Strategies An accredited CPD provider organization has implemented an evaluation system that measures the degree to which the learning objectives for individual activities were achieved using evidence beyond self-report at least some of the time. For example, the organization uses strategies to assess gains in knowledge, skills, attitudes or performance of participants or improvements in patient care or population health.

40. Section 2: Educational Planning, Implementation and Evaluation Standard 2.5 Evaluation Strategies Evaluation Criteria Non-compliance The CPD provider organization’s evaluation system does not measure the degree to which the learning objectives for individual activities were achieved. Partial compliance The CPD provider organization’s evaluation system to measure the degree to which the learning objectives for individual activities were achieved is either in development, is infrequently applied, or only uses evidence based on self- report Compliance The CPD provider organization’s evaluation system consistently measures the degree to which the learning objectives for individual activities were achieved using evidence beyond self-report at least some of the time.

41. Section 2: Educational Planning, Implementation and Evaluation Standard 2.6 External Program Review An accredited CPD provider organization implements written policies and procedures (including quality control measures) to ensure consistent adherence to current educational and ethical standards established by the Collège des Médecins du Québec (CMQ), the College of Family Physicians of Canada (CFPC), or the Royal College, when reviewing the educational activities developed by external organizations

42. Section 2: Educational Planning, Implementation and Evaluation Standard 2.6 External Program Review Evaluation Criteria Non-compliance There are no written policies and procedures to review the educational activities developed by external organizations Partial compliance The policies and procedures to assess adherence to educational and ethical standards of educational activities developed by external organizations do not include quality control measures or are inconsistently applied Compliance There are written policies and procedures to assess adherence to educational and ethical standards of educational activities developed by external organizations that include quality control measures and are consistently applied.

43. Your Feedback on Section 2 Question 1: Are the standards included within this section relevant to the roles and responsibilities of accredited CPD provider organizations? a) Yes b) No c) Not Sure 43

44. Your Feedback on Section 2 Question 2a: Within this section, which of the following standards requires revision to clarify its purpose or intent? a) Standard 2.1 b) Standard 2.2 c) Standard 2.3 d) None of the standards e) All of the above 44

45. Your Feedback on Section 2 Question 2b: Within this section, which of the following standards requires revision to clarify its purpose or intent? a) Standard 2.4 b) Standard 2.5 c) Standard 2.6 d) None of the standards e) All of the above 45

46. Your Feedback on Section 2 Question 3a: Within this section which of the following standards would you benefit most from receiving further education or support? a) Standard 2.1 b) Standard 2.2 c) Standard 2.3 d) None of these standards e) All of these standards 46

47. Your Feedback on Section 2 Question 3b: Within this section which of the following standards would you benefit most from receiving further education or support? a) Standard 2.4 b) Standard 2.5 c) Standard 2.6 d) None of these standards e) All of these standards 47

48. General Reflections What other comments, questions or concerns would you like to raise about the standards in this section?

49. Section 3:Sustainability Standard 3.1 Operations Standard 3.2 Professional and Legal Standards

50. Section 3:Sustainability Standard 3.1: Operations An accredited CPD provider organization implements an operational plan to support the achievement of the CPD program’s goals and strategic priorities including a: budget process that prospectively allocates sufficient financial resources. human resource recruitment and retention strategy for volunteers and/or paid staff; and plan to ensure access to appropriate physical and technical resources.

51. Section 3:Sustainability Standard 3.1: Operations Evaluation Criteria Non-compliance The CPD provider organization has not developed an operational plan to support the achievement of the CPD program’s goals and strategic priorities Partial compliance An operational plan to support the achievement of the CPD program’s goals and strategic priorities is either in development; does not reflect all requirement elements; or as been inconsistently implemented. Compliance The CPD provider organization has implemented an operational plan to support the achievement of the CPD program’s goals and strategic priorities that is comprehensive, includes all required elements and is consistently applied.

52. Section 3:Sustainability Standard 3.2: Professional and Legal Standards The accredited CPD provider organization has implemented policies and procedures to ensure its governance, operations, planning processes and maintenance and retention of records meets applicable professional and legal standards include the protection of privacy, confidentiality, copyright and contractual law regulations.

53. Section 3:Sustainability Standard 3.2: Professional and Legal Standards: Evaluation Criteria Non-compliance The CPD provider organization has no policies and procedures to ensure that its governance, operations, planning processes, and record retention meet applicable professional and legal standards. Partial compliance The policies and procedures to ensure governance, operations, planning processes and record retention meet all applicable professional and legal standards are either in development; do not meet all required elements; or are inconsistently applied. Compliance The CPD provider organization has implemented a comprehensive set of policies and procedures that demonstrate how its governance, operations, planning processes and record retention consistently meet applicable professional and legal standards for all required elements.

54. Your Feedback on Section 3 Question 1: Are the standards included within this section relevant to the roles and responsibilities of accredited CPD provider organizations? a) Yes b) No c) Not Sure 54

55. Your Feedback on Section 3 Question 2: Within this section, which standard requires revision to clarify its purpose or intent? a) Standard 3.1 b) Standard 3.2 c) None of the standards d) All of the above 55

56. Your Feedback on Section 3 Question 3: Which of the standards included in this section would you benefit most from receiving further education or support? a) Standard 3.1 b) Standard 3.2 c) None of these standards d) All of these standards 56

57. General Reflections What other comments, questions or concerns would you like to raise about the standards in this section?

58. Next Steps 1. Create a second survey of all CPD provider organizations for fall 2012 to obtain feedback on the proposed: Standards Questions Documentation requirements 2. Discuss the feedback with CACME and the Royal College CPD Accreditation Committee

59. Next Steps 3. Create examples for each standard and include a glossary of all terms used. 4. Identify further educational and organizational support strategies 5. Discuss and refine the process of accreditation

60. Next Steps 6. Develop an implementation plan including the piloting of the standards and with several volunteer CPD provider organizations during 2013-14 7. Implement the national accreditation standards by the beginning of 2015.

61. Dr Craig M. Campbell Dr Marianne Xhignesse Dr. Ted Toffelmire September 20, 2012