1. Weighting CanMEDS Roles & Competencies Marilyn Singer MD, FCFP Director, Clinician Assessment Programs Brenda Stutsky RN, PhD, Program Advisor, Manitoba Practice Assessment Program Coalition for Physician Enhancement June 3 rd Madison, Wisconsin

2. Conflict of Interest Disclosure None to disclose

3. Objectives Describe the Manitoba Practice Assessment Program (MPAP) Describe the results of the CanMEDS weighting research project Discuss implications

4. Mission/Goal of the MPAP To provide an assessment of physicians in their individual practices to assist with clarification of licensure status To offer an assessment process that is: – Comprehensive – Transparent – Tailored to the physician’s specialty and practice setting

5. Assessment Components Self-Assessment – Self-Assessment Report – Self-Assessment: Reflective Practice (Matches 360’ Questions) – Self-Assessment: Clinical Practice Skills 360 Degree Surveys – Physician Colleagues – Interprofessional Colleagues – Patients Chart Audit/Chart Stimulated Recall Observation of Clinical Practice Key Interviews Comprehensive Report

6. Framework

7. CanMEDS Weighting Project Research Questions: 1.How do physicians rate the complexity, frequency, and criticality of CanMEDS roles? 2.How do the CanMEDS roles compare with one another in terms of their relative levels of perceived importance? Image from Creative Commons http://www.flickr.com/photos/davehamster/531768924/sizes/m/in/photostream/

8. 2 Online Surveys

9. Sample Physicians on Scott’s Directories: Canadian Medical Directory – BC, AB, SK, MB, NWT, Yukon, & Nunavut – Final sample size N=88 (23 family practice, 65 specialists) - limited generalizability Image from Creative Commons http://www.flickr.com/photos/mega1210/4948863429/sizes/m/in/photostream/

10. Analysis Index of Practice Importance – Importance = Frequency x Criticality Activity Weight – Weight = Sum of Mean Importance/Number of Competencies (for each of the 7 CanMEDS roles) References Kane, M. T., Kingsbury, C., Colton, D., Estes, C. (1989). Combining data on criticality and frequency in developing test plans for licensure and certification examinations. Journal of Educational Measurement, 26(1), 17-27. Raymond, M. R. (2001). Job analysis and the specification of content for licensure and certification examinations. Applied Measurement in Education, 14(4), 369-415.

11. Results No significant difference between family practice and specialists in index of practice importance scores. Mean complexity scores: – Medical Expert4.36 – Collaborator4.04 – Communicator4.03 – Health Advocate3.99 – Manager3.96 – Professional3.85

12. CanMEDS Weighting

13. Implications Validation of the CanMEDS framework One set of tools can be used for both family physicians and specialists All roles are important when determining overall competence Curriculum development for undergrad, post- grad, and continuing medical education

14. Thank You! www.umanitoba.ca/cpd b_stutsky@umanitoba.ca singer@cc.umanitoba.ca