1. University of Alberta Pre-survey Visit March 16, 2017
Program Directors
University of Alberta
Pre-survey Visit
March 16, 2017

2. Objectives of the Meeting
To review the:
Accreditation Process
Categories of Accreditation
Standards of Accreditation
Role of the:
Program directors
Department heads
Residents
Program administrators

3. Accreditation Is a process to: Based on standards
Improve the quality of postgraduate medical education
Provide a means of objective evaluation of residency programs for the purpose of Royal College accreditation
Assist program directors in reviewing their own program
Based on standards

4. Principles of the Accreditation Process
Based on General and Specific Standards
Based on CanMEDS
Onsite regular surveys
Peer-review
Input from specialists
Categories of Accreditation

5. Pre-Survey Process University Royal College Specialty Committee
Questionnaires
Specialty Committee
Questionnaires
Royal College
Comments
Questionnaires and
Comments
Comments
Program Director
Surveyor

6. Role of the Specialty Committee
Prescribe requirements for specialty education
Program standards
Objectives of training
Specialty training requirements
Examination processes
FITER
Evaluates program resources, structure and content for each accreditation review
Recommends a category of accreditation to the Accreditation Committee

7. Composition of a Specialty Committee
Voting Members (chair + 5)
Canada-wide representation
Non-voting Members
Chairs of exam committee
National Specialty Society (NSS)
ALL program directors

8. The Onsite Survey Team Chair – Dr. Glen Bandiera
Deputy Chair – Dr. Alan Chaput
Surveyors
will be a specialist from another discipline
one surveyor per program
Resident representatives (RDoC)
one to four representatives (depends on the number of programs)
accompany surveyors on selected program reviews
Regulatory authorities representative (FMRAC)
Teaching hospital representative (HealthCareCAN)
Other observers
DENIS

9. Resident Survey
As part of the accreditation process the resident associations send an anonymous, confidential survey to all residents in all programs four to six months prior to the onsite survey
Results collated, reports are written by the provincial organization president or designate which are sent to RDoC/FMRQ president to review
Final Royal College accreditation report given ONLY to resident representatives on the survey team

10. Role of the Surveyor
Evaluate how the program is meeting standards at the time of survey
Looking for ‘evidence’

11. Information Given to Surveyors
Questionnaires and appendices
Completed by program
Program-specific standards (OTR/STR/SSA)
Report of last regular survey
Plus report of mandated Royal College review since last regular survey, if applicable
Specialty Committee comments
Also sent to PGD / PD prior to visit
Exam results for last six years

12. The Survey Schedule Includes: Document review (45 min) Meetings with:
Program director (75 min)
Department head (30 min)
Residents - per groups of 20 (60 min)
Teaching faculty (60 min)
Residency Program Committee (60 min)
DENIS
There is some flexibility – the meetings highlighted in grey can be moved around but other meetings needs to follow given sequence

13. The Survey Schedule Document review (45 min)
Residency Program Committee minutes
Resident assessment files
Picked at random from each year
Resident in trouble/remediation

14. Residency Program Committee – Minutes (B1)
Does Committee fulfill all of its responsibilities?
Provides information on issues discussed and follow-up action taken by RPC
Appropriate representation on Committee
Site coordinator(s)
Resident(s) - elected
Meets regularly, at least quarterly

15. Resident Assessments (B6)
Looking at the process of how residents are assessed
e.g. timely, face-to-face meetings
* Confidentiality form signed by the surveyor.

16. Meeting with Program Director (75 minutes)
Description of program
Review of strengths and weaknesses
Response to previous weaknesses
Last Royal College review
Resources and support for program director
Clarification of questionnaire
Review of Specialty Committee comments
Sent by prior to visit
Review of standards

17. Meeting with Department Head (30 minutes)
Overview of strengths and weaknesses
Time and support for program director
Faculty assessments
Resources
To support residency program
Research environment
Overall, priority of residency program

18. Meeting with ALL Residents (1 hour / 20 residents)
Group(s) of 20 residents
Looking for balance of strengths and weaknesses
Focus is on standards
Evaluate the learning environment

19. Topics to discuss with residents
Objectives
Educational experiences
Service /education balance
Increasing professional responsibility
Academic program / protected time
Supervision
Assessments of resident performance
Evaluation of program / assessment of faculty
Career counseling
Educational environment
Safety

20. Meeting with Teaching Faculty (1 hour)
Involvement with program
Goals and objectives
Resources
Assessments
of residents
of faculty performance
Is teaching valued?

21. Meeting with Residency Program Committee (1 hour)
Program director attends first half of meeting
All members of RPC attend, including resident members
Review of responsibilities of committee
Opportunity for surveyor to provide feedback on information obtained during previous meetings

22. The Recommendation Survey team discussion Feedback to program director
Typically evening following review
Feedback to program director
Exit meeting with surveyor
Typically the morning after review
Survey team recommendation
Category of accreditation
Strengths and weaknesses

23. Categories of Accreditation
Approved by the Royal College, CFPC and CMQ
Accredited program
Follow-up:
Next regular survey
Progress report (* a category determined only by the Accreditation Committee)
Internal review
External review
Accredited program on notice of intent to withdraw accreditation

24. Categories of Accreditation Definitions
Accredited program with follow-up at next regular survey
Program demonstrates acceptable compliance with standards

25. Categories of Accreditation Definitions
Accredited program with follow-up by College- mandated internal review
Major issues identified in more than one standard
Internal review of program required and conducted by University
Internal review report due within 24 months

26. Categories of Accreditation Definitions
Accredited program with follow-up by external review
Major issues identified in more than one standard AND concerns -
are specialty-specific and best evaluated by a reviewer from the discipline, OR
have been persistent, OR
are strongly influenced by non-educational issues and can best be evaluated by a reviewer from outside the University
External review conducted by two to three people within 24 months
Same format as regular survey

27. Categories of Accreditation Definitions
Accredited program on notice of intent to withdraw accreditation
Major and/or continuing non-compliance with one or more standards which calls into question the educational environment and/or integrity of the program
External review conducted by three people
(two specialists + one resident) within 24 months
At the time of the review, the program will be required to show why accreditation should not be withdrawn

28. RESIDENCY accreditation committee
After the Survey
survey team
Reports
specialty committee
royal college
university
Report and
Response
Reports
Responses
Recommendation
Reports and Responses
RESIDENCY accreditation committee

29. Residency Accreditation Committee
Voting members (24):
Chair and Vice Chair of the Residency AC
Chair of the International residency Program Review (1)
Association of Faculties of Medicine of Canada (2)
Collège des médecins du Québec (1)
Resident Associations (2)
Regulatory Authorities (1)
Fellows (17 including Chair and Vice Chair)
Non-voting / observers (10):
HealthCareCAN
Accreditation Council for Graduate Medical Education
Association of Faculties of Medicine of Canada
Canadian Resident Matching Service
College of Family Physicians of Canada
Collège des médecins du Québec
Committee on the Accreditation of Canadian Medical Schools
Resident Associations

30. Information Available to the Residency Accreditation Committee
All pre-survey documentation available to the surveyor
Survey report
Program response
Specialty Committee recommendation
History of the program

31. Residency Accreditation Committee
Decisions
Residency Accreditation Committee meeting
June 2018
Dean and postgraduate dean attend
Sent to
University
Specialty Committee
Appeal process is available (within 60 days)

32. Preparing for the Survey Role of the Program Director
Pre-Survey
Complete questionnaires
Inform teaching faculty
Meet with residents
Organize documents (in collaboration with P.A.)
Develop schedule (in collaboration with P.A.)
Tours of wards/clinics not necessary
Coordinate transportation arrangements (in collaboration with P.A.)

33. Preparing for the Survey Role of the Program Director
During Survey
Logistics (in collaboration with P.A.)
Schedule
Remind participants
Ensure ALL residents meet with surveyor
Off-site residents by tele/video-conferencing
Transportation
Exit meeting with surveyor
Time and location to be confirmed

34. Questions?

35. General Standards of Accreditation
“A” Standards
Apply to University, specifically the PGME office
“B” Standards
Apply to EACH residency program
“C” Standards
Apply to Areas of Focused (AFC) diploma programs

36. “B” Standards
B1 Administrative Structure B2 Goals and Objectives B3 Structure and Organization of Program B4 Resources B5 Clinical, Academic and Scholarly Content of Program B6 Assessment of Resident Performance
DENIS

37. B1 – Administrative Structure
B1.1 Program Director
Overall responsibility for program
Acceptable qualifications
Royal College certification *NEW*
Sufficient time and support
Generally interpreted as non-clinical time for program administration
Impact to operation of program

38. B1 – Administrative Structure
B1.2 Residency Program Committee (RPC)
Representative from each site and major component
Resident member of Committee
One elected by the residents
Accountability
Meets regularly, four times/year; keeps minutes
Communicates regularly with members of program, department, residents
Members responsible for bilateral flow of communication with the specific constituencies they represent

39. B1 – Administrative Structure
B1.3 Responsibilities of RPC
Planning and operation of program
Selection of residents
Assessment and promotion of residents
Appeal mechanism
Career planning and counseling
Manage stress

40. B1 – Administrative Structure
B1.3 Responsibilities of RPC
Ongoing review of program
Clinical and academic components
Resources and facilities
Teachers
Feedback mechanisms
Learning environment
Anonymous
Resident safety
Written policy

41. B1 – Administrative Structure
B1.4 Site Coordinator B1.5 Research coordinator B1.6 Environment of inquiry and scholarship

42. B1 – Administrative Structure “Pitfalls”
Residency Program Committee dysfunctional
Unclear Terms of Reference (membership, tasks and responsibilities)
Agenda and minutes poorly structured
Poor attendance
Department head unduly influential
RPC is conducted as part of a Dept/Div meeting
No resident voice

43. B2 – Goals and Objectives
B2.1 Overall statement
B2.2 Structured to reflect the
CanMEDS competency
Used in planning and assessment
of residents
B2.3 Rotation-specific
Used in planning and assessment of residents
B2.4 Resident and staff have copies
Used in teaching, learning, assessment
Learning strategies developed at start of rotation
B2.5 Regular review
At least every two years

44. B2 – Goals and Objectives “Pitfalls”
Missing CanMEDS Roles in overall structure
Okay to have rotations in which all CanMEDS Roles may not apply (research, certain electives)
Goals and objectives not used by faculty/residents
Goals and objectives dysfunctional – does not inform assessment
Goals and objectives not reviewed regularly

45. B3 – Structure and Organization
B3.1 Program provides all components of training as outlined in the specialty-specific documents
Objectives of training (OTR)
Specialty training requirements (STR)
Specific standards of accreditation (SSA)
B3.2 Appropriate supervision
According to level of training, ability/competence and experience
B3.3 Increasing professional responsibility
B3.4 Senior resident role
B3.5 Balance of service and education
Ability to follow academic sessions

46. B3 – Structure and Organization
B3.6 Equivalent opportunity for each resident
B3.7 Opportunity for electives
B3.8 Role of each site clearly defined
B3.9 Safe learning/educational environment
Free from intimidation, harassment or abuse
Promotes resident safety
B3.10 Collaboration with other programs whose residents need to develop expertise in the specialty

47. B3 – Structure and Organization “Pitfalls”
Graded responsibility absent
Service/education imbalance
Service provision by residents should have a defined educational component including evaluation
Educational environment poor

48. B4 – Resources B4.1 Sufficient teaching staff
From appropriate health professions
B4.2 Appropriate number and variety of patients, specimens and procedures
Refer to specialty-specific documents
B4.3 Clinical services organized to achieve educational objectives
Training in collaboration with other disciplines
Inpatient, emergency, ambulatory, community
Age, gender, culture, ethnicity

49. B4 - Resources B4.4 Adequate educational resources
Access to computers, on-line references
Close proximity to patient care areas
B4.5 Access to physical and technical resources in the setting where they are working
Direct observation/privacy for confidential discussions
Adequate space for residents
B4.6 Adequate supporting facilities
ICU, diagnostic, laboratory

50. B4 – Resources “Pitfalls”
Insufficient faculty for teaching/supervision
Insufficient clinical/technical resources
Infrastructure inadequate

51. B5 – Clinical, Academic and Scholarly Content of Program
B5.1 to B5.7
Academic program
Organized curriculum
Organized teaching in basic and clinical sciences
"Evidence" of teaching each of the CanMEDS Roles

52. B5 – Clinical, Academic and Scholarly Content of Program “Pitfalls”
Organized academic curriculum lacking or entirely resident driven
Poor attendance by residents and faculty
Teaching of essential CanMEDS Roles missing
Role modeling is the only teaching modality

53. B6 – Assessment of Resident Performance
B6.1 Based on Goals and Objectives
Identified methods of assessment
Level of performance expected
B6.2 Assessment consistent with characteristic being assessed
Based on specialty-specific requirements
B6.3 Timely, regular, documented feedback
Face-to-face meetings
B6.4 Residents informed of serious concerns

54. B6 – Assessment of Resident Performance “Pitfalls”
Mechanism to monitor, promote, remediate residents lacking
Formative feedback not provided and/or documented
Assessments not timely, not face to face
Summative assessment (ITER) inconsistent with formative feedback, unclearly documents concerns/ challenges

55. Conjoint Residency Education Accreditation System Reform

56. The Canadian Residency Accreditation Consortium (CanRAC)
Collaborative initiative between the Royal College, CFPC, and CMQ
New system of residency accreditation:
21st century best practices in accreditation;
Digitized; and,
Aligned with the shift towards competency-based medical education (CBME). 57
11/13/2018

57. 58

58. Shadow Surveyors Objective Role
To leverage the knowledge and expertise of experienced surveyors acting as “shadow surveyors” at an onsite survey and collect the necessary feedback on the new standards and process in accordance to the accreditation reform objectives
Role
Evaluate and validate new standards and process during an onsite survey visit
An “overlay” to the visit for selected programs or the institution to test the process with no impact to accreditation status / recommendations or decisions 59

59. Shadow Surveyor Code of Conduct
The shadow surveyor:
will act strictly as an observer during the onsite visit
will not engage in the discussions at various meetings, will not ask questions or seek clarification
will not influence in any way the recommendation of the surveyor and the survey team during the evening discussions
will not have voting privileges on the program‘s accreditation status recommendation 60

60. University of Alberta Onsite Survey
November 26 to December 1, 2017