1. Terri Cameron, MA Director, Curriculum Programs AAMC May 23, 2016
Documenting Clerkships and Electives
Terri Cameron, MA Director, Curriculum Programs AAMC
May 23, 2016

2. Objectives
Discuss the types of data that should be entered for Clerkships and Electives
Identify sources for gathering Clerkship and Electives data
Explain options for entering one iteration of a clerkship
Describe how nested sequence blocks might be used in documentation of Clerkships and Electives
Describe options for entering ‘Colleges,’ ‘Academies,’ ‘Bootcamp,’ ‘Sub-Internships,’ and other student/content groupings options
Review examples of how schools are documenting Clerkships and Electives
Terri

3. Quick Review Sequence Block Metadata Title Academic Level
Timing: Dates / Duration
Expectations
Events
Duration
Instructional Methods
Assessment Methods
Resources
Nested Sequence Blocks
Integration Blocks
Valerie

4. Documenting clerkships
Terri

5. Clerkship ‘Iteration’
Should represent core activities that majority of students are exposed to
Select one rotation for upload
A specific rotation
First, Last
Create an upload version of a rotation
Upload all iterations – reporting issues
Select one student’s schedule
May be rotation or integrated
May cross Academic Level
Create Sequence Block in each Academic Level
Terri

6. Clerkships What to Document Clerkship (Sequence Block) name
All activities
Didactics
Conferences (M&M, Tumor Board, X-Ray, etc.)
Rounds (Ward, Attending, etc.)
In-patient and Out-patient Activities
Assessment(s) (for each event and/or for Clerkship)
How to document
Clerkships are Sequence Blocks that are tagged as either integrated or rotation
What if the clerkship is both integrated and a rotation?
Choose integrated: Sequence Block dates compared to duration shows that clerkship is both integrated and a rotation
Terri

7. Clerkship ‘Model’ Example
Document ‘scheduled’ activities (e.g, 10-week clerkship)
Didactics (Individual Events; 10 hours)
Conferences (7 am, Noon, M&M, X-ray, Tumor Board, etc. as Individual Events or total, e.g,. 10 hours each; Total 60 hours)
Grand Rounds (Individual Events or Total; 20 hours)
Ward Rounds (Resident Rounds, Attending Rounds, Daily Rounds; Individual Events or Total; 50 hours)
Total documented hours (140 hours)
Total hours for clerkship (500 hours)
Divide 360 hours using ambulatory/in-patient ratio
70/30 Ratio = 252 hours ambulatory; 108 hours in-patient documented as 252 hours Clinical Experience, Outpatient; 108 hours Clinical Experience, Inpatient

8. Clerkship Model Metadata
Didactics (Slides are often included in Syllabus; first slide often includes Expectations/Objectives)
Conferences (7 am, Noon, M&M, X-ray, Tumor Board, etc. – Expectations usually included in Syllabus)
Grand Rounds (Often advertised as CME with Expectations listed OR pull from slides – good use of Nested Sequence Block)
Ward Rounds (Resident Rounds, Attending Rounds, Daily Rounds – Expectations usually included in Syllabus)
Clinical Experience Outpatient / Inpatient (use Student Clinical Experience Requirement Documentation from Syllabus; e.g., abdominal pain, headache, weakness)

9. Single Student Example
All activities pulled from Evaluation Module or Student Schedule
Evaluation Module should include Expectations
Clinical Experience Outpatient / Inpatient (use Student Clinical Experience Requirement Documentation from Logbook; e.g., abdominal pain, headache, weakness)

10. Sequence Block NestingA E B F I J 1 2
Sequence blocks can be nested
No restriction on the number of nest levels
Can represent a track or options where only some of the clerkships or courses are required
Sequence blocks may be:
Required, Optional, Required In Track
Ordered, Unordered, Parallel
You may require a minimum or maximum # of nested sequence blocks (selectives) A E B F I J 1 2 A E B F I J 1 2
Valerie A E B F I J 1 2

11. Clerkships
You have 9 Academic Level 2 clerkships; 5 are required. 8 must be taken. They may occur in any order.
How would we represent that?
Clerkships, required = Required, order = unordered, minimum = 8, maximum = 8
Medicine, required = Required
Duration – 60 days
Surgery, required = Required
Duration – 40 days
Pediatrics, required = Required
Duration – 30 days
OB/Gyn, required = Required
Duration – 30 days
Valerie
Neurology, required = Required
Duration – 20 days
Psychiatry, required = Optional
Duration – 20 days
Ambulatory, required = Optional
Duration – 20 days
Anesthesiology, required = Optional
Duration – 20 days
Emergency medicine, required = Optional
Duration – 20 days

12. Documenting integrated and longitudinal curricula
Terri

13. Integrated / Longitudinal Clerkships
The Consortium of Longitudinal Integrated Clerkships (CLIC) states that longitudinal integrated clerkships (LICs) have the following common core elements:
Medical students participate in the comprehensive care of patients over time
Medical students have continuing learning relationships with these patients’ clinicians
Medical students meet, through these experiences, the majority of the year's core clinical competencies across multiple disciplines simultaneously.
Terri

14. Integrated / Longitudinal Clerkships
Multiple options for documenting:
Show as single Sequence Block
Clerkship Model: Integrated
Nested Sequence Block(s)
Data Requirements/Sources:
Same as for discipline-specific clerkships
Issues:
All / most students participate?
Separate assessments?
Terri

15. Documenting Electives
Terri

16. What data should be entered?
Best case scenario: Elective Course (Sequence Block) documentation should include:
All Elective Courses:
Name
Instructional Method(s)
Assessment Method(s)
Objectives (Expectations)
Terri

17. Electives What to Document Electives Program Requirements
Use Timing_Dates to show entire time period when electives are offered
Use Expectations to describe expectations of the Overall Electives Program
Elective Courses
Use Timing_Duration to show number of days for each Elective (5 days = 1 Week)
Use Expectations to describe expectations of the Elective Course
Events
Each Elective Course must have at least one event (Instructional Methods, Assessment Methods, Resources, Expectations, etc.)
Terri

18. Electives Options for Uploading Best Practice Other Options
Electives Program Requirements as Sequence Block
Elective Courses as Nested Sequence Blocks
Other Options
Elective Courses as individual Sequence Blocks
Electives Program Requirements Only
Use Timing_Dates to show Start and End Dates for entire time period when electives are offered
Use Expectations to describe expectations of the Electives Program
Use Description to describe specific requirements (Sub-I, x number of weeks per specialty, etc.)
Terri

19. Electives Data Sources Curriculum Management Systems
Scheduled activities or ‘Upload Version’
Online Electives Catalogs
Duration – if multiple options are available, choose the most common
Evaluation Systems
Entering data
Conversation between curriculum office and vendor / developer to choose best option
Enter as ‘Representative Data’
Pull one iteration of each scheduled Elective
Other ideas?
Terri

20. Selectives What to Document
Selectives Course (Sequence Block) Requirements
Timing_Dates of the Selective Requirement
Expectations of the Selective Requirement
Selective Courses (Nested Sequence Blocks)
Timing_Duration to show number of days for Selective Courses (5 Days = 1 Week)
Expectations of the Selective Courses
Events
Each Sequence Block must have at least one event (Instructional Methods, Assessment Methods, Resources)
Valerie

21. Selectives Options for Uploading (Best Practice)
Selectives Course Requirements as Sequence Block
Use Timing_Dates to show entire time period when Selective Requirement occurs
Use Expectations to describe expectations of the overall Selectives Course
Selective Courses as Nested Sequence Blocks
Use Timing_Duration to show number of days for each Selective Course (5 days = 1 week)
Use Expectations to describe expectations of the specific Selective Course
Events
Each Sequence Block must have at least one Event (Instructional Methods, Assessment Methods, Resources)
Selectives

22. Selectives You have 8 fourth year sub internships; 1 is required.
How would we represent that?
Sub Internships, required = Required, order = parallel, minimum = 1, maximum = 1
Medicine Sub-I, required = Optional
Duration – 20 days
Surgery Sub-I, required = Optional
Duration – 20 days
Pediatrics Sub-I, required = Optional
Duration – 20 days
OB/Gyn Sub-I, required = Optional
Duration – 20 days
After previous
Emergency Medicine Sub-I, required = Optional
Duration – 20 days
Family Medicine Sub-I, required = Optional
Duration – 20 days
Psychiatry Sub-I, required = Optional
Duration – 20 days
Anesthesiology Sub-I, required = Optional
Duration – 20 days

23. Colleges,’ ‘Academies,’ ‘Bootcamps’
What to document
Sequence Block for Requirement
Timing_Dates (Entire time period)
Expectations
Nested Sequence Blocks for each Option
Timing_Duration (5 Days = 1 Week)
Expectations?
Nested Sequence Block for each ‘Course’ in the Option
Events
Instructional Methods
Assessment Methods
Resources
Valerie

24. Colleges,’ ‘Academies,’ ‘Bootcamps’
Sequence block, required=Required, order=Parallel, minimum=1, maximum=1
College 1
Required=Optional
College 2
Required=Optional
College 3
Required=Optional
College 4
Required=Optional

25. Typical hrs/wk formal instruct**
Multiple Sites
What to Document
One curriculum per institution
LCME: “Show sum of lectures, conferences, and teaching rounds; range of hours if there is significant variation across sites”
*Include the number of sites used for inpatient teaching and the number of sites used for outpatient teaching in the clerkship in the following format: # inpatient / # outpatient **Sum of lectures, conferences, and teaching rounds; show the range of hours if there is significant variation across sites
Clerkship
Total wks
% Amb.
# Sites used*
Typical hrs/wk formal instruct**
Clinical encounter
criteria†
(Y/N)
Patient log
Terri

26. Multiple Sites Options for Uploading
One iteration of each Sequence Block
Representative curriculum that most students experience
Issues?
Terri

27. Tracks What to Document One curriculum per institution
Options for Uploading
Nested Sequence Blocks
Sequence Block for each Academic Level
Parallel Nested Sequence Blocks for each Track
Additional Nested Sequence Blocks to enter individual courses for each Track
Representative curriculum that most students experience
Issues?
Terri

28. Questions?
Terri

29. Examples from medical schools
Our thanks to the following schools for sharing their best practices:
Susan Masters, UCSF
Robin Rivest, Oakland
Cindy Lybrand, ETSU
Matthew Gerber, UCF
Please share your ideas by sending 1-2 slides to: We’ll create a resources section on

30. UCSF Documentation of the Clinical Core & Advanced Studies (Susan Masters, Curriculum Dean)
We document the core curriculum that all students take in the third and fourth years (Clinical Core and Advanced Studies, respectively) at a minimum at the level of clerkship objectives that are linked to graduation competencies.
We layer longitudinal classroom and clinical experiences for all, and longitudinal clerkship programs that are for some students on top of the eight core clerkship objectives that all students complete either in block clerkships or in longitudinal programs. We include the one classroom-based required fourth-year course. We DO NOT document third- and fourth- year electives.

31. UCSF Documentation Map
Clinical Core – Third Year
Intersessions (Classroom Based, required for all)
Longitudinal Clinical Experience (LCE, required for all)
PISCES (Year-Long LIC, subset of students)
KLIC (Year-Long LIC, subset of students)
LIFE (Six-Month LIC, subset of students)
Model SFGH (Six-Month LIC, subset of students)
VALOR (Six-Month Clerkship Program, subset of students)
Anesthesia
FCM
Medicine
Neuro
Peds
Psych
Ob/Gyn
Surgery
Advanced Studies – Fourth Year
Coda (Classroom Based, required for all)

32. UCSF Mapping Linkage Strategy
Graduation competencies are mapped to course/clerkship objectives. Only one graduation competency can be linked to a course/clerkship objective. Session objectives are linked to course/clerkship objectives. Session objectives can be linked to multiple course/clerkship objectives.
All mapping elements can be tagged with UCSF Subjects (as a defined list) and with MeSH terms.

33. UCSF Mapping Flow Chart
Graduation Competency
Course/Clerkship Objective
Session Objective
Course/Clerkship Objective
Graduation Competency
Session Objective
Course/Clerkship Objective
Session Objective
Session Objective
Tags: Topics, MeSH Terms

34. UCSF Subjects
Sixty-nine subjects derived from (1) a review of LCME standards; (2) the AAMC Graduation Questionnaire; and
(3) USMLE reporting formats.
Acute Care
Clinical Reasoning
Embryology
Global Health Issues
Immunology
Pain Management
Professional Development
Ambulatory Care
Clinical Skills
Emergency Medicine
Health Care Financing
Internal Medicine
Palliative Care
Psychiatry
Anatomy
Community Health
End-of-Life Care
Health Care Quality Improvement
Jurisprudence, Medical
Pathology
Public Health
Anesthesiology
Complementary/Alternative Health
Epidemiology
Health Care Systems
Microbiology
Pediatrics
Radiology
Behavioral Science
Continuing Care
Ethics, Medicine
Health Informatics
Neurology
Pharmacology
Societal Problems
Biochemistry
Critical Care
Evidence-Based Medicine
Histology
Neuroscience
Physical Diagnosis
Socioeconomics, Medical
Biostatistics
Cultural Competence
Family Medicine
Human Development/Life Cycle
Nutrition
Physiology
Substance Abuse
Cell Biology
Determinants of Health
Gender and Cultural Bias
Human Sexual/Gender Development
Obstetrics and Gynecology
Population-Based Medicine
Surgery
Chronic Care
Diagnosis of Disease
Genetics
Human Sexuality/Sexual Function
Occupational Health/Medicine
Prevention/Health Maintenance
Surgical Specialties
Clinical and Translational Research
Domestic Violence/Abuse
Geriatrics
Humanities, Medical
Organ System Pathophysiology
Problem Solving

35. OUWB Clerkships – What Data
Academic Level 3
9 Clerkships
0 Integrated; all Rotations; Rotations weeks vary
1 Selective Course
Academic Level 4
5 Clerkships
4 Sub-Internships
160+ Electives – not going into CI
1 Rotation for each (representative)
Not including Electives at this time

36. OUWB – Clerkships – What Data
Title, Dates, Timing, Academic Level, Expectations, & Events
Expectations coded as K, S, A and mapped to Institutional Objectives and to PCRS
Events – 2 ways
On Student Calendar - Didactics, Conferences, Lectures
Bulk Events (to show hours/occurrence, not on calendar) – Rounds, Inpatient/Ambulatory
Includes: Instructional and Assessment Methods, Keywords, Objectives w/ Mapping, & Resources
Curriculum Keyword List – 350+ terms by committee, includes Hot Topics

37. OUWB Clerkships – Data Sources
Syllabi
Moodle Page (LMS)
Student Calendar/Schedule
Clerkship Coordinators
Clerkship Directors

38. ETSU Quillen College of Medicine
Data Used to Document Clerkships
M3 level = eight (six week) clerkships [core + community + specialties]
Objectives mapped to Institutional and PCRS
Event Title
Event Duration
Keywords (baseline from faculty (+ quality improvement is ongoing see next slide)
Instructional Method(s)
Assessment Method(s)
Documented in Curriculum Management System
Using a model within the New Innovation system (Generalist track / core)
Subset of students complete an equivalent within Rural Primary Care Track (two rural sites)

39. ETSU Quillen College of Medicine
Mapping expectations and keywords to clerkship events
Running dual systems as part of migration efforts
Combination of multiple sources (faculty input, syllabi)
Documenting multiple iterations
Use of model for core requirements
Multiple sites have same / common objectives to achieve
Keyword Lists:
Institutional Education Objectives [PCRS]
Institutional Threads / Cross-cutting Themes [can be case threads]
National Standards and/or models for curricular content [Specialty / Field / Core Content]
Reporting Topics
LCME (Elements + DCI + Questionnaires)
AAMC Reporting
USMLE Content Outline / Categories [two / three levels]
Monitoring curriculum committee changes or reviews related to topic / thread
+ EPAs (post BETA)

40. UCF College of Medicine Clerkships and Electives
We are documenting, and submitting to the CI:
Clerkship/Elective Requirements
Clerkship Objectives
Hours of Instruction in Clerkships and Electives
Most data are pulled from official documentation
We map clerkship-level objectives by pulling them from syllabi
No attempt to map objectives at the event level
We don’t have reliable access to the objectives for most of the electives
No attempt to represent individual clerkship iterations as such
Nested sequence blocks used to describe (most of) the orders and combinations in which students can take clerkships and electives
These sequence blocks are the only M3/M4 periods given dates – clerkships, electives and their events are not tracked on the real-world calendar
Each clerkship is given the expected number of hours of each type of instruction and assessment by “fiat”; for electives this is notional
Keyword lists are taken from MedBiquitous or from LCME standards

41. Questions?
Please use Q & A option to submit questions.

42. 2015-2016 Schedule Staging available now School Portal Opens: August 1
School Portal Closes: September 30
Status sent to CI Primary Admin, Curriculum Dean, and School Dean: September 30
Data clean-up/reporting starts: November 1

43. 2015-2016 Initiatives Gateway Exams Release School Name
CEPAER Documentation

44. Best Practice for ‘Gateway Exams’
Major Exams for Promotion or Graduation
Often multi-day exams
Not related to a particular course or clerkship
Often a series of unqueued cases
Examples: Senior Clinical Competency Exam; CCX; Post-Clerkship Exam

45. Best Practice for ‘Gateway Exams’
Documenting Gateway Exams:
Separate Sequence Block
Duration expected for average student
Include all Assessment Methods as Single Event (e.g., All Cases) or Split into Multiple Events (e.g., Case 1, Case 2.)
No Instructional Methods
Include Keywords for Content if possible
Reporting is post-exam and aggregate
If exam is split into multiple Academic Levels, reference the AL where most of the exam occurs
Use Sequence Block Description to provide innovations, information such as year exam was implemented, etc.

46. Using Program Description to Designate Approval of School Name linked to CI Reports
A Brief History
CurrMIT allowed schools to run reports to find schools offering particular content or using particular instructional or assessment methods or resources
Received both positively and negatively
To expedite implementation of CI and ensure highest participation rate, schools were told all reports would be aggregate
Received both positively and negatively 
Many schools, as well as medical education researchers and media, want to be able to contact schools to learn more about what and how they are teaching certain content. From a school standpoint, including school names creates a collegial environment where schools can learn from each other. Our current process for meeting this need is a very manual process that includes contacting schools and asking them to contact the school, researcher, or media.

47. Using Program Description to Designate Approval of School Name linked to CI Reports
Solution:
Allow schools to choose whether they wish to have their name available for benchmarking and research
School name will be in a separate data file – no link to actual data in report
Schools Included in This Report link will include:
School Name, Curriculum Dean Contact Information, Primary CI Contact Information

48. Using Program Description to Designate Approval of School Name linked to CI Reports
Implementation August 1, 2016, for Data:
Use Program Description to include the text:
ReleaseName=Y OR
ReleaseName=N
No ReleaseName text will be the same as ReleaseName=N
This should not require changes to programs – simply adding text to a field
For systems that do not have this option, an can be sent to
will need to be sent each year.

49. Best Practices for CEPAER Documentation
CEPAER: Core EPAs for Entering Residency
13 EPAs
Linked to PCRS Competencies
In Pilot Phase (10 schools)
Not a requirement, but being documented at many schools
Document as CEPAER-01 – CEPAER-13
Detailed information (included PCRS mapping) can be found at:

50. Questions?
List of upcoming Curriculum Inventory Webinars and materials from today’s session: