1. Curriculum Inventory Workshop
Terri Cameron, AAMC
Valerie Smothers, MedBiquitous
Ye Chen, University of Virginia
James Galt, Rutgers Robert Wood Johnson Medical School
Saniya Saifee, New York University School of Medicine

2. Workshop Objectives
By the end of the workshop, participants should be able to:
Describe and implement the CI Specification, Competency Framework, AAMC Business Rules, and Developer Resources
Create XML code that includes PCRS to Program Competencies Mapping
Create XML code that includes Event to Program Competency Mapping
Create XML code that includes Sequence Block and Event Data Elements
Upload a successful XML file to CI Staging

3. MedBiquitous Curriculum Inventory V1
Working Group launched in December 2010
Susan Albright and Marc Triola, co-chairs
Charter:
“to develop XML standards for the exchange of curriculum data for benchmarking and educational research.” (Not just for AAMC Curriculum Inventory)
Reasons for Undertaking Work:
Enable local curriculum management systems to upload data to a central Curriculum Inventory
Identify curricular trends
Support curriculum benchmarking and reform

11. Curriculum Inventory Expectations Events Sequence blocks
Integration blocks A E B F C G D H I J K L M E O N C G S H I J P R Q 3 1 2 1 2 3 4 5 6 7 8 9 10 11
Competency framework

12. Events A curriculum is made up of many educational events
Events have metadata, descriptive information about the events
IDs required! I A E J B F K C G L D H

13. Expectations
Curricula may reference objectives , competencies, learning outcomes, or milestones. These are called competency objects.
IDs required! 3 1 2 6 4 5 9 7 8 12 10 11

14. Competency Objects and Events
An event may be intended to address one or more competency objects (learning objectives, competencies, and milestones)
A competency object may be associated with many events G 1 2 K 6 2

15. Sequence Blocks
Events can be organized into blocks of instruction, called Sequence Blocks (think course/module/ year/phase/clerkship etc)
A recurring event may be associated with more than one Sequence Block
Sequence blocks may be associated with competency objects (competencies, objectives, etc)
Sequence blocks have metadata, too
IDs required! I A E J B F 2 1 K C A L G D H 7

16. Integration Blocks
Curricular themes, or Integration Blocks, may apply to many sequence blocks, events, or objectives/ competencies and describe how the curriculum is semantically integrated.
An event or block may link to many Integration Blocks and an Integration Block may link to many events, blocks or competency objects
IDs required! A E B F C G D H I J K L 1 M E O N C G S H I J P R Q 3 2

17. Sequence
Sequence blocks can be put together to form a the main structure of the curriculum (a sequence of courses, modules, phases, clerkships) A E B F C G D H I J K L 1 M E O N C G S H I J P R Q 3 2

18. Sequence Block NestingA E B F I J 1 2
Sequence blocks can be nested: a module or clerkship might be in a year which is within a phase and so on …
Blocks will inherit any relations to competency objects from their sub-blocks
No restriction on the number of nest levels
Recursion explicitly forbidden A E B F I J 1 2 A E B F I J 1 2 A E B F I J 1 2

19. Types of blocks: ordered
Sequence=ordered
Required=true
- All sub-blocks within an ordered block happen with a prescribed sequence
Block
Required I A E J B F
Block
Required R C J D Q Z
Block
Required T A X Y M B

20. Types of blocks: unordered
- All sub-blocks within an unordered block happen for all students, but students may encounter them in a variable order
Block
Sequence=unordered
Required=true
Block
Required A E B F I J
Block
Required C J Q Z R D
Block
Required A X M B T Y
Block
Required D K N S L C

21. Types of blocks: selectives
- Every student must participate in [specify number] of the blocks, and the order is variable
Block
Required = Required Order=Unordered
Minimum=1
Block
Optional A E B F I J
Block
Optional C J Q Z R D
Block
Optional A X M B T Y OR OR

22. Types of blocks: electives
- Students may or may not participate in any of the blocks
Sequence Block
Required = Optional
Order =Unordered
Minimum=0
Sequence Block
Optional A E B F I J
Sequence Block
Optional C J Q Z R D
Sequence Block
Optional A X M B T Y OR OR

23. Types of blocks: parallel
- All sub-blocks within a parallel block happen at the same time, but are distinct educational units
Sequence Block
Required = Required
Order = Parallel
Sequence Block
Required C J Q Z R D
Sequence Block
Required A E B F I J

24. Clerkships, required = Required, minimum = 2, maximum = 2
Level 1 Main Track, required = Optional, track = true
Level 1 Common Clerkships, required = Required
Med/Geri 1 84 days required in track
Surg 1 56 days required in track
NeuroScience 28 days optional
Elective 21 days required
Neurology 28 days required
Pediatrics 1 42 days required in track
Ambulatory 1 28 days required in track
Anesthesiology 7 days required
Psychiatry 28 days required
Level 1 Ambulatory Track, required = Optional, track = true
Option 1: This diagram shows two tracks for level 1 clerkships that show the varying lengths. The minimum maximum values of the “parent” block indicate that students must take two of the “child” blocks. Since Common Clerkships is required, that means that either the Main track or the Ambulatory track is required. We would need a new attribute called “required in track” to show that medicine, geriatrics, surgery and pediatrics are required in the track. The content for the tracks could be different (ie Surg 1 has different educational events than Surg 2).
When researchers search for all surgery clerkships, they will see two clerkships of varying length with varying content, both marked required in track. The CIP search mechanism would need to understand the nuance of required in track when allowing users to construct search queries.
QUESTION: Should we be able to say that a clerkship model is longitudinal? Is that mutually exclusive to integrated and rotation?
Ob/Gyn 42 days required
Med/Geri 2 63 days required in track
Surg 2 42 days required in track
Pediatrics 2 28 days required in track
Ambulatory 2 77 days required in track

25. Level 2 Clerkships, July 4, 2011– April 27, 2012
Level 2 Clerkships Variable time, July 4, 2011 – April 27, order = unordered
Level 2 Clerkships, second half of the year, January 1, 2012 – April 27, 2012
Subinternship required 28 days
Emergency Med required 28 days
Into to Internship required 14 days
Anatomic Radiology
Required 14 days or online course
Critical Care
Required 14 days
This shows how you would model a course that can occur anytime during the second half of the year.
Electives 147 days (21 are usually taken during level 1 and the rest during the level 2 variable time

26. Relates to other competency objects . . .
A competency object . . .
Statement
Competency Framework
Relates to other competency objects . . .
Learning Object
Assess-ment
Performance data
Which can relate to external resources . . .

27. Business Rules
Program-level competencies are required and must be mapped to PCRS using #related All competencies, including PCRS, program-level, sequence-block level, and event level competencies used in a <Relation> tag must have an <Includes> tag

28. Spec Required Elements
Curriculum Inventory ReportID Institution Program Title ReportDate ReportingStartDate ReportingEndDate Language Description
Events
Event
Title
EventDuration in hours and min
Instructionalmethod (primary=true/false) or AssessmentMethod (purpose=formative/ summative)
Expectations
CompetencyObject
identifier
title
Category *
AcademicLevels
Sequence *
Vocab ids
* - required by AAMC business Rules

29. SequenceBlock Requirements
Attributes: id, required Title Timing (either duration in days or start and end dates – clerkships must specify both) Level SequenceBlockEvent SequenceBlockReference

30. Matching Program Objectives to PCRS (Crosswalk)
2. Knowledge for Practice: Demonstrate knowledge of established and evolving biomedical, clinical, epidemiological and social-behavioral sciences, as well as the application of this knowledge to patient care.
2.1 Demonstrate an investigatory and analytic approach to clinical situations
2.2 Apply established and emerging bio-physical scientific principles fundamental to health care for patients and populations
2.3 Apply established and emerging principles of clinical sciences to diagnostic and therapeutic decision-making, clinical problem-solving, and other aspects of evidence-based health care
2.4 Apply principles of epidemiological sciences to the identification of health problems, risk factors, treatment strategies, resources, and disease prevention/health promotion efforts for patients and populations
2.5 Apply principles of social-behavioral sciences to provision of patient care, including assessment of the impact of psychosocial and cultural influences on health, disease, care-seeking, care compliance, and barriers to and attitudes toward care
2.6 Contribute to the creation, dissemination, application, and translation of new health care knowledge and practices
2.99 Other knowledge for practice
Demonstrate a broad working knowledge of the fundamental science, principles, and processes basic to the practice of medicine.
2.1 Understand the clinical relevance of scientific inquiry.
2.2 Apply this knowledge in a judicious and consistent manner to prevent common health problems and achieve effective and safe patient care.
2.3 Demonstrate the ability to evaluate emerging knowledge and research as it applies to diagnosis, treatment and the prevention of disease.
2.4 Utilize state of the art information technology and tools to retrieve, manage and use biomedical information in the care of individuals.
2.5 Utilize state of the art information technology and tools to retrieve, manage and use biomedical information in the care of populations.
2.6 Understand the indications, contraindications, and potential complications of common clinical procedures.
2.7 Perform the basic clinical procedures expected of a new PGY-1.

31. PCRS Mapping and Sequence Block or Event-level Competencies
Mapping of Sequence Block-level or Event-level objectives / competencies directly to PCRS is not allowed.
Sequence Block and / or Event Objectives / Competencies, if mapped, are mapped to each other and to Program Objectives / Competencies

32. Competency Object Rules
<Competency Object> tags must be used to identify competencies, along with their <Category> (level)
Program-level
Sequence block-level
Event-level
Competency object URIs must be unique
<Competency Object> tags must not be used to identify PCRS competencies

33. Competency Objects and the Curriculum Structure
All sequence block-level and event-level competencies must be referenced by at least one Sequence Block or Event (that was identified in the curriculum structure)
Competency Framework
Curriculum Inventory
skos: narrower
skos: related P2 S5
PCRS 6.99
PCRS 1.5 I
The Sequence Block References the Program-level Competency P2 E J B
P = Program Level Competency
S = Sequence Block Level Competency
E = Event Level Competency
The Sequence Block References the Sequence Block-level Competency F S5 A

34. Circular References
Relationships must not result in circular references.
A circular reference occurs when a child element draws a hierarchical relationship to its parent that infers the parent is lower-level than the child.
skos: narrower
skos: related P2 S1 S2
PCRS 6.99
PCRS 1.5
P = Program Level Competency
S = Sequence Block Level Competency
E = Event Level Competency
skos: narrower S3
Notes: – A c it is nonsensical because a higher level concept can’t be a child of a narrower/lower level concept – e.g., the tire can never be the parent of the car.
skos: narrower
This relationship is not allowed because it creates a “Circular Reference” and violates a Business Rule.
skos: narrower E4

35. Other notes
Set the schema location to public copy: xsi:schemaLocation="
Use namespace prefixes for competencies and competency framework
Use AAMC ids for schools
Competency Object identifiers must be URIs
URLs do not have to be resolvable. As long as they uniquely identify, it will work.  
Add anchors to the URL so that each objective can be distinguished using the URL with the anchor included.
Duration uses xsd format, for example, P40D, PT2H
1 week = 5 days

36. PCRS --Physician Competency Reference Set
“…a list of common learner expectations”
What competencies schools are incorporating into their curricula;
Where in their curricula schools are incorporating expectations and competencies;
How schools are teaching and assessing competencies; and
In what context and/or content competencies are being taught.

37. Step 1: Match Competencies
PCRS
Step 1: Match Competencies

38. Step 2: Create XML File

39. <Expectations>
<CompetencyObject> </CompetencyObject>
<CompetencyFramework>
<cf:Includes> </cf:Includes>
<cf:Relation> </cf:Relation>
</CompetencyFramework>
</Expectations>
Insert one of your program competencies

40. <CompetencyObject>
<lom:lom>
<lom:general>
<lom:identifier>
<lom:catalog>URI</lom:catalog>
<lom:entry>
</lom:identifier>
<lom:title>
<lom:string>Students will convey information honestly and tactfully; will engage in truthful interactions with patients, peers, and in professional work.
</lom:string>
</lom:title>
</lom:general>
</lom:lom>
<co:Category term="program-level-competency"/>
</CompetencyObject>

41. <Expectations> <CompetencyFramework>
<cf:Includes> </cf:Includes>
</CompetencyFramework>
</Expectations>
Insert one of your program competencies
Insert one PCRS

42. <cf:Includes>
<cf:Catalog>URI</cf:Catalog>
<cf:Entry>
</cf:Includes>
<cf:Includes>
<cf:Catalog>URI</cf:Catalog>
<cf:Entry>
</cf:Includes>

43. <Expectations>
<CompetencyFramework>
<cf:Relation> </cf:Relation>
</CompetencyFramework>
</Expectations>
Show the relationship between your program competency & PCRS

44. <cf:Relation>
<cf:Reference1>
<cf:Catalog>URI</cf:Catalog>
<cf:Entry>
</cf:Reference1>
<cf:Relationship>
<cf:Reference2>
<cf:Entry>
</cf:Reference2>
</cf:Relation>

45. Lessons Learned Some benefit to moving a bit slow
Having more than one person understanding the specifications is important
Humans are not the weakest link
Networking is vital
Carving out dedicated time is crucial
Explaining the big picture to faculty is important
Developing a functioning end product is vital
Developing tools not just a product is key for the future

46. Show Events Mapping in XML File
Ye Chen

47. XML File Structure
<CurriculumInventory > <ReportID> </ReportID><Institution></Institution> <Program> </Program><Title> </Title> <ReportDate> </ReportDate> <ReportingStartDate> </ReportingStartDate> <ReportingEndDate> </ReportingEndDate> <Language> </Language> <Description> </Description> <Events> <Event> </Event> </Events>
<Expectations> <CompetencyObject> </CompetencyObject> <CompetencyFramework> </CompetencyFramework> </Expectations>
<AcademicLevels></AcademicLevels> <Sequence> <SequenceBlock > </SequenceBlock> </Sequence> </CurriculumInventory>

48. Events and Competencies
Event (E1): White Coat Ceremony
Event (E2): Orientation Lecture
Event (E3): Peds Part 1
Event (E4): Peds Part 2
Describe the three functions
List the organization,
Gather, synthesize, and organize patient information.
Professionalism
Perform all medical, diagnostic

49. Repeat the same format for other 4 competencies
List All Competencies
<Expectations> <CompetencyObject> List competencies </CompetencyObject> <CompetencyFramework> </CompetencyFramework> </Expectations>
<CompetencyObject> <lom:lom> <lom:general> <lom:identifier> <lom:catalog>URI</lom:catalog> <lom:entry> </lom:identifier> <lom:title> <lom:string>Professionalism</lom:string> </lom:title> </lom:general> </lom:lom> <co:Category term="program-level-competency"/> </CompetencyObject>
This value will be refered in several places (event, include, and relationship)
Repeat the same format for other 4 competencies

50. Show the relationships among competencies
<Expectations> <CompetencyObject> List competencies </CompetencyObject> <CompetencyFramework>Show relationships</CompetencyFramework> </Expectations>
Include all program level competencies, include sequence block level competencies and event level competencies if they have mappings with other competencies.
other information for competencyFrameWork <lom:lom>------</lom:lom>
<cf:Includes> <cf:Catalog>URI</cf:Catalog> <cf:Entry> </cf:Includes> <cf:Includes> <cf:Catalog>URI</cf:Catalog> <cf:Entry> </cf:Includes> <cf:Includes> <cf:Catalog>URI</cf:Catalog> <cf:Entry> </cf:Includes> <cf:Includes> <cf:Catalog>URI</cf:Catalog> <cf:Entry> </cf:Includes>

51. Show the relationships among competencies (cont.)
Describe the three functions
Gather, synthesize, and organize patient information.
program level comptetencies to PCRS……..
Professionalism
<cf:Relation> <cf:Reference1> <cf:Catalog>URI</cf:Catalog> <cf:Entry> </cf:Reference1> <cf:Relationship> <cf:Reference2> <cf:Catalog>URI</cf:Catalog> <cf:Entry> </cf:Reference2> </cf:Relation> <cf:Relation> <cf:Reference1> <cf:Catalog>URI</cf:Catalog> <cf:Entry> </cf:Reference1> <cf:Relationship> <cf:Reference2> <cf:Catalog>URI</cf:Catalog> <cf:Entry> </cf:Reference2> </cf:Relation>

52. Map event to competencies
Only show the direct mappings. Non direct mapping is taken care by the relationship.
Event (E1): White Coat Ceremony
Event (E2): Orientation Lecture
Event (E3): Peds Part 1
Event (E4): Peds Part 2
Describe the three functions
List the organization,
Perform all medical, diagnostic
<Event id="E1"> <Title>White Coat Ceremony</Title> <CompetencyObjectReference>/CurriculumInventory/Expectations/CompetencyObject[lom:lom/lom:general/lom:identifier/lom:entry=' <InstructionalMethod primary="true">IM013</InstructionalMethod>
</Event>
/CurriculumInventory/Expectations/CompetencyObject[lom:lom/lom:general/lom:identifier/lom:entry= ] '
inside <Event id=“E2”><Event id=“E3”>, there should be <CompetencyObjectReference> with different entry value.

53. Event and SequenceBlock
Curriculum Inventory
Event and SequenceBlock
Saniya Saifee
NYU School of Medicine

54. Event
Event contains sub elements that describe a single education or assessment event

55. CompetencyObjectReference
Event-Subelements
Event
Title
EventDuration
Description
Keyword
CompetencyObjectReference
ResourceType ID
InstructionalMethod
AssessmentMethod
0..∞
Required
Optional

56. Event XML <InstructionalMethod primary="true"> IM013
<Events>
<Event id="E1">
<Title>White Coat Ceremony</Title>
<EventDuration>PT2H</EventDuration>
<Description>First year students are given </Description>
<Keyword hx:source="MeSH" hx:id="D010817">
<lom:string>Physician-Patient Relations</lom:string>
</Keyword>
<Interprofessional>false</Interprofessional>
<CompetencyObjectReference>/CurriculumInventory
/Expectations/CompetencyObject[lom:lom/lom:general/lom:identifier/lom:entry='
</CompetencyObjectReference>
<InstructionalMethod primary="true">
IM013
</InstructionalMethod>
</Event>

57. Competency Object
<Expectations> <CompetencyObject> <lom:lom> <lom:general> <lom:identifier> <lom:catalog>URI</lom:catalog> <lom:entry> event_competency/4 </lom:entry> </lom:identifier> <lom:title> <lom:string>Describe the three functions of the medical interview </lom:string> </lom:title> </lom:general> </lom:lom> <co:Category term="event-level-competency"/> </CompetencyObject> </Expectations>

58. Events-checklist
An event must have at least one instructional OR assessment method.
An event can only be reported if we can report timing.
An event may have multiple instructional or assessment methods, but if it’s an instructional session, only one method can be primary.
All assessment methods must be marked as formative or summative.
Within <Event> tags, the <AssessmentMethod> and <InstructionalMethod> tags must reference a unique ID from their respective vocabularies

59. SequenceBlock Events Sequence Block B A B A D D When F 1 F E E IG F H E D I A B G F H E D I
When 1
Combined 2 3
Events
Sequence Block
Expectations

60. SequenceBlock-Subelements
Title
Timing
Level
ClerkshipModel
CompetencyObjectRef… ID
Required
0..∞ ID
0..∞
SequenceBlockEvent
Required
1..∞
SequenceBlockReference
Optional

61. Timing
Timing
Date
StartDate
EndDate
Duration

62. SequenceBlockEvent required SequenceBlockEvent EventReference
StartDate
EndDate
required

63. AcademicLevels
<AcademicLevels> <LevelsInProgram>2</LevelsInProgram> <Level number="1"> <Label>Year 1</Label> <Description>Preclinical</Description> </Level> <Level number="2"> <Label>Year 2</Label> </AcademicLevels>

64. Sequence Block
<Sequence> <SequenceBlock id="e" required="Required"> <Title>Introduction to Medicine</Title> <Timing><Duration>P1M</Duration></Timing> <Level> </Level> <SequenceBlockEvent required="true"> <EventReference> </EventReference> </SequenceBlockEvent> </SequenceBlock> </Sequence>

65. Nested SequenceBlock <SequenceBlock id="c" required="Required">
<Title>Year 1</Title>
<Timing>
<Duration>P12M</Duration>
</Timing>
<Level>
</Level>
<CompetencyObjectReference>
/CurriculumInventory/Expectations/CompetencyObject
[lom:lom/lom:general/lom:identifier/lom:entry= '
</CompetencyObjectReference>
<SequenceBlockReference>
</SequenceBlockReference>
</SequenceBlock>

66. SB-checklist
A SequenceBlock (Year, course, module) can only be reported if it has at least one “valid” event.
The id attribute in the SB must be unique.
SB should not be associated with a level number that has not been defined in <AcademicLevels>
The <Timing> subelement of <SequenceBlocks> tags with a ClerkshipModel attribute must specify <Dates> and <Duration>

67. Is this a valid Event?
<Event id="E2"> <Title>Orientation Lecture</Title> <EventDuration>PT4H</EventDuration> <Description> Faculty discuss what the students are to expect in the coming year. </Description> <InstructionalMethod primary="true">Lecture </InstructionalMethod> </Event>

68. It is not a valid event
<Event id="E2"> <Title>Orientation Lecture</Title> <EventDuration>PT4H</EventDuration> <Description> Faculty discuss what the students are to expect in the coming year. </Description> <InstructionalMethod primary="true">Lecture </InstructionalMethod> </Event> IM013

69. Best Practices and project insights
Saniya Saifee
NYU School of Medicine

70. Process followed Software program that creates XML Curriculum
Management System (CuRIE)
<CurriculumInventory>
…….. ….
</<CurriculumInventory
Validate XML against XSD
CI Portal
Send XML
AAMC Business rules errors
Refactor
And
repeat

71. Lessons learned -1 <CurriculumInventory> ….
“Including ALL Academic levels, courses, events,
Expectations in your first attempt while uploading file on CI portal”
</CurriculumInventory>
“Including ONE Academic levels, courses, events,

72. Create a checklist of minimum requirements
Lesson learned -3
Create a checklist of minimum requirements

73. Lesson learned -3

74. Keep doing frequent and repeated submission
Lesson learned - 4
Keep doing frequent and repeated submission

75. CI Process Validate XML against MedBiq CI Standard Upload to CI
Successful submission results in ‘Submitted Status’
Log into School Portal
Download and Review Verification Report
Download results in ‘Review Status’
Reject or Verify Data
Rejection returns Status to ‘No Response’
Verification results in ‘Verified Status’

76. Resources
Curriculum Inventory Implementers’ Listserv:
Competency Framework Specs and Schemas:
AAMC Curriculum Inventory Developer Resources:
Physician Competency Reference Set URIs:
SKOS: Simple Knowledge Organization System
MedBiquitous:
Curriculum Inventory Working Group:
Competencies Working Group:

77. Questions?