Presentation is loading. Please wait.

Presentation is loading. Please wait.

Curriculum Inventory Workshop

Similar presentations

Presentation on theme: "Curriculum Inventory Workshop"— Presentation transcript:

1 Curriculum Inventory Workshop
Terri Cameron, AAMC Valerie Smothers, MedBiquitous Ye Chen, University of Virginia James Galt, Rutgers Robert Wood Johnson Medical School Saniya Saifee, New York University School of Medicine

2 Workshop Objectives By the end of the workshop, participants should be able to: Describe and implement the CI Specification, Competency Framework, AAMC Business Rules, and Developer Resources Create XML code that includes PCRS to Program Competencies Mapping Create XML code that includes Event to Program Competency Mapping Create XML code that includes Sequence Block and Event Data Elements Upload a successful XML file to CI Staging

3 MedBiquitous Curriculum Inventory V1
Working Group launched in December 2010 Susan Albright and Marc Triola, co-chairs Charter: “to develop XML standards for the exchange of curriculum data for benchmarking and educational research.” (Not just for AAMC Curriculum Inventory) Reasons for Undertaking Work: Enable local curriculum management systems to upload data to a central Curriculum Inventory Identify curricular trends Support curriculum benchmarking and reform








11 Curriculum Inventory Expectations Events Sequence blocks
Integration blocks A E B F C G D H I J K L M E O N C G S H I J P R Q 3 1 2 1 2 3 4 5 6 7 8 9 10 11 Competency framework

12 Events A curriculum is made up of many educational events
Events have metadata, descriptive information about the events IDs required! I A E J B F K C G L D H

13 Expectations Curricula may reference objectives , competencies, learning outcomes, or milestones. These are called competency objects. IDs required! 3 1 2 6 4 5 9 7 8 12 10 11

14 Competency Objects and Events
An event may be intended to address one or more competency objects (learning objectives, competencies, and milestones) A competency object may be associated with many events G 1 2 K 6 2

15 Sequence Blocks Events can be organized into blocks of instruction, called Sequence Blocks (think course/module/ year/phase/clerkship etc) A recurring event may be associated with more than one Sequence Block Sequence blocks may be associated with competency objects (competencies, objectives, etc) Sequence blocks have metadata, too IDs required! I A E J B F 2 1 K C A L G D H 7

16 Integration Blocks Curricular themes, or Integration Blocks, may apply to many sequence blocks, events, or objectives/ competencies and describe how the curriculum is semantically integrated. An event or block may link to many Integration Blocks and an Integration Block may link to many events, blocks or competency objects IDs required! A E B F C G D H I J K L 1 M E O N C G S H I J P R Q 3 2

17 Sequence Sequence blocks can be put together to form a the main structure of the curriculum (a sequence of courses, modules, phases, clerkships) A E B F C G D H I J K L 1 M E O N C G S H I J P R Q 3 2

18 Sequence Block Nesting
A E B F I J 1 2 Sequence blocks can be nested: a module or clerkship might be in a year which is within a phase and so on … Blocks will inherit any relations to competency objects from their sub-blocks No restriction on the number of nest levels Recursion explicitly forbidden A E B F I J 1 2 A E B F I J 1 2 A E B F I J 1 2

19 Types of blocks: ordered
Sequence=ordered Required=true - All sub-blocks within an ordered block happen with a prescribed sequence Block Required I A E J B F Block Required R C J D Q Z Block Required T A X Y M B

20 Types of blocks: unordered
- All sub-blocks within an unordered block happen for all students, but students may encounter them in a variable order Block Sequence=unordered Required=true Block Required A E B F I J Block Required C J Q Z R D Block Required A X M B T Y Block Required D K N S L C

21 Types of blocks: selectives
- Every student must participate in [specify number] of the blocks, and the order is variable Block Required = Required Order=Unordered Minimum=1 Block Optional A E B F I J Block Optional C J Q Z R D Block Optional A X M B T Y OR OR

22 Types of blocks: electives
- Students may or may not participate in any of the blocks Sequence Block Required = Optional Order =Unordered Minimum=0 Sequence Block Optional A E B F I J Sequence Block Optional C J Q Z R D Sequence Block Optional A X M B T Y OR OR

23 Types of blocks: parallel
- All sub-blocks within a parallel block happen at the same time, but are distinct educational units Sequence Block Required = Required Order = Parallel Sequence Block Required C J Q Z R D Sequence Block Required A E B F I J

24 Clerkships, required = Required, minimum = 2, maximum = 2
Level 1 Main Track, required = Optional, track = true Level 1 Common Clerkships, required = Required Med/Geri 1 84 days required in track Surg 1 56 days required in track NeuroScience 28 days optional Elective 21 days required Neurology 28 days required Pediatrics 1 42 days required in track Ambulatory 1 28 days required in track Anesthesiology 7 days required Psychiatry 28 days required Level 1 Ambulatory Track, required = Optional, track = true Option 1: This diagram shows two tracks for level 1 clerkships that show the varying lengths. The minimum maximum values of the “parent” block indicate that students must take two of the “child” blocks. Since Common Clerkships is required, that means that either the Main track or the Ambulatory track is required. We would need a new attribute called “required in track” to show that medicine, geriatrics, surgery and pediatrics are required in the track. The content for the tracks could be different (ie Surg 1 has different educational events than Surg 2). When researchers search for all surgery clerkships, they will see two clerkships of varying length with varying content, both marked required in track. The CIP search mechanism would need to understand the nuance of required in track when allowing users to construct search queries. QUESTION: Should we be able to say that a clerkship model is longitudinal? Is that mutually exclusive to integrated and rotation? Ob/Gyn 42 days required Med/Geri 2 63 days required in track Surg 2 42 days required in track Pediatrics 2 28 days required in track Ambulatory 2 77 days required in track

25 Level 2 Clerkships, July 4, 2011– April 27, 2012
Level 2 Clerkships Variable time, July 4, 2011 – April 27, order = unordered Level 2 Clerkships, second half of the year, January 1, 2012 – April 27, 2012 Subinternship required 28 days Emergency Med required 28 days Into to Internship required 14 days Anatomic Radiology Required 14 days or online course Critical Care Required 14 days This shows how you would model a course that can occur anytime during the second half of the year. Electives 147 days (21 are usually taken during level 1 and the rest during the level 2 variable time

26 Relates to other competency objects . . .
A competency object . . . Statement Competency Framework Relates to other competency objects . . . Learning Object Assess-ment Performance data Which can relate to external resources . . .

27 Business Rules Program-level competencies are required and must be mapped to PCRS using #related All competencies, including PCRS, program-level, sequence-block level, and event level competencies used in a <Relation> tag must have an <Includes> tag

28 Spec Required Elements
Curriculum Inventory ReportID Institution Program Title ReportDate ReportingStartDate ReportingEndDate Language Description Events Event Title EventDuration in hours and min Instructionalmethod (primary=true/false) or AssessmentMethod (purpose=formative/ summative) Expectations CompetencyObject identifier title Category * AcademicLevels Sequence * Vocab ids * - required by AAMC business Rules

29 SequenceBlock Requirements
Attributes: id, required Title Timing (either duration in days or start and end dates – clerkships must specify both) Level SequenceBlockEvent SequenceBlockReference

30 Matching Program Objectives to PCRS (Crosswalk)
2. Knowledge for Practice: Demonstrate knowledge of established and evolving biomedical, clinical, epidemiological and social-behavioral sciences, as well as the application of this knowledge to patient care. 2.1 Demonstrate an investigatory and analytic approach to clinical situations 2.2 Apply established and emerging bio-physical scientific principles fundamental to health care for patients and populations 2.3 Apply established and emerging principles of clinical sciences to diagnostic and therapeutic decision-making, clinical problem-solving, and other aspects of evidence-based health care 2.4 Apply principles of epidemiological sciences to the identification of health problems, risk factors, treatment strategies, resources, and disease prevention/health promotion efforts for patients and populations 2.5 Apply principles of social-behavioral sciences to provision of patient care, including assessment of the impact of psychosocial and cultural influences on health, disease, care-seeking, care compliance, and barriers to and attitudes toward care 2.6 Contribute to the creation, dissemination, application, and translation of new health care knowledge and practices 2.99 Other knowledge for practice Demonstrate a broad working knowledge of the fundamental science, principles, and processes basic to the practice of medicine. 2.1 Understand the clinical relevance of scientific inquiry. 2.2 Apply this knowledge in a judicious and consistent manner to prevent common health problems and achieve effective and safe patient care. 2.3 Demonstrate the ability to evaluate emerging knowledge and research as it applies to diagnosis, treatment and the prevention of disease. 2.4 Utilize state of the art information technology and tools to retrieve, manage and use biomedical information in the care of individuals. 2.5 Utilize state of the art information technology and tools to retrieve, manage and use biomedical information in the care of populations. 2.6 Understand the indications, contraindications, and potential complications of common clinical procedures. 2.7 Perform the basic clinical procedures expected of a new PGY-1.

31 PCRS Mapping and Sequence Block or Event-level Competencies
Mapping of Sequence Block-level or Event-level objectives / competencies directly to PCRS is not allowed. Sequence Block and / or Event Objectives / Competencies, if mapped, are mapped to each other and to Program Objectives / Competencies

32 Competency Object Rules
<Competency Object> tags must be used to identify competencies, along with their <Category> (level) Program-level Sequence block-level Event-level Competency object URIs must be unique <Competency Object> tags must not be used to identify PCRS competencies

33 Competency Objects and the Curriculum Structure
All sequence block-level and event-level competencies must be referenced by at least one Sequence Block or Event (that was identified in the curriculum structure) Competency Framework Curriculum Inventory skos: narrower skos: related P2 S5 PCRS 6.99 PCRS 1.5 I The Sequence Block References the Program-level Competency P2 E J B P = Program Level Competency S = Sequence Block Level Competency E = Event Level Competency The Sequence Block References the Sequence Block-level Competency F S5 A

34 Circular References Relationships must not result in circular references. A circular reference occurs when a child element draws a hierarchical relationship to its parent that infers the parent is lower-level than the child. skos: narrower skos: related P2 S1 S2 PCRS 6.99 PCRS 1.5 P = Program Level Competency S = Sequence Block Level Competency E = Event Level Competency skos: narrower S3 Notes: – A c it is nonsensical because a higher level concept can’t be a child of a narrower/lower level concept – e.g., the tire can never be the parent of the car. skos: narrower This relationship is not allowed because it creates a “Circular Reference” and violates a Business Rule. skos: narrower E4

35 Other notes Set the schema location to public copy: xsi:schemaLocation=" Use namespace prefixes for competencies and competency framework Use AAMC ids for schools Competency Object identifiers must be URIs URLs do not have to be resolvable. As long as they uniquely identify, it will work.   Add anchors to the URL so that each objective can be distinguished using the URL with the anchor included. Duration uses xsd format, for example, P40D, PT2H 1 week = 5 days

36 PCRS --Physician Competency Reference Set
“…a list of common learner expectations” What competencies schools are incorporating into their curricula; Where in their curricula schools are incorporating expectations and competencies; How schools are teaching and assessing competencies; and In what context and/or content competencies are being taught.

37 Step 1: Match Competencies
PCRS Step 1: Match Competencies

38 Step 2: Create XML File

39 <Expectations>
<CompetencyObject> </CompetencyObject> <CompetencyFramework> <cf:Includes> </cf:Includes> <cf:Relation> </cf:Relation> </CompetencyFramework> </Expectations> Insert one of your program competencies

40 <CompetencyObject>
<lom:lom> <lom:general> <lom:identifier> <lom:catalog>URI</lom:catalog> <lom:entry></lom:entry> </lom:identifier> <lom:title> <lom:string>Students will convey information honestly and tactfully; will engage in truthful interactions with patients, peers, and in professional work. </lom:string> </lom:title> </lom:general> </lom:lom> <co:Category term="program-level-competency"/> </CompetencyObject>

41 <Expectations> <CompetencyFramework>
<cf:Includes> </cf:Includes> </CompetencyFramework> </Expectations> Insert one of your program competencies Insert one PCRS

42 <cf:Includes> <cf:Catalog>URI</cf:Catalog> <cf:Entry></cf:Entry> </cf:Includes> <cf:Includes> <cf:Catalog>URI</cf:Catalog> <cf:Entry></cf:Entry> </cf:Includes>

43 <Expectations>
<CompetencyFramework> <cf:Relation> </cf:Relation> </CompetencyFramework> </Expectations> Show the relationship between your program competency & PCRS

44 <cf:Relation> <cf:Reference1> <cf:Catalog>URI</cf:Catalog> <cf:Entry></cf:Entry> </cf:Reference1> <cf:Relationship></cf:Relationship> <cf:Reference2> <cf:Entry></cf:Entry> </cf:Reference2> </cf:Relation>

45 Lessons Learned Some benefit to moving a bit slow
Having more than one person understanding the specifications is important Humans are not the weakest link Networking is vital Carving out dedicated time is crucial Explaining the big picture to faculty is important Developing a functioning end product is vital Developing tools not just a product is key for the future

46 Show Events Mapping in XML File
Ye Chen

47 XML File Structure <CurriculumInventory > <ReportID> </ReportID><Institution></Institution> <Program> </Program><Title> </Title> <ReportDate> </ReportDate> <ReportingStartDate> </ReportingStartDate> <ReportingEndDate> </ReportingEndDate> <Language> </Language> <Description> </Description> <Events> <Event> </Event> </Events> <Expectations> <CompetencyObject> </CompetencyObject> <CompetencyFramework> </CompetencyFramework> </Expectations> <AcademicLevels></AcademicLevels> <Sequence> <SequenceBlock > </SequenceBlock> </Sequence> </CurriculumInventory>

48 Events and Competencies
Event (E1): White Coat Ceremony Event (E2): Orientation Lecture Event (E3): Peds Part 1 Event (E4): Peds Part 2 Describe the three functions List the organization, Gather, synthesize, and organize patient information. Professionalism Perform all medical, diagnostic

49 Repeat the same format for other 4 competencies
List All Competencies <Expectations> <CompetencyObject> List competencies </CompetencyObject> <CompetencyFramework> </CompetencyFramework> </Expectations> <CompetencyObject> <lom:lom> <lom:general> <lom:identifier> <lom:catalog>URI</lom:catalog> <lom:entry></lom:entry> </lom:identifier> <lom:title> <lom:string>Professionalism</lom:string> </lom:title> </lom:general> </lom:lom> <co:Category term="program-level-competency"/> </CompetencyObject> This value will be refered in several places (event, include, and relationship) Repeat the same format for other 4 competencies

50 Show the relationships among competencies
<Expectations> <CompetencyObject> List competencies </CompetencyObject> <CompetencyFramework>Show relationships</CompetencyFramework> </Expectations> Include all program level competencies, include sequence block level competencies and event level competencies if they have mappings with other competencies. other information for competencyFrameWork <lom:lom>------</lom:lom> <cf:Includes> <cf:Catalog>URI</cf:Catalog> <cf:Entry></cf:Entry> </cf:Includes> <cf:Includes> <cf:Catalog>URI</cf:Catalog> <cf:Entry></cf:Entry> </cf:Includes> <cf:Includes> <cf:Catalog>URI</cf:Catalog> <cf:Entry></cf:Entry> </cf:Includes> <cf:Includes> <cf:Catalog>URI</cf:Catalog> <cf:Entry></cf:Entry> </cf:Includes>

51 Show the relationships among competencies (cont.)
Describe the three functions Gather, synthesize, and organize patient information. program level comptetencies to PCRS…….. Professionalism <cf:Relation> <cf:Reference1> <cf:Catalog>URI</cf:Catalog> <cf:Entry></cf:Entry> </cf:Reference1> <cf:Relationship></cf:Relationship> <cf:Reference2> <cf:Catalog>URI</cf:Catalog> <cf:Entry></cf:Entry> </cf:Reference2> </cf:Relation> <cf:Relation> <cf:Reference1> <cf:Catalog>URI</cf:Catalog> <cf:Entry></cf:Entry> </cf:Reference1> <cf:Relationship></cf:Relationship> <cf:Reference2> <cf:Catalog>URI</cf:Catalog> <cf:Entry></cf:Entry> </cf:Reference2> </cf:Relation>

52 Map event to competencies
Only show the direct mappings. Non direct mapping is taken care by the relationship. Event (E1): White Coat Ceremony Event (E2): Orientation Lecture Event (E3): Peds Part 1 Event (E4): Peds Part 2 Describe the three functions List the organization, Perform all medical, diagnostic <Event id="E1"> <Title>White Coat Ceremony</Title> <CompetencyObjectReference>/CurriculumInventory/Expectations/CompetencyObject[lom:lom/lom:general/lom:identifier/lom:entry='’]</CompetencyObjectReference> <InstructionalMethod primary="true">IM013</InstructionalMethod> </Event> /CurriculumInventory/Expectations/CompetencyObject[lom:lom/lom:general/lom:identifier/lom:entry= ] '’ inside <Event id=“E2”><Event id=“E3”>, there should be <CompetencyObjectReference> with different entry value.

53 Event and SequenceBlock
Curriculum Inventory Event and SequenceBlock Saniya Saifee NYU School of Medicine

54 Event Event contains sub elements that describe a single education or assessment event

55 CompetencyObjectReference
Event-Subelements Event Title EventDuration Description Keyword CompetencyObjectReference ResourceType ID InstructionalMethod AssessmentMethod 0..∞ Required Optional

56 Event XML <InstructionalMethod primary="true"> IM013
<Events> <Event id="E1"> <Title>White Coat Ceremony</Title> <EventDuration>PT2H</EventDuration> <Description>First year students are given </Description> <Keyword hx:source="MeSH" hx:id="D010817"> <lom:string>Physician-Patient Relations</lom:string> </Keyword> <Interprofessional>false</Interprofessional> <CompetencyObjectReference>/CurriculumInventory /Expectations/CompetencyObject[lom:lom/lom:general/lom:identifier/lom:entry=''] </CompetencyObjectReference> <InstructionalMethod primary="true"> IM013 </InstructionalMethod> </Event>

57 Competency Object <Expectations> <CompetencyObject> <lom:lom> <lom:general> <lom:identifier> <lom:catalog>URI</lom:catalog> <lom:entry> event_competency/4 </lom:entry> </lom:identifier> <lom:title> <lom:string>Describe the three functions of the medical interview </lom:string> </lom:title> </lom:general> </lom:lom> <co:Category term="event-level-competency"/> </CompetencyObject> </Expectations>

58 Events-checklist An event must have at least one instructional OR assessment method. An event can only be reported if we can report timing. An event may have multiple instructional or assessment methods, but if it’s an instructional session, only one method can be primary. All assessment methods must be marked as formative or summative. Within <Event> tags, the <AssessmentMethod> and <InstructionalMethod> tags must reference a unique ID from their respective vocabularies

59 SequenceBlock Events Sequence Block B A B A D D When F 1 F E E I
G F H E D I A B G F H E D I When 1 Combined 2 3 Events Sequence Block Expectations

60 SequenceBlock-Subelements
Title Timing Level ClerkshipModel CompetencyObjectRef… ID Required 0..∞ ID 0..∞ SequenceBlockEvent Required 1..∞ SequenceBlockReference Optional

61 Timing Timing Date StartDate EndDate Duration

62 SequenceBlockEvent required SequenceBlockEvent EventReference
StartDate EndDate required

63 AcademicLevels <AcademicLevels> <LevelsInProgram>2</LevelsInProgram> <Level number="1"> <Label>Year 1</Label> <Description>Preclinical</Description> </Level> <Level number="2"> <Label>Year 2</Label> </AcademicLevels>

64 Sequence Block <Sequence> <SequenceBlock id="e" required="Required"> <Title>Introduction to Medicine</Title> <Timing><Duration>P1M</Duration></Timing> <Level> </Level> <SequenceBlockEvent required="true"> <EventReference> </EventReference> </SequenceBlockEvent> </SequenceBlock> </Sequence>

65 Nested SequenceBlock <SequenceBlock id="c" required="Required">
<Title>Year 1</Title> <Timing> <Duration>P12M</Duration> </Timing> <Level> </Level> <CompetencyObjectReference> /CurriculumInventory/Expectations/CompetencyObject [lom:lom/lom:general/lom:identifier/lom:entry= ''] </CompetencyObjectReference> <SequenceBlockReference> </SequenceBlockReference> </SequenceBlock>

66 SB-checklist A SequenceBlock (Year, course, module) can only be reported if it has at least one “valid” event. The id attribute in the SB must be unique. SB should not be associated with a level number that has not been defined in <AcademicLevels> The <Timing> subelement of <SequenceBlocks> tags with a ClerkshipModel attribute must specify <Dates> and <Duration>

67 Is this a valid Event? <Event id="E2"> <Title>Orientation Lecture</Title> <EventDuration>PT4H</EventDuration> <Description> Faculty discuss what the students are to expect in the coming year. </Description> <InstructionalMethod primary="true">Lecture </InstructionalMethod> </Event>

68 It is not a valid event <Event id="E2"> <Title>Orientation Lecture</Title> <EventDuration>PT4H</EventDuration> <Description> Faculty discuss what the students are to expect in the coming year. </Description> <InstructionalMethod primary="true">Lecture </InstructionalMethod> </Event> IM013

69 Best Practices and project insights
Saniya Saifee NYU School of Medicine

70 Process followed Software program that creates XML Curriculum
Management System (CuRIE) <CurriculumInventory> …….. …. </<CurriculumInventory Validate XML against XSD CI Portal Send XML AAMC Business rules errors Refactor And repeat

71 Lessons learned -1 <CurriculumInventory> ….
“Including ALL Academic levels, courses, events, Expectations in your first attempt while uploading file on CI portal” </CurriculumInventory> “Including ONE Academic levels, courses, events,

72 Create a checklist of minimum requirements
Lesson learned -3 Create a checklist of minimum requirements

73 Lesson learned -3

74 Keep doing frequent and repeated submission
Lesson learned - 4 Keep doing frequent and repeated submission

75 CI Process Validate XML against MedBiq CI Standard Upload to CI
Successful submission results in ‘Submitted Status’ Log into School Portal Download and Review Verification Report Download results in ‘Review Status’ Reject or Verify Data Rejection returns Status to ‘No Response’ Verification results in ‘Verified Status’

76 Resources Curriculum Inventory Implementers’ Listserv: Competency Framework Specs and Schemas: AAMC Curriculum Inventory Developer Resources: Physician Competency Reference Set URIs: SKOS: Simple Knowledge Organization System MedBiquitous: Curriculum Inventory Working Group: Competencies Working Group:

77 Questions?

Download ppt "Curriculum Inventory Workshop"

Similar presentations

Ads by Google