1. Curriculum Inventory Developer’s Workshop Part II: Competency Framework Terri Cameron, MA Director of Curriculum Management Valerie Smothers, MA Deputy Director, MedBiquitous

2. We will cover: MedBiquitous Competency Objects and Competency Framework How the curriculum structure, the Competency Framework, and the Physician Competency Reference Set (PCRS) work together How to apply the Competency Framework and Competency Object Business Rules to Curriculum Inventory data How to document school expectation mapping to PCRS using the MedBiquitous Competency Framework

3. Continuous Quality Improvement / Accreditation Curriculum Inventory Reports MedBiquitous Data Exchange Standard Curriculum data mapped to an expectations hierarchy (competencies to course objectives to session objectives) are uploaded to the Curriculum Inventory (CI) from school systems using the MedBiquitous CI data exchange standard (www.medbiq.org).www.medbiq.org Standardized Vocabulary for Instructional Methods, Assessment Methods, and Resources Institutionally- developed expectations will be matched to Physician Competency Reference Set Vendor and School- developed Curriculum Management Systems Curriculum Inventory www.aamc.org/medaps Today’s Session

4. Review of Terminology Previous Academic Year (LCME model – if current year is 2016-2017, previous academic year is 2015-2016) Academic Level / Phase (formerly called ‘year of curriculum’) Sequence Block (Course, Module, Unit, Block, Clerkship) Event (Instructional or Assessment Session) (e.g., Lecture, Lab, Discussion, Assessment) www.aamc.org/medaps

5. Review of Terminology, Cont’d. Standardized Vocabulary (Instructional Methods, Assessment Methods) PCRS (Physician Competency Reference Set) Expectations (Competency / Objective / Milestone / EPA) Verification Report (Curriculum Inventory data report used to verify curriculum details) MedBiquitous (www.medbiq.org)www.medbiq.org www.aamc.org/medaps

6. MedBiquitous Competency Object and Competency Framework

7. ® Mission: To develop and promote technology standards for the health professions that advance lifelong learning, continuous improvement, and better patient outcomes. Not-for-profit, member-driven, standards development organization

8. Building the Foundation: The MedBiquitous Architecture (www.medbiq.org) * American National Standards † Candidate standard

9. Competency Concepts Competency Object - an umbrella term used to describe any abstract statement of learning or performance expectations, and information related to the statement. 1 Competency Framework - an organized and structured representation of a set of interrelated and purposeful competency objects. 1 1 ANSI/MEDBIQ CF.10.1-2012 Competency Framework Specifications, October 3, 2012.

10. Competency Object Statements can be learning outcomes, competencies, learning objectives, professional roles, topics, classifications, collections, etc. The Object is abstract in that it does not inherently contain information about connections of the statement to individuals, events or other objects.

11. URL Title (statement) Description Category http://example.com/competency/ 12345.xml Practice-based Learning and Improvement Residents must demonstrate the ability to investigate and evaluate their care of patients... Category = program-level- competency http://example.com/competency/ 12345.xml Practice-based Learning and Improvement Residents must demonstrate the ability to investigate and evaluate their care of patients... Category = program-level- competency From ACGME: Competency Object Example

12. URL Title (statement) Description Category URI http://example.com/competency/12345.xml Practice-based Learning and Improvement Residents must demonstrate the ability to investigate and evaluate their care of patients... URI http://example.com/competency/12345.xml Practice-based Learning and Improvement Residents must demonstrate the ability to investigate and evaluate their care of patients... From ACGME: Competency Object Example

13. Statement A competency object... Relates to other competency objects... Which can relate to external resources... Learning Object Assess- ment Performance data Competency Framework

14. SKOS Mapping: Simple Knowledge Organization System (W3C, World Wide Web Consortium, www.w3.org)www.w3.org http://www.w3.org/TR/skos-reference/#mapping Related (skos:related) = “is related to concept” Broad (skos:broader) = “has broader concept” Narrow (skos:narrower) = “has narrower concept” How Do We Describe Relationships in a Competency Framework? Domain Competency Learning Objective Broader Narrower

15. Example... URI http://www.example.org/A.xml URI http://www.example.org/B.xml URI http://www.example.org/C.xml URI http://www.example.org/A.xml http://www.w3.org/2004/02/skos/core#narrower URI http://www.example.org/B.xml... What competencies are included in this framework? What are the relationships among the competencies included? A B C

16. Questions? Use Chat to submit questions Please send to All Panelists. www.aamc.org/medaps

17. How would you specify this relationship? 1 2 3 Has ________ concept A: Related B: Broader C: Narrower Has ________ concept

18. How would you specify this relationship? 1 2 3 Has narrower concept A: Related B: Broader C: Narrower Has narrower concept

19. The curriculum structure and the Competency Framework

20. Curriculum Inventory A E B F C G DH I J K L A 1 M E O N C G SH I J P R Q 2 3 12 4536 897 1011 The Curriculum Inventory uses Competency Frameworks to define the competencies/ objectives integrated into the curriculum. Competency Framework 9 8 5 The Curriculum Inventory and Competency Frameworks

21. Curriculum Inventory: “Expectations” Contain Competency Objects (e.g., learning objectives, competencies, learning outcomes) Contain a competency framework that describes a set of relationships between competency objects Are referenced from events or sequence blocks

22. Curricular Structure and Competency Objects Sequence Blocks (e.g., courses) reference Events and Competency Objects Events (e.g., sessions) reference Competency Objects and Elements (instructional methods, assessment methods, etc) A E B F C G DH I J K L A 1 2 7 A 1 1

23. Sequence Blocks, Events, and Competency Objects Some schools map relationships between Sequence Block and Event learning objectives Some schools map relationships between Program Level Objectives / Competencies and Sequence Block learning objectives OR Event learning objectives Some schools do not have any mapping for Sequence Block learning objectives OR Event learning objectives The MedBiquitous Competency Framework supports any of these options

24. Indicating the Type of Competency For the Curriculum Inventory, there are three categories of Competency Objects within the Framework: Program-level Sequence Block-level Event-level This program instills the following competencies… By the end of this course, you will be able to… By the end of this session, you will be able to…

25. Relationships in a Competency Framework Program-level competency Sequence block-level competency Program-level competency Program-level competency Has Narrower Concept Event-level competency Event-level competency Event-level competency Event-level competency Has Narrower Concept

26. School Competency Framework Describe the major organ systems, their components and the relationships between them in the normal individual Identify common variations in structure and location of organs within the body Medical Knowledge: Describe common disease entities, including their characteristic signs and symptoms, etiology, epidemiology, and pathophysiology. Medical Knowledge: Demonstrate basic knowledge of normal structure, development, and function of major organ systems and the body as a whole in the context of health and disease. Has Narrower Concept Identify the organs involved in the renal system Explain the interaction of the heart and lungs in the circulatory system D escribe the anatomical structure of the heart Explain how structural variations affect kidney function Has Narrower Concept

27. We build an aggregation tool! Physician Competency Reference Set (PCRS) 58 Competencies across 8 Domains Each school creates a crosswalk between school competencies and PCRS Using MedBiq Competency Framework, School Program Objectives/Competencies are mapped to PCRS Competency Mapping at other levels can be tracked through the Program Objective / PCRS relationship What do we do with 150+ Different Competency Frameworks?

28. Expectation Levels PCRS Program Level Sequence Block Level Event Level U.S./ Canada Medical School Course/ Clerkship Session/ Experience www.aamc.org/medaps Describe the anatomical structure of the heart. Identify common variations in structure and location of organs within the body. Demonstrate basic knowledge of normal structure, development and function of major organ systems and the body as a whole in the context of health and disease. Apply established and emerging bio-physical scientific principles fundamental to health care for patients and populations.

29. Physician Competency Reference Set Partial URI Physician Competency Reference Set (AAMC 2013) aamc-pcrs-comp-c0100 PATIENT CARE: Provide patient-centered care that is compassionate, appropriate, and effective for the treatment of health problems and the promotion of health aamc-pcrs-comp-c0101 Perform all medical, diagnostic, and surgical procedures considered essential for the area of practice aamc-pcrs-comp-c0102 Gather essential and accurate information about patients and their condition through history-taking, physical examination, and the use of laboratory data, imaging, and other tests aamc-pcrs-comp-c0103 Organize and prioritize responsibilities to provide care that is safe, effective, and efficient aamc-pcrs-comp-c0104 Interpret laboratory data, imaging studies, and other tests required for the area of practice aamc-pcrs-comp-c0105Make informed decisions about diagnostic and therapeutic interventions based on patient information and preferences, up-to-date scientific evidence, and clinical judgment

30. Competency Framework and PCRS Program-level competency Sequence block-level competency Program-level competency Program-level competency Has Narrower Concept Event-level competency Event-level competency Event-level competency Event-level competency Has Narrower Concept PCRS 6.99 PCRS 1.5 Is Related

31. Competency Framework and Curriculum Structure: One Example skos: narrower skos: related P3 S5 PCRS 6.99 PCRS 1.5 Competency Framework Curriculum Structure The Program- level Competency has narrower Sequence Block-level Competency The Sequence Block-Level Competency has narrower Event- level Competency P = Program Level Competency S = Sequence Block Level Competency E = Event Level Competency Identify common variations in structure and location of organs within the body Medical Knowledge: Demonstrate basic knowledge of normal structure, development, and function of major organ systems and the body as a whole in the context of health and disease. Has Narrower Concept Explain how structural variations affect kidney function Has Narrower Concept Program (School Curriculum) Sequence Block Event E2 E1

32. Relationships and References skos: narrower skos: related P2 S5 E2 PCRS 6.99 PCRS 1.5 The Sequence Block References the Program-level Competency The Sequence Block References the Sequence Block-level Competency P = Program Level Competency S = Sequence Block Level Competency E = Event Level Competency Competency Framework Curriculum Inventory The Event References the Event-level Competency A E B F C G DH I J K L A S5 P2 7

33. Learning Check How do you know the role of a competency in the curriculum? a) The Competency Object Category is tagged as Program-level, Sequence block-level, or Event- level b) The competency is referenced from a sequence block(s) or event(s) in the curriculum c) The competency framework tells you how the competencies are related hierarchically d) All of the above

34. Learning Check How do you know the role of a competency in the curriculum? a) The Competency Object Category is tagged as Program-level, Sequence block-level, or Event- level b) The competency is referenced from a sequence block(s) or event(s) in the curriculum c) The competency framework tells you how the competencies are related hierarchically d) All of the above

35. Questions? Use Chat to submit questions. Please send to All Panelists. www.aamc.org/medaps

36. AAMC Business Rules

37. General Rules Program-level competencies are required and must be mapped to PCRS using #related All competencies, including PCRS, program- level, sequence-block level, and event level competencies used in a tag must have an tag

38. PCRS Mapping and Program Level Competencies Program-level competencies may not be mapped to domain-level PCRS Program Objectives / Competencies that do not match a PCRS competency can be mapped to the appropriate Domain ‘Other’ (numbered 99 in each Domain)

39. Matching Program Objectives to PCRS (Crosswalk) PCRSProgram Objectives 2. Knowledge for Practice: Demonstrate knowledge of established and evolving biomedical, clinical, epidemiological and social-behavioral sciences, as well as the application of this knowledge to patient care. 2.1 Demonstrate an investigatory and analytic approach to clinical situations 2.2 Apply established and emerging bio-physical scientific principles fundamental to health care for patients and populations 2.3 Apply established and emerging principles of clinical sciences to diagnostic and therapeutic decision- making, clinical problem-solving, and other aspects of evidence-based health care 2.4 Apply principles of epidemiological sciences to the identification of health problems, risk factors, treatment strategies, resources, and disease prevention/health promotion efforts for patients and populations 2.5 Apply principles of social-behavioral sciences to provision of patient care, including assessment of the impact of psychosocial and cultural influences on health, disease, care-seeking, care compliance, and barriers to and attitudes toward care 2.6 Contribute to the creation, dissemination, application, and translation of new health care knowledge and practices 2.99 Other knowledge for practice 2.Demonstrate a broad working knowledge of the fundamental science, principles, and processes basic to the practice of medicine. 2.1 Understand the clinical relevance of scientific inquiry. 2.2 Apply this knowledge in a judicious and consistent manner to prevent common health problems and achieve effective and safe patient care. 2.3 Demonstrate the ability to evaluate emerging knowledge and research as it applies to diagnosis, treatment and the prevention of disease. 2.4 Utilize state of the art information technology and tools to retrieve, manage and use biomedical information in the care of individuals. 2.5 Utilize state of the art information technology and tools to retrieve, manage and use biomedical information in the care of populations. 2.6 Understand the indications, contraindications, and potential complications of common clinical procedures. 2.7 Perform the basic clinical procedures expected of a new PGY-1.

40. Defining the Competency Objects URI http://medicine.edu/competencies/K_2_1 Understand the clinical relevance of scientific inquiry.

41. Matching Program Objectives to PCRS (metadata and included competencies) URI http://medicine.edu/competencies/K_2_1 http://www.w3.org/2004/02/skos/core#related URI http://aamc.org/pcrs/aamc-pcrs-comp-c0201

42. PCRS Mapping and Sequence Block or Event-level Competencies Mapping of Sequence Block-level or Event- level objectives / competencies directly to PCRS is not allowed. Sequence Block and / or Event Objectives / Competencies if mapped, are mapped to each other and to Program Objectives / Competencies

43. Competency Relationships Explanation P = Program Level Competency S = Sequence Block Level Competency E = Event Level Competency related has narrower concept skos: narrower skos: related P2 S5 E2 PCRS 6.99 PCRS 1.5 E3 skos: narrower

44. What’s Wrong with this Mapping? Partial URIPCRS aamc-pcrs-comp-c0100 PATIENT CARE: Provide patient- centered care that is compassionate, appropriate, and effective for the treatment of health problems and the promotion of health aamc-pcrs-comp-c0101 Perform all medical, diagnostic, and surgical procedures considered essential for the area of practice aamc-pcrs-comp-c0102 Gather essential and accurate information about patients and their condition through history- taking, physical examination, and the use of laboratory data, imaging, and other tests aamc-pcrs-comp-c0103 Organize and prioritize responsibilities to provide care that is safe, effective, and efficient aamc-pcrs-comp-c0104Interpret laboratory data, imaging studies, and other tests required for the area of practice Curriculum Structure Program-Level Competency (School Curriculum) Sequence Block- Level Competency Event-Level Competency

45. Correct Mapping Partial URIPCRS aamc-pcrs-comp-c0100 PATIENT CARE: Provide patient- centered care that is compassionate, appropriate, and effective for the treatment of health problems and the promotion of health aamc-pcrs-comp-c0101 Perform all medical, diagnostic, and surgical procedures considered essential for the area of practice aamc-pcrs-comp-c0102 Gather essential and accurate information about patients and their condition through history- taking, physical examination, and the use of laboratory data, imaging, and other tests aamc-pcrs-comp-c0103 Organize and prioritize responsibilities to provide care that is safe, effective, and efficient aamc-pcrs-comp-c0104Interpret laboratory data, imaging studies, and other tests required for the area of practice Curriculum Structure Program-Level Competency (School Curriculum) Sequence Block- Level Competency Event-Level Competency

46. Competency Object Rules tags must be used to identify competencies, along with their (level) Program-level Sequence block-level Event-level Competency object URIs must be unique tags must not be used to identify PCRS competencies

47. Sample Code: Competency Objects tag along with the and a unique URI URI http://med.sample.edu/competencies/Program- Physician-Patient-Rel Physician-Patient Relations

48. Competency Objects and the Curriculum Structure All sequence block-level and event-level competencies must be referenced by at least one Sequence Block or Event (that was identified in the curriculum structure) skos: narrower skos: related P2 S5 PCRS 6.99 PCRS 1.5 Competency Framework Curriculum Inventory The Sequence Block References the Program-level Competency The Sequence Block References the Sequence Block-level Competency P = Program Level Competency S = Sequence Block Level Competency E = Event Level Competency A E B F I J S5 P2

49. Circular References Relationships must not result in circular references. A circular reference occurs when a child element draws a hierarchical relationship to its parent that infers the parent is lower- level than the child. P = Program Level Competency S = Sequence Block Level Competency E = Event Level Competency skos: narrower skos: related P2 S1 S2S2 PCRS 6.99 PCRS 1.5 S3 skos: narrower E4 skos: narrower This relationship is not allowed because it creates a “Circular Reference” and violates a Business Rule.

50. Errors you may receive

51. Error 1: URI 'http://medschool.edu/CourseObjective_9' referred in Competency Object was not found under Sequence Block Competency Reference skos: narrower skos: related P2 S9 PCRS 6.99 PCRS 1.5 Competency Framework Curriculum Inventory P = Program Level Competency S = Sequence Block Level Competency E = Event Level Competency A E B F I J P2

52. Solution: Add a Sequence Block Competency Reference (or delete the competency object) skos: narrower skos: related P2 S9 PCRS 6.99 PCRS 1.5 Competency Framework Curriculum Inventory P = Program Level Competency S = Sequence Block Level Competency E = Event Level Competency A E B F I J S9 P2 Year 1 P12M /CurriculumInventory/AcademicLevels/Level[@number='1'] /CurriculumInventory/Expectations/CompetencyObject[lom:lom/lom:general/lom:identifier/lom:en try='http://medschool.edu/CourseObjective_9']

53. Error 2: URI 'http://medschool.edu/CourseObjective_9' referred in Includes was not found under Competency Object...... URI http://medschool.edu/CourseObjective_9...

54. Solution: Add a competency object for that competency: URI 'http://medschool.edu/CourseObjective_9 Describe cardiovascular anatomy.

55. Error 3: URI 'http://medschool.edu/CourseObjective_9' referred in Sequence Block competency object reference was not found under Competency Object skos: narrower skos: related P2 PCRS 6.99 PCRS 1.5 Competency Framework Curriculum Inventory P = Program Level Competency S = Sequence Block Level Competency E = Event Level Competency A E B F I J S9 P2

56. Solution: Add a competency object for that competency: URI 'http://medschool.edu/CourseObjective_9 Describe cardiovascular anatomy.

57. Error 4 URI 'http://medschool.edu/CourseObjective_9' referred in Relations was not found under Includes... URI http://medschool.edu/EventObjective_2.xml URI http://medschool.edu/CourseObjective_9 http://www.w3.org/2004/02/skos/core#narrower URI http://medschool.edu/EventObjective_2.xml...

58. Solution Add an includes statement for that competency.... URI http://medschool.edu/EventObjective_2.xml URI http://medschool.edu/CourseObjective_9.xml URI http://medschool.edu/CourseObjective_9 http://www.w3.org/2004/02/skos/core#narrower URI http://medschool.edu/EventObjective_2.xml...

59. 2014-2015 Schedule Staging available now School Portal Opens: August 1 School Portal Closes: September 30 Status e-mail sent to CI Primary Admin, Curriculum Dean, and School Dean: September 30 Data clean-up/reporting starts: November 1

62. Resources Curriculum Inventory Implementers’ Listserv: http://medbiq.org/mailman/listinfo/ciimplementer_medbiq.org http://medbiq.org/mailman/listinfo/ciimplementer_medbiq.org Competency Framework Specs and Schemas: http://www.medbiq.org/working_groups/competencies/index.html http://www.medbiq.org/working_groups/competencies/index.html AAMC Curriculum Inventory Developer Resources: https://www.aamc.org/initiatives/cir/about/349624/fordevelopers.html https://www.aamc.org/initiatives/cir/about/349624/fordevelopers.html Physician Competency Reference Set URIs: https://www.aamc.org/external/348732?url=https://www.mededportal.org/icollaborati ve/resource/793 https://www.aamc.org/external/348732?url=https://www.mededportal.org/icollaborati ve/resource/793 MedBiquitous: www.medbiq.org www.medbiq.org Curriculum Inventory Working Group: http://www.medbiq.org/curriculum_inventory Competencies Working Group: http://www.medbiq.org/working_groups/competencies/index.html http://www.medbiq.org/working_groups/competencies/index.html ‘For Developers’ Section of www.aamc.org/cirwww.aamc.org/cir  Sample XML file and Verification Report from UCSF  CI XML Template from UVA  Slides/Recordings from Past Training Webinars  Registration Developer Webinars

63. Questions? Use Chat to submit questions. Please send to All Panelists. www.aamc.org/medaps