Curriculum Inventory Administrators’ Group December 9, 2015 Terri Cameron, MA, Director, Curriculum Programs

Audio:  You will hear the audio through your computer speakers. Please make sure your computer speakers are on and the sound is turned up.  If you still have no sound once the webinar starts, please click on the audio broadcast icon ( ) located in the Participants Panel on the right hand side of your screen. Housekeeping  Please use the Q&A panel located on the right hand side of your screen to submit your questions throughout the webinar. Send to All Panelists. If you experience any technical or audio issues during the webinar, please send a message through the Chat panel to AAMC Meetings. Questions :

Please use the Chat panel located on the right hand side of your screen to submit your questions. Send to All Panelists. Questions

CI Upload Statistics December CI in Context Updated CI Standardized Vocabulary Curriculum Structure Survey? Featured Report: Assessment of Professional Behavior Curriculum Inventory Report Usage Statistics Curriculum Inventory Report Template (New and Improved) CI at 2016 Medical Education Meetings Medical School Highlight: Vanderbilt University School of Medicine Anderson Spickart, III, Assistant Dean of Educational Informatics and Technology Next meeting: Wednesday, December 13, 1 pm ET Agenda

CI Upload Statistics As of October 31 (final): Verified: 135 US MD: 127 CA MD: 5 US DO: 3 Acad Lev # Schools > Academic Level Distribution by Length (Month Increments)

CI Upload Statistics SB Location Academic Level # Schools Reporting Min SBsMax SBsAvg SBs ClerkshipAll Clerkship Clerkship Clerkship Clerkship PreClerkshipAll PreClerkship PreClerkship PreClerkship Sequence Block Summary

CI Upload Statistics Event Summary Event Location Event Type Number of Schools Reporting Events Minimum Events Maximum Events Average Number of Events ClerkshipALL Clerkship Non- Assessment PreClerkshipALL PreClerkshipNon- Assessment

CI in Context: December Importance of Benchmarking Reports for Curriculum Continuous Quality Improvement Robert Noiva, PhD, and Robin Rivest, MS, Oakland University William Beaumont School of Medicine

Curriculum Inventory Standardized Vocabulary Update

Independent Learning: Instructor-/ or mentor-guided learning activities to be performed by the learner outside of formal educational settings (classroom, lab, clinic) (Bowen & Smith, 2010); Dedicated time on learner schedules to prepare for specific learning activities, e.g., case discussions, TBL, PBL, clinical activities, research project(s). Synonymous with/Includes: Independent Study; Self-Study, Homework Does not include: Self-Directed Learning Self-Directed Learning: Learners taking the initiative for their own learning: diagnosing needs, formulating goals, identifying resources, implementing appropriate activities, and evaluating outcomes (Garrison, 1997; Spencer & Jordan, 1999). Does not include: Independent Learning; Independent Study

Curriculum Structure Survey? What is the curriculum structure at your school? Case-based Organ System Integrated Presentation-Based Systems-Based Theme-Based

Featured CI Report: Assessment of Professional Behavior

Curriculum Inventory Reports are accessed over 1,250 times per month Accessed over 1,000 times in 2015 Jan 2015Feb 2015Mar 2015Apr 2015May 2015Jun Jul 2015Aug 2015Sep 2015Oct 2015Nov 2015YTD Total

Curriculum Inventory Reports are accessed over 1,250 times per month Accessed 710 times in 2015 Jan 2015Feb 2015Mar 2015Apr 2015May 2015Jun Jul 2015Aug 2015Sep 2015Oct 2015Nov 2015YTD Total

Curriculum Inventory Report Request Template Drop-down will include ONLY participating schools Will be adding options to include data from: All Schools US MD CA MD US DO

CI at 2016 Medical Education Meetings Upcoming deadlines: MedBiquitous Conference (May): Deadline December 15 Med Ed Meeting (September): Deadline December 21 Not yet announced: GIR Learn Serve Lead Past deadlines: AACOM CCME GEA Regional Meetings IAMSE

22 Capturing Medical Student Performance Data to Guide Personal Learning Plans Vanderbilt School of Medicine Anderson Spickard, III, MD, MS Toufeeq Ahmed, MD, PhD Kim Lomis, MD Bonnie Miller, MD Transforming IT infrastructure to support major curricular reforms. Tch Learn Med. In press, Jan 2016.

Competencies Activities Experiences Activities Experiences Assessments Data Repository Vanderbilt Healthcare Learning System Reflective Summary Personal Learning Plan Reflective Summary Personal Learning Plan Learning Objectives Guided Self-Assessment Guided Self-Assessment Student View ePortfolio Artifacts Assessments Student View ePortfolio Artifacts Assessments Administrative View Program Evaluation Administrative View Program Evaluation

Inter-departmental courses Individualized Vanderbilt Curriculum 2.0 First year Second year Third and fourth year Continuity clinic

PC1 History and Physical Exam PC1a-HistorySub-optimalEntryAspirational Thoroughness of history Not engaged. Not focused on patient interaction. Performs part of a history or a basic history. Important information may be lacking, or may be overly inclusive. Performs a reasonably complete history. Gathers necessary elements to arrive at a correct diagnosis or short differential diagnosis. May be overly inclusive. Performs a complete history. No major information is missed; perhaps a few small details forgotten; may be overly inclusive. Targeted and appropriate history, including pertinent positives and negatives. Thoroughly and efficiently elicits patient’s history. Performs part of a history or a basic history. Important information may be lacking, or may be overly inclusive. Organization of history Disruptive when asking questions or entering the conversation. Captures fragmented information without an intentional approach. Disorganized approach, or heavily relies upon a template, but eventually captures pertinent information. Some organization, with some reliance on template or notes. Misses some helpful information or broadly inclusive without focus. Organized, linear approach to history taking; able to independently obtain sufficient data with minimal reliance on template or checklist. Organized, linear, hypothesis-driven approach. Able to elicit all important aspects of HPI, medical history, current medications, family and social history.

Competencies Activities Experiences Activities Experiences Assessments Data Repository Vanderbilt Healthcare Learning System Reflective Summary Personal Learning Plan Reflective Summary Personal Learning Plan Learning Objectives Guided Self-Assessment Guided Self-Assessment Student View ePortfolio Artifacts Assessments Student View ePortfolio Artifacts Assessments Administrative View Program Evaluation Administrative View Program Evaluation

Assessments Grades Self-assessments Peer assessments Faculty assessments Allied health professionals Patient notes

Assessments Grades Self-assessments Peer assessments Faculty assessments Allied health professional assessments Patient notes

Assessments Grades Self-assessments Peer assessments Faculty assessments Allied health professional assessments Patient notes Automatic scoring of medical students’ notes to monitor learning in the workplace. Med Tch. 2014:36;68-72.

Competencies Activities Experiences Activities Experiences Assessments Data Repository Vanderbilt Healthcare Learning System Reflective Summary Personal Learning Plan Reflective Summary Personal Learning Plan Learning Objectives Guided Self-Assessment Guided Self-Assessment Student View ePortfolio Artifacts Assessments Student View ePortfolio Artifacts Assessments Administrative View Program Evaluation Administrative View Program Evaluation

Faculty Review Portfolio Coach Faculty Review Portfolio Coach Joint Review Reconcile any differences in interpretation of the evidence Joint Review Reconcile any differences in interpretation of the evidence Student ePortfolio Student View I nformed self-assessment Personal Career goals Student View I nformed self-assessment Personal Career goals Personal Learning Plan Promotion Committee

Example

Student 1 Student 2 Student 3 Student 4 Student 5

Competencies Activities Experiences Activities Experiences Assessments Data Repository Vanderbilt Healthcare Learning System Reflective Summary Personal Learning Plan Reflective Summary Personal Learning Plan Learning Objectives Guided Self-Assessment Guided Self-Assessment Student View ePortfolio Artifacts Assessments Student View ePortfolio Artifacts Assessments Administrative View Program Evaluation Administrative View Program Evaluation

Administrative View Program Evaluation Administrative View Program Evaluation Data Repository

First Semester PDx (PDx Instructors) Mid-Clerkships ( Master Clinical Teachers ) End-clerkship OSCE (Faculty Observers) Developmental Progression

End-Clerkship Year Multi-station OSCE assessment RIME Assessment

Mid - year Receptivity to feedback

Mid - year Self-awareness

% of Student-Portfolio coach pairs congruent on all 6 ACGME domain assessments when they met

Next - EPA assessments via mobile Curriculum development for the workplace using Entrustable Professional Activities (EPAs). AMEE Guide No.99. Med Tch. Early online, July 14, 2015;1-20.

Education Informatics Division of VUMC Informatics Center under the leadership of Dr. Toufeeq Ahmed School of Medicine under the leadership of Deans Bonnie Miller, Kim Lomis, and Amy Fleming Office of Educational Informatics and Technology under the management of Jeremy Elgin Acknowledgements

“Best-of-breed” approach

VSTAR Platform

Technologies

Use ‘Chat’ to tell us what’s going on with you! Other updates from participants

Happy Holidays! Thanks so much for all you have done and continue to do on a daily basis to make the CI the success it is! We look forward to continuing to improve and enhance our data and reports in the coming year. Have a wonderful and blessed 2016!

Wednesday, January 13, 1 pm ET (Second Wednesday of each month, 1 pm ET) Registration Links for 2016 will be posted soon Please send agenda items to Next meeting: