1. 1 E*Value Module 2 GME E*Value Training, May 19-21, 2008 Office of Graduate Medical Education Evaluations Management Duty Hours Management

2. 2 Evaluations Management The E*Value Evaluations Module is a web-based system for Residency and Fellowship Training programs, creating efficiencies in time and data management. Programs using E*Value TM for evaluations management have their evaluation notifications, reminders and data compilation fully automated to make resident, faculty, alumni, activity, site and program evaluations easy. Core features of this system include: Resident, Educator & Alumni Evaluations Program Evaluations Course, Rotation & Site Evaluations Low Score Notifications Outcome and Trend Reporting Evaluation Anonymity and Confidentiality Performance, Score & Ranking Reports Automated Report delivery Reporting and Feedback in Real-time Office of Graduate Medical Education Overview

3. 3 Evaluation components User Activity Time Frame Schedule Evaluation Form Goals & Objectives document (if required) EVALUAT ION Office of Graduate Medical Education These are the key components required to establish a schedule in E*Value

4. 4 Add/Edit Schedule Office of Graduate Medical Education Ready to schedule your evaluations? Go to ADD/EDIT schedule …

5. 5 View Schedule Office of Graduate Medical Education This is the schedule view …

6. 6 Preview the evaluation form Office of Graduate Medical Education An evaluation form used by the trainee to evaluate his faculty

7. 7 Evaluation set up rules Evaluation Types –Trainee, Educator, Staff, Patient, Peer-to-Peer, Alumni, Activity/Rotation, Site, Competency, 360-degree, Conference, Mini CEX & other evaluation types Completed Evaluations – Review a permanent record of all evaluations completed On-the-Fly Evaluations – Praise and Concern cards - make impromptu evaluations available to users Creation – Create your own, or have AI create your evaluation forms Question Groups – Define question groups to address core competencies and objectives Photographs – See an image of the person you are about to evaluate at the top of form Survey Set-up – Store, manage, and retrieve all evaluations built for program Reciprocity Rules – Help prevent retaliation or payback evaluation reviews by blocking data Low Score Notification – Receive automatic notifications of low scores being given Oldest Evaluations First – Require evaluators to complete oldest evals first Suspend Evaluations – Suspend evaluations if they were mis-assigned Office of Graduate Medical Education Establish rules for your programs evaluation process

8. 8 Performance Reports Program Selection Menu – Login to the residency, fellowship or specialty E*Value program of your choice Individual reports – View reports about your performance – trainees and faculty can generate reports for themselves. Program-level Management – Program Directors, can generate performance by user, activity, people group, time frame Departmental Oversight – Compare performance between programs, activities, people groups, and time frames. Run reports for individual in multiple programs in the department via parent level access. Cross-Compare Residencies, Clerkships, Activities and People at Departmental Level Viewing Privileges – Customize viewing privileges by individual throughout entire organization Office of Graduate Medical Education Program Directors, Faculty and Trainees have access to their reports.

9. 9 Duty Hours & Violations Management E*Values duty hours module allows the trainee to track training time and the program administration to monitor the violations. The Duty Hours web-based Calendar can be managed to track duty hours and activities, and detailed ACGME and statistical reports can be generated in real-time. http://www.acgme.org/acWebsite/dutyHours/dh_index.asp Office of Graduate Medical Education

10. 10 Duty Hours Violations Rules Office of Graduate Medical Education Duty Hours Violations Rule 1. Less than 4 days off in 4 weeks (28 days) (concatenated results). Rule 2. 80 hour work week. Rule 3. Shift length exceeds 30 hours. Rule 4. Shift break less than 10 hours. Duty Hours Violation Rules established by the Accreditation Council on Graduate Medical Education (ACGME)

11. 11 Duty Hours Calendar Office of Graduate Medical Education The Duty Hours web-based Calendar can be managed to track duty hours and activities. Trainees Duty Hours calendar - open an time tracker entry screen by clicking on the date link

12. 12 Preview the time tracker screen Office of Graduate Medical Education A Trainee enters his duty hours and other time on this pop-up screen. Selection of a task is essential in TRACKING duty hours, non-duty hours, vacation and other time off. Home call can also be tracked here by adding an additional task type. Selection of an activity will ensure the time is mapped to a rotation activity appropriately. Entering a comment will serve as a note about hours and or potential violations.

13. 13 Duty Hours real-time data Office of Graduate Medical Education Duty Hours Data: A range of detailed ACGME and other statistical reports can be generated in real-time. Compliance and Violation Reports are available by individual trainee, groups of trainees, by activity, task or violation.

14. 14 Duty Hours Statistical Report Office of Graduate Medical Education

15. 15 Duty Hours & Educator Hours Management Monthly Hours Calendar – Quickly log shifts of various customized task types Default Settings – Save defaults by user of commonly worked shifts Shift Comments – Enter optional comments about each shift Edit Trainee & Educators Hours – View, log or edit hours for individuals Custom Activity & Task Types – Designate In-house, night float, call, vacation, or custom tasks Violation Red Flags – Trainees receive instant notification/ stop-check if logging a violation Duty Hours for GME – Hours can be fed to E*gme for GME reimbursement purposes Duty Hours Violations Reports – Identify and respond to trainees in violation Violations Rules Settings - Monitor compliance by ACGME, NY State rules ACGME Reports – Answers the 6 duty hours questions posed by ACGME part 7b (see notes below) Stats Reports – Manage hours & violations, notify trainees Compliance Report – Review number & percentage days logged over period of time, by trainee Notes – Part 7b of the Web Accreditation Data System asks for documentation on each trainee for: 1. Avg Hrs/Wk 2. Avg Days of In-house Call/Wk 3. Max continuous hrs/week; 4. How often resident worked 30+ hour shifts in last 28 days 5. How many off days in last 28 6. Avg hours per shift Log shifts of various types on the Web, monitor violations and statistics Office of Graduate Medical Education

16. 16 Contact GME Gitanjali Kapur – Educational Technologies Analyst E*Value consultation, training, technical support. kapurg@medsch.ucsf.edu kapurg@medsch.ucsf.edu Amy Day - Manager, Resident & Fellow Affairs Resident & Fellow Appointments, Credentialling. daya@medsch.ucsf.edu Lorenzo Woo – Director, Office of Graduate Medical Education Policies, Contracts, Program related info. wool@medsch.ucsf.edu UCSF Office of GME: 415-476-6542 Office of Graduate Medical Education