Presentation on theme: "The Challenge and Importance of Evaluating Residents and Fellows Debra Weinstein, M.D. PHS GME Coordinators Retreat March 25, 2011."— Presentation transcript:
The Challenge and Importance of Evaluating Residents and Fellows Debra Weinstein, M.D. PHS GME Coordinators Retreat March 25, 2011
What keeps us up at night? Recruiting top candidates Protecting time for education Duty hours compliance Evaluating residents; providing feedback; dealing with poor performance
Why is this so important? Duty to trainees –promote professional development –help each reach his/her greatest potential Duty to society –ensure minimum standards of competence –promote development of accurate self- assessment, knowledge of individual limits
How do we know theres a gap? Internal reviews ACGME resident survey ACGME citations Individual cases
Where are we falling short? Many assigned evaluations are never done Quality of evaluations is variable –Important questions not asked –Only ratings / no commentary –Intentional avoidance of critique Evaluations not consistently reviewed with trainees 360-degree evaluations not implemented
Goals Protect current and future patients –Ensure appropriate balance of supervision and independence during training –Provide accurate references –Develop physician skills in self-assessment, understanding individual limits Optimize professional development and satisfaction of individual trainees –Focus on exceptional talent and potential as well as weaknesses –Utilize performance reviews to individualize education and career counseling
Trainee assessment must be able to identify the spectrum of problems encountered Insufficient academic progress: knowledge, judgment, decision making Insufficient technical skill Poor professionalism: dishonest, irresponsible, poor work ethic, inappropriate behavior Interpersonal conflicts Substance abuse Mental health issues Learning disabilities; ADD Other: poor organizational skills, carelessness, lack of productivity, etc.
PHS Evaluation/Feedback Task Force: Key Recommendations Stronger and more specific PHS policy Steering Committee to oversee and champion the effort Dedicated champion to implement and assess impact Faculty development, resident/fellow training, improved infrastructure Incentives/mandates for faculty
Faculty Development Initiatives Institution ABIM Hospital Visit Program, May 2011 Department Train the trainer sessions Grand rounds, retreats, new hire orientation Individual 1:1 coaching, role play
Resident/Fellow Training shift in culture: seek, accept and utilized feedback –Formative, not punitive! peer evaluation self-evaluation
OSCE (Objective Structured Clinical Exam) Objective: 1) Teaching 2) ?Early identification at risk trainees Focus: Communication/interpersonal skills; patient safety Plan: 100 incoming intern participates during 2011 orientation program Evaluate cost, feasibility, outcomes in considering future utilization
Targets for improved infrastructure Templates for 360-degree evaluations, can be customized by program Automated compliance reports from web- based evaluation system Electronic vehicle to immediately communicate a concern
How will we measure progress? Pre/post survey of faculty and trainees –knowledge, attitudes, confidence Pre/post comparison of evaluation completion rates, and quality Internal reviews and accreditation reports Discussion with program directors, coordinators, trainees and faculty
Key Elements of An Improved Assessment System Sufficient contact between trainees and faculty to allow for assessment Consistent completion of high quality evaluations Effective feedback to trainees Early identification of performance problems with prompt intervention Accessible resources for remediation
Coordinators Role: Work with Program Director to define this! May include: Address the need for faculty contact/continuity in scheduling Monitor evaluation completion rates; address non-compliance Help to design and implement plan for obtaining 360-degree evaluations Utilize and customize evaluation templates Help to ensure that residents understand the programs system for assessment: how is this communicated?