1. IAMSE 2014 Remediation of Core Medical Competencies Regina Kreisle, MD, PhD Carol Nichols, PhD

2. Faculty Regina Kreisle, MD, PhDrkreisle@purdue.edurkreisle@purdue.edu Professor of Pathobiology Course Director, Pathology and EBM Asst. Director for Curriculum Development Competency Coordinator Indiana University School of Medicine – Lafayette Statewide Competency Director – Medical Knowledge Carol Nichols, PhDCANICHOLS@gru.eduCANICHOLS@gru.edu Associate Professor Phase 1 Modules Co‐Director Medical Gross Anatomy Director Department of Cellular Biology and Anatomy Medical College of Georgia at Georgia Regents University

3. Competency Remediation What is a Competency? – Competencies encompass knowledge, attributes, skills, and attitudes necessary for a particular set of tasks or objectives What constitutes a competency deficiency? What should be done about it?

4. MSOP Report “…The goal of medical education is to produce physicians who are prepared to serve the fundamental purposes of medicine. To this end, physicians must possess the attributes that are necessary to meet their individual and collective responsibilities to society.”

5. MSOP Physicians must be altruistic, altruistic and truthful. Physicians must be knowledgeable about the scientific basis of medicine. Physicians must be skillful in communicating with and caring for patients. Physicians must be dutiful in working with other to promote the health of individual patients and the broader community.

6. ACGME Patient care Medical knowledge Practice-based learning and improvement Interpersonal and communication skills Professionalism Systems-based practice

7. Competency-Based Education Implies that skills, attitudes, and behaviors are as important as knowledge base Incorporates curricular components and formal assessments for each desired competency Competencies are woven throughout curriculum

8. The Competency Agenda Overt agenda: provide better preparation in all aspects of knowledge, attitudes, and skills necessary for the future practice of medicine Covert agenda: identify (and potentially remediate) problems that would otherwise be missed in assessing knowledge base alone (usually in only about 5% of students)

9. Evaluation of Competencies Principle One: The competency curriculum in an integrated part of the entire curriculum. – Courses incorporate assessment of associated competencies – Benchmarks are set for promotion

10. Evaluation of Competencies Principle Two: Competencies must be formally evaluated in a summative manner. – Formal achievement results from demonstration of knowledge, skills, and attitudes. – Specific courses are asked to evaluate specific competencies in a formal manner. – These course evaluations are combined with other statewide means of assessment (NBME Exams, OSCE’s, etc.)

11. Evaluation of Competencies Principle Three: Students must receive regular feedback and multiple opportunities to demonstrate achievement. – Achievement should not be based on a single activity or assessment – Non-achievement usually triggers remediation – Failure to demonstrate satisfactory progress toward achievement of competencies can result in academic probation or even dismissal

13. 2014 CompetencyEventCurricular Location (PC) Patient CareRequired Clinical SkillsPromotion Requirement EOY3 OSCEGraduation Requirement BLSPromotion Requirement ACLSPromotion Requirement (ICS) Interpersonal Communication Skills Patient Feedback on CSNeurology Clerkship Required Clinical SkillsPromotion Requirement EOY3 OSCEGraduation Requirement (MK) Medical KnowledgeUSMLE Step 1Promotion Requirement USMLE Step2 CK, CSGraduation Requirement (P) ProfessionalismEthicsOb/Gyn Clerkship Peer and Self-AssessmentPromotion for Class of 2017* (PBLI) Practice Based Learning and Improvement LLL activityInternal Medicine Clerkship Case DiscussionsSurgery Clerkship (SBP) Systems Based Practice Community/ Family ProjectFamily Medicine Clerkship Tracking of Required Competency Based Activities *Peer and Self-Assessment will be subsumed into the new advising program for Class of 2018 and beyond

14. Practical Evaluation and Assessment Requires… Well-defined criteria Assessment tools that are – Matched for the competency and instructional method – Uniform – Manageable Feedback

15. Example – Effective Communication The Level One student will demonstrate: effectiveness in written communication of an informal nature, such as descriptive reports, history and physical examination write-ups, and tests. competence in oral communication in small groups or in one-on-one settings with an individual faculty member or with a patient who possesses no characteristics which would pose challenges to the student. the ability to communicate by e-mail, use word- processing and use bibliographic databases.

17. Practical Suggestions Assessments are easiest when done in smaller groups with personal contact – Labs, Problem-based learning or discussion groups, physical diagnosis preceptors, etc. Use uniform evaluation tools, especially when multiple faculty are assessing the same competency – Use detailed criteria and expected norms for satisfactory performance – Make evaluation tool as objective as possible

18. Practical Suggestions, Cont’d. Assess activities already in place whenever possible (rather than adding to the curriculum) – Oral presentations, reports, labs, etc. Use mid-term and formative evaluations with feedback – don’t surprise the students at the end Make sure students are aware of the criteria and your expectations

19. Remediating Deficiencies in a Competency-based Curriculum Regina Kreisle, MD, PhD Statewide Competency Director – Medical Knowledge Indiana University School of Medicine - Lafayette

20. Competency-Based Education For each of the six “core” competencies, criteria are established. Curricular components are developed to address and assess these criteria even in basic science courses. Different courses identify and develop different aspects of the core curriculum. Other benchmarks are required for promotion or gradation.

21. Evaluation of Competencies Achievement should not be based on a single activity or assessment. Non-achievement usually triggers remediation. Failure to achieve a single competency results in remediation. Failure to achieve multiple competencies may result in dismissal or repeat of a year.

22. Assessment of Deficiencies Requires clearly communicated expectations and requirements. – Competency criteria Early feedback to students. Documentation! Involvement of the student in identifying and correcting their own deficiencies.

23. Remediation – Years 1 and 2 Most common competency failure is failing a course (medical knowledge competency) – If one course is failed, student remediates in the summer Next most common is professionalism Isolated deficiencies occur with effective communication and basic clinical skills

24. Remediation – Years 1 and 2 Failure of a pre-clinical course automatically requires remediation in Medical Knowledge. Medical Knowledge is usually remediated by allowing the student a period of study followed by and examination. Failure of a competency component of a course or benchmark assessment may result in an isolated deficiency.

25. Remediation – Years 1 and 2 Isolated deficiency – remediates specific issues, usually with help of Statewide Competency Director Complex or multiple deficiencies – student goes to Student Promotions Committee – Yes, we have been able to dismiss students for multiple competency issues, even in the first year

26. Remediation – Years 1 and 2 Isolated deficiencies may uncover underlying specific issues Multiple deficiencies almost always involve health, mental health, learning disabilities, or motivational issues Issues identified in years 1 and 2, but not resolved, nearly always get worse during the clerkships

27. Statewide Competency Director Consulted when specific competency issues arise Assist in defining deficiencies and developing remediation plans

28. It Works! We have remediated issues in students who were otherwise passing to their benefit. We have helped students become aware of issues and change their behavior through behavioral contracts. We have helped students learn new skills such as dealing with anxiety, or discovering learning disabilities. Some students cannot be remediated and should not become physicians.

29. Remediation – Years 3 and 4 Many borderline issues become overt issues in years 3 and 4. Unfortunately, many issues are unresolved issues from the preclinical years. Students develop at different paces; not everyone can master the skills on schedule.

30. Remediation – Years 3 and 4 By far the most common isolated deficiency in the third year is failure of the end-of-clerkship exam. – Students must take time off to study and then retake the exam. – Failure of remediation requires repeat of the clerkship. Students can receive isolated deficiencies (except Clinical Skills) in any of the competency and must remediate before going to the fourth year. Failure to demonstrate adequate Clinical Skills results in failure and repetition of the clerkship. Students receiving deficiencies in multiple competencies can be dismissed.

31. ID-3: failure of medical knowledge competency (failed clerkship exam)

32. Multiple academic difficulties (ID-3’s) coupled with Professionalism Concerns

33. Major concerns with Self-Awareness and Self-Care

34. Issues in Successful Remediation Before you get to remediation… Multiple assessments and evaluations should be made directed at specific competency requirements and objectives. Assessments should be made by more than one individual in more than one setting. Students should receive both formative (low risk) and summative (high risk) evaluations. Students need continuous feedback, not just a summary evaluation.

35. Issues in Successful Remediation Passing requirements should be clearly defined (and be defensable). These should be communicated early and often. Unsatisfactory evaluations should be accompanied by identification of specific deficiencies. There should be enough time between the evaluation and academic consequences to allow for remediation.

36. Issues in Successful Remediation Components of Remediation Identification of specific deficiencies preferably linked to competency criteria and objectives. – Link evaluation tools to specific criteria and objectives – Insist on specifics with unsatisfactory evaluations – Collect as much information as is needed Deficiencies (and successes) need to be communicated in a timely manner along with supporting evidence.

37. Issues in Successful Remediation Evaluation and consequences must be consistent with stated policies. Students need a chance to respond to the documentation of deficiencies. Students and instructors should develop a plan that agrees on specific objectives and means for meeting those objectives. Re-evaluation of the competency should take place that is consistent with the goals of the remediation process.

38. Issues in Successful Remediation Problems crop up when… A deficiency is identified, but not defined. – Insufficient information is provided by the evaluator – Competency criteria or objectives are insufficiently defined The deficiency is not properly documented. Students are not informed of deficiencies until late in the process.

39. Issues in Successful Remediation Students are not involved in the identification of remediation goals or methods of remediation. Remediation is identified as punishment, not as an opportunity for improvement. The student refuses to accept that a deficiency exists.

40. Our experience to date... Students rise to our expectations Early assessment means early remediation We find problems exams can’t find and can insist on remediation of non-academic issues Students take competencies seriously New approaches to the problem student Students take remediation opportunities to better themselves Multiple competency problems = failure

41. Conclusions A successful remediation strategy begins with clear competency criteria and relies on appropriate, timely student assessment and feedback Documentation of deficiencies, particularly from multiple evaluators is important Early identification of isolated deficiencies is important – Learning issues – ADHD, dyslexia, test anxiety – Anxiety, self-confidence issues, time management – Self-awareness and self-care, relationship issues

42. Conclusions Global issues such as personality disorders and motivation issues are a bigger problems, but best identified early Early remediation of small problems may avoid bigger issues in clinical training Remediation should be seen as opportunities for growth, not punishment or corrective action Students need to be actively involved in their remediation planning