Reinvigorating the Feedback Loop: Teaching the ARCH Model for Giving Feedback within a Residents-as-Teachers Curriculum Dennis Baker, PhD Amanda Cuda, MD Courtney Halista, MD Chase Hughes, MD Dina Kurzweil, PhD Karen Marcellas, PhD Carolina Merrigan, MD Dana Nguyen, MD Kristen Wyrick, MD Introduction of Team Members (Each Team Member to provide a short overview of role and location)

Disclosures This presentation does not represent the official opinions or methods of the U.S. Government, the Department of Defense, or the Uniformed Services University.

Objectives By the end of this session, participants will be able to: Recognize the importance of providing feedback Identify actions that facilitate effective feedback Recall the components of the ARCH model Demonstrate the use of the ARCH model to provide feedback Begin planning curriculum interventions 5 Minutes (Nguyen) - Introduction and Objectives: Introduce all presenters, briefly explain roles in development of current curriculum. Give overview for session/agenda review. Residents are largely involved in the education and oversight of medical students. However, structured faculty development sessions are lacking for resident teachers. One of the most important skills for any teacher is that of providing effective feedback to learners. This interactive symposium will share our experiences of implementing our new “Giving Effective Feedback” Resident-as-Teachers sessions from the background theory and curriculum development to the implementation of the actual sessions. We will use video demonstrations to contrast the spectrum of feedback quality using no model, using the “sandwich” model, and finally using the ARCH model. The participants will then have an opportunity to practice the model and debrief their experience. Finally, we will provide opportunity for collaboration so the participants can plan how they could implement a similar session in their programs. Multiple models of feedback exist. Rather than offer a theoretical overview of multiple models, we focus on one practical model, the ARCH model, conceptualized at Florida State University College of Medicine by Dennis Baker, PhD. Teachers at all levels will benefit from the symposia, as they can use this model to enhance the quality of their feedback to junior learners. Additionally, in this symposium, clerkship directors and residency curriculum leaders will enhance their own abilities to successfully plan, create, and execute effective curriculum interventions. By the end of this session, participants will be able to: Describe the background of feedback models Identify actions that facilitate effective feedback Recall the components of the ARCH model Demonstrate the use of the ARCH model to provide feedback Recognize the importance of providing feedback Begin planning curriculum interventions

Carolina Merrigan, MD CPT, MC, USA PGY2 Ft Carolina Merrigan, MD CPT, MC, USA PGY2 Ft. Belvoir Community Hospital, VA

Perspectives 10 Minutes (Merrigan)-Introduction of the problem—residents in general do not have the skills in providing effective feedback; review typical resident feedback video (USU); solicit input from the audience regarding scope of the problem 8 minutes - Mistakes when giving feedback, Share Experiences Reflections on feedback animated video (ReflectionsOnFeedback_final) - This animated video is designed to show common mistakes when providing feedback. Click the image in the PowerPoint to start the video; it will open in a new browser window. Minimize or close that window to return to the presentation. Play the video in full, and then facilitate a discussion from the participants about the “problems” identified with feedback in the video. Can they relate? As a learner? As a teacher? Is it hard to conceptualize or provide constructive criticism? To assist you, a key of feedback concerns is listed below. Scene 1 - resident provides vague feedback; when asked for something specific resident tells student to “read more” (students hate this!) Student 1 was told things were great but inside feels it was average and didn’t seem great. Also when student digs deeper for additional help/feedback everything remains vague. Resident needs to identify a specific issue and work with the student to make a plan to improve Scene 2 - resident provides “feedback sandwich” Two different issues can arise: 1) student focuses only on the good and does not hear the bad; or 2) student focuses only on the bad and does not hear the good. Also, sometimes when doing this, the resident skims quickly over the negative Resident needs to focus on being more clear that feedback is being provided, and provide positive feedback separate from negative feedback Resident also needs to be clear that student needs to work on something specific, and work with student on making a plan for how to improve. Scene 3 - Resident is too busy to provide feedback Student is left not knowing whether they did well or poorly, and what to sustain or improve Resident needs to make time for feedback!

Dina Kurzweil, PhD & Karen Marcellas, PhD Education Technology & Innovation Support Office Uniformed Services University Bethesda, MD

Planning for Success Education Technology & Innovation Support Office, USU Developing and Integrating the Team Distribution of Material to Facilitators Implementation/Presentation of Materials Back up Plans Dr. Kurzweil and Dr. Marcellas 10 minutes (Kurzweil/Marcellas)- integrating with a development team, review of resources required, technology needs, considerations for distribution of material across learning sites Developing and Integrating the Team  - This requires forward thinking and planning. Below are some of the elements you need to consider in order to put a team together: Identify what skillsets you need to develop your product. Possibilities include instructional design, graphic design, animation, video production Determine how will you get access to those skill sets and how you will pay for them. Develop as a team the central mission/goal for your project and a shared vocabulary Establish the lines of communication Determine how the team will review products Identify roles on the team Considerations include who will maintain the schedule, who will make final decisions, who needs to provide approval on which products Coordinate a project timeline that works for everyone Distribution of material to facilitators Review capabilities for receiving materials across sites. Look for the lowest common denominator to ensure equal capabilities. Implementation/Presentation of Materials Questions to consider: Can I hook my outside computer to the training site’s projection and internet systems? Most people like to do presentations from their own computer so that they do not have to rely on the Internet and so that they have the appropriate software available. If you have to use the training site room’s computer, it raises a number of additional considerations: Do the technical versions of the training materials work in the training environment?  Make sure you know what software is needed for your presentation (e.g., PowerPoint, media player) How will you access your presentation materials? Can you access a shared network directory? If not, does what external media can be used (e.g., CD or DVD or flash drive). Consider emailing the materials to yourself as a backup as well. What is the room layout? If you want participants to be able to work in small groups, you’d like to have a room where people (including facilitators) have space to move around easily What is the audio/visual setup of the room in which you will be presenting? How will your presentation be displayed? If they do not have a large screen for display, consider bringing a projector Will you be able to project sound to the room? If not, consider bringing your own speakers. Do they have consistent/reliable Internet access? If not be prepared and have everything downloaded to your laptop as well as a CD or DVD or Flash drive (if allowed). Back up plans -  What will you do if: Materials are missing Always have duplicate copies of everything and try to figure out a way to print onsite or consider a printing center like Fedex. Technology breaks down Have paper copies of everything and bring your computer with elements downloaded so in the worse case scenario you can bring everyone around your computer.

LTC, MC, USAR Uniformed Services University Bethesda, MD Amanda Cuda, MD, MPH LTC, MC, USAR Uniformed Services University Bethesda, MD

How it all Started SEPT 2017 FM Clerkship Site Directors Conference, USU Gathered 16 FM teaching faculty Engaged 2 PhD Education Specialists Dick & Carey and Kern for systematic instructional design March 2018 Video Filming and Editing Done University video studio Gig Harbor home “studio” Spring 2018 Piloted Session at 2 Sites JAN 2018 Facilitator’s Guide Completed Narrowed scope to feedback Iterated objectives Established tasks and team Wrote Facilitator’s Guide Script for video content April 2018 Completion of session “package” Forwarded to clerkship sites Google Drive Vimeo 10 Minutes (Cuda): Walk participants through our history of the process of curriculum/instruction development, review time required for developing instruction material. Best message is - doesn’t matter what model we use, just that you use a systematic process. This takes time and may want to prioritize and decide what is most important. Site director meeting as well as multimedia choices and interactive options. Complete a needs assessment.

Dennis Baker, Ph.D. Assistant Dean for Faculty Enrichment and Professor of Family Medicine Alabama College of Osteopathic Medicine & Emeritus Professor of Family Medicine and Rural Health Florida State University College of Medicine

“Some” Characteristics of Effective Feedback Considers emotions of teacher and learner Start with the learner via self-assessment Expected Right time Right amount Based on direct observation Specific and descriptive language Collaborative, goal oriented Packaged well Dr. Baker

“Some” Characteristics of Effective Feedback Considers emotions of teacher and learner Start with the learner via self-assessment Expected Right time Right amount Based on direct observation Specific and descriptive language Collaborative, goal oriented Packaged well Dr. Baker

ARCH Feedback Model A R C H Ask/Allow for self-assessment Reinforce things done well (KSA) Confirm areas needing correction Help learner with improvement plan A R C Dr. Baker H

ARCH Allow/Ask for Self-Assessment Create safe environment Ask/Allow for self-assessment Use self-assessment as base for RCH ARCH Reinforce Things Done Well (attitudes, skills, knowledge) Discuss learner’s self-addressed strengths first Explore how strengths developed Add strengths you have observed and why important Dr. Baker

ARCH Confirm Areas Needing Correction Ask for learner’s identified areas for correction/improvement Confirm & clarify via restating Add areas for correction you identified and check for understanding ARCH Help With Improvement Plans Dr. Baker Obtain learner’s input for improvement strategies Add suggestions to strategies Clarify goals & plans followed by documentation process

ARCH in Action Dr. Baker

Courtney Halista, MD Capt, MC, USAF Faculty Travis AFB, CA Chase Hughes, MD LT, MC, USN PGY4 Naval Hospital Camp Pendleton, CA

Let’s Practice! Courtney and Chase 10 minutes (Halista/Hughes) – Practice providing feedback using the ARCH model—individual skill development.   All participants will watch a standard scenario (USU video) and then work together in pairs to outline feedback to the student based on the ARCH model. (Do not have time to do the second scenario) (1 Minute) - Introduce activity and timeline (5 Minutes) - View patient encounter videos (click the links to view the videos). (4 Minutes) - Have students pair up and then write out main points of feedback individually and start thinking about what they would say to the medical student if they were the resident.

ARCH Feedback Model A R C H Ask/Allow for self-assessment Reinforce things done well (KSA) Confirm areas needing correction Help learner with improvement plan A R C Courtney and Chase You can use this slide to solicit audience discussion and recommendations for feedback based on the scenario they saw H

Dana Nguyen, MD, FAAFP LTC(P), MC, USA Associate Professor & Clerkship Director Uniformed Services University Bethesda, MD

Preparing for Implementation Facilitator’s Guide Purpose- To allow “anyone” to use content without consulting the design team Not a ‘word for word’ script Provides objectives and key information that needs to be covered Outline 10 minutes (Nguyen) Walk through facilitator guide as a background for building a facilitator guide, specific to structure and implementation rather than specifics of this material Review of Facilitators Guide components The facilitator guide should be designed so that somebody can download it and teach the session without consulting the design team. While it does not need to provide a “word for word” script, it should provide the objectives along with the key information that needs to be covered in order to meet the objectives. Below is the outline of the facilitator guide for our session General information about session Pre and Post Survey Objectives   Introduction Audience Timing - schedule for training event. Format Facilitator Resources Media Requirements Pre-Session Work Session Structure and Content References

Facilitators Guide Components General information Pre and Post Survey Objectives Introduction Audience Timing Format Facilitator Resources Media Requirements Pre-Session Work Session Structure Session Content References 10 minutes (Nguyen) Walk through facilitator guide as a background for building a facilitator guide, specific to structure and implementation rather than specifics of this material Review of Facilitators Guide components The facilitator guide should be designed so that somebody can download it and teach the session without consulting the design team. While it does not need to provide a “word for word” script, it should provide the objectives along with the key information that needs to be covered in order to meet the objectives. Below is the outline of the facilitator guide for our session General information about session Pre and Post Survey Objectives   Introduction Audience Timing - schedule for training event. Format Facilitator Resources Media Requirements Pre-Session Work Session Structure and Content References

LtCol, MC, USAF Uniformed Services University Bethesda, MD Kristen Wyrick, MD LtCol, MC, USAF Uniformed Services University Bethesda, MD

Putting It all Together 8 Minute video Role Play Script 2 Minute Video ARCH Card Video on TeachingPhysician.org STFM Education Column Full article submitted to Medical Teacher 45 minute session Facilitator guide Power Point 4 videos Handout Kristen Briefly review the final products we developed at each of our institutions. Dr. Baker while at FSU (left) and ours at USU (right). Not too much to talk about here but makes an easier transition as to facilitating a large group discussion instead of going abruptly from my last slide.

Time For You! Consider an educational need or concern for your program. Consider your local resources. Do you have education specialists and/or technology developers? What might this project cost? Who else should be on the team? Is this type of curriculum something that could be implemented at your institution? 15 minutes (Wyrick)- Lead audience in discussion/facilitation regarding application to own situation; if large group discussion ensues, Wyrick to facilitate; if participants want to work on planning their own product, all presenters will wander around and assist as resident/faculty subject matter experts

Please evaluate this presentation using the conference mobile app Please evaluate this presentation using the conference mobile app! Simply click on the "clipboard" icon on the presentation page.