1. Learner Centered Feedback – ADAPT Part 1
Shannon Waterman, MD
Swedish Family Medicine Cherry Hill
Seattle, Washington

2. Characteristics of effective feedback
Objectives
Characteristics of effective feedback
Review ADAPT model of giving feedback
Ask-Discuss-Ask (think sandwiches…)
Skills practice and cases
Welcome everyone! Thanks for making time to participate. Before I jump in I want to ask that those of you participating in a small group designate a member of your team who will serve as the typist or responder for your group as there will be several polls and requests for you to enter brief responses throughout this webinar.
Credit to Marah Gotschick for pin the tail on the donkey concept (was Chief at Children’s 2010, now practicing pediatrician in Anchorage.)
And I used this to convey a low-stakes approach to giving feedback. I recognize that medicine and residency are not a game, but it is a helpful simile to bring particularly new residents into a culture of giving/receiving feedback. As part of my feedback workshop with interns we play pin the tail as a way to keep them engaged. You’ll see visual elements of the game throughout the presentation. Some day soon this will no longer be a relevant metaphor, but I’m sticking with it as long as my audience gets the reference.
In 2013 I gave the first of these faculty development webinars on the topic of feedback. You can find that archived on the NDRL. Some of today’s content was taken from that presentation. You can also find there supplemental materials which include a link to Brene Brown’s TED talk on “The Power of Vulnerability” in which she explores ways in which our own discomfort with vulnerability can become a barrier in arenas that might include giving & receiving feedback. You’ll find a link to a New Yorker article by Atul Gwande on Coaching, among other resources. There have been subsequent talks on feedback from wise and experienced faculty including Lisa Johnson, formerly of Olympia’s Providence program, so please wander around the NDRL. We’re working to develop some guides or structure to the webinars that will allow you to access talks around a theme like Feedback, so stay tuned for that over the next year. This may be particularly helpful for newer faculty.
The only new resource I’m providing though a link is something called the One Minute Learner (a companion to the One Minute Preceptor – or 5 Microskills – that guide many of in our precepting.) It was developed by a group of faculty in Boston and I’ll mention that again later in this presentation.

3. Poll Faculty Experience
Let’s start by checking in with who is online with us today. The question is how long you’ve been doing faculty work? If you are viewing this in a small group, please just use the chat section of the webinar to list the years each member of your group has been teaching.

4. Poll
Training
I’m curious, especially for the newer faculty that may be participating, whether or not you received meaningful training in residency or fellowship or as new faculty, in giving feedback (let’s say more than an hour, for example.)

5. Feedback Is Not Evaluation
What words of feedback might he need?
What evaluation might he get?
Is generic praise helpful?

6. Feedback vs. Evaluation
Balanced
Looks toward the future
Coaching
Red ink in the margin
One-way
Reviews recent past
All-Star Voting
Final grade
Coaching: In 2011 Atul Gawande, a surgeon, wrote about the role of advanced practice coaching.

7. Even expert drivers with years of experience can benefit from this sort of feedback.

8. Poll Realms of feedback
About what aspects of family medicine do you find yourself giving the most feedback?

9. Elements of Effective Feedback
1. Expected, timely and routine
2. Based on first-hand information
3. Descriptive rather than evaluative
4. Focused on issues learner can control
5. Specific and concise
6. Private*
7. Reciprocated
Routine, iterative feedback takes an historically high stakes experience and makes it low-stakes. No longer fight or flight sensation of being called into principal’s office.
Descriptive – try to reserve judgement. Say what it is you see. Hold off on theorizing about agendas or motivations.
Specific – few things as unhelpful as generic praise. Becomes a pet peeve for many learners. “Good job.”
* - Judy makes the excellent point that in bedside rounds there can be great benefit to real time feedback in presence of the group – all learners benefit from one another’s growth – and hopefully she will speak to the power of creating a culture or environment of trust that allows for that in her September webinar.
Ende J. Feedback in Medical Education. JAMA. 1983

10. What do we know about feedback?
Learners want more feedback
Learners and faculty perceive feedback differently
Feedback can be effective
Communicating effective feedback is complex
Direct observation is a pre-requisite to feedback
Recently the UW’s GME office presented a half-day workshop on “A new paradigm for feedback: learner engagement and the intentional learning environment” April 12, 2016
Presenters: Judy Pauwels, MD, Kris Patton, MD, Susan Johnston, PhD, Eileen Klein, MD, Tyra Fainstad, MD

11. Poll
Barriers?

12. Barriers to feedback Vulnerability (want to be liked, avoid conflict)
Limited time actually observing (“table rounds”)
Poor observers, unable to “unpack” our observations
Limited time
Different capabilities of learners at different levels
Myth (generational, hold-over among Baby Boomers?)
Adult learners do not need feedback.
“I never got any feedback, so why should you?”
Giving effective feedback is NOT an innate skill for most physicians
Brene Brown’s TED talk “Power of Vulnerability” may foster your own awareness of your hesitancy. I think this can be a more common barrier for certain personality types, as well as something more commonly seen in younger, less experienced faculty. There is very thoughtful commentary in the literature and highlighted in the American College of Physicians about the evolution as a teacher from a fairly ego-centric stance (worried whether learners think I know enough, if I have any street cred or do residents consider me a reliable clinical resource?) to a focus on ensuring the learner is getting what they need.
Lack of appropriate feedback leads to:
Loss or neglect of desired learning
Retention of incorrect learning or bad habits
A false sense of competence

13. Medicine’s Learning Culture
Multiple supervisors for short periods of time
Faculty often asked to evaluate AND coach
Faculty asked to evaluate too many specifics
Limited opportunities for direct observation of learners in action
Culture values efficiency and autonomy
Complexities of a combined working and learning environment
“A new paradigm for feedback: learner engagement and the intentional learning environment” April 12, 2016
Presenters: Judy Pauwels, MD, Kris Patton, MD, Susan Johnston, PhD, Eileen Klein, MD, Tyra Fainstad, MD

14. Prepare Observe Ask Discuss Plan Together
ADAPT model
Prepare Observe Ask Discuss Plan Together
The ADAPT model is a model that Dr. Judy Pauwels and others at the University of Washington GME office are developing and promoting as one way of taking a learner-centered approach to feedback and evaluation. Today’s webinar is the first of a two part series to outline the ADAPT model and some of the advantages and challenges to working in it.
Dr. Pauwels will follow up with September’s Faculty Development Webinar for Part II of learner-centered feedback and help provide insights about the importance of establishing a relationship with those to whom we give feedback, and creating a learning culture based on trust and frequent feedback that allows for planning together for the “next steps” in practice. She’ll look at the importance of our preparation and observation as well, and has experience using bedside rounding as a safe context for individual feedback. So please tune in again in September as she picks up this conversation and offers another perspective.

15. Prepare Observe Ask Discuss Plan Together
ADAPT model
Prepare Observe Ask Discuss Plan Together
I’m going to focus primarily on the ADA portion of the ADAPT model, review characteristics of effective feedback and look at an example or two of how to apply that. I also think that the advantage of these webinars is that it brings together faculty with a range of experience to share with one another and so we’ll take some time to check in about your own experiences.

16. Poll
Bedside Rounds
How often does your inpatient team round as a group with all members?
Possible responses: Daily, a few times a week, once a week, or we don’t round at the bedside as a group.

17. Poll Opportunities for observation & Frequency
Part I: In which clinical contexts do you observe your residents directly? [I want you to ignore times you’ve precepted and just heard presentations, and even those when you’ve gone in to wave at patients without observing meaningful elements of the resident’s exam or closure of a visit, for example. I’m focusing on times when you’ve had a meaningful opportunity to observe a resident in the midst of a clinical encounter.]
Coaching/shadowing
Behavioral health, therapeutic interviewing, etc.
Inpatient admissions and physicals
Inpatient daily rounding (1:1, not as a whole team)
Outpatient specialty clinics (procedure clinic, colposcopy, etc.)
Family conferences or Goals of care meetings
Other – free text
Part II: On average, how many times in a typical 2 week period do you directly observe residents in clinical care [again beyond a phoned review of H&P or clinic based precepting when you didn’t see the patient or only saw them cursorily primarily for the purpose of billing or confirmation of a rash diagnosis, for example?]
Use chat box or free text

18. Abridged history of feedback
The Old Feedback Sandwich
The New Feedback Sandwich
Demo both
Praise / Criticism / Praise
Ask / Tell / Ask
Ask/ Discuss /Ask

19. Ask learner to assess own performance.
Ask – Discuss – Ask
Ask learner to assess own performance.
Have you seen a patient like this before?
What went well?
What could have gone better?
Begins a conversation
Assesses learner’s level of insight
Promotes reflective practice
Establish expectations at the beginning – your preceptors/faculty will tell you about things you’re doing well and things that you can improve upon. That is feedback. It may not come with that label, but that’s what it is.

20. Ask - Discuss- Ask Discuss what you or the learner observed
React to the learner’s observation
Feedback on self-assessment
Include both affirmative and corrective elements
“I observed….”
Give reasons in the context of well-defined shared goals
“You want to become more skilled with cervical exams...”

21. Ask – Discuss – Ask Ask about learner’s understanding. “Teach-back.”
Explore strategies for improvement.
“What could you do differently?”
Replay relevant part of encounter
“Show me how you might phrase…”

22. Reinforce positive behavior – catch them doing something right
General Strategies
Reinforce positive behavior – catch them doing something right
“I appreciated how you incorporated the family into your presentation this morning.”
Redirect negative behaviors
“I’d like to give you feedback on your presentation. When there is a family present on rounds, be sure to start with an introduction of the team.”

23. Feedback: Be Descriptive
Vague: “You relate well to patients.”
Specific: “How did that go for you?
When you asked who would care for her dog when she is admitted for surgery, I saw her visibly relax. Your caring insight helped change the whole tone of the conversation.
Did you notice that change in her affect? Was that a natural thing for you to do?”

24. Environment provides feedback
Patient and patient’s family
Peers (student, residents)
Staff and consultants
Own personal perception*
*To improve accuracy of learner’s “personal perception,” you might ask them to pay attention to environmental cues rather than being directive about specific actions.
It was revealing to me when I polled our incoming interns this year about sources of anticipated feedback in residency, and not one identified clinic precepting as a source of feedback. It may be helpful for our residents to be absolutely explicit during clinic orientation that every precepting encounter is indeed feedback. It is commentary and guidance on communication, on professionalism, on diagnosis and management, and the list goes on. Every clinical encounter contains feedback from patients and families as well, not just feedback from attendings or senior residents.

25. How good are we? Terrible.
Huge discrepancy between perception of attending giving feedback and resident or student learners in multiple specialities.
MSIII students, Internal Med rotation at Ohio VA
1 hour bedside history & physical observed (58 sets)
Feedback given, questionnaire completed
Perception of time spent on feedback similar (26 minutes)
Only 34% agreement in content of the feedback
Sostok M, Coberly L, Rowan G. Feedback Process between Faculty and Students. Acad Med. 2002;77(3), 267.

26. Why learners don’t “hear” feedback?
Receiver doesn’t:
Recognize feedback when it is given
Understand the message
Reflect on the meaning
Giver:
Doesn’t make time to give feedback
Gives feedback in public setting (shaming, humiliating)
Vague examples
Interplay between giver/receiver:
Heard and taken as personal criticism…personality or style conflict…distrust based on gender/culture/race/power
Precepting is the prime example of feedback that isn’t recognized as such.

27. Prepare Observe Ask Discuss Plan Together
ADAPT model
Prepare Observe Ask Discuss Plan Together
So we’ve looked at what an ASK-DISCUSS-ASK might look like, and we’ll plan on taking a look at the preparation and follow-up elements of this model when Dr. Judy Pauwels picks this back up in September. I did want to return to the resource I mentioned, however, The One Minute Learner. The link provided is to a one page handout intended to be copied, folded over or cut out and laminated and made available to faculty (you’ll see there is a Preceptor side), and to clerkship students or residents (you’ll note there is a Learner side) who may be in a context or rotation where it would benefit the learner and their community preceptor to quickly identify their experience and their goals. It is a way of empowering learners to make the most of their precepting sessions and request the highest quality, specific feedback in line with their goals.

28. Objectives
Characteristics of effective feedback
Review ADAPT model of giving feedback
Ask-Discuss-Ask (think sandwiches…)
Skills practice and cases

29. Your feedback is welcomed!
My own experience of giving and receiving feedback, and helping residents to do the same, continues to evolve and I welcome your thoughts about aspects of this topic you think were missed, could be expanded, feedback related topics you’re curious about and so on. Please me with your thoughts while they’re fresh today if you have experience to add to this.

30. Cases – Resident inpatient service
You sit down with the intern for a feedback session at the end of your inpatient week.
You begin with some of the things she has done well, then turn to the areas she needs to work on. Team feedback (and evaluation) suggests she is performing at a solidly average level.
You bring up several patients whose past medical history she had not investigated adequately and comment that she needs to be more succinct in her write-ups.
If there is time (unlikely!) I can leave folks with a case to discuss in their small groups or consider individually.

31. Cases – Giving intern feedback
The resident gets angry. “There is never anytime for me to actually spend with the patients.”
She considered it great time management and commitment on her part that she would wake patients up at 4am to get a more complete history.
She also feels that you have not helped her or understood that she was taught to “write long notes.”

32. Cases – Giving student feedback
What went wrong? How could you have made this a more effective evaluation session?
“ You need to continue working on your efficiency. You improved substantially by keeping a list and learning to prioritize better, but you still need to work on shortening your notes.”
Is this good feedback?
Is this effective feedback?