How to evidence quality Professor Jacky Hayden

How to evidence quality What measurement tools are currently in use What recording tools are currently in use A framework for educational audit Sharing and benchmarking

Why evidence quality?

"A long habit of not thinking a thing wrong gives it a superficial appearance of being right." Thomas Paine (1737–1809)

Why evidence quality? To ensure continuous quality improvement Financial To meet the GMC standards To meet the Deanery standards

Quality Improvement

Quality Improvement

Current reporting tools NACT report form Training programme/School report form Electronic reporting tools

Current tools in use for measuring quality GMC surveys End of placement questionnaire Specialty specific tools eg RCS Systematic feedback from ARCP/portfolio review Exam data Patient satisfaction surveys External review Deanery/College

Tools for measuring the quality of an interaction between a trainer and learner On a ward round In out patients In a tutorial In a joint consultation In theatre In a supervision meeting

What should we measure?

Lessons from Clinical Audit Structure Process Outcome

Focus energy on this group Easy to measure Focus energy on this group Important

Training and trainee/student reported outcomes The Local Education Provider (LEP) can demonstrate that each trainee/student has agreed and defined learning outcomes and that these are achieved. The LEP, working with the training programme directors and educational leaders ensures that educational supervisors are using educationally sound methods

Trainee/student and patient safety The LEP is able to demonstrate that all students and doctors in training work with safe supervision. The LEP works with training programme directors and educational leaders to ensure that the people responsible for workplace based assessments have been trained and calibrated and have time in job plans to undertake their role as well as receiving feedback and review through appraisal.

Trainee/student and trainer experience The LEP ensures that trainees/students and trainers are treated with respect and dignity   The LEP can demonstrate that the culture is one that fosters, team working, clinical engagement and patient focused care.

The Local Education Provider (LEP) can demonstrate that each trainee/student has agreed and defined learning outcomes and that these are achieved

Structure Process Outcome

Structure What evidence would a DME and TPD need to be confident that the learning opportunities of the post had been mapped to the curriculum?

Process What evidence would a DME or TPD need to be confident that the trainee and their nominated supervisor had agreed the specific learning objectives, based on the opportunities of the post and the past experience of the trainee?

Outcomes What evidence would a DME or TPD need: to be confident that the agreed outcomes have been achieved? to be confident that assessments have been robust? to be confident that any remaining objectives have been clearly documented?

Benchmarking

Graph of Programme total audit scores