Why should it be done? Hiring, promotion, and management decisions Help clinicians improve
Intrusion in clinician’s practice? Managers understand patient outcomes Practice profiles are constructed after the fact, when the patient is gone. Both the patient and managers can use benchmarked data Poor clinicians are bad for the patient as well as for the organization
How should the analysis be done? Compare clinician to average peer Compare clinician to average peer taking care of same kinds of patients Compare clinician to expectations on admission Compare clinician and peers on patients matched on certain features In benchmarking, a clinician's performance is compared to an expected value.
Compare Clinician to Average Peer Calculate peer providers and the clinician’s average and standard deviation Compare using test of hypothesis with unequal means Problem: Maybe misleading as providers see different kinds of patients and the clinician with more severely ill patients will naturally have worst outcomes
Example Data 123 internal medicine residents at the New York-Presbyterian Hospital in New York City. The outcomes examined included following outcomes: Patients' satisfaction measured by telephone interviews of at least 10 patients Disease-management profiles for average of 7 diabetes and 11 hypertensive patients. Patient's condition Frequency of use of various medications Faculty-evaluations on seven dimensions: History taking Physical examination Differential diagnosis Diagnostic and/or treatment plan Health care maintenance Act compassionately Team player
Event Tree for Clinician’s Patients Is Kept, Outcomes Change
Is it reasonable to benchmark clinicians? Measurement distorts goals Measurement leads to defensive behavior No adequate measure of severity maybe available Too much time spent on measurement and too little on improvement
How Should Benchmarked Data Be Presented? Before the meeting Schedule a feedback time and date as soon as possible. Check your data to make sure there are no errors. Add text, charts or graphics. Supplement numeric data with anecdotal information and the customer's voice (e.g. a short audio from a patient). Distribute handouts ahead of meeting to participants
How Should Benchmarked Data Be Presented? At the meeting Make it clear that the evaluation is confidential Make a brief introduction of the purpose of the session Acknowledge the limitation of the practice profiling method Present the data and not the conclusions Explicitly ask for clinician's evaluation of the data after each section of the report is presented DO NOT defend the practice profiling method, the benchmarking effort or any aspect of your work Thank the clinicians for their time and describe next steps
How Should Benchmarked Data Be Presented? After the meeting Summarize the comments and append it to the report. Describe resources available. Send a written report to each clinician. Ask the clinicians to comment on: What worked well and what needs improvement? Do they plan to change their practice and in what way? Was it worthwhile? Set the time of next benchmarking report. .
Take Home Lesson Expected Outcomes Can Be Benchmarked Using Severity of Patients’ Illness