Presentation is loading. Please wait.

Presentation is loading. Please wait.

Evaluating the impact of assessing doctors’ performance Dr Julian Archer NIHR Career Development Fellow & Clinical Senior Lecturer in Medical Education,

Similar presentations


Presentation on theme: "Evaluating the impact of assessing doctors’ performance Dr Julian Archer NIHR Career Development Fellow & Clinical Senior Lecturer in Medical Education,"— Presentation transcript:

1 Evaluating the impact of assessing doctors’ performance Dr Julian Archer NIHR Career Development Fellow & Clinical Senior Lecturer in Medical Education, Director of CAMERA, Consultant Paediatrician

2 Evaluating the impact of assessing doctors’ performance Outline Why evaluate impact of assessment? Why evaluate impact of assessment? Programmatic research into medical revalidation Programmatic research into medical revalidation Some earlier work GMC revalidation evaluation framework

3 Why evaluate impact? APA Validity framework – Five headings Content Content Response process Response process Internal structure Internal structure Relationship to other variables Relationship to other variables Consequences Consequences Medical education Predominately all validity areas except consequencesWhy? Consequences are Complex Socio-political Not positivistic

4 EVALUATING IMPACT OR ‘CONSEQUENCES’ Programmatic approach to evaluation

5 What is Revalidation? What is the meaning of Revalidation? The Literature Policy makers What are the intended and unintended consequences of Revalidation in practice? Ward/Theatre /Lab GP Appraisees & Appraisers Focus groups To provide an understanding of Revalidation translational to policy, education and practice To shape and propose positive consequences while minimising negative consequences of Revalidation To disseminate positive responses about Revalidation in order to shape future national policy Stage 1 - PolicyStage 2 - PracticeStage 3 - Public Private Practice Research Questions Setting Method Systematic Review Interviewee drawings Methodology What is the impact of Revalidation on the relationship between the profession and society? Patients Journalist Interviews Analysis of Media Outputs Visual anthropology Mental Health Patient Focus Groups Video appraisals Appraisee Interviews Media Activity Theory Conversation Analysis Ascertain the messages that the public are getting from the media Develop communication about Revalidation more cost effectively Discourse analysis

6 STAGE 1 What is Revalidation in policy? - To explore the origins, definitions, and potential purpose of Revalidation

7 Methods Policy review starting from the Merrison report (1975) starting from the Merrison report (1975) 31 unique elite interviews leading members of Revalidation policy development ranged from the leaders past and present of main stakeholder bodies such as the GMC, AoMRC, BMA, NHS Employers, 4 DHs leading members of Revalidation policy development ranged from the leaders past and present of main stakeholder bodies such as the GMC, AoMRC, BMA, NHS Employers, 4 DHs

8 Discourse

9 DEVELOPING EVALUATION FRAMEWORKS Working with the GMC

10 Methods – evaluative cycle Literature Interviewed major stakeholders Worked with GMC (Main stakeholder) Develop evaluative questions Develop possible methods for an initial pilot Step 1: Engage stakeholders Step 2: Describe the programme Step 3: Focus the evaluation question Step 4: Gather credible evidence Hansen HF. Choosing Evaluation Models: A Discussion on Evaluation Design. Evaluation. 2005;11:

11 Stakeholder interview themes Importance of evaluation “people who go through the process…without any thought of what it actually represents and how it might actually add value to the system” “people who go through the process…without any thought of what it actually represents and how it might actually add value to the system” What is revalidation for? Drive-up standards (performance improvement) vs. picking out ‘bad apples’ Drive-up standards (performance improvement) vs. picking out ‘bad apples’Appraisal “to get inside the closed door of appraisal” “to get inside the closed door of appraisal” RO judgements “I’m not sure exactly how the RO’s decision is made” “I’m not sure exactly how the RO’s decision is made”

12

13 Summary Impact or consequences of assessment is an important part of validity evidence Complex Programmatic research required (drawing on theoretical perspectives)

14 Acknowledgements and thanks Colleagues Sam Regan de Bere Sam Regan de Bere CAMERA team CAMERA teamFunders NIHR NIHR The Health Foundation The Health Foundation General Medical Council General Medical Council

15 Evaluating the impact of assessing doctors’ performance


Download ppt "Evaluating the impact of assessing doctors’ performance Dr Julian Archer NIHR Career Development Fellow & Clinical Senior Lecturer in Medical Education,"

Similar presentations


Ads by Google