Presentation is loading. Please wait.

Presentation is loading. Please wait.

Manual Evaluation: Are we doing what we think we are doing? Associate Professor Dee Mangin Director Primary Care Research Unit Department of Public Health.

Similar presentations


Presentation on theme: "Manual Evaluation: Are we doing what we think we are doing? Associate Professor Dee Mangin Director Primary Care Research Unit Department of Public Health."— Presentation transcript:

1

2 Manual Evaluation: Are we doing what we think we are doing? Associate Professor Dee Mangin Director Primary Care Research Unit Department of Public Health and General Practice Christchruch School of Medicine, University of Otago 20 May 2009 WHA

3 Overview Rationale for testing the manual in a pilot study Protocol for testing Timelines

4 Background WHO/HAI have developed a comprehensive teaching package comprising a manual and accompanying resource material as well as training for teachers implementing and this

5 Background It is planned to evaluate this in a pilot study to look at its effectiveness and at areas for improvement in the final manual WHY DO THIS STUDY?

6 Background Many educational resources and techniques are implemented without ever having any assessment of their effectiveness

7 So we dont know whether our efforts are effective

8 Overview Our wish to think that they are can be misleading…… Many strategies for medical education have little effect on prescribing when they are tested in studies

9 Evidence for prescribing education strategies that have been found effective The effect of dissemination only strategies such as didactic lectures and written information are small at best and of unknown clinical significance Farmer et al. Cochrane Database of Systematic Reviews 2008;3 Facilitated teaching with evidence based resources and audit and feedback have been found to have a moderate effect Richards et al Family Practice 2000 A previous WHO educational resource (WHO Guide to Good Prescribing) using facilitated teaching for medical students on pharmacotherapy showed a significant effect on students skills in a randomised controlled trial Impact of a short course in pharmacotherapy for undergraduate medical students: An international randomised controlled study. de Vries T Henning R Hogerzeil H Bapna J et al The Lancet 1995

10 Study Design Randomised controlled trial 10 volunteer sites 5 intervention sites 5 control sites Quantitative and qualitative methods to assess change in students knowledge attitudes and skills

11 Questions What is the effect on students –knowledge, attitudes and skills of students in understanding and responding to pharmaceutical promotion What are the strengths and weaknesses of the content of the manual and resources provided –assessed across a variety of settings

12 Data collection Before and after assessment of knowledge, attitudes and skills of students Questionnaire Focus groups Key informant interviews and content evaluation by teachers in the intervention group

13 Two comparisons between these randomised intervention and control groups The difference between the before and after manual assessments at the beginning and end of the year compared to the control sites The difference between their assessments at the end of the year and students at the end of the year in the previous year group Again compared to the differences at the control sites

14 Comparison 1 Change in scores at intervention sites compared to control sites 5 5 intervention sites control sites manual teaching as usual Before Before compared to After After difference

15 Comparison 2 Difference in scores between study groups at the end of the year and students at the end of the previous year 5 5 intervention sites control sites Students from previous year tested for comparison Students at end of study year Study start Manual implemented Teaching as usual

16 Comparison 2 Difference in scores between study groups at the end of the year and students at the end of the previous year 5 5 intervention sites control sites Students from previous year tested for comparison Students at end of study year Study start Manual implemented Teaching as usual

17 Volunteer sites Medical or pharmacy students Site prepared to teach all manual chapters Site able to teach within one academic year (12 months) Ethics approval will be gained where required

18 Volunteer sites Training, travel and accommodation will be provided for a teacher from each site The evaluation assessments and analysis will be organised and co-ordinated by the study team based at HAI These involve the before and after questionnaires and focus groups for students/interviews for teachers at each site

19 Volunteer sites: Timeline Plan to implement as academic years begin from later in 2009 through to 2010 Training for teachers from sites randomised to intervention in Amsterdam Sept 2009 Before assessments as academic years start After assessments completed late 2010 as years end Training for teachers from sites randomised to control in Amsterdam in Sept 2010

20 It is not new

21

22 We need to ensure medical and pharmacy graduates are well equipped to understand and deal with the effects of promotion on their prescribing decisions

23

24 It is important that we not just have good ideas But that we know whether our collective efforts are effective

25

26 Volunteers! - Pilot study - Testing the manual in your setting - Expression of interest forms at the back of the room OR - Email: lisa@haiweb.org


Download ppt "Manual Evaluation: Are we doing what we think we are doing? Associate Professor Dee Mangin Director Primary Care Research Unit Department of Public Health."

Similar presentations


Ads by Google