2. Quasi-experimental study designs in evaluating medicines use interventions

3. Lloyd Matowe 1 & Craig Ramsay 2 1 Faculty of Pharmacy, Kuwait University 2 HSRU, University of Aberdeen

4. BACKGROUND  The RCT is seen as the gold standard methodology for evaluating medicine use interventions  There are situations where it is impractical to carry out RCTs  For example in situations where:  It is impractical to randomize subjects  Difficult to find an appropriate control  Intervention has been retrospectively implemented  Quasi-experiments are often used as an alternative

5. Simple before and after studies Time Intervention

6. Interrupted time series analysis Time Intervention

7. before Controlled before and after design BeforeAfter studycontrol control study Intervention

8.  There is suspicion that quasi-experiments are often conducted and analysed inappropriately

9. OBJECTIVE  The objective of this study is to evaluate the use of quasi-experimental designs in medicines use interventions

10. METHODS  A rigorous systematic review  Subset of a large study on the effectiveness of various policy implementation strategies  Search strategies:  EPOC, MEDLINE, EMBASE, HEALTHSTAR, SIGLE  Bibliographies of previously published reviews  Studies included: CBAs & ITS  Interventions: policies/ guidelines to improve medication use  Outcomes- Objective measures of provider behaviour and/ or patient outcome  Time series regression used to reanalyse serial data where reanalysis was necessary

11. RESULTS - 1  Search strategy identified 150 000 hits  5 000 hits identified as potentially relevant  285 studies included  44 studies were on medicines use  17 were CCTs or RCTs  27 studies were quasi-experiments  15 ITS  12 CBAs  24 studies were on multiple interventions  Study areas included: Antibiotics, antipsychotics, analgesics, GIT drugs and anti-hypertensives

12. RESULTS - 2  Time series Overall methodological quality poor Overall methodological quality poor 8 of 15 studies analysed inappropriately 8 of 15 studies analysed inappropriately Effect size different in all 8 studies Effect size different in all 8 studies Common errors included: Common errors included: -Failure to test for autocorrelation and trend -Analysis using non time series methods -Failure to ascertain intervention as independent from other interventions

13. Trends Time Intervention

14. RESULTS-3  Controlled before and after studies Overall methodological quality poor Overall methodological quality poor 7 of 12 had unit of analysis errors 7 of 12 had unit of analysis errors Other common errors included: Other common errors included: - Failure to protect against contamination - Failure to take baseline measurements - Failure to perform power calculation

15. before Controlled before and after design BeforeAfter studycontrol control study Intervention

16. SUMMARY  Quasi-experiments are often designed analysed inappropriately  Methods and design are often inadequately reported  Study methodological weaknesses are often inadequately reported

17. RECOMMENDATIONS  For ITS studies w e recommend: Use of time series regression Use of time series regression Pre-testing for autocorrelation and trend Pre-testing for autocorrelation and trend Ascertain that the intended intervention was independent of other interventions Ascertain that the intended intervention was independent of other interventions  For CBA w e recommend: Protection against contamination Protection against contamination Measurement of baseline characteristics Measurement of baseline characteristics Power calculation Power calculation  For both ITS and CBAs methodological weaknesses should be reported

18. ACKNOWLEDGEMENTS  I would like to thank Professor Jeremy Grimshaw, from Ottawa Health Research Institute and Center for Best Practices, University of Ottawa, Canada for giving me access to the studies used for this evaluation