1. Evidence, guidelines and practice: the way forward in a digital age SYDNEY 11 April, 2013 Bill Runciman Professor – Patient Safety & Healthcare Human Factors – University of South Australia - Centre for Sleep Research Research Fellow, Australian Institute of Health Innovation, University of NSW Clinical Professor – Joanna Briggs Institute, Faculty Health Sciences – The University of Adelaide President, Australian Patient Safety Foundation Adam Dunn Senior Research Fellow, Australian Institute of Health Innovation - Evidence, warts and all -

2. - Evidence - Evidence is and includes everything that is used to reveal and determine the truth Evidence is and includes everything that is used to reveal and determine the truth Giving or procuring evidence is the process of using those things that are: Giving or procuring evidence is the process of using those things that are: Presumed to be true or are Presumed to be true or are Proven by prior evidence (truth) Proven by prior evidence (truth)

3. - Evidence - Eccles et al - Evidence from: Ia - meta-analysis of RCT’s Ib - at least one RCT II - at least one controlled study without randomisation III - comparative studies care-control studies correlation studies IV -consensus (expert reports/opinions)

4. - Consensus - “Experts”: self-selected, sometimes contaminated by vested interests commercial commercial professional professional personal personal financial financial Guidelines that don’t guide

5. - Evidence issues and effects - Over claiming (efficacy, safety) Proclamations of “new standard of care” Over-diagnosisOver-testingOver-monitoringOver-treatingBias-dishonesty

6. - Evidence – Bias - dishonesty Design biases Design biases Publication biases Publication biases Reporting biases Reporting biases Synthesis biases Synthesis biases Marketing biases Marketing biases Media biases Media biases Denial/concealment biases Denial/concealment biases

7. - Design biases - Little comparative effectiveness research Little comparative effectiveness research Deliberate avoidance of unfavourable comparisons Deliberate avoidance of unfavourable comparisons Unethical placebo studies Unethical placebo studies Low doses of comparators for drugs Low doses of comparators for drugs

8. - Publication biases - Non-publication of negative results Non-publication of negative results Refusal to allow scrutiny of data Refusal to allow scrutiny of data Duplicate publications Duplicate publications Re-using subject data Re-using subject data

9. - Reporting biases - Industry influence Industry influence Shadow writers Shadow writers Results not those foreshadowed in trial registers Results not those foreshadowed in trial registers Unwarranted conclusions which do not match the results Unwarranted conclusions which do not match the results Partial or biased reporting leading to distortions (safety, efficacy) Partial or biased reporting leading to distortions (safety, efficacy)

10. - Citation biases - Citation distortions can create unfounded authority Citation distortions can create unfounded authority Unfavourable studies can be ignored Unfavourable studies can be ignored Supportive evidence can be over-cited Supportive evidence can be over-cited

11. - Synthesis bias - Search and citation criteria Search and citation criteria Interpretation of articles found Interpretation of articles found Selection of articles found Selection of articles found Creation of presumptions Creation of presumptions “Low level” evidence ignored or discounted “Low level” evidence ignored or discounted

12. - Other biases - Marketing biases Marketing biases off label prescribing off label prescribing direct marketing direct marketing Media biases Media biases Denial/concealment biases Denial/concealment biases Threats to “whistle-blowers” Threats to “whistle-blowers”

13. - What should we do? - Review the whole area Review the whole area Develop further checklists Develop further checklists Regulation ??? Transparency Regulation ??? Transparency Penalties ??? Discount studies Penalties ??? Discount studies Publicity ??? Publicity ??? Conflicts of interest ??? Conflicts of interest ???