1

2 Levels of evidence Ia Meta-analysis or systematic review of RCTs Ib Randomised trial IIa Controlled non-randomised study IIb Cohort study III Case-control study IV Descriptive study Non-systematic review Consensus report Leading article

3 Survey of 85 physicians (36% from internal medicine) ”Very important” in influencing my prescribing Own training and experience88% Scientific papers62% Advice from colleagues48% Detail men20% Drug adds 4% Patient preference 2% (Avorn J. Am J Med 1982;73:4-8)

4 ”Is impaired cerebral blood flow a major cause of senile dementia?” Yes71% No14% No opinion15% 32% found cerebral vasodilators useful in managing confused geriatric patients

5 Survey of 85 physicians (36% from internal medicine) ”Very important” in influencing my prescribing Own training and experience88% Scientific papers62% Advice from colleagues48% Detail men20% Drug adds 4% Patient preference 2% (Avorn J. Am J Med 1982;73:4-8)

6 Drug adds 287 advertisements for anti-hypertensive or lipid- lowering drugs promotional claims with references - 23 refs. unretrievable (data on file, monographs) - 45 of 102 claims not supported by reference (Lancet 2003;361:27)

7 Randomised trials Unclear randomisation method: - effect exaggerated by 30%, on average No blinding: - effect exaggerated by 14%, on average (BMJ 2001;323:42-6)

8 Outcome reporting bias 102 RCTs approved by the Copenhagen & Frederiksberg Ethics Review Committee and subsequently published Incompletely reported outcomes for meta-analysis: 50% for efficacy, 65% for safety

9 Outcome reporting bias Unreported outcomes 86% of trialists denied unreported outcomes despite evidence in publications & protocols Only half of the trialists responded to the question

10 Outcome reporting bias Full outcome reporting is associated with p<0.05 Odds ratio 2.4 ( ) for efficacy Odds ratio 4.7 ( ) for safety

Are primary outcomes consistent between protocols and publications?

Conclusions u Trial outcomes are often inadequately reported for inclusion in meta-analyses u Reporting of outcomes is biased to favour p<0.05 u Primary outcomes are omitted, changed, or newly-introduced in over 60% of trials Protocols should be publicly available

13

14 ‘Positive’ studies are more likely to be published than ‘negative’ studies u Hazard ratio for 130 clinical trials ‘positive’ (P 0.10) 3.13 (1.76 to 5.58), P= u Median time to publication 4.7 vs 8 years

15 Cochrane Reviews Freely available from

16 Based on Cochrane Reviews when possible

17 NSAIDs “Systematic reviews of RCTs have found no important differences in effect between different NSAIDs or doses but have found differences in toxicity related to increased doses and possibly to the nature of the NSAID itself.” “The only meta-analysis that found one drug to be more effective than another was funded by the manufacturer” Clinical Evidence 1999;2

18 Clinical Evidence (NSAIDs) We have favoured systematic reviews that have not been sponsored or authored by industry... it is easy to seemingly follow the rules for systematic reviews and yet adopt inclusion and exclusion criteria that omit inconvenient studies. In fact, it is hard to find a systematic review sponsored by, or co-authored by, industry that concludes that the company’s product is not better than those of its competitors.

19 Celecoxib Conclusion, industry supported meta-analysis Celecoxib... has significantly improved gastrointestinal safety and tolerability (BMJ Sept 2002) Conclusion, Cochrane Review For an individual with RA the potential benefits of celecoxib need to be balanced against the uncertainty that the short- term reduced incidence of upper GI complications are maintained in the long-term and its increased cost in comparison to traditional NSAIDs.

20 Industrisponsorerede forsøg med lægemidler Ofte problemer med: Design Data-analyse Afrapportering Konklusion

21 Forsøg med psykofarmaka Gamle præparater: - Alt for høj dosis, alt for hurtig dosisøgning. Nye præparater tilsyneladende lige så gode, med langt færre bivirkninger. Men: udbredt manipulation med dosis, data- analyse og afrapportering. (J Nerv Ment Dis 2002;190:583) (BMJ 2003;326:1171)

22 Behandling for skizofreni Olanzapin kr/år Haloperidol kr/år Forbrug i primærsektoren Olanzapin216 mio kr i 2002 Haloperidol 4 mio kr i 2002 ”the new drugs have no unequivocal advantages for first line use” (BMJ 2000;321:1371)

23 Industrisponsoreret forskning versus offentligt sponsoreret forskning Chancen for et positivt udfald 4 gange så stor for forsøg eller systematiske oversigter over flere forsøg. (BMJ 2003;326:1167) Chancen for en positiv konklusion 5 gange så stor for forsøg, trods samme effekt. (JAMA 2003;290:921)

24 Cochrane Reviews

25 Hvilken forskning mangler vi? - Sammenligninger med gamle, billige præparater, og på en fair måde (offentlig finansiering) - Sammenligninger med andet end lægemidler Forebyggelse af ikke-insulinkrævende sukkersyge: - metformin:31% effekt - motion og vægttab: 56% effekt (N Engl J Med 2002;346:393) - Forskning i skadevirkninger

26