1. How to Analyze Therapy in the Medical Literature (part 1) Akbar Soltani. MD.MSc Tehran University of Medical Sciences (TUMS) Shariati Hospital www.soltaniebm.com

2. Objectives What is randomized controlled trial? How to appraise RCT using standard checklist Review different concepts such as –Randomization, allocation concealment, blinding, loss to follow up, intention to treat analysis

3. The hierarchy of evidence Systematic reviews and meta-analyses Randomised controlled trials Cohort studies Case-control studies Cross sectional surveys Case reports Expert opinion

4. Three Step Guide in Using an Article to Assess Therapy 1.Are the results of the study valid? 2.What are the results? What measures of precision of effects were reported (CIs, p- values)? 3.How can I apply these results to patient care?

5. Randomized control trial design

6. Assess Validity and Applicability to my practice setting 1.Is the study a randomized control trial (RCT)? Yes (go on) No (stop) 2.Were the patients properly selected for the trial and randomized with concealed assignment? Yes (go on) No (stop) 3.Were patients and study personnel “blind” to treatment? Yes (go on) No (pause) 4.Were the intervention and control groups similar at the start? (Check “Table 1” of most studies) Yes (go on) No (stop) 5.Was follow-up complete? ii. Were patients analyzed in the groups to which they were randomized (“intention-to-treat” analysis)?

7. Benefits of Random Allocation (Randomization) 1.Reduces bias in those selected for treatment –guarantees treatment assignment will not be based on patients’ prognosis 2.Prevents confounding –known and unknown potential confounders are evenly distributed

8. Assess Validity and Applicability to my practice setting 1.Is the study a randomized control trial (RCT)? Yes (go on) No (stop) 2.Were the patients properly selected for the trial and randomized with concealed assignment? Yes (go on) No (stop) 3.Were patients and study personnel “blind” to treatment? Yes (go on) No (pause) 4.Were the intervention and control groups similar at the start? (Check “Table 1” of most studies) Yes (go on) No (stop) 5.Was follow-up complete? ii. Were patients analyzed in the groups to which they were randomized (“intention-to-treat” analysis)?

9. Ensuring Allocation Concealment BEST – most valid technique  Central computer randomization DOUBTFUL  Envelopes, etc NOT RANDOMIZED  Date of birth, alternate days, etc

10. Do Not Confuse Allocation Concealment with Blinding Allocation concealment seeks to prevent selection bias, protects assignment sequence before and until allocation, and can always be successfully implemented

12. Do Not Confuse Allocation Concealment with Blinding (Cont’d) Blinding seeks to prevent information bias, protects sequence after allocation, and cannot always be successfully implemented

13. Assess Validity and Applicability to my practice setting 1.Is the study a randomized control trial (RCT)? Yes (go on) No (stop) 2.Were the patients properly selected for the trial and randomized with concealed assignment? Yes (go on) No (stop) 3.Were patients and study personnel “blind” to treatment? Yes (go on) No (pause) 4.Were the intervention and control groups similar at the start? (Check “Table 1” of most studies) Yes (go on) No (stop) 5.Was follow-up complete? ii. Were patients analyzed in the groups to which they were randomized (“intention-to-treat” analysis)?

14. Placebo effect Trial in patients with chronic severe itching Cyproheptadine HCL Trimeprazine tartrate No treatment Treatment vs no treatment for itching

15. Placebo effect Trial in patients with chronic severe itching Cyproheptadine HCL Trimeprazine tartrate Placebo No treatment Treatment vs no treatment vs placebo for itching Placebo effect - attributable to the expectation that the treatment will have an effect

17. Blinding and Reporting Usually reduces differential assessment of outcomes (information bias) Authors should explicitly state who was blinded – and how. Many investigators and readers consider a randomized trial as high quality simply because it is “double-blind,” as if double-blinding is the sine qua non of an RCT. –Trials not “double-blinded” should not automatically be deemed inferior trials.

18. Blinding in randomised trials: hiding who got what Double blinding prevents bias but is less important, than adequate allocation concealment. open studies are more likely to favour experimental interventions over the controls and that studies that are not double-blinded can exaggerate effect estimates by 17% Furthermore, in some trials, blinding cannot be successfully implemented, whereas allocation concealment can always be successfully implemented.

19. Assess Validity and Applicability to my practice setting 1.Is the study a randomized control trial (RCT)? Yes (go on) No (stop) 2.Were the patients properly selected for the trial and randomized with concealed assignment? Yes (go on) No (stop) 3.Were patients and study personnel “blind” to treatment? Yes (go on) No (pause) 4.Were the intervention and control groups similar at the start? (Check “Table 1” of most studies) Yes (go on) No (stop) 5.Was follow-up complete? ii. Were patients analyzed in the groups to which they were randomized (“intention-to-treat” analysis)?

20. Significance tests for baseline differences

21. Baseline data AzathioprinePlacebo Mean age 54.754.9 Serum bilirubin (  mol/L) 37.230.9 Stage I disease % 1412 Stage II disease % 4443 Stage III disease % 1515 Stage IV disease % 2730 Christensen et al. Gastro 1985 Effect of azathioprine on the survival of patients with primary biliary cirrhosis

22. Baseline data AzathioprinePlacebo Mean age 54.754.9 Serum bilirubin (  mol/L) 37.230.9 Stage I disease % 1412 Stage II disease % 4443 Stage III disease % 1515 Stage IV disease % 2730 Christensen et al. Gastro 1985 Effect of azathioprine on the survival of patients with primary biliary cirrhosis

23. Assess Validity and Applicability to my practice setting 1.Is the study a randomized control trial (RCT)? Yes (go on) No (stop) 2.Were the patients properly selected for the trial and randomized with concealed assignment? Yes (go on) No (stop) 3.Were patients and study personnel “blind” to treatment? Yes (go on) No (pause) 4.Were the intervention and control groups similar at the start? (Check “Table 1” of most studies) Yes (go on) No (stop) 5.Was follow-up complete? ii. Were patients analyzed in the groups to which they were randomized (“intention-to-treat” analysis)?

24. How complete was the follow up? How many dropouts were there? Conventionally, a 20% drop out rate is regarded as acceptable, but this depends on the study question. Some regard should be paid to why participants dropped out, as well as how many. It should be noted that the drop out rate may be expected to be higher in studies conducted over a long period of time. A higher drop out rate will normally lead to downgrading, rather than rejection of a study.

25. Over-estimation of treatment effect Bias: a one-sided inclination of the mind Not random 40% Not double-blind17% Duplicate information20% Small trials30% Poor reporting quality25%

26. Assess Validity and Applicability to my practice setting 1.Is the study a randomized control trial (RCT)? Yes (go on) No (stop) 2.Were the patients properly selected for the trial and randomized with concealed assignment? Yes (go on) No (stop) 3.Were patients and study personnel “blind” to treatment? Yes (go on) No (pause) 4.Were the intervention and control groups similar at the start? (Check “Table 1” of most studies) Yes (go on) No (stop) 5.Was follow-up complete? ii. Were patients analyzed in the groups to which they were randomized (“intention-to-treat” analysis)?

27. Montorri V, Guyatt G. CMAJ 2001 165(10) p1340 Intention to treat

28. Montorri V, Guyatt G. CMAJ 2001 165(10) p1340 Intention to treat

29. Montorri V, Guyatt G. CMAJ 2001 165(10) p1340 Intention to treat High risk?

30. Assess Validity and Applicability to my practice setting 1.Is the study a randomized control trial (RCT)? Yes (go on) No (stop) 2.Were the patients properly selected for the trial and randomized with concealed assignment? Yes (go on) No (stop) 3.Were patients and study personnel “blind” to treatment? Yes (go on) No (pause) 4.Were the intervention and control groups similar at the start? (Check “Table 1” of most studies) Yes (go on) No (stop) 5.Was follow-up complete? ii. Were patients analyzed in the groups to which they were randomized (“intention-to-treat” analysis)?

32. Thank you!