Presentation on theme: "Design and conduct of randomised controlled trials David Torgerson Director, York Trials Unit"— Presentation transcript:
Design and conduct of randomised controlled trials David Torgerson Director, York Trials Unit firstname.lastname@example.org www.rcts.org
Justification Randomised controlled trials are the key to evidenced based practice. The basic principles of the RCT are relatively straight forward. HOWEVER, many elementary mistakes are made that undermine their design. Whilst placebo trials are widely thought of as being the ‘gold-standard’ other trial designs are as important if not more important to provide evidence for decision making.
Background Traditionally most interventions have been evaluated using a pre-test post-test or before and after design. Participants are tested treated and then tested again any improvements are attributable to the intervention. Currently this is probably the most POPULAR evaluative method.
What is wrong with ‘before and after studies’ "It is incident to physicians, I am afraid, beyond all other men, to mistake subsequence for consequence.“ (Samuel Johnson, 1734)
Problems Problems include: »Temporal changes; »Regression to the mean.
Temporal Change Most people get better or worse irrespective of any medical attention. Similarly in other fields (e.g. education) self-learning irrespective of teaching occurs. Any intervention or treatment is mixed up with these temporal changes difficult to disentangle.
Regression to the Mean As well as temporal changes before and after studies are confounded by a statistical phenomenon known as ‘Regression to or towards the mean’
Regression to the mean This is a GROUP phenomenon and occurs when the group are measured with an inexact measurement tool and then remeasured. Those individuals with ‘extreme’ values will have a high probability of regressing towards the mean on the second measurement.
Components of RTM Statistically we have two components to a test – the ‘true’ value plus a random error component. The larger the value of the random error the greater will be RTM. Many clinicians and others are mislead because of RTM.
Economists and RTM “I suspect that the regression fallacy is the most common fallacy in the statistical analysis of economic data” Milton Friedman 1992
Regression to Mean using bone density Cummings et al. JAMA 2000;283:1318-21
Same data dichotomised Cummings et al. JAMA 2000;283:1318-21
Example of clinician confusion In routine care women not ‘responding’ to a bisphosphonate are given high dose vitamin D to try and reverse bone loss. Heckman et al. BMC Musculoskeletal Disorders 2002;3:6.
Author’s conclusion “In patients not responding to bisphosphonates 1000 vitamin D daily may improve BMD”. So what if they are wrong? Vit D is harmless? BUT an RCT of high dose vitamin D showed a significant INCREASE in hip fractures among the treated group!
Another clinical fallacy A recent conference presentation, using observational before and after data, made the suprising observation that bisphosphonate therapy may reduce the number and incidence of falls.
Effect of Bisphosphonate on Falls Population Risk of Falls
Effect of Bisphosphonate on Falls Population Risk of Falls Change p < 0.001
Do bisphosphonates reduce falls? “This suprising positive effect of bisphosphonates warrants further study as one possible additional explanation as to why fracture rates are reduced with bisphosphonate treatment” Explanation is probably Regression to Mean.
Two further abstracts at same conference One was an RCT of vitamin D injections. Placebo injections were also given. Patients had low BMD – placebo group increased by 3% in one year (not diff from active group). Next abstract before and after with calcitriol bone mass increased by 1.21%, among women selected with low BMD. Conclusion = ‘Calcitriol prevented bone loss at hip and spine’ Did it? Probably RTM.
Policy Changes Regression to the mean is an excellent method of ‘proving’ something works; Failing hospitals can have an ‘expensive’ management change and there is a good chance that regression to the mean will do the job. A NHS authority sent for ‘retraining’ the 10% of cytologists who had the highest error rate: training was highly effective!
Solution to weakness of before and after studies To control both for regression to the mean and temporal effects it is ESSENTIAL that evaluations have a contemporaneous control group.
Choice of Control Groups Not ALL control groups are equal. Only control groups formed by randomisation or a similar procedure (e.g., minimisation) are robust.
Selection Bias All non randomised studies are susceptible to selection bias. Selection bias means that comparator groups are not equal in all variables except for the treatment. Selection for treatment may be based on a variable that predicts outcome.
Hormone Replacement Therapy Non-randomised studies have shown that HRT is associated with reductions in cardiovascular disease, dementia and all cause mortality. This view prevailed until a large (n = 16,000) RCT was undertaken.
Trial Evidence Early trial in men of oestrogens in 1970s stopped increased mortality due to cardiovascular events. HERS trial in mid 1990s among women with Angina showed INCREASED risk of cardiovascular risk. Trial of 16,000 women (WHI study) stopped early in 2001 because of INCREASE in cardiovascular deaths.
HRT: what happened? Women who took HRT tended to be different from those who did not. They took more exercise had a better diet both of which reduces cardiovascular disease. The association was due to these factors NOT HRT. Because of these SELECTION effects we need to undertake randomised trials.
Why Randomisation? Randomisation WILL on average create two or more groups that are equivalent in all important characteristics that affect outcome. Because of this the introduction of a treatment in one group will therefore produce and UNBIASED estimate of effect.
Randomisation Production of equivalent groups will control for regression to the mean, temporal changes and selection bias.
Failure of medicine Because of LACK of RCTs in many areas unproven, harmful, treatments have been given to many patients. History of medicine is littered with examples of introducing ‘unproven’ treatments to patients before controlled rigorous evaluation.
Opposition to RCTs The use of the RCT in health care has met sustained opposition from many clinicians. This has led to many disasters.
Opposition “I found but one person who rigidly adherred to the idea of a placebo control and he is a bio-statistician who, if he did not adhere to this view, would have had to admit his own purposelessness in life” (Jonas Salk).
Health Care Disasters Opposition to RCTs has declined over the years, partly due to a number of catastrophes, from unevaluated treatments. Harmful treatments are still in widespread use today – we just don’t know which ones.
Disasters among babies Routine practice in 40s and 50s to give premature infants pure oxygen. At the same time it was noted that there was an ‘epidemic’ of blindness among babies. Linked to oxygen use. Routine practice in 50s to give prophylactic antibiotics to premature infants, caused brain damage and death. BOTH of these problems only discovered AFTER an RCT was undertaken.
Heart Attacks After heart attack survivors can be placed into two groups: »Those with arrhythmia & those without. »Arrhythmia patients have a much HIGHER risk of subsequent death. »There are drugs that can ‘correct’ cardiac arrhythmias. »These were widely used on millions of patients.
CAST Trials For decades researchers tried to evaluate arrhythmia drugs. Not ETHICAL argued many cardiologists. Finally, CAST trial was started, terminated early in 1989 because of INCREASE in deaths on active drug. 100s of thousands died because of this.
Can it still HAPPEN? Supplementation with folate increases risk of MI (2005) Use of intravenous steroids after head injury increases mortality (2004); HRT INCREASES strokes, myocardial infarction and dementia (2002). Counselling women after difficult labour INCREASES depression (2001)
Development of the Controlled Trial. Many researchers over the centuries have seen the need for a ‘control’ group to avoid the inherent biases in the before and after study. Controlled trials have been conducted for several hundred years probably occasionally using randomisation.
Scurvy Scurvy was a very prevalent condition among sailors before the 19 th Century. A controlled trial in the middle of the 18 th Century of 12 sailors showed that the two sailors allocated to receive lime or orange juice recovered compared to their ship mates allocated to vinegar or salt water.
Medical Care 1810 Standard medical care in the early 1800s included: routine bleeding. Practised by many if not most doctors in the 18th and early 19th Centuries. Chalmers reports a controlled trial undertaken during the Peninsular War comparing bleeding with no bleeding of sick soldiers.
Cambridge-Somerville In 1937 a classic experiment – the Cambridge-Somerville trial was launched. The aim was to show that social worker intervention among ‘delinquent’ boys would reduce ‘criminality’.
Design 650 boys were identified by their teachers has having delinquent behaviour that put them at later risk of criminal activity. 325 pairs were formed and one from each pair was allocated a social worker supported by psychiatrists.
Results – early follow-up % of boys indulging in crime. Pale bar indicates intervention grop
Results later follow-up In 1975 ‘boys’ were followed up again when middle aged men. 58% of intervention group had NOT had a criminal conviction BUT 68% of control group had NOT had a conviction. If a control group had not been used success of the intervention would be assured.
Patulin Trial Perhaps the first placebo controlled trial was the 1944 MRC patulin trial for the treatment of a common cold. 1449 patients included in the trial. Percent cured or improved at 48 hours was 77% for placebo and 73% for patulin.
Streptomycin The first randomised trial in health care of a BENEFICIAL treatment was the MRC’s steptomycin trial. Streptomycin expensive antibiotic, unknown efficacy for pulmonary TB. UK could not afford to buy much due to virtual postwar bankruptcy. Randomisation was seen as fair method of allocating scarce drug.
Streptomycin Results StreptomycinControl Improvement69%33% No change4%6% Worse20%34% Death7%27%
Streptomycin Note 1/3 rd of the control patients improved despite no antibiotic; Patulin Trials were methodologically important: »Used random allocation; »Blinding of doctors, patients, assessors; »Allocation of treatment was concealed; »Placebo (for patulin).
Summary Failure to do timely controlled trials has cost many ‘000s of lives. All new treatments need to be evaluated using the RCT.