1. New Cochrane risk of bias tool for cluster randomised tools
ICTMC
May 2017
New Cochrane risk of bias tool for cluster randomised tools
Sandra Eldridge
Pragmatic Clinical Trials Unit
Centre for Primary Care and Public Health
Marion Campbell (Aberdeen)
Michael Campbell (Sheffield)
Amy Drahota (Portsmouth)
Bruno Giraudeau (Tours)
Jeremy Grimshaw (Ottawa)
Julian Higgins (Bristol)
Barney Reeves (Bristol)
Nandi Siegfried (Cape Town)
Title
Authors
Centre for Primary Care & Public Health
Blizard Institute
I am director of the pragmatic clinical trials unit at Barts and the London school of medicine and dentistry. The core staff in the unit are located in the brick building in this photograph which is the Centre for Primary Care and Public Health, and that in turn is part of the Blizard Institute which is mostly located in the large glass building that our building is reflected in.

2. Outline Why is this work important? Methods and scope
Focus of this talk:
Allocation concealment
Blinding of participants
Missing clusters
Timing of identification and recruitment of individual participants/randomisation

3. Cluster randomised trials
708 trials in last year, only 252 in 2011
In recent review of systematic reviews about 10% include cluster randomised trials

4. Issues in including cluster randomised trials in systematic reviews
Analysis not accounting for clustering
Difficulty interpreting bias
Incorrect weighting when included in meta-analysis (usually given too much weight)
Studies may be deemed as high or low risk of bias incorrectly

5. Methods and overall results
Sub-group of group updating Cochrane risk of bias tool – ROB v2 Mostly teleconferences and s over a long period of time! Adaptation of ROB v2 Used expertise and personal collections of trials Piloting amongst group and externally Continuous referral back to main updating group
Of 5 domains in updated main tool:
Alterations to three domains
Added one new domain

6. Scope
Clusters are schools, villages, medical practices, hospitals, families etc.
Clusters are individuals with observations made on, for example, teeth, eyes etc.
Stepped-wedge designs in which units of randomisation are clusters

7. Outcomes and participants
Tool assumes risk of bias being assessed for one outcome at a time
For cluster randomised trials “participants” = individuals on whom it is planned to collect the outcome of interest
IRIS trial
Clusters: general practices
Intervention: to increase identification and referral for victims of domestic violence
No individual participant recruitment
Data collection: routine records of all women in practice aged 16 plus
Open access urology trial
Clusters: general practices
Intervention: guideline-based open 
access urological investigation service
Outcome: general practitioners’ compliance with referral guidelines
Outcome: waiting time for patients’ referral

8. Allocation concealment
Key fact: clusters can be randomised sequentially, in batches, or all at once
IRIS trial
Clusters randomised sequentially
Cluster identified, details ed to allocator, allocator ed back allocation status
Diabetes manual trial
Clusters randomised in batches
Stratification factors largely aggregated participant baseline data
Comments:
Less easy to subvert randomisation in cluster randomised trials
More difficult to judge whether baseline imbalances are indicative of problems with randomisation process

9. Likely that information reported is limited
Lack of blinding of participants and others who could affect delivery of intervention
Those with opportunity to introduce deviations to the intervention might have little inclination to do so
The more complex the intervention the more difficult to identify such deviations
Likely that information reported is limited
Key fact:
Interventions often multifaceted, aimed at various participants and participants and others cannot be blinded
Do participants (for the outcome of interest) know they are in a trial? (sometimes no)
If they know they are in a trial, are they aware of their assigned intervention? (usually yes)
Are there other people involved in the intervention who know the allocation of participants? (often yes)
If participants or others are aware of assigned intervention did this affect the way the intervention was delivered beyond what one would expect in usual practice?

10. Example of algorithm for judging risk of bias

11. Missing clusters
Key fact: Missingness can occur at both cluster and individual level
Comment:
Need to be careful about interpretation of bias because will depend on reasons for missingness at both levels

12. Timing of recruitment and randomisation
ukBEAM pilot
Aim: To improve back pain
Clusters: UK General Practices
Intervention: offer of exercise classes, physiotherapy etc.
Control group:
66 recruited
Intervention group:
165 recruited, suffering from milder back pain
Explanation: participation in the trial very attractive in intervention arm
Cluster ‘consent’
Cluster randomisation
BIAS
e.g. ukBEAM pilot study
Participant consent
to randomisation?
Consent
Randomisation
UKBEAM 66 in control 54% of predicted, 165 in intervention, 41% more than predicted
In the UKBEAM trial pilot study (table 1.4), 26 practices were randomised to offer active management or usual care to patients presenting with low back pain. Patients within the active management arm were also individually randomised to receive spinal manipulation, exercise classes or advice alone. After one year practices in the control arm (traditional care) had recruited 66 patients, 54% of the number predicted based on practice list size, while those in the active management arm recruited 165 patients, 41% more than predicted. In addition participants from the active management arm were suffering from milder back pain than those in control practices. It is likely that the offer of exercise classes or physiotherapy made participation in the trial an attractive option for the GPs and their patients in the active management arm, whereas there was such benefit for patients in the control arm. Following the pilot study, the trial was redesigned as an individually randomised trial comparing different methods of delivering active management.
Participant consent to data collection, participation

13. Timing of recruitment and randomisation
Key fact:
The timing of cluster randomisation, participant identification and participant recruitment (if relevant) can be a potential source of bias in cluster randomised trials in a way that is not possible in individually randomised trials
New domain: Timing of identification and recruitment of individual participants in relation to timing of cluster randomisation

14. Scenario 4 (identical to 6) Scenario 6 (identical to 4)
Cluster randomization
Identification of potential individual participants
Identification of individual participants
Recruitment of individual participants
Participants not directly recruited
Potential for identification/recruitment bias although this could be avoided through trial design
No potential for identification/recruitment bias because randomization happens after
UK BEAM pilot (Farrin et al 2005)

15. Two further examples in which identification/recruitment bias possible
Scenario 2: Feeding strategies for critically ill patients in intensive care
Clusters: Intensive care unit (ICU) wards
Intervention: Guidelines developed by ICU staff
Outcome: Hospital discharge mortality
Participants not directly recruited but identified by ICU staff (though no evidence of bias)
Scenario 3: Hip protectors for preventing hip fractures
Clusters: Elderly care units within community based health centres
Participants identified prior to randomisation but approached after randomisation
Recruited: 31% in intervention and 9% in control group

16. Conclusion In cluster randomised trials:
Less easy to subvert randomisation but more difficult to judge if this has happened
Lack of blinding could potentially cause bias but, where appropriate, participants not knowing they are in a trial can protect against this
Clusters are less likely to be missing than individuals but if they are potential bias may be more difficult to interpret
The timing of cluster randomisation, participant identification and participant recruitment (if relevant) can be a potential source of bias
Authors and peer reviewers should be aware of key elements to include in their reports to reassure readers regarding bias