Presentation is loading. Please wait.

Presentation is loading. Please wait.

Meta-Analysis: Sports Medicine Canadian Academy of Sport Medicine LAcadémie Canadienne de Médecine du Sport Ian Shrier MD, PhD, Dip Sport Med, FACSM Centre.

Similar presentations


Presentation on theme: "Meta-Analysis: Sports Medicine Canadian Academy of Sport Medicine LAcadémie Canadienne de Médecine du Sport Ian Shrier MD, PhD, Dip Sport Med, FACSM Centre."— Presentation transcript:

1 Meta-Analysis: Sports Medicine Canadian Academy of Sport Medicine LAcadémie Canadienne de Médecine du Sport Ian Shrier MD, PhD, Dip Sport Med, FACSM Centre for Clinical Epidemiology and Community Studies, SMBD- Jewish General Hospital and McGill University Past-president, Canadian Academy of Sport Medicine

2 Meta-Analysis: Sports Medicine Canadian Academy of Sport Medicine LAcadémie Canadienne de Médecine du Sport Ian Shrier MD, PhD, Dip Sport Med, FACSM Centre for Clinical Epidemiology and Community Studies, SMBD- Jewish General Hospital and McGill University Past-president, Canadian Academy of Sport Medicine

3 OBJECTIVES How do we think?How do we think? A Workshop Example: Does Stretching Prevent Injury?A Workshop Example: Does Stretching Prevent Injury? What parameter are you estimating?What parameter are you estimating? RCT vs Obs studies in meta-analysesRCT vs Obs studies in meta-analyses Its all about Bias!Its all about Bias!

4 INTERPRETATIONS Its a rather interesting phenomenon. Every time I press this lever, the graduate student breathes a sigh of relief

5 INTERPRETATIONS Shrier, Platt, Steele. Mega-trials vs. meta-analysis: Precision vs. heterogeneity? Contemp Clin Trials 2007 Shrier et al. Should Meta-Analyses of Interventions Include Observational Studies in Addition to Randomized Controlled Trials? A Critical Examination of Underlying Principles. Am J Epi 2007 Shrier et al. The interpretation of systematic reviews with meta-analyses: an objective or subjective process? BMC Med Inform Dec Making 2008

6 INTERPRETATIONS I believe magnesium has now been shown to be beneficial for patients during the post-MI period (SD-SA) Rev 8 Rev 7 Rev 6 Rev 5 Rev 4 Rev 3 Rev 2 Ag DA Ag DA SA Ag DA - Ag DA - SA Ag SD - Ag DA - Ag Rev 1 I2I2I2I2 Rand. OR Fixed OR Ag DA Ag SD Ag 59% 0.75 (0.61-0.92) 1.01 (0.96-1.07) 69,50561% 0.65 (0.48-0.87) 1.02 (0.96-1.08) 63,047 - DA Ag - SA Ag 14% 0.66 (0.53-0.81) 0.64 (0.52-0.79) 3,6850% 0.38 (0.21-0.66) 0.40 (0.28-0.61) 5970% 0.40 (0.18-0.86) 0.40 (0.19-0.83) 415N 1-231-201-101-51-3 # RCTs

7 HOW DO WE THINK? ***** Clue: want crave, covet, yearn, fancy crave, covet, yearn, fancy (Vandenbroucke et al, 2001)

8 HOW DO WE THINK? JUG U I C**** E Clue: want crave, covet, yearn, fancy crave, covet, yearn, fancy (Vandenbroucke et al, 2001)

9 HOW DO WE THINK? JUG U I C**** EHIS QueuesQueuesQueuesQueues Clue: want crave, covet, yearn, fancy crave, covet, yearn, fancy (Vandenbroucke et al, 2001)

10 HOW DO WE THINK? JUGL UI IN C***E EHIS QueuesQueuesQueuesQueues Clue: want crave, covet, yearn, fancy crave, covet, yearn, fancy (Vandenbroucke et al, 2001)

11 HOW DO WE THINK? JUGL UI IN CRAVE EHIS L E QueuesQueuesQueuesQueues Clue: want crave, covet, yearn, fancy crave, covet, yearn, fancy (Vandenbroucke et al, 2001)

12 OBJECTIVES How do we think?How do we think? A Workshop Example: Does Stretching Prevent Injury?A Workshop Example: Does Stretching Prevent Injury? What parameter are you estimating?What parameter are you estimating? RCT vs Obs studies in meta-analysesRCT vs Obs studies in meta-analyses Its all about Bias!Its all about Bias!

13 Does Stretching Prevent Injury? (adapted from Shrier, Evidence-Based Sports Medicine 2007) (adapted from Shrier, Evidence-Based Sports Medicine 2007)

14 AnalysisAnalysis Yes Yes No No I will now tell my patients to stretch to prevent injury Yes Yes No No I will now tell my patients not to stretch to prevent injury Yes Yes No No I will now tell my patients that I have no idea whether they should stretch to prevent injury Does Stretching Prevent Injury?

15 (adapted from Shrier, Evidence-Based Sports Medicine 2007) (adapted from Shrier, Evidence-Based Sports Medicine 2007)

16 AnalysisAnalysis Yes Yes No No I will now tell my patients to stretch to prevent injury Yes Yes No No I will now tell my patients not to stretch to prevent injury Yes Yes No No I will now tell my patients that I have no idea whether they should stretch to prevent injury Does Stretching Prevent Injury?

17 % Unstretched Condition Acute Stretching: Force (MVC/1RM) (adapted from Shrier, Clin J Sport Med 2004) (adapted from Shrier, Clin J Sport Med 2004)

18 % Unstretched Condition Regular Stretching: Force (MVC/1RM) PNFStatic (adapted from Shrier, Clin J Sport Med 2004) (adapted from Shrier, Clin J Sport Med 2004)

19 Acute Stretching: Force (Isokinetic) % Unstretched Condition Slow (30-60 deg/s) Fast (>180 deg/s) (adapted from Shrier, Clin J Sport Med 2004) (adapted from Shrier, Clin J Sport Med 2004)

20 Regular Stretching: Force (Isokinetic) % Unstretched Condition Slow (30-60 deg/s) Fast (>180 deg/s) (adapted from Shrier, Clin J Sport Med 2004) (adapted from Shrier, Clin J Sport Med 2004)

21 % Unstretched Condition Acute Stretching: Jump Height StaticCMJ (adapted from Shrier, Clin J Sport Med 2004) (adapted from Shrier, Clin J Sport Med 2004)

22 % Unstretched Condition Regular Stretching: Jump Height StaticCMJ (adapted from Shrier, Clin J Sport Med 2004) (adapted from Shrier, Clin J Sport Med 2004)

23 Stretching and Force Overstretching occurs with as little as 20% stretchOverstretching occurs with as little as 20% stretch ProtocolProtocol Skinned bullfrog muscle fibers stretched and released to different lengths Skinned bullfrog muscle fibers stretched and released to different lengths (adapted from Higuchi et al, J Mus Res Cell Motil 1988) (adapted from Higuchi et al, J Mus Res Cell Motil 1988) ResultsResults

24 Regular Stretching: Force ProtocolProtocol Weights attached to left wing of Japanese Quail x 30 days Weights attached to left wing of Japanese Quail x 30 days Animals killed and ant. lat. dorsi. placed in vitro Animals killed and ant. lat. dorsi. placed in vitro (adapted from Alway, J Appl Physiol 1984) (adapted from Alway, J Appl Physiol 1984) ResultsResults

25 Regular Stretching: Injury (adapted from Shrier, Evidence-Based Sports Medicine 2007) (adapted from Shrier, Evidence-Based Sports Medicine 2007)

26 Acute Stretching: Injury Excluding multiple co-intervention studies (adapted from Shrier, Evidence-Based Sports Medicine 2007) (adapted from Shrier, Evidence-Based Sports Medicine 2007)

27 Excluding multiple co-intervention studies Acute Stretching: Injury (adapted from Shrier, Evidence-Based Sports Medicine 2007) (adapted from Shrier, Evidence-Based Sports Medicine 2007)

28 AnalysisAnalysis Yes Yes No No I will now tell my patients to stretch before exercise to prevent injury Yes Yes No No I will now tell my patients not to stretch before exercise to prevent injury Yes Yes No No I will now tell my patients that I have no idea whether they should stretch before exercise to prevent injury Does Stretching Prevent Injury?

29 AnalysisAnalysis Yes Yes No No I will now tell my patients to stretch regularly to prevent injury Yes Yes No No I will now tell my patients not to stretch regularly to prevent injury Yes Yes No No I will now tell my patients that I have no idea whether they should stretch regularly to prevent injury Does Stretching Prevent Injury?

30 OBJECTIVES How do we think?How do we think? A Workshop Example: Does Stretching Prevent Injury?A Workshop Example: Does Stretching Prevent Injury? What parameter are you estimating?What parameter are you estimating? RCT vs Obs studies in meta-analysesRCT vs Obs studies in meta-analyses Its all about Bias!Its all about Bias!

31 Clinical Trials and Meta-Analysis 1994;29:41–47 Effect of RCT on Outcomes

32 RCT vs. Observational: Theoretical HughesHughes Balanced placebo/traditional design Balanced placebo/traditional design RCT Informed: pts told Nicotine or Placebo RCT Informed: pts told Nicotine or Placebo Balanced placebo: Pts randomized to be told Nicotine or Placebo, but random 50% given what they were told (4 groups) Balanced placebo: Pts randomized to be told Nicotine or Placebo, but random 50% given what they were told (4 groups) # Days Smoked NicoPlac Blind % Complete Abstainment NicoPlac Blind Nico Plac Psychopharmacology 1989 ITT: Patient wants to know effect of intervention conditional on them receiving the intervention (per protocol?)ITT: Patient wants to know effect of intervention conditional on them receiving the intervention (per protocol?)

33 OBJECTIVES How do we think?How do we think? A Workshop Example: Does Stretching Prevent Injury?A Workshop Example: Does Stretching Prevent Injury? What parameter are you estimating?What parameter are you estimating? RCT vs Obs studies in meta-analysesRCT vs Obs studies in meta-analyses Its all about Bias!Its all about Bias!

34 RCT vs. Observational: Evidence Linde: Observational studies about 10-20% better for acupuncture/headache (J Clin Epi 2002)Linde: Observational studies about 10-20% better for acupuncture/headache (J Clin Epi 2002) Concato: No difference in well-designed studies (NEJM 2000)Concato: No difference in well-designed studies (NEJM 2000) MacLehose: discrepancies for high quality studies were small but discrepancies for low quality studies were large (HTA 2000)MacLehose: discrepancies for high quality studies were small but discrepancies for low quality studies were large (HTA 2000) Benson: No difference after 1984 (Am J Opthalmol 2000)Benson: No difference after 1984 (Am J Opthalmol 2000) Britton: Non-randomized overestimated magnitude of effect (HTA 1998)Britton: Non-randomized overestimated magnitude of effect (HTA 1998) Linde: Observational studies about 10-20% better for acupuncture/headache (J Clin Epi 2002)Linde: Observational studies about 10-20% better for acupuncture/headache (J Clin Epi 2002) Concato: No difference in well-designed studies (NEJM 2000)Concato: No difference in well-designed studies (NEJM 2000) MacLehose: discrepancies for high quality studies were small but discrepancies for low quality studies were large (HTA 2000)MacLehose: discrepancies for high quality studies were small but discrepancies for low quality studies were large (HTA 2000) Benson: No difference after 1984 (Am J Opthalmol 2000)Benson: No difference after 1984 (Am J Opthalmol 2000) Britton: Non-randomized overestimated magnitude of effect (HTA 1998)Britton: Non-randomized overestimated magnitude of effect (HTA 1998) more extreme

35 RCT vs. Observational: Theoretical All StudiesAll Studies Adjust for known confounders Adjust for known confounders All StudiesAll Studies Adjust for known confounders Adjust for known confounders Unknown confounders likely to be equally distributedUnknown confounders likely to be equally distributedRCTConProDesign Concealed randomization specifically removes the possibility of selection bias or confounding in RCTs, i.e. any differences between the groups are attributable to chance or to the intervention, all else being equal. Deeks et al, Health Tech Asess 2003 Based on assumption of randomization in infinite population, or opposite distribution of confounders if many trials examined Example: Confounder present in 20% of population. N= 400 95% Prob. Distr. = 15.6%-24.4%. If 5 confounders, 23% chance that at least one is outside the range (95% Prob. Dist. = 14.2%-25.8%) (Shrier et al, AJE 2007)

36 RCT vs. Observational: Theoretical All StudiesAll Studies Adjust for known confounders Adjust for known confounders All StudiesAll Studies Adjust for known confounders Adjust for known confoundersDesignProConRCT Unknown confounders likely to be equally distributedUnknown confounders likely to be equally distributed Control participants likely to do better than non- participantsControl participants likely to do better than non- participants Cohort More representative sampleMore representative sample CheaperCheaper Historical Cohort: answer fasterHistorical Cohort: answer faster Increased sample size for adjustmentIncreased sample size for adjustment Confounding by indication: patient/physician - randomConfounding by indication: patient/physician - random There may be important prognostic factors that the investigators do not know about or have not measured which are unbalanced between groups and responsible for differences in outcome. Deeks et al, Health Tech Asess 2003 (Shrier et al, AJE 2007)

37 OBJECTIVES How do we think?How do we think? A Workshop Example: Does Stretching Prevent Injury?A Workshop Example: Does Stretching Prevent Injury? What parameter are you estimating?What parameter are you estimating? RCT vs Obs studies in meta-analysesRCT vs Obs studies in meta-analyses Its all about Bias!Its all about Bias!

38 turn on your IPOD now! IF YOU WANT THE BLUE PILL….

39 FORMS OF BIAS Structural Approach to Bias Confounding Bias Confounding Bias Failure to condition on a common cause Failure to condition on a common cause Do not condition on a variable (or marker of a variable) that lies along the causal pathway Do not condition on a variable (or marker of a variable) that lies along the causal pathway Selection Bias Selection Bias Conditioning on a common effect Conditioning on a common effect (Pearl, Hernán, Greenland)

40 CONFOUNDING BIASOsteoarthritis(indirect) Gait Disorder (direct) ActivityXMY Gait Disorder OAActivity X C Y XX

41 CONFOUNDING BIAS? Exposure: smokingExposure: smoking Outcome: spont. abortionOutcome: spont. abortion Confounding?: previous spont. abortionConfounding?: previous spont. abortion (Weinberg Am J Epid 1993) SmokingSmoking Spont. Abortion Previous Sp. Ab.

42 CONFOUNDING BIAS? Exposure: smokingExposure: smoking Outcome: spont. abortionOutcome: spont. abortion Confounding?: previous spont. abortionConfounding?: previous spont. abortion (Weinberg Am J Epid 1993) SmokingSmoking Spont. Abortion Previous Sp. Ab. TissueAbnormalityTissueAbnormality Underlying abnormality: intrinsic tissue abnormalityUnderlying abnormality: intrinsic tissue abnormality

43 CONFOUNDING BIAS? Exposure: smokingExposure: smoking Outcome: spont. abortionOutcome: spont. abortion Confounding?: previous spont. abortionConfounding?: previous spont. abortion (Weinberg Am J Epid 1993) SmokingSmoking Spont. Abortion Previous Sp. Ab. TissueAbnormalityTissueAbnormality Underlying abnormality: intrinsic tissue weaknessUnderlying abnormality: intrinsic tissue weakness XXXX

44 CONFOUNDING BIAS? (Weinberg Am J Epid 1993) Univariate RR for smoking/no smoking=1.85Univariate RR for smoking/no smoking=1.85 Stratified RRStratified RR RR for smoking/no smoking (Previous Sp. Ab.)=1.32 RR for smoking/no smoking (Previous Sp. Ab.)=1.32 RR for smoking/no smoking (No Previous Sp. Ab.) =1.32 RR for smoking/no smoking (No Previous Sp. Ab.) =1.32 However, whether or not someone had a previous spontaneous abortion does not change the effects of smokingHowever, whether or not someone had a previous spontaneous abortion does not change the effects of smoking Including this covariate results in an invalid estimateIncluding this covariate results in an invalid estimate SmokingSmoking Spont. Abortion Previous Sp. Ab. TissueAbnormalityTissueAbnormality XXXX

45 CONFOUNDING BIAS? (Hernán Am J Epid 2002) CExOutcome ExU Outcome C CCExExOutcomeOutcome UU CCExExOutcomeOutcomeUU

46 Condition on C? (Cole & Hernán Int J Epid 2002) EOutcome CU2U1

47 FORMS OF BIAS Structural Approach to Bias Confounding Bias Confounding Bias Failure to condition on a common cause Failure to condition on a common cause Do not condition on a variable (or marker of a variable) that lies along the causal pathway Do not condition on a variable (or marker of a variable) that lies along the causal pathway Selection Bias Selection Bias Conditioning on a common effect Conditioning on a common effect (Pearl, Hernán, Greenland)

48 (Pearl. Causality Book) Step 4: Connect any two parents sharing a common child. Including colliders opens up path for confounding Including colliders opens up path for confounding X1X1X1X1 X3X3X3X3 X2X2X2X2 X4X4X4X4 X5X5X5X5 Sprinkler Rain Season Wet Slippery If one knows the value of the collider, the parents are associated. If wet:the sprinkler is more likely to be on if there was no rain. PEARLS RULES - EXPLANATION

49 UNBIASED EFFECT ESTIMATE? XOutcome (Pearl. Causality Book) Which measurements should be included in the model if we are interested in the relation between X and Outcome?

50 XOutcome Z1Z1Z1Z1 (Pearl. Causality Book) Z2Z2Z2Z2 Which measurements should be included in the model if we are interested in the relation between X and Outcome? Do Z 1 and Z 2 remove confounding? UNBIASED EFFECT ESTIMATE?

51 XOutcome Z1Z1Z1Z1 (Pearl. Causality Book) Z2Z2Z2Z2 If X is disconnected from Outcome (d-separation), there is no confounding Which measurements should be included in the model if we are interested in the relation between X and Outcome? Do Z 1 and Z 2 remove confounding? UNBIASED EFFECT ESTIMATE!

52 UNBIASED EFFECT ESTIMATE? XOutcome Z1Z1Z1Z1 (Pearl. Causality Book) Z2Z2Z2Z2

53 XOutcome Z1Z1Z1Z1 Z2Z2Z2Z2 X is NOT disconnected from Outcome Which measurements should be included in the model if we are interested in the relation between X and Outcome? Do Z 1, Z 2 and Z 3 remove confounding? Z3Z3Z3Z3 INCLUDING Z 3 INTRODUCES BIAS! UNBIASED EFFECT ESTIMATE?

54 SELECTION BIAS EXAMPLES Observational SpecificObservational Specific Berksons Bias Berksons Bias Volunteer / Self-selection Bias Volunteer / Self-selection Bias Healthy worker Bias Healthy worker Bias Meta-analysis specificMeta-analysis specific Reporting bias Reporting bias Publication bias Publication bias RCT or ObservationalRCT or Observational Differential loss to follow-up Differential loss to follow-up Non-response / Missing data bias Non-response / Missing data bias Adjustment for variables affected by previous exposure Adjustment for variables affected by previous exposure

55 ATTRITION BIAS RCT Complex Attrition bias Treatment Death Side effects Drop Out Mild disease Condition on common effect

56 EFFECT OF BIAS Can one sum probability distributions for different risks of bias?Can one sum probability distributions for different risks of bias? Already being done intuitively and informally Already being done intuitively and informally Some beginnings: response-surface estimation (Greenland), multiple bias modeling (Greenland), adjusted likelihoods (Wolpert), bias against bias (Kaufman) Some beginnings: response-surface estimation (Greenland), multiple bias modeling (Greenland), adjusted likelihoods (Wolpert), bias against bias (Kaufman) Treatment Beneficial Treatment Harmful Bias Towards Benefit Biased Against Benefit Probability of Bias

57 ESTIMATING BIAS? XOutcome (Pearl. Causality Book) Censored Unknown Variable

58 SUMMARY Objective is to obtain an unbiased estimate of the parameter of interestObjective is to obtain an unbiased estimate of the parameter of interest Study design is only one source of biasStudy design is only one source of bias Mathematics underlying statistical analyses do not care what the names of the nodes areMathematics underlying statistical analyses do not care what the names of the nodes are Causal maps make assessing bias more transparentCausal maps make assessing bias more transparent Meta-analyses should be able to treat all potential biases regardless of causeMeta-analyses should be able to treat all potential biases regardless of cause Estimating the wrong parameter - RCT, ITT? Estimating the wrong parameter - RCT, ITT? Conditioning on a variable that lies along the causal path or is a marker for a variable lying along the causal path Conditioning on a variable that lies along the causal path or is a marker for a variable lying along the causal path Absence of conditioning on a common cause Absence of conditioning on a common cause Conditioning on a common effect Conditioning on a common effect

59 INTERPRETATIONS

60 Canadian Academy of Sport Medicine LAcadémie Canadienne de Médecine du Sport


Download ppt "Meta-Analysis: Sports Medicine Canadian Academy of Sport Medicine LAcadémie Canadienne de Médecine du Sport Ian Shrier MD, PhD, Dip Sport Med, FACSM Centre."

Similar presentations


Ads by Google