Presentation is loading. Please wait.

Presentation is loading. Please wait.

Subgroup Analyses: Can We Smooth' out the Rough Edges? Daniel Sargent, PhD Mayo Clinic Sept 28, 2006.

Similar presentations


Presentation on theme: "Subgroup Analyses: Can We Smooth' out the Rough Edges? Daniel Sargent, PhD Mayo Clinic Sept 28, 2006."— Presentation transcript:

1 Subgroup Analyses: Can We Smooth' out the Rough Edges? Daniel Sargent, PhD Mayo Clinic Sept 28, 2006

2 Outline Motivation Motivation Subgroups ARE medicine (especially its future) Subgroups ARE medicine (especially its future) Examples Examples Good and bad conduct Good and bad conduct Strategies Strategies Hierarchical models Hierarchical models Smoothing approaches Smoothing approaches Conclusion Conclusion

3 Subgroups analysis: My Definition & My Bias Definition: An effort to draw inference on an effect of an intervention in a set of patients smaller than the entire experimental cohort Definition: An effort to draw inference on an effect of an intervention in a set of patients smaller than the entire experimental cohort Bias: Such inferences will be more robust when based on a model using all patients than an analysis restricted to just the cohort of interest Bias: Such inferences will be more robust when based on a model using all patients than an analysis restricted to just the cohort of interest

4 Subgroups are medicine If all patients were the same, wouldnt need physicians If all patients were the same, wouldnt need physicians Human Genome Project massively expanding knowledge base Human Genome Project massively expanding knowledge base Technology, biology, chemistry, etc. allowing manufacture of highly specific, targeted compounds Technology, biology, chemistry, etc. allowing manufacture of highly specific, targeted compounds Patients seek tailored treatment recommendations Patients seek tailored treatment recommendations

5 Example: Colon Cancer: Model- Derived Estimates of 5 year DFS (%) with Surgery plus Adjuvant Therapy Gill, JCO 2004; http://www.mayoclinic.com/calcs

6 Example: Breast Cancer Most common cancer in women in the US The HER-2 gene is overexpressed in 25-30% of breast cancers; associated with worse prognosis. Trastuzumab, a humanized monoclonal antibody targets the HER-2 receptor; previous trials have demonstrated activity in the treatment of HER-2 overexpressing late stage breast cancer. Performed a clinical trial testing trastuzumab in subset of HER-2 positive women with early stage breast cancer

7 Disease-Free SurvivalSurvival Years ACT+H H 134 events ACT+H H 62 events ACT 261 events HR=0.48, 2P=3x10 -12 HR=0.67, 2P=0.015 94% 91% 92% 87% 75% 85% 67% ACT 92 events Romond et al, NEJM 2005

8 Avoiding subgroup analysis: Targeted Phase II/III Trials Patient Selection for targeted therapies Test the recommended dose on patients who are most likely to respond based on their molecular expression levels Test the recommended dose on patients who are most likely to respond based on their molecular expression levels May result in a large savings of patients (Simon & Maitournam, Clinical Cancer Research 2004) May result in a large savings of patients (Simon & Maitournam, Clinical Cancer Research 2004)

9 Trials in targeted populations Gains in efficiency depend on marker prevalence and relative efficacy in marker + and marker – patients Gains in efficiency depend on marker prevalence and relative efficacy in marker + and marker – patients Details: Session #13 tomorrow Details: Session #13 tomorrow Prevalence Relative Efficacy Efficiency Gain 25%0%16x 25%50%2.5x 50%0%4x 50%50%1.8x 75%0%1.8x 75%50%1.3x (Simon & Maitournam, CCR 2004)

10 Case Study: Stage II colon cancer Colon cancer: Prognosis defined by stage Colon cancer: Prognosis defined by stage Prior trials generally enrolled patients with both stage II and III disease Prior trials generally enrolled patients with both stage II and III disease Previous randomized trials uniformly demonstrate benefit of chemotherapy in stage III patients (node positive) Previous randomized trials uniformly demonstrate benefit of chemotherapy in stage III patients (node positive) Previous trials & pooled analyses mixed regarding benefit in stage II patients Previous trials & pooled analyses mixed regarding benefit in stage II patients No single trial powered for modest effect seen in stage II ( 2-3% in 5 year survival) No single trial powered for modest effect seen in stage II ( 2-3% in 5 year survival)

11 Meta-analysis Stage II Adjuvant Therapy Benson et al. J Clin Oncol. 2004 N=2,732 RR=0.88 P=0.08

12 American Society of Clinical Oncology Guidelines 2004 Direct evidence from randomized trials does not support routine use of chemotherapy for patients with stage II colon cancer. Direct evidence from randomized trials does not support routine use of chemotherapy for patients with stage II colon cancer. Those who accept the relative benefit in stage III disease as adequate indirect evidence of benefit for stage II disease are justified in considering chemotherapy, particularly for patients with high- risk stage II disease. Those who accept the relative benefit in stage III disease as adequate indirect evidence of benefit for stage II disease are justified in considering chemotherapy, particularly for patients with high- risk stage II disease. Ultimate clinical decision should be based on discussions with the patient. Ultimate clinical decision should be based on discussions with the patient. Benson et al. J Clin Oncol. 2004

13 Primary end-point: disease-free survival (DFS) R LV5FU2 FOLFOX4 : LV5FU2 + oxaliplatin 85 mg/m² N=2246 Stage II: 40% Stage III: 60% New therapy: FOLFOX de Gramont et al., ASCO 2005

14 6.6% Disease-free Survival (ITT) 1.0 0.9 0.8 0.7 0.6 0.5 0.3 0.4 0.2 0.1 0.0 0666121824303642485460 Months Events FOLFOX4 279/1123 (24.8%) LV5FU2 345/1123 (30.7%) HR [95% CI]: 0.77 [0.65 – 0.90] DFS probability p<0.001 de Gramont et al., ASCO 2005

15 Disease-free Survival (ITT) Stage II and Stage III Patients 1.0 0.9 0.8 0.7 0.6 0.5 0.3 0.4 0.2 0.1 0.0 0 FOLFOX4 – 451 Stage II LV5FU2 – 448 Stage II FOLFOX4 – 672 Stage III LV5FU2 – 675 Stage III HR [95% CI]: 0.82 [0.60 – 1.13] Stage II 0.75 [0.62 – 0.89] Stage III Months DFS probability 666121824303642485460 Data cut-off: January 16, 2005 8.6% 3.5% de Gramont et al., ASCO 2005

16 DFS (months) DFS in high-risk* stage II patients 1.0 0.9 0.8 0.7 0.6 Probability *T4 and/or bowel obstruction and/or tumor perforation and/or poorly differentiated tumor and/or venous invasion and/or <10 examined LNs Data cut-off: January 16, 2005 0612182430364248 5.4% HR 0.76 FOLFOX4 – 286 HRStage II LV5FU2 – 290 HR Stage II de Gramont et al., ASCO 2005

17 FDA Action Approval of FOLFOX therapy only in stage III patients, even though trial designed for stage II and III patients Approval of FOLFOX therapy only in stage III patients, even though trial designed for stage II and III patients Possible rationale Possible rationale Standard chemotherapy vs control not shown beneficial in stage II patients Standard chemotherapy vs control not shown beneficial in stage II patients This trial not significant for experimental vs standard chemotherapy This trial not significant for experimental vs standard chemotherapy

18 Stage II trial: QUASAR Chemotherapy (n = 1622)* Observation (n = 1617) No clear indication for chemotherapy (n = 3239) RANDOMIZERANDOMIZE Colon or rectal cancer Stage I-III Complete resection with no evidence of residual disease Gray et al. ASCO 2004. Abstract 3501. At: http://www.asco.org/ac/1,1003,_12-002511-00_18-0026-00_19-0010698,00.asp. Accessed November 2004.

19 % of Patients QUASAR: Overall Survival P =.02 5-year OS, Observation = 77.4% vs Chemotherapy = 80.3% Relative risk = 0.83 (95% CI, 0.71-0.97) Years Gray et al. ASCO 2004. Abstract 3501. At: http://www.asco.org/ac/1,1003,_12-002511-00_18-0026-00_19-0010698,00.asp. Accessed November 2004. 012345678910 0 20 40 60 80 100 Observation (n=1622) Chemotherapy (n=1617)

20 Implication: Stage II patients Compared to control, 5-FU provides 2-3% in OS, statistically significant in a single trial Compared to control, 5-FU provides 2-3% in OS, statistically significant in a single trial Debate over clinical relevance Debate over clinical relevance In a large trial, FOLFOX provides 3-4% in DFS compared to 5-FU, not statistically significant for stage II alone In a large trial, FOLFOX provides 3-4% in DFS compared to 5-FU, not statistically significant for stage II alone No hint of interaction between rx and stage, p = 0.77 No hint of interaction between rx and stage, p = 0.77 On its own, debatable benefit compared to 5-FU On its own, debatable benefit compared to 5-FU Cross trial comparison: FOLFOX may result in 5- 7% improvement vs control, but not approved Cross trial comparison: FOLFOX may result in 5- 7% improvement vs control, but not approved No debate about clinical relevance No debate about clinical relevance Grothey & Sargent, JCO 2005

21 Stage II Colon Cancer: Lessons Learned Decisions based on subgroups may seem rational at the time, but lead to unintended consequences Decisions based on subgroups may seem rational at the time, but lead to unintended consequences Results may make further trials impossible (FOLFOX vs control) Results may make further trials impossible (FOLFOX vs control) Need better approaches to analyze subgroups using modeling (or meta- analyses), not individual trial results Need better approaches to analyze subgroups using modeling (or meta- analyses), not individual trial results

22 Potential solution for prospectively defined subgroups: Hierarchical models Goal: Test a treatment in a number of populations Goal: Test a treatment in a number of populations Hypothesis: Effect may depend vary between populations Hypothesis: Effect may depend vary between populations Example: Targeted cancer therapy Example: Targeted cancer therapy Mechanism of action based therapy Mechanism of action based therapy Multiple tumor types express target, to varying degrees Multiple tumor types express target, to varying degrees

23 Basic statistical formulation Suppose N subgroups, with mean response i, i=1,...N Suppose N subgroups, with mean response i, i=1,...N Assume i Assume i ~ N(, 2 ) If Bayesian, put a prior on 2 Depending on estimate of 2, allows heterogeneity between subgroups Easily extends to non-normal models

24 Hierarchical Model: Example Phase II clinical trial of a new agent specifically targeted at patients with a methylated MGMT promoter Phase II clinical trial of a new agent specifically targeted at patients with a methylated MGMT promoter Prevalence from 10% to 60% across various cancer types Prevalence from 10% to 60% across various cancer types High prevalence seen in Head and Neck, Esophageal, Colorectal, and Non Small-Cell Lung Cancer High prevalence seen in Head and Neck, Esophageal, Colorectal, and Non Small-Cell Lung Cancer Goal: Determine if overall efficacy > 10%, but efficacy may depend on tumor type Goal: Determine if overall efficacy > 10%, but efficacy may depend on tumor type

25 Logistic regression Example Hierarchical logistic model for tumor response Hierarchical logistic model for tumor response Stopping rules for each tumor site Stopping rules for each tumor site P ( Response rate i > 10%) 10%) < 10% OR P (Response rate i > 10%) 10%) 10%) 10%) < 10% Simulation for operating characteristics Simulation for operating characteristics Benefits Benefits Single trial (opposed to 4) Single trial (opposed to 4) Use all data formally but flexibly Use all data formally but flexibly

26 Survival Example Survival following chemotherapy for colon cancer Survival following chemotherapy for colon cancer Pooled analysis of 5 trials, suggestion of a study-specific treatment effect (a different type of subgroup) Pooled analysis of 5 trials, suggestion of a study-specific treatment effect (a different type of subgroup) Fit a random effect Cox model Fit a random effect Cox model (t; x) = 0i (t) exp (x i ) (t; x) = 0i (t) exp (x i ) i i ~ N(, 2 ) Can either model 0 parametrically, or use Cox model

27 Model Results Study Single fixed Treatment Effect Study Specific Fixed Treatment Effect Study Specific Random Treatment Effect Overall -0.22 (0.14) -0.21 (0.20) 1 -0.25 (0.34) -0.21 (0.20) 2 0.24 (0.35) -0.11 (0.22) 3 -0.28 (0.33) -0.22 (0.20) 4 -0.25 (0.20) -0.22 (0.17) 5 -1.10 (0.68) -0.29 (0.28) Prior mean for precision (1/ 2 ) = 50, posterior mean 106, Little evidence of heterogeneity Sargent et al, 2000

28 Another approach: Modeling Interactions using Shrinkage Subgroup analyses are fundamentally looking at interactions Subgroup analyses are fundamentally looking at interactions In multi-factor experiment, the number of interactions can explode In multi-factor experiment, the number of interactions can explode Well known that shrinkage (or model averaging) provides much better performance than all or nothing approach (stepwise) Well known that shrinkage (or model averaging) provides much better performance than all or nothing approach (stepwise) Idea: Include interactions in model, but shrink them away if they are not strongly supported by the data Idea: Include interactions in model, but shrink them away if they are not strongly supported by the data

29 Another approach: Modeling Interactions using shrinkage Dental Experiment Dental Experiment Dentures are often made with a soft liner between the gums and the hard denture base Dentures are often made with a soft liner between the gums and the hard denture base Polishing the liner can cause a gap between the liner and the base Polishing the liner can cause a gap between the liner and the base Such gaps harbor pathogens like Candida Such gaps harbor pathogens like Candida The experiment The experiment Main interest: new vs. standard soft liner material Main interest: new vs. standard soft liner material Factor M:2 materials Factor M:2 materials Factor P:4 polishing methods Factor P:4 polishing methods Factor F:8 finishing methods Factor F:8 finishing methods Fully crossed design, no replication Fully crossed design, no replication Outcome measure: gap btwn liner & base, in log 10 mm Outcome measure: gap btwn liner & base, in log 10 mm Pesun, Hodges & Lai (2002) J. Prosthetic Dentistry

30 Smoothing interactions: Smoothed ANOVA Fit full ANOVA model (include all interactions) Fit full ANOVA model (include all interactions) y = X + y = X + y is 64 x 1, contains log 10 gap y is 64 x 1, contains log 10 gap e is 64 x 1, normal mean 0, precision 0 I 64 e is 64 x 1, normal mean 0, precision 0 I 64 X is 64 x 64 X is 64 x 64 is 64 x 1; we will smooth/shrink its elements is 64 x 1; we will smooth/shrink its elements 12 main effects, 52 interactions 12 main effects, 52 interactions Model interactions Model interactions k ~ N (0,1/ k ), k=13, …, 64 k ~ N (0,1/ k ), k=13, …, 64 Large k implies k shrunk toward 0Large k implies k shrunk toward 0

31 Smoothed ANOVA: The model/prior for the k How to model the interactions How to model the interactions Each interaction smoothed by its own k Each interaction smoothed by its own k Each effect's k are all the same, effect Each effect's k are all the same, effect All two-way interactions are smoothed by a single All two-way interactions are smoothed by a single Mix the above options Mix the above options Use priors on k to specify desired operating characteristics for interactions Use priors on k to specify desired operating characteristics for interactions

32 Use Degrees of Freedom to set priors for the k Hodges & Sargent (2001 Biometrika) extended methods for computing DF in standard ANOVA to linear hierarchical models Hodges & Sargent (2001 Biometrika) extended methods for computing DF in standard ANOVA to linear hierarchical models Hodges et al (Technometrics, 2006) present methodology to use DF to set priors Hodges et al (Technometrics, 2006) present methodology to use DF to set priors Example: I want the 51 2-way interactions to share 5 degrees of freedom Example: I want the 51 2-way interactions to share 5 degrees of freedom See references for technical details See references for technical details Ongoing work: extending to non-linear (Cox) models Ongoing work: extending to non-linear (Cox) models

33 Summary: Smoothed ANOVA Subgroup analyses are fundamentally looking at interactions Subgroup analyses are fundamentally looking at interactions A priori have low probability of a significant interaction, but dont want to exclude the possibility A priori have low probability of a significant interaction, but dont want to exclude the possibility Idea: Include interactions in model, but shrink them Idea: Include interactions in model, but shrink them

34 Summary Subgroup analysis is essential to clinical research Subgroup analysis is essential to clinical research People usually perform such analyses with best of intentions People usually perform such analyses with best of intentions Up-front thought can allow us to Up-front thought can allow us to Carefully define population under study Carefully define population under study Pre-specify sub-populations to be examined Pre-specify sub-populations to be examined Hierarchical/Shrinkage models offer attractive possibilities for addressing subgroups, if defined prospectively Hierarchical/Shrinkage models offer attractive possibilities for addressing subgroups, if defined prospectively

35 Thank You Acknowledgements Acknowledgements Smoothed ANOVA: Jim Hodges Smoothed ANOVA: Jim Hodges Colon Cancer: Axel Grothey, Aimery deGramont, Sharlene Gill Colon Cancer: Axel Grothey, Aimery deGramont, Sharlene Gill


Download ppt "Subgroup Analyses: Can We Smooth' out the Rough Edges? Daniel Sargent, PhD Mayo Clinic Sept 28, 2006."

Similar presentations


Ads by Google