Presentation is loading. Please wait.

Presentation is loading. Please wait.

James Hung, 2004 FDA/Industry Management of Missing Data in Clinical Trials from a Regulatory Perspective H.M. James Hung Div. of Biometrics I, OB/OPaSS/CDER/FDA.

Similar presentations


Presentation on theme: "James Hung, 2004 FDA/Industry Management of Missing Data in Clinical Trials from a Regulatory Perspective H.M. James Hung Div. of Biometrics I, OB/OPaSS/CDER/FDA."— Presentation transcript:

1 James Hung, 2004 FDA/Industry Management of Missing Data in Clinical Trials from a Regulatory Perspective H.M. James Hung Div. of Biometrics I, OB/OPaSS/CDER/FDA Presented in FDA/Industry Workshop, Bethesda, Maryland, September 23, 2004

2 James Hung, 2004 FDA/Industry Collaborators Charles Anello, Yeh-Fong Chen, Kun Jin, Fanhui Kong, Kooros Mahjoob, Robert ONeill, Ohid Siddiqui Office of Biostatistics, OPaSS, CDER Food and Drug Administration

3 James Hung, 2004 FDA/Industry Disclaimer The views expressed in this presentation are not necessarily of the U.S. Food and Drug Administration. Acknowledgment ONeill (2003, 2004) Temple ( )

4 James Hung, 2004 FDA/Industry Outline Informative dropout Statistical analysis methods Methodology consideration Summary

5 James Hung, 2004 FDA/Industry Clinical trial focuses on intent-to-treat population (including completers and dropouts) Response variables often measured over time (e.g., at multiple clinic or hospital visits)

6 James Hung, 2004 FDA/Industry Often the main clinical hypothesis concerns the effect K of a test drug r.t. a control at some time K (e.g., end of study). Statistical null hypothesis H 0 : K = 0 i.e., allow nonzero at other time points? (make sense?)

7 James Hung, 2004 FDA/Industry Unclear why testing only at the last time point is most relevant ( for simplicity? avoid statistical adjustment for testing multiple times? ) Drug effects over time are important information. e.g., inconceivable to market a drug that is effective only at Week 6, say.

8 James Hung, 2004 FDA/Industry For drug effect over time (or some period of time, e.g., at steady state), the relevant null hypothesis is H 0 : 1 = = K = 0 or H 0 : slope difference = 0 (if response follows straight-line model ) or others for relevant time period.

9 James Hung, 2004 FDA/Industry Informative Dropout In many disease areas, dropout rate is high and the results of any analyses for ITT population is not interpretable because of a large amount of missing data, particularly when dropouts are informative.

10 James Hung, 2004 FDA/Industry Dropout problems are multi-dimensional e.g., dropping out due to multiple reasons: side effects of the drug, health state is worsening, unperceived benefit Little knowledge of real causes of missing data, whether missing mechanism related to study outcome or treatment

11 James Hung, 2004 FDA/Industry Informative dropout has many different definitions, e.g., - dependent on observed data, dependent on missing data, treatment-related dropout, … - tied in with missing mechanism MCAR, MAR, MNAR, NIM, … ONeill (2003, 2004)

12 James Hung, 2004 FDA/Industry For regulatory consideration, any treatment related dropout may be a suspect of informative dropout and missing mechanism probably needs to be considered informative (i.e., may severely bias estimates and tests) unless proven otherwise.

13 James Hung, 2004 FDA/Industry In a clinical trial, each cohort of dropout by reason or by dropout time can be very small. Difficult or impossible to assess whether missing values are informative.

14 James Hung, 2004 FDA/Industry

15

16 Based on visual inspection, drug seems to perform better than Placebo.

17 James Hung, 2004 FDA/Industry

18

19

20

21 Difficult to tell whether missing mechanism is ignorable or not… e.g., in a linear response profile, MAR May be NIM.

22 James Hung, 2004 FDA/Industry

23

24

25 These plots show difficulty in classifying dropouts (informative or not) in individual trials where each cohort of dropout is small, (though total dropout rate could be high). These types of analysis should be done with external historical trials, at least for classification purpose.

26 James Hung, 2004 FDA/Industry Statistical Analysis Methods Literature guidance 1) No satisfactory statistical analysis method for handling non-ignorable missing data 2) Likelihood-based methods require assumptions about missing data mechanism (unverifiable from current trial data)

27 James Hung, 2004 FDA/Industry Facts 1)Validity of any analysis method is very much in question. 2) Better alternative method is unclear. Use of current trial data to seek imputation method is futile. 3) Dropouts and missing data are unavoidable.

28 James Hung, 2004 FDA/Industry Glimpse of the analysis problem = µ 1 - µ 2 at last time point n i = # of completers in group i f i = n i /N i If there is no missing value, we have D = Y 1 – Y 2 (unbiased for ) V(D) = estimated variance of D Z = D/[V(D)] 1/2

29 James Hung, 2004 FDA/Industry Missing values { D, V(D), Z } not obtainable. Can try to get E( D | data) and V( D | data). and construct Z* = E( D | data ) / [V( D | data )] 1/2 or Z + = E( D | data ) / [V(D)] 1/2

30 James Hung, 2004 FDA/Industry Y oi = sample mean of completers R i = vector of indicators for completion or dropout Y mi = unobservable sample mean of dropouts

31 James Hung, 2004 FDA/Industry Immediately, when f 1 f 2, this statistic has problem of interpretation, unless R i and Y mi are independent (MI). Under MI, E(Y mi | Y oi, R i ) = E(Y mi ). And if E(Y mi ) = µ i, then completer analysis might offer a reasonable estimate of.

32 James Hung, 2004 FDA/Industry When f 1 = f 2 = f, a linear combination of obs sample mean difference of completers and difference in conditional mean of dropouts (the latter requires models).

33 James Hung, 2004 FDA/Industry What about Var (D | data)? Another formidable task ! Nonlikelihood-based methods are difficult to provide useful solutions unless some kind of ad-hoc conservative imputation is feasible.

34 James Hung, 2004 FDA/Industry LOCF (last observation carried forward) LOCF tests H 0 : K = 0. LOCF can be biased either in favor of test drug (e.g., when its effect decays over time*) or against test drug, even in case of MCAR. *Siddique and Hung (2003)

35 James Hung, 2004 FDA/Industry For assessing drug effect over time, LOCF can seriously underestimate variability of measurement and is unrealistic (i.e., impute a constant value for every visit after the patient dropped out).

36 James Hung, 2004 FDA/Industry LAO (last available observation) Operationally identical to LOCF, this tests some global drug effect over time, H 0 : w 1h µ 1h = w 2h µ 2h W ih = E(dropout rate of drug group i at time h) μ ih = expected response of patients dropping out after time h in drug group i Is this null hypothesis relevant? Shao and Zhong (2003)

37 James Hung, 2004 FDA/Industry v0 v1 v2 v3 Y LOCF versus LAO (in red)

38 James Hung, 2004 FDA/Industry The global mean µ i = w ih µ ih can be unbiasedly estimated by the sample mean. But the usual MSE from ANOVA may not estimate right target (Shao and Zhong). LAO results can be difficult to interpret if dropout reasons or dropout rates are different in treatment groups.

39 James Hung, 2004 FDA/Industry If drug effect over time is at issue, why not use all the pertinent data (longitudinal data analysis should be more efficient than LAO). - need medical colleagues buy in Ex. Analysis of cuff BP over time may be more powerful (value of test statistic is much larger) than LAO Hung, Lawrence, Stockbridge, Lipicky (2000)

40 James Hung, 2004 FDA/Industry MMRM * (mixed-effect model repeated measure with saturated model) Response = µ + treatment + time + treatment*time + baseline + subject (treatment) + error subject (treatment) and error are random effects treatment and time are class variables *Mallinckrodt et al (2001)

41 James Hung, 2004 FDA/Industry MMRM * analysis used to test H 0 : K = 0. - statistically valid under MAR - seem more stable in terms of type I error rate than LOCF under MCAR or MAR*# (LOCF can be very bad, depending on at other visits) *Mallinckrodt et al (2001) #Siddique and Hung (2003)

42 James Hung, 2004 FDA/Industry LOCF, LAO, MMRM can be very problematic in case of informative missing. Dont know how to do conservative imputation with these methods.

43 James Hung, 2004 FDA/Industry Worst rank/score analysis Test drug effect at time K in the presence of events (e.g., death) that cause informatively missing values of the primary study outcome at time K. Example: In congestive heart failure trials, exercise time is missing after death from heart failure. Lachin (1999)

44 James Hung, 2004 FDA/Industry Assign a worst score to any informatively missing values (due to occurrence of an absorbing event related to progression of disease) and perform a nonparametric rank analysis. Valid and efficient for testing H 0 : no treatment difference in distributions of both event time and main study outcome Lachin (1999)

45 James Hung, 2004 FDA/Industry For a drug having little effect on non- mortal outcome (e.g., exercise time), this analysis when used to test non-mortal effect can be anti-conservative if the drug improves survival. Unclear how to perform a reasonable test for the non-mortal effect alone (e.g., labeling issue)

46 James Hung, 2004 FDA/Industry Time to treatment failure analysis In time to event analysis, if test drug has severe side effects that cause more dropouts, then time to treatment failure (event or dropping out due to side effects) analysis may provide a conservative analysis.

47 James Hung, 2004 FDA/Industry Like the worst score/rank analysis, it is unclear how to perform a reasonable test for time to the interested event alone - censoring on dropout due to failure ?

48 James Hung, 2004 FDA/Industry WLP opposite/pooled imputation For binary outcome, opposite imputation imputes sample event rate of completers in one arm for unobserved event rate of incompleters in the opposite arm. Wittes, Lakatos, Prostfield (1989) Proschan et al (2001)

49 James Hung, 2004 FDA/Industry Pooled imputation imputes sample event rate of completers from both arms for unobserved event rate of noncompleters in each arm. Treat imputed rate as ordinary rate. Compute Z statistic in the ordinary manner using a combination of the observed and the imputed rates. Wittes et al (1989), Proschan et al (2001)

50 James Hung, 2004 FDA/Industry WLP is less conservative than the worst case analysis (assign event to dropouts in the test drug group and nonevent to dropouts in the control group). Proschan et al (2001)

51 James Hung, 2004 FDA/Industry Partial list of other well-known methods Likelihood-based method Pattern-mixture model selection model Non-likelihood based method GEE Ad hoc imputation method

52 James Hung, 2004 FDA/Industry Methodology Consideration ONeill (2003, 2004) - better assume NIM in planning stage missing data process not directly verifiable - choice of approach as the primary strategy for handling missing data ? - choice of approaches for sensitivity analysis, robustness analysis ?

53 James Hung, 2004 FDA/Industry Unnebrink and Windeler (2001) adequacy of ad hoc strategy ( e.g., LOCF, ranking, imputation of mean of other group, etc ) for handling missing value depends on whether the courses of disease are similar in the study groups For large dropout rates or different courses of disease, no adequate recommendations can be given

54 James Hung, 2004 FDA/Industry In planning strategies for handling missing values, we need to consider: 1)Null hypothesis should be carefully defined in anticipation of missing data. It should not be altered by the presence of missing data after trial is done, regardless of their pattern.

55 James Hung, 2004 FDA/Industry 2) For design, every attempt needs to be made to minimize dropouts. Alternative designs (e.g., enrichment design*, randomized withdrawal * ) may be used to narrow the study population (recognize problem of generalizability), if ITT population cannot be properly studied. *Temple (2004)

56 James Hung, 2004 FDA/Industry 3) For analysis, the method needs to facilitate conservative imputation to: - adjust the effect estimate toward null - inflate variability (double discounting for possible exaggeration from imputation of missing data), e.g., some type of worst score or rank.

57 James Hung, 2004 FDA/Industry 4) Seek missing mechanism model to help imputation. This needs to use knowledge of disease process (how? Need to get practical experiences) The model needs to be flexible for sensitivity/robustness analysis. Note: such model is not verifiable

58 James Hung, 2004 FDA/Industry 5) Conduct better pilot trials or analyze historical data to explore response profiles of dropouts by reasons to see if missing mechanism may be related to outcome, and propose a reasonably conservative imputation method

59 James Hung, 2004 FDA/Industry Key to reasonable imputation = µ 1 - µ 2 at last time point n i = # of completers in group i If there is no missing value, we have D = Y 1 – Y 2 (unbiased for ) V(D) = estimated variance of D Z = D/[V(D)] 1/2

60 James Hung, 2004 FDA/Industry Missing values { D, V(D), Z } not obtainable. Can try to get E( D | data) and V( D | data). And thus we construct Z* = E( D | data ) / [V( D | data )] 1/2 or Z + = E( D | data ) / [V(D)] 1/2 All need models. Proschan et al (2001)

61 James Hung, 2004 FDA/Industry Goal is to use of a model such that |Z*| |Z| or |Z + | |Z|. Since functional forms of E(D | data) and V(D | data) are unavailable, use of linear model to remove 1 st -order effect of data is the first step. Then, what is the impact of imposing such model on estimation of V(D | data) or V(D)?

62 James Hung, 2004 FDA/Industry SUMMARY Intent-to-treat is the goal. If the dropout rate is high, interpretable intent-to-treat analysis may not be achievable. Alternative designs (e.g., enrichment design) that narrow study population may need to be considered (caveat: generalizability of interpretation).

63 James Hung, 2004 FDA/Industry Intuitively, use of all data seems to be more promising than use of end point data to offer better guidance as to how to reasonably impute missing values. Yet, this advantage comes with a price that unverifiable statistical models must be dependent on. Thus, every method needs to facilitate conservative imputation approach.

64 James Hung, 2004 FDA/Industry For regulatory applications, every attempt needs to be made to: - minimize dropout - explore response pattern of dropout in order to be able to propose a reasonably conservative imputation method - propose conservative strategies for primary analysis and sensitivity analyses

65 James Hung, 2004 FDA/Industry Selected References Lachin (1999, Controlled Clinical Trials) Unnebrink, Windeler (2001, Statistics in Medicine) Shao, Zhong (2003, Statistics in Medicine) Proschan, McMahon, et al (2001, Journal of Statistical Planning and Inference) Wittes, Lakatos, Probstfield (1989, Statistic in Medicine) Mallinckrodt et al (2003, ASA JSM) Siddique, Hung (2003, ASA JSM) Hung, Lawrence, Stockbridge, Lipicky (2000, unpublished manuscript)

66 James Hung, 2004 FDA/Industry ONeill (2003, ASA JSM; 2004, DIA EuroMeeting) Temple ( , Lecture notes on Clinical Trial Designs) Temple (2004, Society of Clinical Trials talk)


Download ppt "James Hung, 2004 FDA/Industry Management of Missing Data in Clinical Trials from a Regulatory Perspective H.M. James Hung Div. of Biometrics I, OB/OPaSS/CDER/FDA."

Similar presentations


Ads by Google