Presentation is loading. Please wait.

Presentation is loading. Please wait.

1 The analysis of longitudinal studies in rheumatology: Is it done correctly? Emmanuel Lesaffre Department of Biostatistics, Erasmus MC, Rotterdam, the.

Similar presentations


Presentation on theme: "1 The analysis of longitudinal studies in rheumatology: Is it done correctly? Emmanuel Lesaffre Department of Biostatistics, Erasmus MC, Rotterdam, the."— Presentation transcript:

1 1 The analysis of longitudinal studies in rheumatology: Is it done correctly? Emmanuel Lesaffre Department of Biostatistics, Erasmus MC, Rotterdam, the Netherlands L-Biostat, K.U.Leuven, Leuven, Belgium 6th Annual Clinical Research Methodology Course 6th Annual Clinical Research Methodology Course Friday, December 16, 2011 In collaboration with Karolina Sikorska, Maurits de Rotte, Jolanda Luime, Mieke Hazes

2 2 Conclusions 1.Many longitudinal studies in rheumatology are not using an appropriate statistical methodology. 2.Repeated significance tests and summary statistics methods are inappropriate in practice and can not show the evolution of an outcome over time. 3.Classical repeated measurement methods are inappropriate in practice. 4.The mixed effects model and the GEE approach can make use of all available cases independently whether they have data on all time points studied. 5.The mixed effects model and the GEE approach are sparsely used in longitudinal studies rheumatology.

3 3 Description JIA study

4 4 4 Juvenile Idiopathic Arthritis (JIA) study Description –Longitudinal study: 5 time points (0, 3, 6, 9, 12 months) –Recruitment at University Medical Centre Utrecht (UMCU) and Wilhelmina Children’s Hospital, the Netherlands –Children diagnosed with JIA according to the ILAR criteria and started methotrexate (MTX) therapy between 1990 and 2006 –All patients and their parents gave their informed consent and approved by the medical ethics committee of the UMCU –Treatment was tight-controlled using a standardized report form on disease activity every 3 months –Information on MTX usage, disease activity, route of administration, dosing of MTX, reasons for ending MTX treatment, concomitant therapy and laboratory parameters were collected

5 5 5 JIA study Description –183 JIA patients of our test cohort were genotyped for ABCB1 3435C>T. –2 groups: genotype CC homozygous (53 patients, 28%)  T-carrier (129 patients, 72%) –Continuous response = erythrocyte sedimentation rate (ESR) was obtained on all 5 visits for 106/183 patients (58%): CC homozygous (31 patients, 29%), T-carrier (75 patients, 71%) = complete cases –ESR was log-transformed because of a skewed distribution. –Binary response = ACR30 response, obtained for all 183 patients on all visits. Of the 183 patients, 46 (25%) responded at 3 months, 85 (46%) at 6 months, 110 (60%) at 9 months and 113 (62%) at 12 months.

6 6 Classical analyses for ESR

7 7 7 JIA study Classical analysis (ESR)

8 8 8 Classical approaches for comparing 2 groups over time 1.Repeated significance tests (t-tests, Wilcoxon tests, etc) 2.Summary Statistic Method: summarizing whole curve by e.g. AUC and compare AUC between 2 groups with t-test 3.Repeated measurements ANOVA 4.MANOVA

9 9 1. Repeated significance tests

10 10 1. Repeated significance tests 1. Repeated significance tests JIA study (ESR) Here unpaired t-tests giving P= 0.066, 0.12, 0.62, 0.33, 0.44 Nowhere significant  no real issue here Suppose P= 0.066, 0.03, 0.62, 0.33, 0.44 … what to conclude? Or P= 0.066, 0.12, 0.62, 0.33, 0.03 … what to conclude? If no adjustment of P-value, then problem of multiple testing

11 11 1. Repeated significance tests Popular approach in medical research But –Inefficient: only patients are included that are present at visit –Likely to be biased: missing data process can bias analysis –No good insight in individual evolution: link between subsequent observations is lost –Turns longitudinal study into several cross-sectional studies Not suitable for contemporary rheumatology studies

12 12 JIA study CC homozygous T-carrier All individual profiles (ESR)

13 13 JIA study 3 individual profiles (ESR)

14 14 2. Summary Statistic Method 2. Summary Statistic Method JIA study (ESR) Compute Area-under-the-Curve (AUC) for each profile On complete cases only: CC-Homozygous – T-carrier= -1.33, P = 0.54, 95% CI=(-5.61; 2.96) Complete the incomplete cases by LOCF: CC-Homozygous – T-carrier= -3.78, P = 0.04, 95% CI=(-7.33; -0.21) In addition one could compare –Averages of the profiles –….

15 15 JIA study 3 individual profiles (ESR): incomplete profiles completed by Last-Observation-Carried-Forward (LOCF)

16 16 2. Summary Statistic Method Not frequently used in medical research Recommended for balanced data –Balanced: all patients are examined at all (regular) visits –Difficult for unbalanced data, i.e. when there are missing observations and patients can come at all times –LOCF is a popular approach to make data more balanced LOCF –Imputes unrealistic values –Underestimates variability of the data Not suitable for contemporary rheumatology studies

17 17 3. Repeated Measurements ANOVA One of only 2 approaches available for analysis of repeated measurements 50 years ago Requires balanced data –Patients with missing values are excluded –Time points must be regular, patients with irregular time points are excluded –Restrictive assumptions on correlation between subsequent responses: correlations are equal (corrections available but not sufficient) –Splits treatment effect up into time, group and group*time effect Not suitable for contemporary rheumatology studies

18 18 3. Repeated Measurements ANOVA 3. Repeated Measurements ANOVA JIA study (ESR) On complete cases only: –CC-Homozygous – T-carrier= Time effect: P < Genotype effect: P=0.47 Time*genotype effect: P=0.02 Complete the incomplete cases by LOCF: –CC-Homozygous – T-carrier= Time effect: P < Genotype effect: P=0.04 Time*genotype effect: P=0.34

19 19 3. Repeated Measurements ANOVA 3. Repeated Measurements ANOVA JIA study (ESR) Assumption equal correlation?? TIME (months)

20 20 4. MANOVA Other of only 2 approaches available for analysis of repeated measurements 50 years ago Requires balanced data –Patients with missing values are excluded –Time points must be regular, patients with irregular time points are excluded –No structure allowed on correlations  large studies are required –Splits treatment effect up into time, group and group*time effect –Output more difficult to understand Not suitable for contemporary rheumatology studies

21 21 4. MANOVA 4. MANOVA JIA study (ESR) On complete cases only: –CC-Homozygous – T-carrier= Time effect: P < Genotype effect: P=0.47 Time*genotype effect: P=0.03 Complete the incomplete cases by LOCF: –CC-Homozygous – T-carrier= Time effect: P < Genotype effect: P=0.04 Time*genotype effect: P=0.36

22 22 Modern analyses for ESR

23 23 “Modern” approaches for comparing 2 groups over time 1.Types of missing data processes 2.Approaches to deal with missing data 3.Mixed effects models 4.GEE models

24 24 1. Types of missing data processes What are missing data? Data that are not observed How are missing data generated? Patients/clinicians forget to fill item(s) in questionnaire Patients refuse to fill item(s) in questionnaire Lost or damaged biological sample Patients miss a visit because … Patients decide not to return to clinician anymore Patient died …  Different reasons why data are missing

25 25 Impact of missing data What is believed Loss of efficiency In reality Loss of efficiency Biased results (often)  Classical statistical methods need to be adapted Approach CLEVER explicit or implicit imputation of missing data ButNEVERBut: can NEVER replace true data

26 26 Terminology Monotone missing Also called dropouts   n Non-monotone missing Also called intermittent missingness   n

27 27 More terminology Missing completely at random Missing at random Missing not at random – informative missing Terminology might be confusing – Rubin (1975)

28 28 Missing completely at random (MCAR) Probability of missingness is independent of all responses Examples A random selection of teeth in mouth are taken in the study A blood tube is dropped Patient died in a car accident, but careful: patient could have experienced a sleep attack when taking a dopamine agonist Then Simple mean of response is unbiased estimate of true mean Classical statistical techniques (repeated t-tests, Summary statistics method, repeated measurements ANOVA, MANOVA) can be used Impact: loss of efficiency

29 29 Missing at random (MAR) Probability of missingness depends on observed responses Examples Study design specifies that if blood pressure is not lowered patient will be removed from anti-hypertensive trial Multi-stage screening: data are missing at subsequent stages due to result at initial stage (negative test) Then Simple mean of response is biased estimate of true mean Classical statistical techniques canNOT be used Statistical tests to distinguish versus MCAR exist Impact: loss of efficiency + bias, but likelihood analysis can correct for bias due to missingness

30 30 Missing not at random (MNAR) Probability of missingness depends on observed responses & unobserved responses Examples Patient shows a flare up in the disease unobserved in the study + patient decides to leave the study Then Simple mean of response is biased estimate of true mean Classical statistical techniques canNOT be used No test to distinguish versus MAR no satisfactory analysis, ONLY sensitivity analysisThere is no satisfactory analysis, ONLY sensitivity analysis Impact: loss of efficiency + bias, only a sensitivity analysis can shed light on the problem

31 31 Bias of simple mean COMPLETE MCAR MAR

32 32 2. Approaches to deal with missing data Prevention and planning Analytical remedies Complete case analysis Available case analysis Imputation techniques Likelihood-based analyses Weighted analyses Sensitivity analyses  Most appropriate statistical solutions are COMPLICATED

33 33 2. Approaches to deal with missing data Complete case analysis (MCAR) –Default analysis in many packages, only ok for MCAR –If not MCAR: substantial bias can be the result Available case analysis (MCAR) –Use for each variable/time point all observations available Single value imputation (MCAR, MAR??) –Mean value imputation –Hot decking –LOCF methods are based on unrealistic models & underestimate variance

34 34 Example: LOCF LOCF time

35 35 2. Approaches to deal with missing data Multiple imputation (MAR) –Explicit imputation of missing data –Incorporate random mechanism –Generate M different completed imputed data sets –Combine M means and M variances  1 overall mean & variance method is based on statistical model for imputation

36 36 2. Approaches to deal with missing data Likelihood-based models (MAR) –Implicit –Implicit imputation of missing values –Model-based: Linear & generalized linear regression Linear & generalized linear mixed models … method is based on statistical model for response GEE models (MCAR) and weighted GEE models (MAR) Bayesian models (MAR)

37 37 2. Approaches to deal with missing data MNAR models –Model also missing data mechanism –Complex modelling sensitivity analysis –Never completely satisfactory  sensitivity analysis –BUT: –BUT: if time points are close to each then MNAR close to MAR

38 38 3. Mixed effects models Two examples

39 39 3. Mixed effects models Assumption in mixed effects models

40 40 3. Mixed effects models Not enough used YET in medical research Recommended for contemporary rheumatology studies –Allows unbalanced data –None of the patients are deleted from the study –Irregular time points are allowed –No explicit imputation of responses –Time evolution, effect of covariates and correlation structure: all can be flexible and modelled BUT –One needs statisticians for the job –Luckily: they are OFTEN cheap

41 41 3. Mixed effects models Linear mixed effects models –Response = continuous –Normality assumptions Generalized linear mixed effects models –Response = continuous or discrete –Normality assumptions

42 42 3. Mixed effects models (ESR) 3. Mixed effects models JIA study (ESR) On ALL cases: –CC-Homozygous – T-carrier= Time effect: P < Genotype effect: P=0.03 Time*genotype effect: P= ESTIMATES and 95% CIs

43 43 3. Mixed effects models (ESR) 3. Mixed effects models JIA study (ESR)

44 44 4. GEE models Can cope only with MCAR Makes only few assumptions about distribution of data Most popular for discrete responses Can also be used for continuous response

45 45 4. GEE models 4. GEE models JIA study (ACR30) Binary response No missing data in the study Possible analyses –Chi-square tests (Fisher’s exact tests) at each time point –Generalized mixed effects model –GEE model Same comments as before Here all analyses gave non-significant difference between genotype groups

46 46 Literature review

47 47 Literature review Literature search (criteria) : –PubMed –January 1- December 31, 2008 –Annals of the Rheumatic Diseases & Arthritis and Rheumatism –Patients with arthritis were followed up in time over > 3 measurements –Response that could vary over time Information collected: –Research aim –Primary outcome, secondary outcomes –Used technique for handling with missing data –Statistical analysis techniques used Eligible studies were ranked according to degree they correctly analyzed the longitudinal research question

48 48 Literature review Study characteristics 203 longitudinal studies in ARD or in AR 156 out of these studies dealt with arthritis –11 studies with JIA patients –58 studies described a RCT –98 studies described a longitudinal cohort study 110 studies were included (excluded: 30 studies not longitudinal, 1 study was not first published in 2008, 16 studies did not include patients with arthritis) Statistical techniques used 110 studies were ranked 17 made optimal use of modern statistical methods

49 49 Literature review Ranking DescriptionNumber of studies Suitable for repeated measurement analysis110 1Repeated character of the data not taken into account78 2Use of Bonferroni correction for multiple testing7 2Use of Repeated Measurements ANOVA or MANOVA8 3Appropriate repeated measurement techniques used on all suitable research questions 17

50 50 Conclusions

51 51 Conclusions 1.Many of the longitudinal studies in rheumatology are not using an appropriate statistical methodology. 2.Repeated significance tests and summary statistics methods are inappropriate in practice and can not show the evolution of an outcome over time. 3.Classical repeated measurement methods are inappropriate in practice. 4.The mixed effects model and the GEE approach can make use of all cases independently whether they have data on all time points studied. 5.The mixed effects model and the GEE approach are sparsely used in longitudinal studies rheumatology.

52 52 Thank you for your attention


Download ppt "1 The analysis of longitudinal studies in rheumatology: Is it done correctly? Emmanuel Lesaffre Department of Biostatistics, Erasmus MC, Rotterdam, the."

Similar presentations


Ads by Google