Biostatistics Case Studies 2014 Session 6 An Overview of Missing Data Youngju Pak Biostatistician

Goals for this talk  Familiarize with conceptual and analytical issues on missing data  Raise the awareness of issues relevant to statistical inference when some data is missing  Introduce general methods to prevent and treat missing data, including multiple imputation.

Contents  When and why data are missing  Consequences  Prevention strategies when design studies  Classification  Diagnosis  Statistical methods  Final remarks

What is missing data?  The term missing data means that we are missing some type of information about the phenomena in which we are interested.  Usually leave blank cells in data sets.  Should be distinguished from “Not Applicable”.

The prevalence of missing data.  Among 3 years of publications (about 300) within a prominent psychological journal, about 90% of the articles had missing data.  Average amount of missing data is above 30%. (Source: McKinght, PE et al. 2007, p.3)

When do we miss the data in the research process?  Participant recruitment Survey non-response  Randomization & Implementation of the treatment Preference toward a particular group Refuse to participate in the study after their assignment of the group.  Data collection & Maintenance Subject drop out in longitudinal studies Biological samples get contaminated  Data entry  Data analysis and reporting

Some consequences of missing data  Reliability of measurements with multiple items # of item ↓ → inaccuracy of the variance and covariance of items ↑  Validity of study results (Sample selection) Difference in characteristics between participants and non-participants → Selection bias → Unrepresentative sample (Randomization) Data missing differentially → I nitial nonequivalence (Data analysis) Sample size ↓ → Power ↓  Generalizability of results Any or all of the above → difficulty with statistical inference and interpretation of findings → inaccurate knowledge base → Misinformed and possibly misleading policy recommendations

“The best solution to handle missing data is to have NONE”. R.A. Fisher

How to Prevent missing data?  Overall study design  Characteristics of the target population and the sample  Data collection and measurement  Treatment implementation  Data entry

How to Prevent missing data? 1. Overall study design  Measurement occasions and the timing of data collection Avoid excessive data collection Use existing information such as expected growth curve  Number of variables A strong justification additional variables. “just in case” is poor justification for additional variables.  Assignment to the intervention group Use separate sites or timing to avoid participants preference, wait list control, etc. Increase incentives as the study progress to avoid the drop out due to improvement or adverse results.

How to Prevent missing data? 1. Overall study design cont.  Attrition and retention strategy Differences between participants with complete data and participant with missing data can introduce bias in parameter estimates Multiple retention strategies  Detailed record  Creating a project identity  Developing a screening measures to identify individuals with high risk of drop out  Training and monitoring of research staffs, etc

How to Prevent missing data? 2. Characteristics of the target population  Some strategies In a survey, use appropriate words for the target population.  e.g., use “did not finish” instead of “dropped out” for a group of native Americans Translate questionnaires into the dominant language or face to face interview with low English proficiency. Provide the breaks during the interview for seniors Assure the confidentiality for the sensitive topics (Singer 1995)

How to Prevent missing data? 3. Data collection and measurement  Physiological indices e.g., blood samples-equipment error To prevent  Firm protocols  Random check  Develop solution prior to data collection  Observation of the behavior e.g., Facial expressions To Prevent  Close enough distance  Multiple observers.

How to Prevent missing data? 3. Data collection and measurement cont.  Interviews Inform participants in advance about the conditions and duration of interviews Consider participants preference towards interviewers Sort interview items from easiest to most difficult Careful selections of interviewers and training A computer assisted interview (e.g.,SurveyMonkey) can reduce the potential error of missing data

How to Prevent missing data? 4.Treatment implementation & Data entry  Reduce the treatment burden e.g., multiple sessions with short intervals might be more burden some than long-term intervention with less frequency  Improve treatment administration Consider the characteristic of providers  e.g., if providers are viewed as unskilled or unfriendly, participants are more likely to drop out Avoid circumstances that subjects do not like such as a parking lot that is far from the study site  Data entry Double entry or random cross checking

How to Prevent missing data? -Summary  Most have to do with reducing the burden of participants in studies.  Feasibility must also taken into consideration along with the costs and benefits, when selection prevention strategies. e.g., shorter questionnaires → less missing → the breadth or depth of knowledge ↓  Should design interventions to facilitate adherence and to prevent attrition.  More details can be found in McKnight, et al. 2007, Chapter 4)

Missing Data Classification  How to best carry out statistical inference in the presence of missing data depends on the missing data “mechanism”.  The most widely used missing data classification system was introduced by Rubin, Donald (1976)  Three distinct missing data type based on missing data mechanism 1. Missing Completely At Random (MCAR) 2. Missing At Random (MAR) 3. Not Missing At Random (NMAR)

Missing data classification 1. Rubin’s categories of missing data (Source: McKnight, et al. 2007)

Rubin’s categories of missing data - An example  Suppose interest centers on determining if the following factors effect plasma of beta- carotene Age Gender Current smoking status BMI Alcohol use (average # of drinks/week) Dietary beta-carotene as a covariate (mcg/day) (Source: StatLib data, Dept of Statistics, Carnegie Mellon University)

Rubin’s categories of missing data - An example cont.  Possibly MCAR Some plasma carotene levels are missing  e.g., Some blood samples lost in transport Some dietary carotene missing  e.g., Subjects recruited on a day when the dietician doing the diet-inventory interview calls in sick Some items missing “here and there” due to erratic scanning of data collection forms  Graduate students have not slept very well the night before the work day

Rubin’s categories of missing data - An example cont.  Possibly MAR Missing demographics  Perhaps females tend to omit reporting weight & age Missing dietary beta-carotene  Overweight individuals tend to refuse the beta- carotene dietary-inventory. Clearly not MCAR May be MAR as missing is related to other available variable

Rubin’s categories of missing data - An example cont.  Possibly MNAR Heavy drinkers tend not to respond to drinks per week questions Smokers reluctant to admit such Elderly subjects skip demographic items, such as age, due to poor design of data collection forms Any variable that the probability of an variable being missing is related to the value of that item

Missing data classification 2. Dimension of Missing Data  Missing on the variable (item nonresponses)  Missing on the occasions (wave nonresponses)  Missing on the individual (unit nonresponses) Individuals Variables/Items Occasions Cattell’s data box (1966)

Missing data classification 3.Mechanism or Dimension?  Less missing data (m.d) is better, in general  A large proportion of MCAR might be better than smaller amount of MAR or MNAR, for parameter estimation. Nonetheless, statistical power will be lower  The amount of m.d, in combination with the reason, dimension, and mechanism should be considered in diagnosis and treatments of m.d

D iagnostic procedure  Diagnosis plays important role in selecting the appropriate missing data techniques as well as interpretation of study findings (inferential limitation)  MCAR diagnostics Two sample t-test  Not effective for multivariate data Little’s MCAR test (1988)  A type of chi-square test  A significant p-value means data is not MCAR.  Available in SPSS

Diagnostic procedure cont.  MCAR or MAR ( ignorable) vs. NMAR(non-ignorable) ? No numerical test or graphical test exists m.d is non-ignorable when no information available to explain why the data are missing. Should look at source data outside studies such as previous findings, double sampling, or intensive follow up for non-respondents, etc. Schafer(1997) provides guidance for cases in which ignorability to be plausible and when it is not.

Handling m.d. in data analytic procedures  Four different methods Data deletion method Data augmentation method Single imputation method (SI) Multiple imputation method (MI)

Handling m.d. in data analytic procedures cont. - Data deletion method  Complete case method (listwise deletion) Discard observations with any missing value & only include complete cases Easy to implement If MCAR, parameter estimates are unbiased Can reduce the power substantially  Available case method (pairwise deletion) Discard data only at the level of the variable Can preserve larger portion of the sample If MCAR, parameter estimates are unbiased Results in different sample size per variables such as correlations → stability ↓

Handling m.d. in data analytic procedures cont. 2. Data Augmentation  Avoid many of the inherent problems of deletion methods.  Does not explicitly replace missing values. Instead, an algorithm is invoked that takes into account the observed data, the missing data, the relationship among observed data, and some underlying statistical assumptions when estimating parameters.  Maximum likelihood (ML), Expectation and Maximization (EM), Markov Chain Monte Carlo (MCMC), dummy variable method, and weighting method. Note: SPSS has default program for listwise, pairwise, EM and regression methods for estimation ( Analyze  Missing Value Analysis )

Handling m.d. in data analytic procedures cont. 3. Single Imputation (SI)  Replace a missing value with a single value  Replace with - Constant: zero, mean, median - Random o Hot deck : Randomly selecting a value from the observed data o Cold deck : Use another data set to replace missing values - Nonrandom o Last Observation Carried Forward (LOCF) o Next Observation Carried Backward (NOCB) o Regression predictions

Handling m.d. in data analytic procedures cont. 3. Single Imputation (SI) cont.  SI generally OK with a small amount ( < 5%) of m.d.  SI tend to underestimate standard errors, increasing type I error Ignores the uncertainty in imputed values Performance may depend on variability of items with missing value  SI tend to perform poorly even the missing data mechanism is ignorable.  MI are considered to be superior alternative, particularly MAR case.

Handling m.d. in data analytic procedures Cont. 4. Multiple Imputation (MI).  MI replace each missing value with a set of plausible values that are drawn from a assumed distribution.  Multiple imputations (from 3 to 10 times), repeat analysis with complete data, aggregate results from the analyses.  Pros Provide sound parameter estimates Most highly praised methods for statistically handling missing data (Allison 2002, Rubin 1996, Schafer & Graham 2002)  Cons Require substantial sample size Optimal choice of technique is often unclear May be difficult for less experience researchers due to the specification of the distribution along with assumptions. Sensitivity analysis recommended

Handling m.d. in data analytic procedures Cont. 4. Multiple Imputation (MI) cont.  Plasma Beta-Carotene Example Dependent variable: Natural log of plasma beta-carotene concentration Independent variables: age, gender, current smoking status, BMI, alcohol use, dietary beta carotene (logged) Complete data N=314 Second data set with data MAR  N=216 complete data cases Regression Analysis  Complete Data  Listwise deletion  Multiple Imputation # of imputation (M): 10 times

Handling m.d. in data analytic procedures Cont. 4. Multiple Imputation (MI) cont. Complete Data (N = 315) Parameter Parameter Estimate Standard Error t ValueProb t Intercept <.0001 Age Female CurSmoke r BMI <.0001 Alcohol LBeta_Diet Listwise deletion(N = 216) Parameter Paramete r Estimate Standar d Error t Valu eProb t Intercept < Age Female CurSmoke r BMI < Alcohol LBeta_Diet  Plasma Beta-Carotene Example cont.

Handling m.d. in data analytic procedures Cont. 4. Multiple Imputation (MI) cont. Complete Data Results Parameter Paramete r Estimate Standar d Error t ValueProb t Intercept <.0001 Age Female CurSmoker BMI <.0001 Alcohol LBeta_Diet Multiply Imputed Analysis (M = 10) Estimat e Standa rd ErrorMinMax t Valu eProb t < <  Plasma Beta-Carotene Example cont

Recommend readings for MI  UW-Madison Social Science computing cooperative readings.htm  UCLA : Institute for Digital Research and Education missing_data/mi_in_stata_pt1.htm

Final Remarks  No recipe for the single best approach !  An optimal solution for a particular analysis requires consideration of: Dimensions of missing data The missing data mechanism Reasons for missing data Data types of variables that are missing Objectives of studies

Final Remarks cont.  Try to minimize missing data when design studies  Nonetheless, some data can be missing.  When data are missing, investigate the reason, dimension, and mechanism to choose the appropriate treatment.  Deletion methods are sometimes OK (e.g., MCAR with the amount of missing 5%).

Final Remarks cont.  Multiple imputation is known to perform well in many cases.  Distributional assumptions along with data types are key component for MI, thus might hard to implement for less experienced researchers.  Recommend to seek professional help when you consider complicated methods such as multiple imputation.

Reference  McKinght, PE, et al.(2007) Missing Data: A Gentle Introduction. The Guilford Press, NY.  Allison, PD (2002). Missing Data. Thousand Oaks, CA:Sage  Little, RL & Rubin, DB. (2002). Statistical analysis with missing data, 2 nd. New York: Wiley.  Rubin, DB (1976). Inference and missing data. Biometrika, 63,  Rubin, DB (1987). Multiple imputation for nonresponse in surveys. New York: Wiley.  Schafer, JL. (1997). Analysis of Incomplete Multivariate Data. London: Chapman & Hall.  White, Royston & Woods(2011). Multiple imputation using chained equations: Issues and guidance for practice, Statistics in Medicine  Van Buuren (2007). Multiple imputations of discrete and continuous data by fully conditional specification, Statistics in Medical Research