Approaches to quantitative data analysis Lara Traeger, PhD Methods in Supportive Oncology Research

Agenda Identify key steps to developing an analysis plan Select specific tests Strengthen study design and reliability 2

1) Identify key sources of support Mentors Collaborators Consultants Prior published research 3

2) Ensure that aims/hypotheses are articulated Clearly use exploratory or confirmatory language Identify hypotheses for each confirmatory aim Delineate primary, secondary, and exploratory aims 4 Brief exercise: review and critique aims in preparation for data analysis

3) Ensure that methods are well developed Identify overall study design Identify recruitment pool (size, feasibility, power) Identify how each outcome will be measured Plan how each measure may be scored or evaluated 5 Brief exercise: select methods for evaluating a patient- reported outcome in preparation for data analysis in preparation for data analysis

Additional notes on sample size, effect size, and statistical power Consider impact of the following on statistical power: – N – Effect size – Standard deviation of the outcome – Alpha level Convention for study planning: power ≥.8 or greater In absence of pilot data, how to conduct power analysis? Estimate study attrition in order to ensure adequate power 6

4) Set up preliminary steps and decisions Analyze sample characteristics Test variable distributions and other assumptions Consider and plan for missing data Set test criteria (p-value, fit indices, etc.) Select appropriate software 7 Brief exercise: explore implications of various methods for addressing missing data

5) Organize a priori and post hoc plans Develop an a priori plan for each specific aim Model building Modeling fitting (vs. over-fitting) Covariates and control variables Pilots versus large-scale trials Address post hoc or exploratory plans Characterize or follow up significant findings Further inform next steps or future studies 8

Selecting specific tests Before selecting a test, ensure the following are clear: What is the study purpose? – Descriptive or inferential What is the type of outcome variable? – Continuous, dichotomous, ordinal, and/or nominal What is the likely distribution of the outcome? – normal, binomial, or skewed What is the proposed purpose of the other model variable(s)? – Correlate, grouping, predictor, or independent variable

Study purpose: descriptive Describe a group Continuous measure Dichotomous or nominal measure (categories) Mean (SD) Median / ranges Relative frequencies What proportion of respondents comprise different subgroups? What is the average and variability of scores? Non-normal distributions: what is the median and interquartile range of scores?

Study purpose: inference – relationship between 2 continuous variables Test correlation Pearson’s Correlation Spearman Rank Correlation Non-normal distribution, or small N

Study purpose: inference – difference between 2 paired/dependent groups Test difference Continuous outcome Dichotomous outcome Paired t test McNemar’s test Wilcoxon signed rank test Ordinal or non- normal distribution, or small N

Study purpose: inference – difference between 2 independent groups Test difference Continuous outcome Dichotomous outcome Student t test Chi-square test Wilcoxon rank sum (Mann-Whitney U) Fisher’s exact test (small N) small N or small cell sizes Ordinal or non- normal distribution, or small N

Study purpose: inference – difference between >2 independent groups Test difference Continuous outcome Dichotomous outcome ANOVA Chi-square test Kruskal Wallis Test Ordinal or non- normal distribution, or small N ANCOVA Control for confounding factors

Study purpose: inference – Multivariate models Test relationships Continuous outcome Dichotomous outcome Multiple linear regression Multiple logistic regression Poisson or negative binomial regression Specific non-normal distributions Nominal Outcome (>2 groups) Multinomial logistic regression Time to event Cox proportional hazards model Outcome with count data

Summary: Developing a data analysis plan 16 Brief exercise: review and critique plan for analyzing a pilot trial versus RCT Brief exercise: identify necessary elements for planning analysis of a large-scale multiple logistic regression model

Advanced statistical modeling – 3 cases: panel data, path analysis, and latent variables Using repeated measures/panel data – Benefits and challenges – Correcting for dependence among observations – Applications in small and large datasets – Applications in intervention trials Generating path models – Benefits and challenges – Model building Applying latent variables – Purpose and utility of latent variables

Considerations: analysis planning will help to inform and strengthen study design Three examples: Power calculation results  Revise target population or modify study design Expected variable distributions  modify method for measuring/evaluating outcomes Expected family-wise error rate  Streamline proposed outcomes or measurement strategy 18

Considerations: analysis planning will help to strengthen reliability of study findings Two examples: Model fitting  – Whether/how to appropriately control for explained or unexplained variance – Whether to maximize explained variance in an outcome vs. focus on variance explained by specific factors Interpretation of test statistics  – Factors that influence p value (N, SE, model type) – Ways to evaluate clinical significance and meaning 19

Questions? 20