Research Study Design and Analysis for Cardiologists Nathan D. Wong, PhD, FACC

Advantages and disadvantages of different research study designs - which is best for you? Calculating sample size and power Which statistical tests to use Fallacies in presenting results Steps for protocol development Recommendations for further assistance

Strength of Studies to Prove Causation Weakest: Observational, cross-sectional Weak: Observational, case-control Modest: Observational, prospective Strongest: Randomized clinical trial Within each of these studies, features that further strengthen or weaken the case include sample size, selection of comparison group (control or placebo), selection of study population, length of time of follow-up, and control for potential confounders

Observational, cross-sectional Examines association between two factors (e.g, an exposure and a disease state) assessed at a single point in time, or when temporal relation is unknown Example: lipids, blood pressure, and C- reactive protein levels Conclusions: Associations found may suggest hypotheses to be further tested, but are far from conclusive in proving causation

Observational, case-control Useful for uncommon or rare outcomes that could take years (or longer) to obtain sufficient cases in a prospective follow-up or population sample Often used for etiological studies of cancer Selection of control group (e.g., hospital vs. healthy community controls) and consideration of possible confounders crucial Cannot always be certain about temporal relation between exposure and disease outcome since historical information on exposure history is obtained

Prospective cohort studies Examples: Framingham Heart Study, Cardiovascular Health Study (CHS), Multiethnic Study of Atherosclerosis (MESA), Nurses Health Study Advantages: large sample size, ability to follow persons from healthy to diseased states, temporal relation between risk factor measures and development of disease Disadvantages: expensive due to large sample size often needed to accrue enough events, many years to development of disease, possible attrition, causal inference not definitive

Randomized Clinical Trial Considered the gold standard in proving causation by “reducing” in risk factor of interest--e.g., cholesterol inconclusive as risk factor until early trial showed that lowering it lowered CHD risk Expensive, labor intensive, attrition from loss to follow-up or poor compliance can jeopardize results, esp. if more than outcome difference between groups Conditions are highly controlled and may not reflect clinical practice or the real world Randomization “equalizes” known and unknown confounders/covariates so that results can be attributed to treatment with reasonable confidence

Guidelines for Sample Size / Power Determination Necessary for any research grant application Need to estimate what “control group” rate of disease or outcome is Need to state what is minimum difference (effect size) you want to detect that is clinically significant--e.g., difference in rates, or risk ratio Either power can be estimated for a fixed sample size at fixed alpha (usually 0.05 two-tailed) for different effect, OR sample size can be estimated for a given power (usually 0.80) for different effect sizes

Statistics and Statistical Procedures for Different Study Designs Cross-sectional: Pearson correlation, Chi-square test of proportions- prevalence odds ratio for likelihood of factor Y in those with vs. w/o X Case-control: Odds ratio for likelihood of exposure in diseased vs. non-diseased-- Chi-square test of proportions / logistic regression Prospective: Relative risk (RR) for incidence of disease in those with vs. without risk factor of interest, adjusted for covariates and considering follow-up time to event--Cox PH regression. Correlations and linear/ transformed regression used for continuous outcomes.

Statistics and Statistical Procedures (continued) Randomized clinical trial: Relative risk (RR) of event occurring in intervention vs. control group - Cox PH regression – For continuously measured outcomes, such as pre-post changes in risk factors (lipids, blood pressure, etc.) initial treatment vs. control differences examined by Student’s T-test, repeated measures ANOVA / ANCOVA used for multiple measures across a treatment period and covariates

Fallacies in Presenting Results: Statistically vs. Clinically Significant? Having a large sample size can virtually assure statistically significant results--but at a very low correlation or relative risk Conversely, an insufficient sample size can hide (not significant) clinically important differences Statistical significance directly related to sample size and magnitude of difference, and indirectly related to variance in measure

Steps to Protocol Development Aims and Hypotheses Background Methods, including subject recruitment, eligibility criteria, screening procedures, treatment phase or follow-up procedures Study power and sample size justification Statistical methods of analysis Potential study limiations

Data Collection / Management Always have a clear plan on how to collect data-- design and pilot questionnaires, case report forms. The medical record should only serve as source documentation to back up what you have coded on your forms Use acceptable error checking data entry screens or spreadsheet software (e.g., EXCEL) that is covertable into a statistical package (SAS highly recommended and avail via UCI site license) Carefully design the structure of your database (e.g, one subject/ record, study variables in columns) so convertible into an analyzable format

Where to Go for Help Epidemiology and statistics books Institutional Review Board - considers mainly subject projection issues Dean’s Scientific Review Committee - considers appropriateness of research design, procedures, statistical considerations Questions?