1. Statistical Analysis and Tips for Designing a Clinical Study DESIGN, METHODS AND COMMON MISTAKES Eneko Larumbe, PhD Biostatistician October 12, 2016

2. WHAT DO I NEED WHEN I REVIEW YOUR PROTOCOL (I) Did the investigators read enough literature? Do they include manuscripts supporting both sides of a question, and do they suggest why they think one is correct? – Do they have support for the techniques they want to use? – Being an excellent clinician does not mean being a good clinical scientist – Number of articles read (do they have a full understanding of what has been done?) – Read reference bases in addition to PubMed: ERIC, EMBASE, psycInfo, Scopus, ISI, sportdiscus, etc. Is the protocol focused? Is it worth the effort? – Investigators perspective: even the smallest research takes a lot of time – Patients perspective: avoid involving “innocent” people in poorly designed research What are the hypotheses? – What are they measuring? What are they manipulating? – Are all variables clearly identified as independent or dependent? – Are the hypotheses specific and testable? Will this design lead them to reach their goal? – Is the design appropriate for the level of evidence they want to reach?

3. THE SCIENTIFIC DESIGNS Max. intervention Max. internal control Most natural Min. internal control Experimental (random assignment) Quasi- Experimental (no-random assignment) Ex post facto (correlational) Longitudinal (across time without manipulation) Case-control and Cohort (without manipulation: OR, RR, etc.) Observation (with or without presence of experimenter) Case (n=1) Survey (opinion, perception) Qualitative (more speculative) Systematic review (PRISMA, Cochrane)

4. WHAT DO I NEED WHEN I REVIEW YOUR PROTOCOL (II) How valid and reliable are the measurements? – Special emphasis on psychological testing and surveys. Do the authors use validated instruments (especially for constructs like depression, communication, happiness, etc.)? What scales of measure are they getting? – Are the units/scales (mm, cl, kg, years, 1-7, nominal) defined? – Do they plan to transform variables or calculate indices? Did they justify the sample size? Did they perform a power analysis? – I can do this for you if you tell me: How many patients you will actually recruit Expected attrition rate What effect size will you be able to detect? With information provided, can I determine what statistical analysis is appropriate? Is the project viable? – Is it feasible to recruit patients, get data, perform analyses and write a report in a reasonable time? – Is there reasonable expectation that it will be accepted for publication?

5. ELEMENTS NECESSARY TO DETERMINE THE SAMPLE SIZE Alpha, usually 0.05, 0.01, 0.001 Power (1-beta), usually beta=4x(alpha) Design (analysis, allocation…) Targeted effect size – Pilot study – Literature – Desired magnitude (to make sense) – Clinical effect size

6. ELEMENTS NECESSARY TO DETERMINE THE ANALYSIS Number of groups / allocation Time design – Single / pre-post / multiple assessments Purpose of the study – Descriptive / difference / association / prediction Independent/predictor variable(s) – What was manipulated or expected to cause differences or predict the outcome Dependent/criterion variables – Outcomes to be measured or predicted Confounders / covariates