Presentation on theme: "Oct-2007 1 Ferran Torres, MD PhD Unidad de Soporte en Estadística y Metodología (USEM) Servei de Farmacologia Clínica (UASP). Hospital."— Presentation transcript:
Oct-2007 Ferran.Torres@uab.es 1 Ferran Torres, MD PhD Unidad de Soporte en Estadística y Metodología (USEM) Servei de Farmacologia Clínica (UASP). Hospital Clínic Profesor Bioestadística. Facultat Medicina. UAB Herramientas básicas para un buen diseño Epidemiológico
Oct-2007 Ferran.Torres@uab.es 2 2 extreme views about observational studies Observational studies aren’t useful. RCTs are the gold standard and the only valid design for “truth” Observational study evidence trumps RCT evidence. RCTs are not applicable to real-world practice
Oct-2007 Ferran.Torres@uab.es 3 Epidemiological studies. Terminology “Observational” studies: Lack investigator allocation to an intervention Studies not trials Include: case series, cross-sectional, case- control & cohort studies, before and after, time-series, database studies, historical controls “Non-randomized studies” is broader term
Oct-2007 Ferran.Torres@uab.es 4 NRS vs RCT evidence –Well-designed cohort or case-control & RCTs have similar effect sizes (24 clinical topics). Concato et al. Benson et al NEJM 2000;342:1887-92. –“Strong evidence that quasi-randomized trials provided biased effect size estimates of about 30%”--at least for medical Rx. Cochrane NRSG approach (http://www.cochrane.dk/nrsmg) http://www.cochrane.dk/nrsmg –Results of RCTs and NRS sometimes but not always differ. Deeks. HTA report. 2003;Vol 7:no. 27.
Oct-2007 Ferran.Torres@uab.es 5 Extending findings from available RCTs Population limitations (homogeneous, limited co-morbidities, unstudied vulnerable groups) Small sample sizes (adverse events, rare events) Short follow-up (maintenance of benefits, adverse events) Important outcomes not available (patient priorities, long-term effects, natural history/background rate)
Oct-2007 Ferran.Torres@uab.es 7 Esperanza de vida al nacer años 47,3 78,1
Oct-2007 Ferran.Torres@uab.es 8 Key points in study design Economical- Budget Logistic-organization Ethical Scientific
Oct-2007 Ferran.Torres@uab.es 9 Errors in research There are basically 2 types of error in research. 1. One is random error due to random variation in subjects’ responses or measurement. Inferential statistics (the p value and 95% confidence interval) measure the amount of random error and thus allow us to draw conclusion based on our research data. 2. However, there is another type of error, Bias or systematic error.
Oct-2007 Ferran.Torres@uab.es Types Of Error: Random Error –Larger sample produce less variable estimate and more likely to reflect the experience of the total population p < 0.05 : 5 %, or 1 in 20, probability of observing a result as extreme as that observed solely by chance BUT, a composite measure affected by both the magnitude of the difference between groups and the sample size
Oct-2007 Ferran.Torres@uab.es 11 Bias - Definition Deviations of results (or inferences) from the truth, or processes leading to such deviation. Any trend in the selection of subjects, data collection, analysis, interpretation, publication or review of data that can lead to conclusions that are systematically different from the truth. (Last Dictionary of epidemiology) Systematic deviation from the truth that distorts the results of research. (Sitthi Lancet 1993)
Oct-2007 Ferran.Torres@uab.es 12 Válidas y precisasVálidas e imprecisas No Válidas y precisasNo Válidas e imprecisas Sesgo Precisas pero con SESGO Imprecisas y con SESGO
Oct-2007 Ferran.Torres@uab.es 14 Selection bias (in entire study group) Error due to systematic differences in characteristics between those who are selected for study and those who are not. (Last Dictionary of Epidemiology) The selected sample is not representative of the universe of which it is a part. (Hill Principles of Medical Statistics 1971) The control or population experience may not be representative of the counterfactual of the cases
Oct-2007 Ferran.Torres@uab.es 15 Types of Selection Bias Berksonian bias – There may be a spurious association between diseases or between a characteristic and a disease because of the different probabilities of admission to a hospital for those with the disease, without the disease and with the characteristic of interest Berkson J. Limitations of the application of fourfold table analysis to hospital data. Biometrics 1946;2:47-53 Response Bias – those who agree to be in a study may be in some way different from those who refuse to participate –Volunteers may be different from those who are enlisted
Oct-2007 Ferran.Torres@uab.es 16 Confounding bias Confounding bias 1. Associated with the exposure being studied –imbalance in the comparison groups 2. Independently associated with the disease 3. Not an effect of the exposure Confounding bias arises when the confounder is unequally distributed between the group with the study risk factor and the control group without the study factor.
Oct-2007 Ferran.Torres@uab.es 17 Types of Information Bias Interviewer Bias – an interviewer’s knowledge may influence the structure of questions and the manner of presentation, which may influence responses Recall Bias – those with a particular outcome or exposure may remember events more clearly or amplify their recollections Observer Bias – observers may have preconceived expectations of what they should find in an examination Loss to follow-up – those that are lost to follow-up or who withdraw from the study may be different from those who are followed for the entire study
Oct-2007 Ferran.Torres@uab.es 18 Types of Information Bias Hawthorne effect – an effect first documented at a Hawthorne manufacturing plant; people act differently if they know they are being watched Surveillance bias – the group with the known exposure or outcome may be followed more closely or longer than the comparison group Misclassification bias – errors are made in classifying either disease or exposure status
Oct-2007 Ferran.Torres@uab.es 22 The 4 types of bias Whole group Comparison between group Subjects Selection bias Confounding bias Measurement Measurement bias Information bias
Oct-2007 Ferran.Torres@uab.es 23 Avoiding bias: Design Avoiding bias: Design Correcting bias: Analysis Correcting bias: Analysis Estimating magnitude and direction of bias: Sensitivity analysis of Bias Estimating magnitude and direction of bias: Sensitivity analysis of Bias Dealing with bias
Oct-2007 Ferran.Torres@uab.es 24 Study design –Sampling –Sample Size –Sources of data collection –Methods of data collection –Content of information –Statistical Analysis Plan Prevention of Bias (?)
Oct-2007 Ferran.Torres@uab.es 25 Avoiding bias: Design Standard source of information –More than one source: Multiple standard sources to confirm information Methods to assure participation and compliance and follow-up Strategy to maximise participation rate (response, consent), and to maximise complete follow up Defining study population: –population based study less vulnerable –Define, a priori, who is a case or what constitutes exposure so that there is no overlap Define categories within groups clearly (age groups, aggregates of person years) “ Prevention is better than cure”
Oct-2007 Ferran.Torres@uab.es 26 Avoiding bias: Design Well defined population In Cohort studies, the population should be chosen independent of the risk of disease in question.In Cohort studies, the population should be chosen independent of the risk of disease in question. In Case control studies, the selection of the controls should be independent of the exposure in questionIn Case control studies, the selection of the controls should be independent of the exposure in question Set up strict guidelines for data collection Train observers or interviewers to obtain data in the same fashion It is preferable to use more than one observer or interviewer, but not so many that they cannot be trained in an identical manner Use of a good control group
Oct-2007 Ferran.Torres@uab.es 27 Be purposeful in the study design to minimize the chance for bias, ex. more than one control group Selection of control in case control study : to equalise incentive or motivation to recall, use a third control arm that has similar disease but not disease under study. Example, congenital abnormality study, case mothers, normal control mothers, a third group of other abnormality Sampling : probability sampling required to ensure representative sample external validity Experimental design for RCT : parallel groups design best. Others for example self-controlled design, historical control etc prone to biases
Oct-2007 Ferran.Torres@uab.es 28 Avoiding bias - Design Randomisation: –random allocation to comparison groups to avoid selection bias by investigators as well as to minimise confounding bias. Randomly allocate observers/interviewer data collection assignments Matching on important confounders Blinding of subjects, investigators and / or statistician
Oct-2007 Ferran.Torres@uab.es 29 Restriction of subjects to obtain homogenous group. Quality control procedures in data collection …all detailed in advance in a written protocol Avoiding bias - Design
Oct-2007 Ferran.Torres@uab.es 31 Med Clin (Barc) Dic-2005. Vol 125, Supl.1 Estudios epidemiológicos (STROBE). Metaanálisis (QUOROM, MOOSE). Estudios de intervención no aleatorizados (TREND). Estudios de precisión diagnóstica (STARD) y pronóstica (REMARK). Otros: –Recomendaciones metodológicas de las agencias reguladoras. –Instrumentos de medida de calidad de vida relacionada con la salud y de otros resultados percibidos por los pacientes. –Estudios de evaluación económica en salud. –Ensayos clínicos aleatorizados (CONSORT). –Ensayos clínicos aleatorizados comunitarios (CONSORT CLUSTER).
Oct-2007 Ferran.Torres@uab.es 32 ICHE9 Statistical Principles for Clinical Trials ICHE9 CPMP/EWP/908/99 CPMP Points to Consider on Multiplicity issues in Clinical Trials (Apr 2003) CPMP/EWP/908/99 CPMP/EWP/2863/99 Points to Consider on Adjustment for Baseline Covariates (Nov 2003) CPMP/EWP/2863/99 CPMP/2330/99 Points to Consider on Application with 1.) Meta-analyses and 2.) One Pivotal study (May 2001) CPMP/2330/99 CPMP/EWP/2158/99 Guideline on the Choice of a Non-Inferiority Margin (Jan2006) CPMP/EWP/2158/99 CPMP/EWP/482/99 Points to Consider on Switching between Superiority and Non-inferiority (Feb 2001) CPMP/EWP/482/99 CPMP/EWP/1776/99 Points to Consider on Missing Data (Jan 2002) CPMP/EWP/1776/99 CHMP/EWP/83561/05 Guideline on Clinical Trials in Small Populations (Feb2007) CHMP/EWP/83561/05 CHMP/EWP/2459/02 Reflection Paper on Methodological Issues in Confirmatory Clinical Trials with Flexible Design and Analysis Plan (Draft) CHMP/EWP/2459/02 Normativas
Oct-2007 Ferran.Torres@uab.es 33 Population of Oldenburg, Germany, 1930-1936 (Ornithologische Monatsberichte 44, Jahrgang, 1936, Berlin) Storks (1000s) Humans (1000s)
Oct-2007 Ferran.Torres@uab.es 34 http://ferran.torres.name/docencia/hcp Do Storks Bring Babies?