Unit 11: Evaluating Epidemiologic Literature

Unit 11 Learning Objectives: 1. Recognize uniform guidelines used in preparing manuscripts for publication in peer reviewed epidemiologic journals. 2.Understand general guidelines used to evaluate epidemiologic literature.

Overview --- Publication in “respected” epidemiologic journals involves “peer review”, a process in which outside “experts” review the suitability of manuscripts submitted for publication. ---In spite this system of checks and balances, numerous examples abound of published epidemiologic studies with poor designs, inappropriate analyses, and unsubstantiated conclusions.

Overview --- In virtually all epidemiologic journals, as well as journals from other disciplines, a standard format is used for manuscript preparation: ---Abstract (summary of the paper) ---Introduction ---Methods ---Results ---Discussion

Overview Within the health sciences, uniform guidelines for manuscript preparation are provided in the document: “Uniform Requirements for Manuscripts Submitted to Biomedical Journals”

Manuscript Format Abstract (summary of the paper): ---Usually 250 words or less. ---Contains a brief summary of each major section of the paper (introduction, methods, results, conclusions). ---Probably the most important part of the paper, since many persons will only read the abstract.

Introduction: ---Typically limited to a few paragraphs. ---Frames the purpose and public health significance of the research by contrasting the study objective with a brief literature review of current state of knowledge. ---The research hypothesis(es) to be investigated/tested should be clearly stated. Manuscript Format

Methods: ---Describes how the study was carried out. ---Includes a description of the study population, exposure and outcome variables, data collection methods, statistical analyses, etc. ---Should be of sufficient detail so that the reader can critically evaluate the work without having to consult outside sources. Manuscript Format

Results: ---Describes what was found in the study. ---Should correspond directly with the stated research hypothesis(es). ---Tables and figures should be judiciously used; text descriptions should not be largely redundant with data in tables and figures. Manuscript Format

Discussion: ---Describes what was learned from the study and public health implications of the findings. ---Should not be a large re-statement of text from the Results section. ---Should not include presentation of “new” findings not presented in Results section. Manuscript Format

Discussion (cont.): ---Should contrast results with similar previous studies, including possible explanations for differences. ---Should candidly acknowledge study limitations (all studies have some limitations). ---Should state to whom the results most likely apply (generalize). Manuscript Format

General guidelines for evaluating epidemiologic literature

In the subsequent slides, the following abbreviations are used: A:Abstract I:Introduction M:Methods R:Results D:Discussion Evaluation Guidelines

1. Presentation and Purpose of Research Hypothesis Section(s) Is the research hypothesis(es) clearly stated? A, I Does the study address an important public health issue? A, I Is the purpose of the research supported by a relevant review of the literature? I

Evaluation Guidelines 2. Selection of Research DesignSection(s) Does the study use an experimental or observational design? A, M Is the study design appropriate for the research hypothesis(es)? M Is the method in which the study design was carried out clearly articulated? M Does the design represent an advance over prior approaches? M

Evaluation Guidelines 3. Study PopulationSection(s) Is it clear how study participants were identified and selected? A, M Were the inclusion and exclusion criteria clearly defined and appropriate? M Were the methods used in selecting study participants appropriate (e.g. cases, controls)? M

Evaluation Guidelines 4. Exposure Ascertainment/AssignmentSection(s) Is it clear how exposure variables/ exposed and unexposed persons were identified? A, M Is the assessment of exposure likely to be precise and accurate? M Was the method of exposure quantification/classification appropriate? M Was exposure ascertainment uniformly applied for all study participants? M

Evaluation Guidelines 5. Outcome Ascertainment/AssignmentSection(s) Is it clear how the study outcome(s) was defined and classified? A, M Is outcome assignment likely to be precise and accurate? M Was the method of outcome classification appropriate? M Was outcome ascertainment uniformly applied for all study participants? M

Evaluation Guidelines 6. Statistical MethodsSection(s) Are the statistical methods used clearly described and appropriate? A, M Is the sample size adequate to answer the research question (e.g. assessed through description of power calculations)? M Have the assumptions underlying the statistical tests in use been met? M Were appropriate methods used to control for possible confounding? M

Evaluation Guidelines 7. Presentation of ResultsSection(s) Do the results presented correspond directly to the research hypothesis(es)? A, R Are the results presented by appropriate use of text, tables, and figures? R Is it clear which potential confounding variables were controlled for in the analysis? R

Evaluation Guidelines 8. Interpretation of Results/ConclusionsSection(s) Overall, are the author’s conclusions justified by the data presented? A, D Do the authors appropriately interpret the clinical, biologic, and statistical significance of the results? D Are the study findings compared and contrasted with similar prior research? D Were measurement errors with regard to exposure/disease classification discussed? D

Evaluation Guidelines 9. Interpretation of Results/ConclusionsSection(s) Has chance been discussed as a potential explanation of the study results? D For non-significant results, do the authors discuss if they had sufficient power? D Is the impact of the major possible sources of bias (e.g. selection bias) discussed? D Do the authors consider/discuss whether confounders could account for the observed study results? D

Evaluation Guidelines 10. Interpretation of Results/ConclusionsSection(s) Do the authors appropriately acknowledge limitations of their study? A, D Are the generalizability of the study findings discussed and appropriate? D Do the authors provide suggestions for future areas of investigation? D

Class Exercise From the article “Snoring as a Risk Factor for Type Diabetes Mellitus: A Prospective Study (AJE 2002; 155: ): Questions from the Abstract: 1)What was the primary exposure variable of interest? 2)What was the primary outcome variable of interest? 3)What type of epidemiologic study design was used? 4)What were the primary study inclusion criteria? 5)How was the primary exposure ascertained? 6)What were the general results overall for the association between the exposure and outcome of interest? 7)Was there any suggestion of effect modification?

Class Exercise From the article “Snoring as a Risk Factor for Type Diabetes Mellitus: A Prospective Study (AJE 2002; 155: ): Questions from Table 2: 8)What was the reference (control) group used in the analysis? 9)The initial results were adjusted for age. Was the evidence that body mass index (BMI) was a confounder? 10) Provide an interpretation for the final results presented in the multivariate adjusted model.