Guidelines for Critically Reading the Medical Literature John L. Clayton, MPH

Purpose of the Study What are the objectives in performing the study? What is (are) the research questions being asked? Why was this study conducted?

Study Design What type of study was performed? –Experimental vs. observational –Prospective vs. retrospective –Cross-sectional vs. longitudinal –Ecological (e.g., vital statistics, mortality trends) If you know the study design, you know which types of analyses are appropriate, and can be looking for the appropriate test statistics

Study Design How was the population or sample selected? Are there possible sources of bias in selecting the sample? –Are there factors that might make the sample unrepresentative?

Study Design Was there a control group? –How was it selected? Was randomization or matching used? –If so, on what characteristics were subjects matched?

Study Design What were the inclusion criteria used to choose the study sample? –Was the definition of a “case” appropriate? After identification of the sample, were there exclusions? –If so, who was excluded, when were they excluded, and why were they excluded?

Measurement and Observation Are there clear definitions of terms used, including diagnostic categories, drugs or compounds used, and interventions performed? Were the outcome criteria defined prior to examination of the data?

Measurement and Observation What was measured during the study? How were the measurements made? Were the classification criteria and measurement processes capable of meeting the study objectives?

Measurement and Observation Were the categories and processes applied consistently for all subjects? Are the observations and measurements reliable (reproducible)? What if anything, was done to assess their reliability?

Presentation of Data How are the data presented? –Tables, charts, graphs, etc. Which measures of association were used? –Relative risks, odds ratios, correlation coefficients, etc.

Presentation of Data Are results presented clearly and in enough detail so that the reader is able to judge them for him/herself? Are the data which are presented relevant to the purpose of the study? Are there data which are not presented which should have been?

Presentation of Data Are the data consistent with each other? –Are the values in one table consistent with those in another? –Do the numbers add up appropriately? –Is everybody accounted for?

Analysis of the Data Are the data worthy of analysis? What methods of statistical analysis, if any should have been performed? Were the assumptions for various statistical tests met?

Analysis of the Data Was the analysis strategy appropriate for the study design? Were potential confounding variables taken into account? Was the chosen analysis strategy able to answer the original hypotheses?

Conclusions What conclusions did the author(s) reach? Are they justified? Are they generalizable? If so, to what population? What conclusions do you reach based on the available evidence?

Conclusions Would you like more information about the study in order to reach a conclusion? If so, what? Are the conclusions reached by the authors relevant to the purpose of the study? Do the authors make any recommendations based on the results of the study? If so, are they justified? What recommendations would you make?

General Discussion Was the study ethical? If (are) the research question(s) being addressed worthy of being studied? What additional research should be done in order to answer the questions addressed by this study?

General Discussion What type of study design should be used? What type of analysis strategy should be used? Why was this paper selected for discussion?

References Colton, T., “Critical Reading of the Medical Literature,” in Statistics in Medicine, Chapter 13, Boston: Little, Brown and Company, Murphy, E.A., “An exercise in Qualitative Criticism,” In The Logic of Medicine, Chapter 17, Baltimore: The Johns Hopkins University Press, R.K. Riegelman, Studying a Study and Testing a Test: How to Read the Medical Literature, Boston: Little, Brown and Company, 1981.