THE NEWCASTLE CRITICAL APPRAISAL WORKSHEET A format for examining journal articles* *(Based on Medical Journal of Australia 1992;157:389-94) Presented by Dick Heller, Professor of Public Health, The University of Manchester, UK <Dick.Heller@man.ac.uk> This Worksheet was developed at the University of Newcastle, NSW, Australia to help read journal articles. Darzins PJ, Smith BJ, Heller RF. How to Read a Journal Article. Med J Aust 1992; 157:389-94. More detailed approaches to this are found in the Users Guides to the Medical Literature from the Evidence Based Medicine Group, and are published in JAMA

11 items, each with 3 sections Can you find this information in the paper? Is the way this was done a problem? Does this problem threaten the validity of the study? You can quickly work through any journal article using this format.

11 items 1. What is the research question? 2. What is the study type? 3. What are the outcome factors and how are they measured? 4. What are the study factors and how are they measured? 5. What important confounders are considered? 6. What are the sampling frame and sampling method? 7. In an experimental study, how were the subjects assigned to groups? In a longitudinal study, how many reached final follow-up? In a case control study, are the controls appropriate? (Etc) 8. Are statistical tests considered? 9. Are the results clinically/socially significant? 10. Is the study ethical? 11. What conclusions did the authors reach about the study question? These are the main items on the worksheet, and can be applied to most journal articles

1.What is the research question? (Is the way this was done a problem?) Is it concerned with the impact of an intervention, causality or determining the magnitude of a health problem? (Does this problem threaten the validity of the study?) Is it a well stated research question/hypothesis?

2. What is the study type? (Is the way this was done a problem?) Is the study type appropriate to the research question? (Does this problem threaten the validity of the study?) If not, how useful are the results produced by this type of study?

3. What are the outcome factors and how are they measured? (Is the way this was done a problem?) a) are all relevant outcomes assessed b) is there measurement error? (Does this problem threaten the validity of the study?) a) how important are omitted outcomes b) is measurement error an important source of bias?

4. What are the study factors and how are the measured? (Is the way this was done a problem?) Is there measurement error? (Does this problem threaten the validity of the study?) Is measurement error an important source of bias?

5. What important potential confounders are considered? (Is the way this was done a problem?) Are potential confounders examined and controlled for? (Does this problem threaten the validity of the study?) Is confounding an important source of bias?

6. What are the sampling frame and sampling method? (Is the way this was done a problem?) Is there selection bias? (Does this problem threaten the validity of the study?) Does this threaten the external validity of the study? We have now considered the major sources of bias – selection, measurement and confounding.

7. Questions of internal validity (Is the way this was done a problem?) In an experimental study, how were the subjects assigned to groups? In a longitudinal study, how many reached follow-up? In a case control study, are the controls appropriate? Note: other issues of relevance to internal validity are considered under the other headings in this critical appraisal system. You can add your own questions, and also design your own questions for other study types such as cross sectional studies and systematic reviews (Does this problem threaten the validity of the study?) Does this threaten the internal validity of the study? It is difficult to include all possible sources of bias that might threaten the internal validity of a study, and you are welcome to add your own favourites. The ones included in this side are those that might be considered as citiical for these types of study

8. Are statistical tests considered? (Is the way this was done a problem?) Were the tests appropriate for the data? Are confidence intervals given? Is the power given if a null result? In a trial, are results presented as absolute risk reduction as well as relative risk reduction? (Does this problem threaten the validity of the study?) If not, how useful are the results?

9. Are the results clinically/socially significant? (Is the way this was done a problem?) Was the sample size adequate to detect a clinically/socially significant result? Are the results presented in a way to help in health policy decisions? (Does this problem threaten the validity of the study?) Is the study useful? The difference between statistical significance and a result that is of use is an important distinction to make

10. Are ethical issues considered? (Is the way this was done a problem?) Does the paper indicate ethics approval? Can you identify potential ethical issues? (Does this problem threaten the validity of the study?) Are the results or their application compromised? There are a number of ethical issues to consider in both the conduct of research and the application of the results

11. What conclusions did the authors reach about the study question? (Is the way this was done a problem?) Do the results apply to the population in which you are interested? (Does this problem threaten the validity of the study?) Will you use the results of the study? The final issue is whether the study has passed your critical appraisal in terms of your decision to use the results in your own practice, teaching or further research