1. Critical Appraisal
Dr Samira Alsenany
Dr SA 2012
Dr Samira alsenany

2. Purpose of this seminar
The purpose of this seminar is to provide a brief overview of the critical appraisal process.
A critical review must identify the strengths and weaknesses in a piece of research and this should be carried out in a systematic manner (Eachus, 2003)
Dr SA 2012

3. Critical Appraisal
“Critical appraisal is the process of systematically examining research evidence to assess its validity, results, and relevance before using it to inform a decision” (Hill and Spittlehouse, 2001, p.1).
Dr SA 2012

4. Critical Appraisal
Critical appraisal is an essential step in the process of putting research into practice. Asking questions about an article’s research methodology, scrutinizing its data collection and analysis methods, and evaluating how its findings are presented will help you to determine whether that article’s conclusions should influence practical decision-making.
Dr SA 2012

5. When should you Critically Appraise?
• Conducting literature reviews for grant proposals. • Evaluating the effectiveness, costs, and benefits of health programs. • Establishing new health programs. • Implementing policies; and Public health decision making, especially at the senior management level.
Dr SA 2012

6. What is critical appraisal?
Critical appraisal is an essential part of evidence-based.
Critical appraisal allows us to make sense of research evidence and thus begins to close the gap between research and practice.
The most appropriate design for studying the effectiveness of a specific intervention or treatment.
Dr SA 2012

7. Appraising systematic reviews
Once you have conducted a literature search and obtained full text articles, you can begin the critical appraisal process.
Consider the following questions to assess the quality of the article
Dr SA 2012

8. Questions to Ask What kind of reports do I want?
Is this article relevant to my issue and setting?
How much detail do I need?
How comprehensive do I need to be?
How far back should I search?
Dr SA 2012

9. Questions to Ask Is it of interest? Why was it done? How was it done?
What has been found?
What are the implications?
What else is of interest?
Dr SA 2012

10. Critically Appraise What You Read.
Simple checklists enable the useful information to be identified.
Dr SA 2012

11. Checklists
Checklists for particular types of literature are a quick and easy way of learning critical appraisal.
They all have 3 stages:
Basic questions.
Essential appraisal.
Detailed appraisal
Dr SA 2012

12. Critical Appraisal: Notes and Checklists
Available at: Scottish Guidelines Network (SIGN) has developed a number of critical appraisal tools in support of their creation of evidence-based guidelines. These appraisal tools used by SIGN to conduct systematic reviews have been made available for public use on their website, including methodology evaluation checklists for each of the following 6 types of studies: systematic reviews/meta analyses, RCTs, cohort studies, case-control studies, diagnostic studies, and economic evaluations.
Dr SA 2012

13. Questions to Ask The abstract
Use the information found in the abstract to answer the questions below:
• Are your issues discussed there?
• What are the main findings of the research?
• Do you want to know more after reading the abstract?
• Was the research done in a similar setting to yours?
• Does it address a related question? (Even research that covers your issue indirectly can be useful.)
• Are there reasons to doubt the findings without reading the whole article?
Dr SA 2012

14. Questions to Ask
You may conclude that the study is not reliable merely by reading the abstract.
If this is the case, move on to another article! Critical
Dr SA 2012

15. Questions to Ask Or they didn’t find anything but wanted to publish!
Why was it done?
Introduction
Should a clear statement of the purpose of the study.
Focus on the time, place and circumstances of the population and setting studied.
How similar or different is the study population or setting to yours? Is a difference likely to matter for the issue at hand?
The absence of such statements can imply that the authors had no clear idea of what they were trying to find out.
Or they didn’t find anything but wanted to publish!
Dr SA 2012

16. Questions to Ask How was it done?
Methods
How was it done?
Brief but should include enough detail to enable one to judge quality.
Must include who was studied and how they were recruited.
Basic demographics must be there.
How the data was collected (interviews, focus groups, questionnaires, surveys, observations, etc.).
Dr SA 2012
Dr Samira alsenany

17. Questions to Ask Results
Three broad issues need to be considered the results of the study :
A Are the results of the study valid?
B What are the results?
C Will the results help locally
The major challenge.
What do I think this really means?
Dr SA 2012
Dr Samira alsenany

18. Questions to Ask Results
What has it found?
The data should be there statistics.
Is the sample large enough to produce significant results? Is the effect size clinically or operationally relevant?
Are the aims in the introduction addressed in the results?
Look for illogical sequences, bland statements of results.
Flaws and inconsistencies.
All research has some flaws , the impact of the flaws need to assessed.
Dr SA 2012
Dr Samira alsenany

19. Questions to Ask
How good are the measures? • Do the measures accurately reflect what the researcher was trying to measure (validity)? • How clear and appropriate are these measures? Too broad? Too narrow? • Are they actual measures or proxy measures? • Are the measures well established in either prior research or through pilot testing?
Dr SA 2012

20. Questions to Ask Discussion
The discussion section is more detailed and precise than the abstract, and will explain:
A review of how these results compare or contrast with prior research? If this report found something different from.
The limitations of the research and possible implications which are not mentioned in
the abstract.
Dr SA 2012

21. Questions to Ask implications
What are the implications?
Abstract / discussion.
The whole use of research is how far the results can be generalised.
All authors will tend to think their work is more important than the rest of us!
What is new here?
What does it mean for health care?
Is it relevant to my speciality?
Dr SA 2012

22. Recommended Reading
The following 3 reading series were published in core medical journals and contain in-depth
information on critical appraisal and evidence-based practice.
From the British Medical Journal (BMJ):
How to Read a Paper: The Basics of Evidence-Based Medicine
Available at:
Evidence-Based Nursing Notebook
Available at:
From the Journal of the American Medical Association (JAMA):
User Guides to Medical Literature
Available at:
Dr SA 2012

23. Online Critical Appraisal Tools
The Centre for Methods and Tools ( is currently developing a critical appraisal framework that is specifically tailored to public health research. This guide will be the definitive critical appraisal resource for public health practitioners in Canada. • AGREE Instrument Available at: The purpose of the Appraisal of Guidelines Research & Evaluation (AGREE) Instrument is to provide a framework for assessing the quality of clinical practice guidelines. It originates from an international collaboration of researchers and policy makers who work together to improve the quality and effectiveness of clinical practice guidelines by establishing a shared framework for their development, reporting and assessment. • A Beginner’s Guide to Judging Research Studies Available at This is a succinct guide to critical appraisal written by John Frank, Scientific Director at the Canadian Institutes of Health Research. Although this article focuses on making sense of media reports of new health research, the astute questions Frank poses can be applied directly to research studies themselves.
Dr SA 2012

24. Online Critical Appraisal Tools
• CASP Critical Appraisal Tools
Available at:
The UK-based Critical Appraisal Skills Programme (CASP), a division of the NHS’s Public Health Resources Unit, has developed a customized critical appraisal checklist for each common type of research study. Direct links to each of the 6 appraisal tools are
listed below:
• Qualitative Research
• Review Articles (including Systematic Reviews)
• Case Control Studies
Dr SA 2012

25. Thanks
Dr SA 2012