1. What are systematic reviews and why do we need them?
Gavin Stewart Centre for Evidence-Based Conservation University of Birmingham, UK
What are systematic reviews and why do we need them?

2. What is the problem? Good policy needs an evidence-base
Quantity of information on a subject is large
Accessibility of information is variable (some may be hard to find)
Information quality is variable and so is the outcome

3. Quantity
million articles were being published each year in 20,000 journals (medical example)
a pile of paper 500 metres high
if researchers tried to stay current by reading two articles per day, in one year they would fall 55 centuries behind!
or, if you try to read everything of possible relevance, you would have to read 5,500 articles per day.
Similar story for ecology? Have you read the latest issues of Nature, Science, Biological Conservation, Journal of Applied Ecology, Conservation Biology, TREE, Journal of Environmental Management etc ?

4. Accessibility & Variability
You manage an upland NNR
There are 58 references in the management handbook (and a further 77 you don’t know about?)
Information from other side of the hill
quality is variable
And the outcomes are different
What do you do?

5. What Is Systematic Review?
Systematic review is a tool that provides empirical answers to scientific research questions using existing available evidence
Key features
Systematically locate data
Critically appraise methodology
synthesise evidence

6. They are not conventional Reviews
Follow a strict methodological and statistical protocol
more comprehensive
minimising the chance of bias
improves transparency, repeatability and reliability

7. Differences Between Traditional and Systematic Reviews
(Adapted from Cook, D. J. et. al. (1997). Ann. Intern. Med. 126: )
Feature
Traditional Review
Systematic Review
Question
Often broad in scope
Focused question
Sources & search
Not usually specified,
potentially biased
Comprehensive sources &
explicit search strategy
Selection
Rarely specified,
Criterion-based selection,
uniformly applied
Appraisal
Variable
Rigorous critical appraisal, uniformly applied
Synthesis
Often a qualitative summary
Quantitative summary* when appropriate
Inferences
Sometimes evidence-based
Evidence-based
*A quantitative summary that includes a statistical synthesis is a meta-analysis

8. Stages of a Review Identification of need for review
Formulating a question
Generating a search strategy
Study relevance
Study quality
Data extraction
Synthesis of data
Recommendations

9. Identify and expand concepts Set the scope of the question
Identification of need for review and formulating a question and protocol
Define the hypothesis
Identify and expand concepts
Set the scope of the question

10. Generating a search strategy
Multiple electronic databases and the internet using a range of Boolean search-terms
Foreign language searches
Include grey literature to avoid publication bias (see subsequent slides)
Search bibliographies and contact experts

11. Appraising study relevance
Use the question elements
Subject e.g. arctic-alpine flora
Intervention e.g. grazing (deer)
Outcome e.g. Change in frequency at a site scale (5% decline is deleterious)
If information is presented about arctic-alpines in relation to grazing and frequency is measured, the article is relevant irrespective of what it says!

12. Appraising study quality
There is no such thing as a perfect study, all studies have weaknesses, limitations, biases
Interpretation of the findings of a study depends on design, conduct and analysis, as well as on the population, interventions, and outcome measures
The researchers in a primary study did not necessarily set out to answer your review question

13. What do we do with quality assessment results?
Determine minimum quality threshold for inclusion
Explore differences in quality as an explanation for heterogeneity in study results
To weight individual study results in relation to their validity or the amount of information they contain
Guide interpretation and overall recommendations

14. Data extraction Undertaken with synthesis in mind
Standardised methodology with a priori rationale
Pilot data extraction and ensure repeatability using two reviewers (good practice for data hygiene in any case)

15. Synthesising evidence
Qualitative synthesis
Meta-analysis to synthesise results across studies
Bayesian synthesis of disparate data types particularly using experience as a prior

16. Review & Dissemination Unit
Needs-led research
Review & Dissemination Unit
Decision-makers
Funding Bodies
Research Community
Needs-Led Research