1. Literature review: How to search, evaluate, synthesize, and present the evidence

2. This workshop will focus on…
Searching through the literature for relevant sources;
Evaluating the findings from your search;
Synthesizing these findings; and
Presenting the results.

3. But WHY do we need to conduct a literature review?

4. A literature review can help us to…
Address an information need
Find out whether the evaluation question has been asked before
Find out what has been done in similar settings
Determine what best practices already exist
Discover what tools might be useful in our evaluation

5. Evaluate your practice
What is the problem we need to solve?
The Evidence
to Practice Process
What is the question we need answered?
Select first choice resource where an answer may be found (e.g. the Cochrane library)
Select second choice resource
Design search strategy and carry out search
Appraise and summarize the evidence
Good Evidence
Poor or insufficient evidence
Assess the relevance of the evidence to your local situation and target groups
Note: The Evidence to Practice Process is included in the Supplementary Materials handout (see page 14).
Consider implications for your practice and resources, and those of the organizations you work with
Improvement
Apply the evidence to practice
Evaluate your practice
No improvement
© 2003, Evidence Network: What Works for Children?

6. Where do we start?

7. Searching through the Literature

8. Define the Problem The target population
E.g., Age, sex, ethnic group, diagnostic group
The intervention
The therapy that is of interest to your program
The outcome
The changes you would like to see in your target population

9. Find the Information Academic libraries Online databases
Primary source: The original publication of new data, results, and theories
Stice, E., Shaw, H., Burton, E., & Wade, E. (2006, April). Dissonance and healthy weight eating disorder prevention programs: a randomized efficacy trial. Journal of Consulting and Clinical Psychology, 74(2), Retrieved September 18, 2008, from PubMed database.
Secondary source: Summarizes or comments on primary sources in the context of the particular idea under study
Pratt, B.M. & Woolfenden, S.R. (2002, April). Interventions for preventing eating disorders in children and adolescents. Cochrane Database of Systematic Reviews, Issue 2. Retrieved September 18, 2008, from the Cochrane database.

10. Online databases with free content…
Centre for Reviews and Dissemination (CRD)
Cochrane Database of Systematic Reviews (CDSR)
Turning Research Into Practice (TRIP) Database
The International Network of Agencies for Health Technology Assessment
British Medical Journal
Directory of Open Access Journals (DOAJ)
Substance Abuse and Mental Health Services Administration (SAMHSA)
Centre of Excellence evidence database
Note: Databases are described in the Supplementary Materials handout (see pages 3 – 5).

11. Online databases with limited free content…
PubMed
York University Health Research Guide
British Medical Association (BMA)
Note: Databases are described in the Supplementary Materials handout (see pages 3 – 5).

12. Databases available only by subscription…
PsycINFO – American Psychological Association (APA)
Allied and Complementary Medicine Database (AMED)
EMBASE
Note: Databases are described in the Supplementary Materials handout (see pages 3 – 5).

13. Guidelines available online…
Guidelines are a rich source of evaluated evidence, particularly if the question is about treatment or diagnosis of a relatively common medical condition
Available for free:
UK National Electronic Library for Health Guidelines Finder
US National Guideline Clearing House at
Guidelines International Network (GIN) at
BMJ Publishing Group’s Clinical Evidence
Note: Guidelines are described in the Supplementary Materials handout (see page 5).

14. Search through the Database
Use key words from your question as search terms
Use restrictions to refine the search
A clearly defined patient group and intervention are the major parameters in most subject searches
3. Search by author, journal title etc.
Note: Tips for searching databases are included in the Supplementary Materials handout (see page 6).

15. Approaches to research…
Qualitative
In-depth information about people’s experiences
Focus on subjective meanings
Typically small samples
Emphasis on credibility and trustworthiness
Quantitative
Information about how different variables are related to one another
Focus on objective measurement
Typically large samples
Emphasis on causality, reliability, validity

16. Types of studies you may find…
Systematic reviews
Randomised controlled trials
Quasi-experimental design
Evaluation studies with non-experimental designs
Case control studies
Cohort studies
Population surveys
Qualitative research
Note: There are descriptions of the types of studies you may find in the Supplementary Materials handout (see page 7 – 9).

17. Evaluating the Results

18. Initial questions to consider…
Do the title and abstract suggest a fit between the source and your evaluation?
Are the articles peer reviewed?
Is the study original?
Who is the study about? Is it reasonable to expect that the results might apply to your target group?
Are the claims made by the study plausible?
Note: Questions to consider are included in the Supplementary Materials handout (see pages 11 – 12).

19. Have the authors addressed all outcomes of interest?
What does the study add to what we already know?
The authors’ credentials: are the authors associated with the field of study? Do they have relevant clinical experience?
Are there any issues related to “researcher bias” that are not addressed?
Note: Questions to consider are included in the Supplementary Materials handout (see pages 11 – 12).

20. The Purpose and Method
Are the research questions clear, specific and answerable?
Is the study design appropriate? Do the question, method and analysis of results match up?
Is the sample appropriate?
If a comparison group was used, was it really comparable to the group receiving the intervention?
Did some people 'drop out' of the study, and if so, have the authors accounted for this in their conclusions?
Note: Questions to consider are included in the Supplementary Materials handout (see pages 11 – 12).

21. Results and Conclusions
Is the material presented in a way that is transparent and detailed in a way that can be easily examined and evaluated?
How large is the effect of the intervention, if there is one?
How precise is the estimate of the effect? How likely is it that the result was due to chance?
Have the authors clearly shown how they came to their findings?
Does the author explain study limitations?
Do the conclusions match the findings? Are the conclusions supported by the analysis?
Note: Questions to consider are included in the Supplementary Materials handout (see pages 11 – 12).

22. Synthesizing the Findings

23. Initial questions to consider…
What are the most relevant sources?
What are some common elements across sources?
What is unique about each source?
What are the key ideas/concepts being conveyed?
Why would this be important?
What are the limitations/gaps in the literature? (Note: this is particularly important, because it speaks to your contribution to the literature – i.e., your work may help fill this gap)
Note: Tips for synthesizing findings are included in the Supplementary Materials handout (see page 13).

24. Develop a Summary Table
Title and Publication Information
Type of Source and Approach
Overall Goal
Main Ideas and Conclusion
Limitations
Note: A sample of this summary table is included in the Supplementary Materials handout (see page 10).

25. Presenting the Findings

26. Planning the Literature Review
What are the goals of the review?
Who is the audience?
What are your key messages?
How should the review be organized?

27. Tips for Writing the Literature Review
Use the right word in the right place
Avoid awkward, rambling and run-on sentences
Avoid sentence fragments
Use clear, plain language
Always use examples or statements to substantiate your point
Feel free to use an active voice
Note: Tips for writing the literature review are included in the Supplementary Materials handout (see page 13).

28. Next Steps
Selecting measures?
Framework design?
Other ideas?

29. Visit our website for more information: www.onthepoint.ca
Note: Sites to visit for more information are included in the Supplementary Materials handout (see page 14).