1. Introduction to Systematic Reviews
September 27, 2018
(Eileen)
Good afternoon! Thank you for joining us for this session on Systematic Review! We are very excited to have such a diverse group of participants. Although the concept of systematic reviews began in the medical and public health disciplines, this type of research has spread out quickly over other subject disciplines. Participants who have registered for these workshops come from the following subject disciplines:
Civil Engineering
Counseling Psychology
Environmental Science & Technology
Geography
Information Studies
International Education Policy
Landscape architecture
Mechanical Engineering
Neuroscience and Cognitive Science
Nutrition and Food Science
Psychology
School of Languages, Literatures, and Cultures
School of Public Health (Family Science, Health Services Administration, Kinesiology, Epidemiology, Health Equity)
Again, welcome to our workshop and hope you will learn the basics for conducting a systematic review research.

2. Who we are? Eileen Harrington Sarah Over Stephanie Ritchie
Health & Life Sciences Librarian
Priddy Library
Sarah
Over
Engineering
Librarian
STEM Library
College Park
Stephanie Ritchie
Agriculture & Natural Resources Librarian
STEM Library
College Park
Nedelina Tchangalova
Public Health Librarian
STEM Library
College Park
We are librarians with various subject expertise.
Everyone introduce herself.

3. Workshop Outline What is a systematic review?
Why use this type of review?
Overall process & steps
How librarians can work with you
(Eileen)
After today you should be able to:  
Know the difference between different types of reviews
Understand the systematic review process
Identify some software tools might want to use to help with this process
Identify ways librarians can support you in conducting a systematic review
Lastly, if you forget all of that, you will leave here knowing how to ask for help! 

4. Systematic Review
“A properly conducted systematic review faithfully summarizes the evidence from all relevant studies on the topic of interest, and it does so concisely and transparently.”
(Cook et. al, 1997)
“The application of strategies that limit bias in the assembly, critical appraisal, and synthesis of all relevant studies on a specific topic.”
(Porta, 2008)
(Eileen)
When talking about studies--focuses on primary literature
Uses a protocol and everything in that seeks to mitigate bias

5. Literature Review vs. Systematic Review
(Eileen)
Key differences:
Systematic review has clear question or hypothesis; Lit review (also called narrative review) might have question, but more often just a general discussion of topic
Narrative does not attempt to review all of the literature and in fact sometimes might be biased as what include if trying to argue for a certain point of view, whereas a systematic review attempts to get all relevant literature to limit bias
Systematic review focuses on methodology in studies, often narrative doesn’t and narrative doesn’t spell out criteria for why included or excluded certain studies. In SR do
Another big difference is time: SR can take years—average is 18 months, while NR takes weeks to months.
Time Average 18 months Weeks or months
(Petticrew, 2001)

6. Other Review Types Scoping Review Rapid Review Meta-Analysis (Eileen)
Scoping review often used to see what gaps exist in the literature, to get an overview of a particular topic/question, make recommendations about areas that need further study. It usually starts with a broader question than a SR and sometimes and inclusion/exclusion criteria might not be determined until after an initial search, or might be modified slightly once have some results. It includes a broader range of study types, while SR often looks at one type of study. Usually “charts” or puts results into tables. Still  require comprehensive and structured searches of the literature to maximize recall and decrease bias, like SR.
Rapid review: shares similar methodology to a systematic review, but done much quicker—often in 5 weeks or less. Because of this there is a limit scope of the question, comprehensiveness of searching, and/or appraisal. Often only includes more recent literature.
Meta-analysis: SR where all of the numerical data from the studies included in it are collated and statistically analyzed, if it is possible to do so.
Image by

7. Why use a systematic review?
Evidence-based practice, policies, decision-making
Saves time and money
Requirement/Strengthen funding proposal for primary research
Can be alternative approach to primary research for master’s thesis or dissertation
(Eileen)
Often thought of as only for health or medicine, but can really be used in any field to help with evidence-based practice and policy development. So for example, to evaluate a particular educational approach or criminal justice reform.
Can save time an money in that professionals are using the most effective health treatment or most efficient technology for a particular task.
Since SR, and particularly scoping reviews can show where there are gaps in the literature, they can help strengthen a grant proposal or some grants might require you do one.
Some advantages of doing SR for master’s or thesis dissertation include not having to recruit participants or get Institutional Review Board approval, depending on inclusion/exclusion criteria can learn about a variety of different research methodologies. Some might do a scoping review as preliminary work for dissertation to help determine experimental study you want to conduct.

8. Examples (#1)
Müller, C., Lautenschläger, S., Meyer, G., & Stephan, A. (2017). Interventions to support people with dementia and their caregivers during the transition from home care to nursing home care: A systematic review. International Journal of Nursing Studies, 71, 139–152.
Question: What are the characteristics of interventions to support people with dementia and their informal caregivers during the transition period from home care to nursing home care?
Records identified: 1, Included in SR: 5 studies
Conclusions: There is still not enough evidence for issuing recommendations to implement psychosocial counseling and support interventions in clinical practice.
(Nedelina)
From bioengineering/health; Many neurological disorders accompanied by unwanted increased nerve activity, which can lead to pain or spasms.
30 of studies were conducted on animals—authors recommend more studies in humans

9. Examples (#2)
Clucas, B., et al. (2018). A systematic review of the relationship between urban agriculture and biodiversity. Urban Ecosystems, 1-9.
Question: Does urban agriculture promote urban biodiversity?
Records identified: Included in SR: 15 studies (2000 – 2017)
Conclusions: More studies needed, particularly in tropics
(Stephanie)
From agricultural and environmental studies: Often supporting biodiversity touted as a benefit of urban agriculture, but lack of empirical evidence. Found that although some studies found higher biodiversity in urban agriculture plots compared to vacant lots, not enough evidence to generalize that urban ag promotes urban diversity.

10. Examples (#3)
Šmite, D., Wohlin, C., Gorschek, T., & Feldt, R. (2010). Empirical evidence in global software engineering: A systematic review. Empirical software engineering, 15(1),
Questions: What is the state-of-the-art in empirical studies of GSE? What is the strength of the empirical evidence reflected in the empirical GSE literature?
Records identified: Included in SR: 59 studies (post 2000)
Conclusions: GSE observed to be still immature and driven by cost reduction strategies, which minimize globalization. Some best practices were identified in GSE as well.
(Sarah)
From science and engineering studies: software engineering

11. (Nedelina)
This diagram illustrates in a visual way and in plain language what review authors actually do in the process of undertaking a systematic review. 
Researchers identify an issue and develop a research question.
Once the questions is refined, resechers prepare a protocol or plan of action.
Find all relevant studies
Review the studies according to inclusion and exclusion criteria
Assess the quality of the studies, how well the studies have been conducted.
Analyze the results of the included studies
Discuss and conclude overall findings
Image:
Centre for Health Communication and Participation

12. Steps of SR - PIECES
P: Planning I: Identifying E: Evaluating C: Collecting/Combining E: Explaining S: Summarizing
(Nedelina)
To remember all the steps of the SR, think of the PIECES acronym.
Plan the search
Identify studies
Evaluate the studies
Collect/combine studies according to a preset form of criteria
Explain the included results
Summarize and make conclusions
Image by
(Foster, 2018)

13. P - Planning Assemble team Frame question
Check literature if SR hasn’t already been done on your topic
Develop (and register) a protocol
Create project management plan
(Nedelina)
The methods of the systematic review are generally decided before conducting it.
During this step: assemble your team, might just be two people, but could be more. Might include a librarian and a statistician if doing a meta-analysis.
Frame question: see LibGuide
Check the literature/protocol registries that SR hasn’t already been done on topic,
Develop your protocol, which will include exclusion and inclusion criteria, idea of where want to search. Things to consider when developing protocol: study populations, study design, intervention types, comparison groups, measured outcomes, languages, years published. It is recommended that you register your protocol so that someone else doesn’t start a SR on the same topic. Might also want to start thinking about where you would like to publish SR since certain journals might have requirements for way a SR is been done—specific databases they want you to search for example.
Create project management plan---start thinking about how you will organize work, manage articles during the retrieval process, document work
Image by

14. I – Identifying (#1) Develop search strategy: Identify major concepts
Term harvesting
Text mining tools
Combine with Boolean operators – AND, OR, NOT
Search Filters/Hedges
Translate for different databases
(Stephanie)
In this step, searching for studies which match the preset criteria in a systematic manner before doing this need to develop search strategy
First you want to look at your research question and identify the major concepts
Can also use term harvesting techniques: look at abstract and assigned keywords from gold standard articles (articles you would expect to be included in final SR); look at search strategies found in published SRs
Can use text mining tools to help with term harvesting
When have terms you are going to use you will be combing them using Boolean Operators
Might also want to add Search Filters/Hedges to overall search string. Search hedges are vetted strategies created by expert searchers to find specific literature. Some are population-based and some are publication-type based
Want to translate search for different databases since between databases they might have different syntaxes that they use
Librarians can be very helpful with this stage
Image by

15. I – Identifying (#2) Implement search: Library databases
Regional/specialized databases
Grey literature
Handsearching
Citation managers
(Stephanie)
Some journals or organizations, such as Cochrane, might require that you search certain databases. Different fields have developed guidelines for conducting SR and include suggestions on databases to search. Links to guidelines and other literature, as well as databases that are recommended for different subject areas on SR subject guide
Depending on topic and/or population might want to check regional databases or there might be smaller databases created by research centers or organizations focused on your topic
Grey literature includes things like government, NGO, think-tanks, research consultants reports, conference proceedings, theses and dissertations. Can be difficult to search because not organized systematically. However, there are some resources that can help, which are on the Subject Guide. Even with difficulties, important to search it to reduce bias
Handsearching is manual page-by-page examination of the entire contents of a journal issue or conference proceedings. Why handsearch? locates relevant --to find articles that are poorly or inaccurately indexed or unindexed (some journals that might be relevant not indexed in your chosen databases);
--allows researchers to scan content quickly for relevant studies from the high-impact journals, and
--ensures that relevant studies are not overlooked
Will want to use a citation manager to export/keep track of articles as find them, such as EndNote (online version is free), Mendeley or Zotero, might consider purchasing Desktop version of EndNote
Documentation key throughout process—how developed search strategy, where searched, etc.
Image by

16. E - Evaluating
Select studies for inclusion based on pre-defined criteria
Open access tools:
Excel
Distiller
Rayyan
RevMan
GradePro
(Stephanie)
Sort all retrieved articles (included or excluded) and assess the risk of bias for each included study
Excel is the most basic tool for the management of the screening and data extraction stages of the systematic review process. Customized workbooks and spreadsheets can be designed for the review process, and lists of references can be exported from citation managers into Excel format for screening.
Basic examples include Database Search Log and Review Matrix.
A more advanced way to using Excel for this purpose is the PIECES approach, designed by a librarian at Texas A&M.
Distiller is designed for screening and data extraction phases of systematic reviews. Free for student as a single user for one project for 1 semester with no collaboration with other users,
Rayyan is a free online tool that can be used for independent screening and coding of studies in a systematic review. Rayyan uses tagging and filtering to code and organize references. Citations can be uploaded from RefMan, EndNote, BibTex, CSV or zip file formats.
RevMan is used by Cochrane reviewers to manage the data extraction and analysis process. It is a free tool that is geared toward Cochrane-like reviews.
GradePro is used to create summary of findings tables. Free for single-users and non-profits. 
Image by

17. C – Collecting/Combining
Assign codes to included articles
Extract and analyze data from studies either quantitatively or qualitatively
(Stephanie)
Each study is coded with preset form, either qualitatively or quantitatively synthesize data (if doing a meta-analysis)
Image by

18. E - Explaining
Summarize and draw conclusions from synthesis of data, such as:
Central focus & problem addressed
Review of key results
Information about the sample population
Evaluate the bias risk of included studies – Cochrane RoB Tool
(Sarah)
Placed results of synthesis into context--outline strengths and weaknesses of the studies
Evaluate the risk of bias of included studies. Use a Risk of Bias tool (such as the Cochrane RoB Tool) to assess the potential biases of studies in regards to study design and other factors. Read the Cochrane training materials to learn about the topic of assessing risk of bias in included studies. You can adapt existing tools (PDF p.5) to best meet the needs of your review, depending on the types of studies included.
Cochrane was designed for health-care and may not be relevant for all fields, but is a good starting point. See the SR Guide for further information.

19. S - Summarizing
Write report with clear and transparent description of methods and results
 Follow guidelines:
EQUATOR Network
PRISMA Checklist
Find the best journal to publish your work
(Sarah)
Clearly present your findings, including detailed methodology (such as search strategies used, selection criteria, etc.) such that your review can be easily updated in the future with new research findings. Perform a meta-analysis, if the studies allow. Provide recommendations for practice and policy-making (depending on discipline) if sufficient, high quality evidence exists OR future directions for research to fill existing gaps in knowledge or to strengthen the body of evidence.
There are guidelines for many different types of SR/disciplines—the EQUATOR Network has The EQUATOR wizard will help you choose the right reporting guideline for your work. The EQUATOR search will allow you to browse a library of reporting guidelines, or search for a specific checklist. The PRISMA Checklist can also be helpful – the main point is to make certain you are following the standards for reporting the results of a systematic review
If you haven’t already done so, find a journal to publish your work—tools on subject guide to help with this.

20. Other Tools for Managing Your Project
Systematic Review Toolbox
Reviews of software tools for SR—University of British Columbia
(Sarah)
Systematic Review Toolbox: community-driven, searchable, web-based catalogue of tools to support the systematic review. Options include a simple keyword search (i.e. Quick Search) to locate tools and a more detailed search (i.e. Advanced Search) to find specific types of tools. Submit new tools too!
Reviews of software gives a list of different packages and tools that may be useful, including different reference managers (key for any large research project)
Image by

21. Support from Librarians
Tier 1: General consulting
Tier 2: Searching assistance
Tier 3: Develop & implement searches, manage results and prepare for screening
(Sarah)
Connecting areas of research, insuring full coverage of research area and full use of resources
Different tiers of service and each one has different requirements:
Tier 1: One hour (in-person) consultation - provide a basic overview of the systematic review process: protocol development, search strategy design, selecting relevant databases, collecting and organizing studies, screening the results, and writing the manuscript, etc.
Tier 2: Acknowledgement - assistance in generating key terms, creating search strings for specific databases, and/or reviewing search strings created by the researcher
Tier 3: Co-authorship (middle author) - develop the search strategy, execute the searching in various databases, manage the studies and prepare them for screening by the researcher and write any portion of the search methodology
For additional information and self-help see our SR Subject Guide
Image by
Self-guided help

22. References
Clucas, B., Parker, I. D., & Feldpausch-Parker, A. M. (2018). A systematic review of the relationship between urban agriculture and biodiversity. Urban Ecosystems, 21(4), 635– Cook, D. J., Mulrow, C. D., & Haynes, R. B. (1997). Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine, 126(5), 376–380. Müller, C., Lautenschläger, S., Meyer, G., & Stephan, A. (2017). Interventions to support people with dementia and their caregivers during the transition from home care to nursing home care: A systematic review. International Journal of Nursing Studies, 71, 139– Foster, M. (2018). Systematic reviews service: Introduction to systematic reviews. Retrieved September 18, 2018, from Petticrew, M. (2001). Systematic reviews from astronomy to zoology: myths and misconceptions. BMJ, 322(7278), 98– Porta, M. S. (Ed.). (2008). A dictionary of epidemiology. (5th ed.). Oxford ; Oxford University Press. Šmite, Darja, et al. (2010). Empirical evidence in global software engineering: a systematic review. Empirical Software Engineering, 15(1), 91–118.
(Sarah)

23. Questions
What would you like to see covered in future systematic review workshops?
(Sarah)
Place blank paper on table for the participants to write down their suggestions for future workshops.
Ask audience for questions they might have, but then also ask them what they would like to see covered in future systematic review workshops—would they like one solely focused on searching?

24. Thanks for attending!