Systematic Review: Interpreting Results and Identifying Gaps October 17, 2012

Stages of Systematic Review 1.Define the Problem 2.Literature Search 3.Data Evaluation 4.Data Analysis 5.Interpretation of Results 6.Presentation of Results

Finishing a systematic review is not an endpoint!

Outcomes of Systematic Review Answer a question with current evidence Describe status of present research literature on a subject Summarize and reflect Highlight what is known and unknown

Looking for an answer? Often cannot give a definitive answer, but… Leads to new theories Provides basis for future research Ideas for new directions in the field Provide information for insightful decisions on policy

Interpreting Systematic Review Results o Quality of included studies Study design Bias Missing data Heterogeneity of effect sizes Generalizability o Certainty vs uncertainty o Known vs unknown

Certainty in Systematic Review Certainty – confidence with which scientific community accepts associations between variables and theories for associations Need to determine how certain conclusions are High certainty – additional research of the kind evaluated is not necessary for further substantiation Low certainty – more research is needed

Certainty in Systematic Review o False certainty Conclusions portrayed as certain when they are not Encouraged by publication expectations? Failure to assess bias, heterogeneity, validity, mechanisms, etc Can discourage needed research o Need to acknowledge areas of uncertainty in systematic review Critical evaluation of goals of review: assessing relationships or assessing moderators of relationships?

Using the Results of Review Highly certain reviews provide a foundation of accepted knowledge o Concise base of theory to direct future work o Direct policy Uncertainty also directs the future o Synthesis can reveal what is lacking o Identify what studies need to be done o Identify the unknown o Uncertain vs negative or neutral results

Interpreting Systematic Reviews: Sometimes, More is Better Interpreting Systematic Reviews: Sometimes, More is Better

Improving the Next Generation of Research: Where are the gaps? Systematic review reveals studies that are completely lacking in the evidence base – important in Conservation Medicine Consider systematic review as a stepping stone Not a final phase for a question or hypothesis Systematic review is a method to generate new theories Use uncertainty to prioritize research

At what stage of a systematic review can you identify gaps?

Identifying Gaps May become apparent during literature search o Complete lack of studies on certain topics o Lack of certain study designs o Lack of proper methods Evaluation of certainty of conclusions o Quality, bias, validity Generalizable results o What research is needed to fill gaps?

Systematic Review and Public Policy Review question may aim to influence policy Provides alternative to single research study, expert consensus, personal or public opinions Answer questions about programs, policies, practices, or products Interest in evidence based practices

Influencing Public Policy Positive and negative influence Bias and utility of the review Who requested or initiated the review? Who funded the review? Influence of business Influence of politics Transparency of results

Public Policy and Conservation Medicine Often identify gaps in research! Systematic review as a tool to influence policy to fund research to fill gaps Prioritize research to fill gaps Institute policies that protect health Communication of review results to policy makers and public

Organic Foods Review What is the conclusion? How certain is the conclusion? What factors influence the certainty?

Organic Foods Review What specific new research is called for in the conclusion? What policy considerations are influenced by the conclusion?

Finishing Review Paper