Analytic Frameworks Prepared for: Agency for Healthcare Research and Quality (AHRQ) Training Modules for Systematic Reviews Methods Guide

Systematic Review Process Overview

 To understand the use of analytic frameworks within systematic reviews  To develop an understanding of principles for developing analytic frameworks  To understand which elements of a review can be represented in an analytic framework Learning Objectives

 A type of evidence model that links and defines clinical concepts, evidence, and populations as they relate to outcomes  Sometimes referred to as causal pathways  Alternative and related concepts are:  Conceptual frameworks  Influence diagrams  Theoretical frameworks  Logic models What Is an Analytic Framework?

 To provide clinical, biological, or health services underpinnings for the mechanism of action  To clarify implicit assumptions  To identify logical flaws as key questions are developed  To provide a starting point for discussions with key informants  To identify distinct bodies of evidence to be included in the review Why Use Analytic Frameworks?

 Specifies populations, interventions, and outcomes, and sometimes comparators, timing, and settings  Identifies potential modifiers and mediators of effectiveness  Clarifies links between intermediate and health outcomes The Importance of an Analytic Framework

 Complex bodies of literature with multiple key questions:  Analytic frameworks help define key questions, and thus direct specific literature searches.  Complex chain of logic between intermediate and clinical outcomes:  Analytic frameworks help users understand decisional context. When To Use Analytic Frameworks

 Population of interest  Intervention  Linkages that demonstrate key questions  Intermediate outcomes  Ultimate health outcomes (including harms) Components

 The population, intervention, and outcomes should be clearly identified in the analytic framework.  All key questions should be represented clearly on the framework.  Contextual information about modifiers and mediators may be appropriate.  Analytic frameworks are drafted with the purpose of clarifying complex questions and will need iterative revisions through the topic refinement process (see the module, “Topic Refinement”). Revising the Framework

Depicting Essential Components: An Example  Graphical chain of logic  Arrows  Linkages  Preventive service or treatment  Questions  Dotted lines  Associations  Rectangles  Intermediate outcomes  Rounded corners  Health states  Square corners  Curved arrows  Lead to ovals  Harms Harris RP, et al. Am J Prev Med 2001;20(Suppl):21-35; Woolf SH, et al. Annu Rev Public Health 1996;17:

Key Research Questions The numbers that represent the key questions should be placed in the relevant position within the framework Whitlock EP, et al. Am J Prev Med 2002;22:

Sample Working Framework Harris RP, et al. Am J Prev Med 2001;20(Suppl):21-35.

Sample Screening and Treatment Framework Hartmann KE, et. Evidence Report/Technology Assessment No Available at: PopulationInterventionOutcomes

Capturing the Context Analytic frameworks can provide a means of capturing contextual issues important to a review Vanderbilt University Evidence-based Practice Center. Systematic review protocol: traumatic brain injury and depression. Available at: care.ahrq.gov/ehc/product s/77/367/TBI%20and%20 Depression%20( ).pdf.

 Do not build the framework and forget it  Use the framework throughout the review process to:  Revisit inclusion/exclusion criteria  Keep a handle on scope  Reconsider the appropriateness of key questions  Guide interviews with key informants  Let the framework help in structuring the report and results Using the Framework

 Analytic frameworks help reviewers clarify specific key questions and direct literature searches of complex bodies of literature.  Analytic frameworks help end-users understand the decisional context of key questions when there is a complex chain of logic.  Analytic frameworks can help the iterative process of identifying specific researchable key questions in the context of complex clinical or policy issues. Key Messages

 Agency for Healthcare Research and Quality. Topic development. In: Methods reference guide for effectiveness and comparative effectiveness reviews. Version 1.0 [Draft posted Oct. 2007], Chapter 2, Finding evidence. Rockville, MD: Agency for Healthcare Research and Quality; Draft Posted October p Available at: f.  Bravata DM, McDonald KM, Shojania KG, et al. Challenges in systematic reviews: Synthesis of topics related to the delivery, organization, and financing of health care. Ann Intern Med 2005;142(Pt 2):  Harris RP, Hefland M, Woolf SH, et al. Current methods of the US Preventive Services Task Force: a review of the process. Am J Prev Med 2001;20(Suppl):  Hartmann KE, McPheeters ML, Biller DH, et al. Treatment of Overactive Bladder in Women. Evidence Report/Technology Assessment No. 187 (Prepared by Vanderbilt University Evidence-based Practice Center under Contract No I). Rockville, MD: Agency for Healthcare Research and Quality, August AHRQ Publication No. 09-E017. Available at: References (I)

 Mulrow C, Langhorne P, Grimshaw J. Integrating heterogeneous pieces of evidence in systematic reviews. Ann Intern Med 1997;127:  Vanderbilt University Evidence-based Practice Center. Systematic review protocol: traumatic brain injury and depression. Rockville, MD: Agency for Healthcare Research and Quality, Posted December 9, Available at: %20Depression%20( ).pdf.  Whitlock EP, Orleans CT, Pender N, et al. Evaluating primary care behavioral counseling interventions: an evidence-based approach. Am J Prev Med 2002;22:  Woolf SH, DiGuiseppi CG, Atkins D, et al. Developing evidence-based clinical practice guidelines: lessons learned by the US Preventive Services Task Force. Annu Rev Public Health 1996;17: References (II)

 This presentation was prepared by Melissa L. McPheeters, Ph.D., M.P.H., a member of the Vanderbilt University Evidence-based Practice Center.  The module is based on an update of chapter 2 in version 1.0 of the Methods Guide for Comparative Effectiveness Reviews (available at: roducts/60/294/2009_0805_principles1.pdf). Author