1. The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community Wellbeing Melbourne School of Population Health Applicability and Transferability Role of review authors Belinda Burford Rebecca Armstrong Emma Tavender

2. The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community Wellbeing Melbourne School of Population Health Some definitions applicability and transferability are two very important and related concepts (often used interchangeably) Two sides of the one coin: –Applicability: whether or not it would be feasible to implement this intervention in a particular setting OR Is it possible? –Transferability: whether we can expect to see similar results after implementing this intervention in a particular setting OR Even if its possible, will it work?

3. The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community Wellbeing Melbourne School of Population Health Some challenges for complex reviews Heterogeneity, heterogeneity, heterogeneity! Population –Interactions between intervention, population characteristics and individual preferences/attitudes –Groups rather than individuals Intervention –Multiple components active components?; interactions between components? –Fidelity/integrity difficult to measure Comparisons –Poorly described Outcomes –Proxy/surrogate outcomes Setting –Important! Often poorly described

4. The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community Wellbeing Melbourne School of Population Health Image from: http://www.harlem- school.com/10TH/math_pdf/math.htmlhttp://www.harlem- school.com/10TH/math_pdf/math.html

5. The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community Wellbeing Melbourne School of Population Health Organising the information What are some approaches you have taken in your reviews to help the reader consider applicability/transferability?

6. The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community Wellbeing Melbourne School of Population Health What do end-users need from our reviews? Who are your end-users? –Differing needs What do they need to know 1 : –Could it work in my jurisdiction? –Will it work? –What would it take to make it work? –Is it worth it? Many resources for end-users for assessing applicability of systematic reviews: –SUPPORT tools to support policy-making –Applicability/transferability checklist 2 –Many, many more… 1 Lavis et al. Use of research to inform public policymaking Lancet 2004; 364: 1615–21 Lancet 2005 2 Wang et al 2005. Applicability and transferability of interventions in evidence-based public health. Health Promotion International, 21(1):76-83.

7. The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community Wellbeing Melbourne School of Population Health Starting from the end-user perspective: Applicability End-user question 1 What can the review provide? 2 Does the political environment of the local society allow for this intervention to be implemented? Study setting/context Is there any political barrier to implementing this intervention? Study setting/context Would the general public and the targeted (sub) population accept this intervention? Does any aspect of the intervention go against local social norms? Is it ethically acceptable? Characteristics of study population Study setting/context Can the contents of the intervention be tailored to suit the local culture? Intervention components Are the essential resources for implementing this intervention available in the local setting? Resources required Does the target population in the local setting have a sufficient educational level to comprehend the contents of the intervention? Characteristics of study population Which organisation will be responsible for the provision of the intervention in the local setting? Organisational context Is there any possible barrier to implementing this intervention due to the structure of that organisation? Barriers to implementation identified in studies Does the provider of the intervention in the local setting have the skill to deliver this intervention? If not will training be available? Who delivered the intervention? Was training provided? 1 Questions from: Wang et al 2005. Applicability and transferability of interventions in evidence-based public health. Health Promotion International, 21(1):76-83. 2 Table from: Burford et al. Forthcoming

8. The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community Wellbeing Melbourne School of Population Health End-user question 1 What can the review provide? 2 What is the baseline prevalence of the health problem of interest in the local setting? What was the difference in prevalence between the study setting and the local setting? Baseline prevalence in study populations Are the characteristics of the target population comparable between the study setting and the local setting? Is it possible that the characteristics of the target population, such as ethnicity, socioeconomic status, educational level etc will have an impact on the effectiveness of the intervention? Characteristics of study population Outcomes analysed by relevant socio-demographic factors Is the capacity to implement the intervention comparable between the study setting in such matters as political environment, social acceptability, resources, organisational structure and the skills of the local providers? Study setting/context Resources required Intervention deliverers Starting from the end-user perspective: Transferability 1 Questions from: Wang et al 2005. Applicability and transferability of interventions in evidence-based public health. Health Promotion International, 21(1):76-83. 2 Table from: Burford et al. Forthcoming

9. The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community Wellbeing Melbourne School of Population Health Organising the information PlacePopulation Geographical Setting Scale plus… consider factors important for your review… Age Gender Race/Ethnicity Income-level Education plus… Intervention CharacteristicsIntervention Components Types of roles involved Interaction between roles Delivery Flexibility/tailoring Component within a broader system/program plus… Number of components Type of components Frequency plus…

10. The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community Wellbeing Melbourne School of Population Health Approaches for dealing with this in your review Extrinsic Approach –Give context-dependent guidance for applying the review findings –1-2 paragraphs in the discussion –Implementation tables Intrinsic Approach –Build into the review planned subgroup analyses Good to do both

11. The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community Wellbeing Melbourne School of Population Health EXAMPLE: Interventions for preventing obesity in children 1 Data extraction –implementation factors based on practice- relevant tool Extrinsic approach –Described these across studies in results section and summarised in the discussion Intrinsic approach –Meta analyses to explore heterogeneity 1 Waters E, de Silva-Sanigorski A, Hall BJ, Brown T, Campbell KJ, Gao Y, Armstrong R, Prosser L, Summerbell CD. Interventions for preventing obesity in children. Cochrane Database of Systematic Reviews 2011, Issue 12. Art. No.: CD001871. DOI: 10.1002/14651858.CD001871.pub3.

12. The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community Wellbeing Melbourne School of Population Health Extrinsic Approach - Results Effectiveness –Maintenance / Sustainability of effects –Equity PROGRESS-Plus –Harm-adverse/unintended effects Implementation –Intervention design and theoretical basis –Process evaluation (including: intensity/dose, acceptability, barriers/facilitators) –Resources needed (including: who delivered, funding, hours of implementation, other?) –Strategies to address disadvantage/diversity Waters E, de Silva-Sanigorski A, Hall BJ, Brown T, Campbell KJ, Gao Y, Armstrong R, Prosser L, Summerbell CD. Interventions for preventing obesity in children. Cochrane Database of Systematic Reviews 2011, Issue 12. Art. No.: CD001871. DOI: 10.1002/14651858.CD001871.pub3.

13. The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community Wellbeing Melbourne School of Population Health Extrinsic approach: Discussion Applicability of the evidence This review included studies from high income countries as well as lower-middle- and upper-middle- income countries, with five studies conducted in countries within the latter two groupings (Thailand, Brazil, Chile and Mexico). This means that, while predominantly conducted within high-income settings, the findings from this review may be generalisable to a number of settings. A total of nineteen studies specifically reported incorporating strategies to target socio-economic and/or cultural diversity or disadvantage. One such study was conducted outside of the high-income country setting, in Chile, an upper-middle-income country. Of the remaining eighteen studies, seven studies conducted in the USA were of interventions targeting African American children and their communities and another two studies targeted Native American communities. Other studies targeted participants of low socio- economic status, or were implemented in areas of social disadvantage. By far the most common setting for interventions included in this review were schools (43 studies). Other interventions were (or included) home-based (14 studies), community-based (six studies), or were set in a health service (two studies) or care setting (two studies). Eleven studies incorporated interventions across multiple settings. Most interventions took a combined dietary and physical activity approach to obesity prevention (31 studies). As a single strategy, targeting physical activity alone was more popular (17 studies) than targeting diet alone (seven studies). The predominant theoretical basis for interventions in this review was behaviour change theory. Other theories represented include environmental change strategies, the socio-ecological framework, social learning theory, health promotion theory, transtheoretical models, and youth development and resiliency based approaches. The theoretical basis for interventions was explicitly reported in approximately half of the included studies. Waters E, de Silva-Sanigorski A, Hall BJ, Brown T, Campbell KJ, Gao Y, Armstrong R, Prosser L, Summerbell CD. Interventions for preventing obesity in children. Cochrane Database of Systematic Reviews 2011, Issue 12. Art. No.: CD001871. DOI: 10.1002/14651858.CD001871.pub3.

14. The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community Wellbeing Melbourne School of Population Health Intrinsic approach: Synthesis Meta analyses to explore heterogeneity: –Age group of participants –Setting: Education only Education plus other settings Non-education –Intervention type: PA Diet PA/diet combined –Intervention duration: <=12 months > 12 months Waters E, de Silva-Sanigorski A, Hall BJ, Brown T, Campbell KJ, Gao Y, Armstrong R, Prosser L, Summerbell CD. Interventions for preventing obesity in children. Cochrane Database of Systematic Reviews 2011, Issue 12. Art. No.: CD001871. DOI: 10.1002/14651858.CD001871.pub3.

15. The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community Wellbeing Melbourne School of Population Health The next update? Intrinsic –meta-regression: investigate multiple factors simultaneously (continuous and categorical) for their possible influence on the size of the intervention effect Caution: –Generally not considered when <10 studies in meta-analysis –Characteristics should be pre-specified (supported by scientific rationale) [ours will be post-hoc] –Select a small number of characteristics –Careful interpretation (Ch 9 of Cochrane Handbook) Extrinsic –Implementation tables

16. The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community Wellbeing Melbourne School of Population Health Using qualitative synthesis to explore heterogeneity of complex interventions 2 reviews: Qualitative review on HIV/AIDS patients perspectives on adherence Cochrane Review of intervention trials to improve adherence to therapy in HIV/AIDS patients Tabulated whether the interventions in the CR corresponded with the patients perspectives on how to help them Candy et al. BMC Medical Research Methodology 2011, 11:124

17. The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community Wellbeing Melbourne School of Population Health Exploring temporal study design: Handwashing behaviour in LMIC Vindigni et al. Systematic review: handwashing behaviour in low- to-middle-income countries: outcome measures and behaviour maintenance. Tropical Medicine and International Health. 2011;16(4):466-477

18. The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community Wellbeing Melbourne School of Population Health Handwashing: Temporal study design Community settings (n=16 studies) –Formative research: average = 4.8 months –Intervention: average = 17.2 months –Evaluation: average = 9.6 months Schools (n=4 studies) –Formative research: 1 study –Intervention: average = 4.8 months –Evaluation: average = 5 months Health care settings (n=7 studies) –Formative research: 1 study –Intervention: average = 2.5 months –Evaluation: average = 3.2 months Vindigni et al. Systematic review: handwashing behaviour in low-tomiddle-income countries: outcome measures and behaviour maintenance. Tropical Medicine and International Health. 2011;16(4):466-477

19. The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community Wellbeing Melbourne School of Population Health Example from the EPOC Group Population type Studies satisfying EPOC Design Criteria Comparative studies not satisfying EPOC Design Criteria Descriptive studies only Total Urban7121635 Urban Disadvantaged0167 Rural141419 Rural Disadvantaged101112 Total9174773 Are there different amounts of relevant evidence? Eg. Specialist outreach clinics in primary care and rural hospital settings

20. The McCaughey Centre VicHealth Centre for the Promotion of Mental Health and Community Wellbeing Melbourne School of Population Health Specialist Outreach review