Evaluation design for a community-based physical activity program for socially disadvantaged groups: Communities on the Move 20th EASM Conferende, 2012-09-2012 Marion Herens Annemarie Wagemakers, Johan van Ophem, Lenneke Vaandrager, Maria Koelen

Content Trend in physical activity Rationale Communities on the Move approach (CoM) Key principles of CoM Research aims (cost)effectiveness CoM Research questions Evaluation design CoM

Physical activity in the Dutch adult population 2000-2009 (Hildebrandt et al, 2010) Figure 1 Persons (%) of 18 years of age and older meeting the Dutch Healthy Physical Activity guidelines

Rationale CoM Persistent health and physical activity inequalities despite policy actions and active health and physical activity promotion in the Netherlands (Van der Lucht et al, 2010; Van Oort et al, 2004) Inactivity among Dutch adults stabilises since 2005 (5,5 %). Groups not meeting guidelines generally people in poor(er) socio-economic conditions (Hildebrandt et al, 2010) Physical inactivity has been identified as the fourth leading risk factor for global mortality by WHO causing an estimated 3.2 million deaths globally (WHO-GAPA, 2012)

NISB: Development of CoM 2002-2012 2001-2002 feasibility study 2003-2004 pilot phase 1 (n=9) Program development (theory and practice) 2005-2006 pilot phase 2 (n=11) evaluation and fine-tuning 2007 dissemination & implementation strategy 2008-2012 Nationwide implementa- tion 2012 (Cost)effectiveness study NISB: Development of CoM 2002-2012 Developed and disseminated by the Netherlands Institute for Sports and Physical Activity (NISB) Aim: to initiate and enhance physical activity in inactive low SES groups Multilevel community based programme based on 7 key principles Achievements: use in over 40 municipalities, over 100 low SES groups

Principle based approach: Seven key principles matter... Fun being active active Social network Environment Participant level: social network approach: find people where they meet active participation: participants part of decision making Fun factor: the fun of being active and play Involvement of social and physical envrionment Organisational level: Group coaching Intersectoral collaboration & networks Local sustainability The results reported by NISB through M&E pilot studies, process evaluation: CoM reaches the ‘difficult to reach’ target groups successfully People start to be more active and have fun Sustainable behaviour change More intersectoral collaboration Learning together Both participant and professionals are enthusiastic Active involvement of the people in the social environment

Implemening organsisation Human ecology CoM Macro system (NISB, University, Ministry of Health & Sports, national policy, laws, norms Exo system (employment situation, community organisation, health services) Meso system (neighbourhood, school, sports club) Micro system (family, relatives, friends) Individual Implemening organsisation CoM Community CoM group

Research aims Overall: Development of a context-sensitive monitoring and evaluation approach in order to measure the effectiveness and cost-effectiveness of community-based physical activity programs at distinct impact levels: individual, group, program, and community Based on: ”What is it about this approach that works, for whom, in what context, and why?”(Pawson & Tilly, 1997) Test the design in the field through action research Challenge to develop a research approach which has both practical and scientific relevance Scientific challenges Dutch Centre for Healthy living: CoM qualifies as theoretically underpinned. Question = efficacy ? Practice: ‘we observe effects’. Question = what are those effects, what does it take to realise effects and under what conditions? Collaboration practice-research: NISB and Wageningen University, Health & Society Challenge is to develop a research approach which is has both practical and scientific relevance

Research questions (1) Specific research questions: Individual level - Which effects can be documented with respect to physical activity behaviour and health, quality of life and life satisfaction? Group level – How do active participation and group learning support behavioural change with respect to changes in physical activity and habitual behaviour?

Research questions (2) Program level - Which mechanisms explain the successes and failures of Communities on the Move? Overall - How can results be interpreted in terms of costs and benefits and what combination of economic evaluation methods and tools is most appropriate to evaluate a community-program on cost-effectiveness?

Study design effectiveness and cost-effectiveness of CoM Multiple case study Cohort – 16 groups/locations; monitored 18 months, 4 moments of measurement (t0-t3) Multi level quantitative data collection (measurements and questionnaires) at participant level (min. n=240) qualitative data collection at group, community and organisational level (interview, focus group, document analysis) 4 locations in depth study (interview, focus group, observation, document analysis)

Data analysis and reporting 6 12 18 24 30 36 42 48  months Preparation of questionnaires Literature review Document analysis Field visits   t3 t0 t1 t2  t3 CiB program (in depth study) 2. CiB program 3. CiB program 4. CiB program 5. CiB program (in depth study) 6. CiB program 7. CiB program 8. CiB program 9. CiB program (in depth study) 10. CiB program 11. CiB program 12. CiB program 13. CiB program (in depth study) 14. CiB program 15. CiB program 16. CiB program Data analysis and reporting

Level & study population Variables Individual level: Participants Socio demografic Quality of life Life satisfaction Physical activity and habitual behaviour BMI Health related behaviour (e.g. smoking) Sense of Coherence Social network & support Personal goals participants Willingness to pay Group level Learning Active Participation in the program Program level Local CoM organizations and NISB Organization CoM, factors coordinated action Support and training Competences Diffusion process (RE-AIM) Cost per QALY Community level Participants CoM, Local organizations Community members Spin-off - new CiB programs - community participation

Outputs of the (cost)effectiveness study of CoM Overall: Recommendations for improving the health of low SES groups through physical activity. Further research results include: An elaborated monitoring and evaluation design for participatory community health and physical activity promotion Assessment of CoM’s (cost-) effectiveness at the individual, program, and community level The facilitation of wider implementation of CoM at both national and local level

Challenges Feasibility comprehensive intervention and research approach: making practice and research work together Combining M&E paradigms: results, learning & reflection orientation Tools and research strategy development in relation to target groups and their competences Essentie onderzoeksopdracht: probleemoplossend of legitimerend. Waar draagt het onderzoek aan bij: verbeteren monitoringssystematiek , beter bereik lage ses Onderzoek zelf intervenieert op niveau van de interventie Communities in Beweging (CiB 2.0) Grondslag Communities in Beweging; community based of sociale netwerktheorieën Paradigmacombinatie of paradigmashift M&E in effecttiviteitsonderzoek Communities in Beweging ? Resultaatgerichte methoden (objsct M&E  resultaten)– Constructivische methoden (Object M&E  leerproces) Reflexieve methoden (object M&E  Kwaliteit van het leerproces) Kennisoverdracht of lerende kennissystemen om de verbinding onderzoek-beleid-praktijk te bouwen Voorgestelde tools in relatie tot onderzoek van doelgroepen in een lage sociaal economische positie, aard en hoeveelheid ? Kosteneffectiviteit, kostenutiliteit, kosten-baten ? Wat speelt een rol op welk niveau, en hoe relateren we dit aan waarde/waarderingssystemen

References Hildebrandt VH, Chorus AMJ, & Stubbe JH (2010) Trendrapport bewegen en gezondheid 2008/2009. 2010, TNO Kwaliteit van Leven Leiden. WHO-Global Advocacy for Physical Activity (2012) Position Statement #2 Support for the inclusion of a global target on physical inactivity, GAPA. 2012, WHO - GAPA. Van Oort FVA, van Lenthe FJ, & Mackenbach JP (2004) Co-occurrence of lifestyle risk factors and the explanation of education inequalities in mortality: results from the GLOBE study. Preventive Medicine, 39(6), 1126-1134. Van der Lucht F, Polder JJ (2010) Van gezond naar beter; Kernrapport van de Volksgezondheid Toekomstverkenning. 2010, Rijksinstituut voor Volksgezondheid en Milieu (RIVM), Bilthoven. Pawson R & Tilley N (1997) Realistic evaluation, Thousand Oaks, CA US: Sage Publications, Inc.

Thank you for your attention Marion Herens PhD Candidate M: 06 31753714  T: +31(0)317 483670 E: marion.herens@wur.nl www.hso.wur.nl Project supervision: Maria Koelen, Annemarie Wagemakers, Johan van Ophem, Lenneke Vaandrager