1. Evaluation design for a community-based physical activity program for socially disadvantaged groups: Communities on the Move
20th EASM Conferende, Marion Herens
Annemarie Wagemakers, Johan van Ophem, Lenneke Vaandrager, Maria Koelen

2. Content Trend in physical activity
Rationale Communities on the Move approach (CoM)
Key principles of CoM
Research aims (cost)effectiveness CoM
Research questions
Evaluation design CoM

3. Physical activity in the Dutch adult population 2000-2009 (Hildebrandt et al, 2010)
Figure 1 Persons (%) of 18 years of age and older meeting the Dutch Healthy Physical Activity guidelines

4. Rationale CoM
Persistent health and physical activity inequalities despite policy actions and active health and physical activity promotion in the Netherlands (Van der Lucht et al, 2010; Van Oort et al, 2004)
Inactivity among Dutch adults stabilises since (5,5 %). Groups not meeting guidelines generally people in poor(er) socio-economic conditions (Hildebrandt et al, 2010)
Physical inactivity has been identified as the fourth leading risk factor for global mortality by WHO causing an estimated 3.2 million deaths globally (WHO-GAPA, 2012)

5. NISB: Development of CoM 2002-2012
feasibility study
pilot phase 1 (n=9)
Program development (theory and practice)
pilot phase 2 (n=11) evaluation and fine-tuning
2007 dissemination & implementation strategy
Nationwide implementa-
tion
2012 (Cost)effectiveness study
NISB: Development of CoM
Developed and disseminated by the Netherlands Institute for Sports and Physical Activity (NISB)
Aim: to initiate and enhance physical activity in inactive low SES groups
Multilevel community based programme based on 7 key principles
Achievements: use in over 40 municipalities, over 100 low SES groups

6. Principle based approach: Seven key principles matter...
Fun being active active
Social network
Environment
Participant level:
social network approach: find people where they meet
active participation: participants part of decision making
Fun factor: the fun of being active and play
Involvement of social and physical envrionment
Organisational level:
Group coaching
Intersectoral collaboration & networks
Local sustainability
The results reported by NISB through M&E pilot studies, process evaluation:
CoM reaches the ‘difficult to reach’ target groups successfully
People start to be more active and have fun
Sustainable behaviour change
More intersectoral collaboration
Learning together
Both participant and professionals are enthusiastic
Active involvement of the people in the social environment

7. Implemening organsisation
Human ecology CoM
Macro system (NISB, University, Ministry of Health & Sports, national policy, laws, norms
Exo system (employment situation, community organisation, health services)
Meso system (neighbourhood, school, sports club)
Micro system (family, relatives, friends)
Individual
Implemening organsisation
CoM
Community
CoM group

8. Research aims
Overall: Development of a context-sensitive monitoring and evaluation approach in order to measure the effectiveness and cost-effectiveness of community-based physical activity programs at distinct impact levels: individual, group, program, and community Based on: ”What is it about this approach that works, for whom, in what context, and why?”(Pawson & Tilly, 1997)
Test the design in the field through action research
Challenge to develop a research approach which has both practical and scientific relevance
Scientific challenges
Dutch Centre for Healthy living: CoM qualifies as theoretically underpinned. Question = efficacy ?
Practice: ‘we observe effects’. Question = what are those effects, what does it take to realise effects and under what conditions?
Collaboration practice-research: NISB and Wageningen University, Health & Society
Challenge is to develop a research approach which is has both practical and scientific relevance

9. Research questions (1) Specific research questions:
Individual level - Which effects can be documented with respect to physical activity behaviour and health, quality of life and life satisfaction?
Group level – How do active participation and group learning support behavioural change with respect to changes in physical activity and habitual behaviour?

10. Research questions (2)
Program level - Which mechanisms explain the successes and failures of Communities on the Move?
Overall - How can results be interpreted in terms of costs and benefits and what combination of economic evaluation methods and tools is most appropriate to evaluate a community-program on cost-effectiveness?

11. Study design effectiveness and cost-effectiveness of CoM
Multiple case study
Cohort – 16 groups/locations; monitored 18 months, 4 moments of measurement (t0-t3)
Multi level
quantitative data collection (measurements and questionnaires) at participant level (min. n=240)
qualitative data collection at group, community and organisational level (interview, focus group, document analysis)
4 locations in depth study (interview, focus group, observation, document analysis)

12. Data analysis and reporting6 12 18 24 30 36 42 48
 months
Preparation of questionnaires
Literature review
Document analysis
Field visits t3 t0 t1 t2
 t3
CiB program (in depth study)
2. CiB program
3. CiB program
4. CiB program
5. CiB program (in depth study)
6. CiB program
7. CiB program
8. CiB program
9. CiB program (in depth study)
10. CiB program
11. CiB program
12. CiB program
13. CiB program (in depth study)
14. CiB program
15. CiB program
16. CiB program
Data analysis and reporting

13. Level & study population
Variables
Individual level:
Participants
Socio demografic
Quality of life
Life satisfaction
Physical activity and habitual behaviour
BMI
Health related behaviour (e.g. smoking)
Sense of Coherence
Social network & support
Personal goals participants
Willingness to pay
Group level
Learning
Active Participation in the program
Program level
Local CoM organizations and NISB
Organization CoM, factors coordinated action
Support and training
Competences
Diffusion process (RE-AIM)
Cost per QALY
Community level
Participants CoM,
Local organizations
Community members
Spin-off
- new CiB programs
- community participation

14. Outputs of the (cost)effectiveness study of CoM
Overall: Recommendations for improving the health of low SES groups through physical activity.
Further research results include:
An elaborated monitoring and evaluation design for participatory community health and physical activity promotion
Assessment of CoM’s (cost-) effectiveness at the individual, program, and community level
The facilitation of wider implementation of CoM at both national and local level

15. Challenges
Feasibility comprehensive intervention and research approach: making practice and research work together
Combining M&E paradigms: results, learning & reflection orientation
Tools and research strategy development in relation to target groups and their competences
Essentie onderzoeksopdracht: probleemoplossend of legitimerend. Waar draagt het onderzoek aan bij: verbeteren monitoringssystematiek , beter bereik lage ses
Onderzoek zelf intervenieert op niveau van de interventie Communities in Beweging (CiB 2.0)
Grondslag Communities in Beweging; community based of sociale netwerktheorieën
Paradigmacombinatie of paradigmashift M&E in effecttiviteitsonderzoek Communities in Beweging ?
Resultaatgerichte methoden (objsct M&E  resultaten)–
Constructivische methoden (Object M&E  leerproces)
Reflexieve methoden (object M&E  Kwaliteit van het leerproces)
Kennisoverdracht of lerende kennissystemen om de verbinding onderzoek-beleid-praktijk te bouwen
Voorgestelde tools in relatie tot onderzoek van doelgroepen in een lage sociaal economische positie, aard en hoeveelheid ?
Kosteneffectiviteit, kostenutiliteit, kosten-baten ? Wat speelt een rol op welk niveau, en hoe relateren we dit aan waarde/waarderingssystemen

16. References
Hildebrandt VH, Chorus AMJ, & Stubbe JH (2010) Trendrapport bewegen en gezondheid 2008/ , TNO Kwaliteit van Leven Leiden.
WHO-Global Advocacy for Physical Activity (2012) Position Statement #2 Support for the inclusion of a global target on physical inactivity, GAPA. 2012, WHO - GAPA.
Van Oort FVA, van Lenthe FJ, & Mackenbach JP (2004) Co-occurrence of lifestyle risk factors and the explanation of education inequalities in mortality: results from the GLOBE study. Preventive Medicine, 39(6),
Van der Lucht F, Polder JJ (2010) Van gezond naar beter; Kernrapport van de Volksgezondheid Toekomstverkenning. 2010, Rijksinstituut voor Volksgezondheid en Milieu (RIVM), Bilthoven.
Pawson R & Tilley N (1997) Realistic evaluation, Thousand Oaks, CA US: Sage Publications, Inc.

17. Thank you for your attention
Marion Herens
PhD Candidate
M:  
T: +31(0) E:
Project supervision:
Maria Koelen, Annemarie Wagemakers, Johan van Ophem, Lenneke Vaandrager