1. Evaluating Health Promotion: Why Does Context Matter? Louise Potvin, PhD Chair Community Approaches and Health Inequalities Université de Montréal 2 nd Brazilian Seminar on Health Promotion Effectiveness Rio de Janeiro, May 14th 2008

2. Acknowledgements Sherri L. Bisset, PhD candidate (U. de Montréal) whose thesis provides empirical illustrations for this presentation Contributors to: Health Promotion Evaluation Practice in the Americas: Values and Research (forthcoming). New York: Springer.

3. Thesis Health promotion programs are systems of actions that operate as socio-technical networks: they produce innovations by creating and strengthening new linkages between technical devices and local actors The role of evaluation is to produce knowledge to guide future action regarding parts of the socio-technical network, in the same or other contexts Evaluation’s important role is to document links between technical devices, local actors, conditions and actions, and changes in local context

4. Plan 1.Programs in context: health promotion as a socio-technical network 2.Program implementation: Operating the expansion and consolidation of a socio- technical network 3.Program evaluation: context matters

5. Part 1: Health Promotion as Socio- Technical Network

6. The Ottawa Charter: Two Innovations Definition of health –Health is produced in every day life; linked to access to local resources/conditions Principles of action –Participation & Empowerment: legitimacy of non expert, local knowledge –Intersectoral: health producing resources are accessible through non health sector providers

7. A Social Definition of Program Programs are social constructions: a tinkering of previously unrelated (or loosely related), and disparate human and non human components: –Knowledge (model of action; best practice; local culture) –People (staff; target population; partners) –Problem (determinants, consequences) –Technical devices (compound; manual; meeting minutes) –Activities (meetings; courses; celebrations) –Resources (financial; material; human) A mix of “local” and “imported” elements Need a social process to explain how such heterogeneous elements can hold together

8. The Actor-Network Theory (ANT) Michel Callon; Bruno Latour; John Law Two underlying stories (J. Law): 1.Relational materiality: « Entities take their form and acquire their attributes as a result of their relations to other things… divisions and distinctions are understood as effects. They are not given in the order of things » 2.Performativity: « Entities are preformed in, by, and through, those relationships. »

9. Socio-Technical Network Focus on actions and interactions: Both actors and networks are outcomes and linked through action. Actors: no ontological rupture between human and non-human (technical) entities; both are capable of agency. Both are defined by, and define, the network Two orientations of action: stabilising existing relations and making new connections Translation links heterogeneous entities into a network

10. Socio-Technical Network of Health Promotion: School-Based Nutrition Problem: feeding children Mothers: time & skills Lunch in schools Problem: lack of success in school Problem: skills & Interest in food Nutritionists Nutrition workshops Didactic links with School subjects External funder School teachers & principals Mothers: support in workshop University health researcher University education researcher Ministry of Education Other school boards Brazilian health promotion evaluators

11. Part 2: Expansion and consolidation of a socio-technical network

12. Translation Operation of linking the network’s heterogeneous entities Ongoing interpretations/reinterpretations by actors of their roles and of the innovative product, going from their respective interests and their power relations and leading to the elaboration of compromises Four operations: –Problematization –Interessement –Enrolment –Mobilization

13. Problematization Setting in motion of actors around a provisional and minimum project Definition of the problem/situation by the project or innovation promoters, Identification of affected actors, their interests and the issues linking them Assingment of roles and identities HEALTH PROMOTION: assigning health meanings to non-health entities

14. Interessement Set of strategies adopted by the various actors with a view to: –rallying the other actors around a shared objective –defining their role Interessement strategies seek to align actors’ new identities and roles with their interests Interessement strategies take shape in material devices HEALTH PROMOTION: actions and artefacts that concretely link existing entities with health; knowledge, partnership agreements,

15. Enrolment Enrolment occurs when the actors take on a role in the network in line with the problematization Successful interessement gives rise to negotiation which leads to acceptance of a precise role enabling the network’s consolidation HEALTH PROMOTION: integration of new health-related roles and identities by networks entities

16. Mobilization Mobilization concerns the involvement of a critical mass of actors in the action system so that innovation becomes relevant, useful, indispensable Actor mobilization, above and beyond their representatives, leads to network extension In contrast, the absence of solidity of representatives leads to controversies HEALTH PROMOTION: capacity to displace entities and orient their actions in a health-related direction

17. Translation PROBLEMATIZATION INTERESSEMENTMOBILIZATION ENROLMENT CONTROVERSIES

18. An Example: Petits Cuistot-Parents en Réseaux Problem: lack of success in school Problem: skills & Interest in food Nutritionists Nutrition workshops Didactic links with School subjects External funder School teachers & principals Mothers: support in workshop

19. Part 3: Context and Program Evaluation

20. Evaluation and Socio-Technical Networks Socio-technical networks are performative: they acquire reality through action Movements in socio-technical networks: consolidating existing linkages or expansion through new linkages Evaluation: systematic knowledge to inform movements for consolidation or expansion of network

21. Knowledge to Support Consolidating Network Focus on the internal context of programs Modeling existing entities (numbers, identities, roles) their links (strength, meaning), actions (number, nature, roles) and controversies (nature, solutions; translations); Methods: systematization Complementarity of qualitative and quantitative information

22. Knowledge to Support Expanding Network Networks expansion : enrolling new entities through new or renewed linkages Focus on the link between existing network (internal context) and external context The distinction between external and internal is contingent

23. Lessons from Health Promotion Evaluation Practices in the Americas Research methods are not « context-neutral »; evaluation activities are part of a program context and there are meaningful linkages between evaluation and program Innovative evaluation practices strengthen links between local context, evaluation and program: Evaluators have difficulties to reflect on their practice beyond the technical aspects of cellecting empirical data