1. IUMSP Institut universitaire de médecine sociale et préventive, Lausanne Evaluators’ perspectives on evaluation – perspectives for the future F. Dubois-Arber, Institut universitaire de médecine sociale et préventive, Lausanne Evaluation in public health – lessons learned and future directions

2. IUMSP Institut universitaire de médecine sociale et préventive, Lausanne 2 Conflicting expectations regarding evaluation  Decrease in the resources available for public policies  Increasing demand for evaluations, especially for policy evaluations  Demand for accountability (politicians)  Demand for « high level of evidence » (policy makers)  Demand for guidance, consensus building (programme managers)  Demand for immediate answers on long term issues

3. IUMSP Institut universitaire de médecine sociale et préventive, Lausanne 3 Consequences  Nature of evaluations  Approaches to evaluation and tender procedures  Methods  Profession

4. IUMSP Institut universitaire de médecine sociale et préventive, Lausanne 4 Less evaluations- higher quality, comprehensive, policy/ programmes evaluations Issues at stakes for increased relevance to various stakeholders :  Earlier in the programme/ policy process  More time for the preparation of the evaluation  Adequate financing  Adequate duration of evaluation  Evolutions in the design, in the expertise brought in the evaluation  Flexibility in responding to multiple evaluation needs

5. IUMSP Institut universitaire de médecine sociale et préventive, Lausanne 5 More user-focused evaluations and more appropriate procedures of call for tender  This approach should be – from the very beginning – embodied in the direct exchanges and the discussions between programme managers/policy makers and evaluators, with the commissioners as facilitators  In their current format, calls for tender are not the best way of choosing the evaluators who will be able to take into account the various needs in evaluation  The process should be more open with the possibility of discussing evaluation questions and possible options for evaluation, scenarii and trade-offs, etc. in direct interaction before proposing protocols  Dedicated resources (time and money) would allow a better appropriation of the matter and of the issues at stakes: investment in evaluation proposal

6. IUMSP Institut universitaire de médecine sociale et préventive, Lausanne 6

7. 7 Evaluating policies, evaluating systems  Need to acknowledge the « systemic » nature of policies/ programmes and evaluate them as such  Better understanding and measurement of the evaluation object (policy/programme)  Appreciation of the possibilities of obtaining results (policy/programme theory)  Appreciation of the outcomes timing  Assessment and measurement of the intensity of interventions  Assessment and measurement of the combination and complementarity/opposition of interventions  Assessment and measurement of interventions coverage  Assessment and measurement of relevant elements of the environment (political will, convergence/ divergence between stakeholders, norms, etc.)

8. IUMSP Institut universitaire de médecine sociale et préventive, Lausanne 8 Setting up and using monitoring systems for outcome/impact measurement  Repeated, consistent over time and valid data collections  Avoid the multiplication of specific data collections  Better use of existing statistics and reporting systems  Sharing the information (coordinating bodies)  Consensus building on the information needed, trade offs  Long term thinking

9. IUMSP Institut universitaire de médecine sociale et préventive, Lausanne 9 Professionalisation of evaluation : necessary not sufficient  Tension between the profession of evaluator and the original discipline(s)  Tension between the craftsman(woman) and the academic  Teams and multidisciplinarity as a solution

10. IUMSP Institut universitaire de médecine sociale et préventive, Lausanne 10