Building evaluation capacity: success factors and approaches in NSW Health NSW Government Evaluation Conference 2015 Andrew Milat PhD, MPH (Hons), BHMS ed (Hons) Associate Director, Evidence and Evaluation NSW Ministry of Health

Research and evaluation in the health cluster

 NSW Ministry of Health  NSW Office of Preventive Health  Agency for Clinical Innovation  Cancer Institute NSW  Kids and Families NSW  Justice Health & Forensic Mental Health Network Research and evaluation support services in the health cluster

Provision of strategic R&E advice and support services Practice guidesResearch and policy exchanges R&E training and capacity building Research funding R&E infrastructure support activities Key Functions

NSW Health Evaluation Capability Building Sub-group Objectives  identify key strategies to build evaluation capability in the Ministry and the Pillars  forge stronger links with the NSW Government Evaluation Community of Practice (ECoP)  Facilitate collaboration and communication across the cluster

Evaluation capacity building is: ‘…intentional efforts to create and sustain organisational processes that make quality evaluation and its uses routine.’ Source: Stockdill, Baizerman, and Compton 2002, p. 14. Evaluation capacity building

Objective:  To identify evaluation capacity building strategies employed by organisations and program developers and to describe success factors and lessons learned. Methods:  Key word search of electronic databases in health and social sciences. The literature review was conducted in 3 phases in January Results:  Phase abstracts reviewed  Phase full papers assessed  Phase papers in final review  Content analysis was used to identify success factors Literature review: evaluation capacity building literature

Review: Success factors  Tailored strategy based on an assessment of need  Multifaceted approach  Organisational commitment to evaluation and capacity building  Practical / experiential training  Ongoing technical support within the workplace

Review: Success factors  Systems that enable program monitoring and information management  Access to support tools and resources  Connecting and networking and sharing ideas  Leadership support and recognition

Strategies Internship Support materials Policy/ researcher exchanges Technical Assistance Involvement in Evaluation Training Communities of Practice - Networking Transfer of learning Diffusion into practice Source: Adaptation of Preskill and Boyle (2008) NSW Health Model of Research & Evaluation Capacity Building Mediators Mentoring Policies and Procedures Funding R& E Performance Monitoring Systems Leadership Strategic intent NSW Govt Eval Framework NSW State Health Plan Pop. Health R & E Strategy Strategies R&E Support Units Priority driven research Centres

Training  Critical appraisal and research synthesis  Program logic  Evaluation methods

In production:  Commissioning Economic Evaluations  Program Logic  Research Priority Setting  Publishing Health Research Evidence and evaluation guidance series

 Sustainable evaluation practice is built through multi-faceted strategies that address:  Individual knowledge and skills  Leadership and organisational commitment  Culture  Systems and processes  Communication Conclusion