Challenges & Strategies for Success Challenges & Strategies for Success Lewe Atkinson Manager, Evaluation and program improvement Evaluator as change agent

Summary What is MLA? Rationale for evaluation at MLA What is the MLA evaluation framework? Strategic framework for change Major challenges for change Successful change tactics Key points When will we know that we have been successful?

What is MLA? Industry-owned company Formed in 1998 Australian red meat industry Marketing, Market access & R&D H/O in Sydney others interstate 7 offices OS (US, Kor, Jap, China, Brussels, Russia, & Bahrain) 241 staff Expenditure about $160M/yr

Rationale for evaluation at MLA Effectiveness = doing the right things Efficiency = doing them well Benefit = doing the right things well Commonwealth stakeholders = matching $ and alignment with NRRPs Industry stakeholders = MLA accountability for levy income

Framework for independent program evaluation

What is the MLA evaluation framework ? Based on the DOFA framework. Input/output/outcome program logic framework. Reports alignment with Rural Research and Development Priorities. Consistent with the Council of Rural Research and Development Corporation “Guidelines for Evaluation”. Three major MLA programs undergo ex-post evaluation each year.

Strategic framework for change People-focused program logic: Jess Dart Brisbane MLA goal = build stakeholder confidence in MLA. 2. End-users = politicians, industry peak councils, and MLA board. 3. Next-users = MLA Staff: MD, EC, & PM’s. 4. We will be successful when; – Evaluations are meaningful to end-users – MLA staff manage for outcomes – Evaluation is core to MLA values

Major challenges for change 1. Making the complex simple 2. Embedding a focus on outcomes 3. Maintaining independence & consistency across programs 4. Building evaluation capability 5. Developing a learning culture

Successful change tactics 1. Alignment with MLA values, vision, and strategy 2. Engaging key stakeholders in setting better outcome KPIs 3. Engaging the Executive Committee 4. Creating cross functional teams, learning by doing & then reflecting 5. Creating layers of independence & transparency

Asking key evaluation questions Linking evaluation to strategy 1.Making the complex simple: Alignment with MLA values, vision, and strategy

What are the key evaluation questions…?

1.Are MLA programs adding value? 2.Are our collaborations efficient and effective? 3.Do we facilitate innovation from concept through to commercialisation? 4.Do we facilitate increased industry capability and investment in innovation? 5.Are we identifying and effectively managing industry risks?

Linking evaluation to strategy?

MLA strategic plan and evaluation

Program evaluation strategy and schedule

Link to MLA strategy

2.Embedding a focus on outcomes: Outcome KPI setting workshops across MLA & with key stakeholders Linking outcomes to employee performance measures Engaging key stakeholders in setting better outcome KPIs

Program evaluation is just the tip of the iceberg

Program logic encourages critical thinking Objectives Inputs Outputs Outcomes Market returns Benefit cost ratio Triple bottom line KPIs SocialEnvironmental Integrated framework

3.Maintaining independence & consistency across programs: Executive learning on the job – Sign-off on the MLA Evaluation Strategy Engaging the Executive Committee

Executive learning on the job 1. Annually determines a minimum of three major MLA programs to undergo an ex-post evaluation. 2. Must approve each program evaluation proposal prior to project commencement; – Scope – Methodology – Team – Budget – Timelines 3. Must approve each brochure in the evaluation series prior to publication.

4.Building evaluation capability: Creating an evaluation-focused organisational culture Creating cross functional teams, learning by doing & then reflecting

Living & learning MLA values workshop series

Monthly review of ex-post program evaluation series Last Friday of each month Theme based on one of MLA’s values and an expert presentation on a related evaluation topic Panel discussion: Economic benefit analysis reinforced by real world case studies Reflection on lessons learned to encourage a culture of continuous improvement

5.Developing a learning culture: Transparency – access via MLA website MLA contribution to RDC aggregate results Collaborating with evaluation experts to address the challenges for measuring MLA’s value add Creating layers of independence & transparency

Transparency – access via MLA website What will you find? – A message from the Chairman – Downloadable PDF version of annual evaluation schedule – Downloadable PDF overview of MLA evaluation framework – Downloadable PDF for each brochure in impact series – Archive of all source documents and references sighted in brochure series

MLA contribution to RDC aggregate results INPUT FROM MLA EVALUATION FRAMEWORK JOINT INPUT INTO COLLABORATIVE RDC PROGRAMS

Challenges for measuring MLA’s value add Credible counterfactual case – independent verification of MLA’s value add Realistic attribution – only outcomes under control of MLA Monitoring adoption – meaningful outcome KPIs, other indicators identified early and monitored Increasing program complexity – requires consistent application of program logic

Independent Consistent Transparent Corporate learning Continuous improvement Key points

When will we know that we have been successful? Evaluations are meaningful to end- users MLA staff manage for outcomes Evaluation is core to MLA values

Questions