3. Set strategic priorities for action The top levels of you model will show your vision and mission Work out the priority areas you’ll attempt to change in the next planning period If you’re just doing monitoring and evaluation, you may not do this step at the start Example: The strategic focus has been identified for organizing future parties (the darker the blue, the higher the priority)
4. Put indicators onto your model Indicators are routinely collected measures of outcomes or steps You may or may not be collecting these indicators at the moment Use the picture you now have of which indicators you are already (or are planning to) collect to think about your indicator coverage Example: Potential indicators mapped onto the outcomes model
5. List indicator projects List any indicator projects you want to undertake These may be projects only you work on or ones you do with other stakeholders who also want to measure the same indicators Example: Under each indicator project there is a list of the indicators which will be developed
6. Put evaluation questions onto your model Put possible evaluation questions next to the outcomes or steps they apply to Put both outcome evaluation questions (proving that high level outcomes have been changed by the program and non-outcome evaluation questions (asking about how to improve the program or about how the process works) Don’t worry whether or not you can answer these evaluation questions at this stage Example: Evaluation questions mapped onto the outcomes model
7. Decide which evaluation questions to try to answer Don’t just assume that you have to, or will be able to, answer high level outcome evaluation questions. If you can that’s good, but often it would be better for your funding or controlling organization to undertake such outcome evaluation (e.g. by intensively evaluating a pilot program rather than demanding all programs have high level outcome evaluation) Example: Evaluation question analysis looking at which evaluation questions you’re going to attempt to answer. The evaluation question ‘Did the party make the guests happy?’ will be examined in more detail in the next step.
8a. Work out what outcome evaluation is possible There’s only a limited range of possible outcome evaluation designs and Easy Outcomes currently divides these into seven possible types Work out whether any of these are possible for your high level evaluation question(s) (if you don’t know how to do this, involve someone else who does) (more information: www.systematicoutcomesanalysis.org/oiiwa/model/designs.html) Example: Analysis of the appropriateness, feasibility and affordability of the first three possible high level outcome evaluation designs to answer the evaluation question ‘Did the party make the guests happy?’
8b. Work out what outcome evaluation is possible (cont.) Note that only some of the possible high level outcome evaluation designs will give you information needed in later stages of Easy Outcomes. In particular, certain types of economic analysis can only be done if there is a quantitative measures of the effect of your activity on high level outcomes (only the first four designs have the potential to provide this) Example: Appropriateness, feasibility and affordability of last four high level outcome evaluation designs
9. List evaluation projects Set out your evaluation projects and list under them the evaluation questions they’ll be answering This helps you to be very clear about exactly which evaluation question(s) are being answered by a particular evaluation project (very helpful in large scale evaluations) Example: Evaluation projects and the evaluation questions they will attempt to answer
10. Work out what economic evaluation to do There are only a limited range of possible economic evaluation designs and Easy Outcomes currently divides these into ten possible types Work out whether any of these are possible (if you don’t know how to do this, involve someone else who does) (more information: www.systematicoutcomesanalysis.org/oiiwa/model/economic.html) Example: Two examples of types of economic analysis which could be done in this case.
11. Work out overall evaluation scheme You can chose between two major overall evaluation schemes: 1. Doing outcome evaluation on the whole program roll-out 2. Just doing outcome evaluation on a pilot and then making sure that the full program roll-out is applying best practice learnt in the pilot (but not doing outcome evaluation on the full roll-out) Example: Shows the two possible major overall evaluation schemes for the party example
12. Work out delegated or contracted accountabilities There are three possible types of contracting in Easy Outcomes: 1. Contracting for outputs 2. Contracting for outputs and ‘Managing for outcomes’ * 3. Contracting for not fully controllable outcomes * It’s very important to be clear about which kind of contracting you’re doing. Example: The waiting staff are contracted to be accountable for indicators related to the food and drink being adequately distributed (outputs) but not further up the outcomes model * Type 2. does not involve being accountable for high level outcomes. Type 3 occurs, for example, in the private sector when a CEOs gets a bonus related to the organization’s share price.
Conclusion: Easy Outcomes lets you See what evaluation and monitoring is planned (outcome, non- outcome & economic evaluation) and make decisions about what should be done See if monitoring indicators are measuring the important (not just the measurable) See what’s appropriate, feasible and affordable for high level outcome evaluation Link monitoring and evaluation to strategic planning Be clear about who is accountable for what in delegation or contracting
Origin of Easy Outcomes Developed by an expert evaluator & strategist – Dr Paul Duignan User friendly version of Systematic Outcomes Analysis – a comprehensive applied outcomes, evaluation & strategy approach (www.systematicoutcomesanalysis.org)www.systematicoutcomesanalysis.org Implemented using DoView – outcomes visualization software (www.doview.com)www.doview.com We’re interested in hearing where Easy Outcomes is being applied and answering any questions you have This material is covered by Creative Commons copyright – you can use it for most commercial or non-commercial purposes just with acknowledgement (check web site for further information)