Download presentation

Presentation is loading. Please wait.

Published byKian Malone Modified over 2 years ago

1
Creative Commons © 2007 Presentations in this powerpoint file This file contains three presentations on Easy Outcomes 1. Easy Outcomes: Introductory Presentation (P1) 2. Easy Outcomes: Summary for Decision Makers (P2) 3. Easy Outcomes: Guidelines for Drawing Outcomes Models (P3)

2
Creative Commons © 2007 Presentation 1 Easy Outcomes Introductory Presentation (P1)

3
Creative Commons © 2007 Easy outcomes Introductory Presentation This presentation is available from P1 V1-0

4
Creative Commons © 2007 Origin of Easy Outcomes Developed by an expert evaluator & strategist – Dr Paul Duignan User friendly version of Systematic Outcomes Analysis – a comprehensive applied outcomes, evaluation & strategy approach (www.systematicoutcomesanalysis.org)www.systematicoutcomesanalysis.org Implemented using DoView – outcomes visualization software (www.doview.com)www.doview.com We’re interested in hearing where Easy Outcomes is being applied and answering any questions you have This material is covered by Creative Commons copyright – you can use it for most commercial or non-commercial purposes just with acknowledgement (check web site for further information)

5
Creative Commons © 2007 Easy outcomes lets you Work out your outcomes, the steps needed to get to them & to prioritize your activity Find out which outcomes you can and can’t measure and track them Figure out which evaluation questions to ask & if you can or can’t answer them Decide who should be accountable for what in delegation or contracting

6
Creative Commons © 2007 How is Easy Outcomes done? A template in DoView outcomes software is filled out * It’s based around drawing an outcomes model (also called a program logic, results chain, strategy map, ways-means diagram) using a set of guidelines Once the outcomes model has been drawn, you put onto it indicators, evaluation questions etc. You the analyse what’s appropriate, feasible and affordable (e.g. for outcomes evaluation you use the 7 possible types of outcome evaluation identified in Easy Outcomes) Once the work is done you are easily able to quickly overview the whole monitoring and evaluation planning * If one wanted to, Easy Outcomes can be implemented using standard office software but it would take longer to do so.

7
Creative Commons © 2007 The process 1.Getting started 2.Draw an outcomes model 3.Set strategic priorities for action (if doing strategic planning) 4.Put indicators onto your model 5.List indicator projects 6.Put evaluation questions onto your model 7.Decide which evaluation questions to try to answer 8.Work out what outcome evaluation is possible 9.List evaluation projects 10.Work out what economic evaluation to do 11.Work out overall evaluation scheme 12.Work out delegated or contracting accountabilities

8
Creative Commons © Getting started There’s only one step absolutely required in Easy Outcomes – drawing an outcomes model You can do all or some of the other steps and sometimes people do them in a different order A DoView template file for doing the process is available at You can get a trial version of DoView from

9
Creative Commons © Drawing an outcomes model Draw your model using the Easy Outcomes guidelines for outcomes models (www.easyoutcomes.org/guidelines/outcomesquidelines.html) At this stage don’t worry if you can’t measure some of the steps or outcomes Also, don’t worry if you can’t prove that you changed some of the steps or outcomes Example: an outcomes model for a social event (a party) (usually there would be more than one diagram in an outcomes model)

10
Creative Commons © 2007

11
3. Set strategic priorities for action The top levels of you model will show your vision and mission Work out the priority areas you’ll attempt to change in the next planning period If you’re just doing monitoring and evaluation, you may not do this step at the start Example: The strategic focus has been identified for organizing future parties (the darker the blue, the higher the priority)

12
Creative Commons © 2007

13
4. Put indicators onto your model Indicators are routinely collected measures of outcomes or steps You may or may not be collecting these indicators at the moment Use the picture you now have of which indicators you are already (or are planning to) collect to think about your indicator coverage Example: Potential indicators mapped onto the outcomes model

14
Creative Commons © 2007

15
5. List indicator projects List any indicator projects you want to undertake These may be projects only you work on or ones you do with other stakeholders who also want to measure the same indicators Example: Under each indicator project there is a list of the indicators which will be developed

16
Creative Commons © 2007

17
6. Put evaluation questions onto your model Put possible evaluation questions next to the outcomes or steps they apply to Put both outcome evaluation questions (proving that high level outcomes have been changed by the program and non-outcome evaluation questions (asking about how to improve the program or about how the process works) Don’t worry whether or not you can answer these evaluation questions at this stage Example: Evaluation questions mapped onto the outcomes model

18
Creative Commons © 2007

19
7. Decide which evaluation questions to try to answer Don’t just assume that you have to, or will be able to, answer high level outcome evaluation questions. If you can that’s good, but often it would be better for your funding or controlling organization to undertake such outcome evaluation (e.g. by intensively evaluating a pilot program rather than demanding all programs have high level outcome evaluation) Example: Evaluation question analysis looking at which evaluation questions you’re going to attempt to answer. The evaluation question ‘Did the party make the guests happy?’ will be examined in more detail in the next step.

20
Creative Commons © 2007

21
8a. Work out what outcome evaluation is possible There’s only a limited range of possible outcome evaluation designs and Easy Outcomes currently divides these into seven possible types Work out whether any of these are possible for your high level evaluation question(s) (if you don’t know how to do this, involve someone else who does) (more information: Example: Analysis of the appropriateness, feasibility and affordability of the first three possible high level outcome evaluation designs to answer the evaluation question ‘Did the party make the guests happy?’

22
Creative Commons © 2007

23
8b. Work out what outcome evaluation is possible (cont.) Note that only some of the possible high level outcome evaluation designs will give you information needed in later stages of Easy Outcomes. In particular, certain types of economic analysis can only be done if there is a quantitative measures of the effect of your activity on high level outcomes (only the first four designs have the potential to provide this) Example: Appropriateness, feasibility and affordability of last four high level outcome evaluation designs

24
Creative Commons © 2007

25
9. List evaluation projects Set out your evaluation projects and list under them the evaluation questions they’ll be answering This helps you to be very clear about exactly which evaluation question(s) are being answered by a particular evaluation project (very helpful in large scale evaluations) Example: Evaluation projects and the evaluation questions they will attempt to answer

26
Creative Commons © 2007

27
10. Work out what economic evaluation to do There are only a limited range of possible economic evaluation designs and Easy Outcomes currently divides these into ten possible types Work out whether any of these are possible (if you don’t know how to do this, involve someone else who does) (more information: Example: Two examples of types of economic analysis which could be done in this case.

28
Creative Commons © 2007

29
11. Work out overall evaluation scheme You can chose between two major overall evaluation schemes: 1. Doing outcome evaluation on the whole program roll-out 2. Just doing outcome evaluation on a pilot and then making sure that the full program roll-out is applying best practice learnt in the pilot (but not doing outcome evaluation on the full roll-out) Example: Shows the two possible major overall evaluation schemes for the party example

30
Creative Commons © 2007

31
12. Work out delegated or contracted accountabilities There are three possible types of contracting in Easy Outcomes: 1. Contracting for outputs 2. Contracting for outputs and ‘Managing for outcomes’ * 3. Contracting for not fully controllable outcomes * It’s very important to be clear about which kind of contracting you’re doing. Example: The waiting staff are contracted to be accountable for indicators related to the food and drink being adequately distributed (outputs) but not further up the outcomes model * Type 2. does not involve being accountable for high level outcomes. Type 3 occurs, for example, in the private sector when a CEOs gets a bonus related to the organization’s share price.

32
Creative Commons © 2007

33
Conclusion: Easy Outcomes lets you See what evaluation and monitoring is planned (outcome, non- outcome & economic evaluation) and make decisions about what should be done See if monitoring indicators are measuring the important (not just the measurable) See what’s appropriate, feasible and affordable for high level outcome evaluation Link monitoring and evaluation to strategic planning Be clear about who is accountable for what in delegation or contracting

34
Creative Commons © 2007 Further information Further information available from: More detailed information available from Information on DoView outcomes software from Comments to:

35
Creative Commons © 2007 Presentation 2 Easy Outcomes: Summary for Decision Makers (P2)

36
Creative Commons © 2007 Easy outcomes Summary for decision makers (Representatives, Parliamentarians & Boards, CEOs & Managers) This presentation is available from P2 V1-0

37
Creative Commons © 2007 The decision maker’s problem Having to make decisions in a world of uncertainty Wanting more and more robust evaluation and monitoring information for guidance Knowing that evaluation and monitoring are costly and when linked to accountability may have unintended consequences No quick way of making decisions about which evaluation and monitoring to do Being unclear about how much uncertainty there is in evaluation & monitoring information they’re given Lack of effective organizational linkages between evaluation & monitoring, strategy development and delegation & contracting

38
Creative Commons © 2007 The risk Decision makers demand more and more evaluation and monitoring They’re given evaluation and monitoring information but have no real idea of the actual level of uncertainty in that information They end up doing the easily measurable (but often badly measured) rather than the strategically important And they have no way of deciding on the trade-off between spending more and more money & effort (including organizational preoccupation) on monitoring and evaluation rather than on program delivery

39
Creative Commons © 2007 Examples of mistakes Demanding only certain types of evaluation design be used (e.g. experimental outcome evaluation) regardless of the appropriateness, feasibility or affordability of these designs in particular cases Demanding all programs be evaluated and then getting weak evaluations of many programs rather than robust evaluations of a selected few which could be done if evaluation resources and effort were more focused Insisting that players ‘contract for outcomes’ or be ‘accountable for outcomes’ and exposing both contractors and contractees to risks in situations where it is difficult to actually attribute change to a particular player

40
Creative Commons © 2007 A solution – Easy Outcomes lets decision makers See what evaluation and monitoring is planned (outcome, non- outcome & economic evaluation) and make decisions about what should be done See if monitoring indicators are measuring the important (not just the measurable) See what’s appropriate, feasible and affordable for high level outcome evaluation Link monitoring and evaluation to strategic planning Be clear about who is accountable for what in delegation or contracting

41
Creative Commons © 2007 It can also show that providers are ‘Managing for outcomes” Easy Outcomes also lets decision makers quickly assess whether a provider is ‘Managing for outcomes’ A peer reviewed Easy Outcomes analysis gives a decision maker some assurance that the provider is attempting to Manage for outcomes* * Managing for outcomes is different from being held accountable for achieving not fully controllable outcomes

42
Creative Commons © 2007 It provides a standard format Think of Easy Outcomes as a standard format for setting out outcomes, monitoring and evaluation planning This is particularly useful where a number of programs are reporting to a decision maker and means the decision maker or his or her staff do not have to grapple with many different reporting formats Not having a standard format for reporting on outcomes and evaluation planning is a bit like not having a standard format for reporting accounting information – likely to lead to a great deal of additional work as you have to come to terms with different frameworks

43
Creative Commons © 2007 How is Easy Outcomes done? A template in DoView outcomes software is filled out * It’s based around drawing an outcomes model (also called a program logic, results chain, strategy map, ways-means diagram) using a set of guidelines Once the outcomes model has been drawn, you put onto it indicators, evaluation questions etc. You do some analysis of what’s appropriate, feasible and affordable (e.g. for outcomes evaluation you use the 7 possible types of outcome evaluation identified in Easy Outcomes) Once the work is done you are easily able to quickly overview the whole monitoring and evaluation planning * If one wanted to, Easy Outcomes can be implemented using standard office software but it would take longer to do so.

44
Creative Commons © 2007 Since strategic advantage often lies in difficult to measure areas… Strategic advantage in the public, private and community sectors often lies in ‘hard to measure’ areas Well measured areas where causation has been established through monitoring and evaluation are usually full of players and activity In the private sector this makes them commodity markets, in the public & community sectors they’re the places where much activity is already being focused

45
Creative Commons © 2007 …It’s important that you get better control of evaluation, monitoring and outcomes planning An unsophisticated ‘go forth and evaluate’ or ‘just do outcomes evaluation’ approach will not work (you’ll end up doing the measurable, not necessarily the strategic) The Easy Outcomes approach means that you and your staff will know what is known, what is not yet known and what can be known to aid decision making If you’re going to get optimal strategic intelligence for decision making you need to make the decisions about how to deploy your limited monitoring and evaluation resources

46
Creative Commons © 2007 Put the outcomes models at the heart of your organization The same Easy Outcomes model that has been built for monitoring and evaluation can also be used for: –Defining your vision and mission –Priority setting in strategic planning –Identifying the outcomes of your organizational units –Organizing the evidence base around your interventions –Talking to outside stakeholders about what you’re doing –Delegating and contracting

47
Creative Commons © 2007 Example of an Easy Outcomes analysis The following diagrams show a simple Easy Outcomes analysis using the example of a social function (party) This example is discussed in more detail in the Easy Outcomes Introductory Presentation* and on the Easy Outcomes website * Available from the Easy Outcomes website

48
Creative Commons © 2007 The process Draw an outcomes model Set strategic priorities for action Put indicators onto your model List indicator projects Put evaluation questions onto your model Decide which evaluation questions to try to answer Work out what outcome evaluation is possible List evaluation projects Work out what economic evaluation to do Work out overall evaluation scheme Work out delegated or contracting accountabilities

49
Creative Commons © 2007 Example: An Easy Outcomes Analysis for a social event (party)

50
Creative Commons © 2007

51

52

53

54

55

56

57

58

59

60

61

62
Origin of Easy Outcomes Developed by an expert evaluator & strategist – Dr Paul Duignan User friendly version of Systematic Outcomes Analysis – a comprehensive applied outcomes, evaluation & strategy approach (www.systematicoutcomesanalysis.org)www.systematicoutcomesanalysis.org Implemented using DoView – outcomes visualization software (www.doview.com)www.doview.com We’re interested in hearing where Easy Outcomes is being applied and answering any questions you have This material is covered by Creative Commons copyright – you can use it for most commercial or non-commercial purposes just with acknowledgement (check web site for further information)

63
Creative Commons © 2007 Further information Further information available from: More detailed information available from Information on DoView outcomes software from Comments to:

64
Creative Commons © 2007 Presentation 3 Easy Outcomes: Guidelines for Drawing Outcomes Models for Easy Outcomes (P3)

65
Creative Commons © 2007 Easy outcomes Guidelines for drawing outcomes models for Easy Outcomes This presentation is available from P3 V1-0

66
Creative Commons © 2007 Guidelines for drawing outcomes models Easy Outcomes provides guidelines for drawing outcomes models (http://www.easyoutcomes.org/guidelines/outcomesguidelines.html) Using the guidelines means that the outcomes models will be able to be used for all steps in the Easy Outcomes process In addition, external stakeholders usually find Easy Outcomes models very accessible and useful

67
Creative Commons © 2007 The guidelines 1.Use outcomes not activities 2.Draw the ‘cascading set of causes in the real world’ 3.Don’t force models into horizontal levels 4.Don’t make ‘siloed’ outcomes models 5.Singular not composite steps and outcomes 6.Keep outcomes short 7.Put outcomes into an hierarchical order 8.Include all relevant steps needed to cause an outcome 9.Keep measurement separate from steps and outcomes 10.Put a ‘value’ in front of steps and outcomes 11.Make just the right number of ‘slices’ 12.An organization does not necessarily need a single high level outcome 13.Include both current and future priorities

68
Creative Commons © Outcomes not activities Activities can be changed into outcomes just by changing the way they are written.

69
Creative Commons © Cascading set of causes in the real world Draw all of the steps (causes) leading to high level outcomes in the outside world. Drill down as far as you like. Draw the full outcomes model as on the left, don’t limit yourself just to measurable outcomes (middle model) or those that it can be proved have been changed by a player (attributable outcomes – as in the right hand model where only outcomes which it can be proved were caused by a player have been included in the model).

70
Creative Commons © Don’t force models into horizontal levels Different outcomes models will have more or less steps at a particular level, don’t force steps into levels or restrict the number at a particular level (as has been done in these diagrams). Outputs can be identified on your model but they don’t have to lie at a particular level.

71
Creative Commons © Don’t make ‘siloed’ outcomes models In the ‘siloed’ model on the left, lower level steps are only allowed to cause a single high level outcome. In the real world, a lower level step may well cause more than one high level outcome (as in the model on the right shows where the blue step causes both the blue and the yellow higher level outcome).

72
Creative Commons © Singular not composite outcomes Make sure that outcomes are not ‘composite’ like the one on the left hand side which contains both an ‘end’ (improve father-son relationships) and a ‘means’ (teaching them to fish together). Composite outcomes lock you permanently into using one strategy to get an outcome. If by, through, to, in order to or similar terms appear in an outcome they show that it is a composite outcome. Divide the composite outcome into two or more outcomes and open up the possibility that there is another way (strategy) of getting to the outcome (e.g. the blue box in the diagram on the right).

73
Creative Commons © Keep outcomes and steps short Keep outcomes and steps short, as in the one on the right rather than the one on the left. If you need to, keep an additional longer description of the outcome separately. For every additional word you use in an outcome name, you will lose or confuse some readers.

74
Creative Commons © Put outcomes into an hierarchical order The outcomes on the left are not in an hierarchical order. The ones on the right are. You can tell that there is an hierarchical order to the outcomes on the right because if you could magically achieve the top one, you would not bother intervening to do the lower ones.

75
Creative Commons © Include all relevant steps needed to cause an outcome or step The model on the left is incomplete because it does not include an essential step (the red one in the model on the right) which is needed to cause the outcome above to occur. Include all notable steps needed. You need to use your common sense, because there will be steps which are needed but which can be taken for granted in some situations (e.g. electricity, telephone systems) you don’t need to include such ‘taken for granted’steps where it can be assumed they will be present.

76
Creative Commons © Keep measurement separate from steps & outcomes First draw your outcomes model without worrying about whether you can measure the steps and outcomes (as in the model on the left). Then insert measurements as indicators (as in the model on the right). If you put measurements into your actual outcomes model you cannot deal with the fact that particular ways of measuring an outcome may be inaccurate. In a few cases, where measurement causes something to happen (e.g. audit processes), they may also appear as steps in their own right.

77
Creative Commons © Put a ‘value’ in front of steps & outcomes Put a ‘value’ such as successful, sufficient, effective or accurate in front of your outcomes and steps. You don't need to define this when you are first building your outcomes model. By putting this value in you often limit long discussions about the exact amount of the step or outcome that is needed. If you take too much time up with such discussions when first drawing your model in a workshop situation you’re likely to run out of time). The question of how much is sufficient etc can then become the subject of an evaluation project if necessary.

78
Creative Commons © Make just the right number of ‘slices’ You model should be made up of just the right number of ‘slices’ needed to represent the world of outcomes and steps in which you are intervening. The actual world of causes is a multi- dimensional world which you need to somehow represent in a finite set of dimensions. Your challenge is to ‘slice’ that multi-dimensional world up into a set of slices which make sense to you and your stakeholders. For instance below you can see a set of three slices (there would be a set of outcomes below each of these). Each one is based on a different conceptual level: national, locality and individual level. It may take some time to work out the best set of slices to use in a particular model. Just try developing different slices as they occur to you and then as you progress in drawing your model, get rid of duplicate steps and outcomes and join or separate them as you think appropriate.

79
Creative Commons © An organization does not necessarily need a single high level outcome Where an organization has been delegated or contracted to intervene in more than one area, it’s appropriate for it to have separate outcomes models for each area. On the left below is an artificial attempt at a single outcomes model. Having separate outcomes models for each area (as on the right) means that the organization is in a better position to collaborate with outside organizations working in the same area. For instance, the two green outcomes on the extreme right (which are the focus of another organization) can be much more easily integrated into the outcomes model on this topic when it has been set up as in the right hand rather than the left hand model.

80
Creative Commons © Include both current and future priorities Your outcomes models should include all relevant causes in the real world and should not be limited just to your current priorities (determined by resourcing or other constraints). The model on the right is limited to only priorities and so does not give a full strategic view of the world in which you’re intervening. Highlighting priorities onto a full outcomes model (as in the middle example) is a better approach and works well in the public sector where public sector employees can give ‘free and frank’ advice about the world as it is and elected officials can then determine current priorities on this model.

81
Creative Commons © 2007 Conclusion Drawing outcomes models using the guidelines makes sure that they will work with all of the steps in Easy Outcomes. More information on the outcomes model standards is available If you have any comment on these standards please let us know:

82
Creative Commons © 2007 Further information Further information available from: More detailed information available from Information on DoView outcomes software from Comments to:

Similar presentations

© 2016 SlidePlayer.com Inc.

All rights reserved.

Ads by Google