Presentation is loading. Please wait.

Presentation is loading. Please wait.

Benedict Wauters 24-10-2012 ESF Com. TWG -Cyprus Introduction to Results based Management and to the Maastricht conference.

Similar presentations


Presentation on theme: "Benedict Wauters 24-10-2012 ESF Com. TWG -Cyprus Introduction to Results based Management and to the Maastricht conference."— Presentation transcript:

1 Benedict Wauters 24-10-2012 ESF Com. TWG -Cyprus Introduction to Results based Management and to the Maastricht conference

2 Overview 1.The COP RBM 2.Results based management: a brief history 3.So what? 4.Introducing the Maastricht conference.

3 Roots of the COP EQUAL programme: working group (led by the EQUAL unit) on introducing PCM (Project Cycle Management) in the ESF Partnership development Toolkit - September 2005 Conference on sound planning - November 2005 Sourcebook and website on sound planning - September 2006 (www.esfsourcebook.eu)www.esfsourcebook.eu Independent report “An Agenda for a Reformed Cohesion Policy” by F. Barca – May 2009

4 Who? Managing Authority of Czech Human Resources and Employment Operational Programme Czech National Coordination Authority of the NSRF Greek Managing Authority for “Health and welfare” OP and Intermediate Body for “Development of Human Resources” OP Lithuanian intermediate body for Operational Programme for the Development of Human Resources OP and Promotion of Cohesion OP Lithuanian Managing Authority Managing Authority for the Italian Region of Molise Regional OP Slovak ESF Managing Authority Swedish ESF Managing Authority Lithuanian ESF Agency Managing Authority for the Walloon Convergence and Regional Competitiveness and Employment Ops Managing Authority of the Flemish obj. 2 OP (coordinator of the COP) National support structure for the ESF for the Italian Regions (Tecnostruttura) European Institute for Public Administration Originally also: MA Italian Region of Piemonte and MA of Romania

5 Definition of results EC guidance document, November 2011*: The intended result is the specific dimension of well-being and progress for people that motivates policy action, i.e. what is intended to be changed. The notion of change also comprises changes in behaviour, social practices, institutions etc. Outputs are the direct products of programmes, they are intended to contribute to results. COP RBM: the essential difference between a result and an output (both as described above) is a shift of ownership from a producer to a user If it cannot be used, it is just an “output”: eg with a training it is the knowledge that can be used, not the actual training *MONITORING AND EVALUATION OF EUROPEAN COHESION POLICY - EUROPEAN REGIONAL DEVELOPMENT FUND AND COHESION FUND –CONCEPTS AND RECOMMENDATIONS.

6 The journey of the COP Warsaw Brussels (x2) Warsaw Vilnius Prague Rome Stockholm Athens Core of the COP work : 8 interactive seminars (of 2-3 days) with experts from around the world (incl. US, Canada, Australia) in debate with practitioners from our own programmes Between seminars: participating in global networks, going deeper into issues derived from the meetings Between seminars: participating in global networks, going deeper into issues derived from the meetings Key outputs: Definitions of Results RBM baseline study Practice descriptions Huge online knowledge base RBM strategic management and self-assessment system Key outputs: Definitions of Results RBM baseline study Practice descriptions Huge online knowledge base RBM strategic management and self-assessment system

7 Some key insights from the seminars-1 not clear what RBM really is, many tools and approaches seem relevant; no clear and generally accepted definition what the word “results” actually means! huge variety and selection of tools for mapping and analysing stakeholders when setting up projects but this tends to obscure the much more important question: how do you involve them and keep them involved? how do you know who are stakeholders in a project? Ask the target group how they run into in real life! bottom-up vs. top down deployment of an ESF OP is very different central to RBM is the “results chain” RBM at organisational level is quite different from RBM at project level; for organisations it is more like a a specific type of strategic planning and management system quality management (ISO) only helps “stabilise” and incrementally improve processes but does not itself provide a results orientation we are focused on changes that depend on long chains of cause and effect but tend not to look at what/who is/are in front of our noses (e.g. intermediariy organisations) Strategy maps / Balanced scorecards have potential for supporting RBM at organisational level and can in some way be linked to results chains

8 Some key insights from the seminars-1 not clear what RBM really is, many tools and approaches seem relevant; no clear and generally accepted definition what the word “results” actually means! huge variety and selection of tools for mapping and analysing stakeholders when setting up projects but this tends to obscure the much more important question: how do you involve them and keep them involved? how do you know who are stakeholders in a project? Ask the target group how they run into in real life! bottom-up vs. top down deployment of an ESF OP is very different central to RBM is the “results chain” RBM at organisational level is quite different from RBM at project level; for organisations it is more like a a specific type of strategic planning and management system quality management (ISO) only helps “stabilise” and incrementally improve processes but does not itself provide a results orientation we are focused on changes that depend on long chains of cause and effect but tend not to look at what/who is/are in front of our noses (e.g. intermediary organsations) Strategy maps / Balanced scorecards have potential for supporting RBM at organisational level and can in some way be linked to results chains RBM at organisational level is quite different from RBM at project level; for organisations it is more like a a specific type of strategic planning and management system We are focused on changes that depend on long chains of cause and effect but tend not to look at what/who is/are in front of our noses (e.g. intermediary organisations ) central to RBM is the “results chain”

9 Some key insights from the seminars -2 measurement via indicators and target setting on them tends to lead to perverse effects when they have direct consequences; they should only be used as flashlights to illuminate an area that raises questions to be strategic implies that choices have to be made and these should become visible when an organisation (e.g. a Managing Authority) sets objectives; a BSC/strategy map can help-bottom up logic and top down indicator setting for aggregation clash with each other an indicator that is just measuring quantity (amounts) without “quality” (how good) is useless accountability should be linked to learning, not to indicators linking money to outputs directly requires deep insight in the local situation and sophisticated processes for preparation and follow-up: it is not an easy fix human systems are all but stable and predictable. They are complex and adaptive. This has huge implications for planning and evaluation. counterfactual impact evaluation is to be used with great care as it assumes stable and sufficient cause and effect relations which rarely applies in practice longer “results chains” require different actions throughout the chain that assume an earlier action already had its effect importance of experimentation and adapation when faced with complexity; innovation is always complex

10 Some key insights from the seminars -2 measurement via indicators and target setting on them tends to lead to perverse effects when they have direct consequences; they should only be used as flashlights to illuminate an area that raises questions to be strategic implies that choices have to be made and these should become visible when an organisation (e.g. a Managing Authority) sets objectives; a BSC/strategy map can help-bottom up logic and top down indicator setting for aggregation clash with each other an indicator that is just measuring quantity (amounts) without “quality” (how good) is useless accountability should be linked to learning, not to indicators linking money to outputs directly requires deep insight in the local situation and sophisticated processes for preparation and follow-up: it is not an easy fix human systems are all but stable and predictable. They are complex and adaptive. This has huge implications for planning and evaluation. counterfactual impact evaluation is to be used with great care as it assumes stable and sufficient cause and effect relations which rarely applies in practice longer “results chains” require different actions throughout the chain that assume an earlier action already had its effect importance of experimentation and adapation when faced with complexity; innovation is always complex measurement via indicators and target setting on them tends to lead to perverse effects when they have direct consequences; they should only be used as flashlights to illuminate an area that raises questions accountability should be linked to learning, not to indicators human systems are all but stable and predictable. They are complex and adaptive. This has huge implications for planning and evaluation.

11 The COP RBM, to its surprise, found itself exposed to a huge variety of (sometimes conflicting) views… Who came up with this Results Based Management in the first place?

12 RBM is a child with many parents… MBO: management by objectives, launched by Peter Drucker in “the Practice of management” in 1954 It emphasised the learning aspect and discouraged mechanical models (of “scientific” management) Key is adaptation of empowered employees to changing circumstances and performance feed-back rather than control and predictability MBO’s current most popular form is through Kaplan and Norton’s strategy focused organisation and XPP (execution premium proces) which revolve around the Balanced Scorecard. LFA: logical framework approach launched by USAID in 1969. This developed into GOPP, OOPP and PCM*. (Vahamaki 2011) 12 *Goal oriented project planning, Objectives oriented project planning, Project cycle management

13 .. Some more parents Planning, programming and budgeting systems in the 60s that emphasised financial planning and cost accounting. This developed into “Programme management by activity” during the 70s and 80s (eg using Gantt charts, network analysis, etc.), emphasising further the planning of activities These streams were the antipode to Druckers’ MBO. Corporate performance management and Performance based budgeting attempted to combine both approaches (trying to cost results) in the 70-90s but were met with relatively little uptake Finally comes the New Public management Philosophy (as a loose collection of doctrines) in the 80s with…. what is now called RBM as its intended emanation in development aid (Vahamaki 2011) 13

14 New public management: a collection of principles Hood (1991) 14

15 Hood (1991) New public management: zooming in on two key principles 15

16 RBM: no singular model exists… …but there are some key aspects: This usually includes granting administrators the autonomy to use their resources in the most efficient and effective way. This generates learning. (Vahamaki 2011) 16

17 RBM: no singular model exists… …but there are some key aspects: This usually includes granting administrators the autonomy to use their resources in the most efficient and effective way. This generates learning. (Vahamaki 2011) An analytic and performance oriented approach 17

18 RBM: no singular model exists… …but there are some key aspects: This usually includes granting administrators the autonomy to use their resources in the most efficient and effective way. This generates learning. (Vahamaki 2011) Centred on end-user level development, only thereafter moving to identify what is necessary to get there 18

19 RBM: no singular model exists… …but there are some key aspects: This usually includes granting administrators the autonomy to use their resources in the most efficient and effective way. This generates learning. (Vahamaki 2011) Generate information and learn from it 19

20 RBM: no singular model exists… …but there are some key aspects: This usually includes granting administrators the autonomy to use their resources in the most efficient and effective way. This generates learning. (Vahamaki 2011) Influence decisions based on this information and learning 20

21 RBM: no singular model exists… …but there are some key aspects: This usually includes granting administrators the autonomy to use their resources in the most efficient and effective way. Key tool: the “results chain” This generates learning. (Vahamaki 2011) 21

22 RBM: no singular model exists… Action

23 RBM: no singular model exists… “Outcome” orientation is a key differentiator from basic New Public Management thinking which is more output oriented; it is the heritage of some of the “other” parents “Outcome” orientation is a key differentiator from basic New Public Management thinking which is more output oriented; it is the heritage of some of the “other” parents

24 RBM: no singular model exists… …but there are some key aspects: This usually includes granting administrators the autonomy to use their resources in the most efficient and effective way. This generates learning. (Vahamaki 2011) 24 It is intended to be an organisationwide approach, not just a tool for planning projects

25 So far the history of RBM… …but how is this relevant for ESF?

26 Question What is the single most important concern, that occupies public servants’ minds, implicitly or explicitly?

27 Answer? Accountability

28 How was this expressed recently? “The most evident weaknesses which indicate the need for reform of cohesion policy are: …Methodological and operational problems that have prevented both the appropriate use of indicators and targets – for which no comparable information is available - and a satisfactory analysis of “what works” in terms of policy impact. A remarkable lack of political and policy debate on results in terms of the well-being of people, at both local and EU level, most of the attention being focused on financial absorption and irregularities.” Independent report “An Agenda for a Reformed Cohesion Policy” delivered at the request of Commissioner for Regional Policy, Ms Hübner.

29 What does it mean for us tomorrow? We are faced with: Ever increasing audit burden essentially to check the materiality and regularity of expenditure (was the money really spent on the project, in compliance with a multitude of rules) A performance framework that attempts to capture a complex programme in a few abstract indicators (common or specific) and associated milestones and targets with consequences if missed Broad use of impact evaluation (just advancing is not good enough, it’s relative to NO action!) That’s the RBM, part right? How likely is it that this really leads to more “well-being of people?”

30 Imagine… Structural Funds- the movie, starring… CEO: EC and national/regional policy-makers Shareholders: EU taxpayers Employees: MAs, IBs,… Clients: target groups Suppliers: project promotors –the final beneficiaries EU 2020 = the objectives

31 Imagine… -2 A new CEO starts and sets out a clear objective for the organisation: in 7 years time, we will become the global market leader -a clear target! For this, a considerable investment budget was secured from the shareholders! Employees are enthusiastic about this bold goal! Clients and suppliers rejoice at the prospect of new products/services and business prospects! Over the next 7 years the CEO ensures the following actions are taken to reach the target: checking invoices were really paid according to procedures, checking that employees logged their activities in time sheets, checking the budget was spent according to the agreed deadlines, checking the checks to see whether there is less than 2% difference in finding errors… checking regularly whether the relative market position is improving according to plan…giving employees hell if it isn’t checking whether the data on market position has been correctly gathered…

32 Imagine… -2 A new CEO starts and sets out a clear objective for the organisation: in 7 years time, we will become the global market leader -a clear target! For this, a considerable investment budget was secured from the shareholders! Employees are enthusiastic about this bold goal! Clients and suppliers rejoice at the prospect of new products/services and business prospects! Over the next 7 years the CEO ensures the following actions are taken to reach the target: checking invoices were really paid according to procedures, checking that employees logged their activities in time sheets, checking the budget was spent according to the agreed deadlines, checking the checks to see whether there is less than 2% difference in finding errors… checking regularly whether the relative market position is improving according to plan…giving employees hell if it isn’t checking whether the data on market position has been correctly gathered… What are the odds that this company will really be nr 1 after 7 years?

33 Lehman reported record profits every year from 2005 to 2007. In 2007, the firm reported net income of a record $4.2 billion on revenue of $19.3 billion. In February 2007, the stock reached a record $86.18, giving Lehman a market capitalization of close to $60 billion. On Monday September 15, 2008, Lehman declared bankruptcy.

34 Let’s think again…. Accountability?

35 Accountability for what? 35

36 Hood (1991) 36

37 Accountability for what? 37

38 Hood (1991) 38

39 Accountability for what? 39

40 Hood (1991) 40

41 Typical expressions of accountability Lean and purposeful: 'just-in-time' inventory control systems payment-by-results systems … Honest and fair: recall systems for removing public officials from office by popular vote public tendering rules Independent anti-corruption investigatory bodies… Robust and resilient: redundancy, the maintenance of back-up systems to duplicate normal capacity; diversity, the maintenance of quite separate, self-standing units (to avoid 'common mode failure‘ e.g. groupthink) use of greater amounts of materials than ordinarily necessary… Adapted from Hood (1991) 41

42 Hood (1991) 42 Lean and purposeful = sole accountability for NPM

43 A broader view of RBM is called for Complexity theory: Harry Jones (ODI) Relationalism vs substantialism: Rosalind Eyben (IDS –Big push) Social change theory: Irene Guyt Cultural theory: Donald Curtis Management theory: Owen Barder (Centre for global development, EU branch) …

44 Keeping in mind new knowledge… New public management under fire: Separation of policy execution and policy development has led to policies that are impossible to execute Loosening of standards of operational management has led to cost increases Financing of agencies on the basis of output targets has led to loss of service quality and bureaucracy Budgeting on the basis of output targets has led to loss of control by Parliament and unreadable budget documentation Outsourcing of intermediate production to the market has led to decrease of service quality and higher costs

45 Keeping in mind new knowledge… Separation of policy execution and policy development has led to policies that are impossible to execute Loosening of standards of operational management has led to cost increases Financing of agencies on the basis of output targets has led to loss of service quality and bureaucracy Budgeting on the basis of output targets has led to loss of control by Parliament and unreadable budget documentation Outsourcing of intermediate production to the market has led to decrease of service quality and higher costs “..sanctions if output targets are not achieved …perverse incentive leads to manipulation of the data (artificially lowered targets…, choice of output indicators that are easy to meet but have nothing to do with the outcomes politicians are interested in, sometimes outright fraud with the numbers) and gaming behaviour (putting emphasis on activities that improve the data).“ “..sanctions if output targets are not achieved …perverse incentive leads to manipulation of the data (artificially lowered targets…, choice of output indicators that are easy to meet but have nothing to do with the outcomes politicians are interested in, sometimes outright fraud with the numbers) and gaming behaviour (putting emphasis on activities that improve the data).“

46 Keeping in mind new knowledge… Separation of policy execution and policy development has led to policies that are impossible to execute Loosening of standards of operational management has led to cost increases Financing of agencies on the basis of output targets has led to loss of service quality and bureaucracy Budgeting on the basis of output targets has led to loss of control by Parliament and unreadable budget documentation Outsourcing of intermediate production to the market has led to decrease of service quality and higher costs …No campaign to promote civil service values or public ethos can compensate for that, the less so if managers and professionals feel frustrated in their own right on how their performance is assessed. Everything we knew already about central planning in socialist states but now limited to the public sector of a market economy.“

47 Results based management (RBM) = Strategic management for the public sector Review and synthesis of tools and approaches from leading organisations in terms of public sector management, Public sector oriented sources New Zealand: Getting Better at Managing for Outcomes and Getting Better at Managing for Shared Outcomes Treasure Board of Canada: The Managing for Results Self-Assessment Tool World Bank: CAP-Scan Managing for Development Results Capacity Scan Asian Development Bank: Readiness Assessment Tool - Implementing a Results Focus in Organizations EIPA: CAF ODI (Overseas development Institute) etc.: integrating elements from complexity theory, … Mainly private sector but also attention to public sector: Palladium: Balanced Scorecard Hall of Fame award for Executing Strategy This led to the development of a RBM self-assessment framework and strategic management system 47 RBM originally aimed to return to Drucker’s Management by Objectives with emphasis on mind-set, essentially of feeling responsible for results rather than just compliant; this is the approach followed by the COP strategic management and self-assessment system

48 48 Results based management (RBM) = Strategic management for the public sector CAUTION: No system can replace a “community of inquirers” that takes the day to day experience in trying to make things work as a starting point of ongoing, joint reflection, figuring out what to do next! Systems do not make something work, people do! But some systems can make it harder to establish this community and others can support it. CAUTION: No system can replace a “community of inquirers” that takes the day to day experience in trying to make things work as a starting point of ongoing, joint reflection, figuring out what to do next! Systems do not make something work, people do! But some systems can make it harder to establish this community and others can support it.

49 49 RBM assessment framework Part 1: core RBM practices 1. Orientation of the PMO (programme management organisation) is clear 2. The PMO strategy is reflected in a strategic results framework 3. The strategy is translated into operations 4. Performance information is collected and supplied 5. Performance information is used 6. External stakeholders are involved in all aspects Part 2: enablers Part 3: the institutional context See http://www.coprbm.eu/?q=node/430http://www.coprbm.eu/?q=node/430

50 / evaluation demand plan 50 Kaplan and Norton, 2008

51 2. The PMO strategy is reflected in a strategic results framework Three levels of detail in the framework!

52

53 53 RBM assessment framework and ESF The ex ante evaluation should assess "the adequacy of human resources and administrative capacity for management of the programme", as well as "the suitability of the procedures for monitoring the programme and collecting the data necessary to carry out evaluations"(Article 48 (i) and (j) of the proposed structural funds regulations).

54 54 RBM assessment framework Part 1: core RBM practices 1. Orientation of the PMO (programme management organisation) is clear 2. The PMO strategy is reflected in a strategic results framework 3. The strategy is translated into operations 4. Performance information is collected and supplied 5. Performance information is used 6. External stakeholders are involved in all aspects Part 2: enablers Part 3: the institutional context See http://www.coprbm.eu/?q=node/430http://www.coprbm.eu/?q=node/430

55 From a Programme Management Organisation results chain… Starting point: PMO organisational VISION for change incl. how to add value!

56

57 57 Position of detailed results chains in the overall PMO strategy …to a programme action results chain

58 Adding value as an ESF PMO 1.Volume effects: ESF action 'adds' to existing action, either by supporting national action in general ('mirroring') or specific areas of national policy ('boosting'); 2.Scope effects: ESF action 'broadens' existing action by supporting groups or policy areas that would not otherwise receive support; 3.Role effects: ESF action supports local/regional innovations that are taken up at national level or national innovative actions that are then 'mainstreamed'; 4.Process effects: ESF action influences Member States administrations and organisations involved in the programmes.

59 Adding value as an ESF PMO 1.Volume effects: ESF action 'adds' to existing action, either by supporting national action in general ('mirroring') or specific areas of national policy ('boosting'); 2.Scope effects: ESF action 'broadens' existing action by supporting groups or policy areas that would not otherwise receive support; 3.Role effects: ESF action supports local/regional innovations that are taken up at national level or national innovative actions that are then 'mainstreamed'; 4.Process effects: ESF action influences Member States administrations and organisations involved in the programmes. An enhancer supports in a highly efficient way delivery partners to produce (a higher volume) of already existing products/services to satisfy existing needs of constituents. This support is being continuously enhanced as are the supported products and services.

60 Adding value as an ESF PMO 1.Volume effects: ESF action 'adds' to existing action, either by supporting national action in general ('mirroring') or specific areas of national policy ('boosting'); 2.Scope effects: ESF action 'broadens' existing action by supporting groups or policy areas that would not otherwise receive support; 3.Role effects: ESF action supports local/regional innovations that are taken up at national level or national innovative actions that are then 'mainstreamed'; 4.Process effects: ESF action influences Member States administrations and organisations involved in the programmes. An innovator supports delivery and other partners to continuously (re)develop new products and services for new constituent needs and deploy them until they become stable and mainstream.

61 Adding value as an ESF PMO 1.Volume effects: ESF action 'adds' to existing action, either by supporting national action in general ('mirroring') or specific areas of national policy ('boosting'); 2.Scope effects: ESF action 'broadens' existing action by supporting groups or policy areas that would not otherwise receive support; 3.Role effects: ESF action supports local/regional innovations that are taken up at national level or national innovative actions that are then 'mainstreamed'; 4.Process effects: ESF action influences Member States administrations and organisations involved in the programmes. For a solutions manager the focus is NOT on specific products and services but on developing detailed knowledge concerning specific challenges (a limited number of) delivery partners (the customers) are facing regarding their constituents and work closely with them to solve these challenges.

62 The journey of the COP: taking stock Warsaw Brussels Warsaw Vilnius Prague Rome Stockholm Athens Maastricht 5-6 November 2014 and beyond: how to ensure delivery of better and more results by the European Social Fund?

63

64

65 The journey goes on after Maastricht… “evidence suggests at least four to five years of consistent effort – and many organizations have been working towards it for much longer. The problem is that key people move on, governance structures change, priorities shift. A lot of relearning has taken place in the history of results-based management. And four to five years is just the timeline to get up and running. Good management requires constant attention. To quote a cliché, integrating performance information into managing and budgeting is a journey, not a destination.” Source: John Mayne, Challenges and Lessons in Implementing Results-Based Management, 2007, Evaluation

66 The journey goes on after Maastricht… “evidence suggests at least four to five years of consistent effort – and many organizations have been working towards it for much longer. The problem is that key people move on, governance structures change, priorities shift. A lot of relearning has taken place in the history of results-based management. And four to five years is just the timeline to get up and running. Good management requires constant attention. To quote a cliché, integrating performance information into managing and budgeting is a journey, not a destination.” Source: John Mayne, Challenges and Lessons in Implementing Results-Based Management, 2007, Evaluation New COP RBM proposal –with a focus on supporting the use of the developed system- has been submitted to DG EMPL with new partners joining

67 Thank you for listening! Questions? More info: www.coprbm.eu or benedict.wauters@esf.vlaanderen.bewww.coprbm.eu benedict.wauters@esf.vlaanderen.be


Download ppt "Benedict Wauters 24-10-2012 ESF Com. TWG -Cyprus Introduction to Results based Management and to the Maastricht conference."

Similar presentations


Ads by Google