1. Core practices in RBM Benedict Wauters

2. Rember these limitations? The following cannot be handled solely by line workers: What external threats and opportunities to deal with, which customers and which needs to target, what capacity to build given longer term estimates of demand: this is strategic management Designing new products/services, with new technology to satisfy new needs and customers: this is innovation management Building structural cooperative relations with which suppliers and other stakeholders: this is stakeholder management As stated, these processes are NOT disconnected from current operations but current operations have a focus on what is… current. 2

3. 3 Organisations need to also take longer term perspectives for the organisation as a whole to capture emerging opportunies that could otherwise be missed and deal with threats that may otherwise catch them by surprise… Preparing for the future triggers innovation and stakeholder management processes. All of this impacts what future operations will be like.

4. 4 H. Mintzberg = influenced by what we learn from acting incl. in daily operations = why there needs to be room for contingencies in the budget (reserve capacity) STRATEGY IMPLEMENTATION

5. Structure of this presentation 5 1.Achieving balance between learning and control with management systems 2.How we translate this in the RBM system for PMOs

6. Boundary systems Balancing learning and control Strategy (plan) Internal control Core values, mission, vision Critical performance variables Strategic uncertainties Risks to be avoided Interactive control systems Diagnostic control systems Belief systems Safeguarding critical assets by ensuring adherence to specific rules and regulations (no arrow as does not rely on the content of the strategy) Bob Simons is the Charles M. Williams Professor of Business Administration at Harvard Business School. Over the last 30 years, Simons has taught accounting, management control, and strategy implementation courses in both the Harvard MBA and Executive Education Programs.

7. Boundary systems Strategy (plan) Core values, mission, vision Critical performance variables Strategic uncertainties Risks to be avoided Interactive control systems Diagnostic control systems Belief systems SEARCH CREATIVELY AND EXPAND OPPORTUNITY SPACE. LEARNING dominates over control.

8. Boundary systems Strategy (plan) Core values, mission, vision Critical performance variables Strategic uncertainties Risks to be avoided Interactive control systems Diagnostic control systems Belief systems CONSTRAIN SEARCH BEHAVIOUR AND ALLOCATE SCARCE ATTENTION. Control dominates learning.

9. Introducing management systems 9 Maturity of organisation Introduction of systems Internal control Diagnostic control Business conduct boundaries Belief systems Strategic boundaries Interactive control

10. 10 http://www.toyota-global.com/company/vision_philosophy/toyota_global_vision_2020.html

11. Belief systems Explicitly (e.g. via documents) providing basic values, mission, credos about added value, vision: Say something about what is of value (to be proud of, responsible for) for the long run, providing a compass for action …and hence that is NOT to be compromised for the short term Due to the high level of abstraction, not specific enough to ensure that staff actions do not create (however well- intended) serious risks Telling people what to do creates the risk they do not innovate/adapt …but only holding them accountable for achieving vision/mission, creates the risk they do take ‘innovative’ actions, driven by their intrinsic motivation, but that are inadvertently damaging to the organisation

12. We therefore also need systems that tell people what NOT to do and that INHIBIT people to do certain things: 1) Internal control (not linked to strategy) 2) boundary systems for strategy 12

13. Internal control systems Internal control systems: things organisations should do to safeguard their critical assets and reliable data from wilful violations and unintentional errors Structure Segregation of duties: accounting for a valuable asset done by other person than has physical custody Levels of authorisation: access to assets is commensurate with level of responsibility someone has for valuable assets Physical security: safes, surveillance (if not easy to reconcile)… Independent audit: examine integrity of internal control; report to boards Systems Accurate and complete records (trail) so a transaction can be tracked to its source Secure databases (cannot be tampered with) Timely management reports Staff: Trained accountants Job rotation (incl. replacement during vacations) Adequate resources for internal controls Note: this is the foundation of many EC regulations in ESFI, but it is not “strategy”

14. Toyota’s drive for growth moved into high gear in 1995 with the appointment of Hiroshi Okuda as the company’s new president. Okuda, known for his aggressive efforts to remake Toyota, was the architect of an ambitious global growth strategy, known as the “2005 vision.” ….turning point at 2003; from then on, sales grew faster than the company could manage…growth had taken priority over the company’s traditional focus on quality. Organizational incentives, especially informal ones, became skewed toward growth. Without specific policies that preserved the traditional quality focus, key decisions affecting product development, supplier management and production became biased in favor of meeting sales, delivery, cost- cutting and profit targets. A high-level Toyota executive publicly acknowledged in 2010 that, facing internal manpower shortages, the company had no choice but to use a large number of new contract engineers to boost engineering capacity. In his view, that contributed to the increases in quality glitches. The reality is that Toyota’s problems were not caused by a faulty production system but by poor management decisions. NOTE: as of 2009, Akio Toyoda became president of Toyota 14 What Really Happened to Toyota? Sloan Management Review, Summer 2011, R.E. Cole

15. Boundary systems-1 Formally stated rules, limits and proscriptions (“no exceptions”) Tied to defined sanctions and credible threats (requires making examples out of some situations) for punishment They are the “brakes” for the organisation, intended to stop people from engaging in unethical and/or harmful actions Also provide legitimate grounds for staff to refuse conflicting requirements from superiors Transgressions to be sought out and stopped actively (by management, auditors, other colleagues …)

16. Boundary systems-2 Two types: Business conduct boundaries: when cost of maintaining reputation is high (e.g. protecting image) e.g. legal constraints, codes of conduct communicating forbidden actions, to be signed and hence declared understood, to be reminded of (eg with regular training) management draws this up with specialist staff relatively simple as not possible to codify every possible situation so NOT 30000 pages of guidelines Strategic boundaries: what opportunities to avoid, services not to deliver, customers not to serve, value propositions to avoid when resources for new initiatives are limited and the organisation needs to stay focused e.g. simple strategy rules for opportunity selection like minimum asset utilisation rates, links with core competences, etc. as present in initiative management systems to be set by management

17. In January 2008, Chris Tinto, Toyota’s U.S. vice president in charge of technical and regulatory affairs, further warned his fellow executives that “some of the quality issues we are experiencing are showing up in defect investigations (rear gas struts, ball joints, etc…).” These and other early warnings were ignored. In a pattern not uncommon in large organizations, politically powerful executives shrugged off early warnings of lower-ranking executives. Top management in Japan has been less sensitive to the expectations of regulators, culture and politics in overseas markets, and consequently, they have been slower to respond to local problems. 17 What Really Happened to Toyota? Sloan Management Review, Summer 2011, R.E. Cole

18. Interactive control-1 Interactive control systems: focus attention on strategic uncertainties and provide a lever to fine-tune and alter strategy, responding to changes in the environment Search for and identify patterns of change, emerging threats and opportunities, strengths and weaknesses that could invalidate the assumptions underpinning the strategy Data is used to challenge the current assumptions -actively looking for surprises- and force to make sense of changing conditions behind the surprise Not just reporting but interactive discussion on data and assumptions at ALL levels, leading to organisational learning Always face-to-face, direct involvement of management This is where bottom-up action plans and experiments create emergent strategy But not in a chaotic way, as management does guide the search for new opportunities and stimulates the experiments R. Simons, Performance measurement and control systems, ch. 4,2000

19. Interactive control-2 Requirements for interactive control systems: Information must be straightforward: there should be no need to understand complex algorithms or calculations to understand what is presented The information must relate to strategic uncertainties (relating to assumptions that underpin the current strategy) New action plans must be generated based on answers to the questions: What has changed? Why? What will we do about it? The information must be used at all levels of the organisation: higher levels stimulate lower levels to search for, analyse and discuss new information Hence the info must be widely available Management itself must interpret the information; this cannot be outsourced to analysts Paperwork and forms should never become more important than dialogue and action: ultimately, it is about adjusting in real time to what emerges R. Simons, Performance measurement and control systems, ch. 4,2000

20. Interactive control-3 Types of interactive systems: Business planning systems: Uncertainty: changing needs affect projected demand relating to new services/solutions Technology / innovation project management systems: Uncertainty: changes in technologies (in the broadest sense) affect attributes of current and potential services/solutions Demand tracking systems: Uncertainty: changes in attributes of existing services/solutions affect who make(s) how much use of them Intelligence systems: Uncertainty: changes in the social, economic, political, technological… environment Human resources systems (competence management, manpower / succession planning): Uncertainty: changes in required competencies 20 R. Simons, Performance measurement and control systems, ch. 4,2000

21. 21 R. Simons, Performance measurement and control systems, ch. 4,2000 The same systems can be used diagnostically or interactively! The difference lies in how data is used! The same systems can be used diagnostically or interactively! The difference lies in how data is used!

22. Diagnostic control-1 Diagnostic control systems: communicate what is critical in terms of the strategy and measure this with critical performance indicators Failure there would lead to failure of the organisation and its strategy Reports are exceptions (variance) based: How much, what? Implies that we identify variables that can vary Warning light goes on and this triggers action to correct situation Implies that someone knows how to interpret variance Implies that people who are doing the work are also interpreting this; they should not be waiting for managers to act Of course, some variances will have causes that cannot be tackled just by the people doing the work on a daily basis. This should be triggering interactive control systems! R. Simons, Performance measurement and control systems, ch. 4,2000

23. Diagnostic control-2 One of the key uses of diagnostic control in manufacturing is the monitoring of reaching standards: specify the work in terms of process (how it is done) or output (what is delivered) not meeting a specification may create problems for others involved downstream in a wider process or with inter-operability e.g. an output from one part of the process turns out problematic for the next step where it is an input however, this becomes a problem when (over)specifying the work for the sake of it: one should strive for minimal critical specifications otherwise there is waste from doing more than is necessary 23

24. Diagnostic control-3 Translating standards to service delivery is problematic: the key question is whether or not a service provider can respond to the customer demand and these demands (unlike specifications for manufactured outputs that are the focus of production plants) vary the service provider can only deal with that demand variation by matching it with its own variety in terms of response (regular demands all front line workers should be able to deal with, more exceptional demands they can pull in knowledge from someone else for) input control, usually in the form of people through the recruitment and training process (e.g. highly capable and integer individuals) is crucial here while YES/NO standards on the work itself lead to reducing response variety in service delivery and hence to increase variation in terms of properly serving the demands of customers Studying whether the process is predictably satisfying demand (end to end time, accuracy/value, one stop) is what matters! NOT yes/no standards on parts of the process! 24

25. 25 Strategy Learning Tactics (strat. initiatives) Actions (incl. business as usual) Strategy Critical performance variables KPI’s Input process output Top down pressure Diagnostic Interactive Series of interlocking dialogues Exception reporting on transaction points Strategy testing and adaptation Strategy review (tactics) and operational review (action level ie ongoing improvement) Operational review R. Simons, Performance measurement and control systems, ch. 4,2000

26. 26 time UCL, LCL and center are NOT YES/NO targets but bounds of natural variation. Process is under control Exception reports. Can trigger interactive control if cause outside control of operational teams.

27. 27 Tools like Sensemaker are also of use What is it that makes this cluster of stories different from that one?

28. 28 Tools like Sensemaker are of use…especially when using them over time Why this “deviance”? All stories from various sources were related to only 1 staff member!

29. 29 H. Mintzberg DIAGNOSTIC CONTROL

30. 30 H. Mintzberg INTERACTIVE CONTROL

31. Boundary systems Management systems and motivation Strategy (plan) Core values, mission Critical performance variables Strategic uncertainties Risks to be avoided Interactive control systems Diagnostic control systems Belief systems Extrinsic: fear of punishment or… Intrinsic: desire to do right (belonging) if …no undue pressure/temptation Extrinsic: reward (judgmental) or… Intrinsic: desire to achieve (mastery) and control (autonomy) if … focus and resources Extrinsic: pride, glory or… Intrinsic: desire to contribute to bigger whole (belong) if… sure of purpose Extrinsic: reward (formula) or… Intrinsic: desire to innovate/create (autonomy) if… opportunity for dialogue /dissent without fear

32. 32 How is all this translated in core RBM practices?

33. Results Based Management an organization should be ready to respond to changes in circumstances or performance and alter one’s approach if necessary, which requires… an analytical and performance-oriented approach to understanding what to do; a view of performance that is centred on beneficiary level outcomes; only thereafter moving toward identifying what resources, actions and outputs are necessary to get there; a need for information about the subject and a capacity to collect, process and analyse this information over time; a mechanism by which policy, planning and decision-making can be influenced by performance information. Source: REVIEW: RESULTS BASED MANAGEMENT IN DEVELOPMENT COOPERATION by the Riksbankens Jubileumsfond synthesised in the COP RBM Sourcebook ch. 3.3

34. Results Based Management an organization should be ready to respond to changes in circumstances or performance and alter one’s approach if necessary, which requires… an analytical and performance-oriented approach to understanding what to do; a view of performance that is centred on beneficiary level outcomes; only thereafter moving toward identifying what resources, actions and outputs are necessary to get there; a need for information about the subject and a capacity to collect, process and analyse this information over time; a mechanism by which policy, planning and decision-making can be influenced by performance information. Source: REVIEW: RESULTS BASED MANAGEMENT IN DEVELOPMENT COOPERATION by the Riksbankens Jubileumsfond synthesised in the COP RBM Sourcebook ch. 3.3 Demand orientation Focus on the system

35. 35 Managing and Delivering Performance: How Government, Public Sector and Not-for-profit Organisations Can Measure and Manage What Really Matters Bernard Marr is a global enterprise performance expert and a best-selling business author. He helps companies to better manage, measure, report and analyse performance. His leading-edge work with major companies, organisations and governments across the globe makes him an acclaimed and award-winning keynote speaker, researcher, consultant and teacher. Bernard is acknowledged by the CEO Journal as one of today’s leading business brains. He has written a number of seminal books and over 200 high profile reports and articles on enterprise performance. This includes the best-sellers 'Key Performance Indicators', ‘The Intelligent Company’, ‘More with Less’, ‘Managing and Delivering Performance’ and ‘Strategic Performance Management', a number of Gartner Reports and the world’s largest research studies on the topic. His expert comments regularly feature in high-profile publications including The Times, The Financial Times, Financial Management, the CFO Magazine and the Wall Street Journal.

36. 36 ? ?

37. Six core RBM practices 1. Orientation of the PMO (programme management organisation) is clear 2. The PMO strategy is reflected in a strategic results framework 3. The strategy is translated into operations 4. Performance information is collected and supplied 5. Performance information is used 6. External stakeholders are involved in all aspects 37

38. 38 1. Orientation of the PMO (programme management organisation) is clear 2. The PMO strategy is reflected in a strategic results framework 3. The strategy is translated into operations 4. Performance information is collected and supplied 5. Performance information is used

39. 39

40. 40

41. Mission and values of the entity E.g. “We support – with EU and national resources – organisations to initiate actions that improve the functioning of the labour market. We acquire and share knowledge to contribute to solutions for today and tomorrow.” E.g. “customer focus: staff have an open and caring attitude, towards project promotors as well as other stakeholders. The promotor can easily get in touch and communicate smoothly with staff. The relation with stakeholders is based on mutual respect and appreciation. Agreements and promises are kept. All interests are taken into account. All of these aspects lead to a relationship of trust.” 41

42. 42

43. 43 Example of expanded vision Going beyond current capabilities is crucial!

44. Vision can have many different aspects the ambition can be broad, generic e.g. drive innovation in the labour market e.g. be champions of simplification e.g. …or it can relate to specific thematic policy challenges high on the political agenda at a give time e.g. improve the life of Roma… e.g. tackle poverty… if ambition is too low or perceived to be irrelevant as it does not connect with the policy context, those who decide on its mandate may attack it 44

45. Vision and current operations When current operations are a mess… … then vision becomes a vision of survival: how to keep the boat from sinking! This reaffirms the importance of systems thinking for your current operations: If daily work life is a struggle for survival… 45 …then senior managers coming along with wonderful statements about vision, mission and values are bound to be seen as coming from another planet

46. Warning! Mission, values and vision can be actively used to drive out critical voices, thus eradicating diversity in the organisation which is problematic for accountability as “robust, resilient and adaptive” The idea is to foster “a community of inquirers”: practitioners start from their own day-to-day experience of trying to do the work, within the framework of a mission and a vision, but where the aim is to make subjective experience more objective through reflection and discussion with the manager and others in doing so the practitioner and manager could open themselves up to different ways of working and perhaps move themselves on from stuck and less productive ways of working with peers This entails that what the mission and the vision “mean” at a practical level is constantly being (re)invented 46

47. 47

48. 48 Example output of a SWOT exerciseWhat are key strategic priorities? Eg what S can we use to capture O? What W must we amend? Etc. Once, again, if current operations are mess, this will crowd out any thinking about longer term opportunities and threats

49. A key question here is to define how you want to add value as a PMO? We will cover this when we discuss delivery mechanisms! 49

50. PMO strategy = strategy concerning added value PMO strategy = technical assistance priority OP Priority 1 OP Priority 1 OP Priority 2 OP Priority 2 OP Priority 3 OP Priority 3 PMO strategy = technical assistance priority OP Priority 1 OP Priority 1 OP Priority 2 OP Priority 2 OP Priority 3 OP Priority 3 Added value by leveraging the technical assistance! 50

51. 51

52. Strategic results framework The COP RBM system focuses on SF PMO and what these organisations should be aiming for and doing to increase their added value to a country or region. 52 The top level reflects policy challenges. But the PMO strategy will not typically emphasise all thematic context objectives from an OP and may add other types of expectations to/ from these stakeholders. The top level reflects policy challenges. But the PMO strategy will not typically emphasise all thematic context objectives from an OP and may add other types of expectations to/ from these stakeholders. Expectations in different perspectives are bi-directional (toward and from)

53. 53

54. 54 Enable citizens, communities and society to be resilient and treat them honest and fair

55. These are the outcome objectives (beyond control hence only influencable). SHOULD BE: what our partners/constituents REALLY care about and WE are willing to fullfill given our interpretation of our vision 55

56. These are the input/process/output objectives (deemed under control) 56

57. 57 The three components of the value adding core process are reflected. only outputs and outcomes deemed critical to achieve a vision this means that the map is not intended to reflect all possible internal processes and associated outputs nor all possible external process, constituent and legitimizing authority outcomes this would overload such a map and render it useless as an instrument for strategic action.

58. 58 Example of expanded vision Same vision can be realised with different strategies!

59. Innovator strategy map vs enhancer 59

60. 60 BUSINESS UNIT MAP Focus on core value adding processes CORPORATE MAP -Focus on synergies (coordination and cooperation ) -Focus on common priorities/principles -…or fix completely BU maps (then BU maps are the same)

61. Cascading From the business unit strategy is cascaded to operational teams within the business unit They should conduct a mission, vision and values exercise at their own level, within the framework of the higher level ones, and then to look at the higher level strategy map from that perspective this may lead to requesting adjustments to the higher level map or to adjusting mission, vision and values most frequently unit level mission, vision and values will just be helpful in translating a higher level map into a suitable lower level one. The strategic outputs and outcomes on the map are arrived at in consultation with staff in the cascading process The business unit strategy may therefore be adjusted in the cascading process The idea is that there is a discussion where a lower level can argue for its interpretation of a higher level map and can even argue for modification of the higher level Contestation is very much at the core of this process and it is crucial that higher level management incites this and creates openness for it Within operational business units, individual staff member objectives are then similarly aligned to unit objectives 61

62. Regulatory/Support unit alignment Regulatory/Support unit strategy map should have: a top level with a focus on both effectiveness (this contains the objectives from the corporate, business and operational unit maps that the support unit can impact on) and efficiency (in terms of use of resources) a client level that identifies who the units serve and what these clients (or client groups, e.g. staff, managers,...) expect an internal process level an internal capacity level 62

63. 63 Skip for now

64. Strategic initiatives The vision relates to the next 3-10 years. Hence so does the strategy. This entails the strategy map is future oriented. If the vision was challenging enough (as it should be) then it would be hard to imagine that this can be realized by doing more of whatever the entity is already doing or by doing it better with whatever means it is already using. Rather, the entity will have to do things it does not do yet at all or acquire new capacities, competences and/or relationships it does not have yet. 64

65. Strategic initiatives Strategic initiatives do not represent: business as usual or … ongoing improvement/maintenance of existing processes and outputs They proactively prepare the organisation for the future e.g. by addressing NEW products/services via new (types of) calls for proposals …and/ or NEW relations, competences, expanded capacities hence there is considerable uncertainty about how to run them and what they can achieve E.g. a full Vanguard method based redesign Require coordination from many parts of the organization As some strategic initiatives may be related (complementing each other or feeding each other), they should be grouped under overarching themes that can be visualized in the strategy map. 65

66. 66

67. 67 1.Select a process objective Quality of care: solve my health issues in a sustainable way as quickly as possible *illustrative real example not be taken as an ideal example *

68. 68 2. Break up the process objective into its key components THREE MAJOR TRANSACTION POINTS OF CONTACT THAT FEED INTO EACH OTHER

69. 69 3. Break down the components into drivers To understand better what the situation is, you can engage in measurement *illustrative real example not be taken as an ideal example *

70. 70 4. Reflect on what the measurement tells you from a systemic point of view Ask: “How are things currently organised / done – as a system- that needs to be different?” Be careful with measures: should be “systemic”. Ask: “How are things currently organised / done – as a system- that needs to be different?” Be careful with measures: should be “systemic”. *illustrative real example not be taken as an ideal example

71. 71

72. Strategic versus operational plans Demand plan: planned (increase of) outputs by type (ranging from simple outputs to complex calls for proposals) Capacity requirements + free capacity maintained (in % used of available people and facilities) Factoring in expected process improvements Direct operational, maintenance AND improvement costs = OPEX Capital expenditure (replacement/expansi on of existing facilities for current business) =CAPEX Indirect operational cost =OPEX Strategic initiatives STRATEX (can be both expenses or capitalisa tion) PLANSPLANS BUDGETBUDGET Operational plans (with a one year time horizon) specify which outputs will be delivered by which parts of the entity.

73. Remember! 73 This concerns planning and score keeping, NOT operational management (developing, improving)!

74. 74

75. 75 / quarterly Does not question strategy as such = strategy reviews are mainly interactive control systems! They focus on learning! -what comes up from strategic initiatives? -what comes up from operations that they cannot handle? = strategy reviews are mainly interactive control systems! They focus on learning! -what comes up from strategic initiatives? -what comes up from operations that they cannot handle?

76. Cascading to be performed again 76 / quarterly Does not question strategy as such (could also be prompted by new threat or opportunity emerging from regular environmental scanning) = strategy testing and adaptation meetings are also interactive control systems, now questioning the strategy as such = strategy testing and adaptation meetings are also interactive control systems, now questioning the strategy as such

77. 77 / quarterly Does not question strategy as such (could also be prompted by new threat or opportunity emerging from regular environmental scanning) Operational reviews are mainly a diagnostic control system –focusing on “control”- but, as it concerns continuous improvement, is also used interactively

78. Strategic measurement-1 This measurement should reflect key performance questions (KPQs) that require data for informed discussion regarding possible answers: open questions (what, why, how) rather than closed questions (yes, no, how much): e.g. Not: “was the budget spent? Rather, “what has happened with the budget?”, “why?”, “how will we act?” relate to the strategy map formal “measurement” does not necessarily equate to an indicator but may require qualitative/quantitative research Strategy testing questions relate to the map in its entirety and to the underlying strengths, weaknesses, opportunities and threats e.g. does it really matter to invest in networks to get our delivery partner to be more innovative… We were not sure about this. How will we find something out? e.g. we thought our programme would have 30% less financial resources? What happened? What will we do? 78 Interactive questions Interactive questions

79. Strategic measurement-2 Strategy review questions related to objectives on the strategy map and associated initiatives e.g. (among others) How are strategic initiatives developing (what has changed, why)? How are they contributing to realising strategic objectives as hoped (what has changed / why)? What will we do? …e.g. we wanted to make sure staff were able to deal with requests one stop so we trained them for predictable demand. What have we learned? How is the programme deployed by the PMO having an impact on Roma? To what extent are innovative actions of DPs for Roma developing properly? Why? What will we do? Frequency of strategic measurement depends on the nature of the KPQs: minimum quarterly for questions looking at progress of strategic initiatives and possibly their impact on the strategic objectives minimum yearly basis for questions relating to the overall logic of the strategy as visualised by the strategy map 79

80. Strategic measurement-3 80 E.g. Action research (probe, sense, act / plan, do, check,act) responding to KPQs such as “What is getting developed, what is not getting developed, why and what are the implications?” Traditional arsenal of social research (incl. theory based and counterfactual impact evaluation) Interactive questions Interactive questions Programmme evaluation is one specific kind of measurement activity!

81. 81 Tools like Sensemaker are also of use What is it that makes this cluster of stories different from that one?

82. 82 Tools like Sensemaker are of use…especially when using them over time Why this “deviance”? All stories from various sources were related to only 1 staff member!

83. Strategic measurement and cascading If relevant, questions and associated measurement ideas are cascaded at the same time as the objectives they measure This means that cascaded objectives also have cascaded measurement at unit level: staff think about KPQs at their level, just as they thought about objectives and engage in discussion with higher level A baseline is established when measurement is conducted for the first time discuss with relevant staff whether and how improvement can be achieved and how the PMO will know if things are moving into the right direction when using quantitative research, it entails ensuring that the direction of the desired trend is discussed when using more qualitative research, the key characteristics of the desired situation should be discussed in both cases we will want to answer “why” change has occured which is not something we can derive from descriptive info 83

84. Operational measurement Operational reviews are: Diagnostic: By now it should be clear that we want to use systemic measures and statistical process charts Interactive Look for “signals” in diagnostic system and then look into reasons behind these (interactive) To what extent are we achieving what we expect (what has changed)? Why not? What to do? Inline experimentation initiatives to improve service How are process initiatives (not) improving KPIs? Why? Tools in the hands of staff that do the work to self- regulate (decide what to do about it), NOT for managers to control them 84

85. 85 time UCL, LCL and center are NOT YES/NO targets but bounds of natural variation. Process is under control Exception reports. Can trigger interactive control if cause outside control of operational teams.

86. Knowledge is shared 86 Ways to stimulate organisational learning