1. Water today, water tomorrow Water and Regulation Improving the value from statistics Derek Parsons Engineering Adviser – Ofwat April 2010

2. Contents Water today, water tomorrow Welcome to WSUG Ofwat Strategy & Programme Regulatory issues – Re-Cap PR09 observations and frustrations Ofwat – WSUG common interest

3. Ofwat strategy – our vision Water today, water tomorrow Ofwat strategy – sustainable water published March 2010 “A sustainable water cycle in which we are able to meet our needs for water and sewerage while allowing future generations to meet their own needs”

4. Our strategy…has been… Protecting consumers, promoting value, safeguarding the future Keeping consumers at the heart of what we do Enforcing compliance Keeping companies accountable Introducing competition where it benefits consumers Regulating effectively Shaping the regulatory agenda at national and EU level Taking a long- term view of sustainability

5. Our strategy…has been ‘refreshed’ Water today, water tomorrow

6. New organisation, new projects, new programme Finance and Networks Corporate Services and Programme Management Policy and Communications Markets and Economics Future water charging Future price limits Regulatory compliance Future scenarios Sustainable drainage Market reform Europe and environmental policy Water today, water tomorrow

7. Ofwat plan 2010-11 to 2012-13 Ofwat forward programme published March 2010 “we are developing an integrated programme of work that will look at why and how we do things. Each of the projects will take into account relevant recommendations from the Pitt, Cave and Walker reviews, as well as developing new thinking on solutions to the challenges the sectors face.”

8. Ofwat Projects Future scenarios Retail market reform Future price limits Sustainable drainage Future water charging Regulatory compliance [Accounting separation] [Upstream market reform] Water today, water tomorrow

9. Routine work – doesn’t stop Keeping consumers at the heart of what we do Keeping companies accountable Enforcing compliance Regulating effectively

10. Water today, water tomorrow Data Both our project and routine work require “…reliable, accurate and complete data…” and the WSUG aim of “...improving the …understanding of the use and value of statistics within the Water Industry”. are aligned …..so we welcome WSUG and support your objectives

11. Water today, water tomorrow Some issues – assistance needed Use of data confidence grades Modelling assumptions Awareness of and balancing risk

12. Water today, water tomorrow Data Reporting Formalised approach to presenting data “Confidence grades” A1, B3, C4, DX matrix Clear what ‘alpha’ element ‘reliability’ is Clear what ‘numeric’ element ‘accuracy’ is – see June Return Guidance BUT…

13. Water today, water tomorrow Data confidence grading Not clear how to assign values Not clear how to tell if assigned consistently Less clear what it means in terms of precision and confidence Even less clear how to use the confidence grading in calculations Especially if grading is applied to a profile (condition grading) So how does anyone know that a central estimate has been suitably derived?

14. Modelling assumptions Models are typically developed by expert third parties Use company data as much as possible Are guided in their use by expert third party The results are owned by the Company BUT… Water today, water tomorrow

15. Modelling assumptions Models have assumptions and / or default values Company Board not necessarily aware of their impact Companies can find it hard to resist model results (+ / -ve) Is there scope for a modelling protocol template? Where - Assumptions are stated - Impact of assumptions stated - Sensitivity around different assumptions stated - Statistical parameters around results - Guidance needed on what must be considered - How to identify a reliable modelling output - Expert peer review / QA - Best practice Water today, water tomorrow

16. Awareness of and balancing risk Companies have adopted a risk based approach Use company data as much as possible Are sometimes guided in their use by expert third party The results are owned by the Company BUT… Water today, water tomorrow

17. Awareness of and balancing risk Risk = likelihood x consequence Uncertainty around data, both parameters Some risks are very low likelihood high consequence - how to model around consequence quantum? Water today, water tomorrow

18. Scope for templates? Passive (June returns etc) - simple check for ‘correct’ application of grading - is there a better format, without compromising past? Active (business planning) - More complex - inform ‘best practice’ on measurement - inform ‘best practice’ on modelling - inform ‘best practice’ on stats calculation - inform ‘best practice’ on risk balance Water today, water tomorrow

19. Areas of interest See previous presentation Sep 2009 - AMA assessment - Capital estimating scorecard - Capex estimates - Delivering stable serviceability - Climate change - CBA benefits, WTP + Risk awareness, uncertainty and balance Water today, water tomorrow

20. Common theme Knowing what is unknown Measuring uncertainty Understanding uncertainty Communicating uncertainty Enumerating risks associated with uncertainty Facilitate Risk Balancing Demonstrating ‘in control of’ risk management Water today, water tomorrow

21. Ofwat - WSUG Common interest? Development of processes and associated tools that facilitate understanding and communication of uncertainty to inform and enable better risk management Water today, water tomorrow

22. ANNEX – some slides from Sep 09 presentation Water today, water tomorrow Re-cap on observations brought to first meeting…

23. Water today, water tomorrow PR09 observations 1 – asset planning leadership; policy and strategy; management; planning processes; planning systems; (data collection, processing, business as usual) data; (asset, service, performance and intervention) analysis; (performance modelling, prediction and future intervention) reporting; and overall approach to risk and balance. (optimising risk, co and customer) To assess PR09 plans for capital maintenance expenditure (circa £12bn over 5 years) Ofwat used its ‘AMA assessment’. This assesses company plans over 31 assessed components divided between the following headline areas. Statistical issues are very pertinent to some of these as indicated above.

24. Water today, water tomorrow Industry PR09 draft determination AMA scores - example This is the best scoring sub-service (out of the four) – so room for improvement!

25. Water today, water tomorrow PR09 observations 2 – Capex estimates – understanding estimating confidence?

26. Water today, water tomorrow PR09 observations 2 – Capex estimates – understanding estimating confidence? How is the uncertainty priced?

27. Water today, water tomorrow PR09 observations 3 – CBA benefits, WTP Significant uncertainty across many attributes - WTP, impacts of schemes, costs Many different techniques - Monte-Carlo, Scenario analysis, switching Which are the best methods? Can results be pooled to reduce the uncertainty? - What variance is there across companies/regions? Is there a minimum survey size? An optimum?

28. Water today, water tomorrow PR09 observations 4 – serviceability Conundrum – forecast confidence insufficient despite historic performance Modelling gives limited forecasting capability: 50 % confidence of achieving range B 95% confidence of achieving range C Need 100 % confidence of achieving range A, AND oscillate around a target figure (reference level), as historically observed ABC

29. Water today, water tomorrow PR09 observations 5 – climate change UKCIP02 provided one future climate change outcome – best evidence Size of change UKCP09 will relative measures of the considered evidence that supports plausible future climate change outcome Time Size of change Time Consistency in the argument and accounting for uncertainty

30. Water today, water tomorrow A water regulation perspective Questions?