Dubious Tales from Deep Dark Waters Past (Being a very selective history of hydro management and NZEM origins.. with some observations on various kinds of bias) E.G.Read with contributions from S. Starkey Presented to EPOC Winter Workshop Auckland, New Zealand 5 September 2013

Caveat This presentation should not be quoted, or relied upon It has been largely prepared from memory which is inevitably selective and biased (and not aided by the fact that any files I still have are currently in post-quake limbo-land)

Some Overview Papers J.G. Culy, E.G. Read, and B. Wright: "Structure and Regulation of the New Zealand Electricity Sector", in R Gilbert and E Kahn (eds.) International Comparison of Electricity Regulation, Cambridge University Press, 1996, p E.G. Read, J.G. Culy, and S.J. Gale: "Operations Research in Energy Planning for a Small Country", European Journal of Operational Research, vol. 56, 1992, p E.G. Read: "OR Modelling for a Deregulated Electricity Sector", International Transactions in Operations Research, vol. 3, no. 2, 1996, p E.G. Read: "Electricity Sector Reform in New Zealand: Lessons from the Last Decade” Pacific Asia Journal of Energy Vol 7, No 2, 1997, p

Episode 1: Guidelines 1958 Meremere: – The Basic Rule Curve/ Forbidden Zone/Guideline – Basically a critical probability calculation (as in the classic “newsagent” problem) – Backwards projection of minimum storage to survive “Design Dry Year” (e.g. 1:20 if marginal cost of running out is 20 times marginal cost of fuel) 1967 Marsden: - Multiple Guidelines

Guideline Diagram (From Read and Boshier 1989)

Episode II: VALWAT/STAGE Experimented with SDP involving a variety of forward simulation strategies (Programmed by Chris Lusk c.1979, reported by Read and Boshier, 1989) Lead to abandonment of: – Deterministic LP model (Boshier and Lermit, 1977) – Deterministic decomposition model (Read, 1983)

Produced STAGE Developed 1980ish, reported by Boshier et al (1983) Based on Swedish model by Stage and Larsson : – Uses “Marginalistic SDP” - Setting MWV t = marginal value of release in t = E{MWV t+1 } …. Subject to bounds – But with about 6 months “forward simulation” - Setting MWV t =E{ MWV t+26, or spill/shortage value if storage bound reached sooner} – Trying to capture effects of correlation - Spreading trajectories over a wider range - So typically raising expected MWVs (see later)

Optimal (SDP/STAGE) guidelines

Optimal deterministic guidelines assuming average inflows

Optimal guidelines from averaging deterministic optima for historical inflows

Optimal guidelines assuming naïve future management

Biases? The optimum is actually rather flat: – The two extreme solutions performed badly, but – While the SDP But was preferred for its conservatism – It actually produced similar simulated performance to the average of deterministic policies Why do these biases occur, though?

The System Marginal Cost Curve is basically convex Optimal management will try to control variability of marginal cost as best it can

Bias arises due to sub-optimal/unrealistic control of variability True expected marginal cost with optimal (well controlled) degree of variation Under-estimated expected marginal cost from expected situation /perfect foresight Over-estimated expected marginal cost due to sub-optimal (poorly controlled) degree of variation

Episode III: RESOP/PRISM>SPECTRA “2-D Constructive Dual Dynamic Programming” – so named retrospectively by Read and Hindsberger(2010) – Conceived by Read (1984ish) – Programmed by Culy, Davies, and many more over the years – Reported by Read et al(1987) RESOP reservoir optimisation module: – Originally developed within a long term planning model (PRISM) – Soon used for operational planning and much more – PRISM was later re-developed as SPECTRA

Basic Concept: “Guideline augmentation” by inserting “flats” corresponding to varying utilisation level of one thermal station

“Uncertainty Adjustment” to produce new expected MWV curve

Another source of bias: decision period length Notice that, a longer decision period means: – The flats get wider in proportion – The uncertainty adjustment interpolates over a wider range But the MWV curve is still basically convex, so: – The Expected MWV curve may be expected to rise – And also lose its detailed structure – Release decisions become more “moderate” (since we could not stick with more extreme release levels for any length of time) This was very evident in early RESOP experiments

A more recent example (using the model of Dye et al, 2012)

And another source of bias: Interpolation True expected marginal cost Over-estimated expected marginal cost due to naïve linear interpolation on a (relatively coarse) grid

Yet another source of bias: assuming independence Cumulative pdf of independent in flows Effect of correlation

Counteracting biases in SPECTRA Independent inflows: – Too optimistic (lowering MWV) Linear interpolation on a fairly coarse grid – Too pessimistic/cautious (raising MWV) Originally a coarse enough grid was used to give a cautious bias But tuning was possible using a finer grid – And it was tuned to achieve the 1:20 criterion exactly – In 1992, which was worse than a 1:20 event – So we should have run out of water – And nearly did

After the 1992 crisis… Operational reliability standard raised: – From 1:20 DDY with 7% load reduction – To 1:60 DDY without load reduction (The operational optimum being quite flat due to excess supply due to overbuilding in NZED/MoE period)

Heuristic added to account for correlation in optimisation phase by: – Inflow pdf spreading σ'= σ(t) / √t where σ(t) is the s.d of t week cumulative flow distribution – So assuming independent inflows in optimisation gives effective cumulative inflow s.d of: σ‘(t) = t * σ‘ / √t = σ(t)

Heuristic added to account for correlation simulation/operation phase by:

Episode IV: Decentralisation The 1992 crisis provided some impetus for change, by denting ECNZ’s reputation, – Quite unfairly IMHO But some kind of “competitive” market was always on the agenda – And ECNZ’s dominance was obviously a much more significant issue

Impact of decentralisation? Less precise coordination – Less detailed information – Less sophisticated modelling Greater risk aversion – Less pooling available to meet contract commitments More innovation???????? Greater diversity of perspective/technique – Lower overall risk

Benefits of Diversity? full empty Manager 1 Manager 2 Diverse strategies reduce probability of extremes Both empty Both full

A Comment on Bias in Performance Evaluation Performance evaluation seems much more difficult than most OR papers recognise Classic paradigm: – I make assumptions/collected data/built a model – I show that my policy is better than yours… … given my assumptions/data/model – I get my work published, and you get no say in it But reality is very different… … not least in that it is neither known nor agreed

What About Reality? Even if reality was as simple as an (agreed) inflow distribution: – We will not observe that reality, only randomised outcomes – Policies will tend to be judged by what happened, not what might have happened Observers differ widely in their assessment of: – Underlying driving forces – Current data values – Forecasts – What ‘should’ drive decision-making – Their personal situation and stake – The interpretation of historical events

A Rational Approach? Given this variety – What seems “optimal” to one analyst may seem totally irrational to another. – And vice versa!! – We should be (a)ware of (the extent to which we are) privileging our own perspectives when comparing our own recommendations with those of others That leaves us with a much more difficult task: – Assessing internal consistency with stated assumptions – Assessing robustness of outcomes in terms of a range of criteria, across a range of possible assumptions

Or… We could just entrust decision-making to those who have most at stake – And hope for “robust” outcomes Knowing that these will always seem sub- optimal – From (almost) everyone’s perspective

Reservoir Management Outcomes? We should expect storage coordination to look “wrong”, from any individual perspective But Tipping and Read (2010) Tuned a model to show that, in aggregate, hydro was operating as if using a plausible looking MWV curve Subsequently tested to find storage policy – About as cautious as that under MoE – Somewhat more cautious than that under ECNZ – As should be expected?

BUT None of this was really what motivated the NZEM reforms Advantages were seen in increased decision- making diversity – But maybe only enough to offset loss in coordination efficiency The real issue was creating a competitive market as a means of controlling cost and prices – Rather than relying on “Government”

At the time… History and common sense both seemed to suggest that it was politically impossible for a Government (of any stripe): – To make unbiased growth forecasts for the economy (and hence for electricity) – To back away from plans and promises that turned out to be unwise – To set fair, honest and realistic prices for industry, commerce, or domestic consumers – To resist biasing technology choices

The fear was that: Gains in operational efficiency at ECNZ would be lost by gradual reversion to public sector norms, but particularly Forecasting, planning and pricing would be re- politicised – With potentially severe consequences for allocative and investment efficiency – Which are where most of the sector’s costs are incurred

Concluding Perspective Harker (2013) claims that the true energy component of domestic power bills has risen in recent years – Almost to the level reached under the MoE in 1982

But… was the only year, in the MoE period, in which -After rising by 124% in 12 months -Prices (briefly) reached LRMC levels -As calculated using cost projections from SCM and MWD -Most of which turned out to be significant under- estimates 30 years later, with cheap gas and hydro both gone: – The New Zealand electricity sector is still (apparently) producing at those prices – And actually paying its own way!

If Dr Harker is correct … I am personally astounded How did such a marvellous thing ever come about? Maybe Max was right?

More References J.F. Boshier and R. J Lermit: A Network Flow Formulation for Optimum Reservoir Management of the New Zealand Power Generating System, NZOR vol5 #2., 1977, p J.F. Boshier, G.B. Manning, and E.G. Read: "Scheduling Releases from New Zealand's Hydro Reservoirs" Transactions of the Institute of Professional Engineers in New Zealand, vol. 10, no. 2/EMCh, July 1983, p S. Dye, E.G. Read, R.A Read, S.R. Starkey “Easy Implementations of Generalised Stochastic CDDP Models for Market Simulation Studies” Proceedings 4th IEEE/Cigré International Workshop on Hydro Scheduling in Competitive Markets. Bergen, Norway, 2012 B. Harker “Chairman's Address”, TrustPower AGM, July E.G. Read: "Reservoir Release Scheduling for New Zealand Electricity - A Non-Linear Decomposition Algorithm", New Zealand Operational Research, vol. 11, no. 2, July 1983, p E.G. Read, J.G. Culy, T.S. Halliburton, and N.L. Winter: "A Simulation Model for Long-term Planning of the New Zealand Power System", in G.K. Rand (ed.) Operational Research 1987, North Holland, p E.G. Read: "A Dual Approach to Stochastic Dynamic Programming for Reservoir Release Scheduling", in A.O. Esogbue (ed.) Dynamic Programming for Optimal Water Resources System Management, Prentice Hall NY, 1989, p E.G. Read and J.F. Boshier: "Biases in Stochastic Reservoir Scheduling Models", in A.O. Esogbue (ed.) Dynamic Programming for Optimal Water Resources System Management, Prentice Hall NY, 1989, p E. G. Read and M. Hindsberger “Constructive Dual DP for Reservoir Optimisation” in S. Rebennback, P.M. Pardalos, M.V.F. Pereira and N.A. Iliadis (eds) Handbook on Power Systems Optimisation Springer, 2010, Vol I p3-32 J. Tipping and E. G. Read “Hybrid bottom-up/top-down modelling of prices in hydro-dominated power markets” in S. Rebennback, P.M. Pardalos, M.V.F. Pereira and N.A. Iliadis (eds) Handbook on Power Systems Optimisation Springer, 2010, Vol II, p