EPOC Winter Workshop, October 26, 2010 Slide 1 of 31 Andy Philpott EPOC ( joint work with Vitor de Matos, Ziming Guan Advances in DOASA

EPOC Winter Workshop, October 26, 2010 Slide 2 of 31 What is it? EPOC version of SDDP with some differences Version 1.0 (P. and Guan, 2008) –Written in AMPL/Cplex –Very flexible –Used in NZ dairy production/inventory problems –Takes 8 hours for 200 cuts on NZEM problem Version 2.0 (P. and de Matos, 2010) –Written in C++/Cplex –Time-consistent risk aversion –Takes 8 hours for 5000 cuts on NZEM problem DOASA

EPOC Winter Workshop, October 26, 2010 Slide 3 of 31 Motivation Market oversight in the spot market is important to detect and limit exercise of market power. –Limiting market power will improve welfare. –Limiting market power will enable market instruments (e.g. FTRs) to work as intended. Oversight needs good counterfactual models. –Wolak benchmark overlooks uncertainty –We use a rolling horizon stochastic optimization benchmark requiring many solves of DOASA. We don’t have access to SDDP. We seek ways that SDDP can be improved. DOASA

EPOC Winter Workshop, October 26, 2010 Slide 4 of 31 Source: CC Report, p 200 Counterfactual 1 The Wolak benchmark

EPOC Winter Workshop, October 26, 2010 Slide 5 of 31 What is counterfactual 1? –Fix hydro generation (at historical dispatch level). –Simulate market operation over a year with thermal plant offered at short-run marginal (fuel) cost. –“The Appendix of Borenstein, Bushnell, Wolak (2002)* rigorously demonstrates that the simplifying assumption that hydro-electric suppliers do not re-allocate water will yield a higher system-load weighted average competitive price than would be the case if this benchmark price was computed from the solution to the optimal hydroelectric generation scheduling problem described above” [Commerce Commission Report, page 190]. (* Borenstein, Bushnell, Wolak, American Economic Review, 92, 2002) The Wolak benchmark

EPOC Winter Workshop, October 26, 2010 Slide 6 of 31 Counterfactual 1 What about uncertain inflows? wet dry Stochastic program counterfactual The optimal generation plan burns thermal fuel in stage 1 in case there is a drought in winter. The competitive price is high (marginal thermal fuel cost) in the first stage, but zero in the second (if wet). Counterfactual 1 In the year under investigation, suppose all generators optimistically predicted high inflows and used all their water in summer. They were right, and no thermal fuel was needed at all. Counterfactual prices are zero. summerwinter

EPOC Winter Workshop, October 26, 2010 Slide 7 of 31 Yearly problem represented by this system S N demand WKOHAWMAN H demand EPOC Counterfactual

EPOC Winter Workshop, October 26, 2010 Slide 8 of 31 Cost-to-go recursion DOASA

EPOC Winter Workshop, October 26, 2010 Slide 9 of 31 DOASA: Cutting planes define the future cost function DOASA

EPOC Winter Workshop, October 26, 2010 Slide 10 of 31 SDDP versus DOASA DOASA SDDP (literature) DOASA Fixed sample of N openings Fixed sample of forward pass scenarios (50 or 200) Resamples forward pass scenarios (1 at a time) High fidelity physical modelLow fidelity physical model Weak convergence testStricter convergence criterion Risk model (Guigues)Risk model (Shapiro)

EPOC Winter Workshop, October 26, 2010 Slide 11 of 31                   p 11 p 13        p 12 How DOASA samples the scenario tree

EPOC Winter Workshop, October 26, 2010 Slide 12 of 31          p 11 p 13 p 12 How DOASA samples the scenario tree

EPOC Winter Workshop, October 26, 2010 Slide 13 of 31                            p 11 p 13 p 21        How DOASA samples the scenario tree

EPOC Winter Workshop, October 26, 2010 Slide 14 of 31 DOASA run times

EPOC Winter Workshop, October 26, 2010 Slide 15 of 31 Why do it this way? Lower bounds converge faster

EPOC Winter Workshop, October 26, 2010 Slide 16 of 31 Why do it this way? Upper bound convergence: 5000 forward simulations

EPOC Winter Workshop, October 26, 2010 Slide 17 of 31 Takeaways In this case terminating SDDP after 4, or 5, or even 10 iterations (of 200 scenarios each) does NOT guarantee a close to optimal policy. Confidence intervals with 200 scenarios are 5 times bigger than with 5000 scenarios. Single forward pass is better as it does not duplicate cut evaluation. Iterations slow down as cut sets increase. Cut-set reduction needed. SDDP

EPOC Winter Workshop, October 26, 2010 Slide 18 of 31 Rolling horizon counterfactual –Set s=0 –At t=s+1, solve a DOASA model to compute a weekly centrally-planned generation policy for t=s+1,…,s+52. –In the detailed 18-node transmission system and river-valley networks successively optimize weeks t=s+1,…,s+13, using cost-to-go functions from cuts at the end of each week t, and updating reservoir storage levels for each t. –Set s=s+13. Application to NZEM

EPOC Winter Workshop, October 26, 2010 Slide 19 of 31 We simulate an optimal policy in this detailed system MANHAW WKO Application to NZEM

EPOC Winter Workshop, October 26, 2010 Slide 20 of 31 Gas and diesel industrial price data ($/GJ, MED) Application to NZEM

EPOC Winter Workshop, October 26, 2010 Slide 21 of 31 Heat rates Application to NZEM

EPOC Winter Workshop, October 26, 2010 Slide 22 of 31 Load curtailment costs Application to NZEM

EPOC Winter Workshop, October 26, 2010 Slide 23 of 31 Market storage and centrally planned storage New Zealand electricity market

EPOC Winter Workshop, October 26, 2010 Slide 24 of 31 New Zealand electricity market =(NZ)$12.9 million per year (=2.8% of historical fuel cost) Estimated daily savings from central plan

EPOC Winter Workshop, October 26, 2010 Slide 25 of 31 Savings in annual fuel cost Total fuel cost = (NZ)$400-$500 million per annum (est) Total wholesale electricity sales = (NZ)$3 billion per annum (est) New Zealand electricity market

EPOC Winter Workshop, October 26, 2010 Slide 26 of 31 The next steps How does risk aversion affect prices and efficiency? How to model this? Use CVaR (Rockafellar and Urysayev, 2000) Actually, need a time-staged version of this. (Ruszczynzki, 2010), (Shapiro, 2010) Application to NZEM

EPOC Winter Workshop, October 26, 2010 Slide 27 of 31 CVaR 1-  = Conditional value at risk (tail average) Application to NZEM 90% 10% VaR 0.9 = $420M CVaR 0.9 = $460M

EPOC Winter Workshop, October 26, 2010 Slide 28 of 31 Average 2006 storage trajectories minimizing (1- )E[Z]+ CVar(Z) A risk-averse central planner

EPOC Winter Workshop, October 26, 2010 Slide 29 of 31 “Fuel and shortage cost – residual water value” CDF A risk-averse central planner 0 1

EPOC Winter Workshop, October 26, 2010 Slide 30 of 31 Conclusions DOASA is well-tested tool for benchmarking. We now have a good empirical understanding of convergence behaviour. We can model risk aversion effectively. Next steps –include inflow data –simulate central plans with different levels of risk aversion –How much risk can be avoided for $50M fuel cost? –Examine winter 2008 in more detail – especially price outcomes. Interested in feedback from participants – is this worth pursuing? If so how should industry fund it?