1. SPXI Tutorial, August 26, 2007 Andy Philpott The University of Auckland www.esc.auckland.ac.nz/epoc Stochastic Optimization in Electricity Systems

2. SPXI Tutorial, August 26, 2007 Electricity optimization Optimal power flow [Wood and Wollenberg, 1984,1996, Bonnans, 1997,1998] Economic dispatch [Wood and Wollenberg, 1984,1996] Unit commitment Lagrangian relaxation [Muckstadt & Koenig, 1977, Sheble & Fahd, 1994] Multi-stage SIP [Carpentier et al 1996, Takriti et al 1996, Caroe et al 1999, Romisch et al 1996-] Market models [Hobbs et al, 2001, Philpott & Schultz, 2006] Hydro-thermal scheduling Dynamic programming [Massé *, 1944, Turgeon, 1980, Read,1981] Multi-stage SP [Jacobs et al, 1995] SDDP [ Pereira & Pinto, 1991] Market models [Scott & Read, 1996, Bushnell, 2000] Capacity expansion of generation and transmission LP [Massé & Gibrat, 1957] SLP [Murphy et al, 1982] Multi-stage SP [Dantzig & Infanger,1993] Multi-stage SIP [Ahmed et al, 2006, Singh et al, 2006] Market models [Murphy & Smeers, 2005] * P. Massé, Applications des probabilités en chaîne à lhydrologie statistique et au jeu des réservoirs Journal de la Société de Statistique de Paris, 1944

3. SPXI Tutorial, August 26, 2007 Uncertainty in electricity systems System uncertainties Long-term electricity demand (years) Inflows to hydro-electric reservoirs (weeks/months) Short-term electricity demand (days) Intermittent (e.g. wind) supply (minutes/hours) Plant and line outages (seconds/minutes) User uncertainties (various time scales) Electricity prices Behaviour of market participants Government regulation

4. SPXI Tutorial, August 26, 2007 What to expect in this talk… I will try to address three questions: –What stochastic programming models are being used by modellers in electricity companies? –How are they being used? –What will be the features of the next generation of models? I will not talk about financial models in perfectly competitive markets (see previous tutorial speakers). I will (probably) not talk about capacity expansion models. Warning: this is not a how-to-solve-it tutorial.

5. SPXI Tutorial, August 26, 2007 Economic dispatch model

6. SPXI Tutorial, August 26, 2007 Uncertainty in economic dispatch Plant and line outages (seconds/minutes) –Spinning reserve (N-1 security standard) Uncertain demand/supply(e.g. wind) –Frequency keeping stations (small variations) –Re-dispatch (large variations) –Opportunity for stochastic programming (see Pritchard et al WIND model)

7. SPXI Tutorial, August 26, 2007 Unit commitment formulation

8. SPXI Tutorial, August 26, 2007 Stochastic unit commitment model

9. SPXI Tutorial, August 26, 2007 Lagrangian relaxation decouples by unit See sequence of papers by Romisch, Growe-Kuska, and others (1996 -)

10. SPXI Tutorial, August 26, 2007 Hydro-thermal scheduling

11. SPXI Tutorial, August 26, 2007 Hydro-thermal scheduling literature Dynamic programming Massé ( 1944) * Turgeon (1980) Read (1981) Multi-stage SP Jacobs et al (1995) SDDP Pereira & Pinto (1991) Market models Scott & Read (1996) Bushnell (2000) * P. Massé, Applications des probabilités en chaîne à lhydrologie statistique et au jeu des réservoirs Journal de la Société de Statistique de Paris, 1944

12. SPXI Tutorial, August 26, 2007 (Over-?) simplifying assumptions Small number of reservoirs (<20) System is centrally dispatched. Relatively complete recourse. Stage-wise independence of inflow process. A convex dispatch problem in each stage.

13. SPXI Tutorial, August 26, 2007 p 12 p 11 p 13 p 21

14. SPXI Tutorial, August 26, 2007 Outer approximation

15. SPXI Tutorial, August 26, 2007 Outer approximation of C t+1 (y) Θ (t+1) Reservoir storage, x(t+1) θ t+1 α t+1 (k) + β t+1 (k) T y, k

16. SPXI Tutorial, August 26, 2007 Cut calculation

17. SPXI Tutorial, August 26, 2007 Sampling algorithm

18. SPXI Tutorial, August 26, 2007 p 11 p 13 p 12

19. SPXI Tutorial, August 26, 2007 p 11 p 13 p 12

20. SPXI Tutorial, August 26, 2007 p 11 p 13 p 21

21. SPXI Tutorial, August 26, 2007 p 11 p 13 p 21

22. SPXI Tutorial, August 26, 2007

23. Case study: New Zealand system HVDC line MANHAW TPO

24. SPXI Tutorial, August 26, 2007 A simplified network model S N demand TPOHAWMAN

25. SPXI Tutorial, August 26, 2007 2005-2006 policy simulated with historical inflow sequences

26. SPXI Tutorial, August 26, 2007 Computational results: NZ model 10 reservoirs 52 weekly stages 30 inflow outcomes per stage Model written in AMPL/CPLEX Takes 100 iterations and 2 hours on a standard Windows PC to converge Larger models have slow convergence

27. SPXI Tutorial, August 26, 2007 Computational results: Brazilian system 283 hydro plants AR-6 streamflow model –about two thousand state variables 271 thermal plants 219 stages 80 sequences in the forward simulation 30 scenarios (openings) for each state in the backward recursion 7 iterations 11 hours CPU (Pentium IV-HT 2.8 GHz 1 Gbyte RAM ) Source: Reproduced with permission of Luiz Barossa, PSR

28. SPXI Tutorial, August 26, 2007 Electricity pool markets Chile (1970s) England and Wales (1990) (NETA 2001) Nordpool (1996) New Zealand (1996) Australia (1997) Colombia, Brazil, … Pennsylvania-New Jersey-Maryland (PJM) New York (1999) New England (1999) Ontario (May 1, 2002) Texas (ERCOT, full LMP by 2009)

29. SPXI Tutorial, August 26, 2007 Uniform price auction (single node) price quantity price quantity combined offer stack demand p price quantity T 1 (q) T 2 (q) p

30. SPXI Tutorial, August 26, 2007 Nodal dispatch-pricing formulation p q T m (q) [ i ]

31. SPXI Tutorial, August 26, 2007 Residual demand curve for a generator S(p) = total supply curve from other generators D(p) = demand function c(q) = cost of generating q R(q,p) = profit = qp – c(q) Residual demand curve = D(p) – S(p) p q Optimal dispatch point to maximize profit

32. SPXI Tutorial, August 26, 2007 A distribution of residual demand curves (Residual demand shifted by random demand shock ) D(p) – S(p) + p q Optimal dispatch point to maximize profit

33. SPXI Tutorial, August 26, 2007 One supply curve optimizes for all demand realizations The offer curve is a wait-and-see solution. It is independent of the probability distribution of

34. SPXI Tutorial, August 26, 2007 This doesnt always work There is no nondecreasing offer curve passing through both points. Optimization in this case requires a risk measure. We will use the expectation of profit with respect to the probability distribution of.

35. SPXI Tutorial, August 26, 2007 p q If (S-D) -1 is a log concave function of q and c(q) is convex then a single monotonic supply curve exists that maximizes profit for all realizations of. Monotonicity Theorem [Anderson & P, 2002]

36. SPXI Tutorial, August 26, 2007 The market distribution function [Anderson & P, 2002] p q quantity price Define: (q,p) = Pr [D(p) + – S(p) < q] = F(q + S(p) – D(p)) = Pr [an offer of (q,p) is not fully dispatched] = Pr [residual demand curve passes below (q,p)] S(p) = supply curve from other generators D(p) = demand function = random demand shock F = cdf of random shock

37. SPXI Tutorial, August 26, 2007 q(t) p(t) quantity price Expected profit from curve (q(t),p(t))

38. SPXI Tutorial, August 26, 2007 Finding empirical Use small dispatch model Aggregated demand DC-load flow dispatch Piecewise linear losses Solved in ampl/cplex Draw a sample from demand Draw a sample from other generators offers Solve dispatch model with different offers q Increment the locations where dispatch occur by 1

39. SPXI Tutorial, August 26, 2007 Estimation of using simulation Dispatch count on segment increases by 1 Sampled residual demand curve

40. SPXI Tutorial, August 26, 2007 The real world Transmission congestion gives different prices at different nodes. Generators own plant at different nodes. Generators in New Zealand are vertically integrated with electricity retailers, with demand at a different node. Generators have contracts with purchasers at different nodes. Maintenance and outages affect generation and transmission capacity.

41. SPXI Tutorial, August 26, 2007 Contracts A contract for differences (or hedge contract) for a quantity Q at an agreed strike price f is an agreement for one party (the contract holder) to pay the other (the contract writer) the amount Q(f- ) where is the electricity price at an agreed node. A generator having written a contract for Q seeks to maximize E[R(q,p)] = E[qp - c(q) + Q(f- )]

42. SPXI Tutorial, August 26, 2007 Generators real objective Owner of HLY station might want to maximize gross revenue at HLY + TOK –$35/MWh fuel cost at HLY –cost of purchases to cover retail base of 25% at OTA 5% at ISL 5% at HWB accounting for hedge contracts at $50/MWh of 250MW at OTA 150MW at HAY 50 MW at HWB (Numbers are illustrative only!)

43. SPXI Tutorial, August 26, 2007 Implementation in the real world BOOMER code [Pritchard, 2006] Single period/single station simulation/optimization model. Construct discrete on a rectangular grid. For every grid segment record all the relevant dispatch information (e.g. nodal prices at contract nodes) Use dynamic programming to construct a step function maximizing expected profit. A longest path problem through acyclic directed graph, where increment on each edge is the overall profit function times the probability of being dispatched on this segment

44. SPXI Tutorial, August 26, 2007 Longest path gives maximum expected profit

45. SPXI Tutorial, August 26, 2007 without retail and contracts with retail and 450MW of contracts

46. SPXI Tutorial, August 26, 2007 with retail customers moved to be more remote

47. SPXI Tutorial, August 26, 2007 What is wrong with this model? Single period Competitors response not modelled Extreme solutions: no comfort factor Can be used as a benchmark for traders

48. SPXI Tutorial, August 26, 2007 Challenges for SP Electricity systems have been a happy hunting ground for stochastic optimization. What are the SP success stories in electricity? Tractability is only part of the story – model veracity is more important. In markets the dual problem is as important as the primal (e.g. WIND model). Are the assumptions of the models valid e.g. perfect competition? Are the answers simple enough to verify (e.g. by out-of- sample simulation)? Models are used differently from their intended application.

49. SPXI Tutorial, August 26, 2007 The End