1. Energy Cost Optimization in Water Distribution Systems Using Markov Decision Processes Paulo T. Fracasso, Frank S. Barnes and Anna H. R. Costa University of Sao Paulo Department of Electrical and Computer Engineering Intelligent Techniques Laboratory

2. Agenda Anatomy of Water Distribution Systems Problem relevancy Markov Decision Process Modeling a Water Distribution System as an MDP Monroe Water Distribution System Experiment results Conclusions 1

3. Water distribution system It is a complex system composed by pipes, pumps and other hydraulic components which provide water supply to consumers. Focus of my work 2

4. Problem relevancy About 3% of US energy consumption (56 billion kWh) are used for drinking water (Goldstein and Smith, 2002). $2 billion/year Source: Electric Power Research Institute,1994. 3

5. MDP is a model for sequential decision making in fully observable environments when outcomes are uncertain. Advantages of MDP compared to other techniques:  Real world – operates in uncertain and dynamic domains  Planning – generates control policies to sequential decisions  Optimal solution – guarantees to achieve a higher future payoff Disadvantages of MDP:  Discrete domains (state and action)  Course of dimensionality Markov Decision Process - MDP 4

6.  MDP is defined as a tuple where:  S is a discrete set of states (can be factored in Nv features):  A is a discrete set of actions:  T is a transition function where  R is a reward function where Markov Decision Process - MDP 5

7.  Solving an MDP consists in finding a policy, which is defined as a mapping from states to actions, s.t.  Bellamn’s equation allows to break a dynamic optimization problem into simpler sub-problems:  The optimal value of the utility is:  The optimal policy are the actions obtained from : Markov Decision Process - MDP

8.  Topology of a typical water distribution system:  States (everything that is important to control):  Time – range: discrete:  Tank level – range: discrete: Water Distribution System modeled as an MDP 7

9.  Actions (what you can manipulate):  Triggered directly:  Associated with a VFD – range: discrete:  Transition function (how the system evolves):  Calculated by EPANET  Reward function (how much an action cost):  Consumption:  Demand: Water Distribution System modeled as an MDP 8

10. Markov Decision Processes Constraints Control policy Demand Electrical power Final result: Energy price schema 9 Water Distribution System modeled as an MDP

11. Understand MDP results  Control policy:  Maps state variables into a set of actions  States variables: everything that is important to control (tank level and time)  Set of actions: what you can manipulate (pumps)  Indicates controllability (avoid black region)  Correlated to demand curve Tank level Time 10

12. Understand MDP results  Controller:  Uses control policy map to produce actions  Actions are based just on tank level and time  Easy to implement and fast to run in PLC (lookup table) Tank level Time Pump trigger 11

13. Monroe Water Distribution System  Characteristics:  11 pumps  1 storage tank  4 pressure monitoring  40k people served  182 miles of pipes  Diameters varying from 2 to 42 inches 12

14. Monroe Water Distribution System 13  Demand curve (during summer season): Average: 6 700 GPM Minimum: 4 188 GPM Maximum: 8 389 GPM  Pressure restrictions (in PSI):  J-6: 65 ≤ P ≤ 70 ▪ J-131: 45 ≤ P ≤ 55  J-36: 50 ≤ P ≤ 60 ▪ J-388a: 40 ≤ P ≤ 90

15. Monroe Water Distribution System 14  Pumps (E2, E3, E4, E5, E6, E7, W8, W9, W10, W11 and W12):  Energy price schema:  On-peak (09:00 – 20:59): $0.04014/kWh  Off-peak (21:00 – 08:59): $0.03714/kWh  Demand (monthly): $13.75/kW

16. MDP apply to Monroe WDS Mathematical model:  Set of states: where and  Set of actions:  Transition function:  Reward function: Data flux diagram: EPANET DLL.INP FILEMATLAB 15

17. MDP results in Monroe WDS  Expected electrical power : 16 E5 and E7 consume 144.3kW W11, E2 and E6 consume 320.4kW

18. MDP results in Monroe WDS  Number of activated pumps (27 possibilities): 17 on={E2,E6}on{E5,E7} on={W12,E3,E4,E5}on={E2,E3,E4,E5}

19. MDP results in Monroe WDS  SCADA records:  obtained from historical data (July 6 th, 2010)  75% of WTP consumption is considered to be used in pump  One day is extrapolated to one billing cycle (30 days)  Both approaches started in the same level (19.3 ft) Energy expensesSCADA recordsMDPDifference Off-peak energy [$/month] 3 210.572 608.32-23.1% On-peak energy [$/month] 3 750.783 768.51+0.5% Demand [$/month] 3 836.253 603.67-6.5% Total [$/month] 10 797.609 980.50-8.2% 18

20. Conclusions  MDP avoids restrictions (level, pressure, and pumps) and reduces expenses with energy  To reduce energy consumption is different to reduce expenses with energy (demand is the biggest villain)  Summer season imposes small quantity of feasible actions  Verify if it is possible to reduce the number of pump combination  MDP policy is easy to implement in a non-intelligent device (PLC) 19

21. Contact Thank you for your attention PAULO THIAGO FRACASSO paulo.fracasso@usp.br Av. Prof. Luciano Gualberto, trav.3, n.158, sala C2-50 CEP: 05508-970 - São Paulo, SP - Brazil Phone: +55-11-3091-5397 20