1. On Learning in Policy Space by Oligopolists in Electricity Markets
Steve Kimbrough
Fred Murphy

2. Organization of Presentation
Bidding process in day-ahead market
Supply function equilibria
Agent-based modeling
Bid representation
Results
Conclusions

3. Day-ahead Market Each firm bids a step-function supply curve
LSE’s offer demand curves
The supply and demand curves are put into an LP with transmission capacity constraints and the LP solved for the equilibrium

4. Our Day-ahead Market Demand curves are continuous
No transmission constraints
Firms bid step-function supply curves
Combine individual supply curves into single supply curve
Find market equilibrium, where curves cross

5. Alternative models of Equilibria
Cournot - bid quantities
Problem: utilities bid prices as well
Bertrand - bid prices
Problem: oddly discontinuous behavior
Add capacity game and same as Cournot
Supply function – bid a supply curve

6. Supply Function Equilibria
Klemperer and Meyer (1989), Green and Newbery (1993)
Literature uses continuous curves to derive results
Models have multiple equilibria unless severely restricted in functional form and/or domain

7. A Problem with all of the Standard Models
Firms are part of a community
The models assume myopic optimization and pure non-cooperative behavior
There are institutions where firms can talk without breaking the law
Leading companies look to the interests of the industry as well as the firm

8. Agent-based Modeling Two approaches Our approach is to
Agents are decisions/policies and fittest survive
Agents try different decisions and mostly go with the decisions that have better outcomes
Our approach is to
Give agents alternative objective functions
Let agents make decisions based on the values in the assigned objective function
Evaluate outcomes based on firm profitability

9. The Model of the Market Player objective functions Supply functions
Firm profitability
Industry profitability
Industry profitability subject to a “fair share” constraint
Supply functions
n plants, n steps
n plants, n+1 steps

10. The Model of the Market, cont.
Episode - one round of play
Players randomly adjust prices and quantities around trial values on steps
Epoch - a collection of episodes
Players evaluate the outcomes from the random trials using the assigned objective function
They adjust the trial values in the direction of the increased objective function

11. Monopolist, N
File: elec-Own-N.png

12. Monopolist, N+1
File: elec-Own-N+1.png

13. File: elec-Own-Own-N-N.png
Duopoly, Own-Own,N-N
Players find “Cournot” outcome. High-bidding player is exploited.
File: elec-Own-Own-N-N.png

14. File: elec-Own-Own-N1-N1.png
Duopoly, Own-Own, N+1-N+1
Players find “Cournot-plus” outcome. Neither player is exploited.
File: elec-Own-Own-N1-N1.png

15. File: elec-Industry-Industry-N1-N1.png
Duopoly, Ind-Ind, N+1-N+1
Players individually maximize industry profits and jointly find monopoly outcome.
File: elec-Industry-Industry-N1-N1.png

16. File: elec-Industry-Own-N1-N1.png
Duopoly, Ind-Own, N+1-N+1
Players find near-monopoly outcome. Cooperative, “Industry Returns” player, is exploited.
File: elec-Industry-Own-N1-N1.png

17. Duopoly, IndOwn-Own, N+1-N+1
Players find “Cournot-plus” outcome. Cooperative, “Industry Returns, s.t. Own Returns” player, is not exploited.
File: elec-IndustryOwn-Own-N1-N1.png

18. Duopoly, IndOwn-IndOwn, N+1-N+1
Players find monopoly outcome. Neither player is exploited. (But the customers are.)
File: elec-IndustryOwn-IndustryOwn-N1-N1.png

19. A Pattern
Classical results hold for learning agents who are very myopic.
Agents learn to collude tacitly by being less myopic and trading off exploration and exploitation, tilting more towards exploration. (“Exploring rationality”)
Simple tilts towards exploration are subject to exploitation.
There exist simple constraints on exploration that greatly reduce exposure to exploitation. In this mode agents may mutually achieve tacit collusion safely.

20. Conclusions
In repeated play the structure of the supply curves can be exploited to find the Pareto optimum
At the same time it is possible for a firm to protect itself from self-serving players
To get its fair share, each player has to make the other see the consequences of its actions on aggregate demand in its choices of prices and quantities