1 Market Based Control of Complex Computational Systems Nick Jennings

2 The Complex Systems Challenge Building software that operates effectively in environments that: –Have no centralised control –Are highly interconnected –Are in constant state of flux –Are highly unpredictable –Involve multiple, individually-motivated actors

3 The Complex Systems Landscape Grid Computing Semantic WebWeb Services Agent Based Computing Service description Service discovery Service composition Flexible interoperation & reasoning in heterogeneous environments Robust, large scale open systems Autonomy Rich interactions Brain meets Brawn Semantic integration Semantic Grid OGSA uses WS standards Pervasive Computing Peer-to- Peer eCommerceAutonomic Computing

4 Entities offer services in an institutional setting Entities connect to services –Service discovery –Service composition –Service procurement Entities enact services –Flexible & context sensitive service delivery The Computational Model Agent Environment Sphere of influence Electronic institution Interaction Organisational relationship (Jennings, 2000 & 2001)

5 encapsulated computer system, situated in some environment, and capable of flexible autonomous action in that environment in order to meet its objectives Agents as Service Providers & Consumers

6 encapsulated computer system, situated in some environment, and capable of flexible autonomous action in that environment in order to meet its objectives Agents as Service Providers & Consumers control over internal state and over own behaviour

7 encapsulated computer system, situated in some environment, and capable of flexible autonomous action in that environment in order to meet its objectives Agents as Service Providers & Consumers control over internal state and over own behaviour experiences environment through sensors and acts through effectors

8 encapsulated computer system, situated in some environment, and capable of flexible autonomous action in that environment in order to meet its objectives Agents as Service Providers & Consumers reactive: respond in timely fashion to environmental change proactive: act in anticipation of future goals control over internal state and over own behaviour experiences environment through sensors and acts through effectors

9 Negotiation as de facto Form of Interaction Agree appropriate service contracts –Service composition –Service procurement Fixed price offerings –Catalogues Dynamic pricing –Negotiations –Auctions Historical precedent Economic efficiency

10 permissible participants e.g. buyers, sellers & third parties interaction states e.g. accepting bids, auction closed events causing state transitions e.g. bid, time out, bid accepted valid actions bid, ask, propose, accept, reject, counter-proposal, critique reward structures who pays & who gets paid for what Computational Service Economies Mechanism Design rules of the game (Dash et al., 2003)

11 Computational Service Economies shaped by interaction protocol decision making employed to achieve trading objectives –from very simple to very complex maximise benefit –to self (self interest) and/or –to group (social welfare) Mechanism Design rules of the gamehow to succeed in the game Agent Strategies (Dash et al., 2003) permissible participants e.g. buyers, sellers & third parties interaction states e.g. accepting bids, auction closed events causing state transitions e.g. bid, time out, bid accepted valid actions bid, ask, propose, accept, reject, counter-proposal, critique reward structures who pays & who gets paid for what

12 Computational Service Economies shaped by interaction protocol decision making employed to achieve trading objectives –from very simple to very complex maximise benefit –to self (self interest) and/or –to group (social welfare) Mechanism Design rules of the gamehow to succeed in the game Agent Strategies (Dash et al., 2003) permissible participants e.g. buyers, sellers & third parties interaction states e.g. accepting bids, auction closed events causing state transitions e.g. bid, time out, bid accepted valid actions bid, ask, propose, accept, reject, counter-proposal, critique reward structures who pays & who gets paid for what Game theory analyses interactions to determine likely outcomes and equilibria

13 The Market-Based Control Project Market-Based Control (MBC): –paradigm for controlling computer systems using economically-inspired techniques Market mechanisms used to: –generate and predict emerging system properties, although decisions are made independently by local agents that each have their own aims and objectives –a market is a self-organising system, directed by mechanism The proposition: –MBC is good for effectively controlling and managing complex, adaptive, distributed computational systems

14 Objectives Develop and evaluate core MBC technologies Automated mechanism design –Automate design of market mechanisms to achieve a desired set of global goals –Adapt to a changing environment and changing (priority of) objectives –Predict and automate design of agent strategies Apply MBC solutions to design and manage complex, distributed computational systems

15 Project Applications Utility data centres –MBC to allocate computational resources & achieve a robust, scalable service Distributed content delivery within p2p networks –MBC to regulate sharing of content Decentralised control and scheduling of multiple robots –MBC to provide incentives for cooperation and to achieve global goals

16 Research Highlights Competing sellers in online auctions Strategies for bidding in multiple auctions Empirical game theory to select mechanisms and strategies for complex markets Adaptive auctions

17 Research Highlights Competing sellers in online auctions Strategies for bidding in multiple auctions Empirical game theory to select mechanisms and strategies for complex markets Adaptive auctions

18 Often strong competition among sellers in online auctions –How many eBay auctions yesterday? A)10 B)100 C)1000

19 Often strong competition among sellers in online auctions Sellers choice of mechanism & auction parameters affect buyers choice of seller –How should bidder choose between auctions/sellers? –How should a seller set its parameters? Focus on sellers reserve price & sealed-bid auctions

20 Mediator Model of Competing Sellers Auction Buyers Seller Set & announce Auction Fees Set & announce Reserve Price Select seller Bid in auctions Seller Auction

21 Shill Bidding Competing sellers reduces optimal reserve price and expected revenue (compared to isolated auctions) Avoid by shill bidding: –Seller disguised as buyer to bid in own auction. Illegal and undesired, but hard to detect –But mediator can use auction fees to deter it Use Evolutionary Simulation to: – Evaluate effectiveness of different types of auction fees in deterring shill bidding – Measure market efficiency

22 Results with Auction Fees Fraction of auctions won by shill bids Allocative efficiency CP= closing price RD = difference between reserve and closing prices

23 Observations Competition among sellers affects choice of mechanism and auction parameters –Important to take competition into account when designing mechanisms and bidder strategies Sellers can decide to shill bid in order to improve profits Mediator (such as eBay) can deter shill bidding and increase efficiency by setting appropriate auction fees

24 International Competition Made proposal to have new game in the Trading Agents Competition Foundation –TAC Market Design Reverse Trading Agents Competition –Design mechanisms with varying: Clearing policy Information revelation policy Auction fees

25 Research Highlights Competing sellers in online auctions Strategies for bidding in multiple auctions Empirical game theory to select mechanisms and strategies for complex markets Adaptive auctions

26 Bidding in Multiple Auctions Different start/finish times –Simultaneous, sequential, or hybrid Heterogeneous: –N single-unit auctions –1 st /2 nd price sealed bid, English or Dutch –Each can have different number of bidders Multiple items Find optimal best response simultaneous sequential hybrid

27 Heuristic Strategies Setting too complex to analyse theoretically and find optimal strategies Heuristic strategies: –Choose the thresholds Single auction dominant strategy (DOM) Equal threshold (EQT) –Choose the auction Exhaustive search (ES) Knapsack utility approximation search (KS) Trade-off between speed and complexity

28 Heuristics close to optimal for this restricted case EQT better than DOM KS much more computationally efficient than ES

29 Research Highlights Competing sellers in online auctions Strategies for bidding in multiple auctions Empirical game theory to select mechanisms and strategies for complex markets Adaptive auctions

30 Empirical Game Theory Game Theory is a mathematical theory which underpins auction- and mechanism-design –very powerful and, at least in theory, can tell us what are the optimal mechanism and strategies. But some markets too complex to analyse in practice using game theory. –too many participants and too many possible moves. Evolutionary methods do not always converge on robust strategies Empirical Game Theory: –emerging field combines principled game-theoretic analysis together with computer simulation methods. –amenable to automation, so it may be used by agents themselves to decide on market mechanisms.

31 Empirical Game Theory Analysing strategies in Double Auctions Find payoffs for strategies by repeated simulations Find mixture of these pure strategies that constitute a evolutionary game-theoretic equilibrium

32 Research Highlights Competing sellers in online auctions Strategies for bidding in multiple auctions Empirical game theory to select mechanisms and strategies for complex markets Adaptive auctions

33 Discrete Bid English Auctions Fixed bid increment

34 Research Questions What effect do these discrete bid levels have on the auction properties? How should the auctioneer determine the discrete bid levels to use in any situation in order to maximise his revenue?

35 E º = m X i = 0 e º [ F ( l i+ 1 ) ¡ 1 ] ¡ e º [ F ( l i ) ¡ 1 ] F ( l i + 1 ) ¡ F ( l i ) h l i £ 1 ¡ F ( l i ) ¤ ¡ l i + 1 £ 1 ¡ F ( l i + 1 ) ¤ i We calculate the auction revenue by considering the probability of these three cases: Gives the final result: We can optimise this expression (analytically or numerically) to find the optimal discrete bid levels. Calculating Auction Revenue E = m X i = 0 l i [ P ( case 1 ; l i ) + P ( case 2 ; l i ) + P ( case 3 ; l i )] Discrete bid levels implemented Mean number of bidders Bidders valuation distribution l 0 ::: l m (David et al., 2005)

36 Optimal Bid Levels Uniform bidders valuation distribution Reserve price increases Bid increment decreases

37 Optimal Bid Levels Increases expected revenue. Decreases expected auction duration. Increases expected auction efficiency. Optimal discrete bid levels Fixed bid increment Optimal discrete bid levels Fixed bid increment Optimal discrete bid levels Fixed bid increment

38 Learning Auction Parameters To calculate optimal discrete bid levels we must know: –The bidders valuation distribution. –The number of participating bidders. Typically we do not know these parameters. –However, we can use Bayesian Machine Learning to estimate these parameters – online.

39 Parameter Estimates Optimal Bid Levels Auction Prior Knowledge Auction Closing Price Learning Auction Parameters (Rogers et al., 2005)

40 Bayesian machine learning is attractive for this application: –Makes use of our knowledge of how the auction closes. –Allows us to incorporate prior knowledge or experience. –Makes efficient use of the sparse training data (observations of auctions). –Computationally efficient (no need to maximise multi- dimensional functions). Bayesian Machine Learning

41 Learning the Number of Bidders

42 Learning the Number of Bidders

43 Conclusions MBC prima facie candidate for controlling complex, distributed computational systems with autonomous self-interested components: –Computational game theory / Mechanism design –Evolutionary algorithms / Machine learning –Decision theory Ongoing research and goals: –design of mechanisms and strategies for MBC –gain understanding of and predict dynamic properties of market-based computational systems –develop formal representation and tools Ultimate goal: automated mechanism design

44 Partners