1. TFG - MARA, Budapest, September 20051 Modeling Complex Multi-Issue Negotiations Using Utility Graphs Valentin Robu, Koye Somefun, Han La Poutré CWI, Center for Mathematics and Computer Science, Amsterdam, The Netherlands

2. TFG - MARA, Budapest, September 20052 Multi-issue (multi-item) negotiation Negotiation = method of competitive (or partially cooperative) allocation of goods, resources, tasks between agents Applications: E-commerce: Bundling can be an effective method to increase sales (use in recommender systems) High degree of customization – possible through negotiations Logistics: mechanism for task allocation Many deals are negotiated bilaterally or in closed groups of companies (e.g. transportation contracts) Utility functions are not (or partially) revealed => indirect revelation mechanism Search with incomplete information

3. TFG - MARA, Budapest, September 20053 Utility functions for multi-issue negotiations Linearly additive: Linear combination of issue utilities: Search space is structured -> more accesible to heuristics [Faratin Sierra & Jennings. 2002], [Jonker & Robu 2004], [Coehoorn & Jennings 2004] [Gerding & La Poutre, 2004] “Auction-type”: XOR of ANDs K-additive: Captures local substitutability/complementarity effects between k issues Finding optimal allocation can become hard even for the 2- additive case Exiting solutions: assume a trusted mediator, computationally expensive (3000-5000 bids for 50 issues) [Klein, Faratin, Sayama & Bar-Yam, 2003] [Lin 2004]

4. TFG - MARA, Budapest, September 20054 Utility graphs: basic ideas Inspiration: probabilistic graphical models Each node = one issue under negotiation (or item in a bundle) Nodes grouped into clusters of connected nodes Cost of representation Exponential in size of the cluster Linear in the number of clusters Use in negotiation Opponent modelling: seller maintains & updates a model of buyer’s preferences

5. TFG - MARA, Budapest, September 20055 Utility graphs: an example Global utility is a sum of utility over clusters, rather than individual issues Buyer - cluster potentials: u(I1) = $7, u(I2) = $5, u(I3) = $0 u(I4) = $0, u(I1, I2)= - $5, u(I2, I3)=$4,u(I2, I4)=$4 Seller - all items have cost $2. u BUYER (I1=1, I2=0, I3=1, I4=0) = $7 Gains from Trade = Buyer_utility – Seller_Cost Optimal combination? GT(I1=0, I2=1, I3=1, I4=1)=$13 - 3*$2 = $7

6. TFG - MARA, Budapest, September 20056 Utility graphs: Use in negotiation Bundles with maximal G.T.  Pareto-optimal bundles [Somefun, Klos & La Poutré 2004] Seller keeps a model of the utility graph of the buyer and aims for a bundle with maximal GT After each counter-offer, he updates this model (true graph of the buyer remains hidden) Seller knows a super-graph of possible buyer utility graphs (qualitative assumption)

7. TFG - MARA, Budapest, September 20057 Partitioning a utility graph Q: How to select the bundle with a maximal GT, with respect to a utility graph learned so far? A1 (Brute force answer): generate all possible bundles and select the best one. Complexity for 50 issues: 2 50 > 10 15 bundles A2: Partition the graph into sub-graphs Nodes belonging to more than 1 subgraph = cutset nodes For all possible instantiations of cutset nodes, compute local sub-bundle combination Merge them, such that a local optimum is achieved

8. TFG - MARA, Budapest, September 20058 Partitioning a utility graph (2) Complexity of exploring all bundles: 2 c * (2 p + 2 q ) Partitions can be found in polynomial time (always for graphs of tree-width 2)

9. TFG - MARA, Budapest, September 20059 Learning in utility graphs (1) Seller has a super-graph for possible inter- dependencies in the buyer population This graph contains tables for each cluster, with size 2 at the power of size of the cluster Initial values = proportional to the Hamming distance Values are adjusted as follows:, for the combination induced from buyer’s bid, for all other combinations

10. TFG - MARA, Budapest, September 200510 Learning: a simple example Two complementary issues: I1 and I2 I1I2time t t+1t+2 00000 01$7$8.4$10 10$5$4$3.2 11$17$13.6$10.9 Buyer asks, for several rounds: I1=0, I2=1 This combination gets updated with (1+α), the others with (1-α) Supposing costs are c(I1)=c(I2)=$3, α=0.2 the bundle with maximal GT changes from (1,1) to (0,1) after 2 steps

11. TFG - MARA, Budapest, September 200511 Learning in utility graphs (2) The cluster update factor is clique-specific: |C| = total number of cliques; α, β = learning parameters Where the clique Gains from Trade Ratio is defined as ratio of “local” (per clique) vs. total (bundle-wide) GT: We adjust the model more towards the other’s value for clusters which are less important, and less for the others

12. TFG - MARA, Budapest, September 200512 Experimental validation: set-up Graph with 50 issues, 28 clusters: 3 of size 4, 16 of size 3, 6 of size 2, 3 of size 1 Costs and strength of interdependencies: drawn from a independent, normal distributions (i.i.d-s): Means around 1*(Hamming Distance) Spreads between 0 and 5 => highly non-linear search space Results averaged for 100 tests/configuration

13. TFG - MARA, Budapest, September 200513 Experimental results

14. TFG - MARA, Budapest, September 200514 Negotiation part: Conclusions It is possible to reach Pareto-efficient outcomes reasonably fast, by exploiting the decomposable structure of utility functions Consequence: We can handle complex negotiations even in time constrained domains / with buyer impatience Assumption: A structure of the super-graph for the population of likely buyers Solution: collaborative filtering past negotiation data

15. TFG - MARA, Budapest, September 200515 Structure of the initial utility graph Preferences of buyers are in some way clustered Class (population) of buyers with similar preference structures => largely overlapping utility graphs Can we estimate which items can be potentially complementary/substitutable by looking at previous buying patterns? Collaborative filtering asks the same questions ! Not all relationships hold for all users – only a super-graph of these relationships is required

16. TFG - MARA, Budapest, September 200516 Architecture & simulation model view

17. TFG - MARA, Budapest, September 200517 Collaborative filtering: Overview Output recommendations to buyers, based on previous buy instances User-based: for each user, select a neighbourhood of users with a similar preferences Item-based: identify relationships between items, based on previous buying patterns In our case, recommendation step is completely replaced by negotiation => more customization possible

18. TFG - MARA, Budapest, September 200518 Step 1: Data preparation Items Previous negotiations I1I1 I2I2 I K...I 50 Neg. 10110 Neg. 2 … 1…1… 1…1… 0…0… 1…1… Neg. N (eg. N=2000) 1100 Negotiation outcomes matrix Item pairs I1I1 I2I2 I K...I 50 I1I1 N134…220 I2I2 134N…… IKIK ………… I 50 220……N 1-1 pairs: N i,j (1,1) 1-0 pairs: N i,j (0,1) 0-1 pairs: N i,j (1,0) 0-0 pairs: N i,j (0,0) Total no. buys (out of N) N 1 (1)N 2 (1)N K (1)..N 50 (1) 260130…50 4 Item-item matrixes

19. TFG - MARA, Budapest, September 200519 Step 2: Data analysis (1) Compute item-item similarity, based on the appearance data ItemI1I1 I K...I 50 I1I1 N…220 IKIK ……… I 50 220…N ItemI1I1 I K...I 50 I1I1 1…0.84 IKIK 0.23…… I 50 0.84…1 Number Buys / item N 1 (1)…N 50 (1) 26050 4 Item-item matrixes Total number buys/item Cosine / correlation matrix 2 matrixes for cosine- based similarity 1 matrix for correlation- based similarity

20. TFG - MARA, Budapest, September 200520 Criteria 1: Cosine-based similarity Measure of distance between the buying vectors for two items i, j Intuitive, but not so precise Complementarity effect: Substitutability effect:

21. TFG - MARA, Budapest, September 200521 Criteria 2: Correlation-based similarity Average buys per item: Similarity between items i and j:

22. TFG - MARA, Budapest, September 200522 Results: Correlation-based similarity

23. TFG - MARA, Budapest, September 200523 Conclusions & discussion Utility graphs efficient way to guide online learning of buyer preferences in electronic negotiations Learning a starting structure of these graphs – possible through collaborative filtering By combining the two techniques => relatively short negotiations (around 20 steps/50 issues) Intuition: we explicitly utilize the clustering effect between utility functions of typical buyers Personalization techniques used in collaborative filtering can be successfully combined with personalization through agent-mediated negotiation

24. TFG - MARA, Budapest, September 200524 Questions Thank you very much for your attention! Full paper(s) available from: homepages.cwi.nl/~robu