1. Output URL Bidding Panagiotis Papadimitriou, Hector Garcia-Molina, (Stanford University) Ali Dasdan, Santanu Kolay (Ebay Inc) Related papers: VLDB 2011, InfoLab TR-939, AdAuctions 2009

2. Search Engine Results Page (SERP) Organic Results Sponsored Ads Query Sponsored Search Ads

3. Keyword Bidding Advertiser Search Engines the social network lord of the rings the matrix lotr III... # keywords = ~ 10K KEYWORDS

4. Example SERPs en.wikipedia.org/wiki/The_Social_Network www.imdb.com/title/tt1285016/ www.imdb.com/title/tt133093/ en.wikipedia.org/wiki/The_Matrix en.wikipedia.org/wiki/The_Lord_of_the_rings www.imdb.com/title/tt167260/ www.imdb.com/title/tt120737/ the social network the matrix the lord of the rings lotr iii

5. Output Bidding Advertiser Search Engines imdb.com AND wikipedia.org # URLs = 2 URLs

6. Outline Architectures Bid Language Output bid/expression generation Spill Evaluation Experiments

7. Architectures Current Search Engine Architecture

8. Architectures Serialization Overview – First, retrieve organic results – Then, retrieve ads Pros – Simplicity Cons – Results Latency O: Organic Search System S: Sponsored Search System SERP

9. Architectures Pipelining Split organic search system to – Or: retrieval subsystem (retrieve relevant docs) – Op: post-processing subsystem (create result snippets) Op and S run in parallel Pros – No additional latency Cons – Sponsored search system depends on organic system O: Organic Search System = Or + Op S: Sponsored Search System SERP

10. Architectures Parallelization URLs with ads are known a priori S can use – Or’: Or replica that indexes only URLs with ads Pros – No additional latency – Independent organic and sponsored search system Cons – More resources O: Organic Search System (Or + Op) S: Sponsored Search System Or’: Small replica of Or V: Ad validation SERP

11. Bid Language Model Output Expression – e.g., a := (u 1  u 2 )  u 3  (h 1  h 2 ) – u: URL e.g., en.wikipedia.org/wiki/The_Social_Network – h: host e.g., en.wikipedia.org Questions – URLs or hosts or both? – complex or simple?

12. Output Expression Generation Motivation Use existing keyword campaigns to generate realistic output expressions to study The social network lord of the rings the matrix lotr III … Output Expression Generator imdb.com AND wikipedia.org

13. Problem – INPUT: keyword set R – OUTPUT: expression a that “covers” R Candidate solutions – a 1 := u 1  u 2  u 3 – a 2 := u 1  u 4 – a 3 := u 5 Output Expression Generation Motivating Example

14. Compactness Contain few URLs Spill minimization: Do not match “irrelevant” queries Output Expression Generation Objectives Output Expression Size |a| Spill spill(a,R) a 1 := u 1  u 2  u 3 3{} a 2 := u 1  u 4 2{q 5 } a 3 := u 5 1{q 4,q 5, q 6 }

15. Query Set Output Cover minimize γ|a| + (1-γ) |spill(a, R)| subj. to m(a,q),  q  R γ : regularization parameter Related to – Set Cover – Red-Blue Set Cover Output Expression Generation Problem Statement

16. Output Expression Generation Greedy Algorithm Pre-compute – C[u]: Queries covered by URL u – S[u]: Spill of URL u w.r.t. R Algorithm

17. Spill Evaluation Spill queries may be relevant to R Divide spill(a, R) to – positive: relevant – negative: irrelevant Use query clustering for evaluation Example: – a := u 2  u 3 – Positive spill = {q 1 } – Negative spill = {q 5 }

18. Experimental Evaluation Goals Compare output URL bidding variations – 1-URL, 2-URL, 3-URL e.g, 2-URL: use only URLs, up to 2 URLs in a disjunct – 1-host, 2-host, 3-host – 1-mixed, 2-mixed Comparison criteria – Compactness – Spill tradeoff – Spill Evaluation

19. Experimental Evaluation Setup Dataset (from Yahoo query logs) – 12,931,117 queries – 62,666,514 URLs – 7,185,392 hosts – 2,251 ads Process – For each variation (1-URL, 2-URL, …) For different γ values – Generate output expressions for all 2,251 ads

20. Experimental Evaluation Compactness vs Spill

21. Experimental Evaluation Positive and Negative Spill

22. Experimental Evaluation Summary Compactness-spill trade-off – Using both URLs/hosts outperform other options – Up to 2 conjuncts in a disjunct is sufficient Spill evaluation – Output expressions can bring additional queries Other experiments in Combining keyword and output bidding – Output expression are suitable for half of the keywords – Using only hosts seems to be sufficient

23. Conclusions Output URL bidding can be implemented efficiently Advantages over keyword bidding – Bid Compactness – More relevant queries

24. THANK YOU!