Optimal Stopping

Have you searched for something lately? Can you give examples for what you’ve searcher for?

Searching What? Everything! Searching for a partner Searching for a job Searching for a product Searching for a parking space Searching for a java class (reuse) Search for a thesis advisor … The goal here is to optimize the process rather than ending up with the optimal search object

Example - Marriage Market legacy domain (search “pioneers”) f(x) Lifetime Utility

Statistics Reminder given a continuous random variable X, we denote: The probability density function, pdf as f(x). (also known as the probability distribution function and the probability mass function) The cumulative distribution function, cdf, as F(x). The pdf and cdf give a complete description of the probability distribution of a random variable

PDF The pdf of X, is a function f(x) such that for two numbers, a and b with a≤b: That is, the probability that X takes on a value in the interval [a, b] is the area under the density function from a to b.

CDF The cdf is a function F(x), defined for a number x by: That is, for a given value x, F(x) is the probability that the observed value of X will be at most x.

דוגמה: התפלגות אחידה f(x)=0.01 200 300

התפלגות בדידה במקום f(x) אנו מדברים על P(x) למשל בהטלת קוביה, P(2)=1/6

Example - Marriage Market legacy domain (search “pioneers”) f(x) Lifetime Utility Should I try to do better?

Can we do better? Yes we can! However, it has a cost Thus a search strategy is needed Strategy: (opportunities, time, cost)->(terminate, resume)

Search Characteristics A distribution of plausible opportunities The searcher is interested in exploiting one opportunity Unknown value of specific opportunities Search costs

Searching What? Application Cost Opportunity Marriage Market Time / money / loneliness Better partner Job Market Time / money / confidence Better job Product Time / money Better price / performance Parking time Closer parking space Looking for a thesis advisor Working with him a little More interesting thesis … Anyone searched for an apartment in her life? What made you take the one you are living in? Anyone sold an apartment in her life? What made you accept the “winning” bid? The key concept – don’t attempt to find the best opportunity, instead find the best policy

The search strategy After each draw, the searcher has a choice: Keep what he has Draw another opportunity from the distribution F(), at a cost c Notice: the net profit is a random variable whose value depends both on the actual draws and on his decisions to accept or reject particular opportunities

The Goal Maximize the expected value of the net profit Application Cost Opportunity Marriage Market Time / money / loneliness Better partner Job Market Time / money / confidence Better job Product Time / money Better price / performance Parking time Closer parking space

The optimal strategy Let V* be the expected profit if following the optimal strategy If infinite decision horizon: If rejecting an opportunity, he is in the same situation as a searcher who is starting anew: expect profit V* Therefore the optimal strategy is threshold based The reservation property of the optimal search rule is a consequence of the stationarity of the search problem (a searcher discarding an opportunity is in exactly the same position as before starting the search)

Example - Marriage Market f(x) Reservation Value - x Lifetime Utility Should I try to do better? Ring the door bell - just before a blind date In a simple infinite horizon model - doesn’t depend on history

Example - Marriage Market f(x) Resume Search - sample one more Terminate Search Reservation Value - x Lifetime Utility Should I try to do better? Ring the door bell - just before a blind date In a simple infinite horizon model - doesn’t depend on history

The optimal Reservation Value f(x) Resume Search - sample one more Terminate Search Lifetime Utility x Distribution of utilities in the environment (p.d.f / c.d.f) Expected utility when using reservation value x Search cost F(x)

The Reservation Value Concept Distribution of utilities in the environment (p.d.f / c.d.f) Expected utility when using reservation value x Search cost F(x) What is x that maximizes V(x)?

The Reservation Value Concept

Example - Marriage Market f(x) Resume Search - sample one more Terminate Search Reservation Value - x Lifetime Utility The expected utility from accepting only “better” partner than the optimal reservation value woman will yield an expected overall utility equal to the “lowest’ partner I’m willing to accept Should I try to do better? Ring the door bell - just before a blind date At the end of the evening the question is: “should I try and do better” (I.e. try to find a better wife)

Some more interesting interpretations

Some more interesting interpretations (2) Stop searching and keeping x* Searching exactly one more time

Myopic rule Searcher cares only about whether or not he wants the opportunity now Therefore, we don’t care for the recall option

Also notice that… and: Bernoulli trial is an experiment whose outcome is random and can be either of two possible outcomes, "success" and "failure".

Problem 1 You are about to purchase an iPod touch over the internet You estimate the price distribution of the product over the different sellers to be uniform between 200-300 dollars You can search by yourself, by visiting different web-sites – the cost of time for obtaining a price quote is $1 How will you search? What will be your expected cost? What’s the mean of the number of merchants you’ll visit?

Solution f(x) 0.01 200 300

Find the minimum cost

Verification V(x)=x? Mean number of merchants visited: Mean payment to merchant: 214.14-7.14=207 (notice it’s less than minimum of sampling 7 merchants) V

The finite case Assume we need to choose within N periods of time We’ll use backward induction – start at period N: Now consider N-1 Stopping resuming

The finite case (2) If we use x*>x then: For any best value so far x<z<x* we are supposed to search now (and necessarily terminate after) Therefore: However this cannot hold because z>x

The finite case (2) If we use x*<x then: For any best value so far x*<z<x we are supposed to stop now (and necessarily search after) In that case, a dominating strategy would be to search earlier than later

The finite case (4) Expected benefit: Either we managed to find something above the reservation value x (with probability (1-F(x)^N) Or, we have searched all N opportunities and everything was below x.

f(x) Reservation Value - x Lifetime Utility

מתי נעדיף לדגום N בו-זמנית? מועמד השולח בקשות למספר מוסדות חברה הפונה למספר ספקים לקבל הצעות תהליך הבודק מספר שרתים בו זמנית

דוגמה המוכרים מתחלקים (באופן שווה) בין כאלה המבקשים 2$ עבור המוצר וכאלה המבקשים 3$ מהי התפלגות המחיר המינימלי אם דוגמים מספר מוכרים? 0.75*2+0.25*3 0.875*2+0.125*3

Distribution of the Minimum Price fN(x) fN(x)

תוחלת מינימום המחיר דוגמה – uniform distribution f(x) 1 P=1 1 F(x) 1 1

תוחלת מינימום המחיר (המשך) You can easily check this with Excel… Can you guess what is EN(x) for the maximum of a sample?

התועלת שבדגימה נוספת יורד בקצב קטן (הוכחה על-ידי Robert Solow)

עלות החיפוש בד"כ פרופורציונאלית למספר המוכרים שנדגום כמות החיפוש האופטימלית תתקבל על-ידי השוואת הרווח השולי מחיפוש נוסף ועלות החיפוש cost of search marginal benefit N

דוגמה מחירים מתפלגים בצורה אחידה בין 50 ל- 100 (רציף) עלות כל מחיר נוסף – 1 דולר f(x) 0.02 N=5: E_5+5=63.33 N=6: E_6+6=63.14 N=7: E_7+7=63.25 50 100

EN+N N

Problem 1 You are about to purchase an iPod touch over the internet You estimate the price distribution of the product over the different sellers to be uniform between 200-300 dollars You can search by yourself, by visiting different web-sites – the cost of time for obtaining a price quote is $1 How will you search? What will be your expected cost? What’s the mean of the number of merchants you’ll visit?

Problem 1 (cont.) Alternatively, you can access one of the comparison-shopping web-sites over the internet Web-site “A” offers you searching 100 web-sites for a total of $10 (average of 10 cent per searched site). Web-site “B” offers you searching 20 web-sites for a total of $2 (average of 10 cent per searched site)

Solution The minimum of a sample of size 100 can be calculated using:

Solution (cont.) Calculating the expected minimum: Integration by parts:

Solution (cont.) 214.14>200.99+10 So we better take this offer… and here is a simpler way to come up with that: The minimum of 100 quotes is very close to the 200 lower bound In fact, we know for the uniform distribution that: f(x) 0.01 200 300 In the 201 vicinity

It can become even simpler… f(x) In our problem: 0.01 200 300 f(x) 1 1

Solution for web-site B Conclusion: it’s better to use web-site B