Topics in Search Theory The Secretary Problem

Theme Paper Who Solved the Secretary Problem? – Thomas Ferguson, 1989

Background Problem appeared in late 50’s and early 60’s Also known as “marriage problem”, “the dowry problem” Highly appealing problem: – Easy to state – Striking solution Since then the problem has been extended and generalized in many directions to become a “field” of study (see [Freeman, 1983])

Problem Statement Simplest form: – There is one secretarial position available – The number of applicants is known – The applicants are interviewed sequentially in random order (each sequence is equally likely) – You can rank all the applicants from best to worst without ties – The decision to accept or reject the applicant must be based only on the relative ranks of those applicants interviewed so far – An applicant once rejected cannot later be recalled – You will be satisfied with nothing but the very best (meaning: your payoff is 1 if choosing the best of the n applicants and 0 otherwise)

Analysis Attention can be restricted to the class of rules that for some integer r≥1 rejects the first r-1 applicants and then chooses the next applicant who is best in the relative ranking of the observed applicants (since we don’t learn anything new about the population over each interview) timeline n reject first r-1accept (and stop) if better than max so far

Analysis What is the probability of selecting the best applicant? – For r=1 or r=n, 1/n – For r>1: timeline n max of r-1max of j-r> the probability that the maximum number in a sample of j-1 is one of the first r-1 numbers the probability that this is the best candidate

Analysis The optimal r is the one that maximizes this probability For small values of n the optimal r can be easily computed (see excel file) Approximation can be supplied for large n

Kepler’s Problem In 1611, the German astronomer Johannes Kepler lost his wife Since his first marriage had been arranged for him, he decided this time to make his own decision He arranged to interview and to choose from among no fewer than eleven candidates The process consumed much of his attention for nearly 2 years, investigating: – Virtues and drawbacks – Dowry – Negotiation with the parents – Advice of friends – …

Kepler’s Problem Of the eleven candidates interviewed, Kepler eventually decided on the fifth What a surprise! Check the excel sheet for the optimal strategy here… Perhaps if Kepler had been aware of the secretary problem, he could have saved himself a lot of time and trouble Of course, it is not exactly the same problem: – Recall option – Discount factor? – Other utility aspects? – Maybe it’s an optimal stopping rule problem?

Back to Secretary problem: Analysis This is actually a Riemann approximation to an integral! When:

Optimal Solution First derivative: Substituting in : Interpretation: wait until 37% of applicants have been interviewed and then select the relative best one. The probability of success in this case will be 37%

Differences from One-Sided Search Rankings No search costs Decision horizon Recall Goal function (find the best vs. optimize expected value)