1. Lecture 3
Axioms & elicitation

2. Contents Axioms Elicitation Quick recap
”But we just studied these…” -> these are vital to understanding!
Elicitation
How can we elicit someone’s utility function?
What does a utility function look like?

3. Axioms Translation We can compare any two alternatives in utility
If 10 > 5 and 5 > 2, then 10 > 2
Lotteries of consequences make sense
Reduction of compound events
Sure-thing principle
No ”joy in gambling”
Utility is bounded

4. If the axioms are satisfied…
We can model alternatives with utilities
The optimal or rational choice is maximizing expected utility
Utility is an interval scale
Size of unit is arbitrary
Zero point is arbitrary
You can question whether people truly behave like the axioms describe (lecture 4 onwards)

5. Violation of solvability
Continuity
It has been argued that with extreme outcomes (such as death), one alternative is strictly preferred to the other, unless p = 1.
Counterexample:

6. Violation of monotonicity
The axiom implies that, for two uncertain events with identical payoffs, an individual’s preference will depend only on the probability of winning.
Suppose that:
C represents a flip of a coin today with a payoff in 1 year, and
D represents a flip of the same coin 1 year from now with immediate payoff;
Would you still be indifferent?
On the other hand, are the outcomes really the same?

7. Violation of substitution
P=0.10
P=0.01
P=0.89 A 1M B 5M
P=0.10
P=0.01
P=0.89 A’ 1M B’ 5M

8. Substitution - continued
A frequent choice pattern is A and B’! We have: A = M M B = M M A’ = M B’ = M This problem is called Allais’ paradox

9. Problem of payoffs
What are actually the outcomes or payoffs? Are these the same[3]:
[3] Sen, A. (1997). Maximization and the Act of Choice. Econometrica, vol. 65(4) pp

10. Elicitation

11. Interlude: what is probability?
Frequency
The number of events in a sequence, taken to the limit
Naturally, no infinite sequences exist
So typically estimated from data
Example
Of all the courses you have taken, how many times did you get a grade 5?
Subjective
Your subjective opinion on how likely something is
Can be influenced by data/experience, doesn’t have to be
Hard to show someone is wrong!
Example
How likely is it that you will get a grade 5 on this course?

12. The components of a decision
What we need to elicit:
Probabilities
How likely is it that you receive consequence A…B…C…N?
Utilities
How desirable is consequence A…B…C…N?

13. Components example Suppose you have a few extra euros. Should you
Play the lottery
Buy a coffee
Save the money p U EU
Play lottery
0.01
100 1
Buy coffee
0.99 10
9.9
Save 3

14. Encoding methods Self-elicitation versus interviewer elicitation
Encoding methods
Self-elicitation versus interviewer elicitation
Direct questioning versus inferring from choices
The process (interviewer recommended!)
Motivate the subject and investigate the subject’s biases
Structure the uncertain quantity by having it clearly defined
Make the subject think fundamentally about the problem and avoid biases (if possible)
Encode the probability judgments
Verify the responses by checking for consistency (multiple estimations)

15. Chesley’s taxonomy Elicitation approach Who elicits? Direct
Indirect (or hybrid)
Self
Interviewer

16. Direct asking techniques
Direct asking techniques
Magnitude or direct estimation
What is p(x<5)?
What is k such that p(x < k) =0.8?
Odds method
Estimate the ratio of two probabilities: what are the odds that boxer A will win boxer B? (e.g. 2-to-1 odds = 66% vs 33% probability)
Quartile or fractile assessment
Seeks to find equally likely points
Quartile assessments can be made by three splits of the range of the distribution

17. Direct asking techniques
Direct asking techniques
Graphical techniques
Histograms or smooth curves; either ask the individual to provide a graphical distribution or solicit his or her reaction to a given graph
Direct specification of the distribution parameters
Make assumptions about the distribution (for example normal), assess its parameters (mean and variance).
Note: difficulty of assessing variances!

18. Indirect methods Subject makes a set of bets Difficult to do properly!
Example: tumor probability [1]
Difficult to do properly!
We need the utility function over the prizes to do this!
[1] (p. 120)

19. Hybrid: Lotteries, Bids
Lottery
Replacing p estimation with lottery
Given lotteries ApB and CqD, which would you pick?
Repeat with different values to find estimate for p
Bid
Given a gamble ApB
Asking: how much would you pay to take part?

20. Recommendations Unfortunately there is no ”one best method”
Depends on what you and the DM is comfortable with
Some methods (quartiles, parameters) benefit from statistics skills of the DM
Using an interviewer and consistency checks is a good idea
Use density functions, not cumulative distribution functions!

21. Example: which distribution does the CDF represent?

22. Exponential
Normal
Gamma

23. Properties of a good elicitation process
Two measurement issues: reliability and validity
Reliability: relatively free of random error, repeatable, stable, and consistent
Validity: accurately represents the person’s opinion
Ease of use
Not too much cognitive effort for the DM
Not too long

24. Next lecture
Behavioral decision making! “On traditional economic theory: We do not play chess as if we were a grandmaster, invest as if we were Warren Buffett, or cook like an Iron Chef. It is more likely we cook like Warren Buffett, who loves to eat at Dairy Queen.” - Richard Thaler (Nobel Prize in Economics 2017)