1. Decision Analysis Lecture 7
Tony Cox My
Course web site:

2. Agenda Project ideas Comments on problem set 6 (due March 7)
Family out: Typos in Charniak
Car buying: Hint – calculate EMV for each act.
Decision psychology (cont.)
Heuristics and biases
Probabilistic expert systems (Netica)
Wason selection task, Simpson’s Paradox
Other probability models and practice problems
Discrete: geometric, Poisson
Continuous: exponential, beta, normal

3. Goals for applied projects
Extract good problems from available knowledge and data
“Good” = high value of analysis = large improvement in decisions, results, etc.
Apply DA techniques to produce valuable answers and insights
Unexpected directions are ok!
Present results so that value is clear
If possible, discuss impact and/or next steps
Research projects: Algorithm research, software project, etc.

4. General ideas for projects
Read and report on a book or a collection of about 3-5 technical papers in an area where DA methods are extensively used
Marketing, pricing, A/B testing;
Location, operations, inventory and logistics;
Bidding
Policy analysis
Research and report on algorithms behind Netica, simulation-optimization, or other analytics software
Create a piece of software for advanced analytics
Apply statistical consulting methods and software to a real-world problem from work or life.
Propose your own novel research or application

5. Some project ideas Optimizing buyer discounts given history
Psychology of self-control and will power
Report on Silver’s The Signal and the Noise; report on Poundstone’s Priceless: The Myth of Fair Value
Excel simulation-optimization (SO) solver for multi-item inventory control decisions
SO for call center intent-based routing to reduce call handling times
NFL decision analytics; Improving 4th down decisions
Providing more valuable information to help change customer/patient behaviors

6. Charniak – two typos
Figure 2, last entry with value 0.3 should be not-fo and not-bp.
“I can calculate the conditional probability of family-out given these pieces of evidence. (For this case, it is .5.)”
Netica model says “0.5” is wrong.
It should be 0.448

7. Car-buying: Structuring the problem and solution
Not trivial to formulate well in Netica
Multiple decision nodes (what test to choose, if any, and what to do after each outcome) and possibility of an option disappearing complicate the BN diagram
Easier approach may be to develop an EMV formula for each possible (undominated) act
Calculate EMV(a) directly from the problem data
Use Netica only as a Bayes Rule calculator

8. Defining acts
Act = one or more if-then rules determining what to do in all cases: If I see Y, then I will do X
Examples of acts:
Buy old
Buy new
Buy old if AAA test says good and car is available, else buy new
Buy old if garage test says good, else buy new

9. Defining states A state resolves relevant uncertainties
Car is good or bad
AAA test says good if good
AAA test says bad if bad
Car remains available if AAA test
Garage says good if good
Garage test says bad if bad
Only some state components are relevant to each decision.
Decisions may depend on states and their probabilities via observations and Bayes’ Rule

10. Decision psychology heuristics and biases (cont.)

11. Seeking and interpreting data for probabilities: Simpson’s Paradox, Wason selection task

12. Using data to guide decisions
Evaluate how well interventions, policies, and decisions have worked in creating significant improvements in outcomes
Compare treatments or interventions
Which has the greatest effect?
How big is it?
How sure can we be?
Decide: Use data to help guide risky choices, adaptively optimize choices

13. Choosing a kidney stone treatment
Which kidney stone treatment should you choose?
How confident should you be?
Are available data sufficient to support a confident decision?
Data/evidence:
Treatment A
Treatment B
78% (273/350) success rate
83% (289/350) success rate

14. Which treatment should you recommend?
Treatment A
Treatment B
Small stones
Group 1 93% (81/87)
Group 2 87% (234/270)
Large stones
Group 3 73% (192/263)
Group 4 69% (55/80)
Both
78% (273/350)
83% (289/350)

15. Detecting discrimination
You are hired to investigate discrimination in admissions at a state university
What valid statistical conclusions can you draw from the following data?

16. Discrimination data revisited
Data from 6 largest departments

17. Simpson’s Paradox - discussion
Q: Which relations should be used, aggregate or sub-population-specific?
A: Could be either, depending on causality

18. Example: Designing data collection to test a hypothesis
Vowels:
A, E, I, O, U
Even numbers:
0, 2, 4, 6, 8

19. Variation: Testing a more concrete hypothesis

20. Wrap-up on exercises
Continual open-mindedness about uncertainties + technical skills to reduce them are essential for using data to learn how to act more effectively.
A combination of humility (willingness to learn from data) and skepticism (insistence on learning from data) about what is assumed or considered known boosts performance in forecasting, decision-making, diagnosis, research, and management