# Use of available data to probe methods for decision under uncertainty Raphael T. Haftka Department of Mechanical and Aerospace Engineering.

## Presentation on theme: "Use of available data to probe methods for decision under uncertainty Raphael T. Haftka Department of Mechanical and Aerospace Engineering."— Presentation transcript:

Use of available data to probe methods for decision under uncertainty Raphael T. Haftka (haftka@ufl.edu) Department of Mechanical and Aerospace Engineering University of Florida Gainesville, Florida 32611 With: Raluca Rosca (UF), Efstratios Nikolaidis (University of Toledo)

Outline Why validating design methods is trickier than validating analysis models Domino tower data base Invented design/decision problem based on domino tower data Performance of probabilistic and possibilistic models and effect of inflation of variance with little data Conclusions

Validation of methods for design under uncertainty When data is missing, it may not be possible to validate a method. Must compare against other methods Choice of method may depend on amount of information I give simple instructions how to get to my home to people who do not know Gainesville. I tell locals what is the shortest route.

Gigerenzer’s Approach Use existing data bases to compare methods for making binary decisions (choose A or B) Invent decision problem that can be based on data Give decision maker part of data base to formulate decision rule Test performance of decision rule on entire data Showed advantage of simple decision-making methods Gigerenzer and Todd (1999), Simple Heuristics That Make Us Smart, Oxford University Press

Our objectives Extend approach to non-binary decisions, involving choice of variable (s) Focus on use of existing data sets for probing weaknesses rather than strengths –Experiments are more effective for disproving rather than for proving Demonstrate lessons learned from validation

Domino tower data base Dominoes stacked until tower collapses One set of 50 towers built by Rosca Another set of 90 towers built by 16 students in a competition

Competition and Rosca’s histograms Analytical model of tower collapse fit best to Gamma distribution Data passed chi-test also for normal distribution

Guaranteed performance decision We use data to test decisions on level of guaranteed performance Company gains advantage over competitors by guaranteeing performance How to select guaranteed level? High target risks failure, low target risks loss of customers

Decision invented for domino data Rosca guarantees the height of the tower she will build She wins if she fulfills her guarantee and if competitor cannot build a tower higher by a handicap than her guarantee High guarantee risks failure to fulfill Low guarantee risks failure due to competitor’s tower exceeding guarantee plus handicap

Examples Handicap of 5, guarantee of 30 tiles Competitor wins by building 35, no matter what Rosca builds Rosca loses by building 29 even if competitor builds only 20 Rosca wins by building 32 even if competitor builds 36

Test procedure Give decision maker (Rosca) five data points for both her and competitor results Decision maker fits distributions to each set of five points Guarantee selected on basis of distribution Repeat 80 times for random quintets Performance based on percentage of success for all possible pairs (50x90)

Decision methods compared Probability based on two distributions Possibility based on triangular membership function spanning data Probability and possibility with inflated variance to compensate for scarce data

Optimum guarantees – all data

Effect of inflation

Five data points

Conclusions from experiments Comparable performance of probability and possibility may indicate need to explain and improve probabilistic performance Adverse effect of inflation indicates need for theoretical study of this effect Probability downgraded importance of inflated mode of uncertainty and possibility emphasized it. This we only later derived analytically. Experiment surprised us and led us to useful insight into difference between the methods

Concluding remarks We extended Gigerenzer’s method for using existing data bases to test methods for decisions under uncertainty –Optimum value instead of binary decision –Repeated random partitioning of data Demonstrated that experiments can yield insights into decision making methods –Possible weaknesses in standard probabilistic approach and inflation –Unreported difference between probability and possibility

Similar presentations