1. The COTOR Challenge Committee on the Theory of Risk November 2004 Annual Meeting

2. History of the Challenge Last spring a COTOR member challenged actuarial geeks to estimate 500k xs 500k layer based on list of 250 claims Last spring a COTOR member challenged actuarial geeks to estimate 500k xs 500k layer based on list of 250 claims Emails flew back and forth furiously Emails flew back and forth furiously A number of different approaches were used A number of different approaches were used Literature about heavy tailed distributions was recommended Literature about heavy tailed distributions was recommended Winner was Phil Heckman using mixture of 2 lognormals Winner was Phil Heckman using mixture of 2 lognormals

3. History cont. Criticism existed around the sample since some sample statistics were too far from the real distribution Criticism existed around the sample since some sample statistics were too far from the real distribution COTOR feels that the solution of this problem is of interest ot the actuarial community COTOR feels that the solution of this problem is of interest ot the actuarial community Our data is almost never normal/lognormalOur data is almost never normal/lognormal Our data is typically heavy tailedOur data is typically heavy tailed It is likely that in many real situations, a sample of 250 claims would not represent a random draw from any distributionIt is likely that in many real situations, a sample of 250 claims would not represent a random draw from any distribution

4. History cont. Another challenge was issued under well defined conditions Another challenge was issued under well defined conditions Stuart Klugman picked the sample Stuart Klugman picked the sample 250 claims randomly generated from an inverse transformed gamma 250 claims randomly generated from an inverse transformed gamma Challenge was to estimate severity in the $5M xs $5M layer (mean and 95% confidence intervals) Challenge was to estimate severity in the $5M xs $5M layer (mean and 95% confidence intervals)

5. The Sample Claim Size Count Greater than 5,000,000 1 500,000 to 1,000,000 2 100,000 to 500,000 7 50,000 to 100,000 10 25,000 to 50,000 8 10,000 to 25,000 26 5,000 to 10,000 30 2,500 to 5,000 56 1,000 to 2,500 74 500 to 1,000 32 250 to 500 4 Under 250 0 250 claims randomly selected from an inverse transformed gamma

6. Purpose of Session Raise awareness of audience of how frequently extreme values need to be dealt with Raise awareness of audience of how frequently extreme values need to be dealt with Present relatively easy to use approaches Present relatively easy to use approaches Make audience aware of how difficult this problem is to solve Make audience aware of how difficult this problem is to solve

7. Normal Distribution Assumption The normal or lognormal assumption is common in finance application The normal or lognormal assumption is common in finance application Option pricing theoryOption pricing theory Value at riskValue at risk CAPMCAPM Evidence that asset return data does not follow the normal distribution is widely available Evidence that asset return data does not follow the normal distribution is widely available 1968 Fama paper in Journal of the American Statistical Association1968 Fama paper in Journal of the American Statistical Association

8. Test of Normal Distribution Assumption Normal Q-Q Plot of Monthly Return on S&P

9. Test of Normal Distribution Assumption

10. Consequences of Assuming Normality The frequency of extreme events is underestimated – often by a lot The frequency of extreme events is underestimated – often by a lot Example: Long Term Capital Example: Long Term Capital “Theoretically, the odds against a loss such as August’s had been prohibitive, such a debacle was, according to mathematicians, an event so freakish as to be unlikely to occur even once over the entire life of the universe and even over numerous repetitions of the universe”“Theoretically, the odds against a loss such as August’s had been prohibitive, such a debacle was, according to mathematicians, an event so freakish as to be unlikely to occur even once over the entire life of the universe and even over numerous repetitions of the universe” When Genius Failed by Roger Lowenstein, p. 159 When Genius Failed by Roger Lowenstein, p. 159

11. Criteria for Judging New and creative way to solve the problem New and creative way to solve the problem Methodology that practicing actuaries can use Methodology that practicing actuaries can use Clarity of exposition Clarity of exposition Accuracy of known answer Accuracy of known answer Estimates of confidence interval Estimates of confidence interval

12. Table of Results Respond er Mean Lower CL Upper CL Method A 9,500.0 0 450.00 17,500. 00 Inverse Logistic Smoother B 6,000.0 0 0.00 26,000. 00 Kernel Smoothing/Bootstrapping C 12,533. 00 2,976.00 53,049. 00 Log Regression of Density Function on large claims D 2,400.0 0 ?? Generalized Pareto E 6,430.0 0 1,760.00 14,710. 00 Fit distributions to triple logged data. Used Bayesian approach for mean and CI F1 10,282. 00 2,089.00 24,877. 00 Scaled Pareto F2 30,601. 00 6,217.00 74,038. 00 Pareto G 4,332.6 5 297.34 7,645.8 6 Empirical Semi Smoothing H1 2,700.0 0 0.00 17,955. 00 Single Parameter Pareto/Simulation for Confidence Intervals H2 8,772.0 0 0.00 54,474. 00 Generalized Pareto/Bayesian Simulation True Mean 6810.00

13. Observations Regarding Results These estimations are not easy These estimations are not easy Nearly 13 to 1 spread between lowest and highest mean Nearly 13 to 1 spread between lowest and highest mean Only 10% of answers came within 10% of right result Only 10% of answers came within 10% of right result All responders recognized tremendous uncertainty in results (range from upper to lower CL went from 8 to infinity) All responders recognized tremendous uncertainty in results (range from upper to lower CL went from 8 to infinity) Our statistical expert could not understand the description of the method of 30% of the respondents Our statistical expert could not understand the description of the method of 30% of the respondents

14. Observations All but 2 of the methods relied on approaches commonly found in the literature on heavy tailed distributions and extreme values All but 2 of the methods relied on approaches commonly found in the literature on heavy tailed distributions and extreme values It is clear that it is very difficult to get accurate estimates from a small sample It is clear that it is very difficult to get accurate estimates from a small sample The real world is even more challenging than this The real world is even more challenging than this 250 claims probably don’t follow any known distribution250 claims probably don’t follow any known distribution TrendTrend DevelopmentDevelopment Unforeseen changes in environmentUnforeseen changes in environment Consulting with claims adjusters and underwriters should provide valuable additional insightsConsulting with claims adjusters and underwriters should provide valuable additional insights

15. Observations The closest answer was 5% below the true mean The closest answer was 5% below the true mean Half of the responses below the true mean, Half were above Half of the responses below the true mean, Half were above Average response was 40% higher than the mean Average response was 40% higher than the mean Average response (ex outlyer) was within 2% of the mean Average response (ex outlyer) was within 2% of the mean Read: Read: “The Wisdom of Crowds: Why the Many are Smarter than the Few and How Collective Wisdom Shapes Business, Economics, Societies and Nations” by: James Surowiecki Implications for Insurance Companies? Implications for Insurance Companies?

16. Speakers Meyers Meyers Evans Evans Flynn Flynn Woolstenhulme Woolstenhulme Venter Venter Heckman Heckman

17. Announcement of Winners Louise Francis – COTOR Chair Louise Francis – COTOR Chair

18. Possible Next Steps Make the results of the challenge available to the membership Make the results of the challenge available to the membership COTOR subcommittee to evaluate how to make techniques readily available COTOR subcommittee to evaluate how to make techniques readily available Another round making the challenge more real world Another round making the challenge more real world Include trend and development Include trend and development Give multiple random samples Give multiple random samples