1. Distinguishing the Forest from the Trees 2006 CAS Ratemaking Seminar Richard Derrig, PhD, Opal Consulting www.opalconsulting.com Louise Francis, FCAS, MAAA Francis Analytics and Actuarial Data Mining, Inc. www.data-mines.com

2. Data Mining Data Mining, also known as Knowledge-Discovery in Databases (KDD), is the process of automatically searching large volumes of data for patterns. In order to achieve this, data mining uses computational techniques from statistics, machine learning and pattern recognition. www.wikipedia.org

3. The Problem: Nonlinear Functions

4. An Insurance Nonlinear Function: Provider Bill vs. Probability of Independent Medical Exam

5. Common Links for GLMs The identity link: h(Y) = Y The log link: h(Y) = ln(Y) The inverse link: h(Y) = The logit link: h(Y) = The probit link: h(Y) =

6. Nonlinear Example Data Provider 2 Bill (Bonded) Avg Provider 2 Bill Avg Total PaidPercent IME Zero-9,0636% 1 – 2501548,7618% 251 – 5003759,7269% 501 – 1,00073111,46910% 1,001 – 1,5001,24314,99813% 1,501 – 2,5001,91517,28914% 2,501 – 5,0003,30023,99415% 5,001 – 10,0006,72047,72815% 10,001 +21,35083,26115% All Claims54511,2248%

7. Desirable Features of a Data Mining Method: Any nonlinear relationship can be approximated A method that works when the form of the nonlinearity is unknown The effect of interactions can be easily determined and incorporated into the model The method generalizes well on out-of sample data

8. Decision Trees In decision theory (for example risk management), a decision tree is a graph of decisions and their possible consequences, (including resource costs and risks) used to create a plan to reach a goal. Decision trees are constructed in order to help with making decisions. A decision tree is a special form of tree structure. www.wikipedia.org

9. Different Kinds of Decision Trees Single Trees (CART, CHAID) Ensemble Trees, a more recent development (TREENET, RANDOM FOREST) A composite or weighted average of many trees (perhaps 100 or more) There are many methods to fit the trees and prevent overfitting Boosting: Iminer Ensemble and Treenet Bagging: Random Forest

10. The Methods and Software Evaluated 1) TREENET5) Iminer Ensemble 2) Iminer Tree6) Random Forest 3) SPLUS Tree7) Naïve Bayes (Baseline) 4) CART8) Logistic (Baseline)

11. CART Example of Parent and Children Nodes Total Paid as a Function of Provider 2 Bill

12. CART Example with Seven Nodes IME Proportion as a Function of Provider 2 Bill

13. CART Example with Seven Step Functions IME Proportions as a Function of Provider 2 Bill

14. Ensemble Prediction of Total Paid

15. Ensemble Prediction of IME Requested

16. Naive Bayes Naive Bayes assumes conditional independence Probability that an observation will have a specific set of values for the independent variables is the product of the conditional probabilities of observing each of the values given category cj

17. Bayes Predicted Probability IME Requested vs. Quintile of Provider 2 Bill

18. Naïve Bayes Predicted IME vs. Provider 2 Bill

19. The Fraud Surrogates used as Dependent Variables Independent Medical Exam (IME) requested Special Investigation Unit (SIU) referral IME successful SIU successful DATA: Detailed Auto Injury Claim Database for Massachusetts Accident Years (1995-1997)

20. Results for IME Requested

21. Results for IME Favorable

22. Results for SIU Referral

23. Results for SIU Favorable

24. TREENET ROC Curve – IME AUROC = 0.701

25. TREENET ROC Curve – SIU AUROC = 0.677

26. Logistic ROC Curve – IME AUROC = 0.643

27. Logistic ROC Curve – SIU AUROC = 0.612

28. Ranking of Methods/Software – 1 st Two Surrogates

29. Ranking of Methods/Software – Last Two Surrogates

30. Plot of AUROC for SIU vs. IME Decision

31. Plot of AUROC for SIU vs. IME Favorable

32. References Brieman, L., J. Freidman, R. Olshen, and C. Stone, Classification and Regression Trees, Chapman Hall, 1993. Friedman, J., Greedy Function Approximation: The Gradient Boosting Machine, Annals of Statistics, 2001. Hastie, T., R. Tibshirani, and J. Friedman, The Elements of Statistical Learning, Springer, New York., 2001. Kantardzic, M., Data Mining, John Wiley and Sons, 2003.