1. The Analysis and Estimation of Loss & ALAE Variability Section 5. Compare, Contrast and Discuss Results Dr Julie A Sims Casualty Loss Reserve Seminar Boston, MA September 13, 2005

2. And the Winner is… It depends on the aims of the analysis It depends on the data you are analysing Finding the model that works best “on average” is a huge amount of work – more than this Working Party could do DataModel

3. More Limited Aim Give some examples and ideas of how to use the criteria Get people thinking and talking about the need to do more

4. 3 Star Modelling Process Fit for purpose: Criteria 1, 2, 3, 4 Adequate fit: Criteria 14, 15 Best in class: Criteria 5, 6, 7, 8, 10, 11, 13, 16, 17, 18, 20 Orphans 9, 12, 19

5. Fit For Purpose: Criterion 1 Aims of the Analysis Expected Range (ER): unreliable estimates of parameter uncertainty and percentiles Overdispersed Poisson (ODP): no estimates of percentiles Mack chain ladder equivalent (distribution free): no estimates of percentiles Murphy average ratio equivalent (with normal distribution): full distribution

6. Fit For Purpose: Criterion 4 Cost/Benefit ER: low cost Mack & Murphy: moderate cost ODP: higher cost “Cost” here is based on complexity Benefits? – see later

7. Adequate Fit: Criterion 14 Distributional Assumptions Essential if you want percentiles ER, Mack & ODP: no distribution Murphy on IL40: poor normality = poor fit

8. Adequate Fit: Criterion 14 Distributional Assumptions Murphy on IL40

9. Adequate Fit: Criterion 14 Distributional Assumptions Murphy on IL40

10. Adequate Fit: Criterion 15 Residual Patterns Patterns in residuals likely to give a poor estimate of the mean ER: residuals not defined Murphy on IL40 and ODP on PL40: poor fit

11. Adequate Fit: Criterion 15 Residual Patterns Murphy on IL40: residuals trend up in later accident periods, forecast means likely to be too low

12. Adequate Fit: Criterion 15 Residual Patterns ODP on PL40: residuals trend up and down over calendar periods, forecast means might be high or low

13. Best in Class: 11 Criteria! No surprising behaviour Parsimony - as few parameters as is consistent with good fit

14. Best in Class: Criterion 5 CV Decreases in Later Accident Periods ER on PL40: surprising increases in coefficient of variation of accident totals

15. Best in Class: Criterion 10 Reasonability of Parameters ODP on PL40: surprising increase in accident parameter in last period

16. Best in Class: Criterion 11 Consistency with Simulation Murphy on PL10: pick the real data…

17. Best in Class: Criterion 18 Parsimony (Ockham’s Razor) ODP on IL10: 18 parameters can be reduced to 6 with little loss of fit

18. Fit For Purpose: Criterion 4 Cost/Benefit Caveats: small sample of data, personal opinion ER: low benefit ODP, Mack & Murphy: moderate benefit More parsimonious models: higher benefit More data and more models should be evaluated!!!