1. © Deloitte Consulting, 2005 Loss Reserving Using Claim-Level Data James Guszcza, FCAS, MAAA Jan Lommele, FCAS, MAAA CLRS Boston September, 2005

2. © Deloitte Consulting, 2005 Agenda  Motivations for Reserving at the Claim (or Policy) Level  Outline one possible modeling framework  Sample Results  Bootstrapping to estimate variability of o/s loss estimates

3. © Deloitte Consulting, 2005 Motivation Why do reserving at the claim level?

4. © Deloitte Consulting, 2005 2 Basic Motivations 1. Better reserve estimates for a changing book of business  Do triangles “summarize away” patterns that could aid prediction?  Could adding predictive variables help? 2. More accurate estimate of reserve variability  Insight of bootstrapping: empirical distribution can often serve as a proxy for the true distribution.  But loss triangles summarize away this distributional information.

5. © Deloitte Consulting, 2005 (1) Better Reserve Estimates  Key idea: use predictive variables to supplement loss development patterns  Most reserving approaches analyze summarized loss/claim triangles.  Does not allow the use of covariates to predict ultimate losses (other than time-indicators).  Actuaries use predictive variables to construct rating plans & underwriting models.  Why not loss reserving too?

6. © Deloitte Consulting, 2005 Why Use Predictive Variables?  Suppose a company’s book of business has been deteriorating for the past few years.  This decline might not be reflected in a summarized loss development triangle.  However: The resulting change in distributions of certain predictive variables might allow us to refine our ultimate loss estimates.  Predictive variables can be “leading indicators” of changes in development patterns.

7. © Deloitte Consulting, 2005 Examples of Predictive Variables  Policy metrics: limits written, classification, jurisdiction, distribution by product type, rating territory… age of policy, agent characteristics, bill paying history, financial information on the policyholder, motor vehicle records  Operational metrics: case reserve philosophy, new rating strategies…  Changes in policy processing: Role of agent vs. underwriter, new system  Financial Metrics: Rate Changes

8. © Deloitte Consulting, 2005 Examples of Predictive Variables  Qualitative metrics: Mix of Preferred/standard/non-standard, schedule rating, premium discounts, renewal credits, multiple policy credits  Claim metrics: claimant information, date of accident, time of day, report date, coverage… nature of injury (BI, PD, medical vs. indemnity etc) type of injury (e.g. back strain, broken bone, damaged vehicle, house fire, car theft…) cause of loss/injury, diagnosis and treatment codes, financial information on the claimant, attorney retained by claimant or not

9. © Deloitte Consulting, 2005 More Data Points  Typical reserving projects use claim data summarized to the year/quarter level.  Probably an artifact of the era of pencil-and- paper statistics.  In certain cases important patterns might be “summarized away”.  In the computer age, why restrict ourselves?  More data points  less chance of over-fitting the model.

10. © Deloitte Consulting, 2005 Danger of over-fitting  One well known example  Overdispersed Poisson GLM fit to loss triangle.  Stochastic analog of the chain-ladder  55 data points  20 parameters estimated  parameters have high standard error.  How do we know the model will generalize well on future development?  Claim-level data: 1000’s of data points

11. © Deloitte Consulting, 2005 Out-of-Sample Testing  Claim-level dataset has 1,000’s of data points  Provides more flexibility for various out-of- sample testing strategies.  Use of holdout samples  Cross-validation  Uses:  Model selection  Model evaluation

12. © Deloitte Consulting, 2005 (2) Reserve Variability  Variability Components  Process variance E[(y-E[y]) 2 ]  Parameter varianceE[(y ^ -E[y ^ ]) 2 ]  Model specification risk  Predictive var = process + parameter var  E[(y-y ^ ) 2 ] ≈ E[(y-E[y]) 2 ] + E[(y ^ -E[y ^ ]) 2 ]  Variability of the forecast  The “reserve variability” we want to measure  Includes both variability of underlying data and the variability of our parameters estimates.  Reserve variability should also consider model risk.  Harder to quantify

13. © Deloitte Consulting, 2005 Reserve Variability  Can claim-level data give us a more accurate view of reserve variability? Prediction variance: variability of the forecast. estimate using brute force “bootstrapping” Process risk: inherent variability of the data. We are not summarizing this away Parameter risk: variability in the estimation. more data points should lead to less estimation error  Leaves us more time to focus on model risk.

14. © Deloitte Consulting, 2005 Disadvantages  Expensive to gather, prepare claim-detail information.  Still more expensive to combine this with claim/claimant/policy-level covariates.  More open-ended universe of modeling options (both good and bad).  Requires more analyst time, computer power, and specialist software.  Less interactive than working in Excel.

15. © Deloitte Consulting, 2005 Modeling Approach Sample Model Design

16. © Deloitte Consulting, 2005 Philosophy  Provide an illustration of how reserving might be done at the claim or policy level.  Simple starting point: consider a generalization of the chain-ladder.  Just one possible model  Analysis is suggestive rather than definitive  No consideration of superimposed inflation  No consideration of calendar year effects  Model risk not considered here  etc…

17. © Deloitte Consulting, 2005 Notation  L j = {L 12, L 24, …, L 120 }  Losses developed at 12, 24,… months  Developed from policy inception date  AY i = {AY 1, AY 2, …, AY 10 }  Accident Years 1, 2, …, 10  {X k } = covariates used to predict losses  Assumption: covariates are measured at or before policy inception

18. © Deloitte Consulting, 2005 Model Design  Build 9 successive GLM models  Regress L 24 on L 12 ; L 36 on L 24 … etc  Each GLM analogous to a link ratio.  The L j  L j+1 model is applied to either  Actual values @ j  Predicted values from the L j-1  L j model  Predict L j+1 using covariates along with L j.

19. © Deloitte Consulting, 2005 Level of the Data  Data could be summarized to the  Policy level  Claim level  Claimant level  Finer level of summarization  a broader array of predictive variables can be considered in modeling.  e.g.: it is more natural to use type-of-claim variables in a claim-level model than a policy-level model

20. © Deloitte Consulting, 2005 Model Design: Claim-Level Data  Idea: model each claim’s loss development from period L j to L j+1 as a function of a linear combination of several covariates.  Claim-level generalization of the chain-ladder idea: Consider case where there are no covariates

21. © Deloitte Consulting, 2005 Model Design  Over-dispersed Poisson GLM:  Log link function  Pragmatic choice – often used in reserving  Variance of L j+1 is proportional to mean  Treat log(L j ) as the offset term  Or simply use the ratio (L j+1 /L j ) as the target and use L j as a weight.

22. © Deloitte Consulting, 2005 Using Policy-Level Data  This framework can handle policy-level data as well as claim-level data.  The data would contain many zeros  Poisson assumption places a point mass at zero  How to handle IBNR  Include dummy variable indicating $0 loss @12 mo  The model will allocate a piece of the IBNR to each policy with $0 loss as of 12 months.  You get back the total IBNR by adding up all of the pieces allocated to $0-loss policies.  The indicator can be interacted with other predictive variables to assign variable amounts of IBNR to different types of policies.

23. © Deloitte Consulting, 2005 Case Study Simulated Claim Data Comparing the proposed method with the chain-ladder

24. © Deloitte Consulting, 2005 Simulation Study  We can illustrate the technique by applying it to simulated data.  By construction: 1. Policies are identical except for one characteristic: Good credit vs. bad credit 2. Bad credit policies develop more slowly.  Build a steeper development pattern into the simulation. 3. A greater proportion of bad credit policies have been written in more recent years.  The chain ladder will not reflect the changing mix of business in a timely way.

25. © Deloitte Consulting, 2005 Simulated Data  500 claims each year * 10 years  Each record represents 1 claim IBNR not included in this example.  Each record has multiple loss evaluations  @ 12, 24, …,120 months  “Losses @ j months” means:  j months from the beginning of the accident year  Assume losses are fully developed @120 months.

26. © Deloitte Consulting, 2005 Loss Distribution  Each of the 5000 claims was drawn from a lognormal distribution with parameters μ=8; σ=1.3  Different average loss development patterns assumed for bad, good credit policies. L i+j = L i * (link + ε) “link” is a function of credit  Bad credit polices develop more slowly ε is a random error term

27. © Deloitte Consulting, 2005 Loss Distribution Histograms of log(loss) and loss 5000 claims (500/year) lognormal(μ=8, σ=1.3)

28. © Deloitte Consulting, 2005 By Design  losses from “bad credit” policies on average have slower development patterns.  Proportion of “bad credit” policies higher in more recent accident years

29. © Deloitte Consulting, 2005 Covariates  In this example, credit is the only covariate. Bad credit indicator  This suffices to illustrate the point. Other covariates could easily be built into the simulation.  Here, we know the “true” covariates In real life, this is a significant source of model risk.

30. © Deloitte Consulting, 2005 Approach  We simulate data for all 10 accident years to ultimate.  We therefore know the “true” outstanding losses by year.  Compare the results of both our method and the chain-ladder to the “truth”.

31. © Deloitte Consulting, 2005 The Data  Summarize the simulated data to the accident year level.  We know the “true” o/s losses Can calculate the implied development factors.  Losses in the blue-shaded cells will be treated as unknown  not used in the models.

32. © Deloitte Consulting, 2005 Comparing the Results  The chain ladder produces results that are too low AY 1999 CL reserves are too low by $.6M  Unlike the C-L, the proposed method does pick up on the shifting distribution of bad credit policies. Proposed method’s results very close to the “truth”

33. © Deloitte Consulting, 2005 Comparing the Results  In more detail…  Chain-ladder LDFs, ultimate losses too low in recent accident years

34. © Deloitte Consulting, 2005 Eliminating the Covariate  What if we run our claim-level model without including the credit predictive variable?  Our method exactly reproduces the chain ladder results!  The method is a proper generalization of the chain ladder. Improved predictions are due to inclusion of relevant covariates.

35. © Deloitte Consulting, 2005 Reserve Variability The Bootstrap Estimating the probability distribution of one’s outstanding loss estimate

36. © Deloitte Consulting, 2005 The Bootstrap  An estimate of outstanding losses is a point estimate.  In statistics, we always want to complement a point estimate with a confidence interval.  Or better yet the distribution of the point estimate.  In the case of loss reserving, this is hard to do.  Bootstrapping is a simulation-based technique for estimating the variability of any estimator.  No distributional assumptions needed.

37. © Deloitte Consulting, 2005 The Key Idea  Everything we know about the “true” probability distribution comes from the data.  So let’s treat the data as a proxy for the true distribution.  We draw multiple samples from this proxy…  This is called “resampling”.  Resampling = sampling with replacement  And compute the statistic of interest on each of the resulting pseudo-datasets.  You thereby estimate the distribution of this statistic!

38. © Deloitte Consulting, 2005 Motivating Example  Let’s look at a simple case where we all know the answer in advance.  Pull 500 draws from the n(5000,100) dist.  The sample mean ≈ 5000 Is a point estimate of the “true” mean μ. But how sure are we of this estimate?  From theory, we know that:

39. © Deloitte Consulting, 2005 Visualizing the Raw Data  500 draws from n(5000,100)  Look at summary statistics, histogram, probability density estimate, QQ-plot.  … looks pretty normal

40. © Deloitte Consulting, 2005 Sampling With Replacement Now let’s use resampling to estimate the s.d. of the sample mean (hopefully ≈ 4.47)  Draw a data point at random from the data set. Then throw it back in  Draw a second data point. Then throw it back in…  Keep going until we’ve got 500 data points. You might call this a “pseudo” data set.  This is not merely re-sorting the data. Some of the original data points will appear more than once; others won’t appear at all.

41. © Deloitte Consulting, 2005 Sampling With Replacement  In fact, there is a chance of (1-1/500) 500 ≈ 1/e ≈.368 that any one of the original data points won’t appear at all if we sample with replacement 500 times.  any data point is included with Prob ≈.632  Intuitively, we treat the original sample as the “true population in the sky”.  Each resample simulates the process of taking a sample from the “true” distribution.

42. © Deloitte Consulting, 2005 Resampling  Sample with replacement 500 data points from the original dataset S Call this S* 1  Now do this 999 more times! S* 1, S* 2,…, S* 1000  Compute X-bar on each of these 1000 samples.

43. © Deloitte Consulting, 2005 Results  From theory we know that X-bar ~ n(5000, 4.47)  Bootstrapping estimates this pretty well!  And we get an estimate of the whole distribution, not just a confidence interval.

44. © Deloitte Consulting, 2005 Calculating a Confidence Interval  Easiest way: j ust take the desired percentiles of the bootstrapped distribution. 95% conf interval ≈ (4987.6, 5004.8)  Another way: rely on the approximate normality of the bootstrapped distribution. X-bar ±2*(bootstrap dist s.d.) Agrees fairly well in this case. mean(norm.data) - 2 * sd(b.avg) [1] 4986.926 mean(norm.data) + 2 * sd(b.avg) [1] 5004.661

45. © Deloitte Consulting, 2005 Bootstrapping Reserves  S = our database of 5000 claims  Sample with replacement all policies in S  Call this S* 1  Same size as S  Now do this 499 more times!  S* 1, S* 2,…, S* 500  Estimate o/s reserves on each sample  Get a distribution of reserve estimates

46. © Deloitte Consulting, 2005 Bootstrapping Reserves  Compute our reserve estimate on each S* k  These 500 reserve estimates constitute an estimate of the distribution of outstanding losses  Note: this involves running 9*500 GLM models!  Notice that we did this by resampling our original dataset S of claims.  Note: this bootstrapping method differs from other analyses which bootstrap the residuals of a model.  These methods rely on the assumption that your model is correct.

47. © Deloitte Consulting, 2005 Distribution of Outstanding Losses  Blue bars: the bootstrapped distribution  Dotted line: kernel density estimate of the distribution  Pink line: superimposed normal

48. © Deloitte Consulting, 2005 Distribution of Outstanding Losses  The simulated dist of outstanding losses appears ≈ normal. Mean: $21.751M Median: $21.746M σ : $0.982M σ/μ ≈ 4.5%  95% confidence interval (19.8M, 23.7M) Note: the 2.5 and 97.5 %iles of the bootstrapping distribution roughly agree with $21.75 ± 2σ

49. © Deloitte Consulting, 2005 Distribution of Outstanding Losses  We can examine a QQ plot to verify that the distribution of o/s losses is approximately normal. However, the tails are somewhat heavier than normal. Remember – this is just simulated data! Real-life results have been consistent with these results.

50. © Deloitte Consulting, 2005 Outstanding Losses by Year

51. © Deloitte Consulting, 2005 Interpretation  This result suggests:  With 95% confidence, the total o/s losses will be within +/- 9% of our estimate.  Assumes model is correctly specified.  Too good to be true?  Yes: doesn’t include model risk! In the real world we don’t know the ‘true’ model  Bootstrapping methodology can be refined.  Advantages:  We are really using our 1000s of data points.  We’re not throwing away heterogeneity info.

52. © Deloitte Consulting, 2005 Bootstrapping What?  The approach discussed here differs from bootstrapping discussions often found in the literature.  We bootstrap the original observations Run the model on each “false history”.  Most accounts: bootstrap the residuals of a model. Create a “false history” by adding the residuals to the original data points. Run the model on each “false history”.  This choice always available when bootstrapping a regression model.

53. © Deloitte Consulting, 2005 Closing Thoughts  The bootstrapping technique illustrated here does provide an estimate of predictive variance.  It takes into account both process and parameter risk  Advantage: it is simple to understand and execute.  However it can be refined by more subtle bootstrapping approaches.

54. © Deloitte Consulting, 2005 Closing Thoughts  The loss development triangle is not necessarily a sufficient statistic for estimating o/s losses. Variability information is summarized away Allows us to bootstrap observations, not just residuals or link ratios. Information about differential loss development patterns might be suppressed.

55. © Deloitte Consulting, 2005 Closing Thoughts  The ± 9% result is only suggestive The data is just a primitive simulation. However is consistent with real-world results that we’ve seen. Simulating 1 / 10 the # of claims with 10* the average size:  yields a similar reserve estimate…  …but a (-15%,+30%) 95% confidence interval...