1. Chris Nyce, FCAS, MAAA KPMG Senior Manager
AAA Updated Research on the NAIC Risk Based Capital (RBC) Formula
Research on the Effectiveness of a Trend Test in the Property/Casualty RBC Formula
Chris Nyce, FCAS, MAAA
KPMG Senior Manager

2. Disclaimer
These results are based on research conducted by a subgroup of the American Academy P/C RBC Committee
Views expressed today are based on the research, but do not necessarily reflect the views of the Academy, CAS, KPMG, or the NAIC who of course makes all decisions about changes to the RBC formula
Examples used are illustrative, and not a reference to any specific company
Anyone who says otherwise is not only wrong, but is itching for a fight

3. The Mission of the AAA RBC Committee
Began the research with a charge
NAIC expressed concern that RBC was becoming more of a lagging indicator, not useful in predicting companies that become weaker
“Given the use of a trend test in the life RBC formula, is the application of a trend test in the Property/Casualty RBC formula a good idea?”
Our interpretation-
Not a “Yes/No” question
Instead-”What is the most effective way of differentiating between companies above the Company Action Level that are likely to fall below it, and those that are likely to remain above it.”
We approached this with a one year future time horizon, i.e. based on observable data this year, what will happen next year

4. Status of the Work
These ideas were presented in the Spring 2004 CAS meeting, prior to release of the Academy report
Academy issued a report on August 26, currently featured on the casualty cover page of the Academy website
The Academy report did not recommend incorporating the finding in RBC formulas, instead concluded the one certain approach is a “best predictor” of those examined
NAIC PC RBC subgroup did recommend on August 31, 2004 that this trend test be incorporated into the RBC formula, and a company being flagged results in CAL
Exposure period ends November 11, 2004
NAIC considers comments, votes at December meeting
Could vote yes, no, or further deliberation
If yes, likely implementation in 2005

5. Background Life test currently uses a trend test
Applies to companies with RBC between 250% and the company action level (“CAL”) of 200%
RBC ratio is the ratio of capital to RBC required capital
Life test looks at past changes in RBC
Max of last year and the three year average decline in margin for each company
Apply that change to the current RBC position
If below 190%, company is deemed to be at the CAL
Note that even before our work, the feeling of committee members was that the life trend test did not work well for P/C companies
We quickly confirmed this to be true

6. Our Approach
Basic question-”What is the most effective predictor of decline in capital adequacy?”
In general terms, used “Hypothesis Testing”
Examined specific cases of past company failures
Formulated hypotheses on the causes of RBC decline
Tested the hypotheses using statistical tests on annual statement data
Conducted additional tests by examining the effectiveness retrospectively
Measured the results using a specific set of metrics
Selected one approach that produced the best metrics

7. Boundaries of our Study
Did not constrain ourselves to examining the life formula
Based on publicly filed data from the NAIC blank
And used company level data, due to conclusions in the work leading to the original formula that this was appropriate
Outcome has to be intuitively correct, and simple
All research also from public data sources
For NAIC data, company names remained confidential
Outcome had to be based on empirical data, not on our preconceived opinions

8. Data Considerations
For “micro” analysis we used public data sources such as AM BEST and press reports
NAIC provided 5 year history of all requested data elements
Confidential as to company identifier
About 2400 companies
Used data through 2002 for statistical tests, updated through 2003 for retrospective test
We scrubbed the data, in general separately for each test to maximize data points utilized
Screened out invalid entries and extreme values

9. Micro Results-Initial Hypotheses
Companies we examined could be characterized as experiencing trouble due to various causes, such as:
High levels of reinsurance recoverables, causing high leverage in estimating reserves, and exposure to disputed balances
High leverage of premiums and reserves to surplus
Reserve inadequacies coming to roost
Poor operating results
Fraud and misrepresentation
Ill-liquid or incorrectly valued assets
Under-funded pensions: (usually a contributor, not a cause)

10. What is the Best Early Indicator of Future Capital Declines?
Lack of Liquidity
Reserve Inadequacies
Fraud
Bad Assets
Past Capital Declines
Leverage
Poor Profitability

11. Overall “Macro” Approach
Performed statistical tests on the NAIC database
Explored the basic relationships behind each hypothesis
Performed retrospective tests on characteristics of companies just prior to falling to the CAL
Set up metrics to evaluate the outcome of the retrospective test
Determined recommendations based on all of the above

12. Statistical Tests
Explored relationships between hypothesized variables
Performed tests on the NAIC database of 2400 companies for 5 years ending 2002
Looked for statistical tendencies
Generally used correlation and regression analysis
Examined the percentage of variation explained
Calculated the measures of significance
Used to corroborate and explain retrospective result
Note that a poor result in our tests does not necessarily mean that the measure is not good for IRIS or other financial evaluations
And high correlations don’t necessarily mean the hypothesis would form a good trend test

13. Statistical Test of Life Type Trend Test
Does a simple life type of trend test work?
Correlation between year to year changes in RBC ratio for all companies= -23% (wrong sign)
For only companies near the CAL = 1%
In 2001 and 2002, the direction of the change in subsequent years was only the same 41% of the time
Changes in market asset valuations dominated any characteristics of companies themselves
Implication: Life type of trend test is worse than random guessing for P/C Companies

14. What About Underwriting Results and Reserve Runoff?
Correlation between subsequent year combined ratios= 25% to 34% between
For only companies near CAL correlation is 33% to 75% (highly significant)
Reserve Runoff
Correlation between subsequent year runoff ratios=33% to 37% between 2000 and 2002
For only companies near CAL correlation is 29% to 35%
This is good and bad news
Statistical relationship is strong
But still only predicts a portion of the subsequent year outcome

15. What is the Predictive Power of Leverage?
Gross Leverage
Correlation between gross leverage and subsequent year RBC ratio change= -1% to 1% between
For only companies near CAL correlation is –5% to –3% (not significant)
Net Leverage
Correlation between net leverage and subsequent year RBC ratio change= 3% to 4% between
For only companies near CAL correlation is 1% to 16% (wrong sign)
This is not a good outcome
Statistical relationship is weak and sign is sometimes wrong

16. Well then it must be Liquidity?
Correlation between liquid assets to surplus and subsequent RBC change is –4% to 1% over 2001 to 2002
Depending on sample, relationship is not significant, or sign is wrong

17. In 2002 and 2003, Portion of Companies Falling to CAL
RBC Ratio in Prior Year
Total Companies in Sample
Number of Companies Falling to CAL
Percentage Falling to CAL
200% to 300%
314 30
9.6%
300% to 350%
166 9
5.4%
350% to 400%
205 4
2.0%
400% to 450%
176 3
1.7%
Greater than 200%
3582 55
1.5%

18. Retrospective Tests
Performed on NAIC database of 2400 companies ending 2003
Generally “Yes/No”
Measured whether the hypothesis accurately predicted the subsequent year outcome, or not
Therefore, scrubs were oriented toward invalids, but not toward extremes
Measured on three metrics
Effectiveness-Percentage of overall correct predictions
False alarms-Percentage of companies flagged that did not deteriorate to CAL
Failing Companies Flagged-Percentage of companies that subsequently declined to the CAL that were correctly flagged

19. Retrospective Approach
Started by setting a threshold such as leverage above industry average, or combined ratio above 110%, etc.
Based on the threshold, companies were “flagged” or “not flagged”
Allowed for mixed approaches;
Leverage, reserve runoff, and combined ratio
Reserve runoff and combined ratio
Three year tests of reserve runoff and combined ratio
Adjusted the threshold to optimize the metrics
Based on trial and error
Understand, this test tells us not what causes RBC decline, but what best predicts it
Although the implication for the cause is pretty clear

20. Retrospective Metrics

21. An Effective Approach Based on Tests
RBC Ratio
Current Year Combined Ratio
Company Status
200%-300%
Greater than 120%
More Likely to Decline to CAL-”flagged”
Less than 120%
Less Likely to Decline to CAL
300%-350%
Greater than 134%
Less than 134%
Above 350%
All

22. Further Metrics from the Combined Ratio Trend Test
Total Companies
% Flagged
% Of Companies Falling <200% in Subsequent Year
Flagged
Not Flagged
Total
Two-Tiered CR Test for Companies 200%<RBC<250%
143
47%
17.9%
6.6%
11.9%
Two-Tiered CR Test for Companies 250%<RBC<300%
171
33%
2.6%
7.6%
Two-Tiered CR Test for Companies 300%<RBC<350%
166
16%
14.8%
3.6%
5.4%
Total Population 200%<RBC<350%
480
31%
17.3%
3.9%
8.1%

23. Why not 100% Effective
Formula approach doesn’t account for capital changes (contributed, dividend)
Financial statements can always be subject to restatement
RBC ratio decline could involve fraud, or an other wise solid looking asset losing value
Or pension funding
The statistical relationship is strong, but is not 100% predictive of direction and magnitude
Need to keep the test simple

24. Could a PC RBC Trend Test be Appropriate?
Hypothetical Formula Change
Additional Companies at CAL Resulting
True Alarms, Those Falling <200%
False Alarm
True Alarms to Total Flagged
Change the CAL to 250%
143 17
126
11.9%
Implement a “Life” type trend test for Companies 200%<RBC<250%
117 13
104
11.1%
Implement a combined ratio trend test for Companies 200%<RBC<250% 67 12 55
17.9%
Implement a combined ratio trend test for Companies 200%<RBC<300%
123 22
101
Implement a combined ratio trend test for Companies 200%<RBC<350%
150 26
124
17.3%