Download presentation

Presentation is loading. Please wait.

Published byDamien Rodriguez Modified about 1 year ago

1
© National Bank of Belgium

2
Failure Prediction Models: Disagreements, Performance, and Credit Quality Janet MITCHELL and Patrick VAN ROY National Bank of Belgium “Small business banking and financing: a global perspective” Cagliari, May 25th

3
© National Bank of Belgium Motivation The paper explores empirically a number of comparative issues relating to models assigning failure predictions (credit scores or PDs) to non-listed firms. Failure prediction models are important in Basel II (PDs key input for the calculation of regulatory capital under IRB approach). Focus on four models: the National Bank of Belgium (NBB) bankruptcy prediction model, two vendor models (Model 1 and Model 2) and the Z-score (Altman).

4
© National Bank of Belgium Four main issues Disagreements between models: do different models yield significantly different "rankings" for the same firm? Model power: are some models better at differentiating between failing and non-failing firms? Combining models: are combinations of models more powerful than single models? Design of internal ratings systems: does model power change as the number of rating classes and the distribution of borrowers across classes vary?

5
© National Bank of Belgium Data and sample 40,000 small to medium-sized non-listed Belgian firms. Inputs, statistical methods and calibration differ across models. Data are obtained from the Belgian central balance sheet office and from one vendor. Bankruptcy data is used to estimate 1-year and 5- year credit scores or PDs. The presentation focuses on 1-year failure predictions.

6
© National Bank of Belgium 1-year bankruptcy rates (in %) Risk class% firmsNBBModel 1Model 2Z-score 101.40.00 221.50.000.010.060.05 321.50.090.060.110.25 418.80.220.160.200.45 522.00.340.430.570.40 611.61.442.001.121.28 703.37.855.526.855.46 Methodology Output of each model (PDs or credit scores) is rank- ordered before being mapped to 1 of 7 risk classes based on the output of one vendor model: low risk high risk

7
© National Bank of Belgium Disagreements (1 vs. 4,5,6 or 7) Percentage of class-1 firms (= lowest risk firms) of a given model classified in or above the median risk class (= class 4) by another model: Class 1Class 4,5,6 or 7 NBBModel 1Model 2Z-score NBB-01.436.754.2 Model 100.5-18.921.9 Model 234.329.4-42.3 Z-score07.507.744.2-

8
© National Bank of Belgium Disagreements (7 vs. 1,2,3 or 4) Percentage of class-7 firms (= highest risk firms) of a given model classified in or below the median risk class (= class 4) by another model: Class 7Class 1,2,3 or 4 NBBModel 1Model 2Z-score NBB-16.314.516.4 Model 108.7-15.633.2 Model 218.119.0-29.3 Z-score18.426.611.5-

9
© National Bank of Belgium Model power: ROC curves type 1 error 1 type 2 error

10
© National Bank of Belgium ROC curves of the 4 models

11
© National Bank of Belgium ROC areas of the 4 models ModelArea under the ROC curve NBB0.876 Model 10.868 Model 20.833 Z-score0.779 Area of model with no discriminatory power = 0.5 Area of model with acceptable discriminatory power > 0.7 Area of model with perfect discriminatory power = 1.0

12
© National Bank of Belgium ROC areas of selected combinations NBB (N) = 0.876 ; Model 1 (M1) = 0.868 ; Model 2 (M2) = 0.833; Z-score (Z) = 0.779 CombinationMin.Max.MedianMean N - M10.8780.8980.908 N - M20.8610.8920.898 N - Z0.8540.8550.880 N - M1 - M20.8670.9010.8990.916 N - M1 - Z0.8610.8860.8940.911 N - M2 - Z0.8450.8790.8830.899 N - M1- M2 - Z0.8520.8900.9140.917

13
© National Bank of Belgium ROC areas of selected combinations NBB (N) = 0.876 ; Model 1 (M1) = 0.868 ; Model 2 (M2) = 0.833; Z-score (Z) = 0.779 CombinationMin.Max.MedianMean N - M10.8780.8980.908 N - M20.8610.8920.898 N - Z0.8540.8550.880 N - M1 - M20.8670.9010.8990.916 N - M1 - Z0.8610.8860.8940.911 N - M2 - Z0.8450.8790.8830.899 N - M1- M2 - Z0.8520.8900.9140.917

14
© National Bank of Belgium ROC areas of selected combinations NBB (N) = 0.876 ; Model 1 (M1) = 0.868 ; Model 2 (M2) = 0.833; Z-score (Z) = 0.779 CombinationMin.Max.MedianMean N - M10.8780.8980.908 N - M20.8610.8920.898 N - Z0.8540.8550.880 N - M1 - M20.8670.9010.8990.916 N - M1 - Z0.8610.8860.8940.911 N - M2 - Z0.8450.8790.8830.899 N - M1- M2 - Z0.8520.8900.9140.917

15
© National Bank of Belgium ROC areas of selected combinations NBB (N) = 0.876 ; Model 1 (M1) = 0.868 ; Model 2 (M2) = 0.833; Z-score (Z) = 0.779 CombinationMin.Max.MedianMean N - M10.8780.8980.908 N - M20.8610.8920.898 N - Z0.8540.8550.880 N - M1 - M20.8670.9010.8990.916 N - M1 - Z0.8610.8860.8940.911 N - M2 - Z0.8450.8790.8830.899 N - M1- M2 - Z0.8520.8900.9140.917

16
© National Bank of Belgium ROC areas of selected combinations NBB (N) = 0.876 ; Model 1 (M1) = 0.868 ; Model 2 (M2) = 0.833; Z-score (Z) = 0.779 CombinationMin.Max.MedianMean Z - N0.8540.8550.880 Z - M10.8470.8710.890 Z - M20.8080.8580.855

17
© National Bank of Belgium ROC areas of 9 possible internal ratings systems (NBB model) Number of classes Mapping of firms based on Vendor model distribution Moody's distribution Equal distribution 70.8760.8740.858 100.8830.8820.873 170.8870.8850.883 Note: NBB continuous credit score has an ROC area of 0.889 0.018 0.010 0.004 0.011 0.025

18
© National Bank of Belgium Conclusion High disagreements rates between models: model choice can have a significant impact on loan pricing and origination decisions. High power of each model: the definition of failure as well as the statitical method used by the models may not matter as much as one would have expected. Larger differences between differing combinations of models than between differing internal rating systems.

19
© National Bank of Belgium

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google