1. 1 Mapping the State of Financial Stability 14th Annual DNB Research Conference 3rd November 2011 DNB, Amsterdam Peter Sarlin (Åbo Akademi / TUCS) & Tuomas Peltonen (European Central Bank)

2. 2 An example of a SOM output at certain time

3. 3 1. Introduction - motivation Global Financial Stability Map (GFSM) IMF GFSR September 2011 (Dattels et al., 2010) Current state-of-the-art: Six composite indices describing the state of financial stability (area) and various aspects of vulnerabilities (dimensions)… Current state-of-the-art: Six composite indices describing the state of financial stability (area) and various aspects of vulnerabilities (dimensions)…

4. 4... fall short in disentangling the sources of vulnerability as further indicators are needed.... fall short in disentangling the sources of vulnerability as further indicators are needed. IMF/Dattels et al. (2010) use market intelligence and other types of technical adjustments to GFSM for describing the vulnerabilities IMF/Dattels et al. (2010) use market intelligence and other types of technical adjustments to GFSM for describing the vulnerabilities ➨ no attempt to measure the precision of out-of- sample prediction of systemic events. ➨ What does a certain level of vulnerabilities wrt. future crisis? ➨ no attempt to measure the precision of out-of- sample prediction of systemic events. ➨ What does a certain level of vulnerabilities wrt. future crisis? 1. Introduction - motivation

5. 5 Common limitations of spider charts: The area does not scale 1-to-1 with increases in vulnerabilities and depends on the order of dimensions Common limitations of spider charts: The area does not scale 1-to-1 with increases in vulnerabilities and depends on the order of dimensions In the charts (e.g. two different countries), the only actual difference in the aggregate degree of vulnerability is the dimensions and their order In the charts (e.g. two different countries), the only actual difference in the aggregate degree of vulnerability is the dimensions and their order

6. 6 1. Introduction - What do we do in the paper? Create the Self-Organizing Financial Stability Map (SOFSM)Create the Self-Organizing Financial Stability Map (SOFSM) A model that can visualize multidimensional macro-financial vulnerabilities and the state of financial stability across countries and over timeA model that can visualize multidimensional macro-financial vulnerabilities and the state of financial stability across countries and over time A model that has good out-of-sample predictive capabilities of future systemic events / financial crisesA model that has good out-of-sample predictive capabilities of future systemic events / financial crises

7. 7 2. Self-Organizing Financial Stability Map (SOFSM) Building blocks for creating the SOFSM:Building blocks for creating the SOFSM: Self-Organizing MapsSelf-Organizing Maps Identifying systemic eventsIdentifying systemic events Vulnerability indicatorsVulnerability indicators Model trainingModel training Model evaluationModel evaluation Mapping the State of Financial StabilityMapping the State of Financial Stability

8. 8 2.1 Self-Organizing Maps (SOMs) – what are they? SOM is an Exploratory Data Analysis (EDA) technique by Kohonen (1981) ➨ Viscovery SOMineSOM is an Exploratory Data Analysis (EDA) technique by Kohonen (1981) ➨ Viscovery SOMine It is a clustering and projection technique:It is a clustering and projection technique: –Spatially constrained form of k-means clustering –Preserves the neighbourhood relations of the data (instead of trying to preserve the distances between data) –Projects data onto a grid of nodes (rather than projecting data into a continuous space) Enables visualization of high-D data ➨ 2D grid of nodes without losing the topological relationships of data and sight of individual indicators.Enables visualization of high-D data ➨ 2D grid of nodes without losing the topological relationships of data and sight of individual indicators. Enables a flexible distribution and interactions.Enables a flexible distribution and interactions. Kohonen’s group has continuously reviewed the SOM literatureKohonen’s group has continuously reviewed the SOM literature The SOM has been used in approx. 10 000 worksThe SOM has been used in approx. 10 000 works Applied to currency and debt crises: Arciniegas and Arciniegas Rueda (2009), Resta (2009), Sarlin (2011) and Sarlin and Marghescu (2011)Applied to currency and debt crises: Arciniegas and Arciniegas Rueda (2009), Resta (2009), Sarlin (2011) and Sarlin and Marghescu (2011)

9. 9 2.1 Self-Organizing Maps (SOMs) – training algorithm mbmb xjxj Radius of the neighborhood σ 1.Compare all data points x j with all nodes m i to find for each data point the nearest node m b (i.e., best-matching unit, BMU) 2.Update each node m i to averages of the attracted data, including data located in a specified neighbourhood σ 3.Repeat steps 1 and 2 a specified number of times. The SOM parameters are radius of the neighbourhood σ, number of nodes M, map format (ratio of X and Y dimensions), and number of training iterations t.

10. 10 2.1 Self-Organizing Maps (SOMs) – interpreting the output This is a 2D map that represents multi-D data with a 2-level clusteringThis is a 2D map that represents multi-D data with a 2-level clustering For each indicator, we create a „feature plane“ where the color coding represents the distribution of its values on the 2D map.For each indicator, we create a „feature plane“ where the color coding represents the distribution of its values on the 2D map. Indicator 1 Indicator 2 Indicator 3 Indicator 4

11. 11 2.2 Identifying systemic events and creating financial stability cycle Use the data set from Lo Duca and Peltonen (2011): 28 countries (18 EMEs & 10 AEs), Quarterly data 1990Q1-2010Q3 Use the data set from Lo Duca and Peltonen (2011): 28 countries (18 EMEs & 10 AEs), Quarterly data 1990Q1-2010Q3 Identification of systemic events: Identification of systemic events: The Financial Stress Index (FSI) includes 5 components for each country, measuring volatilities and sharp declines in key market segments (stock, foreign exchange and money markets) The Financial Stress Index (FSI) includes 5 components for each country, measuring volatilities and sharp declines in key market segments (stock, foreign exchange and money markets) A systemic event occurs when the FSI is above the 90 th percentile of the country-specific distribution (on average, negative real consequences) A systemic event occurs when the FSI is above the 90 th percentile of the country-specific distribution (on average, negative real consequences) Using the FSI, we identify four classes to describe the financial stability cycle: Using the FSI, we identify four classes to describe the financial stability cycle: Pre-crisis periods (18 months before the systemic event) Pre-crisis periods (18 months before the systemic event) Crisis periods (systemic events defined by a financial stress index) Crisis periods (systemic events defined by a financial stress index) Post-crisis periods (18 months after the systemic event) Post-crisis periods (18 months after the systemic event) Tranquil periods (all other periods) Tranquil periods (all other periods)

12. 12 2.2. Financial Stress Index (FSI) A simple Financial Stress Index (1) the spread of the 3-month interbank rate over the 3-month Government bill rate (Ind 1 ) (2) negative quarterly equity returns (Ind 2 ) (3) the realized volatility of the main equity index (as average daily absolute changes over a quarter) (Ind 3 ) (4) the realized volatility of the nominal effective exchange rate (Ind 4 ) (5) the realized volatility of the yield on the 3-month Government bill (Ind 5 ). Each component j of the index for country i at quarter t is transformed into an integer that ranges from 0 to 3 according to the country-specific quartile of the distribution:

13. 13 2.3 Vulnerability indicators 14 indicators of country-level macro-financial vulnerabilities: 14 indicators of country-level macro-financial vulnerabilities: Domestic = inflation, GDP growth, CA deficit, budget balance, credit growth, leverage, equity price growth, equity valuation Domestic = inflation, GDP growth, CA deficit, budget balance, credit growth, leverage, equity price growth, equity valuation Global = inflation, GDP growth, credit growth, leverage, equity price growth, equity valuation Global = inflation, GDP growth, credit growth, leverage, equity price growth, equity valuation Test several transformations of the indicators (over 200 transformations of the indicators tested). Test several transformations of the indicators (over 200 transformations of the indicators tested). Select best-performing (as a leading indicator) transformations of the variables Select best-performing (as a leading indicator) transformations of the variables

14. 14 2.4 Model training „Static“ model, i.e. model is not re-estimated recursively:„Static“ model, i.e. model is not re-estimated recursively: Training set (estimation sample): 1990Q4 - 2005Q1Training set (estimation sample): 1990Q4 - 2005Q1 Test set (out-of-sample): 2005Q2 - 2009Q2Test set (out-of-sample): 2005Q2 - 2009Q2 In the benchmark, we use 18 months as a forecast horizonIn the benchmark, we use 18 months as a forecast horizon Account for policymakers’ preferences when evaluating the performance as in Alessi and Detken (2011) (benchmark μ=0.5 )Account for policymakers’ preferences when evaluating the performance as in Alessi and Detken (2011) (benchmark μ=0.5 ) Data as an input to the SOFSMData as an input to the SOFSM Class variables + vulnerabilities for trainingClass variables + vulnerabilities for training Only vulnerabilities for mapping and evaluatingOnly vulnerabilities for mapping and evaluating Crisis probabilities as an output of the SOFSMCrisis probabilities as an output of the SOFSM Map data onto SOFSM and retrieve a crisis probabilityMap data onto SOFSM and retrieve a crisis probability

15. 15 2.5 Model evaluation Evaluate output using the „usefulness“ criterion (see e.g. Alessi and Detken (2011)) and compare it with a Lo Duca and Peltonen (2011)-type logit model Evaluate output using the „usefulness“ criterion (see e.g. Alessi and Detken (2011)) and compare it with a Lo Duca and Peltonen (2011)-type logit model Find the threshold that that depends on policymaker’ preferences between Find the threshold that minimizes a loss function that depends on policymaker’ preferences between type I and type II errors Define Define Usefulness “U” In the benchmark model, we set µ = i.e. we assume the policymaker is (“neutral external observer”)In the benchmark model, we set µ = 0.5, i.e. we assume the policymaker is equally concerned of missing systemic events and issuing false alarms (“neutral external observer”)

16. 16 2.5 Model evaluation Defining early warning nodes Defining early warning nodes When calibrating the policymakers’ preferences, we vary the thresholds. This changes the number of “early warning nodes”. When calibrating the policymakers’ preferences, we vary the thresholds. This changes the number of “early warning nodes”. µ =0.4 µ =0.5 µ =0.6 µ =0.4 µ =0.5 µ =0.6

17. 17 2.5 Model evaluation Training the SOM: Training the SOM: While a higher number of nodes M improves in-sample performance, it decreases generalization, i.e. out-of-sample performance. While a higher number of nodes M improves in-sample performance, it decreases generalization, i.e. out-of-sample performance. We increase M and findthe first model with Usefulness ≥ 0.25 (logit model). We increase M and find the first model with Usefulness ≥ 0.25 (logit model). In terms of “Usefulness”, when µ=0.5, the models are by definition very similar on in-sample data, while the SOM performs better on out-of-sample data In terms of “Usefulness”, when µ=0.5, the models are by definition very similar on in-sample data, while the SOM performs better on out-of-sample data Robustness is tested with respect to three aspects Robustness is tested with respect to three aspects SOM parameters: radius of neighborhood and number of nodes SOM parameters: radius of neighborhood and number of nodes Policymakers’ preferences Policymakers’ preferences Forecast horizon Forecast horizon The SOM detects more crises but issues also more ‘false’ warnings Reminder: Recall positives = TP/(TP+FN), Recall negatives = TN/(TN+FP), Precision positives = TP/(TP+FP), Precision negatives = TN/(TN+FN), FP rate = FP/(FP+TN), TP rate = TP/(FN+ TP), Accuracy=(TP+TN)/(FN+FP+TN+ TP).

18. 18 3. Mapping the State of Financial Stability – The two dimensional SOFSM This is the 2D SOFSM that represents multi-D data (i.e. maps the state of financial stability using vulnerabilities).This is the 2D SOFSM that represents multi-D data (i.e. maps the state of financial stability using vulnerabilities). The stages of the financial stability cycle are derived using the class variables (pre-crisis, crisis, post-crisis and tranquil periods)The stages of the financial stability cycle are derived using the class variables (pre-crisis, crisis, post-crisis and tranquil periods)

19. 19 3. Mapping the State of Financial Stability – Constructing the four clusters according to the financial stability cycle Clustering is performed using hierarchical clustering based on class variables. The map is partitioned into four clusters according to the financial stability cycle: a pre-crisis, crisis, post-crisis and tranquil cluster.Clustering is performed using hierarchical clustering based on class variables. The map is partitioned into four clusters according to the financial stability cycle: a pre-crisis, crisis, post-crisis and tranquil cluster. Co-occurrence of pre- and post- crisis

20. 20 3. Mapping the State of Financial Stability – The distribution of 14 indicators across the 4 clusters Domestic: early signs of crisis - equity growth and valuation, budget deficit, followed by real GDP and credit growth, leverage, budget surplus, and CA deficit. Domestic: early signs of crisis - equity growth and valuation, budget deficit, followed by real GDP and credit growth, leverage, budget surplus, and CA deficit. Global: early signs of crisis - equity growth and level, followed by real GDP growth, while global credit growth and leverage are more concurrent with crises. Global: early signs of crisis - equity growth and level, followed by real GDP growth, while global credit growth and leverage are more concurrent with crises.

21. 21 3. Mapping the State of Financial Stability – Temporal dimension Evolution of macro-financial conditions (all 14 indicators) for the United States and the Euro area (2002-10, first quarter) In 2010Q1, the euro area aggregate, did not reflect the crisis in GR, IE, PT. Financial Stress Index also decreased for the euro area aggregate in 2010Q1 US 2002 2010 US 2008–09 US 2007 US 2006 US 2004–05 US 2003 Euro 2002 Euro 2003 Euro 2004–05 Euro 2006 Euro 2007 Euro 2008 Euro 2009 Euro 2010

22. 22 3. Mapping the State of Financial Stability – Cross section Visualizing current macro-financial vulnerabilities in key advanced and emerging economies (2010Q3) Contagion through similarities in macro-financial vulnerabilities

23. 23 3. Mapping the State of Financial Stability – Regional evolution Evolution of the macro-financial conditions in Emerging Market Economies and Advanced Economies (2002-10, first quarter) Pre-crisis Crisis Post-crisis Tranquil

24. 24 4. Conclusions Self-Organizing Financial Stability Map is a useful model for financial stability surveillance:Self-Organizing Financial Stability Map is a useful model for financial stability surveillance: mapping the state of financial stability and visualizing multidimensional macro-financial vulnerabilitiesmapping the state of financial stability and visualizing multidimensional macro-financial vulnerabilities has good out-of-sample predictive capabilities of future systemic events / financial crises (EWS)has good out-of-sample predictive capabilities of future systemic events / financial crises (EWS) the SOFSM is flexible with respect to, e.g., events of interest, vulnerability indicators, forecast horizons, policymaker‘s preferencesthe SOFSM is flexible with respect to, e.g., events of interest, vulnerability indicators, forecast horizons, policymaker‘s preferences

25. 25 Thank you for your attention! Thank you for your attention!

26. 26 Extra slides

27. 27 2. Self-Organizing Maps (SOMs) – Kohonen training algorithm 1. Initialization using the two principal components 2. Finding the best-matching units where each data point x is matched with its BMU m c. 3. Update the reference vectors (or units or nodes) where the neighborhood h is given by where r is the 2D coordinates of m c and m i and is a specified tension value. 4. Cluster the units using Ward ’ s hierarchical clustering

28. 28 Descriptive statistics of the data Notes: Transformations: a, deviation from trend; b, annual change; c, level. KSL: Lilliefors' adaption of the Kolmogorov-Smirnov normality test. AD: the standard Anderson-Darling normality test. Significance levels: 1%, *.

29. 29 The logit model Notes: Significance levels: 1%, ***; 5 %, **; 10 %, *.

30. 30 Evaluation measures Recall positives(RP) = TP/(TP+FN), Recall negatives(RN) = TN/(TN+FP) Precision positives(PP) = TP/(TP+FP) Precision negatives(PN) = TN/(TN+FN) Overall accuracy = (TP+TN)/(TP+TN+FP+FN), FP rate = FP/(FP+TN) FN rate = FN/(FN+ TP) ROC curve = TP rate vs. FP rate AUC = Area under the ROC curve (probability that a randomly chosen positive instance is ranked higher than a randomly chosen negative instance)

31. 31 3. The SOM Model – the training framework Attempt to find a parsimonious, objective and interpretable model:Attempt to find a parsimonious, objective and interpretable model: 1. Train and evaluate in terms of in-sample U. Set threshold to max U. 2. For each M-value, order the models in a descending order. Find for each M-value the first model with U ≥ 0.25 (logit model). 3. 3. Evaluate the interpretability of the models chosen in Step 2. Choose the one that is easiest to interpret.

32. 32 3. The SOM Model – a comparison with a logit model In terms of U, the models are very similar in-sample, while out-of-sample they perform equally well when µ=0.4 and the SOM is better when µ=0.6. In terms of U, the models are very similar in-sample, while out-of-sample they perform equally well when µ=0.4 and the SOM is better when µ=0.6. Similarly as for µ=0.5, the classification of the models are of opposite nature for µ=0.6. For µ=0.4, the SOM model issues less false alarms and misses more crises. Similarly as for µ=0.5, the classification of the models are of opposite nature for µ=0.6. For µ=0.4, the SOM model issues less false alarms and misses more crises. Reminder: Recall positives = TP/(TP+FN), Recall negatives = TN/(TN+FP), Precision positives = TP/(TP+FP), Precision negatives = TN/(TN+FN), FP rate = FP/(FP+TN), TP rate = TP/(FN+ TP), ROC curve = TP rate vs. FP rate

33. 33 3. The SOM Model – a comparison with a logit model Robust over different horizons (24, 18, 12 and 6 months) Reminder: Recall positives = TP/(TP+FN), Recall negatives = TN/(TN+FP), Precision positives = TP/(TP+FP), Precision negatives = TN/(TN+FN), FP rate = FP/(FP+TN), TP rate = TP/(FN+ TP), ROC curve = TP rate vs. FP rate

34. 34 3. The SOM Model – a comparison with a logit model Reminder: Recall positives = TP/(TP+FN), Recall negatives = TN/(TN+FP), Precision positives = TP/(TP+FP), Precision negatives = TN/(TN+FN), FP rate = FP/(FP+TN), TP rate = TP/(FN+ TP), ROC curve = TP rate vs. FP rate, AUC = Area under the ROC curve (probability that a randomly chosen positive instance is ranked higher than a randomly chosen negative instance)

35. 35 3. Mapping the State of Financial Stability – Global evolution Evolution of the global macro-financial conditions (all 14 indicators) (2002-10, first quarter) Pre-crisis Crisis Post-crisis Tranquil

36. 36 4. Visual Analysis of Systemic Events – financial stability cycle Evolution of the macro-financial conditions in China (2007Q4-2010Q3, quarterly observations) Pre-crisis Crisis Post-crisis Tranquil

37. 37 4. Visual Analysis of Systemic Events – financial stability cycle Evolution of the macro-financial conditions in Russia (2007Q4-2010Q3, quarterly observations) Pre-crisis Crisis Post-crisis Tranquil

38. 38 Deleted slides

39. 39 1. Introduction - motivation Frequent occurrence of financial distressFrequent occurrence of financial distress EWS literature based on conventional statistical techniques:EWS literature based on conventional statistical techniques: –Univariate - signalling approach (e.g. KLR, 1997) –Multivariate - logit/probit models (e.g. Berg & Pattillo, 1998) The above methods have certain limitations:The above methods have certain limitations: –No interactions between indicators –Distributional assumptions on crisis probabilities –No visualization of multivariate data Non-parametric methods (Peltonen, 2006)Non-parametric methods (Peltonen, 2006) –Improve accuracy and predictive power, but do not focus on visualization

40. 40 1. Introduction - motivation Visualizing the state of financial stability is a challenging task:Visualizing the state of financial stability is a challenging task: –Dimensionality problem: as a large number of indicators are required to describe various aspects of vulnerabilities and to assess systemic risks –How to visualize multidimensional indicators on a static two/three dimensional chart? –Temporal and cross-section aspects complicates the matter Combining a method with a good visualization capacitiy with a good out-of-sample predictions of systemic events is even more challenging…Combining a method with a good visualization capacitiy with a good out-of-sample predictions of systemic events is even more challenging…