1. SOME PRACTICAL ISSUES IN COMPOSITE INDEX CONSTRUCTION Lino Briguglio and Nadia Farrugia Department of Economics, University of Malta Prepared for the INTERNATIONAL CONFERENCE ON SMALL STATES AND ECONOMIC RESILIENCE Organised by the Commonwealth Secretariat, London and the Islands and Small States Institute of the Foundation for International Studies at the University of Malta VALLETTA, MALTA 23 - 25 April 2007

2. Layout 1. Introduction 2. Normalisation: Rescaling Formula 3. Weighting: Regression Method 4. Imputation of Missing Data 5. Scoring on a Multi-Point Mapping Scale 6. Conclusion

3. 1. Introduction This presentation discusses some of the methodological approaches commonly used in constructing composite indices. This presentation discusses some of the methodological approaches commonly used in constructing composite indices. The main problems encountered relate to: (1) normalisation procedures (2) weighting procedures (3) missing data, and (4) unavailability of quantitative observations. The main problems encountered relate to: (1) normalisation procedures (2) weighting procedures (3) missing data, and (4) unavailability of quantitative observations. There are variant methodologies, other than the ones described in this presentation. There are variant methodologies, other than the ones described in this presentation.

4. 2. Normalisation: The Rescaling Formula Components (sub-indices) of a composite index are often measured in different units and so straightforward summation would not be valid. Components (sub-indices) of a composite index are often measured in different units and so straightforward summation would not be valid. The Re-scaling formula for a particular observation is: The Re-scaling formula for a particular observation is:R=(X-MinX)/(MaxX-MinX) Where: R is the rescaled observation X is the actual observation Min and Max are the Minimum and Maximum In this way the observation is rescaled to take a value between 0 and 1. This is done for all the observations of a particular component of the index. In this way the observation is rescaled to take a value between 0 and 1. This is done for all the observations of a particular component of the index. The sub-indices are then summed up using a weighting system. The sub-indices are then summed up using a weighting system.

5. Example: Rescaling Formula

6. Comments on the Rescaling Formula This procedure is commonly used. This procedure is commonly used. It is the procedure used in the Economic Resilience. Index computed by the University of Malta. It is the procedure used in the Economic Resilience. Index computed by the University of Malta. It is also used in the Human Development Index. It is also used in the Human Development Index. Its main defect is that the distribution of the normalised variables are heavily influenced by outlier observations. Its main defect is that the distribution of the normalised variables are heavily influenced by outlier observations.

7. 3. Weighting Very often the components (sub-indices) are assigned equal weights to compute an average. Very often the components (sub-indices) are assigned equal weights to compute an average. Sometimes unequal weights are assigned on the basis of prior knowledge or expert views. Sometimes unequal weights are assigned on the basis of prior knowledge or expert views. It is often difficult to use a scientific method to assign weights in composite indices. It is often difficult to use a scientific method to assign weights in composite indices. Atkins et al use the regression method approach as described in the next slide. Atkins et al use the regression method approach as described in the next slide.

8. Weighting: Regression Method Weighting: Regression Method The basic assumption is that a dependent variable can be found as a proxy for the composite index. The basic assumption is that a dependent variable can be found as a proxy for the composite index. This is then regressed on a number of explanatory variables which represent the index components. This is then regressed on a number of explanatory variables which represent the index components. It lets the data produce the weights and does not require normalisation of the observations. It lets the data produce the weights and does not require normalisation of the observations. The weights for averaging the components of the index are taken as the coefficients on the explanatory variables of the estimated equation. The weights for averaging the components of the index are taken as the coefficients on the explanatory variables of the estimated equation. This procedure is used in the computation of the Commonwealth Vulnerability Index. This procedure is used in the computation of the Commonwealth Vulnerability Index.

9. Example: Weighting Regression Method Income volatility = Susceptibility to environmental events and hazards + Economic Exposure + Diversification Incvol= 1.5448 (a regression constant) +0.0103 Hazards +0.0290 Economic exposure +3.2523 Diversification

10. 4. Imputation of Missing Data Sometimes data for certain countries is missing. One approach is to use the regression method to impute the missing data. One approach is to use the regression method to impute the missing data. A cross section regression is run among the countries for which complete data is available. A cross section regression is run among the countries for which complete data is available. Two variables are used in the regression, namely (a) the average of all the components and (b) the average of all the components less that where data is missing. Two variables are used in the regression, namely (a) the average of all the components and (b) the average of all the components less that where data is missing. This is done in order to find out how the omission of the survey data affects the area rating. This is done in order to find out how the omission of the survey data affects the area rating.

11. Imputation of Missing Data The regression estimates are then used to adjust the averages for countries with a missing component. The regression estimates are then used to adjust the averages for countries with a missing component. While these statistical adjustments enhance the overall comparability among the countries, comparisons between the nations that have the complete data set and the nations that do not should be made with a degree of caution. While these statistical adjustments enhance the overall comparability among the countries, comparisons between the nations that have the complete data set and the nations that do not should be made with a degree of caution. This approach is used by the Economic Freedom of the World Index. This approach is used by the Economic Freedom of the World Index.

12. Example: Imputing Missing Observation The regression equation is The regression equation is AVGX1-5 = 0.7667 + 0.7548 AVGX1-4

13. Comment on Missing Data The main defect of this approach is that for every country the missing observation is conditioned by the observations in other countries.

14. 5. Scoring on a Mapping Scale This involves categorising an occurrence (in terms of intensity or frequency) along a scale of say 1 to 7 with 1 being the lowest possible occurrence and 7 the highest possible. This involves categorising an occurrence (in terms of intensity or frequency) along a scale of say 1 to 7 with 1 being the lowest possible occurrence and 7 the highest possible. The wider the spread of the scale, the more possible it will be to derive meaningful standard deviations of the averages obtained. Usually 5- or 7-point scale is used. The wider the spread of the scale, the more possible it will be to derive meaningful standard deviations of the averages obtained. Usually 5- or 7-point scale is used. 1234567 Absent Very Rare RareAverage Moderately higher than average frequency Markedly higher than average frequency Highest possible frequency

15. Scoring on a Mapping Scale This approach is very useful when data is qualitative, and when the researcher desires to transform it into a quantitative format. This approach is very useful when data is qualitative, and when the researcher desires to transform it into a quantitative format. It also permits non-linearity (exponential, U- or S-shaped) e.g. Calorie intake is very bad in small and in big doses It also permits non-linearity (exponential, U- or S-shaped) e.g. Calorie intake is very bad in small and in big doses This approach is used by Kaly et al in the construction of the Environmental Vulnerability Index. This approach is used by Kaly et al in the construction of the Environmental Vulnerability Index. Its main defect relates to the subjectivity of the category groupings and the choice between linear and non-linear relationships. Its main defect relates to the subjectivity of the category groupings and the choice between linear and non-linear relationships.

16. Example: Multi-Point Mapping Scale 1234567 VolcanoesX=01X<55X<1010X<1515X<2020X3535X Toxic Industries X55<X1010<X2020<X5050<X100100<X200X>200 Land Slides X=00<X0.50.5<X11<X1.5 1.5< X 22<X2.5X>2.5 Selected Variables from Kaly et al (2003): Environmental Vulnerability Index

17. 6. Conclusion Composite indices construction involves subjective decisions. Composite indices construction involves subjective decisions. It is important that one fully understands the implications of a methodological procedure used as this has serious implications on the quality of the results of the composite index, its reliability and its operationality. It is important that one fully understands the implications of a methodological procedure used as this has serious implications on the quality of the results of the composite index, its reliability and its operationality.

18. Thank you for your attention