1. Benchmarking money manager performance: Issues & evidence Louis K. C. Chan University of Illinois Urbana- Champaign March 2006

2. Objectives The evaluation and attribution of investment performance is crucial for investment research and practice –Money manager performance –Results of investment strategies & trading rules –Effects of managerial decisions on shareholder wealth Academic and practitioner research has produced a large array of methods for evaluating and attributing investment performance

3. Objectives Question: are conclusions sensitive to the choice of evaluation and attribution methods? why? We compare the results from various methods applied to common samples –Set of active institutional money managers –Passive indexes

4. Evaluating method performance Many widely-used methods draw on evidence from asset pricing studies that size, value/growth describe much of the variation in returns (notably Fama and French (1992), Fama and French (1993)) We concentrate on benchmarking methods based on size, value/growth –Characteristic-matched control portfolios –Time-series factor model regressions –Effective asset mix regressions –Cross-sectional regressions on characteristics 1998 – 2000 market boom as stress test of benchmarking methods

5. Evaluating manager performance Much previous work on evaluating performance of mutual and closed-end funds (e.g. Jensen (1968), Elton et al. (1993), Malkiel (1995), Gruber (1995), Carhart (1997), Daniel et al. (1997), Kothari and Warner (2001), etc.) Managers of pension plan equity assets are just as important, but much less previous research (see LSV 1992, Coggin et al. 1993)

6. A first look: characteristic-matched portfolios vs. 3 factor model

8. Benchmark details Benchmarks vary according to –Characteristics or loadings –Measuring size, value/growth style –Treating size, value/growth effects separately –Portfolio weighting scheme –Frequency of benchmark reconstitution

9. Benchmark details Characteristics versus loadings –Predict benchmark return using portfolio’s attributes (size, book-to-market …) or predict benchmark return using portfolio’s loadings on factors –Some evidence that attributes predict returns better than loadings (Daniel and Titman 1997) –Data on holdings not generally accessible

10. Building performance benchmarks Measuring size, value/growth style –Size: market capitalization (float?) –Value/growth orientation usually measured by book-to-market ratio (book value of equity divided by market value of equity) –Book value of equity does not record value of intangible assets; includes goodwill from acquisitions

11. Building performance benchmarks Treating size, value/growth effects separately –E.g. independent 2-way sorts by size, BM –In one-way sorts by book-to-market equity large stocks typically are classified as growth –Under an independent size/BM sort procedure large-cap managers, regardless of large value/large growth style, will tend to be compared against a growth benchmark

12. Building performance benchmarks Weighting scheme for stocks in benchmark –Equal-weighting –Value-weighting Benchmark reconstitution frequency –Over time benchmark becomes more heterogeneous and may no longer correspond to managed portfolio’s features

13. Data Holdings and returns every quarter for 199 portfolios offered by money managers to clients, 1989Q1 - 2001Q4 Domestic U.S. equity portfolios only Different styles (large/mid/small, value/blend/growth) Some selection bias

14. Results outline Performance relative to benchmarks based on characteristics –Overall active manager sample –Classified by investment style –Diagnostics Performance relative to benchmarks based on loadings –Overall active manager sample –Classified by investment style –Diagnostics

15. Performance measures Abnormal return = portfolio’s return minus return on benchmark portfolio Tracking error volatility = standard deviation of quarterly difference between portfolio’s return and benchmark’s return

19. Benchmark performance

21. Benchmark comparisons

24. Performance based on regression benchmarks Three factor model excess return is ( r pt – r ft ) – benchmark return benchmark return is from fitted regression β (r mt – r ft ) + s SMB t + h HML t

25. Regression-based benchmark details Exposures estimated –over full period (including the quarter when we measure performance) –or leaving out the quarter when we measure performance Measuring size, value/growth factors –High versus low book-to-market –Other indicators of value/growth orientation

26. Building regression-based benchmarks 3 factor model accounts for size, value/growth separately E.g. benchmark return for small value manager = return for market exposure plus return for smallness plus return for value Benchmark credits manager for smallness even though small stocks’ performance is because small growth does better than small value

27. Regression-based benchmarks Alternative: compare manager to a selection of passive benchmarks (effective asset mix regressions) r pt = α + w 1 *LG t + w 2 *LV t + w 3 *MCG t + w 4 *MCV t + w 5 *SG t + w 6 *SV t + υ pt w 1, …,w 6 portfolio weights (between 0 and 1, add up to 1)

28. Building regression-based benchmarks Another widely-used alternative: each stock’s predicted return is from a cross- sectional regression using stock characteristics, industry dummy variables r it = α + β 1 *X 1i + β 2 *X 2i + …

32. Regression-based benchmark comparisons

35. Conclusions Benchmarking methods that appear similar on the surface can lead to very different conclusions about investment performance Popular methods (characteristic-matched reference portfolios, 3 factor time series regression models, cross-sectional regression) have disappointing ability to track managed active portfolios and passive benchmarks

36. Conclusions Methods based on within-size classifications, use multiple measures of value-growth orientation, improve ability to track managed and passive portfolios Given the fragility in reliably separating skill from style, detailed decomposition and attribution of performance should be treated with caution