Catalyst Stocks Premium Stock Pick Service

Tests of market efficiency look at the whether specific investment strategies or portfolio managers beat the market. But what does beating the market involve? Does it just imply that someone earns a return greater than what the market (say, the S&P 500) earns in a specific year? We will begin by looking at what beating the market involves and define what we mean be excess returns. We will then follow up by looking at three standard ways of testing market efficiency and when and why we may choose one over the other.

The fundamental question that we often attempt to answer when we test an investment strategy is whether the return we earn from the strategy is above or below a benchmark return on an alterative strategy of equivalent risk. But what should that benchmark return be? As we shall see, it is almost impossible to measure the success or failure of an investment strategy without taking a point of view on how risk should be measured.

If you can estimate the returns that you could have made by adopting an investment strategy in the past or observed the returns made by a portfolio manager or investor over a period, you can evaluate those returns. To make the evaluation, you have to choose an appropriate benchmark. In this section, we will consider the alternatives that are available to us in making this choice.

When you have estimated the returns on a strategy, the simplest comparison you can make is to the returns you would have made by investing in an index. Many portfolio managers and investors still compare the returns they make on their portfolios to the returns on the S& P 500. While this comparison may be simple, it can also be dangerous when you have a strategy that does not have the same risk as investing in the index and the bias can cut both ways. If you have a strategy that is riskier than investing in the index - say investing in small, high growth stocks - you are biasing yourself towards concluding that the strategy works (i.e., it beats the market). If you have a strategy that is much safer than investing in the index, such as buying high dividend paying, mature companies, you are biasing yourself towards concluding that the strategy does not work.

There are slightly more sophisticated versions of this approach that are less susceptible to this problem. For instance, some services that judge mutual funds do so by comparing them to an index of funds that have the same style as the fund being judged. Thus, a fund that invests in large market cap companies with low price to book ratios will be compared to other large-cap, value funds. The peril remains, though, since categorizing investors into neat boxes is easier said than done. A fund manager may begin the year calling herself a large-cap, value investor and during the course of the year shift to being an investor in high growth, risky companies.

In chapter 3, we considered the basics of risk and put forth several risk and return models. All of these models tried to measure the risk in an investment, though they differed on how best to measure it, and related the expected return on the investment to the risk measure. You could use these models to measure the risk in an investment strategy, and then examine the returns relative to this risk measure. We will consider some of these risk-adjusted measures of performance in this section.

a. Mean Variance Measures

The simplest measures of risk-adjusted performance have their roots in the mean-variance framework developed by Harry Markowitz in the early 1950s. In the mean-variance world, the standard deviation of an investment measures its risk and the return earned is the reward. If you compare two investments with the same standard deviation in returns, the investment with the higher average return would be considered to be the better one.

Extending this concept to investment strategies, you could look at the payoff to each unit of risk taken by dividing the return earned using the strategy by the standard deviation of return, in a measure called the Sharpe ratio.4

Sharpe Ratio = Average Return on Strategy/ Standard deviation of Returns from Strategy To compute the standard deviation, you would need to track the returns each period for several periods. For instance, the average monthly returns over the last 5 years at mutual funds can be divided by the standard deviations in monthly returns at mutual funds over the 5 years to come up with Sharpe ratios for mutual funds. Once you have the Sharpe ratios for individual funds, you can compare them across funds to find the funds that earn the highest reward per unit of risk (standard deviation) or you can compare the fund's ratios to the Sharpe ratio for the entire market to make a judgment on whether active investing pays.

The Sharpe ratio is a versatile measure that has endured the test of time. Its focus on the standard deviation as the measure of risk does bias it against portfolios that are not diversified widely across the market. A sector-specific mutual fund (such as a bio-tech or health care fund) will tend to do poorly on a Sharpe ratio basis because its standard deviation will be higher because of the presence of sector-specific

Funds with highest Sharpe Ratios: Take a look at the 50 mutual funds with the highest Sharpe ratios.

4 This measure was developed by Bill Sharpe in the mid 1960s as a measure of mutual fund performance.

risk. Since investors in these funds can diversity away that risk by holding multiple funds, it does seem unfair to penalize these funds for them.

Illustration 6.1: Computing Sharpe Ratios for largest U.S. Mutual funds - 1997-2001

In table 6.1, we compute the Sharpe ratios for the 15 largest mutual funds in the United States in March 2002, using data from 1997 to 2001.

Table 6.1: Sharpe Ratios for Large U.S. Mutual Funds: 1997-2001

Fund Name |
Average Return |
Standard deviation |
Sharpe Ratio |

10.72% |
20.82% |
0.51 | |

Vanguard 500 Index |
10.14% |
19.63% |
0.52 |

American Funds Inv Co Amer A |
12.84% |
15.69% |
0.82 |

American Funds Wash Mutual A |
12.36% |
16.64% |
0.74 |

American Funds Growth Fund A |
17.66% |
25.00% |
0.71 |

Fidelity Growth & Income |
10.32% |
16.10% |
0.64 |

Fidelity Contrafund |
11.47% |
18.33% |
0.63 |

American Funds New Prspctv A |
11.82% |
18.33% |
0.64 |

American Funds EuroPacific A |
7.20% |
18.75% |
0.38 |

AmCent Ultra Inv |
10.17% |
26.15% |
0.39 |

Janus |
9.57% |
27.31% |
0.35 |

Vanguard Instl Index |
10.27% |
19.65% |
0.52 |

Fidelity EquityIncome |
10.39% |
17.89% |
0.58 |

S & P 500 |
10.35% |
19.55% |
0.53 |

Note that the American funds delivered Sharpe ratios that were much higher than the S&P 500. In contrast, the Janus fund delivered a lower Sharpe ratio. The two biggest funds, Fidelity Magellan and the Vanguard 500 (not surprisingly), delivered Sharpe ratios that were very similar to the S&P 500.

A close relative of the Sharpe ratio is the information ratio. It is the ratio of the excess return earned by a fund over an index to the excess volatility of this fund to the volatility of the index. To measure the latter, we estimate what is commonly called tracking error, which measures the deviations of the fund returns from the index returns each period over several periods. In its most common form, the excess return over the S&P 500 for a fund is divided by the tracking error of the fund relative to the S&P 500.

Information Ratio = (Return on Strategy - Return on Index)/ Tracking Error versus the Index

Information ratios differ from Sharpe ratios because of their fidelity to an index. In other words, you can have a portfolio with low standard deviation but it can have high tracking error, if it contains stocks that are not in the index. For instance, a portfolio of low risk stocks with low market capitalization may have low standard error but it will still have a high tracking error versus the S&P 500.

In the 1990s, these measures were refined slightly to come up with a measure called M squared.5 Instead of dividing the total return of a strategy or fund by its standard deviation, you compute the expected return you would have had on the fund, if you had to adjust its standard deviation down to that of the index and compare this expected return to the return on the index. For instance, assume that you have a fund with a return of 30% and a standard deviation of 50% and that the return on the S&P 500 is 15% and the standard deviation of the S&P 500 is 20%. To make the fund's standard deviation comparable to that of the S&P 500, you would have had to invest 60% of your money in T.Bills (earning 3%) and 40% in the fund:

Adjusted Standard Deviation of Portfolio = .4 (Std deviation of fund) + .6(0)

= .4 (50%) = 20% The return on this portfolio can then be calculated: Expected return on portfolio = .4 (30%) + .6 (3%) = 13.8%

Since this return is lower than the S&P's return of 15%, you would categorize this fund as an underperformer.

This measure of performance is closely related to the Sharpe ratio and is susceptible to the same biases. Since the expected return is adjusted to make the risk of the mutual fund similar to that of the index, funds that are not diversified widely across the market will score poorly.

b. Capital Asset Pricing Model

The capital asset pricing model emerged from the mean-variance framework to become the first model for risk and return in finance. In the CAPM, as described in chapter 3, the expected return on an investment can be written as function of its beta:

5 The developed of this measure was Leah Modigliani, a strategist at Morgan Stanley. Without taking anything away from her accomplishments, it is worth noting that she is the daughter of Franco Modigliani, Nobel price winner in Economics.

Expected return = Riskfree rate + Beta (Expected return on market - Riskfree rate) In chapter 2, we used the model to estimate the expected returns for the next period, using the current riskfree rate, the beta and the average premium earned by stocks over the riskfree rate as inputs. In this section, we will consider how the capital asset pricing model can be adapted to judge past performance.

i. Excess Return (Alpha or Jensen's Alpha)

The simplest way to use to the capital asset pricing model to evaluate performance is to compare the actual return to the return your investment or strategy should have made over the evaluation period, given its beta and given what the market did over the period. As an example, assume that you are analyzing a strategy that generated 12% in returns over the last year. Assume that your calculations indicate that the strategy has a beta of 1.2, and that the riskfree rate at the beginning of the last year was 4% and that the return on the market over the last year was 11%. You can compute the excess return as follows: Expected return over last year = 4% + 1.2 (11% - 4%) = 12.4% Excess Return = Actual Return - Expected Return = 12% - 12.4% = -0.4% This strategy under performed the market, after adjusting for risk, by 0.4%. This excess return is also called an abnormal return.

What are the differences between what we are doing here and what we did in chapter 2 to forecast expected returns in the future? The first is that we use the riskfree rate at the beginning of the evaluation period when we do evaluation, whereas we use the current riskfree rate when making forecasts. The second is that we use the actual return on the market over the period, even if it is negative, when we do evaluation rather than the historical or an implied equity premium that we use when computing forecasted returns. Finally, the beta we use in evaluation should measure the risk you were exposed to during your evaluation period, while a forward-looking beta should be used for forecasts.

The excess returns on a strategy can be computed for any return period you want -daily, weekly, monthly or annual. You would need to adjust your riskfree rate and market return appropriately, using, for instance, the weekly riskfree rate and a weekly market return if you want to compute weekly excess returns from a strategy. An alternative approach to estimating the excess return, which should yield the same results, is to run a regression of the returns on your strategy, in excess of the riskfree rate, against return on a market index, in excess of the riskfree rate.

(Return on strategy - Riskfree rate) = a + b (Returns on market index - Riskfree rate)

The slope of this regression gives you the historical beta, but the intercept of this regression yields the excess return by period for your strategy.6 Using the statistical term for the intercept, the excess return is often called an alpha. In some quarters, it is called Jensen's alpha, reflecting the fact that it was first used in a study of mutual funds in the 1960s by Michael Jensen, one of the pioneers in empirical finance from the University of Chicago.

There is one final point that should be made about excess returns. When you compute excess returns by day or week over a longer period (say six months or a year), you may also want to compute how the strategy performed over the entire period. To do this, you usually look at the compounded return over the period. This compounded return is called a cumulative excess return or a cumulative abnormal return (CAR). Defining the excess return in each interval as ER^ you can write the excess return over a period as follows:

Cumulative Abnormal return over n intervals= (1+ ER1) (1+ ER2) (1+ ER3)... (1+ ERn) A cumulative abnormal return that is greater than zero indicates that the strategy beat the market, at least over the period of your test.

Unlike the variance-based measures in the last section, Jensen's alpha does not penalize sector-specific funds that are not diversified because it looks at the beta of a portfolio and not its standard deviation. The measure's fidelity to the capital asset pricing model, however, exposes it to all of the model's limitations. Since the model has historically under estimated the expected returns of small cap stocks, with low PE and low price to book ratios, you will tend to find that strategies that focus on stocks with these characteristics earn positive excess returns.

There are variations that have appeared on Jensen's alpha. An early variation replaced the capital asset pricing model with what is commonly called the market model, where the expected return on an investment is based upon a past regression alpha.7 In the

6 To see why, lets work through the algebra. The expected return in the CAPM can be written as: Expected Return on Strategy = Riskfree Rate + Beta (Return on market - Riskfree rate) Expected Return on Strategy - Riskfree Rate = Beta (Return on market - Riskfree rate)

In other words, if your stock did exactly as predicted by the CAPM, the intercept should be zero. If the intercept is different from zero, that must indicate under performance (if it is negative) or out performance (if it is positive)

7 In the market model, the excess return is written as Excess return = Actual return - (a + b Return on Market)

Where a is the intercept and b is the slope of a regression of returns on the stock against returns on the market index.

last decade, for instance, researchers have developed a version of the measure that allows the beta to change from period to period for a strategy - these are called time varying betas. This is clearly more realistic than assuming one beta for the entire testing period.

ii. Treynor Index

The excess return is a percentage measure. But is earning a 1% excess return over an expected return of 15% equivalent to earning a 1% excess return over an expected return of 7%? There are many who would argue that the latter strategy is a more impressive one. The Treynor Index attempts to correct for this by converting the excess return into a ratio, relative to the beta.8 It is computed by dividing the difference between the returns on a strategy and the riskfree rate by the beta of the investment. This value is then compared to the difference between the returns on the market and the riskfree rate. Treynor Index = (Return on Strategy - Riskfree Rate)/ Beta

To illustrate, assume that you are considering the strategy that we described in the last section with a beta of 1.2 that earned a return of 12% in the most recent year. In that example, the return on the market over the same year was 11% and the riskfree rate was 4%.

The Treynor Index for this strategy would be

Treynor Index for Strategy= (12% - 4%)/ 1.2 = 6.67%

This strategy underperformed the market.

The Treynor Index is closely related to the alpha measure described in the last section. The measures will always agree on whether a strategy under or out performs the market, but will disagree on rankings. The Treynor Index will rank lower beta strategies higher than the alpha measure, because it looks at excess returns earned per unit beta.

Illustration 6.2: Estimating Jensen's Alpha and Treynor's Index: Large U.S. Mutual Funds

In table 6.2 below, we estimate Jensen's alpha and Treynor's Index for the 15 largest mutual funds in the United States. For simplicity, we assumed that the average annual risk free rate during the period was 5%.

Fund Name |
Return on Fund |
Beta |
Expected Return |
Jensen's Alpha |
Treynor's Index |

Fidelity Magellan |
10.72% |
1.02 |
10.46% |
0.26% |
5.61% |

Vanguard 500 Index |
10.14% |
1.00 |
10.35% |
-0.21% |
5.14% |

American Funds Inv Co Amer A |
12.84% |
0.71 |
8.80% |
4.04% |
11.04% |

American Funds Wash Mutual A |
12.36% |
0.58 |
8.10% |
4.26% |
12.69% |

American Funds Growth Fund A |
17.66% |
1.07 |
10.72% |
6.94% |
11.83% |

Fidelity Growth & Income |
10.32% |
0.70 |
8.75% |
1.58% |
7.60% |

Fidelity Contrafund |
11.47% |
0.67 |
8.58% |
2.89% |
9.66% |

American Funds New Prspctv A |
11.82% |
0.82 |
9.39% |
2.43% |
8.32% |

American Funds EuroPacific A |
7.20% |
0.78 |
9.17% |
-1.97% |
2.82% |

AmCent Ultra Inv |
10.17% |
1.26 |
11.74% |
-1.57% |
4.10% |

Janus |
9.57% |
1.35 |
12.22% |
-2.65% |
3.39% |

Vanguard Instl Index |
10.27% |
1.00 |
10.35% |
-0.08% |
5.27% |

Fidelity EquityIncome |
10.39% |
0.70 |
8.75% |
1.65% |
7.70% |

S&P 500 |
10.35% |
1 |
10.35% |
0.00% |
5.35% |

While Fidelity Magellan and the Vanguard 500 index fund have alphas close to zero and Treynor indices that match the market, the American Funds Growth Fund earned an annual alpha of 6.94% over the period.

c. Arbitrage Pricing and Multi-factor Models

In chapter 2, we noted that the assumptions that we need to arrive at the single market beta measure of risk in the capital asset pricing model are unrealistic and that the model itself systematically under estimates the expected returns for stocks with certain characteristics - low market capitalization and low PE. We considered the alternative of the arbitrage pricing model, which allows for multiple market risk factors which are unidentified or a multi-factor model, which relates expected returns to a number of macro-economic factors such as interest rates, inflation and economic growth. These models, we argued, allow us more flexibility when it comes to estimating expected returns.

You could use either the arbitrage pricing or multi-factor model to estimate the return you would have expected to earn over a period on a portfolio and compare this return to the actual return earned. In other words, you could compute an excess return on alpha for a strategy or portfolio using these models instead of the capital asset pricing model.

To the extent that the arbitrage pricing and multi-factor models are less likely to yield biased returns for small cap and low PE stocks, you could argue that the excess returns from these models should give you better measures of performance. The biggest problem that you run into in using these models to evaluate the excess returns earned by a portfolio manager or a strategy is that the portfolios themselves may be constantly shifting. What you measure as an alpha from these models may really reflect your failure to correct for the variation in exposure to different market risk factors over time. While this is also a problem with the capital asset pricing model, it is far easier to adjust a single beta over time than it is to work with multiple betas.

d. Proxy and Composite Models

The alternative to conventional risk and return models is the use of a proxy model, where the returns on stocks are correlated with observable financial characteristics of the firm. Perhaps the best known proxy model was the one developed by Fama and French, which we presented in chapter 2. They found that between 1962 and 1990, stocks with lower market capitalization and price to book ratios consistently earned higher returns than larger market capitalization companies with higher price to book ratios. In fact, market capitalization and price to book ratio differences across firms explained far more of the variation in actual returns than betas did.

Building on this theme, traditional risk and return models may fall short when it comes to estimating expected returns for portfolios that have disproportionately large exposures to small cap or low price to book value stocks. These portfolios will look like they earn excess returns. Using a proxy model, where the returns on the portfolio are conditioned on the market cap of the stocks held in the portfolio and their price to book ratios may eliminate this bias:

Expected return on portfolio = a + b (Average Market Capitalization)Portfolio + c (Average Price to Book Ratio^^

This model can even be expanded to include a conventional market beta, yielding what is often called a three-factor model:

Expected return on portfolio = a + b (Market beta) + c (Average Market Capitalization)Portfolio + d (Average Price to Book Ratio)Portfolio

The perils of incorporating variables such as market capitalization and price to book ratios into expected returns is that you run a risk of creating a self-fulfilling prophecy. If markets routinely misprice certain types of companies - small companies, for instance - and we insist on including these variables in the expected return regressions, we will be biased, with a complete enough model, towards finding that markets are efficient. In fact, in recent years, researchers have added a fourth factor - price momentum - to these factor models, because of recent findings that companies that have done well in the recent past are likely to continue doing well in the future.

There are two closing points that we would like to emphasize about the use of risk and return models and tests of market efficiency. The first is that a test of market efficiency is a joint test of market efficiency and the efficacy of the model used for expected returns. When there is evidence of excess returns in a test of market efficiency, it can indicate that markets are inefficient or that the model used to compute expected returns is wrong or both. While this may seem to present an insoluble dilemma, if the conclusions of the study are insensitive to different model specifications, it is much more likely that the results are being driven by true market inefficiencies and not just by model misspecifications.

In terms of which approach you should use to come up with expected returns, it is worth noting that each approach has its own built in biases that you need to be aware of. Table 6.3 below summarizes the alternative approaches to evaluating returns and the types of strategies and portfolios that they are likely to be biased towards and against.

There are a number of different ways of testing for market efficiency, and the approach used will depend in great part on the investment scheme being tested. A scheme based upon trading on information events (stock splits, earnings announcements or acquisition announcements) is likely to be tested using an 'event study' where returns around the event are scrutinized for evidence of excess returns. A scheme based upon trading on a observable characteristic of a firm (price earnings ratios, price book value ratios or dividend yields) is likely to be tested using a 'portfolio' approach, where portfolios of stocks with these characteristics are created and tracked over time to see if, in fact, they make excess returns. An alternative way of testing to see if there is a relationship between an observable characteristic and returns is to run a regression of the latter on the former. This approach allows for more flexibility if you are testing for interactions among variables. The following pages summarize the key steps involved in each of these approaches, and some potential pitfalls to watch out for when conducting or using these tests.

An event study is designed to examine market reactions to, and excess returns around specific information events. The information events can be market-wide, such as macro-economic announcements, or firm-specific, such as earnings or dividend announcements. The steps in an event study are as follows -

(1) The event to be studied is clearly identified, and the date on which the event was announced pinpointed. The presumption in event studies is that the timing of the event is known with a fair degree of certainty. Since financial markets react to the information about an event, rather than the event itself, most event studies are centered around the announcement date9 for the event.

(2) Once the event dates are known, returns are collected around these dates for each of the firms in the sample. In doing so, two decisions have to be made. First, you have to decide whether to collect weekly, daily or shorter-interval returns around the event. This will, in part, be decided by how precisely the event date is known (the more precise, the more likely it is that shorter return intervals can be used) and by how quickly information is reflected in

9 In most financial transactions, the announcement date tends to precede the event date by several days and, sometimes, weeks.

prices (the faster the adjustment, the shorter the return interval to use). Second, you have to determine how many periods of returns before and after the announcement date will be considered as part of the 'event window'. That decision also will be determined by the precision of the event date, since more imprecise dates will require longer windows.

Return window: -n to +n where,

Rjt = Returns on firm j for period t (t = -n, ...,0, +n)

(3) The returns, by period, around the announcement date, are adjusted for market performance and risk to arrive at excess returns for each firm in the sample. You could use any of the risk and return models described in the last section to estimate excess returns. For instance, if the capital asset pricing model is used to control for risk -

Excess Return on period t = Return on day t - (Riskfree rate + Beta * Return on market on day t)

ER-jn ERj0 ER+jn

Return window: -n to +n where,

ERjt = Excess Returns on firm j for period t (t = -n, ...,0, +n) You can also look at how a portfolio held over multiple periods would have done by measuring a cumulated abnormal return (CAR) by compounding the excess returns over the periods. Thus, if your excess return on day 1 is +2%, day 2 is -1% and day 3 is +1.5%, your cumulative excess return over all three days would be: Cumulated Excess Return = (!+ ERj) (!+ ER2) (!+ ER3) - 1

(4) Once the excess returns are estimated for each firm in the sample, the average excess returns can be computed across the firms and it will almost never be equal to zero. To test to see whether this number is significantly different from zero, however, you need a statistical test. The simplest is to compute a standard deviation in the excess returns across the sampled firms, and to use this to estimate a t statistic. Thus, if you have N firms in your sample and you have computed the excess returns each day for these firms:

Was this article helpful?

Get All The Support And Guidance You Need To Be A Success At Investing In Stocks And Shares. This Book Is One Of The Most Valuable Resources In The World When It Comes To

## Post a comment