Beware of Threshold Distortion: Edited

This is heavily edited. For the full article, see Beware of the Threshold Distortion.
Beware of the Threshold Distortion

There is one trap that I have seen much too often. It is using thresholds in the presence of randomness (noise). It is used in arguing against the existence of investor skill. I say this very bluntly: this method of presentation propagates a lie.

Here is the method:

A person observes the initial price of a stock or mutual fund to see how well it does over a fixed number of years. This can be as short as one year. It is seldom longer than five years. He looks at prices (or total return) once more after a second interval. Then he compares what has happened during the first interval to what happens during the second.

There are three relevant times: the start, the middle time and the end.

For purposes of discussion, let us examine what happens when prices are completely random (after subtracting out the average). Prices at each of the three times can be positive or negative (relative to the average). The typical price is zero (relative to the average).

Let us now introduce a threshold at the middle time and use it for selecting stocks (or mutual funds). We will set the threshold high. Only a few stocks (or mutual funds) will exceed the threshold.

Consider how these stocks (or mutual funds) behave. Typically, they start close to zero, rise high enough to pass the threshold. Typically, they should fall back to a value close to zero. In a few instances, they would have started or ended up significantly higher or lower than zero. But typically, they would have been close to zero. We should see a sharp price increase in the first interval followed by a comparable price decrease in the second.

When stock (or mutual fund) prices are truly random, selecting the best performers during the first interval is the same as selecting which stocks (or mutual funds) are most likely to do the worst in the second interval.

If there were no randomness whatsoever, selecting the best performers during the first interval should be the same as selecting the best performers during the second interval.

What we see is critically dependent upon the degree of randomness when compared to the level of skill (i.e., the noise level as compared to a signal).

What Does This Mean?

In the stock market, the single-year randomness of the S&P500 index is plus and minus 10% (i.e., plus and minus one standard deviation). A very high level of skill corresponds to a 2% improvement in the long-term annualized total return....

The randomness in the market is much more than the level of skill that we should expect. You tell a lie if you use thresholds as if there were no randomness. When you take randomness into account, your baseline for comparison should be for the selected stocks or mutual funds to underperform during the second interval by as much as they outperformed during the first. Unless they give back everything, skill exists.

[Actual comparisons need to include confidence intervals and to report the power of any statistical test.]

What Should You Do?
…
If you are inclined to invest in mutual funds or to follow a newsletter, focus on the latest nine years. Use caution. [Some of that information can be misleading.]

Failure Mechanisms and Success Mechanisms
…
We have some early indications of what to expect during today’s secular bear market. The mantra for investing in all-stocks, all of the time, will no longer hold. Low cost actively managed mutual funds are likely to perform better than passively managed index funds. I expect some of our better newsletter writers to lose their advantage in this new environment. I expect the performance of some other newsletter writers to shine.

I expect the most important success mechanism to be varying stock allocations according to overall stock market valuations.
…
Additional Remarks

Value investors will recognize the threshold effect. Value investors buy when prices are below normal. They set negative thresholds (compared to average prices).
…
Readers may be surprised at my enthusiasm for an advantage of 1% or 2% (annualized, long-term). Compounded over a long period of time, a 1% or 2% advantage builds up to a large amount. A 10% return compounded over 30 years multiplies an initial investment by 17.4 times. An 11% return compounds to 22.3 times the initial investment. A 12% return compounds to 30.0 times the initial investment. Over 40 years, the multiples are 45.3, 65.0 and 93.1, respectively.

Have fun.

John Walter Russell
June 29, 2005.