Friday, January 23, 2015

Forecasting Folly

"There are two kinds of forecasters: those who don't know, and those who don't know they don't know." - John Kenneth Galbraith

With this Galbraith quote in mind, consider what Professor Daniel Kahneman wrote in an article back in 2011.

In that article, Kahneman explains that he was responsible for evaluating candidates for officer training during his time in the military several decades ago. The methods he and others at the time used were apparently developed by the British Army during World War II. Part of his job was to, after careful observation of potential officers, offer what were thought to be useful predictions about how these candidates were likely to perform in the future. Seems straightforward enough: simply figure out who was clearly qualified and who was not via a sound methodology.

Since certain individuals appeared to have strong leadership skills while others plainly did not, Kahneman (and others) felt quite comfortable making definitive predictions.

Unfortunately, that confidence was unfounded:

"...despite our certainty about the potential of individual candidates, our forecasts were largely useless. The evidence was overwhelming."

In the same article Kahneman also added -- and this might at least partially help explain why prognosticators continue to confidently prognosticate despite the folly of it -- the following:

"The statistical evidence of our failure should have shaken our confidence in our judgments of particular candidates, but it did not. It should also have caused us to moderate our predictions, but it did not. We knew as a general fact that our predictions were little better than random guesses, but we continued to feel and act as if each particular prediction was valid. I was reminded of visual illusions, which remain compelling even when you know that what you see is false. I was so struck by the analogy that I coined a term for our experience: the illusion of validity.

I had discovered my first cognitive fallacy."

If it's difficult to predict how one individual is going to perform, then the inherent difficulty of predicting what will happen with the stock market or something as complex as the global economy shouldn't exactly be a surprise.

Forecasting is tough to do reliably well. This article by Barry Ritholtz puts its more bluntly:

Pro Forecasters Stink, You're Worse

That doesn't stop many from trying to predict what is mostly just not predictable. There is, and there will continue to be, no shortage of experts making forecasts about, among other things, the markets and the economy. Many of them are extremely smart, informed, well-intentioned, credible sounding, and a number even have some interesting things to say.

The problem is that those well-intentioned experts may not necessarily be producing something that's genuinely useful. There naturally will be exceptions but, especially as the forecasts become more macro-oriented, I think it increasingly makes sense to be skeptical. The world has always been an uncertain place and will continue to be that way. Being flexible and open-minded beats rigid certitude.

Expert forecasters will no doubt continue looking into their crystal ball and offer what at least sounds like compelling thoughts about the future.

The fact that they continue to do so with a high level of confidence just might be, at least in part, the "illusion of validity" at work.

Unfortunately, some of us will also likely pay way too much attention to it.

From a separate article written by Ritholtz late last year:

"Despite the abysmal track record of almost all forecasters, the news media still loves them. It has air time and pages to fill and seems little concerned about giving space to money-losing prognosticators.

As I first wrote a decade ago, to forecast is folly. Today, we have Google Search to help us prove it. Pundits may forget, but not the Internet."

At a minimum, it seems like not a bad idea at all to at least pause for a second or two and consider carefully whether someone's predictions deserves any more consideration than the outcome of a coin flip.

Adam

Related posts:
Henry Singleton: Why Flexibility Beats Long-Range Planning
Forecasters & Fortune Tellers
Charlie Munger: Snare and a Delusion
On Forecasting
James Grant on Economic Forecasting

This site does not provide investing recommendations as that comes down to individual circumstances. Instead, it is for generalized informational, educational, and entertainment purposes. Visitors should always do their own research and consult, as needed, with a financial adviser that's familiar with the individual circumstances before making any investment decisions. Bottom line: The opinions found here should never be considered specific individualized investment advice and never a recommendation to buy or sell anything.