In this post I want to explain why you can’t find a Black Swan in a heap of straw. But first I’d better explain what I’m talking about.
The idea of a Black Swan comes from the book The Black Swan by Nassim Nicholas Taleb. In The Black Swan, a Black Swan is something new that cannot be predicted from your experience. If you have only seen white swans in your life, a black swan is basically unimaginable until you go to the place where black swans live and see one, after which it is self-evident. A Black Swan (capitalized) is anything that makes this leap from unimaginable to self-evident. This happens all the time, more often than we care to admit, so why are we so often caught off guard?
I can’t answer that without explaining a little of our intuitive idea of probability. And the easiest way to do that is with a heap of straw.
Imagine a blower throwing straw toward a point on the ground. This creates a kind of pile that, in technical discussions, we call a heap. Most of the straw lands near the center, where the heap is highest, but the heap also spreads out and there is some straw that settles a good distance away from the center. If you bundle up all the straw and turn the blower on again, it will produce another heap of almost identical shape. Blow in the same amount of straw, and you can easily imagine that it is the same heap all over again. Do this a few more times, and you can predict where the straw is going to fall even before you turn the blower on.
The shape of the heap of straw is what in probability is known as a normal distribution. A normal distribution is found in nature wherever a process combines a large number of small movements that can go in any direction — just the way a single piece of straw is small and can point in any direction when it lands.
The normal distribution is also one of the simplest probability distributions, so it is the favorite of economists, statisticians, and scientists generally as they try to explain the tendencies of any group of phenomena that they have identified. In education, the normal distribution is usually called the bell curve, based on the idea that the shape it creates resembles that of a bell, and it is used by educators to justify the low achievement of large numbers of students. Someone, after all, has to be well below average — the bell curve says so. In economics, though, the normal distribution is widely used in attempts to explain practically anything economists can measure.
We look at unemployment rates, for example, and we say that they are sometimes low, sometimes high, but usually pretty close to average — the same way that the straw mostly falls close to the center of the pile.
This kind of thinking can quickly lead us astray, however, if we start to imagine that it is the nature of unemployment to line up according to the tendencies we have observed in the past. It may do so with remarkable regularity for a period of many years, only to change completely one day, so that suddenly, we are no longer able to explain it. That kind of event is the Black Swan that Taleb warns us about. The inherent difficulty in predicting the future based on your experience is that one day things will change, and you won’t be able to understand the nature of the change until after it has taken place.
If you cannot predict the future from your experience of a phenomenon, then it also follows that you cannot predict the future from data, because data is just an organized numerical way of keeping track of past experience. Any predictions based on data will be approximately right for some length of time and then will be wrong.
We are led astray because of the intellectual appeal of being able to say what is going on. It is not easy to admit that what we have learned with considerable effort over a period of time is no longer of value.
Imagine that you are very efficiently making bales of straw, one after another, and a black swan wanders in. Will your experience with the straw tell you where in the pile of straw you are likely to see the black swan? No, because the nature of the black swan is different from the nature of straw. It doesn’t follow the same rules or respond to the same forces that you expect from straw. There is no reason to imagine that the black swan will be in the pile of straw at all. And that is the nature of a Black Swan: its properties are learnable, but are not able to be learned in advance.
This, as has been repeated endlessly elsewhere, is what killed the derivatives markets. These markets were trading based on quantitative models that had been tested only over a period of five years, in extremely limited economic circumstances. Quadrillions of dollars, more money than actually exists, were put on the line based on these models. Any surprising change in the economic surroundings would be enough to bring this party to a crashing halt. The popular explanation is that the quantitative models could not adapt to falling real estate prices, an economic circumstance for which they had never been tested. Yet the derivatives market began to fall apart a year before real estate prices began to decline nationally in the United States, so the real estate prices may not have been the original cause of the failure. Regardless of the specifics, something changed in the outside world, and the quantitative models that Wall Street had come to utterly rely on blew up in short order.
What Taleb is saying, translated to the metaphor I have been employing here, is that we spend too much time studying the heap of straw when the black swan is bound to wander by and change everything before too long. Yet we cannot study the black swan until it arrives.
If we must study the normal distribution of things, we should at least be prepared to drop our “normal” expectations and take a closer look when something comes along that doesn’t fit the pattern. Yet those are the exact observations that scientists are most likely to disregard. They are treated as “outliers” and routinely excluded from statistical studies on the suspicion that they may be trying to deceive us.
I know about this because I have worked in dozens of these studies. There is no time to find out about the outliers, but the statisticians are convinced they will ruin the study, so they are simply edited out of the data, as if they never happened.
Taleb suggests the opposite: ignore observations that fit what we are expecting, and look instead at those that are surprising. Find out what makes them different. Here we may discover the next Black Swan.
What does this mean for our current economic situation? You can see how the experts who expected the current recession to follow the pattern of all recent recessions have been disappointed. Those who expect it to be a repeat of the Great Depression will also get the wrong idea. Just the fact that journalists are using the phrase, “the worst recession since the Great Depression,” tells you that this is something different, more than just a repeat.
The insistence on seeing the recession as a repeat of past recessions has led to costly policy decisions. Former Treasury Secretary Henry Paulson justified the whole Wall Street bailout to Congress based on the scenario that housing prices could begin to rise again by December. It was a laughably optimistic prediction, excused only by the possibility that Paulson had had formal training in economics and was trying to fit this recession into the textbook model of a normal recession. The prediction that the bottom is just around the corner continues to be repeated, in spite of having been wrong for two years now.
The mistake is trying to fit the recession into our comfortable, familiar pile of straw — the normal distribution we got by measuring and analyzing past recessions. Economists are, on the whole, the worst offenders because they know better than everyone else how a recession “should” behave. If we could persuade economists that the current event is not a “recession,” but instead an “unidentified financial disturbance,” they could throw their preconceptions out the window, and their forecasts would improve.
The Black Swan is getting so much attention partly because it is a bold, brilliantly conceived book with important new ideas, and partly because investors are trying to understand how an investment model can just break down one day with no warning and never work again. But it is not just investors who should be paying attention. Every mental model we use to do anything will eventually fail; everything we take for granted will someday change.
