Friday, March 28, 2008

The Black Swan Giveth, and the Black Swan Taketh Away

On the 1000th day of its life, Bertrand Russell's turkey felt fat and happy. The next day, Thanksgiving, he was stuffed with bread and eaten to the great satisfaction of the Russell family. Russell's turkey met a black swan.[1]

A black swan was an idea put forward by the Enlightenment philosopher, David Hume, to represent the unexpected, the stuff you know you don't know or don’t know that you don't know. It was a play on the popular idea in the 17th century that the only color of swan found in nature was white. Hume argued that although no one had ever seen a black swan, their existence was not logically ruled out by their lack of being observed, at least by Europeans, as Europeans would soon find out. In fact, black swans do exist, but they weren't recorded in natural histories until Europeans (also) discovered Australia. As native Australians already knew, black swans are quite numerous indeed.[2]

Russell’s turkey met a black swan. We’ve all met black swans. Two good friends of mine met a familiar black swan this past Christmas, the one few of us ever anticipate. Both friends had enjoyed decades long careers at a single employer. Suddenly, they were let go, seemingly out of the blue. Sometimes black swans bear teeth.

But black swans sometimes bear gifts, too. Another friend of mine experienced a virtuous swan just after the new year. He was waiting for his privately held company to go through it’s quarterly valuation and release its updated stock price. If one followed the price history of the company’s stock and believed that past performance indicated future performance, one would reasonable expect an increase of $1 to $5. Imagine his surprise when he opened his email to learn that the company’s stock had jumped $17 per share, an increase of 107%! The value of the ownership he held in the company doubled in one day.

This beautiful bird is about to wreak havoc on those who fail to comprehend its predatory ambitions, or it may deliver a golden egg.

Now both sets of friends face more potential black swans. Will my former example set of friends continue to believe that employment always means stability, or will they take more proactive steps to manage their careers? Will my latter friend be honest enough to understand that his company’s stock price can go back down just as dramatically as it went up?

These surprises are the most general grist for consideration in Nassim Nicholas Taleb’s most recent book, The Black Swan: The Impact of the Highly Improbable,

a more thorough and extensive consideration than his previous book, Fooled by Randomness: The Hidden Role of Chance in Life and in the Markets.

Taleb, literary essayist, Dean’s Professor of Uncertainty at the University of Massachusetts, and mathematical options trader, lays out the foundation for his ideas from his experiences in the financial markets. These experiences lead him to discuss three interrelated themes that he ultimately broadens to an understanding of uncertainty in general.
  1. The Ludic Fallacy: The word "ludic" comes from the Latin ludus, for “games.” Most of us were taught to think about systems that involve uncertainty and chance using structured, discrete symmetries like those found in games of chance, such as craps, card games, etc. Unfortunately, these analogies don’t frequently hold up into real world situations; but because it’s comfortable and expected to do so, we persist in their use. Use of binomial distributions are often a result of ludic framing, as is its continuous cousin, the bell curve (or Normal Gaussian for those more trained in quantitative analysis). These distributions have appropriate applications, but they do not apply to all cases of uncertainty.
  2. Mediocristan/Extremistan: Most of us live in Mediocristan, a land not governed so much by mediocrity, but by a persistent belief in the applicability of a characteristic median to all types of uncertainty. When most of us think of uncertainty in some domain, we have been taught to think about that in terms of the bell curve. Taleb demonstrates that many types of uncertainty can be thought of in this way (Class I uncertainty), but not all of them. One characteristic of uncertainties described by a bell curve is that they do not scale, that is, the probability of occurrence of outcomes far from the mean (and median) fall exponentially fast. In this world, no one event outweighs the significance of all the other events. Many physical processes with natural physical constraints are accurately described by such distributions. Black swans can show up here, but they are extremely rare. Unfortunately, most black swans live in Extremistan, the land where uncertainty scales according to a power law (Class II uncertainty), where it is possible for a single event in a domain to outweigh the significance of all the others. Extremistan exists beyond the Platonic fold, where our typical representations of reality fail to apply. The processes that govern such distributions tend to be social, emergent, financial, maybe not entirely physical. This will have an important bearing on seeing the applicability to maneuver conflict.
  3. Confirmation Bias: Oftentimes we become inebriated with hope, that outcomes will go the way we wish and hope. To convince ourselves that such is the case, we develop narratives from cherry-picked data and information that confirm our bias. Using these narratives as a guide to decision making, we are disabused of our fallacious reasoning in sometimes spectacular ways. Yet we still fail to learn if we are still alive to face the next black swan. “Beware the scalable,” Taleb enjoins.
The scandalous malpractice, as Taleb shouts, is that the rules that apply to Mediocristan are too often misappropriated to understand and manage systems that don’t obey such laws, often at the expense of lives and immense fortunes. The most pointed cases involve applications of options and modern portfolio theory in which billions of dollars of investors’ fortunes are lost by the malpractice of Nobel “intellectuals” who should know better (anyone remember the tragedy of the Amaranth fund or the trading company Long-Term Capital Management?); the poignant disaster of the unsinkable Titanic; the current woes of Bear Stearns and the sub-prime lending industry; and, in Taleb’s case, the decade and a half long civil war in his centuries-long peaceful Lebanon, a war that he and all too many others sadly believed would end soon after it started.

How does understanding the black swan inform our understanding of maneuver conflict? Consider the martial arts version of the Ludic Fallacy offered by Mark Spitznagel.
Organized competitive fighting trains the athlete to focus on the game and, in order not to dissipate his concentration, to ignore the possibility of what is not specifically allowed by the rules, such as kicks to the groin, a surprise knife, et cetera. So those who win the gold medal might be precisely those who will be most vulnerable in real life. (Black Swan, pg. 127)
John Boyd leads us to understand that conflict is often a non-cooperative contest for limited resources by novelty generating agents. Novelty is the black swan of conflict. When we become convinced that our side will win on the basis of strength or numbers, when we believe that the other side will follow our rules of engagement, we will be exposed to cruel novelty. This is precisely what Chet Richards describes as a disease of orientation called fixation: “...attachments to appearances, conclusions, institutional positions, dogmas, ideologies — pretty much anything that keeps the people inside the organization from recognizing that the world is changing or being changed by competitors.”

How do we escape the tyranny of the black swan? We have to learn to do at least two things. First, we have to learn how to really learn, always looking for disconfirming evidence to the self-justifying narratives we generate from the first cousins of confirmation & my-side bias, availability bias, and anchoring that keep us from considering a wide range of possible outcomes, their appropriate degrees of likelihood, and their consequences. We have to learn that images in the mirror tell us scant little about the road ahead. To do this, next, we have to learn how to properly discern systems governed by the laws of Mediocristan from those governed by the laws of Extremistan, and act accordingly.

Nassim Nicholas Taleb delivers what may be the only book on epistemology that I would describe as both a blustery and a rollicking good read. If only all the other text books in philosophy of knowledge I read in school had been so fun. If only all the others had been so honest. In that sense, Taleb's book is its own black swan.

[1] The reader should almost immediately recognize that Bertrand Russell was English and did not observe Thanksgiving. In fact, as Russell himself tells this story, he uses a chicken as the example of the doomed bird. Taleb acknowledges this, but adapts the story to an American audience. [back]

[2] I am reminded of an event in my 9th grade year in which my algebra teacher convinced a sizable portion of our class that the state of Nevada did not exist, that it was a ruse invented by the US Air Force to deter investigation into super secret military programs. His “proof” was a simple question: “Have you ever seen a car tag from Nevada?” For kids in rural mid-Georgia, his scam was based on a rather safe bet that Nevadans rarely drove to our sleepy little town. [back]


Robert said...

This review was also published at Chet Richard's website.

Robert said...

Another good review of The Black Swan is posted here: REVIEW: Taleb's "Black Swan".

Although I got the feeling that deichmans seemed to think that I offered "glowing praise" of the book that likely bordered on sycophantism, I did post a response suggesting more of a balanced view of The Black Swan. I think deichmans' observations are excellent and worth considering.

Robert said...

Taleb provides a great interview here.