THE SIGNAL AND THE NOISE
NO NOISE WITH THIS BUZZWORTHY BOOK
Nate Silver, the creator of the New York Times political blog FiveThirtyEight.com, is probably America’s most well-known statistician, especially after his assertion that a 300-plus electoral college vote victory for Barack Obama was highly probable proved correct. In The Signal and the Noise, Silver explores a wide variety of areas — including electoral politics, sports, weather, terrorism, war and gambling — in which prediction plays a major role. He examines some of the major failures, from Pearl Harbor, 9/11 and the 2008 financial crisis to a whole series of failed earthquake forecasts, to try to understand what went wrong and what can be learned for the future. The problem is not the lack of data — today there is an almost endless supply of information on any topic or question. Instead, the problem is learning how to interpret the information, paying attention to the right information and recognizing the rest as just distraction — in other words, recognizing the difference between a predictive signal and the distracting noise.
Colossal Misinterpretation
Silver begins by examining one of the most colossal errors in prediction in recent times: the world financial crisis that no one predicted. As Silver explains, the signals were present, but were dismissed. As analysts and decision-makers watched continuously rising house prices, they failed to see the potential for a bubble; as they watched the explosion of mortgage-backed securities, they failed to understand the risk of those securities; as they started to consider the potential of a housing crisis, they failed to see that such a crisis could trigger a global financial crisis; and as they dealt with the financial crisis, they failed to predict the long-term impact.
Be Foxy
For Silver, the best forecasters are going to be “foxes,” rather than “hedgehogs.” The metaphor comes from Isaiah Berlin’s famous essay, “The Hedgehog and the Fox.” Hedgehogs believe in big governing ideas from which everything flows, Silver writes. Foxes, on the other hand, believe in little ideas and multiple approaches. Hedgehogs are specialized, stubborn and confident. Foxes are multidisciplinary, self-critical and cautious. The principles of Silver’s FiveThirtyEight blog reflect his “foxy” approach to forecasting. Instead of confidently making one specific forecast and sticking to it, especially if it’s different from what everyone else is saying, Silver offers a range of probable outcomes and is willing to change the predictions in light of new information.
The Human Factor
Despite his mastery of data, Silver notes that the human factor is equally important in making the right predictions. “What is it, exactly, that humans can do better than computers that can crunch numbers at 77 teraFLOPS?” Silver asks, as he considers two meteorologists staring at every screen on the forecasting floor of the National Weather Service headquarters. The answer: “They can see.”
Of course, they can also misinterpret what they see. But Silver is hopeful for the potential of predictions as long as those making the predictions, he writes, adhere to the principles of an 18th century English minister called Thomas Bayes. In a chapter titled “Less and Less and Less Wrong,” Silver describes the probabilistic theory of Bayes, in which someone makes a prediction, looks at the new evidence, confirms or changes the prediction, looks at the new evidence, confirms or changes the prediction, each time getting closer to making the correct prediction. As Silver explains, Bayes taught us that we learn about the universe “through approximation, getting closer and closer to the truth as we gather more evidence.” Bayes theorem implies that “we must become more comfortable with probability and uncertainty,” he writes. “We must think more carefully to the assumptions and beliefs that we bring to a problem.” As Silver demonstrates in detail in such diverse areas as sports, global warming, and terrorism, the more predictions are based on a Bayesian approach, the more likely they are to come true.
The Signal and the Noise is a sprawling, dense book that is as filled with stories and personalities as it is with data. This book will fascinate those who care about what is happening in the world — or what may happen.
Sign up for Executive Book Alert today and receive a review of a top business book every week.
