by Patrick Callahan
As one of the founders of a company that helps others come up with predictions – I found the book Superforecasting particularly interesting (and necessary to read) for our company. Who is a Superforecaster? Are they the brainiacs that always outscore anyone in anything when it comes to math? Or are they the passionate Sunday morning news pundits that show they knew more through their passion as displayed on the talk shows?
At the end of the day – I learned that it is quite the opposite: the Superforecaster is humble, cautious, curious, questioning and an open-minded person. They are the retiree that is not the smartest – but the most patient; one that is willing to question themselves. The Superforecaster is the exact person you want to hang out with (or at CompassRed – one you want to work with!).
Five things I learned from Superforecasting that will stick with me:
- Man and Machine. Anyone can become a Superforecaster. It isn’t that there is an innate gift – it’s just that the Superforecaster is measured and she constantly checks her assumptions. And though computers are apt to be a big part of forecasting, the human element will become even more important because it is “reflective”. As Tetlock observes: “I think it’s going to get stranger and stranger” for people to listen to the advice of experts whose views are informed only by their subjective judgment”.
- Follow the Medical Industry. The medical industry is a perfect example of where there was no methodological testing for the longest time and was changed by the basics of the fundamentals of Superforecasting. Prior to testing and for hundreds of years, it was common practice for Doctors to “bleed out” people out to remove their diseases simply because that was thing to do. It was not until after WWII that we started to do randomized tests and questioning the practices of the past. We’ll start to do this in other institutions and in our everyday behavior in the future and we need to start questioning ourselves. Do we trust our own intuition? What do the numbers tell us?
- Approach Through Fermi. The Fermi estimation is a good way to approach an analysis: Break things down into smaller components and test those assumptions or forecasts individually. “Unpack the questions into components”. “Forecasts aren’t like lottery tickets that you buy and file away until the big draw. They are judgements that are based on available information and that should be updated in light of change information”.
- The Hedgehog Fox Dichotomy. Tetlock defines the hedgehog-fox dichotomy: The Fox knows many important things – but the Hedgehog knows one thing really, really well. In categorizing forecasters, Tetlock summarizes that “The [fox] group consisted of more pragmatic experts who drew on many analytical tools, with the choice of tool hinging on the particular problem they faced. These experts gathered as much information from as many sources as they could.” While the hedgehog approached forecasters “…sought to squeeze complex problems into the preferred cause-effect templates and treated what did not fit as irrelevant distractions. Allergic to wishy-washy answers, they kept pushing their analyses to the limit (and then some), using terms like “furthermore” and “moreover” while piling up reasons why they were right and others wrong. As a result, they were unusually confident and likelier to declare things “impossible” or “certain.”’
In the end, it was the foxes that were the better forecasters. They were more open-minded and less sure to be the ones that would be affected by an ideology.
- System 1 and System 2. Being aware of System 1 and System 2 allows you to question the things you see and logically draw up the things you don’t. System 1/System 2 is based on the Dual Process Theory: System 2 is our conscious theory and System 1 is the autopilot function. Superforecasters are able to not see one versus the other – but consider and use both approaches to thinking.
Thinking about how to think is a mind-bending experience – but one that should be done by everyone. Questioning the very foundations of what you believe to be true is one of the central themes of Superforecasting. Tetlock lays out a manifesto of why and how this should be done.