Superforecasting

It's been a while since I did a book review. I finished this one a few weeks back, but got distracted before writing about it. Not something I'd usually pick up but it was on sale and sounded interesting:

From one of the world's most highly regarded social scientists, a transformative book on the habits of mind that lead to the best predictions. 

Everyone would benefit from seeing further into the future, whether buying stocks, crafting policy, launching a new product, or simply planning the week's meals. Unfortunately, people tend to be terrible forecasters. As Wharton professor Philip Tetlock showed in a landmark 2005 study, even experts' predictions are only slightly better than chance. However, an important and underreported conclusion of that study was that some experts do have real foresight, and Tetlock has spent the past decade trying to figure out why. What makes some people so good? And can this talent be taught? 

In Superforecasting, Tetlock and coauthor Dan Gardner offer a masterwork on prediction, drawing on decades of research and the results of a massive, government-funded forecasting tournament. The Good Judgment Project involves tens of thousands of ordinary people - including a Brooklyn filmmaker, a retired pipe installer, and a former ballroom dancer - who set out to forecast global events. Some of the volunteers have turned out to be astonishingly good. They've beaten other benchmarks, competitors, and prediction markets. They've even beaten the collective judgment of intelligence analysts with access to classified information. They are "superforecasters". 

In this groundbreaking and accessible book, Tetlock and Gardner show us how we can learn from this elite group. Superforecasting offers the first demonstrably effective way to improve our ability to predict the future - whether in business, finance, politics, international affairs, or daily life - and is destined to become a modern classic. 

This article on the BBC reminded me about it. 

It interested me because I quite like statistics and I often wonder how so many people get to positions where they are influencing major decisions with what appears to be subjective opinion rather than tested prediction models or at least a relatively evidence-based, educated guess. 

This book also wondered this aloud.

Unfortunately, due to an Audible glitch, which I'll talk about later, I lost all my notes on this one, so it'll be a pretty short review. All I can say is that it was interesting. It went into quite a bit of detail about how we guestimate probability and touched on a few of the things covered in Eagleman's The Brain and You, such as there being two parts to your brain, the slower-thinking, reasoning part and the snap decision, quick-calculating part, and how we need to quiet the second in order to make better decisions, which isn't easy when you're put on the spot or in a stressful situation.

The bit about the 'wisdom of the crowd' was interesting, and how the greater the individual input, the more accurate the overall prediction tends to be. There were also some clever examples of how accurate someone can be when they break down a large problem - such as how many piano tuners there are in Chicago - into ever smaller parts: how many pianos, how many times they need tuning, how long tuning takes. By taking a very rough guess at potential figures for each unknown, the final figure proved to be not far off. 

I don't think I'd make a superforecaster, but I do enjoy thinking things through, and it was heartening that, although maths is an important component in superforecasting, you don't have to be a brilliant mathematician to be a good reasoner. I've always been fairly numerically challenged, but I like looking at trends in statistics. 

Anyway, it was nice to know that forecasting accuracy is something that people are pushing for. It does appear that news is more and more driven by personality and misinformation, and it would be wonderful if the people we turn to for insight actually had to show their track records on prediction. 

[UPDATE: Trevor Noah just nailed the above point about accuracy.]