Predictions are commonplace. From sports to weather, elections to climate change, prognostications are everywhere.
In some arenas, we want accuracy. When we check a weather forecast, we want it to hit the mark. If the local TV forecast consistently fails, then we’ll look for (and find) another forecast that works better. The same goes for financial predictions. The person with the best grasp of the future will do better in the markets.
Predictions about finance and weather have one thing in common: we all have a stake in getting the right answer.
But when it comes to sports and politics, we often care less about accuracy. We want someone to tell us what we want to hear. One person wants to hear a pundit say that the Packers will win; someone else yearns for good news about the Bears (good luck with that!). Republicans want to know how the GOP will succeed; Democrats listen to those who say that they’re going to be victorious.
Because we disagree, pundits can often be “successful” by not being accurate.
According to pundittracker.com, most political and sports predictions are only slightly more accurate than a coin flip. Take ESPN’s talking heads. I pulled together their “ESPN Experts” predictions so far this NFL season (week 13) and compared them to the general sports fan (ESPN’s Pick ‘Em), bettors (based on point spread predictions), and predictions that merely predicts that the home team will win.
Here’s the results:
- General sports fan 68%
- Point spread 66%
- Average ESPN Pundit 64%
- Picking the home team 58%
When we take into account random variation, bettors and pundits are statistically the same as someone who just picks the home team. The average sports fan is the only one who does (slightly) better than the simplistic home team strategy.
It turns out there’s a reason that pundits do poorly — they have an incentive to play to our biases. It turns out that people gravitate toward predictions that are given with gusto, no matter how accurate. Economists Jadrian Wooten and Ben Smith studied the popularity of sports pundits. Looking at twitter followers, they found that a sports pundit who is perfectly accurate has an increase in popularity of only 3 1/2 percent. A pundit who is always confident, however, will increase his popularity by 17 percent.
In other words, it’s five times more effective for a pundit to be confident than accurate.
Wooten and Smith conclude that this is why pundits are often so poor at predictions
When you watch a pundit on television, remember that their job is to maximize eyeballs, not accuracy. Their employer is in the business of selling advertising, which means the networks will choose those pundits who provide the most advertising revenue. That might not necessarily be the most accurate pundit. It could just be the one who is most outrageously overconfident.
This may be changing. Nate Silver’s FiveThirtyEight is one sign that there is a market for accuracy in predictions. Ezra Klein credits blogs run by political scientists (albeit those with a strong statistical bent) for helping to change how political journalism is done.
We can hope, but I’m not confident that this is what the future holds.
