Nate Silver, David Axelrod agree: Yes, there is a crisis in polling

Earlier in the day, I pointed out the string of bad polling over the last six months or more that has managed to miss right-leaning victories in Western elections. Others have begun to notice, too, especially in the UK after David Cameron’s resounding victory, preceded by predictions of doom from the media. For instance, the British Polling Council wants to conduct an investigation into how the UK pollsters got the call so badly wrong:

David Axelrod, who worked on behalf of Labour (while Jim Messina worked for the Tories), also thinks that UK pollsters have some ‘splaining to do:

It’s not just a problem in the UK, though. US pollsters and media organizations have blown calls here too, and poll aggregator Nate Silver says that it’s time to “worry about the state of the polling industry”:

[I]n the U.S., some pollsters have been caught withholding results when they differ from other surveys, “herding” toward a false consensus about a race instead of behaving independently. There may be more difficult times ahead for the polling industry.

Later in the evening, Silver added:

Our pre-election forecast, put together in conjunction with, did project a narrow Conservative plurality. And it was about in line with or slightly more bullish on the Conservatives than other forecasts and betting markets.

However, that’s not much of an excuse. The forecast assigned too little of a chance to an outcome like this one, especially given that there have been significant polling errors in the U.K. before. It’s a good lesson as we begin to plan our coverage for the 2016 U.S. election.

Politico’s Dylan Byers wondered whether Silver had talked himself out of his niche:

This is quite a notable statement. The former New York Times statistician gained national fame for correctly anticipating the outcome of the 2008, 2010 and 2012 U.S. elections. He did this largely by understanding how to read the polls, and by knowing which polls were worth reading. …

If Silver is declaring that the world has a polling problem, and that there may be more difficult times ahead for the polling industry, what is Silver’s added value in an election cycle? His ability to forecast elections is largely dependent on the accuracy of polling. Without that, what is his raison d’etre — other than to point out how bad polling caused him to make inaccurate forecasts?

It sounds more like Silver wants to preserve his niche. Bad polling and “herding” make his job much more difficult, especially the latter. Speaking out for reform may at first make it difficult to explain his analyses and projections without demonstrating some significant nod to the uncertainty in the polling industry, but the polls for 2016 now are nearly meaningless anyway, even if they’re hyper-accurate. The polling industry won’t get any better if those who rely on it just keep their mouths shut about its problems, as the string of examples Silver uses demonstrates.

Consumers of news and especially the analysts have to challenge pollsters to improve their batting average, and sooner rather than later. Otherwise, Silver and others like him will have nothing to add to Election Nights except to continue to recap the industry’s failures — as necessary as that may be.