I’ve been waiting for this for a few days. Nate Cohn is the NY Times’ polling expert, sort of their alternative to Nate Silver. Today Cohn has published his first take of what went wrong with polling in 2020.

The first big point worth highlighting is that Cohn believes there definitely was a big problem this year, i.e. the polls were pretty far off nationally and in several swing states. In fact, he suggests it may have been worse than 2016:

It’s not too early to say that the polls’ systematic understatement of President Trump’s support was very similar to the polling misfire of four years ago, and might have exceeded it…

The national polls were even worse than they were four years ago, when the industry’s most highly respected and rigorous survey houses generally found Hillary Clinton leading by four points or less — close to her 2.1-point popular-vote victory. This year, Mr. Biden is on track to win the national vote by around five percentage points; no major national live-interview telephone survey showed him leading by less than eight percentage points over the final month of the race.

But the big question is why? Why were polls as bad or worse this time out even after pollsters made efforts to correct for the mistakes made in 2016. In answer to this question, Cohn offers five possible explanations for what went wrong. I’m going to take these out of order from the way Cohn presents them:

A version of the shy Trump voter theory

The idea here isn’t that Trump voters are lying to pollsters but that maybe pollsters just aren’t reaching Trump’s voters:

“We now have to take seriously some version of the Shy Trump hypothesis,” said Patrick Ruffini, a Republican pollster for Echelon Insights. It would be a “problem of the polls simply not reaching large elements of the Trump coalition, which is causing them to underestimate Republicans across the board when he’s on the ballot.”

The Pandemic made Democrats more responsive to pollsters

Cohn presents this point later but I think it’s sort of related to the point above. Maybe Trump voters were harder for pollsters to reach and meanwhile, Democrats started responding much more often once the pandemic set in:

“The basic story is that after lockdown, Democrats just started taking surveys, because they were locked at home and didn’t have anything else to do,” said David Shor, a Democratic pollster who worked for the Obama campaign in 2012. “Nearly all of the national polling error can be explained by the post-Covid jump in response rates among Dems,” he said.

The resistance also made Dems more likely to respond to pollsters

Another related point:

Are the “MSNBC moms” now excited to take a poll while they put Rachel Maddow on mute in the background? Like most of the other theories presented here, there’s no hard evidence for it — but it does fit with some well-established facts about propensity to respond to surveys.

This jibes with my own idea about how the polls shaped the election.

Turnout made polls of likely voters misleading

The idea here is that as the number of voters increases, polls of likely voters become less accurate and polls of registered voters become more accurate. In this case, polls of registered voters showed a tighter race than the LV polls we usually pay attention to.

The Hispanic vote

A point many have remarked upon which seems to have had an impact in Florida and in Texas especially. Trump just did better with minority voters than anyone expected. There were some late surveys that suggested this was happening and that made Democrats very nervous but pollsters seem to have missed this possibility for most of the race.

To wrap this up I just want to return to my contention that polling wasn’t merely bad this year it was bad in ways that shaped the race it was supposedly reflecting. I think you can at least argue that the first three points above boil down to the idea that Democrats saw pollsters as friends while Republicans saw them as enemies. Dems were eager to talk and Republicans were the opposite. And instead of correcting for this, the pollsters and the media kept amplifying it. Democrats were predicting a blue wave and in fact what we had was a razor thin race for President, Republicans holding the Senate (at least probably) and gaining as many as 10 seats in the House.

Yesterday Allahpundit argued the opposite point, i.e. polls were as likely to make Democrats complacent as they were to make Republicans discouraged. He pointed to this Huff Post piece which suggested Democrats may have lost the Senate in part because the GOP and independents at some point stopped being willing to cross party lines.

GOP internal polling showed that as voters across the board became more aware Biden was likely to win the presidency in the final weeks, it became harder for Democratic candidates to win over the Republicans and conservative-leaning independents they needed to win. The chance to compete slipped away from Democrats in Kansas and Montana. Greenfield’s advantage over Ernst in Iowa ― one poll showed the Democrat winning 10% of Trump voters ― evaporated.

Maybe it’s true that the GOP held the Senate, at least in part, because GOP and independent voters became less persuadable. But if so, the paragraph suggests the turning point was the assumption that Trump was going to lose. Where did this come from? From the polls of course.

I think this suggests that as Republicans became convinced Trump was going to lose (polls showed Biden winning nationally by +10) they decided to at least hold the Senate.

It’s hard to know what would have happened if polls hadn’t been so wrong. Maybe the GOP wouldn’t have held the Senate. Then again, maybe they wouldn’t have become discouraged about Trump and they’d have held the White House.

Elections happen at the margins and in this case several states were decided by a few thousand votes. Ultimately it’s a counterfactual and we can’t ever really know how it might have gone. But I think there’s at least some evidence the GOP was beaten down a bit by the polls and not much indication Democrats were complacent.