If you missed part 1, in which I highlighted some of the worst polling we saw in 2020, it’s here.

Now let’s move on to something which I don’t think is terribly controversial at this point. Having done this 2 elections in a row, the media’s credibility is pretty well shot. You don’t have to take my word for it. Plenty of observers on the left are saying it right now. Writing at the Atlantic, David Graham has a piece whose headline calls it a catastrophe for American democracy:

This is a disaster for the polling industry and for media outlets and analysts that package and interpret the polls for public consumption, such as FiveThirtyEight, The New York Times’ Upshot, and The Economist’s election unit. They now face serious existential questions. But the greatest problem posed by the polling crisis is not in the presidential election, where the snapshots provided by polling are ultimately measured against an actual tally of votes: As the political cliché goes, the only poll that matters is on Election Day. The real catastrophe is that the failure of the polls leaves Americans with no reliable way to understand what we as a people think outside of elections—which in turn threatens our ability to make choices, or to cohere as a nation.

In short, there’s no reason anyone should trust the polls or the media that relies on them at this point. NPR’s media analyst David Folkenflik reached a similar conclusion:

Hard not to conclude that it’s another black eye for polling and for the news organizations that rely on them. And why do I say that? Well, most of these polls, you read them in newspapers, you see them on TV, you might hear them on our network. They have media branding on them, even those done by colleges or private firms. The polls and the medias are intertwined on this stuff. So when you see a lot of wrong results, that reflects on us, too, regardless of whether Biden or Trump ultimately wins. And it looks like Biden – things are going in his direction. That huge blue wave, I don’t think we’re seeing it. The Dems were projected pick up seats in the House. They’re losing seats in the House…

People thought 2016 was a hot mess, and this may end up being worse for polling.

I like the bit about polls and the media being intertwined. Another way to put it is that pollsters produce a product and the media distributes that product. In fact, in many cases, the media is directly funding the product.

But it’s worth noting that early on election day David Folkenflik published a story at NPR headlined “Down In The Polls, Trump Seeks Familiar Embrace of Conservative Media.” Maybe David needs to do some reflection of his own on the ways in which polling and his own work are intertwined.

Finally, even at Jacobin, a socialist/communist online magazine, the conclusion is that “polling is dead.”

After the debacle of the 2016 election — when polls were off by an average of 5 points — polling institutions revised their methods. Some started dialing more cell phones. Others shifted focus toward swing states. Many began weighting responses by education. “Perhaps, after four years of hand-wringing, the polls will show they were all right after all,” FiveThirtyEight surmised after conducting its own survey of leading pollsters less than a month before the 2020 general election.

So far, the error rate looks the same. Bookmakers, looking at the polls, put their money on Democrats taking back the Senate. That now seems vanishingly unlikely. Polling outfits divined a blue wave that would expand the Democratic majority in the House. The Democrats lost seats.

If 2016 didn’t prove it, 2020 certainly did: the polling industry is in crisis.

This will make the media’s pre-existing credibility problem worse

Let’s not dance around the elephant in the room here. Trust in the media is largely divided by political party. Republicans have always trusted the media less than Democrats (and for good reason) but GOP trust in the media took a steep dive in 2016. This polling is from Gallup (yes, I’m relying on polling to criticize polling, but what else can you do):

Note also that Democrats trust in the media seemed to take off shortly thereafter. Twenty years ago you had 5-15 point media trust gap. Since 2016 you have a 50 point trust gap. There is now a pro-media party (70+%) and an anti-media party (10-20%). And all of that was true before this election where polling was once again badly off both nationally, in critical swing states and in important Senate elections. Not all of those errors favored Democrats but most of them did. So there’s every reason to think trust in the media will decline further, especially on the right.

Why did this happen…again?

I’ve seen various attempts to explain how the media-polling industrial complex got this wrong for a second election in a row. One leading theory is that “shy” Trump voters are not sharing their real preferences with pollsters. Does that mean respondents are lying? Not necessarily. As Jacobin points out, people don’t have to lie when they can simply chose not to answer the phone. In fact, that’s increasingly what happens:

In 2019, Pew Research reported a continuation of the long drop in response rates to a new all-time low: six responses in every one hundred contacts. Commercial pollsters are thought to receive even lower response rates, but they decline to publish that figure. The sampling pool for opinion polling techniques has become so small that it fundamentally skews the data they return. Surveyors know this well and are doing what they can to compensate, but the truth is that random selection polling methods have been obsolescing for decades.

Some more transparency about the actual process might help. Is it really true that pollsters are calling 20 people to get one response? And if so, doesn’t that suggest a kind of self-selection among respondents (most people can see who is calling) that renders the polls suspect.

A Washington Post piece on polling suggests that inconsistent response rates force pollsters to make a lot of guesses about the shape of the electorate. And those guesses may be as significant as the data they actually collect: