Ed wrote about this topic last week. His jumping off point was a story by Nate Cohn at the NY Times in which Cohn argued that the Democratic strength we’re seeing in polling right now looks a lot like a repeat of 2020 polling (which was awful). Cohn also noted that we probably shouldn’t expect polling to look much different since pollsters hadn’t made any significant changes to how they approach their work despite getting it wrong in 2016 and 2020.

It raises the possibility that the apparent Democratic strength in Wisconsin and elsewhere is a mirage — an artifact of persistent and unaddressed biases in survey research.

If the polls are wrong yet again, it will not be hard to explain. Most pollsters haven’t made significant methodological changes since the last election. The major polling community post-mortem declared that it was “impossible” to definitively ascertain what went wrong in the 2020 election.

But there is another major voice on polling who has argued the alternative. Nate Silver at FiveThirtyEight published an analysis earlier this month which came to a different conclusion:

My contention is that while the polls could have another bad year, it’s hard to know right now whether that bias will benefit Democrats or Republicans. People’s guesses about this are often wrong. In 2014, for example, there was a lot of discussion about whether the polls would have a pro-Republican bias, as they did in 2012. But they turned out to have a pro-Democratic bias instead.

Silver then offers seven separate reasons why he thinks evidence of consistent poll bias in favor of Republicans is not really there. You can read all of his arguments by following the link but I’m just going to zero in on what Silver had to say about 2018: [emphasis added]

As you can see in the table in the first point, polls did not have a systematic Democratic bias in 2018. That seems relevant, considering that was the most recent midterm.

Polls have also generally not had a Democratic bias in other elections in the Trump era when Trump himself was not on the ballot…

Elections have consequences, and they’re relatively infrequent events. So the second-guessing and recriminations tend to linger for a while.

But that doesn’t change the fact that people’s concerns about the polls stem mostly from a sample of exactly two elections, 2020 and 2016. You can point out that polls also had a Democratic bias in 2014. But, of course, they had a Republican bias in 2012, were largely unbiased in 2018, and have either tended to be unbiased or had a Republican bias in recent special elections.

Silver is not completely leaning on 2018 as proof the polls could go either way but it is an example he points to twice in his list of reasons. And that brings me to this piece from Vox today which notes the back and forth between Nate Cohn and Nate Silver and then sort of kicks Silver in the shins a bit by undercutting his argument about 2018:

I have a different interpretation of polls’ performance in 2018 than Silver, though. According to his numbers, polling averages underestimated Democrats by about 1 point on average in the House and in governor’s races, and there was no partisan bias in Senate polls on average that year.

But there’s a catch: The Senate map that year had an unusually large amount of contests in solidly blue states, none of which proved to be competitive. Democrats outperformed polls in nearly all of those contests.

Yet if we look at 2018’s actually competitive races — which that year were in purple and red states — most Democratic candidates underperformed their polls, and often by quite a lot.

The final margin was more than 3 points more unfavorable to the Democrat than FiveThirtyEight’s final polling averages in Florida, West Virginia, Michigan, Ohio, Tennessee, Missouri, and Indiana. There was only one competitive state — Nevada — in which the Democrat outperformed polls by more than 3 points.

So, for the purposes of someone trying to figure out which way the Senate would tip, the polls did functionally underestimate Republicans in 2018 too.

Here’s the graph he presents of how polls performed in the close contests in 2018:

If you take his point, that the battleground polls in 2018 tended to underestimate Republicans just as polls in 2016 and 2020 did then there does seem to be more of a pattern forming here. But Andrew Prokop, the author of the Vox piece, does note that 2022 doesn’t look a lot like 2018. So it’s ultimately anyone’s guess if the lessons of 2018 will apply this year.

Still, I don’t think anyone will be completely shocked if polls once again overestimate Democratic strength. We’ll know for sure in a few more weeks.