ObamaCare: Fuzzy polling and fuzzy pollsters

posted at 1:02 am on December 1, 2009 by Karl

Cliff Young and Aaron Amic of the Ipsos McClatchy Poll have a guest post at Pollster.com on what the polling on ObamaCare — and its seeming contradictions — might teach us about polling on non-electoral policy issues. Some of their analysis is dead on and some of it dead wrong, but all of it is interesting.

The focus of the piece is the debate among pollsters about differences between generic healthcare questions (generally showing plurality disapproval) and more specific ones referring to the “public option” (generally showing approval). Young and Amic first observe that, unlike election polling, “issues like healthcare reform are quite fuzzy as no bill typically exists at the beginning of the process. This makes the construction of a single question impossible if not simply disingenuous.” For some reason, pollsters were not advancing this argument at the beginning of the debate, when there was no actual legislation, and ObamaCare was popular. The further along we get in the legislative process we get, the less support there is for ObamaCare.

Young and Amic attempt to show the problem of their poll’s generic favor/oppose question:

[A] simple follow up question shows that about a quarter (25%) of those that oppose the reform bills actually think the proposals “do not go far enough” (see table 2 above)! This same 25% actually is much more likely to be Democrat and more likely to support the public option. People, once again, read into the question what they want.

Let’s take a look at that table 2:

As we can see, table 2 does not give us a partisan breakdown for that 25%. Perhaps it is in their crosstabs. It also shows that 63% of Democrats opposing ObamaCare do so because they think it goes too far. And what are we to make of the Republicans who do not think the bill goes far enough? Perhaps they think the bills under discussion should address tort reform, removing barriers to interstate competition, boost health savings accounts, etc. And we have no way of knowing what the Independents really think, either. Indeed, at least one recent poll showed near-majority support for tort reform among Democrats. It seems like Young and Amic are the ones reading things into their data, equating reform with particular Democratic proposals.

At the risk of reading too much into their analysis, I think Young and Amic also mean to imply that the generic question is misleading because those Democrats may ultimately support ObamaCare as better than nothing. There is no way of knowing how that number actually breaks down (though we know that it one lefty Senator decides he or she doesn’t like the final Senate bill, it could easily die). We do know that Gallup asks a different generic question — whether people would advise their member of Congress to vote for or against a healthcare bill. Last week’s Gallup poll still put ObamaCare under water, with 49% of against it — a result consistent with the other polls that Young and Amic would like us to doubt (and with the disapproval level currently shown in Pollster.com’s own polling average). Recent polling by PPP also suggests that Democrats are hurt on the generic ballot if a bill is passed with a “public option.”

Speaking of which, Young and Amic argue that questions which refer to the “public option” and other specific policy measures introduce greater certainty into the ballot question. First, their own polling shows the “public option” with 52% support — a fair amount lower than other polling. Second, Young and Amic suddenly drop the concept of testing that support with a follow-up question. As it turns out, the popularity of the “public option” tends to plummet when follow-ups are asked. Indeed, the same is true of ObamaCare generally, if you ask people to pay the price of a cheese pizza per week to finance it.

I do not mean to completely snark on Young and Amic. They usefully remind us that even basic polling questions may be open to interpretation, and that follow-up questions can reveal those subtleties. I just wish they had been more consistent in applying the follow-up principle.

Blowback