Which polls are, or aren’t, legitimate?
posted at 6:45 pm on September 26, 2012 by Allahpundit
I want to hear from commenters on this, as I think all bloggers are dealing with some variation of this problem right now. Standard practice on the site is for Ed and I to post any poll that we think you’ll find interesting, whether the numbers are good or bad; normally the readers are fine with that, if only because they can use the thread to goof on me for being a dirty, dirty eeyore. But for two months every four years, the calculus changes for some and they start screeching that posting bad numbers is an act of treason that might actually damage the GOP nominee’s chances. And in fairness to those readers, there’s a wisp of truth in that, sort of. As pollster John McLaughlin said to Jim Geraghty:
What Obama and his allies are doing now: “The Democrats want to convince [these anti-Obama voters] falsely that Romney will lose to discourage them from voting. So they lobby the pollsters to weight their surveys to emulate the 2008 Democrat-heavy models. They are lobbying them now to affect early voting. IVR [Interactive Voice Response] polls are heavily weighted. You can weight to whatever result you want. Some polls have included sizable segments of voters who say they are ‘not enthusiastic’ to vote or non-voters to dilute Republicans. Major pollsters have samples with Republican affiliation in the 20 to 30 percent range, at such low levels not seen since the 1960s in states like Virginia, Florida, North Carolina and which then place Obama ahead. The intended effect is to suppress Republican turnout through media polling bias. We’ll see a lot more of this.
The “anti-Obama voters” whom McLaughlin has in mind are swing-state undecideds who either voted for Obama in 2008 or stayed home and are now persuadable by Romney due to their disgruntlement over Hopenchange. They’re low-motivated fence-sitters. People who read partisan blogs every day are not. My guess is that our readership consists of two groups: 99 percent of you would walk barefoot through a snowstorm to get to your polling place to vote for Romney even if I was following you in an Eeyore costume, rattling chains and moaning, “Dooooon’t vooooote.” (I won’t actually do that, except maybe to Ed.) The other one percent are media types and/or liberals who are curious about what righty bloggers are saying on a particular issue. Neither of those groups will be discouraged by poll news, whether good or bad for their guy. Nor should they be: In case there’s any ambiguity as to the point of posting these polls, needless to say it’s not to discourage anyone from voting for Romney. You must vote, and the worse the numbers are, the more determined you should be to get out there because the deficit will have to be made up in higher turnout. Ed and I have spent four years explaining why another four years of Hopenchange dreck would be terrible; why you’d suddenly lose your determination to vote O out now because of bad numbers from the NYT or wherever is utterly beyond me.
The point of posting polls is to track trends in the race and try to get a rough sense of which states will ultimately decide the election, which strategies are working or aren’t, whether one side or the other has momentum, etc. Sometimes, like today, you get some highly dubious samples and you toss them out. Sometimes you don’t. My question is, if for some reason you’re not convinced that partisan blog readerships are essentially immune from being discouraged by polls, what should the rule be on filtering them? There seem to be three schools:
1. The “give us everything” crowd. These are the people who want the good and the bad. They’ll decide for themselves whether a poll is credible or not, but they want the data so that they can make a judgment.
2. The “give us bad news too but make sure you debunk it” crowd. They’ll accept discouraging numbers if a case can be made against the partisan split in the pollster’s sample to debunk it. Ed and I oblige on that whenever we can, but I’m not sure what to do with a poll like, say, today’s Gallup tracker, which has Obama suddenly out to a 50/44 lead among registered voters. Five days ago we were high-fiving over Gallup when they had Romney tied. Is the poll suddenly less credible now than it was then? Rasmussen seems to be the gold standard in credibility on the right, but what should we do if Romney’s numbers tick down there too? And what are we to do with the fact that Romney’s own pollster recently told Guy Benson that he’s expecting a national turnout advantage on election day of something like D+3? Should we be demanding a more even sample from pollsters than even Team Mitt is?
3. The “give us only good news” crowd. They think that posting bad numbers legitimizes those numbers and gives them wider reach, even if there’s an effort to debunk the sample. Essentially, they want a total blackout on downers until election day in the interest of leaving nothing to chance. Question: Does it mitigate the problem if we post a downer poll and post thoughtful analyses like Jay Cost’s and Brandon Gaylord’s that challenge the assumptions of the downer polls lately? If it doesn’t mitigate it, what are we to make of the fact that conservative warriors like Newt Gingrich, Erick Erickson, and Michael Walsh all seem to think that Romney’s campaign is underperforming and that the polls are a reflection of that? (Read Walsh’s conclusion, especially.) Is that higher or lower treason than posting a bad poll in the first place?
Those three schools broadly represent the spectrum of opinion on whether a partisan news site should be more newsy or more partisan. Group one wants to know what’s driving the news, even if it doesn’t trust the underlying data; group three wants victory above all else, even if that means suspending normal operations and ignoring bad news entirely. Group two wants a compromise. I prefer group one, especially since I think the fears of influencing the race by posting glum polls is baseless, but I have a lot of sympathy for group three even though they tend to be the nastiest with their criticism. We all want to win (even Eeyore!), and if you’re a sports fan, you know the special agony of being heavily invested in a contest whose outcome you’re helpless to influence. You’re not helpless in this one, of course — you can vote, and should — but the idea that merely mentioning bad news might sink Romney’s chances when we have fully seven weeks and four debates still to go is like sincerely believing that the Yankees lost because you forgot to wear your rally cap.
Like I say, I’m interested in reading your comments. I’ll leave you with this, from senior Romney advisor Ed Gillespie. Quote: “We have a no-whining rule in Boston about coverage in the media.” Click the image to watch.
Update (Ed): I was thinking about writing a post along these same lines after my analysis of the WaPo/ABC polls in Ohio and Florida. I’m in Camp One, at least theoretically, and I’d hope most of our readers would be as well. Otherwise, if we’re blowing sunshine up your skirts all year long and then it doesn’t end well, we’re all going to have that apocryphal Pauline Kael moment and wonder what happened. I have sympathy for Camps Two and Three, and in practice I’d say we’re probably Camp One Point Seven Five anyway.
Polling really isn’t that mysterious, as I tried to explain in this post yesterday, but there is one other thing to keep in mind: you can have a good, predictive poll sample and still get the wrong conclusion. Talking to 1,000 likely voters in Florida with a D+1 split is still just talking to 1,000 out of 8.2 million voters, roughly the number of ballots cast in the 2008 presidential election in that state. There is a ton of math and statistical analysis that can estimate how well a poll can predict an outcome, but it’s a snapshot in time, and it can still come up with an outlier even with the most predictive sample. That’s why it pays to watch all of the polls, even the ones with questionable samples, and keep an eye on intraseries trending at least as much as a single outcome. (RCP is a great resource for that purpose — as well as a great site overall.)
Just to reaffirm what AP wrote, we’re going to err on the side of more coverage rather than less. That means not all of the news will be rosy, but you won’t get blindsided by the ups and downs that way, either. One final thought to leave you with: we probably have a 42/42 split of die-hard partisans, with the middle 18% still open to be convinced. Wait until after the first debate for numbers to actually start firming up.