Which polls are, or aren’t, legitimate?

posted at 6:45 pm on September 26, 2012 by Allahpundit

I want to hear from commenters on this, as I think all bloggers are dealing with some variation of this problem right now. Standard practice on the site is for Ed and I to post any poll that we think you’ll find interesting, whether the numbers are good or bad; normally the readers are fine with that, if only because they can use the thread to goof on me for being a dirty, dirty eeyore. But for two months every four years, the calculus changes for some and they start screeching that posting bad numbers is an act of treason that might actually damage the GOP nominee’s chances. And in fairness to those readers, there’s a wisp of truth in that, sort of. As pollster John McLaughlin said to Jim Geraghty:

What Obama and his allies are doing now: “The Democrats want to convince [these anti-Obama voters] falsely that Romney will lose to discourage them from voting. So they lobby the pollsters to weight their surveys to emulate the 2008 Democrat-heavy models. They are lobbying them now to affect early voting. IVR [Interactive Voice Response] polls are heavily weighted. You can weight to whatever result you want. Some polls have included sizable segments of voters who say they are ‘not enthusiastic’ to vote or non-voters to dilute Republicans. Major pollsters have samples with Republican affiliation in the 20 to 30 percent range, at such low levels not seen since the 1960s in states like Virginia, Florida, North Carolina and which then place Obama ahead. The intended effect is to suppress Republican turnout through media polling bias. We’ll see a lot more of this.

The “anti-Obama voters” whom McLaughlin has in mind are swing-state undecideds who either voted for Obama in 2008 or stayed home and are now persuadable by Romney due to their disgruntlement over Hopenchange. They’re low-motivated fence-sitters. People who read partisan blogs every day are not. My guess is that our readership consists of two groups: 99 percent of you would walk barefoot through a snowstorm to get to your polling place to vote for Romney even if I was following you in an Eeyore costume, rattling chains and moaning, “Dooooon’t vooooote.” (I won’t actually do that, except maybe to Ed.) The other one percent are media types and/or liberals who are curious about what righty bloggers are saying on a particular issue. Neither of those groups will be discouraged by poll news, whether good or bad for their guy. Nor should they be: In case there’s any ambiguity as to the point of posting these polls, needless to say it’s not to discourage anyone from voting for Romney. You must vote, and the worse the numbers are, the more determined you should be to get out there because the deficit will have to be made up in higher turnout. Ed and I have spent four years explaining why another four years of Hopenchange dreck would be terrible; why you’d suddenly lose your determination to vote O out now because of bad numbers from the NYT or wherever is utterly beyond me.

The point of posting polls is to track trends in the race and try to get a rough sense of which states will ultimately decide the election, which strategies are working or aren’t, whether one side or the other has momentum, etc. Sometimes, like today, you get some highly dubious samples and you toss them out. Sometimes you don’t. My question is, if for some reason you’re not convinced that partisan blog readerships are essentially immune from being discouraged by polls, what should the rule be on filtering them? There seem to be three schools:

1. The “give us everything” crowd. These are the people who want the good and the bad. They’ll decide for themselves whether a poll is credible or not, but they want the data so that they can make a judgment.

2. The “give us bad news too but make sure you debunk it” crowd. They’ll accept discouraging numbers if a case can be made against the partisan split in the pollster’s sample to debunk it. Ed and I oblige on that whenever we can, but I’m not sure what to do with a poll like, say, today’s Gallup tracker, which has Obama suddenly out to a 50/44 lead among registered voters. Five days ago we were high-fiving over Gallup when they had Romney tied. Is the poll suddenly less credible now than it was then? Rasmussen seems to be the gold standard in credibility on the right, but what should we do if Romney’s numbers tick down there too? And what are we to do with the fact that Romney’s own pollster recently told Guy Benson that he’s expecting a national turnout advantage on election day of something like D+3? Should we be demanding a more even sample from pollsters than even Team Mitt is?

3. The “give us only good news” crowd. They think that posting bad numbers legitimizes those numbers and gives them wider reach, even if there’s an effort to debunk the sample. Essentially, they want a total blackout on downers until election day in the interest of leaving nothing to chance. Question: Does it mitigate the problem if we post a downer poll and post thoughtful analyses like Jay Cost’s and Brandon Gaylord’s that challenge the assumptions of the downer polls lately? If it doesn’t mitigate it, what are we to make of the fact that conservative warriors like Newt Gingrich, Erick Erickson, and Michael Walsh all seem to think that Romney’s campaign is underperforming and that the polls are a reflection of that? (Read Walsh’s conclusion, especially.) Is that higher or lower treason than posting a bad poll in the first place?

Those three schools broadly represent the spectrum of opinion on whether a partisan news site should be more newsy or more partisan. Group one wants to know what’s driving the news, even if it doesn’t trust the underlying data; group three wants victory above all else, even if that means suspending normal operations and ignoring bad news entirely. Group two wants a compromise. I prefer group one, especially since I think the fears of influencing the race by posting glum polls is baseless, but I have a lot of sympathy for group three even though they tend to be the nastiest with their criticism. We all want to win (even Eeyore!), and if you’re a sports fan, you know the special agony of being heavily invested in a contest whose outcome you’re helpless to influence. You’re not helpless in this one, of course — you can vote, and should — but the idea that merely mentioning bad news might sink Romney’s chances when we have fully seven weeks and four debates still to go is like sincerely believing that the Yankees lost because you forgot to wear your rally cap.

Like I say, I’m interested in reading your comments. I’ll leave you with this, from senior Romney advisor Ed Gillespie. Quote: “We have a no-whining rule in Boston about coverage in the media.” Click the image to watch.

Update (Ed): I was thinking about writing a post along these same lines after my analysis of the WaPo/ABC polls in Ohio and Florida.  I’m in Camp One, at least theoretically, and I’d hope most of our readers would be as well.  Otherwise, if we’re blowing sunshine up your skirts all year long and then it doesn’t end well, we’re all going to have that apocryphal Pauline Kael moment and wonder what happened.  I have sympathy for Camps Two and Three, and in practice I’d say we’re probably Camp One Point Seven Five anyway.

Polling really isn’t that mysterious, as I tried to explain in this post yesterday, but there is one other thing to keep in mind: you can have a good, predictive poll sample and still get the wrong conclusion.  Talking to 1,000 likely voters in Florida with a D+1 split is still just talking to 1,000 out of 8.2 million voters, roughly the number of ballots cast in the 2008 presidential election in that state.  There is a ton of math and statistical analysis that can estimate how well a poll can predict an outcome, but it’s a snapshot in time, and it can still come up with an outlier even with the most predictive sample.  That’s why it pays to watch all of the polls, even the ones with questionable samples, and keep an eye on intraseries trending at least as much as a single outcome. (RCP is a great resource for that purpose — as well as a great site overall.)

Just to reaffirm what AP wrote, we’re going to err on the side of more coverage rather than less.  That means not all of the news will be rosy, but you won’t get blindsided by the ups and downs that way, either.  One final thought to leave you with: we probably have a 42/42 split of die-hard partisans, with the middle 18% still open to be convinced.  Wait until after the first debate for numbers to actually start firming up.


Related Posts:

Breaking on Hot Air

Blowback

Note from Hot Air management: This section is for comments from Hot Air's community of registered readers. Please don't assume that Hot Air management agrees with or otherwise endorses any particular comment just because we let it stand. A reminder: Anyone who fails to comply with our terms of use may lose their posting privilege.

Trackbacks/Pings

Trackback URL

Comments

Comment pages: 1 3 4 5

I’m more in Camp 2: show us ALL the polls AND the D/R/I split and debunk them when they can be debunked.

If Obama leads by 9% in Florida with a D+9 sample where Indies are evenly split, that shows that Romney can win if he either (1) wins Indies by a bigger margin and/or (2) turns out the Republican “base” better than Obama turns out Democrats.

Voter registration in Florida is 40%D / 36% R / 24% “other” in terms of percentages, and 4.63 million Democrats, 4.17 million Republicans, and 2.78 million “others” in terms of voters.

Obama won Florida in 2008 by 4.28 million to 4.05 million votes, and neither candidate even got all the registered voters of his own party (Obama got 92.5% of registered Democrats, McCain got 96.9% of registered Republicans). Even in a GREAT year for Democrats, McCain turned out the Republican “base” better than Obama turned out the Democrats!

Many polls (including Gallup) have shown that Republicans are more enthusiastic about voting this year than Democrats. Even if Romney and Obama split Independent voters evenly, Romney can win by turning out a higher percentage of the partisan “base” than Obama. If we set a target of say, 3.7 million Republicans vs. 3.6 million Democrats in Florida, that represents 88.7% of registered Republicans vs. 77.8% of registered Democrats, or a turnout gap of about 11%. This can be done…especially if lukewarm Democrats are looking at skewed polls and saying “Obama has this in the bag, why bother voting?”

Steve Z on September 27, 2012 at 10:17 AM

How about 4) Give me accurate figures, whether they be leaning R or D. I’m not into avoiding reality. But I’m not into being lied to by pollsters either. I would personally not bother posting links to polls that oversample either party so overwhelmingly that you get a figure like Obama +10 in Ohio. That isn’t a poll. That’s an insult to the average person’s intelligence.

moonbat monitor on September 27, 2012 at 10:56 AM

All of these polls are designed to do the same thing that the never-ending stream of “Romney gaffes!” reporting is designed to do: 1) to mute Romney appeal to independents, 2) to depress Republican enthusiasm and 3) to create a bandwagon effect – where the casual observer unconsciously wants to be a part of the “winning” team – that benefits Obama with fence-sitters.

Meanwhile…back in the real world…everywhere that Romney or Ryan speaks, they’re setting up speakers outside for all the people who can’t get in. And everywhere that Obama or Biden speaks, they’re busing in people from out of state to get the place half full.

Here in the South, we have a saying that describes the veracity of these “polls” – “That dog won’t hunt.”

Again…

From Rasmussen:

Aug 2004
Party ID: 35%R – 38%D – 27%I
Voter Enthusiasm: 51%R – 68%D
Election result: Bush wins narrowly

Aug 2006
Party ID: 32%R – 37%D – 31%I
Voter Enthusiasm: 40%R – 65%D
Election result: Dems wins solidly

Aug 2008
Party ID: 33%R – 39%D – 28%I
Voter Enthusiasm: 35%R – 61%D
Election result: Obama wins solidly

Aug 2010
Party ID: 34%R – 35%D – 31%I
Voter Enthusiasm: 48%R – 50%D
Election result: Republicans win blowout election unlike anything since the Great Depression.

And the party ID as of last month?

Aug 2012
Party ID: 38%R – 33%D – 29%I

For Democrats to win elections, they need three things: 1) to win the independent vote, 2) to have at least a 3-point lead in party ID and 3) for Republicans to stay home. On the other hand, Republicans win elections even when they are TRAILING in party ID, as long as we are close in voter enthusiasm and win Independents.

What do the voter enthusiasm numbers show? From Gallup last week, Democrats 68% to Republican 62%. But a couple of points. 1) Those are numbers for registered voters, not likely voters. 2) The Democrats are enjoying a post-convention bounce, which is understandable since the Dem convention is really the first Democrat party “event” of the election cycle. 3) Those numbers came out less than week after the Sept 11th Libyan embassy attack – with more and more info coming out everyday about Obama’s utter incompetence in that regard, expect that enthusiasm to be blunted. And finally, 4) even at the boiling point of Obamamania in 2008, Dem voter enthusiasm topped out at 61% – anyone who believes that Dems are going to be more enthusiastic to vote for this clown this time around is a fool, and they are also left to explain that if the Dems are sooo enthusiastic about voting for Obama again, why he is talking to half-empty arenas that only hold 5,000 people anyway? The Dems enthusiasm numbers will wane, so that going into election day, we’ll see voter enthusiasm numbers that are equal or with a slight Republican advantage.

Finally, wWhat do the Independent voter numbers show? Even in these corrupt polls where Dems are being egregiously oversampled, Romney has a DOUBLE DIGIT lead amongst Independents.

Nuff’ said.

Bottom line: anyone who thinks that Republicans aren’t going to turn out in numbers that dwarf 2008 is an abject moron. As such, any poll with a Dem +2 sample or greater is absolute bunk. And since the majority of these Democrat media polls are consistently oversampling Dems by +7 or greater, they are absolute bunk.

rvastar on September 27, 2012 at 11:29 AM

For those who keep spewing the “Romney campaign isn’t leaking their internal polling numbers because they’re losing”…just shut up. Seriously. Shut. Up.

Let’s say the Romney campaign leaks an internal poll from FL showing him winning. What happens within 24 hours? The Democrat propaganda machine would immediately focus its every effort on FL like the Death Star. And when I say their propaganda machine, I mean the US news media. Within 24 hours of Romney releasing that poll, every story on NBC, CBS, ABC, CNN and MSNBC would be about Romney/Ryan wanting to make health care for seniors illegal and their secret plans to begin euthanizing people at age 55.

IOW: giving out information is giving out ammunition for your enemies to use against you. It would be incredibly stupid to do so; hence, it’s not being done.

rvastar on September 27, 2012 at 11:47 AM

I do not have problem with any poll as long as I can get the internals on party affiliation or vote record in the last election. Missing that data the poll is worthless as I cannot tell if it has merit or not.

JIMV on September 27, 2012 at 12:02 PM

Jim Geraghty calls the Democrats out on their “hypothetical electorate that looks different from everything we’ve seen before…”

John McLaughlin had an interesting point:

“The Democrats want to convince [these anti-Obama voters] falsely that Romney will lose to discourage them from voting. So they lobby the pollsters to weight their surveys to emulate the 2008 Democrat-heavy models. They are lobbying them now to affect early voting. IVR [Interactive Voice Response] polls are heavily weighted. You can weight to whatever result you want. Some polls have included sizable segments of voters who say they are ‘not enthusiastic’ to vote or non-voters to dilute Republicans. Major pollsters have samples with Republican affiliation in the 20 to 30 percent range, at such low levels not seen since the 1960s in states like Virginia, Florida, North Carolina and which then place Obama ahead. The intended effect is to suppress Republican turnout through media polling bias. We’ll see a lot more of this. Then there’s the debate between calling off a random-digit dial of phone exchanges vs. a known sample of actual registered voters. Most polls favoring Obama are random and not off the actual voter list. That’s too expensive” for some pollsters.

I’m in the camp that wants to know–good and bad. I need to be prepared; and I’d rather have the truth, with whatever good or bad news that goes along with it.

In that vein, let’s have some in-depth analysis of the recent “surge” in Catholic voter support for Obama. I just don’t believe it.

mountainaires on September 27, 2012 at 2:57 PM

rvastar on September 27, 2012 at 11:29 AM

THIS!

And, thank you very much for that research and analysis.

mountainaires on September 27, 2012 at 3:02 PM

Hey, I wish you’d analyze these 2 new polls….I’d like to know the reality here. And, the liberal press is out in full force, posting these new pro Obama polls every few hours. It’s like they’ve got an army of pollsters out there doing their bidding suddenly. I just think this is so crazy; there is no way Obama has suddenly pulled out ahead on every issue. No freaking way.

I just don’t SEE IT.

Voters like Obama on Medicare: WaPo Poll

http://www.washingtonpost.com/politics/decision2012/medicare-working-to-boost-obama-in-swing-states-poll-finds/2012/09/27/b8a53a0e-0822-11e2-858a-5311df86ab04_story.html?hpid=z1

Catholics shift to Obama

http://andrewsullivan.thedailybeast.com/2012/09/catholics-flee-romney-ryan-dolan.html

mountainaires on September 27, 2012 at 5:27 PM

Hey, I wish you’d analyze these 2 new polls….I’d like to know the reality here. And, the liberal press is out in full force, posting these new pro Obama polls every few hours. It’s like they’ve got an army of pollsters out there doing their bidding suddenly.

mountainaires on September 27, 2012 at 5:27 PM

Well now you see how non-’bots felt during about 4 years of steady, hokey, manipulative pro-Romney polls.

ddrintn on September 27, 2012 at 5:39 PM

Ivastar, you point out this from Rasmussen:

From Rasmussen:

Aug 2004
Party ID: 35%R – 38%D – 27%I
Voter Enthusiasm: 51%R – 68%D
Election result: Bush wins narrowly

Aug 2006
Party ID: 32%R – 37%D – 31%I
Voter Enthusiasm: 40%R – 65%D
Election result: Dems wins solidly

Aug 2008
Party ID: 33%R – 39%D – 28%I
Voter Enthusiasm: 35%R – 61%D
Election result: Obama wins solidly

Aug 2010
Party ID: 34%R – 35%D – 31%I
Voter Enthusiasm: 48%R – 50%D
Election result: Republicans win blowout election unlike anything since the Great Depression.

And the party ID as of last month?

Aug 2012
Party ID: 38%R – 33%D – 29%I

The only think I would add is that Voter Enthusiasm from a Gallup poll in July of this year now favors Republicans by over 12 points.

http://drawandstrike.blogspot.com/2012/09/republican-voter-enthusiasm-is-way-up.html

manofaiki on September 27, 2012 at 5:59 PM

What I want is information and hopefully a bit more information than if I just read the press release’

As an example, if your gave me three polls, that is a first step in information (aka group 1). If you then said that the polls were D+7, D+5 and D+2, that is more information that was probably not highlighted in the press release (aka group 2 sort of).

But what if you gave me all three polls, the split and re-weighted all three polls to the same standard. Whether that is D+3. even, Rasmussen’s D/R/I split or whatever. Make the polls consistent. That provides a lot more information. I would suggest making them consistent based on even dems and GOP. That makes it easy to calculate what dem advantage would have to be to have Obama win (or conversely, the GOP advantage for Romney). Just add 1 to the D+? for every point Obama would need to win the poll.

Personally, one of the key numbers I focus on is the indies. As long as Romney is winning the indie vote, he will win the election. The idea that there are enough dems out there to outweigh the GOP and indie vote together doesn’t hold water. All of the battle ground states and nationally the dems and GOP are close enough that the winner of the indie vote will carry the day.

yetanotherjohn on September 28, 2012 at 9:58 AM

Here are two links that help to explain the current status of the polls. Romney is doing Very well

and

caprian on September 28, 2012 at 1:38 PM

As long as Romney is winning the indie vote, he will win the election. The idea that there are enough dems out there to outweigh the GOP and indie vote together doesn’t hold water. All of the battle ground states and nationally the dems and GOP are close enough that the winner of the indie vote will carry the day.

Bingo. Look at the 2008 numbers again:

Aug 2008
Party ID: 33%R – 39%D – 28%I
Voter Enthusiasm: 35%R – 61%D

Only 1/3rd of registered GOP voters where enthusiastic about voting, and yet, McCain only lost the election by 6% points. No matter how much they want to spin Obama’s election as some landslide victory, it simply wasn’t.

Again, when GOP voter enthusiasm is high (45%+), it is very difficult for Dems to win elections. When you add in a double-digit GOP lead with independents, it becomes impossible for Dems to win elections.

All of these “polls” are nothing more than a psy-ops operation. Period. The fact that, as we get closer to the election, they are relying on ever-more-outlandish samples – D+9-13%!…really?!? – is evidence of their desperation to “move the needle” while they can still do so under the guise of “we were just basing the polls on the voter participation from last election!”.

But don’t be fooled for one second: somewhere in these organizations, there’s someone who is looking at real numbers. And what they are watching for is to see if their endless lies can move the needle towards Obama. This crap is going to continue through October, as they hope that some miracle will happen during the debates to save him. But when it doesn’t, there’s going to come a point where they grab the parachutes and head for the exits. Why? Because while they may be willing to lie for Obama, they’re not going to be willing to die for him.

I think that next week – week of the 1st debate – is the key. If by next Friday – two days after the debate – we hear Brian Williams saying “A shocking new poll shows Mitt Romney gaining momentum”, you’ll know that their real polling numbers are telling them that the gig is up. They’ll then switch tactics from trying to depress Romney support to trying to scare Democrats into voting.

Tune in next Friday :)

rvastar on September 28, 2012 at 3:34 PM

Comment pages: 1 3 4 5