About that CNN poll …

posted at 9:51 am on April 17, 2012 by Ed Morrissey

Allahpundit wrote a lengthy piece about last night’s CNN poll, complete with some insightful commentary about the cross-tabs.  If you missed it — and by the number of comments on the post, it doesn’t appear than many of you have — be sure to read it now.  However, the sample data, as AP pointed out, lacked a few details, most prominent of which was the partisan split of the sampling.  Given the inclination of media polls to use wildly unrepresentative D/R/I splits in their samples, the lack of transparency on that point is telling.

That might not be the biggest problem with the poll, though.  Its biggest problem is … math.  Reader Raymond O did some math and asked a rather interesting set of questions in an e-mail last night about how CNN did theirs.  First, let’s start with the topline results, as reported by CNN: Obama 52%, Romney 43% among registered voters, 53/41 among all respondents.  If that’s the case, then the number of respondents in the latter case voting for Obama should be 538, and the number supporting Romney 416.

However, when reading the questions on page 3 of the poll report, that’s not at all what we see:

BASED ON 484 RESPONDENTS WHO PLAN TO VOTE FOR OBAMA — SAMPLING ERROR: +/- 4.5 PERCENTAGE PTS.
3. (Asked of Obama voters) Is that more a vote FOR Barack Obama or more a vote AGAINST Mitt
Romney? …

BASED ON 476 RESPONDENTS WHO PLAN TO VOTE FOR ROMNEY — SAMPLING ERROR: +/- 4.5 PERCENTAGE PTS.
4. (Asked of Romney voters) Is that more a vote FOR Mitt Romney or more a vote AGAINST Barack
Obama?

Since the combined total of the two exceeds their count of registered voters in the survey (910), we have to assume this refers to the general-population response.  That’s wildly different than the 53/41 split that CNN reports from the poll.  In fact, it’s only a 48/47 split for Obama.    And given that the poll shows a slightly better result for Romney among registered voters, it’s not difficult to conclude that Romney probably led in that category before CNN’s pollster shifted the results around to this extent.

Obviously, the pollster weighted the results — dramatically.  But on what basis?  Pollsters weigh results in likely-voter models; in fact, that’s a vital part of the likely-voter modeling process.  I’m not aware of the need to weight gen-pop polls unless the demos in the survey are wildly deficient, and since CNN didn’t disclose those, we don’t know — but it hardly imbues these results with confidence.  Speaking of which, the actual 484 responses for Obama is slightly more than 10% less than what it would take to get to a 53/41 split, far outside of the margin of error announced in the data (+/- 4.5%).  In other words, CNN’s pollster produced results that we’re supposed to believe represents the general population’s views within 4.5%, and then proceeded to weight them by increasing Obama’s advantage by almost 11%.

Again, I’m not aware of a need to weight results in general-population polls, and certainly not to the extent seen here, nor did CNN disclose that they did weight the results, especially in the general-population results. Without the rest of the demographic data included in this survey and the weighting methodology, this poll should be treated as utter fantasy.


Related Posts:

Breaking on Hot Air

Blowback

Note from Hot Air management: This section is for comments from Hot Air's community of registered readers. Please don't assume that Hot Air management agrees with or otherwise endorses any particular comment just because we let it stand. A reminder: Anyone who fails to comply with our terms of use may lose their posting privilege.

Trackbacks/Pings

Trackback URL

Comments

Comment pages: 1 2 3

It does the left no good to put out polls showing Obama with a good lead. All that does is pacify the liberal voters, and activate the conservative ones.

keep the change on April 17, 2012 at 11:55 AM

you’d think they had enough time to figure that out…at a second glance though, never ‘misunderestimate’ the left’s stupidity…combine that with the giant hubris of their dear leader and you have a preview of Nov 2012…

jimver on April 17, 2012 at 12:13 PM

This kind of nonsense happens too often and I’m convinced the MSM is going to try and do everything in its power to push the meme that Obama is invincible to dispirit conservatives.

Ever notice how these mistakes NEVER break our way?

You damn well know if CNN produced a poll that put Romney up by nearly 10 points they’d be going through it with a fine tooth comb before it was released, they couldn’t have Dear Leader embarrassed like that.

BradTank on April 17, 2012 at 12:15 PM

It does the left no good to put out polls showing Obama with a good lead. All that does is pacify the liberal voters, and activate the conservative ones.

keep the change on April 17, 2012 at 11:55 AM

It indeed does those two things, but it also makes the spineless handwringers in the middle want to jump on the bandwagon of whoever appears more popular. That’s why they do it.

Kataklysmic on April 17, 2012 at 12:05 PM

… and that is the same reason why, in the Republican primary, Romney has been given every benefit of the doubt by the media.

The true Not-Romneys will vote for Not-Romney in the upcoming primaries, but “the spineless handwringers in the middle want to jump on the bandwagon of whoever appears more popular” will likely vote for Romney, because they’ve been told that he’s the “inevitable” nominee, even when he has only 573 of the 1,044 delegates that he needs to win the nomination.

ITguy on April 17, 2012 at 12:17 PM

I roughed out the registered-voter partisan split yesterday based on the crosstabs’ margin of error (rounded to the nearest 0.5 percentage point) for the subgroups – 47% independent (MoE 4.5%), 28% Democrat (MoE 6.0%), 25% Republican (MoE 6.0%). It’s not exactly standard, even in a state where there is no explicit partisan registration.

As for the weighting, if that 484-476 split in favor of Obama was meant to be a measure of all respondents rather than registered voters, it’s an even worse example of weighting, as the “all-respondents” precentage was presented as 53%-41% in favor of Obama.

Steve Eggleston on April 17, 2012 at 12:19 PM

Last night in the original CNN poll thread I mentioned that CNN had tossed Gallup under the bus as a polling partner in 2007, and replaced them with longtime Clinton Fluffer Vinod Gupta. This of course was in anticipation of the inevitable Election in 2008 of Hillary. Vinnie even donated to her campaign.

Here’s a followup to that whole sordid story…

Del Dolemonte on April 17, 2012 at 12:20 PM

You’re right, it’s too early to even make a difference. I think time will help Romney because of all the wacky birth control and other social agenda issues applied to the GOP brand by Santorum Obama/Sebelius/Stephanopoulos.

All fixed.

bayam on April 17, 2012 at 10:33 AM

bmmg39 on April 17, 2012 at 12:21 PM

You’re right, it’s too early to even make a difference. I think time will help Romney because of all the wacky birth control and other social agenda issues applied to the GOP brand by Santorum Obama/Sebelius/Stephanopoulos.
bayam on April 17, 2012 at 10:33 AM

Fixed better.

bmmg39 on April 17, 2012 at 12:23 PM

even Chriz Cillizza at WaPo gets it

jimver on April 17, 2012 at 12:24 PM

There is no group of people in the world as dishonest as the U.S. media, not even politicians.

The initial article/segment is what is important because that’s what makes headlines and it’s what people notice. The corrections/retractions are ignored.

bw222 on April 17, 2012 at 12:24 PM

Del Dolemonte on April 17, 2012 at 12:20 PM

You may have missed my recent comment.
InfoUSA sold Opinion Research Corporation last July.
Lake City now has majority interest.
Lake City is heavily Republican.
Any idea what’s going on?

topdog on April 17, 2012 at 12:30 PM

Looks like this will be another 7 months of handwringing over polls like 2008. The problem then was no one was believing the weighting showing such strong Dem affiliation. In the end it was an enormous D +7. Even in 2010, it was even, and the GOP swept in historic numbers. So it can lean D and not be a disaster.

RCP has R +4 on generic congressional vote this morning. That’s HUGE. It was D +3 when the GOP swept in 1994.

So this will be a close one as it looks now.

Pollster average today (now owned by HuffPo) is Obama 45.4/Romney 45.2. And that includes this CNN poll.

Chuck Schick on April 17, 2012 at 12:34 PM

There is no group of people in the world as dishonest as the U.S. media, not even politicians.

The initial article/segment is what is important because that’s what makes headlines and it’s what people notice. The corrections/retractions are ignored.

bw222 on April 17, 2012 at 12:24 PM

couldn’t agree with you more…I lived in the UK most of my adult life and while their tabloid media is among the worst and most horrid I’ve ever seen, and the bias of their ‘decent’ side of the media is clearly to the left in general, I’ve never seen them carrying water for one or the other leader of the Labour, the way they do with Obama here, shamelessly…They (left-biased British media)skewered Gordon Brown, for instance for his crass incompetence and inane policies…and needless to say that they demolished effectively Blair’s premiership…

jimver on April 17, 2012 at 12:35 PM

Speaking of polls, PPP just released one for Florida. Shows Obama up by 5, 51/45 favorability, and that it would be worse than 50-45 if Rubio is on the ticket.

When you discount the PPP bias, the real number is probably between 48-47 Obama and 49-46 Romney. Either way, it is a toss-up. And favorability, it is either 49-47 approve, or 49-47 disapprove.

Either way, Florida is now a deadheat, and it can only get worse. Plus, considering a popular Republican is running for Senate, the Governor is a Republican, and Republicans should be able to organize better in Florida, I think Romney will win Florida easily.

milcus on April 17, 2012 at 12:43 PM

The true Not-Romneys will vote for Not-Romney in the upcoming primaries, but “the spineless handwringers in the middle want to jump on the bandwagon of whoever appears more popular” will likely vote for Romney, because they’ve been told that he’s the “inevitable” nominee, even when he has only 573 of the 1,044 1,144 delegates that he needs to win the nomination.

ITguy on April 17, 2012 at 12:17 PM

Fixed it for myself.

He has 573 and needs 571 more to get to the 1,144 delegates that he needs to win the nomination.

Romney has exactly 1 more than half of the number he needs.

But if all of the “spineless handwringers in the middle want to jump on the bandwagon of whoever appears more popular” think that Romney has already won, they’ll vote for him.

ITguy on April 17, 2012 at 12:51 PM

It’s not all that hard. Most of us only want to see two things out of a poll:

1) A reasonable split between D/R/I. I don’t have numbers near at hand, but 2010 exits (a great R election) were even. 2008 exits (a great D year) were D+7 or so. There is absolutely no rational basis for ever putting D more than 7 over R. None. The Dems are not remotely in as good a position as they were in 2008. Now they may not be in as bad a position as in 2010; we don’t know. But the polls that we see (and there have been more than a few, usually CNN/WaPo/ABC) with Dem advantages of +9 to +11 are just silly. A party ID of D +0 to 4 is more in line with the truth.

2) Some attmept to screen for people who are at least somewhat likely to actually vote. A the bare minimum, an RV screen should be applied. People can argue over LV models all they want, but the RV screen should be bare minimum to take a poll seriously.

Guess what Rasmussen does? And Gallup? And Fox? They have generally reasonable D/R/I and restrict responses to RV or LV.

A pollster can come up with any damned result they want. If I tried hard enough, I could get a poll proclaiming broccoli as the nation’s favorite food. It still wouldn’t be true.

FuzzyLogic on April 17, 2012 at 10:44 AM

Not me.

I don’t believe in weighting for party ID – and most reputable pollsters don’t do that – and I don’t care about screening likely voters this far from the elections; legitimate polls may be conducted among a variety of population samples, be it registered voters or adults. Obviously one should factor the population sample when analyzing the top lines.

Weighting for party self-identification has always been a contentious issue in election polling but, again, most credible pollsters don’t adopt that procedure (and IMO for good reason; there is no fixed amount of Ds/Rs/Is – in fact, it’s very fluid. Adjusting the sample for party ID would make impossible to capture those variances and it would always imply a subjective guess from the pollster. For example, should the pollsters be weighting their pollsters for the 2010 breakdown? The 2008 one, which was quite different? Something in between? And why? As long as the poll is well conducted, a random sample will tend to reflect the partisan views of the electorate).

What I want to see out of a poll is quite simple: meet the transparency standards.

joana on April 17, 2012 at 1:08 PM

Gallup presidential track: Romney 48 (+1) Obama 43 (-2). Obama approval 46% (+1)

Romney +5 today in Gallup.

rubberneck on April 17, 2012 at 1:20 PM

So this will be a close one as it looks now.

Pollster average today (now owned by HuffPo) is Obama 45.4/Romney 45.2. And that includes this CNN poll.

Chuck Schick on April 17, 2012 at 12:34 PM

The difference now is that there has been a significant change in voter self-identified party affiliation since 2008.

http://www.rasmussenreports.com/public_content/politics/mood_of_america/partisan_trends

Rasmussen is showing a 3 point edge nationally for Republicans currently.

crosspatch on April 17, 2012 at 1:25 PM

Gee someone working with poll (stat) numbers to say what they want to say?

SHOCKED I TELLS YA SHOCKED. Sigh.

This is how they try to control us.

kim roy on April 17, 2012 at 1:34 PM

CNN = Clintonian News Nimrods

Bitter Clinger on April 17, 2012 at 10:00 AM

Communist Nudging Newspeakers

PastorJon on April 17, 2012 at 1:38 PM

Least it not be forgotten; when Obama was campaigning for his cousin, Ordinga, in Kenya, for the presidency, Obama told him that if he loses to claim vote fraud and demand recounts. That advice did not rely on fraud actually occuring.

Ordinga followed Obama’s advice, then took it a step further and demanded the vote show he won. To help them make the decision he threw the nation into turmoil by having his people kill people from tribal areas he believed did not vote for him and forcing thousands to flee. The new Government compromised by creating the position of Prime Minister and giving it to him.

If Obama loses the election, would he, or those advising him, follow his own advice?

There does exist some worrisome facts that point to the possibility.

Polls that show him winning are a possible basis for claiming that the vote was stolen and demanding recounts that could take years, as shown by the Bush-Gore recounts. Of course it would have been shorter if the Supreme Court had ruled the Florida Supreme court had the power to create new law that favored the democrats in a recount. For those who don’t know, Bush won every recount that was later done.

Obama has enjoyed the immunity of impeachment action that would have removed any president that has held the office, many times in the three years he has been there. Is it because he, or those that put him in office, has the power to silence any impeachment proceeding, or fear of what might happen if he was impeached?

Obama has been quietly increasing the power of the Administration while at the same time reducing the relevancy and control of Congress and The Supreme Court with is defiance of Constitutional law, Presidential decrees and changes in law that he insisted on having, that give him the ultimate power to arrest without cause, citizens that are determined to be a threat, and a transfer of power to Homeland security which has been steadily built into a fully equipped citizen army or police that he controls.

According to Sheriff Joe Arpaio investigation findings, the media has been brought under his control with threats if certain subjects negative to Obama and his legitimacy are reported on.
Obama has consistently divided the nation along racial and ideological lines that separate those who support him from those who do not. The later is consistently attacked.

If Obama loses, will he declare voter fraud and take steps to control the flow information to prevent his removal from office?

Obama’s stakes are high. He has been groomed since childhood to be in this position of power yet he has failed to achieve the required goals in his first term. He needs a second term where he has more flexibility to complete those goals.

Franklyn on April 17, 2012 at 1:41 PM

Franklyn on April 17, 2012 at 1:41 PM

Ever since Clinton, the paranoid on either side have worried about the President refusing to give up power.

Fortunately, our Armed Services swore their oath to the Constitution, not some individual, and must defend it against all threats, “foreign and domestic.” I have faith in their oaths.

Adjoran on April 17, 2012 at 2:02 PM

I just want to know what psycho eyes is getting ready to do.

roy_batty on April 17, 2012 at 2:15 PM

As I said before it is going to be a reasonably close election because of Obama’s built-in welfare constituency so it’s a big mistake in April to ride an emotional roller coaster of good and bade polls for an election which will be held in November. Obama also has the unions, MSM and most of the big money Wall Street contributors.Obama also has 95% of all foreign countries on his side-giving him an international propaganda advantage. Also do not underestimate the Democratic propaganda machine–historically the Jews and Arabs agree on nothing and fight each other like competing inheritors-yet in America, Obama got nearly 80% of the Jewish vote and 93% of the Arab-American vote. People who can talk out of both sides of their mouths this successfully are indeed a force to be reckoned with.

This propaganda machine is also adept at digging up dirt-so if Romney picks a vice-presidential candidate who pinched a school girl’s bottom at 13 or was dismissed from class for cheating on a test at 17, you can be sure that this will be plastered all over the MSM-and if no scandal actually occurred-one can quickly be manufactured– and you can be sure that even the most minor verbal faux pas on the campaign trail will be blown out of proportion or taken out of context.

What this all means is a nail-biting election.

MaiDee on April 17, 2012 at 2:42 PM

The corrections/retractions are ignored back paged

bw222 on April 17, 2012 at 12:24 PM

DevilsPrinciple on April 17, 2012 at 2:49 PM

“lacked a few details, most prominent of which was the partisan split of the sampling”

Any poll that doesn’t reveal its sampling is illegitimate, good for nothing. Real Clear Politics shouldn’t include such polls, or polls with a clearly left-skewed sampling, in their average.

anotherJoe on April 17, 2012 at 3:01 PM

Oh the irony..

Both candidates suck.

It’s so fitting that we’re heading into 2012 with such horrible candidates.

It’s like they’re the manifest realities of our shizophrenic political nature as a country.

They both have numerous contradictions.

Wait till the debates.

They’ll be continually accusing eachother of “contradictions” and lies.

And they’ll BOTH be RIGHT.

This is why I want Ron Paul up there.

The President would find it impossible to accuse him of hypocrisy.

Even on the supposed pork spending – Paul never complains about “earmarks” so even with that there’s no hypocrisy.

So we Paulites are looking forward to bringing out 500 or so delegates to the convention just to let you guys know where the future of the party REALLY is.

fatlibertarianinokc on April 17, 2012 at 3:03 PM

http://www.gallup.com/poll/150743/Obama-Romney.aspx

Romney 48
Barry 43

Romney +5.

galtani on April 17, 2012 at 3:51 PM

Irony:

David Axelrod, President Obama’s top campaign strategist, responded to Gallup polls showing Mitt Romney leading the president by criticizing the pollster.

“Gallup is saddled with some methodological problems,” Axelrod tweeted today. He also directed Twitterati to a National Journal piece arguing that Gallup polls showing Romney in the lead “has a sample that looks much more like the electorate in 2010 than the voting population that is likely to turn out in 2012.”

Buy Danish on April 17, 2012 at 4:04 PM

What is the picture that links to this post from?

What is it about? The woman’s eyes make her look like she’s about to “go off”.

williamg on April 17, 2012 at 4:14 PM

What is the picture that links to this post from?

What is it about? The woman’s eyes make her look like she’s about to “go off”.

williamg on April 17, 2012 at 4:14 PM

Could be Coach K just got an irate call from the Bracketeer-in-Chief bitc#ing about Duke’s defense. The woman can’t believe it either.

Barnestormer on April 17, 2012 at 5:13 PM

Wow,

Triple face in palm and a pair of bug eyes.

banzaibob on April 17, 2012 at 5:37 PM

oh goody. Apparently somebody did read my earlier criticisms about taking polls at face value and ignoring their sampling shenanigans.

http://hotair.com/archives/2012/04/15/poll-tea-party-support-on-the-decline/

rayra on April 17, 2012 at 6:01 PM

He also directed Twitterati to a National Journal piece arguing that Gallup polls showing Romney in the lead “has a sample that looks much more like the electorate in 2010 than the voting population that is likely to turn out in 2012.”

Buy Danish on April 17, 2012 at 4:04 PM

and he knows exactly the voting population that is likely to turn up in 2012 because…err, he’s got a list with all the dead people supposed to vote this year??….

jimver on April 17, 2012 at 7:38 PM

williamg on April 17, 2012 at 4:14 PM

dunno where it came from but I did the “right-click, save picture as…” thing…I LOVE that pix!!!

teejk on April 17, 2012 at 8:02 PM

Without the rest of the demographic data included in this survey and the weighting methodology, this poll Nearly everything coming from CNN should be treated as utter fantasy.

I fixed it for you. :) :) :)

Theophile on April 18, 2012 at 12:45 AM

I don’t believe in weighting for party ID – and most reputable pollsters don’t do that – and I don’t care about screening likely voters this far from the elections; legitimate polls may be conducted among a variety of population samples, be it registered voters or adults. Obviously one should factor the population sample when analyzing the top lines.

This is a sort of subtle distinction, but from my perspective it isn’t about the weighting. The point is just to see whether the sample is honest or not. A D+11 sample is completely useless, and any poll based on such a sample can be safely ignored – it just doesn’t reflect reality.

Yes, party ID is fluid, which is why there was such a swing from 2008 to 2010. But look at how many pollsters insisted on showing polls reflecting a 2008ish D+7 sample in the run-up to 2010 … right up until the end, when they had to start showing polls with realistic samples to try to maintain credibility. The same thing is happening this year, and this is why you see such huge divergences – +5 Romney to +9 Obama.

Which poll’s sample actually reflects the current American electorate, and which one is crap?

Texastoast on April 18, 2012 at 1:29 AM

So what’s the deal with the face that girl in the cover photo is making?

dentarthurdent on April 18, 2012 at 2:22 PM

Comment pages: 1 2 3