Is polling science, art or witchcraft?

posted at 3:31 pm on September 30, 2012 by Jazz Shaw

One of the stories I’ve been following here at good ole’ Hot Gas this month has been Allahpundit’s ponderings on precisely what value – if any – the polls have in gauging the temperature of the electorate. (Here and here this week.) I’ve not only read AP’s questions and coverage, but many of your responses on the subject, and I’ve got to be honest here… I was still completely confused on a couple of fundamental points.

One of the first has to do with a subject which the current Oval Office occupant likes to trumpet… arithmetic. (Or, as Joe Biden might put it, a simple three letter word: MATH.) I mean, given all of the time that we spend obsessing over the polls here – as well as on every cable news channel – you’d think there was some actual, er… science behind it, wouldn’t you? Polling has been going on for longer than I’ve been alive and there are major elections every two years. Surely by this time somebody could have looked at the results of the various polling agencies and compared them to the final vote totals in the myriad races and determined who was hitting on all cylinders, right?

Apparently not. But some of AP’s questions have raised the specter of criteria we could use to figure out if anyone is putting their thumb on the scale and – more to the point – when. For the polls which publish their cross tabs, is there any metric where we could see if some of them are running an incredible D+ 1 bazillion in September and early October, but then suddenly push the margins down toward reality during the final two weeks so their “final” predictions would be more in line with reality? It seems to me like that would be a useful piece of information which could have been compiled by now. But has it?

Over the last couple of days I’ve been trying to track down some answers. This effort included speaking with polling analysts and pollsters themselves. The answers I received ran the gamut from things which sort of made sense when I heard them to contradictory responses which left me scratching my head. The last interview I did, and one of the most enlightening, was with Brad Coker of Mason-Dixon Polling and Research. The following are a few of my conclusions about the questions above when it comes to the science of polling.

First, there are varying claims about how pollsters arrive at the total number of people from each party affiliation (plus independents) who are to be surveyed in any given poll. One analyst said that it’s barely even a concern; they just pick a target number of interviews and let the chips fall where they may. (Within reason.) Brad wasn’t exactly that “hands off” about it. He said that they know from experience and previous results how many people of each affiliation are out there – more or less, because it does shift – but it’s one of the less reliable demographics. If he gets a number that’s totally out of reality in either direction and he has the time, he’ll try to get some more interviews to even it out. But the general consensus was that pollsters don’t start out shooting for “x” number of Democrats, “y” number of Republicans and “z” independents. Take that as you will.

Second, everyone seems to agree that poll results shift as you get closer to this election. But as opposed to some nefarious plot to influence the vote, the virtually unanimous response from these industry insiders is that polling is extremely tenuous a year in advance of an election. Identifying “likely voters” at that point is impossible to pin down for a variety of reasons, including people who move, people who only come of age to vote shortly before the election, people who die, and folks who simply aren’t paying attention that early and may not have any clue if they’ll be voting or not. In the final weeks before the election you can structure a much better likely voter model, and this will tend to shift the numbers when it happens.

Next, the subject of “weighting” was addressed by a couple of people. This, in my opinion, is where we really get into the “man behind the curtain” mystery ride. The people I spoke with were pretty much in agreement that weighting is done primarily as a matter of experience in the field and it happens on an ad hoc basis. For example, young people are harder to get a full interview with than seniors, so they typically get less responses from them. In response, they will “weight” the results to increase the influence of younger voters and decrease that of seniors if the number of responses is too far off from the usual turnout numbers. The precise numbers for that weighting don’t come from any handbook or specific formula… you just have to know how to do it.

The only thing everyone seemed to agree on was that no legitimate pollsters are influenced by the media, by partisan bias or money. (The term “legitimate” in this case is meant to exclude campaign push polls and marketing calls.) They’re just producing data based on research and they all seem to feel that their results have panned out pretty closely to actual results over the years, with the notable exceptions of 1980 and 2000, and to a lesser degree, 1996.

As usual, we leave it up to you to judge these explanations.


Related Posts:

Breaking on Hot Air

Blowback

Note from Hot Air management: This section is for comments from Hot Air's community of registered readers. Please don't assume that Hot Air management agrees with or otherwise endorses any particular comment just because we let it stand. A reminder: Anyone who fails to comply with our terms of use may lose their posting privilege.

Trackbacks/Pings

Trackback URL

Comments

Comment pages: 1 2

and you have to be sure of a set of ideas if you want to have any chance of persuading independent voters, because hate alone will not suffice.

nathor on September 30, 2012 at 5:04 PM

Love of Country comes to mind.

VegasRick on September 30, 2012 at 5:15 PM

@b9 :)

Media leverage came from its conditional coverage, but when its coverage is unconditionally hostile then what the media makes of a given thing no longer matters. A completely predictable media is also a completely boring media. It is a media that fewer and fewer people bother with anymore. The media’s power is the power of a troll, the power of a noise machine, the power of being paid attention to. The more people stop paying attention to it, the more it dies.

http://sultanknish.blogspot.com/2012/09/the-great-media-noise-machine.html

Axe on September 30, 2012 at 5:15 PM

I just read that 91% of people nowadays — even among those contacted — refuse to answer polls. How is anyone ever going to “scientifically” determine the effect of that?

Is there a built-in bias in that giant data gap?

Personally, I get asked for my opinion about a hundred times a day. For people with real jobs, it gets to be a giant pain in the ass. But for most liberals it’s a completely novel experience, and it’s the closest thing to self-esteem they will ever feel.

logis on September 30, 2012 at 5:16 PM

This is completely untrue. Go back and see for yourself if polls were accurate this far out? We’re not days before the election. What if the pollsters update their polls to show a trend towards Mitt just before the election? What happens to your narrative? Every single year, the polls are knowingly skewed. This is not a fantasy or wishful thinking. There is proof of it happening. Look at the Walker recall. Look at 2008 two weeks out. Look at 2010. Look at what they did to Hillary during the primaries. What they did against Hillary was insane. They will do this against Mitt. It’s going to get a WHOLE lot worse before election day.

They want you to give up.

MrX on September 30, 2012 at 4:49 PM

It’s amusing that falsehoods like this are so engraved into the minds of conservatives. Case in point, since you made a note of Walker, three weeks before the recall Nate Silver gave him a 95% chance of winning with the poll average margin of 6%.

In 2010 Nate called Senate GOPs total gain by missing only one. He predicted 55 GOP gains in the house, and that was not 100% perfect but still within confidence interval. He got 36 out of 37 governors right too.

lester on September 30, 2012 at 5:18 PM

FYI, one of the people I talked to *was* Nate Silver, but we elected not to use that for various reasons. But we heard from both sides. They’re a pretty close-mouthed group by and large, but it’s clear that most of them include a variety of fudge factors into their final numbers based on “experience and historical data” but I couldn’t make much of that, since they don’t share the details of their predictive models. Really, I was left with more questions than answers by the time I was done.

Jazz Shaw on September 30, 2012 at 4:17 PM


“If you can’t explain it simply, you don’t understand it well enough.”

You made the simple mistake of talking to the wrong people, Jazz.

Pollsters don’t practice art, science or witchcraft – they practice soothsaying. If they could do it reliably, they wouldn’t be pollsters …

… they would be rich, retired and living on a beach.

Who should you talk to?

Easy people who use statisitics to make life and death decisions.

In any field of that nature, the professionals would say the EXACT SAME THING,

“A 9% sample rate is statistically insignificant and NO reliable estimate can be based on such a small number.”

I’ll bet not a single one of the pollsters you talked to said anything along those lines, did they?

Which is the bottom line for anyone looking at polls basec on the standard polling approach this year. Not one of them has large enough of a participation sample to be worth a bucket of warm spit.

PolAgnostic on September 30, 2012 at 5:19 PM

Ummmm Okay. ? You implying something about me or in general?

bazil9 on September 30, 2012 at 5:09 PM

in general

nathor on September 30, 2012 at 5:20 PM

lester on September 30, 2012 at 5:18 PM

Yes, we’re liars, boy. No, go get to the top of the Space Needle and jump.

Lanceman on September 30, 2012 at 5:21 PM

Love of Country comes to mind.

VegasRick on September 30, 2012 at 5:15 PM

this implying that obama does not love his country which comes off as an hateful remark and bound to persuade no one.

nathor on September 30, 2012 at 5:24 PM

Axe on September 30, 2012 at 5:15 PM

Thanks axe. :)

I dont waste a precious moment of my life
indulging liars or propaganda machines.
The media is fairly dead to me.
Prob will be forever.

bazil9 on September 30, 2012 at 5:25 PM

this implying that obama does not love his country which comes off as an hateful remark and bound to persuade no one.

nathor on September 30, 2012 at 5:24 PM

according to you. And I am not implying that obambi does not love this Country. I’m stating it as a fact.

VegasRick on September 30, 2012 at 5:28 PM

Subotai Bahadur on September 30, 2012 at 4:15 PM

Haven’t seen you on here before, but I like the cut of your jib. Nice, well thought out post.

AZfederalist on September 30, 2012 at 5:29 PM

you guys need to realize Mitt is losing

DBear on September 30, 2012 at 5:31 PM

you guys need to realize Mitt is losing

DBear on September 30, 2012 at 5:31 PM

You need to realize you’re a douchebag.

Lanceman on September 30, 2012 at 5:32 PM

One does not have to walk in hatred to vote AGAINST what the incumbent’s policies will do to the country if given another term.

look, obamas policies are similar to what many in the dem party have been defending for years. before obamacare, there was hilary pushing for health care reform as well.

Dislike Mitt ? Fine. Dislike O’s plans even more than the alternative ?? That does not equal being motivated by hate, just doing our part to stem the tide of destruction anticipated.

pambi on September 30, 2012 at 5:11 PM

the problem with mitt is that he is not offering anything very different from obama.

nathor on September 30, 2012 at 5:33 PM

you guys need to realize Mitt is losing

DBear on September 30, 2012 at 5:31 PM

You need to realize you’re a douchebag.

Lanceman on September 30, 2012 at 5:32 PM

He knows. Just like we do.

VegasRick on September 30, 2012 at 5:33 PM

nathor on September 30, 2012 at 5:33 PM

Says you ! You refuse to see any differences. I can’t make you see them.
I simply do not agree with your assessment.

pambi on September 30, 2012 at 5:39 PM

I always lose the election in the polls, and I always win it on election day.
-Benjamin Netanyahu

lynncgb on September 30, 2012 at 5:47 PM

Is polling science, art or witchcraft?

As practiced by the MSM, it’s NONE OF THE ABOVE: it’s simply PROPAGANDA!!

landlines on September 30, 2012 at 5:50 PM

Polling is an election sideshow, an exercise in futility that creates jobs for a few people and gives hours of fun and pleasure to statistics mavens.

It just occurred to me that the seeds of mistrust of polls were planted this past June. Remember the crap sandwich the slimestream media fed us in June when they crapped LIEs all over the exit polls in Wisconsin, claiming Walker and Barrett were in a dead heat and oh, look, Walker won bigger than he did the first time. And further, the filling to that crap sandwich was how the slimestreamers made a bigger stink about how those phony exit polls showed Obama was still up in Wisconsin which seemed to matter more to them than the result of the governor’s race cuz it wasn’t the result they wanted and campaigned for.

As far as presidential elections, well, the pollsters have been wrong 3 out of the past 8 times which is closer to half of the time than not, and they were also terribly wrong in the 1994 and 2010 mid-terms. But isn’t it really all about turnout and whose supporters turn out? Once the voting begins, the polls are useless as the slimestream media proved to us all beyond a doubt in June.

stukinIL4now on September 30, 2012 at 5:52 PM

If this race was the runway for Obama that the Trolls claim it is, they would not have to be here, trying to demoralize us.

kingsjester on September 30, 2012 at 5:54 PM

It’s amusing that falsehoods like this are so engraved into the minds of conservatives. Case in point, since you made a note of Walker, three weeks before the recall Nate Silver gave him a 95% chance of winning with the poll average margin of 6%.

In 2010 Nate called Senate GOPs total gain by missing only one. He predicted 55 GOP gains in the house, and that was not 100% perfect but still within confidence interval. He got 36 out of 37 governors right too.

lester on September 30, 2012 at 5:18 PM

Obama’s approval rating currently looks a lot like it did at this point in 2010. How did that work out for the Dems. Teh fact is that polls aren’t terribly predictive, aside from their value in helping identify trends, and in proving a snapshot of current conditions. This doesn’t make polling unscientific or useless. Polling is a lot like taking a person’s temperature is extremely helpful in helping in determining current health status, but it doesn’t say alot about what that person’s temperature will be in several weeks. To make this kind of a prediction, one has to consider other factors and information. The imprecision associated with election outcomes has alot more to do with shortcomings and limitations in political science and with the limitations of data collection.

ghostwriter on September 30, 2012 at 6:00 PM

“That they got it wrong with Walker’s recall was denial?”

They didn’t get it wrong with Walker’s recall. The RCP average was spot-on.

Poll Date Sample Walker (R) Barrett (D) Spread
Final Results – – 53.1 46.3 Walker +6.8
RCP Average 5/17 – 6/3 – 51.5 44.8 Walker +6.7
WeAskAmerica 6/3 – 6/3 1570 LV 54 42 Walker +12
PPP (D) 6/2 – 6/3 1226 LV 50 47 Walker +3
Marquette University 5/23 – 5/26 600 LV 52 45 Walker +7
WPR/St. Norbert 5/17 – 5/22 406 LV 50 45 Walker +5

gumbyandpokey on September 30, 2012 at 6:01 PM

“A 9% sample rate is statistically insignificant and NO reliable estimate can be based on such a small number.”

I’ll bet not a single one of the pollsters you talked to said anything along those lines, did they?

Which is the bottom line for anyone looking at polls basec on the standard polling approach this year. Not one of them has large enough of a participation sample to be worth a bucket of warm spit.

PolAgnostic on September 30, 2012 at 5:19 PM

ghostwriter on September 30, 2012 at 6:02 PM

Here is analysis of a recent Rasmussen poll where it was basically a dead heat between zero and Romney. Even in this Rasmussen uses 5.1% more dems for some odd reason. supposedly its a compilation of previous elections. But what the heck will that show? I just don’t get what is going on with these polls at all.

http://blogs.middlebury.edu/presidentialpower/2008/09/19/how-many-democrats-how-many-republicans/

rodguy911 on September 30, 2012 at 6:02 PM

“A 9% sample rate is statistically insignificant and NO reliable estimate can be based on such a small number.”

I’ll bet not a single one of the pollsters you talked to said anything along those lines, did they?

Which is the bottom line for anyone looking at polls basec on the standard polling approach this year. Not one of them has large enough of a participation sample to be worth a bucket of warm spit.

PolAgnostic on September 30, 2012 at 5:19 PM

It’s unlikely that any of the pollsters said that because it isn’t true. Depending on the application, a nine-percent sample rate is more than sufficient if the sample is drawn properly, with sufficient randomness.

ghostwriter on September 30, 2012 at 6:06 PM

‘Sup, Gumbo?

Lanceman on September 30, 2012 at 6:07 PM

Here is analysis of a recent Rasmussen poll where it was basically a dead heat between zero and Romney. Even in this Rasmussen uses 5.1% more dems for some odd reason. supposedly its a compilation of previous elections. But what the heck will that show? I just don’t get what is going on with these polls at all.

http://blogs.middlebury.edu/presidentialpower/2008/09/19/how-many-democrats-how-many-republicans/

rodguy911 on September 30, 2012 at 6:02 PM

Frankly, I think that these polls are all saying the same thing, but folks like us miss it, because we are studying it too closely. We’re missing the forest for the trees. The polls are saying that the race is close, that Obama seems to be clinging to a small, insignificant lead, that this dynamic has persisted throughout the summer, that Romney hasn’t closed the deal, and that the people are open to considering a change.

This thing is going to get settled over the next five weeks. If I was a betting man, I would put my money on Romney, because this current dynamic doesn’t seem to favor the incumbent.

ghostwriter on September 30, 2012 at 6:11 PM

If the polls are wrong, it won’t be because of sample rate or people not answering the phones…that stuff always goes on. It would have to be intentionally using the polls as propaganda.

The manipulation of the polls will be a national scandal if Romney wins. But that’s not going to happen, imo.

The trends in the polls are more important than the actual numbers, and Romney is hemorrhaging in every poll.

gumbyandpokey on September 30, 2012 at 6:12 PM

EDIT

“A 9% sample rate is statistically insignificant and NO reliable estimate can be based on such a small number.”

I’ll bet not a single one of the pollsters you talked to said anything along those lines, did they?

Which is the bottom line for anyone looking at polls basec on the standard polling approach this year. Not one of them has large enough of a participation sample to be worth a bucket of warm spit.

PolAgnostic on September 30, 2012 at 5:19 PM

It’s unlikely that any of the pollsters said that because it isn’t true. Depending on the application, a nine-percent sample rate is more than sufficient if the sample is drawn properly, with sufficient randomness, and if the sample is sufficiently representative of the larger population.

ghostwriter on September 30, 2012 at 6:06 PM

ghostwriter on September 30, 2012 at 6:13 PM

gumbyandpokey on September 30, 2012 at 6:12 PM

Yes, Romney’s done, Gumbo. A done tom turkey.

Lanceman on September 30, 2012 at 6:14 PM

If the polls are wrong, it won’t be because of sample rate or people not answering the phones…that stuff always goes on. It would have to be intentionally using the polls as propaganda.

The manipulation of the polls will be a national scandal if Romney wins. But that’s not going to happen, imo.

The trends in the polls are more important than the actual numbers, and Romney is hemorrhaging in every poll.

gumbyandpokey on September 30, 2012 at 6:12 PM

That isn’t necessarily true.

ghostwriter on September 30, 2012 at 6:15 PM

“Is polling science, art or witchcraft?”
It’s a business, with winners and losers bought and paid for. Polls show BO up by 9% in Ohio. If so, why is he here (Ohio) twice a week? I know he wants to get out of the house and have a pizza, but why Ohio if he’s so far ahead?

NotEasilyFooled on September 30, 2012 at 6:19 PM

unprincipled conservatives and Establishment types who insisted that Mitt was the only solution.
newtopia on September 30, 2012 at 4:55 PM

First “unprincipled conservatives” is an oxymoron; By definition and practice, true Conservatives are very principled. Perhaps you meant Republicans?

And Second; Lumping in Conservatives with Establishment types is like lumping bleach in with ammonia. Not very smart unless your goal is just to poison the local atmosphere.

But then, That seems to be your goal, so…..

Nevermind

LegendHasIt on September 30, 2012 at 6:20 PM

The trends in the polls are more important than the actual numbers, and Romney is hemorrhaging in every poll.

gumbyandpokey on September 30, 2012 at 6:12 PM

So what exactly is your purpose here? Are you looking for everyone to say, “yep, Romney’s done, nothing can be done about it, the polls are just too overwhelming, no way he can win now”? To what end is your endless negative trolling attempting? Well, let me add, endless negative, misleading trolling. You consistently ignore the democrat oversample, you tell us we didn’t see what we saw during previous elections by consistently citing the final polls of various election cycles.

So what’s your point here? You just want us to give up so that your negativism becomes a self-fulfilling prophecy and we stay home and don’t vote?

Let me clue you in, even if the polls showed Romney down 20, everybody on this blog would still crawl through broken glass to go vote against the marxist in the White House.

Frankly, you had better pray that your negativism is wrong and that Romney does win. If Jugears wins, it’s the end of this country as you know it. Your freedoms will not matter a whit to the marxists in the regime, they don’t care about you — look how they’ve thrown their own close acquaintances under the bus. You think they care about some nameless liberal blog commenter like you? Another four years like the past four and this country as founded is done. It either splits into two, with one maintaining the original freedoms as founded (and you will not be welcome there) and the other the socialist utopia you desire or all wind up with a marxist state that increasingly encroaches the remaining freedoms we have.

AZfederalist on September 30, 2012 at 6:26 PM

….this implying that obama does not love his country ….
nathor on September 30, 2012 at 5:24 PM

Oh, 0bama may very well love his country a lot…..

We just wonder what country that may be, because it quite obviously is not the U.S.A.

LegendHasIt on September 30, 2012 at 6:27 PM

“It’s a business, with winners and losers bought and paid for. Polls show BO up by 9% in Ohio. If so, why is he here (Ohio) twice a week? I know he wants to get out of the house and have a pizza, but why Ohio if he’s so far ahead?

NotEasilyFooled on September 30, 2012 at 6:19 PM”

Where is Obama supposed to go?

All he has to do is win one of Ohio, FLorida or Virginia and the election is locked up. So he just keeps going to those states over and over again, and squeezes a trip in to IA and WI once in a while to hold his “softest” Blue states.

gumbyandpokey on September 30, 2012 at 6:30 PM

All he has to do is win one of Ohio, FLorida or Virginia and the election is locked up. So he just keeps going to those states over and over again, and squeezes a trip in to IA and WI once in a while to hold his “softest” Blue states.

gumbyandpokey on September 30, 2012 at 6:30 PM

False. There are paths to victory for Romney that do not include Ohio.

ghostwriter on September 30, 2012 at 6:33 PM

Oh, 0bama may very well love his country a lot…..

We just wonder what country that may be, because it quite obviously is not the U.S.A.

LegendHasIt on September 30, 2012 at 6:27 PM

yay, a birther! birtherism is perhaps the most toxic expression of obama hatred that manage to turn off more independent voters from the GOP. well done! are you proud?

nathor on September 30, 2012 at 6:33 PM

All he has to do is win one of Ohio, FLorida or Virginia and the election is locked up. So he just keeps going to those states over and over again, and squeezes a trip in to IA and WI once in a while to hold his “softest” Blue states.

gumbyandpokey on September 30, 2012 at 6:30 PM

And Obama has made an inordinate number of trips to Iowa this year. I would bet that he has been to Iowa more this time around than in 2008 (obviously, excluding the primaries).

ghostwriter on September 30, 2012 at 6:35 PM

If this race was the runway for Obama that the Trolls claim it is, they would not have to be here, trying to demoralize us.

kingsjester on September 30, 2012 at 5:54 PM

Indeed KJ. They spend a lot of their time
typing away
day after day.
Gee-why is that?

bazil9 on September 30, 2012 at 6:38 PM

If this race was the runway for Obama that the Trolls claim it is, they would not have to be here, trying to demoralize us.

kingsjester on September 30, 2012 at 5:54 PM

Indeed KJ. They spend a lot of their time
typing away
day after day.
Gee-why is that?

bazil9 on September 30, 2012 at 6:38 PM

Let ‘em gloat now; I’d rather do my gloating on November 7.

ghostwriter on September 30, 2012 at 6:39 PM

yay, a birther! birtherism is perhaps the most toxic expression of obama hatred that manage to turn off more independent voters from the GOP. well done! are you proud?

nathor on September 30, 2012 at 6:33 PM

If ever there was a party that could, and would perpetrate a manchurian candidate, with absolutely no reservation whatsoever, which ‘progressive’ party would that be?

WHY are all of obama’s records sealed?

Lanceman on September 30, 2012 at 6:41 PM

Just the fact that you have “left leaning” and “right leaning” polling outfits shows there’s a problem. The fact is the “left leaning” outfits like PPP have not only cooked the books, they’ve found it necessary to REALLY cook the books, to show Obama ahead by 10 points in some cases. Why? Because there are many people looking at the Real Clear Politics average so, to cook those numbers, there have to be polls with huge differences, hence PPP.

The point is, forget the polls, look at the facts on the ground. Ohio has had half as many Democrats request absentee ballots and the margin betwen Democrats and Republican requests is closer. Is that a sign of a 9 point Obama victory?

I firmly believe the Democrats are in big trouble and, if this were the republicans, the press would be on top of this story. Just imagine, if you will, George Bush winning an election like Obama, passing legislation that the press didn’t like because he had majorities in Congress, and then, losing the Kennedy seat in Massachusetts, losing the governorship in New Jersey and losing the governorship in Virginia. Then, after that, losing over 60 seats in the House and 7 seats in the Senate, the following year. Can you tell me that wouldn’t be the press narrative right now? Of course not, but the fact that it happened to Obama is the reason the polls are being cooked and everything else because the press knows what’s going on and they know, if they gave Obama the same press they’d give a Republican, he’d be down 10 points himself right now.

bflat879 on September 30, 2012 at 6:42 PM

nathor on September 30, 2012 at 6:33 PM

Comprehension and detection of sarcasm and wit are not exactly your forte are they? Well, on the wit side, you do score 1/2 credit.

AZfederalist on September 30, 2012 at 6:46 PM

I firmly believe the Democrats are in big trouble and, if this were the republicans, the press would be on top of this story. Just imagine, if you will, George Bush winning an election like Obama, passing legislation that the press didn’t like because he had majorities in Congress, and then, losing the Kennedy seat in Massachusetts, losing the governorship in New Jersey and losing the governorship in Virginia. Then, after that, losing over 60 seats in the House and 7 seats in the Senate, the following year. Can you tell me that wouldn’t be the press narrative right now? Of course not, but the fact that it happened to Obama is the reason the polls are being cooked and everything else because the press knows what’s going on and they know, if they gave Obama the same press they’d give a Republican, he’d be down 10 points himself right now.

bflat879 on September 30, 2012 at 6:42 PM

The press is definitely biased. I’m not so sure about the polling outfits. It simply could be their methodological choices that lead to the skew in their polls. I wonder if there have been any studies on the possibility of moving public opinion by showing polls with a particular outcome…

ghostwriter on September 30, 2012 at 6:48 PM

If ever there was a party that could, and would perpetrate a manchurian candidate, with absolutely no reservation whatsoever, which ‘progressive’ party would that be?

crazy argument! not only i not sure which party is more vulnerable to “manchurian candidates” but the whole idea that obama was inserted in early life onto america so that he could somehow manage to get hold of the presidency is preposterous.

WHY are all of obama’s records sealed?

Lanceman on September 30, 2012 at 6:41 PM

why romney does not show his taxes? the same reason, it gives no political benefit. but I very much doubt that is to hide a ridiculous conspiracy theory.

nathor on September 30, 2012 at 6:48 PM

“False. There are paths to victory for Romney that do not include Ohio.”

If Ohio goes to Obama, it’s highly unlikely that Romney will carry both Colorado and Iowa, which he would need to do. WI is off the table, imo.

It’s interesting that Ryan is coming to WI soon, but it won’t be for a rally, but a fundraiser at a local hotel.

gumbyandpokey on September 30, 2012 at 6:51 PM

The trends in the polls are more important than the actual numbers, and Romney is hemorrhaging in every poll.

gumbyandpokey on September 30, 2012 at 6:12 PM

…poll…pole…poll…hemorrhagingmypole…poll…pole…poll…whackingmypole…poll…pole…poll…noknobonmypole…poll…pole…poll…

KOOLAID2 on September 30, 2012 at 6:57 PM

nathor on September 30, 2012 at 6:33 PM

Did I mention, anywhere, his place of birth?

No, I did not.

I know many people who were not born in the nation that they consider ‘theirs’. And many born in the USA hate it, and would prefer to live elsewhere, but lack the guts to actually emigrate.

I only wrote that 0bama OBVIOUSLY does not love the U.S.A. He campaigned on FUNDAMENTALLY CHANGING IT.

And as far as ‘turning off’ from the GOP, a bunch of squishes with no values at all… I could care less, because the modern GOP Establishment is just as valueless as they are.

LegendHasIt on September 30, 2012 at 7:02 PM

why romney does not show his taxes? the same reason, it gives no political benefit. but I very much doubt that is to hide a ridiculous conspiracy theory.

nathor on September 30, 2012 at 6:48 PM

Man, you are one disingenuous, lying fvck.

Lanceman on September 30, 2012 at 7:03 PM

The trends in the polls are more important than the actual numbers, and Romney is hemorrhaging in every poll.

gumbyandpokey on September 30, 2012 at 6:12 PM

It’s not a trend, if it were then the pollsters wouldn’t need to have samples with a D+9. They could do a D+4 sample and show the same, but the problem is they can’t. If they did it would show something similar to Rasmussen, a statistical tie.

opustx on September 30, 2012 at 7:03 PM

…1/2 credit.
AZfederalist on September 30, 2012 at 6:46 PM

Oh, Snap!!!

LMAO

LegendHasIt on September 30, 2012 at 7:04 PM

Man, you are one disingenuous, lying fvck.

Lanceman on September 30, 2012 at 7:03 PM

wtf?

nathor on September 30, 2012 at 7:07 PM

wtf?

nathor on September 30, 2012 at 7:07 PM

You heard me, simpleton.

You honestly think if there was anything wrong with Romney’s tax returns obama’s IRS wouldn’t be all over him?

But your messiah is supposed to be the smartest person ever to become precedent, yet his school records are sealed and millions are spent by him keeping them that way.

Yeah. All politics.

Lanceman on September 30, 2012 at 7:11 PM

You heard me, simpleton.

You honestly think if there was anything wrong with Romney’s tax returns obama’s IRS wouldn’t be all over him?

But your messiah is supposed to be the smartest person ever to become precedent, yet his school records are sealed and millions are spent by him keeping them that way.

Yeah. All politics.

Lanceman on September 30, 2012 at 7:11 PM

I dont think there is nothing illegal in romney tax returns, just more unsavory information that dems would demagogue and the same with obama records. probably nothing illegal but still repubs would demagogue and extrapolate all kinds of nonsense.

nathor on September 30, 2012 at 7:22 PM

Surely by this time somebody could have looked at the results of the various polling agencies and compared them to the final vote totals in the myriad races and determined who was hitting on all cylinders, right?

Well, why would they? The polling firms all say they were extremely accurate once you take into account the wild swings in voter sentiment that occurred over the 24 or 36 hours just prior to the election.

Fred 2 on September 30, 2012 at 7:24 PM

I dont think …..
nathor on September 30, 2012 at 7:22 PM

Admitting the problem is the first step on the road to recovery.
Congratulations!

;-)

LegendHasIt on September 30, 2012 at 7:26 PM

I dont think there is nothing illegal in romney tax returns, just more unsavory information that dems would demagogue and the same with obama records. probably nothing illegal but still repubs would demagogue and extrapolate all kinds of nonsense.

nathor on September 30, 2012 at 7:22 PM

You see, your moral equivalence game is just one big fail here. You have Romney’s income, tax, and charitable contribution numbers for his working life as well has his last two years worth of tax returns. Heck, we’ve been told Romney cut somebody’s hair off when he was in high school and even have a note from the teacher saying Mitt needed to apply himself more. You have zero, zip, zilch, nada single bit of data regarding Obama’s performance at any college he attended. Your side continually makes lofty claims about how brilliant he is, what a genius, etc. You’ve got no data to back that up. We don’t know what his grades were or even a GPA. We don’t know what classes he took. We don’t know how he managed to go from Occidental to Columbia. We don’t know who paid for his education. Your candidate has the most opaque backgrounds of any presidential candidate in modern times. The MSM refused to vet him in 2008 and are giving him a pass now.

It’s not demagoguing to want to know if the most brilliant president in history really is or is just a chump saying he is (well and having the MSM assert that he is as well)

AZfederalist on September 30, 2012 at 7:36 PM

I dont think there is nothing illegal in romney tax returns, just more unsavory information that dems would demagogue and the same with obama records. probably nothing illegal but still repubs would demagogue and extrapolate all kinds of nonsense.

nathor on September 30, 2012 at 7:22 PM

==============

It really would be an inconvenient for Obama if the so-called smartest president in history was a mediocre student (you know, like Gore was…). And it also would be nice to know if Obama pulled a Fauxcohontas to get into Harvard, trying to pass himself off as a foreign student to get favorable treatment .

ghostwriter on September 30, 2012 at 7:37 PM

Just received a phone call for a “random political poll” it rapidly became obvious that the poll was anything but unbiased and being conducted for the State Democrats. The main concern was that the voice inflections used by the poll taker would lead you into the answer they wanted. If the question asked was to lead you into a negative response for a certain question then the question asked ended in a lower pitched “negative tone”, if the desired response was meant to be positive then the interviewer ended the question in a higher pitched uplift tone. These action are very subtle but then these political parties have gotten very very good and manipulating are views.

jpcpt03 on September 30, 2012 at 7:47 PM

I dont think there is nothing illegal in romney tax returns

As my fifth grade grammar teacher would have said, if I had used a double negative like that:
“So, you are, in effect, saying that you think that there is something illegal in THE Romney tax returns.”

Sorry to keep picking on you. But the guy I really wanted to pick on seems to have left.

LegendHasIt on September 30, 2012 at 7:47 PM

It’s unlikely that any of the pollsters said that because it isn’t true. Depending on the application, a nine-percent sample rate is more than sufficient if the sample is drawn properly, with sufficient randomness, and if the sample is sufficiently representative of the larger population.

ghostwriter on September 30, 2012 at 6:06 PM


Except …

One analyst said that it’s barely even a concern; they just pick a target number of interviews and let the chips fall where they may. (Within reason.) Brad wasn’t exactly that “hands off” about it. He said that they know from experience and previous results how many people of each affiliation are out there – more or less, because it does shift – but it’s one of the less reliable demographics. If he gets a number that’s totally out of reality in either direction and he has the time, he’ll try to get some more interviews to even it out. But the general consensus was that pollsters don’t start out shooting for “x” number of Democrats, “y” number of Republicans and “z” independents. Take that as you will.


And …

Next, the subject of “weighting” was addressed by a couple of people. This, in my opinion, is where we really get into the “man behind the curtain” mystery ride. The people I spoke with were pretty much in agreement that weighting is done primarily as a matter of experience in the field and it happens on an ad hoc basis. For example, young people are harder to get a full interview with than seniors, so they typically get less responses from them. In response, they will “weight” the results to increase the influence of younger voters and decrease that of seniors if the number of responses is too far off from the usual turnout numbers. The precise numbers for that weighting don’t come from any handbook or specific formula… you just have to know how to do it.

.
Please cite ANY profession (other than polling) wehich would rely on a 9% sample rate as being anything other than STATISTICALLY INSIGNIFICANToh, and grounds for being sued out of existence for gross negligence.

PolAgnostic on September 30, 2012 at 7:48 PM

.
Please cite ANY profession (other than polling) wehich would rely on a 9% sample rate as being anything other than STATISTICALLY INSIGNIFICANT … oh, and grounds for being sued out of existence for gross negligence.

PolAgnostic on September 30, 2012 at 7:48 PM

Quality control is an obvious example. If you produce a widget, and wish to sample some finished items to ensure they are up to specifications, 5 or 10 percent should be more than sufficient, with a random sample.

It seems to me that you are not using the term statistically significant in a correct fashion. A small sample can provide statistically significant results if is representative of the larger population. I don’t need to flip a times coin a million times to identify that in the limit it the probability of getting heads will be close to 50 percent (with a true coin).

ghostwriter on September 30, 2012 at 7:57 PM

He said that they know from experience and previous results how many people of each affiliation are out there – more or less, because it does shift – but it’s one of the less reliable demographics. If he gets a number that’s totally out of reality in either direction and he has the time, he’ll try to get some more interviews to even it out.

PolAgnostic on September 30, 2012 at 7:48 PM

Could I rephrase that to say, “if he gets a number that he doesn’t like, he’ll try to get some more interviews …”?

AZfederalist on September 30, 2012 at 8:01 PM

jpcpt03 on September 30, 2012 at 7:47 PM

Yep.

In an earlier thread on the same subject, I mentioned that when I was designing and running bolls back in the dark ages, that I was always watching my poll takers to make sure they weren’t leading for a particular answer by their tone of voice.

And during ‘in person’ interviews, subtle ‘body language’ can skew the results.

To complicate matters further, as in Physics, the mere observation of a subject may alter its state.

LegendHasIt on September 30, 2012 at 8:02 PM

Just received a phone call for a “random political poll” it rapidly became obvious that the poll was anything but unbiased and being conducted for the State Democrats. The main concern was that the voice inflections used by the poll taker would lead you into the answer they wanted. If the question asked was to lead you into a negative response for a certain question then the question asked ended in a lower pitched “negative tone”, if the desired response was meant to be positive then the interviewer ended the question in a higher pitched uplift tone. These action are very subtle but then these political parties have gotten very very good and manipulating are views.

jpcpt03 on September 30, 2012 at 7:47 PM

Could it have been a push poll?

ghostwriter on September 30, 2012 at 8:09 PM

Not that these polls shouldn’t be taken with a grain of salt due to their oversampling of Dems, but on the other hand we wouldn’t be questioning them, but rather taking them as Gospel if they showed Romney to be way ahead.

Dr. ZhivBlago on September 30, 2012 at 4:13 PM

No we wouldn’t, you f’n retard.

rayra on September 30, 2012 at 8:23 PM

isent RCP a republican leaning site? I dont thing they would allow themselves to be easily manipulated.

nathor on September 30, 2012 at 4:48 PM

That’s quite enough out of this trolling moron, too. RCP’s claim to fame is averaging all the garbage polls in an attempt to divine an accurate result. But it’s Garbage In, Garbage Out. Their ‘average’ doesn’t mean anything when 19 out of 20 polls are grossly skewed for Obama.

rayra on September 30, 2012 at 8:28 PM

That’s quite enough out of this trolling moron, too. RCP’s claim to fame is averaging all the garbage polls in an attempt to divine an accurate result. But it’s Garbage In, Garbage Out. Their ‘average’ doesn’t mean anything when 19 out of 20 polls are grossly skewed for Obama.

rayra on September 30, 2012 at 8:28 PM

If 19 out of 20 polls were “grossly skewed” in foavor of Obama, I would be concerned that they were showing a result I steadfastly hoped against. However, most of the polls on RealClearPolitics show Obama under 50 percent against Romney, and the race within a few points. The Realclear average does give a reasonable indication the the state of the race. Right now, it’s just showing that Romney has had a couple of tough weeks. It’s not a big deal unless the pattern persists…

ghostwriter on September 30, 2012 at 8:36 PM

Quality control is an obvious example. If you produce a widget, and wish to sample some finished items to ensure they are up to specifications, 5 or 10 percent should be more than sufficient, with a random sample.

It seems to me that you are not using the term statistically significant in a correct fashion. A small sample can provide statistically significant results if is representative of the larger population. I don’t need to flip a times coin a million times to identify that in the limit it the probability of getting heads will be close to 50 percent (with a true coin).

ghostwriter on September 30, 2012 at 7:57 PM

.
I an unsurprised at your choice of quality control for a couple of reasons.

Quality control is massively out of favor since it is not predictive because it’s attempts to use small precentages as indicative of future outcomes on the quality of a product (or in many cases, even the current product) has been proven by market results to be of little practical value. It is, however, the most common use of statistics laypeople have come up against in their lives.

Quality control is based on the same assumptions as polling and is, as a result, the one for which most people feel they have an “instinctive feel”.

The automotive industry is an excellent example of why if you “wish to sample some finished items to ensure they are up to specifications, 5 or 10 percent should be more than sufficient, with a random sample” you will find yourself with a warehouse full of unpaid for returned items and charges for production delays.

Anyone interested in succeeding in industry has been focused on quality assurance (which is not based on statistical sampling) for the last 20 years at a minimum.

For the casual reader, a lot of engineering science has been based on statistical sampling AFTER a sufficiently large body of reinforcing results have been obtained which validate one another with only limited variation from the mean and then the engineers will usually apply a factor fo safety of 2 – which would be equivalent to saying, “Our polling results are X and Y but you should only count on them being accurate to 1/2 of either X or Y.”

PolAgnostic on September 30, 2012 at 9:07 PM

Js, next time you talk to one of these pollsters, a few more questions for them:

How do they figure the people who refuse to be polled into the polls? Do they assume it’s just as many on each side of the issue who refuse? How do they make the determination?

Do the various pollsters coordinate their polls by looking at one anothers’ numbers before they release their poll results? I’m guessing they do and they’ll claim that they just want to make sure there aren’t any outliers, but this is how they would maintain credibility, by showing all the polls show something similar, so it would be fraudulent for them to do this.

I know they do that and it’s like the climate scientists’ explanation for their “hide the decline” technique, whereby they coelesce numerous data streams by arranging them like gramma’s bingo cards and rub and pad the outliers until all their polls look similar.

The interesting thing about the pollster community is that they don’t really compete:

Surely by this time somebody could have looked at the results of the various polling agencies and compared them to the final vote totals in the myriad races and determined who was hitting on all cylinders, right?

You see, you wouldn’t have to ask that question of any other sector because a business would be bragging up their brand and showing that their competitors suck, but these pollsters are all dependent on one another because their “science” wouldn’t survive if they didn’t conspire.

Buddahpundit on September 30, 2012 at 9:56 PM

I an unsurprised at your choice of quality control for a couple of reasons.

Quality control is massively out of favor since it is not predictive because it’s attempts to use small precentages as indicative of future outcomes on the quality of a product (or in many cases, even the current product) has been proven by market results to be of little practical value. It is, however, the most common use of statistics laypeople have come up against in their lives.

Quality control is based on the same assumptions as polling and is, as a result, the one for which most people feel they have an “instinctive feel”.

The automotive industry is an excellent example of why if you “wish to sample some finished items to ensure they are up to specifications, 5 or 10 percent should be more than sufficient, with a random sample” you will find yourself with a warehouse full of unpaid for returned items and charges for production delays.

Anyone interested in succeeding in industry has been focused on quality assurance (which is not based on statistical sampling) for the last 20 years at a minimum.

For the casual reader, a lot of engineering science has been based on statistical sampling AFTER a sufficiently large body of reinforcing results have been obtained which validate one another with only limited variation from the mean and then the engineers will usually apply a factor fo safety of 2 – which would be equivalent to saying, “Our polling results are X and Y but you should only count on them being accurate to 1/2 of either X or Y.”

PolAgnostic on September 30, 2012 at 9:07 PM

It seems to me that you are taking a specific case, and extrapolating it to the general. As it stands, it’s news to me that quality control is out of favor. At any rate, small sampling is likely sufficient for any number of fields, including soil sampling or pharmaceutical testing.

ghostwriter on September 30, 2012 at 10:28 PM

Surely by this time somebody could have looked at the results of the various polling agencies and compared them to the final vote totals in the myriad races and determined who was hitting on all cylinders, right?

It seems to me that this is not the correct question to be asking about the poll, unless it has explicitly been designed as a forecasting instrument.

ghostwriter on September 30, 2012 at 10:30 PM

soil sampling or pharmaceutical testing.

ghostwriter on September 30, 2012 at 10:28 PM

.
Seriously, stop pretending to knowledge you do not have.

Soil sampling is a prime example of where you can take a statistically large body of samples and STILL get reality completely at odds with thousands of samples which have been rigourously tested. Anyone who has worked in the associated fields has personal “Oops!” stories they have experieinced first hand.

Pharmaceutical companies have had production plants shut down because more rigorous analysis of of the data they had drawn, analyzed and submitted showed their products they were making did NOT meet regulatory requirements. One of particular note involved shutting down the majority of insulin production in this country due to failed statistical sampling providing inaccurate data.

PolAgnostic on October 1, 2012 at 4:08 AM

Any science in the hands of propagandists is propaganda

Jimbobby on October 1, 2012 at 5:39 AM

PR disguised as witchcraft. If many in the country are like me, for every polling call I get, I don’t have time for more than 1 in 10. If it starts with something pre-recorded, that’s an automatic hang-up. I don’t listen to or respond to any incoming recorded messages. Much like the Obama regime’s statistics, poll numbers are pulled out of their asses and massaged into stinking little Rodin statuettes.

ironked on October 1, 2012 at 8:55 AM

First and foremost, none of the polling right now is worth much. Past polling hasn’t coalesced to a point of accuracy (when it was accurate) until the final week before the election and it is not unusual for polls to swing significantly right before an election, usually in a direction away from the incumbent, when the economy is struggling. Second, the ever declining participation rate is a real problem for pollsters. They cannot determine if the 9% who participate are representative of the 91% who don’t. They simply assume that they are. Four years ago, the participation rate was double what it is now. Eight years ago, it was almost three times what it is now. And, considering the higher degree of mistrust that Romney supporters have right now in the media and in the polls, a reasonable assumption is that the 91% is populated with more Romney voters than with Obama voters. That said, there is simply no way of measuring the actual makeup and of the 91% or the representativeness of the 9%. Polling, as a science, depends on assumptions. Since these assumptions are sometimes wrong, the pollsters are periodically wrong and need to realign their assumptions in order to be closer to reality in the future. In my opinion, we are in uncharted territory when it come to polling accuracy for this election. The pollsters continue to put on a confident game face, in order to sell their product. But they understand the problems associated with the volatility that they are seeing in poll results. My guess is that the polls will begin to move toward Romney in the final days before the election, simply because he is not the incumbent during tough economic times. When you combine this entirely reasonable assumption with the likely oversampling of Obama supporters in the polls, the possibility of a Romney win, perhaps even a landslide, looks to be the more likely outcome in November. So, unless Obama can pull some type of historically unlikely rabbit out of his hat this November, he will most likely be a one-term President.

NuclearPhysicist on October 1, 2012 at 9:53 AM

Is polling science, art or witchcraft?

LOL! That’s an easy one to answer.

YES.

rspock on October 1, 2012 at 10:01 AM

Sample rate as a determination of outcome accuracy is widely variable based on numerous premises.

If you are testing results which are variable along an analog scale, then the smaller the sample rate, the less significant the results. If you are testing a binary set of options, then a smaller sample rate may still be adequate to project reasonable results, if all other variables can be accounted for.

They aren’t.

Refusals to participate in the poll is a variable which is never factored into results. The common suggestion is that conservatives are far more likely to hang up on, or walk away from, a pollster than are liberals. But if a person walks away, you can’t know of their affiliation. I agree that liberals are more likely to want their opinion known and shared, that conservatives are more likely to remain private.

Likely to vote results are likely based on lies. Again, some folks will want to claim that they will vote, just so that their opinion is recorded, even if they don’t really care. Which group is more likely to behave this way? One could guess, but it would be uncertain.

And this could go on and on. The point is, the variables are not accounted for, and this drives the deviation up beyond the claimed “margin of error”. It is only within the last few days that the pollsters seem to get “better” results, but that is also as likely because they’ve been holding their thumb to the scale up to then, but want to be remembered for being remotely accurate, so they true up their sample structure for the final polls.

When I see a Gallup result today that has Romney within 2%, I feel good.

Freelancer on October 1, 2012 at 12:16 PM

Seriously, stop pretending to knowledge you do not have.

Soil sampling is a prime example of where you can take a statistically large body of samples and STILL get reality completely at odds with thousands of samples which have been rigourously tested. Anyone who has worked in the associated fields has personal “Oops!” stories they have experieinced first hand.

Pharmaceutical companies have had production plants shut down because more rigorous analysis of of the data they had drawn, analyzed and submitted showed their products they were making did NOT meet regulatory requirements. One of particular note involved shutting down the majority of insulin production in this country due to failed statistical sampling providing inaccurate data.

PolAgnostic on October 1, 2012 at 4:08 AM

Really. I could say the same about you. I’ve been giving you the benefit out the doubt, waiting for you to make a positive contribution to this discussion. Try substantiating your claim that ““A 9% sample rate is statistically insignificant and NO reliable estimate can be based on such a small number.”

ghostwriter on October 1, 2012 at 11:24 PM

Sample rate as a determination of outcome accuracy is widely variable based on numerous premises.

If you are testing results which are variable along an analog scale, then the smaller the sample rate, the less significant the results. If you are testing a binary set of options, then a smaller sample rate may still be adequate to project reasonable results, if all other variables can be accounted for.

They aren’t.

Refusals to participate in the poll is a variable which is never factored into results. The common suggestion is that conservatives are far more likely to hang up on, or walk away from, a pollster than are liberals. But if a person walks away, you can’t know of their affiliation. I agree that liberals are more likely to want their opinion known and shared, that conservatives are more likely to remain private.

Likely to vote results are likely based on lies. Again, some folks will want to claim that they will vote, just so that their opinion is recorded, even if they don’t really care. Which group is more likely to behave this way? One could guess, but it would be uncertain.

And this could go on and on. The point is, the variables are not accounted for, and this drives the deviation up beyond the claimed “margin of error”. It is only within the last few days that the pollsters seem to get “better” results, but that is also as likely because they’ve been holding their thumb to the scale up to then, but want to be remembered for being remotely accurate, so they true up their sample structure for the final polls.

When I see a Gallup result today that has Romney within 2%, I feel good.

Freelancer on October 1, 2012 at 12:16 PM

These issues don’t speak to sample size. The standard errors for most of these samples are reasonable (generally about 3 percent). This suggests that the sample is large enough. I don’t think that anybody is arguing that the sample sizes are too small. However, as many on this site have noted, a number of the samples may not be representative of the population at large, oversampling Democrats. In other words, it is possible for a sample to obtain a very precise measure of a nonrepresentative sample, and thus, wildly miss the mark, even though the estimates are statistically efficient.

The classic example of this phenomenon is the “Literary Digest” poll of 1936 that predicted victory for Alf Landon before he got crushed by FDR in a landslide. Literary Digest had an enormous sample size, but because they conducted the poll in the mail, they essentially self-selected a nonrepresentative subset of the population.

ghostwriter on October 1, 2012 at 11:35 PM

Comment pages: 1 2