CJR: Harris poll on GOP extremism not credible

posted at 2:55 pm on March 30, 2010 by Ed Morrissey

The Columbia Journalism Review isn’t known for its stirring defenses of conservatives, so their analysis of the Harris poll that suggested that 45% of Republicans are Birthers and 24% believe Obama to be the Antichrist is remarkable for its conclusion.  Greg Marx pegs his analysis on a deconstruction by ABC News polling director Gary Langer and wonders why the national media was so credulous in reporting a poll obviously aimed to help author John Avlon sell his book:

But in order both to be credible and to be perceived as credible, that undertaking has to come from a place of open inquiry, not from an expectation that the results will boost book sales, create a news cycle, or confirm a political narrative. The whole appeal of polling is its promise (sometimes oversold) to produce objective, almost scientific data that tells us something about the state of our politics. When that enterprise itself becomes an exercise in political gamesmanship, it may foster cynicism about the utility and quality of any effort to collect information. If any given piece of news is valuable only insofar as it advances a particular political view, then they are all in a sense equivalent, which is to say worthless.

That’s not to say that political perspectives don’t have a place in journalism (they do) or that our political views won’t shape the way we understand information (they will). But we still need to protect a place for facts and evidence in our political debate, and to do that we need to push back against rhetorical opportunism and statistical sloppiness on all sides. Journalists can do this by not cutting corners when reporting and compiling data—but also by making the case for why methodological rigor is valuable, and by providing readers with the tools to evaluate information themselves.

This, in the end, is the real value in Langer’s post—it explains why the Harris poll is flawed in a way that will hopefully prompt closer scrutiny of storyline it perpetuated. While this type of direct engagement between media outlets is somewhat unusual, Langer has a clear incentive here: as a pollster, he needs to uphold the integrity, credibility, and rigor of his profession. It’s a lesson for other members of the press to keep in mind.

Langer ripped Harris for its methodology, starting with its interactive model based on incentivizing through cash and gifts, but that was just to start:

The problems are fundamental. “Some people have said” is a biasing introductory phrase; it imbues the subsequent statements with an air of credibility – particularly when you don’t note that others say something else. (That approach can have problems of its own; the “some people” vs. “other people” format implies equivalence.)

The subsequent statements, for their part, are classically unbalanced – there’s no alternative proposition to consider. A wealth of academic literature, neatly summarized here, demonstrates that questions constructed in this fashion – true/false, agree/disagree – carry a heavy dose of what’s known as acquiescence bias. They overstate agreement with whatever’s been posited, often by a very substantial margin. (This reflects avoidance of cognitive burden, which tends to happen disproportionately with less-educated respondents, as is reflected in Harris’ results.)

Using all negative statements, rather than a mix of negative and positive ones, reflects another non-standard approach, one that can further bias responses. (The ordering of  items, unclear in the Harris release, can be troublesome as well.)

Another problem, which I discuss here, is the challenge of over-literalism in evaluating survey results of this type. Rather than answering disparaging poll questions literally, people who are ill-disposed toward the subject may simply use these questions as an opportunity to express their general antipathy – not as a thought-out endorsement of the specific posit. And the use of hot-button invective is ill-advised in its own right; respondents may just blow it back.

Langer also compares the result to more traditional polls taken by Pew during and just after the election by linking to an observation by Time’s Michael Scherer.  Instead of 57% of Republicans believing that Obama is a secret Muslim, Pew found only 17% of Republicans believing that, clearly a fringe view in the GOP even when those sentiments burned most hotly.

The big question here isn’t that Harris did a push poll, or that Avlon exploited it to sell his books.  Push polls have been around for a very long time.  CJR’s point is that the media apparently fell for it, spreading misinformation, rather than doing some routine fact-checking to determine the poll’s credibility.  I’d say that indicates another age-old trend of accepting data points without confirmation when they fit an accepted worldview — and in the national media, a poll declaring Republicans to mainly be lunatics would certainly fit that explanation.

Update: Kathy Shaidle has more thoughts.


Related Posts:

Breaking on Hot Air

Blowback

Note from Hot Air management: This section is for comments from Hot Air's community of registered readers. Please don't assume that Hot Air management agrees with or otherwise endorses any particular comment just because we let it stand. A reminder: Anyone who fails to comply with our terms of use may lose their posting privilege.

Trackbacks/Pings

Trackback URL

Comments

Watches for the ban-hammer.

upinak on March 30, 2010 at 2:58 PM

The Left is so freaked out by the Grass Roots Astroturf Tea Party Rebellion that it has to make up lies to keep the “Usefull Idiots” in line.

Puts a Big Ol’ Smile on my face:)

SayNo2-O on March 30, 2010 at 2:59 PM

Avlon is just as despicable as Frum.

Mark1971 on March 30, 2010 at 2:59 PM

“Langer has a clear incentive here: as a pollster, he needs to uphold the integrity, credibility, and rigor of his profession. It’s a lesson for other members of the press to keep in mind.”

BBBBBBWWWWWWWWWWWWAAAAAAAAAAAAHAHAHAHAHAHAHAH…

(chortle, snort, choke, cough, gasp)

… Stop it Ed, you’re killing me!

Seven Percent Solution on March 30, 2010 at 3:00 PM

The left are in panic mode as they didn’t really take into account that Americans (unlike other countries) was not going to take “a fundamental transformation of America) lightly…

Push back is a b!tch.

katy on March 30, 2010 at 3:04 PM

JR’s point is that the media apparently fell for it,

If you believe they “fell” for it, you need to think clear about what we are up against. They didn’t care if it was true, but the firm provided enough cover that they could use the poll and not have to worry about repercussions when it turned out to be false. That isn’t falling for anything, its journalism done with a slant and just enough CYA so they could sleep at night.

rob verdi on March 30, 2010 at 3:04 PM

I would say that 70% of Democrats are socialists and 60% think Bush was behind the attacks on 9/11. No doubt I could come up with a poll that said that.

Terrye on March 30, 2010 at 3:08 PM

As an experiment, how about a poll here asking whether it was a collision with Obama that sank the Titanic? I bet we can get above 50% on that.

year_of_the_dingo on March 30, 2010 at 3:10 PM

Liars liars with pants afires.

disa on March 30, 2010 at 3:10 PM

I find no coincidence that the “right-wing extremists” were arrested in Indiana/Ohio/Michigan within days after the “alleged” threatening calls to DEM pols.

Furthermore, as disgustingly pathetic as it is, I find no coincidence that the demon beast of priest-child sex abuse has made its way back to the headlines. I guess the Pope should be fully discredited since the Vatican issued a statement condeming Obamacare. Doesn’t help that the Church has been complicit for decades. Timely nonetheless.

I’m a non-practicing Catholic so don’t flame me!

Oink on March 30, 2010 at 3:12 PM

This has become an all too familiar theme with the MSM — grab onto any negative about the GOP, Republicans, Tea Partiers, or conservatives and run with it. Then, once the lie/exaggeration is exposed, bury the apology (if even given) where no one will see it.

Where do these people think they are going to live after they have helped cause the demise of this country? I truly don’t get it.

GnuBreed on March 30, 2010 at 3:13 PM

You cannot simultaneously embrace and reject journalism, journalists and their respective institutions.

You are a joke.

The Race Card on March 30, 2010 at 3:16 PM

Push back is a b!tch.

I believe the favorite lefty phrase for this is BLOWBACK.

Well, they are now getting and will continue to get BLOWBACK from their interventionist policies designed to interfere, coerce and alter the lives of average people.

Poor little babies. They sure do scream and whine like babies, don’t they? When they’re not lying like children caught with their hands in the cookie jar, that is.

Good Lt on March 30, 2010 at 3:19 PM

What makes you so certain that the media “fell for it” and that they were not part of it from the beginning?

I honestly can no longer see any daylight between what is regurgitated from our major media outlets, and what was put forth by ITAR-TASS in the bad old days of the USSR.

NavyspyII on March 30, 2010 at 3:20 PM

I can’t believe CJR gave this bogus poll such a brutal take down, but, really, it’s far too late for the legacy media to save itself.

Rational Thought on March 30, 2010 at 3:21 PM

I would say that 70% of Democrats are socialists and 60% think Bush was behind the attacks on 9/11. No doubt I could come up with a poll that said that.

There was in fact a Rasmussen poll suggesting that 35% of Democrats thought Bush knew about the 9-11 attacks in advance.

Good Lt on March 30, 2010 at 3:23 PM

Really Ed? Is it as bogus as the NEMJ poll you were touting not too long ago.

lexhamfox on March 30, 2010 at 3:24 PM

…57% of Republicans believing that Obama is a secret Muslim…

I don’t believe he is a secret muslim; I believe he is raving muslim.

behiker on March 30, 2010 at 3:24 PM

OT: The Boss Emeritus linked to this post today. Treat yourself because you deserve.

http://pajamasmedia.com/zombie/2010/03/29/searchlight-vs-l-a-rival-rallies-reveal-stark-rightleft-divide/?singlepage=true

BuckeyeSam on March 30, 2010 at 3:27 PM

The media “fell for it”?

Ed, you really don’t know what’s going on.

faraway on March 30, 2010 at 3:28 PM

Why no pic with this thread?

jwolf on March 30, 2010 at 3:29 PM

Really Ed? Is it as bogus as the NEMJ poll you were touting not too long ago.

lexhamfox on March 30, 2010 at 3:24 PM

Looks like Ed published a correction/clarification. It’s something the State Media folks should learn how to do.

forest on March 30, 2010 at 3:30 PM

See the link above for examples of violent lefty rhetoric. This should be linked on the front page of every conservative website on the Net. We need to spread this around liberally (pun intended).

College Prof on March 30, 2010 at 3:33 PM

http://townhall.com/columnists/JohnHawkins/2010/03/30/violent_liberal_hate_rhetoric_fifteen_quotes?page=2

Well, the link didn’t appear, but the URL is above. Sorry about that.

College Prof on March 30, 2010 at 3:33 PM

I would say that 70% of Democrats are socialists and 60% think Bush was behind the attacks on 9/11. No doubt I could come up with a poll that said that.

There was in fact a Rasmussen poll suggesting that 35% of Democrats thought Bush knew about the 9-11 attacks in advance.

Good Lt on March 30, 2010 at 3:23 PM

See? What did I tell ya?

Terrye on March 30, 2010 at 3:36 PM

Jon Avlon claims he’s an independent. Yet his whole schtick is slamming conservatives. It gets him invites to Tina Brown’s dinner parties.

PS – He’s a homo.

kevinkristy on March 30, 2010 at 3:37 PM

You cannot simultaneously embrace and reject journalism, journalists and their respective institutions.

You are a joke.

The Race Card on March 30, 2010 at 3:16 PM

I could say that works both ways.

Terrye on March 30, 2010 at 3:39 PM

You cannot simultaneously embrace and reject journalism, journalists and their respective institutions

One must accept all journalists and institutions or reject all journalists and institutions? No middle ground?

There are good journalists and bad journalists just like there are good plumbers and bad plumbers. And even good journalists, like good plumbers, can do bad work.

There’s no single standard, it seems to me.

SteveMG on March 30, 2010 at 3:39 PM

The left are in panic mode as they didn’t really take into account that Americans (unlike other countries) was not going to take “a fundamental transformation of America) lightly…

Push back is a b!tch.

katy on March 30, 2010 at 3:04 PM

I agree with this idea entirely. The success of the progressive socialist agenda in Europe is used as the model for what they are attempting here in the United States. But Europe has a long history of Kings and dictators that have shaped their cultural outlook. This cultural mindset of Europeans leads them to be accepting of social policy driven from the top-down.

America has no history with tyrants other than to rebel against them. The American cultural identity is to create social policy from the bottom up. We look for popular consensus to bring about transformational social change. That is why the counter-culture of the 1960s and 1970s was successful in changing our nation. Enough people embraced a change in outlook to adopt it as valid social philosophy.

The problem that our politicians have is that the 1960s-1970s counter-culture is dead. Most of the hippies have grown up enough to realize it was all bunk. Those that embrace that philosophy are no longer a dominant part of society but they cannot admit it. So they continue to press in a last ditch effort to change our nation before they fade into obscurity.

Make no mistake on this. The efforts we see by the political left are their death throes. That is why they are so desperate. They know it too and have to get their policies entrenched before they have no hope. What they do not realize, is that transformational change can mean going back to our traditional American values as well. All of their damage can be undone.

Hawthorne on March 30, 2010 at 3:41 PM

I believe he is raving muslim.

Rest assured, he’s not.

Thanks.

Good Lt on March 30, 2010 at 3:41 PM

As the ABC pollster pointed out, there are good polls and there are bad polls (and everything in between). Poorly written one and well written ones. Ones with poor questions and ones with careful questions.

To say we have to accept all journalistic polls is as short-sighted as saying we have to reject all polls.

I’m not sure what the point is?

SteveMG on March 30, 2010 at 3:42 PM

CJR’s point is that the media apparently fell for it, spreading misinformation, rather than doing some routine fact-checking to determine the poll’s credibility.

They fell for it simple because they wanted to believe it was true.

Or as noted Democrat philosopher George Costanza once said:

“A lie isn’t a lie if you believe it.”

Del Dolemonte on March 30, 2010 at 3:43 PM

Even if Obama was a Muslim (and I don’t think he is), who cares?

It’s his policies I (mostly) oppose, not his religion (or the man).

SteveMG on March 30, 2010 at 3:44 PM

You cannot simultaneously embrace and reject journalism, journalists and their respective institutions.

You are a joke.

The Race Card on March 30, 2010 at 3:16 PM

Harris Polling is a polling firm, they’re not journalists.

Of course nowadays most journalists aren’t journalists either.

Del Dolemonte on March 30, 2010 at 3:46 PM

Push back

This kind of overheated, violent rhetoric, so typical of rightwingnuts, is what’s wrong with politics today and has no place on this blog!

/sarc

Insomniac on March 30, 2010 at 3:47 PM

Even if Obama was a Muslim (and I don’t think he is), who cares?

It’s his policies I (mostly) oppose, not his religion (or the man).

SteveMG on March 30, 2010 at 3:44 PM

One of our resident bridge-dwellers made the mistake of bringing that up last week here. When gently reminded that racism had nothing to do with health care reform’s defeat in the Clinton era, said bridge dweller was suddenly at a loss for words.

Del Dolemonte on March 30, 2010 at 3:48 PM

So to begin a poll by saying something like, <blockquote>”Some people say that just because our president is a member of the American Communist Party and Joe Biden has been pronounced clinically brain-dead for the last 22 years, that those things don’t affect their job performance,” that would be wrong?

av8tr on March 30, 2010 at 3:50 PM

I called this immediately after the poll data became available. It was a Harris-Interactive poll, and the respondents were H-I members…online. And the questioning wasn’t available. And Avlon hyped the results before they were even released. All of this combined made the poll a farce in my view. Even though we all instinctively knew it to be so, the media will put aside instincts in an effort to kiss Barry’s a$$. CJR doesn’t have the influence to change that fact.

BKeyser on March 30, 2010 at 3:54 PM

Whatev.

HornetSting on March 30, 2010 at 4:01 PM

I’d say that indicates another age-old trend of accepting data points without confirmation when they fit an accepted worldview

Hence the prevalence of Rasmussen polls in HA posts.

Grow Fins on March 30, 2010 at 4:12 PM

Thanks for the link, Ed!

fivefeetoffury on March 30, 2010 at 4:17 PM

Hence the prevalence of Rasmussen polls in HA posts

Heaven forbid an opinion blog would cite a polling firm – shown historically to be accurate – that supports those opinions.

SteveMG on March 30, 2010 at 4:44 PM

Awww, people are calling me names again! I’m so hurt!

Oh, wait. Why, I have male gentitalia! Name-calling doesn’t bother me!

Nevermind.

Doorgunner on March 30, 2010 at 4:55 PM

I’d say that indicates another age-old trend of accepting data points without confirmation when they fit an accepted worldview

Hence the prevalence of Rasmussen polls in HA posts.

Grow Fins on March 30, 2010 at 4:12 PM

If you can provide credible evidence that Rasmussen skews its polling samples to arrive at a pre-determined conclusion, please do so.

Or as your cool dude from Kenya says, “bring it on”.

Del Dolemonte on March 30, 2010 at 4:59 PM

the Harris poll that suggested that 45% of Republicans are Birthers and 24% believe Obama to be the Antichrist

I believe that Barry Soreto’s Constitutional qualifications to be eligible to occupy the American Presidency are highly questionable to say the least and his having so much in the way of his records locked up in some kind of Full Body Burka just adds to it.

As to his being the Antichrist, I see him as the AntiJefferson and the AntiWashington and the AntianitGeorgeIII.

His circle of friends and advisers all suck too.

MB4 on March 30, 2010 at 5:03 PM

Journalists can do this by not cutting corners when reporting and compiling data—but also by making the case for why methodological rigor is valuable, and by providing readers with the tools to evaluate information themselves.

As far as I am aware, it doesn’t take any kind of special training to get a job writing articles for a publication, like college or an apprenticeship (although I am sure it might help, as well as who you know).
It’s a job that doesn’t pay very well & this quote above indicates to me that the writer believes that journalists are some sort of elite group held to a higher standard than say someone who blogs news on the internet.
From the quote, I would almost say this person is attributing scientific skills to journalists, which the majority clearly do not have.
The news czar should make it mandatory for all practicing journalists to become experts in the fields that they are reporting in, just like they have mandated teachers have a major in the subject that they are teaching.
After all, it is journalists who are ‘teaching’ AKA informing us about/of events.
Let’s call it No News Reader Left Behind or NNRLB.
And their performance will be evaluated by a series of tests in which the reader must complete.
If they do not pass, that journalist will be put on probation & if there is no improvement at the end of the year, the govt will replace the journalist with a beuracrat.

Badger40 on March 30, 2010 at 5:04 PM

Greg Marx pegs his analysis on a deconstruction by ABC News polling director Gary Langer and wonders why the national media was so credulous in reporting a poll obviously aimed to help author John Avlon sell his book

Why? Because most of those in the national media have a special fondness and a compassion for idiots on account of personal experience and heredity. That’s why.

MB4 on March 30, 2010 at 5:07 PM

I’d say that indicates another age-old trend of accepting data points without confirmation when they fit an accepted worldview — and in the national media, a poll declaring Republicans to mainly be lunatics would certainly fit that explanation.

Ed,

Perhaps instead of ‘an accepted worldview’ you meant to say ‘a narrative’.

Or am I reading you wrong?

Freddy on March 30, 2010 at 5:07 PM

I’d say that indicates another age-old trend of accepting data points

What is a “data point” as opposed to “data” in a poll? I have no idea.

MB4 on March 30, 2010 at 5:14 PM

MB4 on March 30, 2010 at 5:14 PM

Hey! Don’t you know? It’s an age old thing!

Badger40 on March 30, 2010 at 5:24 PM

The problems are fundamental. “Some people have said” is a biasing introductory phrase; it imbues the subsequent statements with an air of credibility – particularly when you don’t note that others say something else. (That approach can have problems of its own; the “some people” vs. “other people” format implies equivalence.)

Matt Lauer tried this trick on Laura on the today show. We need 100 Lauras to start taking back our “news prgrams”

r keller on March 30, 2010 at 5:51 PM

Really Ed? Is it as bogus as the NEMJ poll you were touting not too long ago.

lexhamfox on March 30, 2010 at 3:24 PM

Looks like Ed published a correction/clarification. It’s something the State Media folks should learn how to do.

forest on March 30, 2010 at 3:30 PM

Ed merely pointed out that the poll was done by a firm and not the NEJM. Ed did not say what that firm did or why that poll would be for a commercial purpose (just like this poll). The poll was done by a recruitment firm which wants more business so they conduct a poll which say… hey you are going to need more doctors and care givers.

lexhamfox on March 31, 2010 at 12:07 AM