Numerous outlets, from FireDogLake to MSNBC’s Rachel Maddow and Dylan Ratigan, ran with the results of a poll that essentially “proved” that Republicans are extremists, many of whom think Obama is like Hitler and wasn’t born in this country. The poll was mentioned (with no methodology) in a Daily Beast column by John Avlon (“Scary New GOP Poll”), who is pitching a book about the extremism of the right. You’re welcome to seek out the book, but I’ll only mention the subtitle: “How the Lunatic Fringe is Hijacking America.”
I contacted Harris to find the methodology and the specific data to check whether the firm cut corners to achieve a desired result. Alyssa Hall, a spokesperson for the company informed me:
“The author of a book who contributed to our release was granted the exclusive to certain data points. I’m sure you are aware that exclusives are not uncommon. The full release with all data will be out tomorrow and you can see from the data whether republicans are extremists.”
I asked a spokesperson for Harris if the author, the author’s publication, or the author’s publisher paid for the poll, and the reply was no.
Exclusives are one thing, but poisoning a news cycle with faulty data is another. MSNBC jumped on the poll and hosts David Shuster, Dylan Ratigan, and Rachel Maddow each used the poll as a jumping off point for lamenting the faulty logic of the opposition party, without checking the source whatsoever.
As ABC News’s Gary Langer writes, the poll uses questionable methodology to achieve a desired result:
It nails the negativity, all right; this project purports to tote up responses to a list of harsh criticisms of the president – e.g., that he’s “anti-American,” “a racist,” “wants… an excuse to take dictatorial powers,” “is doing many of the things that Hitler did” and “may be the Antichrist.”
Hot words, those. The survey, done by Harris Interactive, apparently was designed to test the theories in a book claiming the “lunatic fringe is hijacking America.” The purpose seems to have been to see how many people the pollsters could get to agree to pejorative statements about Obama. Quite a few, it turns out – but with what I see as a highly manipulative approach to questionnaire design.
Among the problems with the poll:
Sampling: The poll was done among those who sign up for questionairres in exchange for cash and gifts. An extensive Stanford study found that such surveys were less accurate than truly random polls.
Phrasing: Beginning questions with “some people have said” biases the respondent because, as Langer points out, the phrase imbues the statement with an air of credibility.
Imbalance: With a lack of alternative propositions to consider, the respondent is pigeonholed into a particular response, despite a likelihood of not agreeing with the statement fully.
Difficulty of measuring the topic: The agree/disagree approach to asking questions is hardly a way of measuring the nuances of someone’s antipathy toward the president.
Republican pollster David Winston, in casual conversation yesterday, put it well. “It doesn’t seem serious at all.”
Harris’s Regina Corso defended the poll saying that she and Langer could spend hours debating the positives and negatives of their methodology. “We do all our polling online,” she said. Asked about the Stanford study, she said they disagree with them too. “We’ve argued with them about it.”
Seems like if Harris gets into that many scraps about its methodology and then receives partisan support for their polling, they might want to reconsider it.