Polling science, art and witchcraft Pt. 2
posted at 10:01 am on November 10, 2012 by Jazz Shaw
More than a month ago, I undertook some digging based on the assumption that this election, win or lose – though “win” would have been much better – could produce some significant data in terms of polling. I asked the question, “Is polling science, art or witchcraft?” At the close of that piece I was left feeling that, once this was all over, somebody would be right and somebody would be unemployed. We now know the unfortunate values which were plugged into that formula.
During the day after the bloodbath, both Ed and AP posted some compelling pieces on the subject of precisely how so many of us got it so wrong. (Both of those linked items are very much worth a read, by the way, in case you managed to miss them the first time.) I agree with some parts of the analysis offered, but not with others. More on that shortly.
The one thing that is no longer in question is that we need to stop trying to read sinister motives into the professional pollsters and poll analysts and see what might be gained from the product that they’re putting out there. And since we’re on the subject of bad news, it’s also worth asking.. which pollsters were the most accurate, though most of them did pretty well? It was PPP. Gallup and Rasmussen – who we quote here frequently – came in tied for 24th. And really, people who get paid to conduct or analyze polls have little motive to fudge the numbers to try to “help” a candidate or party since they would be unemployed the following season for missing the target. So with that unpleasant tablespoon of cod liver oil safely down the hatch, we continue.
So what can we do with this data going forward which can help us? Here’s one place where Ed and I will disagree to some extent. He focused a fair bit of his analysis on the premise that our major error was assuming that the model for 2012 was more similar to 2010 than 2008, so we got caught with our pants down. I think that’s not the answer, and in fact is part of the problem. The model this year was neither 2010 nor 2008. It was 2012. The model is not stagnant. It shifts all of the time. Predicting human behavior is hard enough to begin with, but people shift based on conditions and their own personal experiences and evolution as they age. Nothing stays the same. As far too many generals have told us, you are doomed to lose if you keep fighting the last war.
The most consistent answer pollsters gave me when I asked about the dreaded “D+9” sample – which Ed once quipped as being in “an alternate universe” – was that they didn’t pay much attention to the split. It wasn’t a significant cross tab. They were far more likely to weight a poll if odd numbers showed up in gender or age groups. Why? They were listening to the answers they got this year, not from two or four years ago. These numbers can be of value in determining where we need to focus the most resources rather than screaming about the sample rate and clapping all the louder for Tinkerbell.
And how can that focus be achieved? Just as one humble suggestion I have a thought. This cycle proved that there is a lot of money out there for conservative political activism, but clearly dumping it all into an endless series of TV ads doesn’t get the job done. If such resources could be channeled into better analytics and a wider, more targeted ground game, we could catch up in the significant areas where Team Obama took our lunch money. While keeping state laws closely in mind, activists from safe states could have their travel and expense costs covered to go into battleground areas and get the door to door work done.
Good analysts (yes… people in the same line of work as Nate Silver) could be found and employed to pour through the demographic data and target voters to contact. Issue items could be weighed to make sure that a positive, pro-growth conservative message was being brought effectively to the people we most need to reach. This would be a major shift in intelligent use of large, monetary resources. I’d be willing to bet that more than a few of you – if the travel, meals, hotel costs, etc. were covered – would be willing to take some time out in the closing month of the election to get out with a targeted list of potential voters and hit the doors, the local watering holes and such. And that’s an effective way to get some face time to combat the barrage of media spin.
Just a few more things to chew over as we pick up the pieces. Your suggestions, as always, are welcome.