A brief note on polling and unemployment data analysis

posted at 1:51 pm on October 5, 2012 by Ed Morrissey

I’m noticing a couple of odd reactions to today’s jobless data and the analysis it has produced so far today.  On one hand, we have people proclaiming the difference between the household and establishment data as proof of a conspiracy to re-elect Barack Obama at the BLS.  On the other, we have a number of voices calling anyone who applies any kind of critical analysis to these figures as “truthers” or “deniers.” Tom Elia has collected some of the latter at The New Editor, but I’ll focus on explaining what exactly the figures are, and why critical analysis does not amount to “trutherism.”

The BLS conducts two surveys each month to determine employment data.  The first is the establishment survey, which polls 410,000 businesses each month.  The second is the household survey, which polls 60,000 households each month.  As one might expect, the larger survey provides a more stable series and more reliable data.  The smaller one is still a very significant sample, but in a nation of around 150 million households, it’s hardly an exact science.

Any poll series can produce an outlier result.  That’s true even of larger sample surveys, even when the sample is properly balanced, and even without malicious intent to tweak the results.  It’s more likely to happen with smaller samples than larger samples, but can happen any time in surveys.  That’s why it’s important to look at trending more than a single result within polling series, although some applications (jobless rates, elections) are intended more for single-result reporting.

In today’s case, the establishment survey showed a result that corresponds closely to other economic trends and that doesn’t deviate much from the intraseries trend.  The household survey, from which the jobless rate is derived, showed a very large deviation from its own trending and from the growth data in the economy.  The last time we had that many added in the household survey, the GDP growth rate was around 9%, and it’s currently 1.5%.

That’s why people who understand data and surveys look skeptically at the result of the household survey.  It doesn’t mean a conspiracy is in place; it does strongly suggest that this month’s sample of 60,000 households threw an outlier, especially when compared with the establishment survey and other economic data.  If so, it will likely correct itself in the next report.  That’s not “trutherism” or denial, but straightforward data analysis.

Update: Just to remind everyone, the next report comes out before the election — the Friday before, actually.  And if the BLS wanted to cook the numbers, I’m pretty sure they’d have cooked the establishment survey, too, to show more than +114K.

Breaking on Hot Air



Trackback URL


Need more coffee…

claudius on October 6, 2012 at 11:39 AM

Here is an article from none other than the NYT, going on about the anomaly in the numbers for 20-24 year olds. What jumps out at me is the steady, very rapid decrease in the number of 20-24 year olds leaving the labor market according to the chart from 1948-2012.

The last time there was a change like that it was during the boom times from around 1995-2001, which I remember well. The problem is that 2009-2012 have not been boom times at all.


claudius on October 6, 2012 at 11:35 AM

It wouldn’t surprise me if that cohort wouldn’t go back to grad school for a couple of years to wait out the bad economy. Grad school has got to be a much more preferable option to living in your childhood bedroom.

ghostwriter on October 6, 2012 at 11:47 AM

ghostwriter on October 6, 2012 at 11:47 AM

It’s possible, I suppose that a bunch of 20-24 year olds are actually employed rather than in school. They are the group that is most likely to work cheap and part time, and it is possible that their parents are unemployed or underwater. Still from 1948 the only similar period is during the late 90s boom, a completely opposite economic era.

We see some similar spurts, but they are all shorter and less dramatic statistically. I’d like to see someone that knows what they’re doing take this on and correlate it with other economic and social trends.

claudius on October 6, 2012 at 12:20 PM

Here is a more detailed account of the employment situation, that may explain some things, although it still does deal with my 20-24 conundrum. Still 584,000 part time workers —they would have had to have been largely in that age group.


I still think it’s some smelly BS.

claudius on October 6, 2012 at 1:03 PM

Steve Eggleston on October 5, 2012 at 10:50 PM

Thank you!

ITguy on October 7, 2012 at 4:23 AM

And Steve, ignore the trolls. Ad Hominem attacks are all they have. They can’t attack the argument, so they attack the man.

ITguy on October 7, 2012 at 4:36 AM