See if you can find the logical fallacy in this opening from the Washington Post about a new study on Internet searches for sexually-related material:
Specifically, Cara C. MacInnis and Gordon Hodson of Brock University found that residents of more religious and more politically conservative states — often in the South — are more likely to Google things like ‘‘sex,’’ ‘‘gay sex,’’ ‘‘porn,’’ ‘‘xxx’’, ‘‘free porn,’’ and ‘‘gay porn” than their peers in more secular states. The study, published this month in the journal Archives of Sexual Behavior, analyzed state-level Google Trends data for 2011 and 2012, and combined it with measures of religiosity and political conservatism from Gallup surveys.
“Overall,” the authors say, “a reliable positive association of moderate-to-large association size exists between state-level religiosity and searches for the term ‘sex.'” They observed similar patterns for Google image searches for sex with political conservatism.
This might be convincing … if politics and religion were the only environmental differences that exist in the United States. One could make the same argument about hominy grits, warm weather, or drawls. Did you know that a tendency to speak in a Southern accent has a reliable positive association between state-level religiosity? And for Google searches about sex?
In the Washington Post’s defense, they appear to be reporting fairly on the study itself based on its abstract, if not with any reasonable portion of journalistic skepticism. The abstract proposes, “These findings were interpreted in terms of the paradoxical hypothesis that a greater preponderance of right-leaning ideologies is associated with greater preoccupation with sexual content in private internet activity.” Except that the researchers didn’t do the hard work of actually identifying and studying those specific populations of “right-leaning ideologies”; that would require time and effort in crafting a study of several populations, along with control groups, and then perhaps reaching some conclusions that actually show real correlation and perhaps causation.
Instead, they based their studies on entire states’ Google trends without any control over which populations did the searching. There’s nothing in the abstract or the Post’s recap, for instance, that even posits that religious and/or political conservatives use the Internet overall at the same rate as other populations, or more or less so. There is no data presented at all that assigns that traffic to specific subgroups; the authors just assumed that the controlling factor had to be “right-leaning ideologies” without ever establishing that as a fact, or even a data-supported hypothesis.
One has to read all the way down to the penultimate paragraph to get this disclaimer:
It’s important to note the study’s claims don’t necessarily hold true at the individual level. “Aggregate data should not be extrapolated to individual behavior, particularly in the field of sex research, where sensitive topics are often examined,” MacInnis and Hodson write.
That’s actually the problem with this study in a nutshell, and an astoundingly hypocritical statement. Aggregate data, in this case, shouldn’t have been used at all. If the study’s authors wanted to make a claim about religious and political conservatives, then they should have done a study on that population group, rather than just assume that a state’s general political tendencies allows for a correlation between specific kinds of Internet usage and a subset of its population.
That didn’t keep them from actually making those value judgments, however:
The authors look at two different groups: political conservatives and religious conservatives. Political conservatism was associated specifically with an increased interest in sexually explicit images.
On the other hand, religious conservatives– measured by the share of residents who say religion is important in their lives — are highly correlated with searching for sex online, but not necessarily “non-traditional” sex. The study notes: “It may be that these ‘sex’ searches were conducted with the intention of delivering ‘traditional’ sexual content (e.g., information regarding monogamous, married, heterosexual sex).”
According to the abstract, at least, they didn’t take a look at any groups. Their abstract is pretty specific: “We examined associations between state-level religiosity/conservatism and anonymized interest in searching for sexual content online using Google Trends (which calculates within-state search volumes for search terms).” All they looked at was state-level data and made their own assumptions about the causative impact of conservatism and religion. That may be described in several ways, but scientific isn’t one of them.