Read the Study First, For the Nth Time
There’s a story getting a ton of play on Facebook about how one-third of men say they’re willing to force a woman to have sex against her will. It’s horrible, it’s attention-grabbing, and it plays into peoples’ worst fears about men (and people in general). And at least unlike some of the “studies” that get press coverage, it’s at least a paper in a peer-reviewed journal with an acceptable sample size (86). It has the usual problem of convenience sampling (e,g,, all white college males who received college credit for answering). The issue goes deeper than that; this is a horribly flawed paper with a research design that verges on deceptive.
The study is set up to maximize the number of people who say they would commit rape. The authors use questions from a few different scales to assess whether the respondents are inclined to commit sexual violence or violence in general. After spending the survey making the recipients answer questions about violence and sexual violence, they then ask the recipients about their willingness to engage in sexual violence. The problem is that by asking all these questions about violence before asking the key question about violence, they are likely to activate violent tendencies in their respondents. This is called “priming”, and can be a real problem. Survey responses are volatile, and even small primes can affect them. Conscientious researchers should have put the key question first – or even better, used treatment and control groups where they didn’t ask the preceding questions about sexual violence.
A headline-grabbing result with a super-questionable research technique usually means a bunk result. It’s hard to blame journalists and activists for grabbing onto it, but the researchers and journal editors should have known better.