Read the Study First, For the Nth Time

There’s a story getting a ton of play on Facebook about how one-third of men say they’re willing to force a woman to have sex against her will.  It’s horrible, it’s attention-grabbing, and it plays into peoples’ worst fears about men (and people in general). And at least unlike some of the “studies” that get press coverage, it’s at least a paper in a peer-reviewed journal with an acceptable sample size (86).  It has the usual problem of convenience sampling (e,g,, all white college males who received college credit for answering).  The issue goes deeper than that; this is a horribly flawed paper with a research design that verges on deceptive.

The study is set up to maximize the number of people who say they would commit rape.  The authors use questions from a few different scales to assess whether the respondents are inclined to commit sexual violence or violence in general. After spending the survey making the recipients answer questions about violence and sexual violence, they then ask the recipients about their willingness to engage in sexual violence.  The problem is that by asking all these questions about violence before asking the key question about violence, they are likely to activate violent tendencies in their respondents.  This is called “priming”, and can be a real problem.  Survey responses are volatile, and even small primes can affect them.  Conscientious researchers should have put the key question first – or even better, used treatment and control groups where they didn’t ask the preceding questions about sexual violence.

A headline-grabbing result with a super-questionable research technique usually means a bunk result.  It’s hard to blame journalists and activists for grabbing onto it, but the researchers and journal editors should have known better.

Tags: , , , , ,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: