On Good Bad Research

There’s a viral study going around – you might have run into it on Facebook – on porn consumption and marriage.  It went viral because duh, and its conclusion is that porn is destroying marriage.  Unfortunately, it’s bunk.  Jordan Weissman does an admirable job debunking the study; long story short, the authors use a terrible instrumental variable.  It’s a nice little look at the use and abuse of social science tactics, and how fancy math can be used to confidently frame complete bull.

This is bad research, and there’s a big market for bad research.  People love to be able to point to “studies” supporting their point of view, whether it’s on economic or cultural issues.  Or the real beating blackened heart of terrible science, diets.  The academy has many problems, but something like this would never make it past peer review because the design is obviously bunk to someone who understands the instrumental variable technique.  On the other hand, it can be released as a “working paper” and run around the world twice before the truth can put its proverbial boots on.

Lying with math is a hard problem to deal with.  The tools of social science can be used and abused, and often even a reasonable and well-intentioned researcher can create work that is willfully misinterpreted and abused by bad actors.  I wish I had some good answers as to what can be done.  Actual working academics have professional ethical responsibilities, and peer review to keep them honest.  Unfortunately, the people who are most likely to abuse the public trust are the ones sitting outside that system.  And misinformation is hard to correct once people have internalized it.