Today in Abuse of Social Science

“Campaign Donations Buy Politicians, Says Science“. The actual survey described does this:

Kalla and Broockman worked with CREDO Action, a progressive organizing group you may be aware of, to develop a pitch for a meeting with Congressional staff that they sent out to 191 Congressional offices. The meeting asked for a meeting about a bill CREDO was organizing on, but varied whether the people asking were identified as “local constituents” or “local campaign donors.” There was no lying; the people in question were actually local donors, and agreed to participate. CREDO just switched up whether their pitch for a meeting identified them as a donor.

Okay, that seems like a reasonable experiment – are Congresspeople more willing to take constituent meetings or donor meetings?  The answer seems pretty obvious.  The answer seems especially obvious because “a local campaign donor” is both a donor AND a constituent. But let’s lay aside the condescension for a moment and see what they found:

The professors then compared who got to meet with important people (the member, senior staff), less important people (junior staff, outreach staffer), or no one at all. The evidence was very clear — people who identified themselves as donors in the pitch letter were significantly more likely to get a plum meeting. As you can see from this chart, people who didn’t identify themselves as donors were more likely to get no meeting or meet with the less important staffers in the left columns, whereas donors were much more likely to meet the members themselves or almost-as-important staff on the right:

So donors were 4 times more likely to get a meeting with a member of Congress!  Except…well.  First of all, given the fact that the majority of people got blown off (a district staff meeting counts as a blow-off), the expected outcome for both treatments was the same.  Secondly, given the sample size – this means a grand total of two constituents and eight donors got a chance to meet with the member of Congress.  The error bars on this ought to be large.  Thirdly, there’s probably some bias associated with the fact that CREDO is the sole agency doing this.  Maybe a local Chamber of Commerce would have different results instead of a liberal pressure group.  Or maybe not.  Maybe Republican Congressmen would respond differently.  It’s really hard to generalize away from this specific example.

Finally and by far most importantly – CREDO didn’t buy shit.  This legislation has not been passed.  No money changed hands.  There was certainly no quid pro quo, or even the hint of one.  Meetings are cheap – there is a world of difference between a fifteen-minute sit-down with a Congresscritter and having them actually marshall votes for a piece of legislation you drop into their hot little hands.  This study, even if it is completely valid, proves exactly nothing about donors’ ability to “buy” a Congressperson.  And even a casual glance at Kalla and Broockman’s paper shows they claim nothing of the sort.

American journalists, this is why you can’t have nice things.

Tags: , , , , , , , , ,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: