The unsexy threat to government data

I don’t think it’s quite appreciated how much economic and civic life in the United States is underpinned by government data – from the Census to the Weather Service to the Bureau of Labor Statistics, the government produces a ton of data that is used on an everyday basis.  Businesses use ACS data to locate stores and markets, banks rely on the unemployment rate to make forecasts, courts rely on Census data to look for disenfranchisement, and so on.  This is an area that scares me.

The President is relatively indifferent to the needs of wonks and bureaucrats, and as a result much of this data will be under threat.  It is highly unlikely, given this indifference, that he will order meddling to juice the unemployment rate – but political appointees will be under a lot of pressure to produce good results, and this means giving him good news.  A lot of this data can be degraded through spontaneous efforts in the middle management layer of the government without a real “plan” from the top.

Researchers should keep an eye on official government data products – particularly looking out for markers of fraud such as violations of Benford’s Law or data that looks suspiciously normal.  It would be fairly easy to develop a battery of tests that will monitor official data outputs to see whether there is evidence of manipulation – this could be a good project for enterprising nerds out there.

The sad fact is that if the government is caught meddling in the data, most likely a lot of this data is ruined for use forever.  Institutional trust is easy to destroy, but difficult to rebuild, and the question of data integrity is one that will be warred over by rival groups of nerds with statistical instruments and arguments impossible for everyday citizens to adjudicate.  Once government’s data integrity is called into question the mere fact of the back-and-forth dispute, even if eventually resolved with a full solution, will serve as an argument in itself.  We’ll likely have fallen into a low-trust equilibrium, which is more stable than a high-trust one.

The knock-on effects aren’t good.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: