Who's been a naughty boy?

In Freakonomics, Steven Levitt documents how he identified teachers that massaged their students' test results to make them look better. This paper (free access, but (quick) registration required) suggests that some developing countries might also be giving their figures a little boost:

[...] we present a technique for assessing data quality based on conformity with Benford's Law, which states that the first digits of numbers generated from natural phenomena do not occur with equal frequency. If data do not conform to the Benford distribution, then questions arise about the process that generated it.

Benford's Law is applied to one of the most commonly used data sets in economics: international macroeconomic statistics. We find that the World Bank international GDP data and purchasing power parity (PPP) corrected Penn World tables for OECD countries conform well to Benford's Law. But some subsets of the data – particularly GDP figures from the developing world -- show non-conformity consistent with deliberate manipulation of the underlying series.

The results are suggestive only; Benford's law is not terribly well understood, and it is difficult to assess the properties of the test used by the authors before more research is carried out - this is virgin territory for economists. That said, for practitioners of the most inquisitive and cynical of professions, this could prove to be an invaluable statistic.

Thanks to voluntaryXchange for the pointer.

by datacharmer | Wednesday, September 12, 2007
  | | Who's been a naughty boy? @bluematterblogtwitter