Bluematter.

Mark Buchanan, via Andrew Gelman, provides another example of ignorance breeding evil (O Socrates, were you ever wrong?):

Later this month – or it could be next month – a group of three judicial “wise men” in the Netherlands should finally settle the fate of a very unlucky woman named Lucia de Berk. A 45-year-old nurse, de Berk is currently in a Dutch prison, serving a life sentence for murder and attempted murder. The “wise men” – an advisory judicial committee known formally as the Posthumus II Commission – are reconsidering the legitimacy of her conviction four years ago.

Lucia is in prison, it seems, mostly because of human susceptibility to mathematical error – and our collective weakness for rushing to conclusions as a single-minded herd.

When a court first convicted her, the evidence seemed compelling. Following a tip-off from hospital administrators, investigators looked into a series of “suspicious” deaths or near deaths in hospital wards where de Berk had worked from 1999 to 2001, and they found that Lucia had been physically present when many of them took place. A statistical expert calculated that the odds were only 1 in 342 million that it could have been mere coincidence.

Open and shut case, right? Maybe not. A number of Dutch scientists now argue convincingly that the figure cited was incorrect and, worse, irrelevant to the proceedings, which were in addition plagued by numerous other problems.

For one, it seems that the investigators weren’t as careful as they might have been in collecting their data. When they went back, sifting through hospital records looking for suspicious cases, they classified at least some events as suspicious only after they realized that Lucia had been present. So the numbers that emerged were naturally stacked against her.

Mathematician Richard Gill of the University of Leiden, in the Netherlands, and others who have redone the statistical analysis to sort out this problem and others suggest that a more accurate number is something like 1 in 50, and that it could be as low as 1 in 5.

More seriously still – and here’s where the human mind really begins to struggle – the court, and pretty much everyone else involved in the case, appears to have committed a serious but subtle error of logic known as the prosecutor’s fallacy.

The big number reported to the court was an estimate (possibly greatly inflated) of the chance that so many suspicious events could have occured with Lucia present if she was in fact innocent. Mathematically speaking, however, this just isn’t at all the same as the chance that Lucia is innocent, given the evidence, which is what the court really wants to know.

To see why, suppose that police pick up a suspect and match his or her DNA to evidence collected at a crime scene. Suppose that the likelihood of a match, purely by chance, is only 1 in 10,000. Is this also the chance that they are innocent? It’s easy to make this leap, but you shouldn’t.

Here’s why. Suppose the city in which the person lives has 500,000 adult inhabitants. Given the 1 in 10,000 likelihood of a random DNA match, you’d expect that about 50 people in the city would have DNA that also matches the sample. So the suspect is only 1 of 50 people who could have been at the crime scene. Based on the DNA evidence only, the person is almost certainly innocent, not certainly guilty.

This kind of error is so subtle that the untrained human mind doesn’t deal with it very well, and worse yet, usually cannot even recognize its own inability to do so. Unfortunately, this leads to serious consequences, as the case of Lucia de Berk illustrates. Worse yet, our strong illusion of certainty in such matters can also lead to the systematic suppression of doubt, another shortcoming of the de Berk case.

Of course, the de Berk case is hardly an isolated example of statistical error in the courtroom. In a famous case in the United Kingdom a few years ago, Sally Clark was found guilty of killing her two infants, largely on the basis of testimony given by Roy Meadows, a physician who told the court that the chance that the two both could have died from Sudden Infant Death Syndrome (SIDS) was only 1 in 73 million. Meadows arrived at this number by squaring the estimated probability for one such death, which is an elementary mistake. Because SIDS may well have genetic links, the chance that a mother who already had one child die from SIDS would have a second one may be considerably higher.

Here, too, the prosecutor’s fallacy seems to have loomed large, as the likelihood of two SIDS deaths, whatever the number, is not the chance that the mother is guilty, though the court may have interpreted it as such.

Even our powerful intuitive belief that “common sense” is a reliable guide can be extremely dangerous. In Sally Clark’s first appeal, statistician Philip Dawid of University College London was called as an expert witness, but judges and lawyers ultimately decided not to take his advice, as the statistical matters in question were not, they decided, “rocket science.” The conviction was upheld on this appeal (although it was subsequently overturned).In ordinary usage, “common sense” is taken to be something of value. Albert Einstein had a less charitable view. “Common sense,” he wrote, “is nothing more than a deposit of prejudices laid down by the mind before you reach age 18.”

By the way, Statistical Modeling, Causal Inference and Social Science is one of the best blogs around and a personal favourite. An absolute treasure chest of knowledge and insightful commentary, it is highly recommended to anyone with more than a passing interest in statistics and econometrics.