Editorials

Conviction by mathematical error?

Doctors and lawyers should get probability theory right

In a recent case of DNA evidence the probability of a chance match was quoted as 20 million to one. The accurate statementthat the defendant or two other unknown people in the United Kingdom could have committed the offenceis much less impressive. Other evidence was overwhelming, but this may not always be true, especially with matches from DNA databases. Even more problematic than the issue of presenting statistical evidence fairly is the problem of getting it wrong.

On 9 November at Chester Crown Court Sally Clark, a Cheshire solicitor, was convicted, by 10-2 majority, of smothering her two infant children. With conflicting forensic evidence, the Crown's case was bolstered by an eminent paediatrician testifying that the chances of two cot deaths happening in this family was vanishingly small1 in 73 million. This seriously misunderstands probability theory. It is speculation whether Sally Clark would have been acquitted without this evidence. But with this mathematical error prominent the conviction is unsafe.

Imagine an archery target with two arrows sticking in the very centre of it. This provides greater evidence of the skill of the archer if the target was in place before the arrows were fired than if it was drawn around them afterwards. Probability theory requires calculation of the probability not only of the event in question but also of all events that are as extreme or more extreme. When the target is drawn first you calculate the chance of both arrows hitting the centre of the target. But when the target is drawn round the arrows afterwards you calculate the chance of both arrows hitting the same point, whatever that point. With two independent arrows one probability is the square of the other.

Suspicion was drawn to Sally Clark by the occurrence of two deaths so the probabilities should not have been squared. The odds of 1 in 73 million shrink to 1 in 8500. But this figure is itself meaningless. There is in fact a wall full of arrows with the target drawn around the two that are close together and the others ignored. Mathematical formulas for this situation often surprise people. For example, with only 23 people in a room the odds are better than 50% that two of them have the same birthday.

From whole population data Reese calculates the square of the population risk of cot death as 1 in 2.75 million.¹ There are 378 000 second or subsequent births each year in England. So if cot deaths are random events two cot deaths will occur in the same family somewhere in England once every seven years. But cot deaths are not random events. There have been several studies of recurrence. At least one study did show no increase in recurrence rates.² But several others showed recurrence rates about five times the general rate,^3-5 implying recurrence somewhere in England about once every year and a half. Two studies showed even higher rates. ^{6 7}

The fact that studies of recurrence have been done means this event is not vanishingly rare. In a case series of recurrent infant death Emery classified two cases as recurrent cot death out of 12 cases occurring in Sheffield in 20 years.⁸ Wolkind et al found five cases in their unsystematic English case series of 57 recurrent infant deaths.⁹ Both these studies distinguished cot death from accident, illness, murder, and neglect.

The prosecution used the figure of 1 in 73 million rather than 1 in 2.75 million because of the family's affluence. Yet taking data from an epidemiological group and applying it stereotypically to all members is an example of the ecological fallacy. Social class is a complex reality of interassociated circumstanceseducation, work, income, lifestyle, culture, contacts, residence, opportunities, social class of origin, etcstatistically summarised for use in population studies by selecting the one variable which performs best as an indicator. This does not mean that individuals have the attributes of the statistical group.

Guidelines for using probability theory in criminal cases are urgently needed. The basic principles are not difficult to understand, and judges could be trained to recognise and rule out the kind of misunderstanding that arose in this case. Never again must mathematical error be allowed to conflict with mathematical fact as if each were a legitimate expert view.

What is our profession's responsibility for the quality of expert evidence given by doctors? Medical evidence is trusted, and we must retain that situation and ensure that it is not abused. It is possible to be an extremely good doctor without being numerate, and not every eminent clinician is best placed to give epidemiological evidence. Doctors should not use techniques before they have acquainted themselves with the principles underlying them.

When errors occur we expect them to be admitted, learnt from, and corrected. Should clinical governance extend to the courtroom? Expert witnesses can hold a substantial part of defendants' lives in their hands. Defendants deserve the same protection as patients.

Stephen J Watkins, director of public health. 

Stockport Health Authority, Stockport SK7 5BY

Acknowledgments

Predisposing biases: SJW is a vice president and immediate past president of the Medical Practitioners' Union, which is predisposed to support the civil liberties movement. He has no personal acquaintance with people involved in this case.