Another example of making mountains out of pimples

I regularly (in FOOD & HEALTH SKEPTIC) characterize medical research as making mountains out of pimples so another example of it might help reinforce what I mean by that. I mentioned this study on the 10th so I will not reproduce it again. Instead I want to point out that its conclusions, although correct, were not only trivial but rather misleading.

What was found was that autism was 7 times more common among mathematics students at Cambridge university than it was among students in other fields at that university. But how was that finding arrived at? It was arrived at by taking 378 mathematics students and comparing them with a group of 414 non-mathematics students. And there were 7 autistic people in the mathematics group but only one autistic person in the other group. So the "7 times" conclusion was totally correct. But was it important?

It was in fact totally trivial. What was really found was that the incidence of autism among students at Cambridge was very rare -- even among mathematics students. Putting it another way, it is very rare for mathematics students to be autistic. Only 1.8% of them were autistic in fact. And putting it that way gives roughly the opposite impression to the impression given by the original report. What the original report presented as interesting is in fact of no importance whatsoever.

And that sort of finding is absolutely typical of what is reported in epidemiological research. In fact it is a much stronger finding than is generally reported. Where a 700% difference was reported above, a 50% difference would be about the average in epidemiological research. Most epidemiological research is the modern-day equivalent of the old mediaeval debate about how many angels can dance on the head of a pin: An argument about trivia.



Posted by John Ray.