In science it often happens that scientists say, ‘You know, that’s a really good argument; my position is mistaken,’ and then they would actually change their minds and you never hear that old view from them again. They really do it. It doesn’t happen as often as it should, because scientists are human and change is sometimes painful. But it happens every day. Carl Sagan
It happened again, in the prestigious New England Journal of Medicine. Stay with me – there’s statistics – but it’s worth the read.
A 2013 study of the Mediterranean diet claimed proof that people eating the fruits/vegetables/olive oil/nuts/fish diet were less likely to experience a heart attack or stroke than people eating a low-fat diet.
Statistical problems were discovered and the study retracted and revised to say, while the subjects had fewer heart attacks and strokes, the diet wasn’t proven to be the reason.
Okay, this may not seem earthshaking. But how it happened is so cool.
We can thank John Carlisle, a British anesthesiologist.
He wrote a letter to an anesthesiology journal bemoaning the fact that his field was polluted by one researcher’s data that many suspected were problematic. The journal editor told Carlisle to prove it.
He studied statistical methods so he could prove it, and got over a hundred papers in his field retracted. But Carlisle wasn’t done. He looked at many more papers in many fields and found 2% were flawed – that they claimed to use a gold standard of randomized trials but had blocks of non-randomized data. (There, that’s the statistics part.)
The lead author of the 2013 study quickly acknowledged the problems when Carlisle pointed them out, and revised the paper. That’s got to hurt. Studies cost a lot of time and money, but he did it. Carl Sagan would be proud.
If that still doesn’t sound earthshaking, consider that studies like these can impact the health of millions of people all over the world. And that “paper mill” journals with fewer scruples than the NEJM have sprung up recently, further complicating our lives.
Carlisle praised the journal’s response. ‘I think that the NEJM editorial team responded very maturely to my paper,’ he says. ‘They took the possibility of a problem seriously and acted quickly and thoroughly.’
That’s science. That’s why science transforms our lives.