Scientists are Only Human – Drat #ResearchCulture #researchers

Don’t we all love something new, unexpected, and exciting? Scientists do too, and that can lead to bad science.

You may have seen a recent report analyzing information on Twitter that showed false news spreads faster and reaches more people than verified true claims. “Novelty is a culprit: the false news that spread rapidly online was significantly more unusual than the true news.” If you thought that only applies to the hoi polloi and social media, guess again.

Both academics and laypeople experience surprises as more interesting (and certainly more entertaining) than the predictable, the normal and the quotidian. [Results reported] in Science Advances are disturbing: papers that couldn’t be replicated were cited more than average, even after the news of the reproducibility failure had been published, and only 12 percent of postexposure citations acknowledged the failure. ScientificAmerican

If no one else can get the same results you did, you probably did something wrong. This can happen even when you are trying to be diligent. As Richard Feynman said:

The first principle is that you must not fool yourself – and you are the easiest person to fool.

Or, as a corollary to Murphy’s Law puts it, no matter how hard you try, you will fail to find your own mistakes. Fortunately, the first yahoo who wanders by will spot them immediately.

New studies based on flawed old studies aren’t likely to advance knowledge. The most encouraging part of all this is that people are trying to improve the situation. Science has shown us, over and over, that our gut reactions, instincts, and even common sense can lead us astray. Science is still the best way to learn about the physical world, and that means supporting good, sound, well conceived and valuable research that may not inspire headlines screaming “break-through.”

Thanks to Scientific American for their article. You’ll find more on transparency, openness, and reproducibility – and on problems with academic reward systems – here. Thanks also to Retraction Watch, where you’ll discover how scientists stay true to their calling.

Advertisement

Bad News is Good News, Because This is How Science Works #science #diet #research

In science it often happens that scientists say, ‘You know, that’s a really good argument; my position is mistaken,’ and then they would actually change their minds and you never hear that old view from them again. They really do it. It doesn’t happen as often as it should, because scientists are human and change is sometimes painful. But it happens every day. Carl Sagan

It happened again, in the prestigious New England Journal of Medicine. Stay with me – there’s statistics – but it’s worth the read.

Gazpacho Ingredients

A Mediterranean diet is still a good idea – make yourself some gazpacho

A 2013 study of the Mediterranean diet claimed proof that people eating the fruits/vegetables/olive oil/nuts/fish diet were less likely to experience a heart attack or stroke than people eating a low-fat diet.

Statistical problems were discovered and the study retracted and revised to say, while the subjects had fewer heart attacks and strokes, the diet wasn’t proven to be the reason.

Okay, this may not seem earthshaking. But how it happened is so cool.

We can thank John Carlisle, a British anesthesiologist.

He wrote a letter to an anesthesiology journal bemoaning the fact that his field was polluted by one researcher’s data that many suspected were problematic. The journal editor told Carlisle to prove it.

He studied statistical methods so he could prove it, and got over a hundred papers in his field retracted. But Carlisle wasn’t done. He looked at many more papers in many fields and found 2% were flawed – that they claimed to use a gold standard of randomized trials but had blocks of non-randomized data. (There, that’s the statistics part.)

The lead author of the 2013 Mediterranean diet study quickly acknowledged the problems when Carlisle pointed them out and revised the paper. That’s got to hurt. Studies cost a lot of time and money, but he did it. Carl Sagan would be proud.

If that still doesn’t sound earthshaking, consider that studies like these can impact the health of millions of people all over the world. And that “paper mill” journals with fewer scruples than the NEJM have sprung up recently, further complicating our lives.

Carlisle praised the journal’s response. ‘I think that the NEJM editorial team responded very maturely to my paper,’ he says. ‘They took the possibility of a problem seriously and acted quickly and thoroughly.’

That’s science. That’s why science transforms our lives.

Thanks to npr.org for reporting to those of us who don’t read the NEJM. Also check out Retraction Watch, a website that reports on scientific retractions and related issues.