Don’t we all love something new, unexpected, and exciting? Scientists do too, and that can lead to bad science.
You may have seen a recent report analyzing information on Twitter that showed false news spreads faster and reaches more people than verified true claims. “Novelty is a culprit: the false news that spread rapidly online was significantly more unusual than the true news.” If you thought that only applies to the hoi polloi and social media, guess again.
Both academics and laypeople experience surprises as more interesting (and certainly more entertaining) than the predictable, the normal and the quotidian. [Results reported] in Science Advances are disturbing: papers that couldn’t be replicated were cited more than average, even after the news of the reproducibility failure had been published, and only 12 percent of postexposure citations acknowledged the failure. ScientificAmerican
If no one else can get the same results you did, you probably did something wrong. This can happen even when you are trying to be diligent. As Richard Feynman said:
The first principle is that you must not fool yourself – and you are the easiest person to fool.
Or, as a corollary to Murphy’s Law puts it, no matter how hard you try, you will fail to find your own mistakes. Fortunately, the first yahoo who wanders by will spot them immediately.
New studies based on flawed old studies aren’t likely to advance knowledge. The most encouraging part of all this is that people are trying to improve the situation. Science has shown us, over and over, that our gut reactions, instincts, and even common sense can lead us astray. Science is still the best way to learn about the physical world, and that means supporting good, sound, well conceived and valuable research that may not inspire headlines screaming “break-through.”
Thanks to Scientific American for their article. You’ll find more on transparency, openness, and reproducibility – and on problems with academic reward systems – here. Thanks also to Retraction Watch, where you’ll discover how scientists stay true to their calling.