When does science self-correct? Lessons from a replication crisis in early 20th century chemistry
https://goodscience.substack.com/p/when-does-science-self-correct-lessons
INTRO: Science is self-correcting—or so we are told. But in truth it can be very hard to expunge errors from the scientific record. In 2015, a massive effort showed that 60 percent of the findings published in top psychology journals could not be replicated. This was distressing news, but it led to several healthy reforms in experimental psychology, where a growing number of journals now insist that investigators state their hypotheses in advance, ensure that their sample size is adequate, and publish their data and code.
You might also imagine that the credibility of the non-replicable findings took a hit. But it didn’t—not really. In 2022—seven years after the poor replicability of many findings was revealed, the findings that had failed to replicate were still getting cited just as much as the findings that had replicated successfully. And when they were cited, the fact that they had failed to replicate was rarely mentioned.
This pattern has been demonstrated several times in psychology and economics... (MORE - details)
_
https://goodscience.substack.com/p/when-does-science-self-correct-lessons
INTRO: Science is self-correcting—or so we are told. But in truth it can be very hard to expunge errors from the scientific record. In 2015, a massive effort showed that 60 percent of the findings published in top psychology journals could not be replicated. This was distressing news, but it led to several healthy reforms in experimental psychology, where a growing number of journals now insist that investigators state their hypotheses in advance, ensure that their sample size is adequate, and publish their data and code.
You might also imagine that the credibility of the non-replicable findings took a hit. But it didn’t—not really. In 2022—seven years after the poor replicability of many findings was revealed, the findings that had failed to replicate were still getting cited just as much as the findings that had replicated successfully. And when they were cited, the fact that they had failed to replicate was rarely mentioned.
This pattern has been demonstrated several times in psychology and economics... (MORE - details)
_