SMALL SAMPLES—ANOTHER PROBLEM WITH PUBLISHED ARTICLES.

SMALL SAMPLES—ANOTHER PROBLEM WITH PUBLISHED ARTICLES. I have posted, for example here, about studies by John Ioannidis and others which find that about one third of frequently cited scientific papers turn out to be wrong. Among the reasons is a bias toward publishing positive results. Another factor is that it is hard to do science. I posted here on Richard Feynman’s speech about the difficulty of eliminating alternative explanations. This article reports on a study which highlights the problems which sample sizes present for neuroscience papers. (link via Instapundit). The study by researchers at the University of Bristol reviewed 48 neuroscience articles from 2011 and “concluded that most had an average power of around 20 per cent – a finding which means the chance of the average study discovering the effect being investigated is only one in five.” This means that the papers are much more likely to miss effects that are there. And also: “Another consequence is that the findings are overestimated because smaller studies consistently give more positive results than larger studies.”