SMALL SAMPLES—ANOTHER PROBLEM WITH PUBLISHED ARTICLES. I have posted, for example here, about studies by John Ioannidis and others which find that about one third of frequently cited scientific papers turn out to be wrong. Among the reasons is a bias toward publishing positive results. Another factor is that it is hard to do science. I posted here on Richard Feynman’s speech about the difficulty of eliminating alternative explanations. This article reports on a study which highlights the problems which sample sizes present for neuroscience papers. (link via Instapundit). The study by researchers at the University of Bristol reviewed 48 neuroscience articles from 2011 and “concluded that most had an average power of around 20 per cent – a finding which means the chance of the average study discovering the effect being investigated is only one in five.” This means that the papers are much more likely to miss effects that are there. And also: “Another consequence is that the findings are overestimated because smaller studies consistently give more positive results than larger studies.”

This entry was posted in Economics, Science. Bookmark the permalink.


  1. Henry Nejako says:

    Just came across this fascinating New York Magazine article about a 28-year-old graduate student, Thomas Herndon, who found a major flaw in an influential 2010 economic paper by Carmen Reinhart and Kenneth Rogoff, “Growth in Time of Debt.”

    “What Herndon had discovered was that by making a sloppy computing error, Reinhart and Rogoff had forgotten to include a critical piece of data about countries with high debt-to-GDP ratios that would have affected their overall calculations. They had also excluded data from Canada, New Zealand, and Australia — all countries that experienced solid growth during periods of high debt and would thus undercut their thesis that high debt forestalls growth.”

    This strengthens my admiration for Nobel Economics Cassandra Paul Krugman.


  2. Pingback: SPREADSHEET ERRORS (COMMENT). | Pater Familias

Leave a Reply

Your email address will not be published.