WHY SO MUCH MEDICAL RESEARCH IS ROT.

WHY SO MUCH MEDICAL RESEARCH IS ROT. The title of this article could be generalized to read WHY SO MUCH STATISTICAL RESEARCH IS ROT rather than WHY SO MUCH MEDICAL RESEARCH IS ROT. Kids, I know you are surprised that I have waited so long to get to this hobby horse of mine, but I knew that an occasion to raise it would occur. If the threshold for a statistically significant result is that it would occur only five per cent of the time, running one hundred separate tests will on average give you five statistically significant results. Put facetiously, five per cent of the time on average, a result will occur which can be expected to occur only five per cent of the time. There is another flaw which is repeated over and over. “Statistically significant” means roughly “nonrandom”; it does not mean that a nonrandom finding is big enough to be important. Deirdre McCloskey has been explaining flaws in tests of statistical significance for years, and she says that misuse is growing rather than declining. (link to the Economist article from Instapundit)