WHAT DOES IT MEAN TO SAY THAT CERTAIN STATISTICAL STUDIES CAN BE REPLICATED ONLY 20% OF THE TIME?

WHAT DOES IT MEAN TO SAY THAT CERTAIN STATISTICAL STUDIES CAN BE REPLICATED ONLY 20& OF THE TIME? I posted here about estimates by Dr. John Ioannidis that observational studies (multiple regression studies) in general can be replicated only 20% of the time versus 80% of the time for controlled random trials. Another expert estimates the replication rate for observational studies at 5% to 10%. You may well ask: How accurate are these estimates? I think the differing estimates provide a guide to the range of uncertainty. These are ballpark estimates by experts in the field. They do not have the rigor of a randomized experiment—or, for that matter—of a multiple regression study. The message is similar to that given by the statement that “Infections cause 16% of cancers”. Ed Yong interpreted that statement to mean that infections “cause more cancers than people might expect.” Studies such as those by Dr. Ioannidis show that statistical studies are a lot less reliable than people expect.

This entry was posted in Economics, Science. Bookmark the permalink.

Leave a Reply

Your email address will not be published.