“THE TRUTH WEARS OFF.”

“THE TRUTH WEARS OFF.” Jonah Lehrer had an important article in the New Yorker (December 13 )about how the results of some randomized statical experiments can deteriorate over time. He begins with a finding from an experiment in 1990 that if subjects were shown a picture of a face twice, they were less likely to identify it if they had verbally described it after the first viewing than if they hadn’t. When the experiment was replicated in 1995, the effect was 30% smaller; when it was replicated in 1996, the effect was 30% less than in 1995. Lehrer also describes a series of experiments on mouse behavior in Albany, Edmonton and Portland. Although the experiments were apparently identical (as far as anybody could tell), the results varied importantly in each city. (I quoted Richard Feynman here on the difficulty of designing experiments to eliminate alternative explanations.) Lehrer describes the work of John Ioannidis on clinical-research studies that had not stood up when replicated and points out that many of them were “randomized controlled trials—the ‘gold standard’ of medical evidence. I have posted on the work of Professor Ioannidis, for example here. Lehrer concludes that: “The decline effect is troubling because it reminds how difficult it is to prove anything.”