ECONOMICS HAS A DIFFERENT REPLICATION PROBLEM. Megan McArdle posted here on the Bloomberg website about a recent paper from the staff of the Federal Reserve Board which “argues that economics has problems similar to those recently found in psychology: A lot of research results are getting published, and a lot of the interesting findings can’t be replicated, often because key data or instructions aren’t available.” The paper by Andrew C. Chang and Phillip Li is entitled: Is Economics Research Replicable? Sixty Published Papers from Thirteen Journals Say â€Usually Notâ€.
However, this replication issue for economists is different from the replication issue for psychology. The usual issue in psychology is whether if an experiment were run again in the same way as the first experiment but with a different set of subjects, would the result be the same. The replication issue that Chang and Li take up is whether there is enough information available about an article that the calculations can be checked. Chang and Li were able to replicate less than half of the 60 papers they looked at. They say: “The most common cause of our inability to replicate findings is that authors do not provide files to the journal replication archives.”