“THE CRUSADE AGAINST MULTIPLE REGRESSION ANALYSIS”. Professor Nisbett has concluded that the villain in the reproducibility failures is multiple regression analysis, saying: “A huge range of science projects are done with multiple regression analysis. The results are often somewhere between meaningless and quite damaging….The thing I’m most interested in right now has become a kind of crusade against correlational statistical analysis—in particular, what’s called multiple regression analysis.”
It is hard to get results with multiple regression analysis because it is used to deal with difficult problems—attempting to identify causes where there are multiple possible explanatory factors and causal relationships running the problems running both ways. I pointed out here that Freakonomics works because Steven Levitt (of Freakonomics) “is gifted at finding variables that move independently of other variables so that he is able to disentangle cause and effect.â€
Nisbett acknowledges that other causal factors are a major problem for reproducibility. For example, he describes a famous experiment (what he calls the “granddaddy” of many experiments): “Students hear the words ‘cane,’ ‘Florida,’ ‘gray,’ and then they walk more slowly out of a laboratory. That sometimes replicates and sometimes doesn’t. But the whole point about a trivial fleeting stimulus that might be powerful is that, in a slightly different context, or when the person’s attention is directed slightly differently, you may not get the effect.”
Multiple regression addresses the role of multiple possible causal factors by considering them explicitly. Nisbett wants to avoid doing using it—to the extent of conducting a crusade against it.