HOW TO MASSAGE DATA FOR PUBLICATION. Kids, this article by Stephanie Lee on the Buzzfeed site describes some of the techniques that can be used to massage data to come up with a publishable “study”. (link via Instapundit). The article presents a young researcher arriving at the Cornell Food and Brand Lab. Before she arrives, she is given a data set based on an experiment on pizza eating. In the experiment, some subjects had paid full price and some got a half price discount, and information about their behavior was collected. The intern was given guidance on how to find a pattern in the results. She should break the subjects into many groups—- “males, females, lunch goers, dinner goers, people sitting alone, people eating with groups of 2, people eating in groups of 2+, people who order alcohol, people who order soft drinks, people who sit close to buffet, people who sit far away, and so on…” Presumably the behavior of one or more of the groupings would be different from the others and would create a “publishable” result.
Stephanie Lee says: “Researchers should set out to prove a specific hypothesis before a study begins. [This researcher], in contrast, was retroactively creating hypotheses to fit data patterns that emerged after an experiment was over.”
Many political consultants are feasting off of similarly ridiculous, facially broken statistical methods.