SHOOTING AT THE WRONG TARGET. I posted here about Goodhart’s Law: “once a social or economic indicator is made a target for the purpose of conducting social or economic policy, it will lose its value as a measure.” I wrote in the post: ” Take a sales manager who observes that his best salesmen have the most meetings with customers. He creates incentives for the next sales year which reward increases in meetings with customers. The plan fails. The Stubborn Mule [blog] says: ‘According to Goodhart’s Law, the very act of targeting a proxy (client meetings) to drive a desired outcome (sales performance) undermines the relationship between the proxy and the target.’ (Think of salesmen gaming the system and overscheduling meetings with customers).”
Wells Fargo has just been fined for opening a couple million fake accounts. About 5300 employees had been fired in the last five years for signing customers up for fake accounts.
Goodhart’s Law predicted this would happen. Matt Levine begins his article on Bloomberg about the Wells Fargo frauds: “Two basic principles of management, and regulation, and life, are:
You get what you measure.
The thing that you measure will get gamed.”
I recently heard a radio spot from the author of “Weapons of Math Destruction” and she was talking about how often algorithms simply mask, amplify, and cloak in false objectivity biases and flaws in an institution / process.
e.g. Police have historically paid more attention to “high crime neighborhoods,” and thus more policing yields more arrests. Then, the high arrest numbers are used to justify more high policing in a feedback loop.