USING THE WRONG ALGORITHM (COMMENT). In a comment on yesterday’s post, Nick called my attention to a new book by Cathy O’Neil, WEAPONS OF MATH DESTRUCTION, which deals with the social and human costs of choosing the wrong algorithm as a target. Nick gave the useful example that: “Police have historically paid more attention to “high crime neighborhoods,†and thus more policing yields more arrests. Then, the high arrest numbers are used to justify more high policing in a feedback loop.”
This review on the Spectrum website by Kristen Clark gives another example from O’Neil’s book of using a seemingly wrong measurement as a target. Car insurance companies use credit scores as a measurement of driver reliability. Regularly paying bills may be correlated with good driving but Consumer Reports has found that: “people with low credit scores and clean driving records were being charged much more for car insurance that people with high credit scores and DUIs on their driving records.”