PREDICTIONS SHOULD USE NUMBERS. I posted here in 2007 about how there was a time when weather forecasts were expressed in words rather than probabilities.The forecast would say “partly cloudy” or “partly sunny” and they meant the same thing. I said: “when I was growing up, it was unthinkable that you could have a weather forecast which gave the probability of rain the next day. Now, everybody is comfortable with the concept of a 10% chance of rain.”
Bryan Caplan in a review of Tetlock’s book (link via the Marginal Revolution blog) quotes an example that Tetlock gives of a similar confusion on an important issue. In 1951 the National Intelligence Estimate said: “an attack on Yugoslavia in 1951 should be considered a serious possibility.” An official who asked each member of the team who prepared the Estimate what “serious possibility” meant. The answers ranged from a 20% probability to an 80% probability.
Another example: In his book Tetlock says that the Joint Chiefs of Staff told President Kennedy that the Bay of Pigs plan had a “fair chance” of success. Tetlock continues: “The man who wrote the words ‘fair chance’ later said he had in mind odds of 3 to 1 against success.
If you don’t use numbers, people can’t communicate their thinking about the future to each other.