Regression to the mean

Extreme results will tend—on average—to be followed by (“regression”) less extreme results (“the mean”). Thus, it is important to avoid attaching “a causal interpretation to the inevitable fluctuations of a random process”.^1 The excitement of a highly favorable outcome early on may predispose you to overestimate your prediction of what this means for you (or your company, the industry, etc.) As Kahneman explains:

…it is natural for System 1 [the emotionally-driven part of our minds] to generate overconfident judgments, because confidence, as we have seen, is determined by the coherence of the best story you can tell from the evidence at hand. Be warned: your intuitions will deliver predictions that are too extreme and you will be inclined to put far too much faith in them.

The solution? Larger sample sizes and brutal assessment of reality are good places to start.


#cognition #statistics