In the 1960s, psychologist Daniel Kahneman was working as a consultant for the Israeli Air Force.

His job was to explain to flight instructors the latest research in behavior psychology; specifically, how rewarding positive behavior is far more effective than punishing mistakes.

Suddenly, one of his students stood up in irritation.

He argued that, in his experience, those whom he praised for well-executed flight patterns always did worse the next time, while those whom he screamed at for poorly executed ones invariably performed much better the next time. “Don’t tell me that reward works and punishment doesn’t work,” he said. “My experience contradicts it.”

The other flight instructors agreed. But in Kahneman’s mind, a different idea was forming. Surely the instructors’ experiences were legitimate, but there was something else going on…

What Kahneman realized, and what his students failed to account for, was the existence of a statistical phenomenon known as regression towards the mean.

Think about the flip of a coin for a second. There’s a fifty percent chance it’ll land on heads, and the same for tails as well. Given a fair coin, this probability holds true no matter how many times you flip—even if you got ten heads in a row, there is still only a fifty percent chance you’ll get heads.

In fact, if you were to graph a coin toss experiment as a probability curve, you’d get a normal distribution, where the data centers around the mean:

This is where regression into the mean comes into play.

Suppose that I flipped a coin and got heads for ten straight runs. While I can never be certain that the next toss will also result in heads, I can be fairly certain that the next run of heads will be less than ten. Why?

Well, it’s because getting ten heads in a row is what we consider “extreme behavior” of the random variable; it’s just not something you’d see happen regularly!

So, since our fair coin has a fifty-fifty chance by default, it would be an extremely rare to observe ten heads in a row—and, as a result, we can be certain that once such an event occurs, it’s most likely that it will return back to normal behavior.

In other words, the coin will “return” back to its average behavior, which is around the mean of our probability distribution.

So, how does this work into Kahneman’s story?

See, any student pilot that starts out is going to have some level of experience with flying planes. They would gradually be improving their skills with training, which means the change wouldn’t be instantly noticeable.

Therefore, any exceptional or terrible performance would be based more on luck than actual skill. In the context of the above, this would be an instance of extreme behavior in an otherwise random variable.

So after such an outlier performance, the pilot would naturally return back to their norm afterwards.

If the outlier was pos...