Let’s say you find a newsletter one day and it indicates that a certain stock is going to rise by the end of the week. Scoffing at the idea that anybody can predict the stock market, you humor the notion and check the stock the next week and see that it has in fact risen. Sure that it is just luck, you wait for the next issue and you find out from it that the stock is going to go down the following week. And sure enough, the prediction comes true once again.
Ten weeks go by like this and each time you see that the news letter correctly predicts the stock out for the next week. At this point, you have to give it to whoever is coming up with these predictions. The odds of the newsletter being accurate for ten straight weeks using random guesses is less than 0.1% (more precisely [math]\frac{1}{1024}[/math]) They must have some serious forecasting skills.
And sure enough, this mysterious stockbroker behind the newsletters sends you a solicitation on the following day to invest money in his forecasting company. And at this point, this sounds like a fantastic deal! He has shown significant skill. There’s nothing really fishy going on here, right? After all, the results of his work are right in front of you!
But there is some clever trickery going on here. Let’s take a look at this story from stockbroker’s perspective.
On the first day, the stockbroker sends 10,240 newsletters, in which half of them predict the stock would go up, and the other half predict it will go down. Now 5,120 of the receivers of this newsletter immediately see an inaccuracy. But they never hear from the stockbroker again. However, the 5120 people who did receive the correct prediction get another newsletter the following week. These 5,120 newsletters are once again split into two groups of guesses, of which now 2,560 people received predictions that were accurate two days in a row.
You can see where this is going. By the end of the tenth week, you and nine other people receive the solicitation to invest in the “skilled” stockbroker’s company. And if you didn’t know any better, you would sign the offer without a second thought.
This story is known as the Baltimore stock broker story, and it really shows the importance of considering multiple perspectives when viewing modern day events, data, or statistics.