Fooled by Randomness is a famous book by Nassim Taleb. As it turns out it is quite difficult for humans to overcome their tendency to see patterns and not get fooled by randomness. In a player report created by a consultancy agency explained how a defender in de Premier League had a 0.11 goal scored in the 2016/2017, but that the same player had a 0.17 goal scored in the 2017/2018 season. That put the defender in the 2.5% best defenders. In the same report they also indicated that this defender had played 3000 minutes in 2016/2017, but only played 2000 minutes. But this wasn’t flagged as a warning. When I then started to look at the underlying data, it turns out that this defender had scored twice in 2016/2017 and had scored three times in season 2017/2018. Whether a player scores two or three times in a season is so much due to luck that it makes very little sense to base decision making on. In the season 2018/2019 the defender played only 1000 minutes and didn’t score.

First of all that the defender didn’t score in 2018/2019 doesn’t mean that he isn’t a good defender or that his contribution to the team’s attack aren’t better than most defenders. Because even with the third season data there still isn’t enough data to reach conclusions. The number of minutes played is a much more meaningful number because it is based on a lot more underlying data.

It is very hard to proof that an analysis is correct and more than happenstance. For that reason it is better to give an indication of how probable an analysis is. When analysts add predictions and the probability of those predictions then we can track how well these predictions hold up. Then we can use Brier’s Rule to see which analysts came up with the right predictions and which analysts didn’t. That is the best indication that the good analysts are indeed onto something that goes beyond happenstance.

Also, for almost all draws and matches that are won by a single goal difference, it is very hard to argue that it is not happenstance. Luck plays such a large part in football. If a match is won with a two goal difference then one can safely say that the outcome was based on more than luck alone, in most cases as there are always exceptions.