Grading players comes with a myriad of pitfalls. To begin with: talent is a right skewed distribution like this:

What this means is that in professional football the probability of finding a player scoring a 10 is much smaller than finding a player scoring a 1 or a 2. Yt, if you look into the player database of football clubs you almost never see the above distribution. What this means is that the scouts in that club are overestimating the players they are grading.

There are some mitigating circumstances. For instance, for most clubs it is too much work to grade players with a 1 or a 2. So they simply skip those players to save time. Skipping badly performing players is a risk though, because now it becomes harder to check whether the grades given by the scouts match this distribution. So a club that has the time and budget to also grade badly performing players really ought to do so as it is a check on the quality of their scouts. Because when you see a scout only grading players with a 7 or higher, you know that the judgment of that scout has very little value.

Football Behavior Management (FBM) stats follow the above distribution.

Relative or absolute grading?

So the second pitfall is whether you grade the player relative to his current team or the team you are scouting for, or whether you grade him with a 10 being the best player who has ever played and a 0 being the worst player ever played. Relative grading is much easier than absolute grading. But absolute grading has the advantage of being less time constrained. If you do relative grading then you ought to decide whether you grade the player for his current team (or even match) he is playing in, or whether you grade him for the team you are scouting him for, his potential future team. A player who is currently playing in an easier league with more time and space might score a high grade for his current team, but a good scout might see that he would struggle in a team in a tougher league with less time and space and for that reason give him a lower grade.

If you grade a player as he is playing in his current team or even for the actual match you are seeing, then the problem arises that it becomes much harder to compare this grade to other players graded playing for other teams in other matches. So if a club chooses this kind of grading, then they need to be very watchful of people not generalizing the same grade as meaning the same level of performance. This is very hard for people, even if they are conscious of the issue. Because if player X scores a 8 and player Y scores a 9, it is very hard for our unconsciousness to understand that player X is actually the better player as he scored the 8 in a tougher league than player Y scoring a 9.

So it is much better to grade players for the team you want them to play in in the future. But even then the grade is time constrained as the team you are scouting for might change over time. Players that scored an 8 for the first season, might be a 7 in the season thereafter and even only a 6 two seasons later as the team becomes stronger and stronger. If the club has the right infrastructure to keep track of all the judgments of players, in almost all cases, older grades are never adjusted for a team that has been developed and has become stronger. What this means is that a decision to hire a player might have been the right decision in the first season turned out to be the wrong decision in a later season because the club failed to take into account that the majority of the reports for that player were made for a time period where the team was weaker.

To avoid all of these pitfalls, FBM stats are absolute.

One number to rule them all

Modern football requires high quality decision making at high speed. In order to speed up decision making, a single grade helps a lot. Yet, a single grade can lower the quality of decision making. For a single grade a club first needs to decide how to deal with the difference between specialists and generalists. Most players are specialists. If a single grade covers every aspect of playing football, a single grade has a strong bias in favor of the few generalists. This doesn’t have to be a problem, as long as the club is aware of this bias or prefers generalists over specialists.

One can use a single grade to cover specialists. For instance if the grade is simply only for that aspect where the player excels in. For most defenders (but not all!) their single grade would reflect their ability to defend. For most strikers (but not all!) their single grade would reflect their ability to score goals. Yet, even if this is setup correctly, it will still lead to a lot of confusion not in the least for the unconscious mind of the decision maker. Again, if player X scores a 8 and player Y scores a 9, most decision makers will assume, consciously or unconsciously, that player Y is the better player, whereas in reality player Y might be an excellent defender and player X a very good striker, with the striker being the better player as scoring goals is much harder than defending.

So the best solution is to have a single grade for the player to speed up decision making and to have this single grade cover every aspect of playing football. Yet, to also have subgrades that tells the decision maker how the player scores in various aspects of football. There really ought to be only a very small set of subgrades, because counterintuitively more data means less information. The more data you have, the more you can prove. But the more you prove, the less value your proof has and the higher chance that what you think is valuable information in reality is nothing but confirmation bias.

Always translate grades into probabilities

Because speedy decision making is so important in modern football, grades are often used to steer the club into action. Many clubs use a grading from A to E, whereas an A grade means try to immediately sign the player and an E grade means to never hire the player. The pitfall here is that scouts find it hard to see the difference between an A grade and a B grade. Or between a B grade and a C grade. Everyone sees the difference between an A graded player and an E graded player, but in reality a club never has to make that decision.

Another issue is that coming up with an A to E grade, or even a grade from 1 to 10, makes it hard to measure how well the scouts and decision makers at the club are at predicting the future development of the player. For these reasons, it is important as a club to know how to translate grades into probabilities. For instance in the following way:

1. An A grade means that you find it highly likely that this player will be a success at the club. The probability of success for this player in your estimation lies between 80% and 99%. Even if all the other scouts within the club are against hiring this player, you still think that the club ought to hire this player directly.
2. A B grade means that you find it likely that this player will be a success at the club. The probability of success for this player in your estimation lies between 60% and 79%. But you would only recommend the club to hire this player if there is no A graded player as an alternative AND the majority of the other scouts of the club have given this player an A or a B grade.
3. A C grade means that you find it unlikely that this player will be a success at the club. The probability of success for this player in your estimation lies between 40% and 59%. Even if most of the other scouts of the club are in favor of hiring this player, you still have your doubts and would see hiring this player as an emergency measure.
4. A D grade means that you find it highly likely that this player will be a success at the club. The probability of success for this player in your estimation lies between 20% and 39%. You feel that hiring this player is a mistake even as an emergency measure.
5. A E grade means that you find it highly likely that this player will be a success at the club. The probability of success for this player in your estimation lies between 1% and 19%. No matter what other scouts think about this player, for you it is clear: never hire this player.

When you use a system like this, this makes it easier for the scouts to give players the right grade. More importantly, this enables the club to measure how good the scouts are in predicting the future development of the player. For a club it is very important to know who the best predictors of the future development path of players are within the club. Once that is known it becomes possible to create a risk management model that uses the success at predicting the future development of players as weights so the club can actually calculate the risks and opportunities that a specific player has.

Potential or progression?

Grading players in terms of their potential is a pitfall. “Potential” means that something is possible. Statistically speaking everything you can think of has a chance of becoming real. In that sense every player has a potential of a 10. Much better than looking for future potential is looking at realized progression in the past and extrapolate that progression into the future. Besides grading players, also predict how their development path and career will continue from the moment of reporting on.