How the Blindsided Rating System Works

What are we trying to measure?

We set about trying to define – based on the metrics available to us – how we'd define our perfect player. Each jersey in a team has a specific role to play, and we reasoned that it should be possible to define what those roles are. So 40% or a player's score should come from defence, and 60% from attack. And of that defense, a flanker is expected to win more turnovers than a second row, so a higher percentage of their defensive score should come from turnovers.

Using this approach we broke down our perfect player by position - defining roles based on the data metrics available to us.

What Data is collected?

In an effort to settle these disputes, we’ve turned to data that records a players performance to measure how well he played in a given game. Using records available from RugbyPass and other web sites over the years, we have built a database that records player, coach, referee and club performances by recording 54 different variables that measure performance.

How is the rating calculated?

For each jersey number, we built a profile which we considered how our idea player in that jersey should perform. For example, for a flanker, we theorised that 60% of his score should come from attacking data and 40% from defensive data.

Summing individual performance data on attack (metre’s run, offloads and carries) and defence (turnovers won, tackles and dominant tackles), we then subtract negative performance factors such as missed tackles and turnovers conceded. From this, we further subtract a percentage of a players score if he is penalised (8%), yellow carded (20%) or orange/red carded (50%).

We then measure this numerical score against all other numerical scores in our sample, and convert this to a percentage of the average score for a player wearing seven. So, if the players numerical score is equal to the average, he scores 100%. Because of our enormous dataset, we can closely define exactly what constitutes an average performance. We know how many tackles a second row makes in a game, and we can plot that average over time. Because the game is changing, we limit this average to games within the last five calendar years – it is a rolling average.

This shows how an individual player can score higher – carry more, and the sum increases, run more meters? The score increases. But there are also negative factors to a player’s performance. How do we account for penalties conceded?

Why don’t we rate players who feature for less than 40 minutes?

Reasoning that players are more than capable of pacing themselves, a prop may be told to go flat out for 30 minutes knowing that they will be subbed when exhaustion means performance dips. So we felt that a 40 minute performance was the minimum required to constitute “a full game”. Players may be injured early in the game and this would also distort the records.

Pitfalls in our method

We assume equitability between leagues, so we use the same method to score a player regardless of the league in which they play. Now although intra-league competitions exist in Europe, Super Rugby clubs and Japan Leage One clubs do not face teams from outside their league structure. We considered adding a weighting based on respective league strength, but decided that – at this stage – we value a global view over weighting performances in prospective leagues. We recognise this deficiency offers room for future improvement, and instead chose to generally compare players who appear in the same competition.

Clubs in France has a much larger fixture list that those in New Zealand. This requires a different type of player – perhaps one more resilient over the long haul rather than one able to peak in short bursts. Quantifying where the tipping point exists between these two types of players is beyond the current scope of our analysis.

Unit skills play an important part in rugby but we make no use of scrum success data. We decided that allocating scrum success rates amongst all eight members of the pack was something that was too complex a task to be built on any theoretical framework. For sure, a tight head maybe at the core of a strong scrum, but that tight head is nothing without the power of the second row. Whilst the scrum undoubtedly remains an integral part of the game, data shows that the number of scrums per game is falling.

Other facts affect performance such as weather and the quality of the surface on which the game is played. Once more, building a framework based on an arbitrary measure of weather or pitch quality was something we felt would undermine the objectivity of our model. We reasoned, that as our database is so large, over time, factors such as weather or surface quality would normalise and their effect would become negligible.

Our ratings aim to provide some objectivity to the analysis of player performance, but they do not measure the reserves a player has to play in excess of previous performance. In the comparison between form and class, we are measuring consistent form, not class or talent. Take a look at specific player records and decide for yourself whether there really is a difference between form and class.

How are we trying to improve on our analysis?

We’ve been collecting data since 1999, but our records are far from complete and we are consistently

Even though we use 54 points of performance data, our actual data analysis per player is based on a limited number of variables. This is because we simply have no access to the data. All the data we have comes from manual inputs per game. We would very much like to add more data and automate the data input process, but for this, we need resources.

As this project is a two-man team, we lack time and resources to accomplish this. For this reason, if you’d like to contribute to make a small financial contribution to our efforts, should we secure sufficient funds we will use this money to buy access to more detailed data which will improve our analysis.

Does this rating system work?

We’ll leave that for you to decide, but from our analysis we believe that the rating correlate fairly closely with more subjective viewpoints on which player truly rank as greatest of all time. Sadly, in Gareth Edwards’ playing days, analytical data was not collected so we still can’t pretend to objectively measure his relative performance records versus those of Antoin Dupont, but we still think GEO is the GOAT!