This post walks through the process we developed to rank the Top 500 soccer clubs in the CONCACAF region. As with our top 100 soccer clubs in the United States and Canada ranking the Elo rating system is what drives the ranking. However, this time we coupled the Elo system with the use of the Poisson distribution to set the crucial weights that the Elo system uses.
This rating includes the results of over 22,500 competitive games that occurred in over 35 countries going back to the 2011 season and through December of 2014. Only league games were used in the final ranking but games used in tournaments were used to set the initial weights and the weights of individual games.
The Elo rating system, developed by Hungarian-American mathematician Dr. Árpád Élő, is used by FIDE, the international chess federation, to rate chess players. In 1997 Bob Runyan adapted the Elo rating system to international football and posted the results on the Internet. Given how the soccer intelligentsia romanticizes a connection between soccer and chess, the use of this formula makes sense.
Footballdatabase.com uses Bob Runyan's methodology to rank club teams around the world. FIFA uses a modified version of Bob Runyan's adaptation to rank Women's International Soccer teams. The Elo system was adapted for soccer by adding a weighting for the kind of match, an adjustment for the home team advantage, and an adjustment for goal difference in the match result.
Each team starts with an initial score which sets an expectation for the quality of the team and the competition they will face. This initial weighting is critical to the final outcome, so we compiled hundreds of games between the teams and leagues to get the weights as justifiable as possible.
After each team has an initial weight, each game results in an adjustment to the team's score based on the following factors:
- The team's old rating
- The importance or weight of the match
- The goal difference of the match
- The result of the match including home field advantage
- The expected result of the match
The details of the actual score are below but first we'll begin with how the model works and adjustments we've made to make the ranking as accurate as possible.
Strengths of the Rating System
The advantage of the Elo system is that it is a thorough assessment of a team's historical results. The system uses the results of every league match, including the goal difference, and weights the outcome based on the importance and the expected outcome.
Weaknesses of the Rating System
The importance of each match is pre-determined by the rating designer. Therefore the weighting of each match could potentially have a subjective element embedded in the score.
In addition, each team must start with a score. That score is pre-determined by the rating designer based on the perceived strength of the team's schedule. Teams with a strong strength of schedule will start with a higher score. Teams with a lower strength of schedule will start with a lower score. The difference between these starting scores is an important and could be a subjective choice by the designer.
The goal difference factor as determined in the original adaptation was very strong. A team that wins a game by two goals will get 50% more points for that outcome than if they had won by 1 goal. A three goal difference gets 75% more than 1 goal difference.
Changes to the Elo System to Address Weaknesses
The key to a good statistical ranking is to create a solid estimate of the quality of the leagues the teams play in. The leagues strength is used to set the initial weights and game weights used by Elo. You can look at the methodology for how that was estimated here. The short version is that we used games from CONCACAF Champions League, US Open Cup and Copa MX tournaments drive the majority of the weights used here. We created a goal difference index to estimate what the expected goal difference that would occur if average teams from two leagues played each other on a neutral site. For example, if an average Liga MX team played an average MLS team on a neutral site, our model estimates that the Liga MX team would win on average by 1.0 goals.
The use of the Poisson distribution
But linking goal difference to the critical Elo initial weights is challenging. This is where the Poisson distribution comes in. Using Poisson, we can use the expected goals of the two leagues to drive a likelihood of winning percentage. This is a common distribution used to predict the outcome of soccer matches. Just input expected goals scored and voila, you can estimate the probability of all scoring outcomes occurring. Elo scores are used to predict winning percentage as well. So we can use Poisson and goal expectation to get a winning percentage and then take that winning percentage and back into initial weights.
Here is a table that walks through that process for a number of the leagues included.
|Starting Elo Weight||League||GDI vs. Liga MX||Win % vs. Liga MX|
|1020||Trinidad & Tobago||3.1||5%|
The initial and game weights of the 40 leagues included are as follows:
|Country Div||ELO Starting Weight||ELO Game Weight|
|Mexico - Liga MX||1550||30|
|Mexico - Ascenso MX||1425||26|
|USA & Canada - MLS||1400||25|
|Costa Rica - Primera||1335||23|
|Guatemala - Liga Nacional||1255||20|
|Panama - LPF||1320||22|
|Honduras - Liga Nacional||1310||22|
|USA & Canada - NASL||1300||22|
|Costa Rica - Asenco||1110||15|
|Trinidad & Tobago - T&T Pro League||1020||12|
|El Salvador - Primera Division||1185||18|
|Nicaragua - Primera Division||1150||17|
|USA & Canada - USL PRO||1180||18|
|Guatemala - Primera Division||1030||13|
|Jamaica - Premier League||1000||12|
|Puerto Rico - LNFPR||1000||12|
|Panama - Liga Nacional||1095||15|
|Haiti - Championnat National||1000||12|
|Guyana - Super League||950||10|
|Mexico - Segundo Division||1000||12|
|Antigua and Barbuda - Premier||800||5|
|Aruba - Division di Honor||800||5|
|Bahamas - Senior League||800||5|
|Barbados - Premier||800||5|
|Belize - Premier||800||5|
|Bermuda - Premier||800||5|
|British Virgin Islands - Premier||800||5|
|Cayman Islands - Premier||800||5|
|Cuba - Primera||800||5|
|Turks and Caicos Islands - Football League||800||5|
|Suriname - Hoofdklasse||800||5|
|St. Kitts and Nevis - Premier League||800||5|
|Martinique - Division d'Honneur||800||5|
|Guadeloupe - Division d'Honneur||800||5|
|Grenada - Premier Division||800||5|
|French Guiana - Division d'Honneur||800||5|
|Dominican Republic - Liga Mayor||800||5|
|Dominica - Premiere League||800||5|
|Curaçao - Sekshon Paga||800||5|
|Suriname - Eerste Klasse||770||4|
A couple of quick notes. Second divisions with the exception of Asenco MX, which had good data, was weighted 225 points less at the start. The low score of 800 was based on expected win percentage between Liga MX teams and these teams. The game weights are calculated by assuming all game weights are between 30 to 5 and that the initial weights range between 800 and 1550 and everything was extrapolated in between. Suriname's 2nd division was placed below these levels. Also, for playoff games in each league 10 points was added to the game weight.
This initial weight represents the value for R_o (below) for the team's first match in the database.
The Basic Calculation
The Elo system has one formula which takes into account the factors mentioned above.
The ratings are based on the following formula:
R_n = R_o + K*G (W - W_e)
R_n = The new team rating
R_o = The old team rating
K = Weight index regarding the tournament of the match
G = A number from the index of goal differences
W = The result of the match
W_e = The expected result
Goal Differential = G
The number of goals is taken into account by use of a goal difference index. G is increased by 25% if a game is won by two goals, and if the game is won by three or more goals by a number decided through the appropriate calculation shown below;
If the game is a draw or is won by one goal
G = 1
If the game is won by two goals
G = 1.25
If the game is won by three or more goals
G = (11+N)/10
Where N is the goal difference
Result of the Match = W
W is the result of the game (1 for a win, 0.5 for a draw, and 0 for a loss).
Expected Result of Match = W_e
W_e is the expected result (win expectancy with a draw counting as 0.5) from the following formula:
W_e = 1 / (10^(-dr/400) + 1)
Where dr equals the difference in ratings plus 100 points for a team playing at home. So dr of 0 gives 0.5, of 120 gives 0.666 to the higher ranked team and 0.334 to the lower, and of 800 gives 0.99 to the higher ranked team and 0.01 to the lower.
This formula is calculated for each team in each game and the resulting score is carried forward to set the expectation for each team's next match.
Calculating the Probability of Game Outcomes
The expected result formula can be used to predict the outcome of games between two scored teams.
Take an average USL PRO team with 1100 points at home against an average MLS team with 1400 points. Add 100 to the USL PRO team for home field advantage.
To calculate the USL PRO team's odds of winning (including a draw being worth .5 points) use the formula above
W_e = 1 / (10^(-dr/400) + 1)
The dr for the USL PRO team is 1200-1400 = -200. Plugging -200 into the equation yields a result of 24%. Plugging in 200 would yield the result of 76%, the odds of winning for the MLS side.