Methodology for ranking the 15 best leagues in CONCACAF

Note: This article has been updated to support the most recent best 15 leagues in CONCACAF.

Thanks for checking out the methodology for ranking the top 15 leagues in CONCACAF. I will try to be as thorough and brief as possible.

The foundational assumptions

The first decision was to use major interleague tournaments in the CONCACAF region and apply the transitive property to the results. Basically this means that US Open Cup results count the same as CONCACAF Champions League results.

The next decision was to use individual club teams that represent the leagues in these tournaments, and their position in the standings to approximate the quality of the entire league. You will see how I adjust the index to make an assessment of the entire league as we walk through this.

The next decision was to use goal differential as the base metric. Obviously points earned or some system of rewarding teams for advancing are options, but I felt goal differential does a good job separating what is a wide talent level across the leagues. Points told don't tell enough of a story, as in the case of a 6-0 win compared with a 1-0, which gives important information about both the winner and the loser in the match - much more than just giving the winner 3 points and the loser 0 points would do. And as we'll see goal differential is correlated strongly with advancing in these tournaments. Goal differential it is.

The Tournament Goal Difference

The index essentially looks for interleague games in the CONCACAF Champions League, US Open Cup, CFU Caribbean Club Championship, and Copa MX and calculates the goal differential between all the leagues, using those individual teams as a proxy. However, when doing this you quickly run into sample size issues, especially amongst the smaller countries. To counter that I only looked at interactions of leagues and countries against Liga MX and MLS. They both have by far the most games played in these competitions and every country I scored has a team that played at least four games played against both MLS and Liga MX, for a minimum of eight games on which to base the index. In the end, all of the top 10 leagues had significantly more games than eight comprising their index.

The next step I took was to index the two main leagues against each other. I'll walk through an example and this example, and the same will follow for all countries indexed (with a few alterations I will note). In CCL play, Liga MX home teams have a +0.7 GD against other Liga MX teams. At home against MLS teams, Liga MX teams have a +1.7 GD. I subtract the two and get a -1.0 index for MLS. On the road Liga MX has a +0.1 GD against MLS teams, whereas MLS home teams enjoy at 0.6 GD advantage against other MLS teams. In that case MLS scores a road index of -0.7. The average of the home and away index results in a -0.9 score for MLS. That's how I derived the Tournament Goal Difference for MLS.

For the remaining countries I did that same math with leagues against both Liga MX and MLS (and then added 0.9 to the MLS index) and then averaged those two indexes together. That is how I collected all the data for CCL participating leagues.

For the CFU Caribbean Club Championship I anchored results off of Trinidad and Tobago since they have the most appearances in the CCL. This methodology was similar to how I handled the

Next I had to apply this logic to the US Open Cup and Copa MX, but these tournaments have two unique challenges that need to be addressed. First, it's certainly true that the top leagues take these tournaments less seriously, certainly early on (the same can even be said of the CCL but I believe to a lesser degree). To account for this, if there were enough games in later rounds to create an index, then I threw out the early rounds. This was definitely the case with USL who had a good number of quarterfinal appearances in the USOC and those games were preferred over the early round games.

The second issue is the differing levels of competition between the leagues. For example, the best Liga MX teams (or those at least playing in the CCL) do not also compete in Copa MX. So really that tournament is the Ascenco MX league against roughly the bottom 80% of Liga MX. The same can be said for the USOC. By the time NASL and USL teams are paired with MLS, they have a positively skewed selection of teams to play MLS. I attempted to deal with all of this. For now just be aware that for the Copa MX and USOC I did the same math as for the CCL competitors, and then made adjustments based on which part of the league was actually competing.

Adjusting for league depth

The CCL only showcases the top teams in each league, but a league analysis must have a method of examining the whole table. I looked at the final league tables over the last two campaigns, and for the CCL leagues, I added up the goal differential of the number of teams that qualify for the CCL. For Mexico that is four teams, for MLS that is four teams, for Costa Rica three, etc. I then calculated the average goal differential per game between that group of teams against the rest of the league. The home and away considerations vanish in this case because all teams have played an equal number of home games. I'll continue with the example of Liga MX and MLS. The top four Liga MX teams average a +0.4 goal differential per game against the other teams in the league. In MLS that number is +0.3. Therefore I subtract 0.1 from the MLS Goal Difference to account for the fact that there is slightly better quality depth in MLS.

Again, I applied this same math against all the leagues that have appeared in the CCL over the last four years. In the case of Copa MX and USOC I made further adjustments based on which part of the league table was likely to be playing in games against MLS. For example I assume the top 75% of teams in NASL end up playing average MLS teams. It's a small but important adjustment to make.

Total Goal Difference

Once I've gotten the Tournament Goal Difference calculated and ensured I have only counted games with enough sample size, then I add the league depth adjustment. Weak leagues will fall further in the rankings as the weaker club teams pull down the overall league. And that is basically the ranking.

This is something that is certainly repeatable for UEFA and CONMEBOL and the other confederations with enough sample size. I'm certain if someone doesn't beat me to it I will do the math at some point and write up another ranking.

Thanks for checking out this page. Here's the final table that lists the first 12 teams and breaks out the components.

Please leave any questions in the comments sections or ping me on twitter @jaredeyoung and I'll gladly answer them.