For any given set of athletes, total points earned over a series of workouts should determine the overall fittest athlete. With sufficiently designed workouts, the strongest athletes should excel in strength workouts; the fastest athletes should excel in speed workouts. Consider power, endurance, agility, accuracy, and so forth, and the scores should balance themselves out. While this may hold true for CrossFit Regionals and Games, this does not quite work for the Open due to the large number of athletes and disparate fitness levels.
Jacob Heppner is an example of a very high level athlete who scored 104 reps in workout 18.4, placing 1796th in that workout for the North Central region. Through 5 workouts
, a normalized ranking system would show that he has already demonstrated fitness over 1790 of those athletes, and would rank 6th instead of 200th.
Dakota Rager, in Central East, posted first place finishes in his region in 18.2 and 18.3. His 310lb clean, which seems like a decent amount of weight for someone his size, placed 295th in 18.2a. Through 5 workouts
, a normalized ranking system would show that he has demonstrated more fitness than 292 other athletes that can move heavier weight, and would rank 2nd instead of 36th.
Did you know that in Central East, 2906 athletes posted a score for 18.4, but posted a ZERO for at least one other workout? A normalized ranking system would rank athletes who DNF one or more workouts even lower than those who performed 1 rep in every workout. Graham Holmberg performed the minimum work requirement of 1 rep in 18.3, placing 7521st, but would place at most 4615th for that workout in a normalized ranking.
Nicolai Duus, in Europe North, had two first place finishes in 18.1 and 18.2. Not only were they first place in the region, but first place worldwide! He is currently ranked 53rd in the region, but through 5 workouts
a normalized ranking system would rank him 3rd in the region.
Brendan Willis had two first place finishes in North Central in 18.2 and 18.4, throwing down some very impressive scores. Considering the 5 workouts
, a normalized ranking system would rank him 2nd in the region instead of 42nd.
I'd be interested to hear of other outliers that seem like they should be ranked higher on the leaderboard. Maybe they dominate your local competitions, or are consistently at the top of the whiteboard in your box.
---How does the current Open ranking system not reward the best overall athletes?
Let’s look at an example from Central East (a very competitive region), where Scott Cottrill lifted 301 lbs to place 421st in 18.2a (1RM Clean). This means 420 athletes had a better score in that workout, so 420 points are added to his total. In order to be ranked in the top 20, this is a large number of points to be made up over the other workouts.
Actually though, 26 points are due to athletes who posted a score higher than 301 lbs, but failed to submit a score for at least one other workout. His normalized score improves from 420 to 394 in 18.2a... still a large number of points to be made up.
Additionally, 17 points are due to seventeen athletes who lifted 302 lbs, but after 5 workouts, Scott’s total points are 475
, while those seventeen range from 751 to 16013 total points. The normalized system does not penalize his single workout score for athletes who are less fit overall, and his normalized score then improves from 394 to 377 for that single workout.
Then improves further for the four athletes who lifted 303 lbs but are less fit overall, from 377 to 373, and so forth.
In reality, there are 411 athletes who lifted heavier but had more total points, and the normalized ranking system awards him 10 points for 18.2a instead of 421, ranking him 7th overall in the region instead of 50th.
--Why would you have to make up an unreasonable amount of points from 18.2a because so many other athletes are worse overall except the heavy lift?
The other thing to remember is that athletes below a certain ranking do not need to submit video evidence of their performance. If you are a regional level athlete, adhering to the judging standards with a video to prove it, how many places are you willing to lose to athletes who are just giving their word when they submit a score?
One point here and there, over 10,000 athletes, over 5 workouts, does make a difference.
The math is explained in detail below, but you can see a spreadsheet for each region here:
Africa Middle East
Asia
Australasia
Canada East
Canada West
Central America
Central East
Europe Central
Europe North
Europe South
Mid Atlantic
North Central
South America
South Central
South East
South West
West Coast
Each sheet has a column for the overall ranking by CrossFit (CF) and the normalized overall ranking (JM), along with the placements for each workout. It also shows the total points for each athlete through 5 workouts.
To see if you agree with the results, compare two athletes with similar normalized total points and look at their actual workout scores (on the right side of the sheet). You win some workouts, you lose some workouts. To see whether you won by a lot or a little, and how relevant that difference is, look at the normalized ranking for each individual workout. Some of the results made me scratch my head at first, but the math is there and it checks out. I can explain for any given athletes, why they are ranked they way they are, just ask.
---- WARNING: MATH ----
To demonstrate the math, let's compare some athletes and you tell me which is fitter.
I'll show 5 scores as the athlete's placement in each workout, and total points (sum of 5 workouts).
Wait... what? If you win more workouts shouldn't you win overall? When we say you "win" a workout, it's because your score is at least one point better than the next athlete. It's the same for total points. You only "win" overall if your total points are lower than the next athlete. We should still be able to agree on that.
OK, let's continue. Comparing some athletes we have already seen, in context with one another.
At this point, can we agree that
and
are fitter than
? And since they have already demonstrated fitness over him, can we also agree that his placement is no longer relevant to their scores? Meaning, in order to directly compare
, what would happen if
is dropped, and placements of the remaining athletes are re-calculated?Anyone placing higher than 2nd in workouts 1-3 would gain a position.
Big deal... they tied before, they tied after. I agree, this example is not terribly interesting. I'm demonstrating the most basic examples so that we can build up to the more complicated ones.
Let's throw a different athlete into the equation
So let's remove
and re-calculate
We don’t want to take anything away from
, whose points total was better than average. But since we decided that he is not as fit as the other two athletes, we don't want his score to interfere with their ranking so that we can directly compare the two of them. Remember from above, it only matters that you are fitter than the next athlete by one total point. It doesn't matter whether that translates to 10 minutes on the clock or 100 lbs on the barbell.
In other sports (basketball, baseball, football), teams are awarded a final ranking at the end of the season, based on a number of factors including record and strength of schedule, that is normalized in a way that makes sense. Think of these re-calculations as a sort of playoff system, where we start with the highest total points to find the least fit athlete, compare that athlete to the next athlete, and so on until we arrive at number 1.
I want to put one more example out there: Specialist vs Generalist
From my understanding, CrossFit aims to define the fittest athlete as having a good balance of strength and speed, someone who performs highly in every workout. It does not aim to reward the specialist who excels in just one thing. Conversely, I would argue that the generalist should not be penalized by the performance of the specialist.
In track and field, decathletes aren't penalized by dedicated runners, jumpers, shot putters, etc. Their scores are calculated only with respect to other decathletes. Similarly, CrossFit athletes should not be ranked in a workout against an athlete who only excels at lifting, or only excels at body weight movements. The scores should reflect the nature of the sport.
If we can agree with these statements, and the logic above, we should re-calculate:
Whoa... so now we're saying that
is actually fitter than we previously thought? Now that we think about it, the scores were actually quite good for 4 of the workouts. Maybe workout 3 was not great, but comparing total points to other athletes,
turned out to be fitter. It seems that the score for workout 3 was actually inflated by athletes that were calculated to be less fit overall.
We have shown that your ranking can actually improve by removing the scores of less fit athletes, which means that the normalized ranking might no longer reflect the actual difference in performance (time, reps, or lbs) for an individual workout. This is a change from the norm, but remember above we said it doesn't matter whether you win a workout by 10 minutes or 100 lbs, you win overall by placing one total point lower than the next athlete.
Note that it is also possible for your overall rank to be worse than the original. To be clear, it's not because you lose places in an individual workout. Your individual workout placement can only be the same or better than the original number. But your overall rank can be worse because others have moved up.
In a race series (auto racing, obstacle course racing, marathons), points are distributed for the first several places (maybe 10pts for 1st, 1pt for 10th), and then everyone else earns zero. This means that finishing last in one event still leaves a shot at the title if you can win out the rest of the series. Even with a sufficently large field of competitors, you are not given an unreasonable number of negative points that can never be made up within the series.
Even in CrossFit Regionals and Games, with a limited number of athletes, there are point disparities that cannot be made up because of the difference awarded for the top score vs the bottom score for a single event.
By re-calculating the rankings in this manner, the total points become much more competitive, and there are no longer hundreds of points between single positions.
Oh, and by the way, I've also noticed that CrossFit does not severely penalize the overall ranking of athletes in the Open who failed to submit a score for one of the workouts. A good ranking system should sort those athletes to the bottom of the list, below the athlete that performed 1 rep of each workout. We can compare athletes who did not finish a workout to other athletes that did not finish a workout, then drop them out as less fit than the athlete who performed the minimum work requirement, and re-calculate accordingly.
---- Putting it all together ----
Now let's put all the athletes together. Of course this is a limited data set with made-up numbers, and only 9 of 50 athletes are accounted for in the example calculations. But we can see how it works, and what it could mean for real data.
If you are the best at everything, you are still the best at everything. If you are the worst at everything, you are still the worst at everything. If you fall anywhere in between, then you will be ranked appropriately, and the total points will more accurately represent the overall fitness level of each athlete with respect to the overall fitness level of every other athlete.
Jacob Heppner is an example of a very high level athlete who scored 104 reps in workout 18.4, placing 1796th in that workout for the North Central region. Through 5 workouts
Code:
(placing 5-4-122-3-1796)Dakota Rager, in Central East, posted first place finishes in his region in 18.2 and 18.3. His 310lb clean, which seems like a decent amount of weight for someone his size, placed 295th in 18.2a. Through 5 workouts
Code:
(placing 18-1-295-1-20)Did you know that in Central East, 2906 athletes posted a score for 18.4, but posted a ZERO for at least one other workout? A normalized ranking system would rank athletes who DNF one or more workouts even lower than those who performed 1 rep in every workout. Graham Holmberg performed the minimum work requirement of 1 rep in 18.3, placing 7521st, but would place at most 4615th for that workout in a normalized ranking.
Nicolai Duus, in Europe North, had two first place finishes in 18.1 and 18.2. Not only were they first place in the region, but first place worldwide! He is currently ranked 53rd in the region, but through 5 workouts
Code:
(placing 1-1-404-8-114)Brendan Willis had two first place finishes in North Central in 18.2 and 18.4, throwing down some very impressive scores. Considering the 5 workouts
Code:
(placing 22-1-513-1-21)I'd be interested to hear of other outliers that seem like they should be ranked higher on the leaderboard. Maybe they dominate your local competitions, or are consistently at the top of the whiteboard in your box.
---How does the current Open ranking system not reward the best overall athletes?
Let’s look at an example from Central East (a very competitive region), where Scott Cottrill lifted 301 lbs to place 421st in 18.2a (1RM Clean). This means 420 athletes had a better score in that workout, so 420 points are added to his total. In order to be ranked in the top 20, this is a large number of points to be made up over the other workouts.
Actually though, 26 points are due to athletes who posted a score higher than 301 lbs, but failed to submit a score for at least one other workout. His normalized score improves from 420 to 394 in 18.2a... still a large number of points to be made up.
Additionally, 17 points are due to seventeen athletes who lifted 302 lbs, but after 5 workouts, Scott’s total points are 475
Code:
(placing 11-35-421-3-5)Then improves further for the four athletes who lifted 303 lbs but are less fit overall, from 377 to 373, and so forth.
In reality, there are 411 athletes who lifted heavier but had more total points, and the normalized ranking system awards him 10 points for 18.2a instead of 421, ranking him 7th overall in the region instead of 50th.
--Why would you have to make up an unreasonable amount of points from 18.2a because so many other athletes are worse overall except the heavy lift?
The other thing to remember is that athletes below a certain ranking do not need to submit video evidence of their performance. If you are a regional level athlete, adhering to the judging standards with a video to prove it, how many places are you willing to lose to athletes who are just giving their word when they submit a score?
One point here and there, over 10,000 athletes, over 5 workouts, does make a difference.
The math is explained in detail below, but you can see a spreadsheet for each region here:
Africa Middle East
Asia
Australasia
Canada East
Canada West
Central America
Central East
Europe Central
Europe North
Europe South
Mid Atlantic
North Central
South America
South Central
South East
South West
West Coast
Each sheet has a column for the overall ranking by CrossFit (CF) and the normalized overall ranking (JM), along with the placements for each workout. It also shows the total points for each athlete through 5 workouts.
To see if you agree with the results, compare two athletes with similar normalized total points and look at their actual workout scores (on the right side of the sheet). You win some workouts, you lose some workouts. To see whether you won by a lot or a little, and how relevant that difference is, look at the normalized ranking for each individual workout. Some of the results made me scratch my head at first, but the math is there and it checks out. I can explain for any given athletes, why they are ranked they way they are, just ask.
---- WARNING: MATH ----
To demonstrate the math, let's compare some athletes and you tell me which is fitter.
I'll show 5 scores as the athlete's placement in each workout, and total points (sum of 5 workouts).
Wait... what? If you win more workouts shouldn't you win overall? When we say you "win" a workout, it's because your score is at least one point better than the next athlete. It's the same for total points. You only "win" overall if your total points are lower than the next athlete. We should still be able to agree on that.
OK, let's continue. Comparing some athletes we have already seen, in context with one another.
At this point, can we agree that
Code:
[E]
Code:
[F]
Code:
[C]
Code:
[E] vs [F]
Code:
[C]Big deal... they tied before, they tied after. I agree, this example is not terribly interesting. I'm demonstrating the most basic examples so that we can build up to the more complicated ones.
Let's throw a different athlete into the equation
So let's remove
Code:
[G]We don’t want to take anything away from
Code:
[G]In other sports (basketball, baseball, football), teams are awarded a final ranking at the end of the season, based on a number of factors including record and strength of schedule, that is normalized in a way that makes sense. Think of these re-calculations as a sort of playoff system, where we start with the highest total points to find the least fit athlete, compare that athlete to the next athlete, and so on until we arrive at number 1.
I want to put one more example out there: Specialist vs Generalist
From my understanding, CrossFit aims to define the fittest athlete as having a good balance of strength and speed, someone who performs highly in every workout. It does not aim to reward the specialist who excels in just one thing. Conversely, I would argue that the generalist should not be penalized by the performance of the specialist.
In track and field, decathletes aren't penalized by dedicated runners, jumpers, shot putters, etc. Their scores are calculated only with respect to other decathletes. Similarly, CrossFit athletes should not be ranked in a workout against an athlete who only excels at lifting, or only excels at body weight movements. The scores should reflect the nature of the sport.
If we can agree with these statements, and the logic above, we should re-calculate:
Whoa... so now we're saying that
Code:
[H]
Code:
[H]We have shown that your ranking can actually improve by removing the scores of less fit athletes, which means that the normalized ranking might no longer reflect the actual difference in performance (time, reps, or lbs) for an individual workout. This is a change from the norm, but remember above we said it doesn't matter whether you win a workout by 10 minutes or 100 lbs, you win overall by placing one total point lower than the next athlete.
Note that it is also possible for your overall rank to be worse than the original. To be clear, it's not because you lose places in an individual workout. Your individual workout placement can only be the same or better than the original number. But your overall rank can be worse because others have moved up.
In a race series (auto racing, obstacle course racing, marathons), points are distributed for the first several places (maybe 10pts for 1st, 1pt for 10th), and then everyone else earns zero. This means that finishing last in one event still leaves a shot at the title if you can win out the rest of the series. Even with a sufficently large field of competitors, you are not given an unreasonable number of negative points that can never be made up within the series.
Even in CrossFit Regionals and Games, with a limited number of athletes, there are point disparities that cannot be made up because of the difference awarded for the top score vs the bottom score for a single event.
By re-calculating the rankings in this manner, the total points become much more competitive, and there are no longer hundreds of points between single positions.
Oh, and by the way, I've also noticed that CrossFit does not severely penalize the overall ranking of athletes in the Open who failed to submit a score for one of the workouts. A good ranking system should sort those athletes to the bottom of the list, below the athlete that performed 1 rep of each workout. We can compare athletes who did not finish a workout to other athletes that did not finish a workout, then drop them out as less fit than the athlete who performed the minimum work requirement, and re-calculate accordingly.
---- Putting it all together ----
Now let's put all the athletes together. Of course this is a limited data set with made-up numbers, and only 9 of 50 athletes are accounted for in the example calculations. But we can see how it works, and what it could mean for real data.
If you are the best at everything, you are still the best at everything. If you are the worst at everything, you are still the worst at everything. If you fall anywhere in between, then you will be ranked appropriately, and the total points will more accurately represent the overall fitness level of each athlete with respect to the overall fitness level of every other athlete.
