[warning: data] The Outliers -- 2018 Open results with normalized rankings

[braiser]

@spinny This is great write up thanks for taking the time. I have always thought that for the sort of statistical, I know it's not really statistical but the idea is represented, view CrossFit takes of fitness the way points are assigned is imperfect. It always bugged me in the earlier years of the games, I don't think it happens anymore, they would cut a portion of the field even though the best is determined by comparing placements across events. I know that isn't very connected but it's bugged me for a long time and I have needed to vent.

Thanks again interesting read.

 

[georgearriolamusic]

@spinny So there is a strategy how to win the Open if you are good at weightlifting: bring 1000+ weightlifters who are a bit worse than you but better than average crossfitter into 18.2A and voila: average crossfitter gets 1000+ points.

 

[aimee3915]

@spinny I get the point of alternative scoring methodologies to prevent one poor finish from killing your open season. However, any of your proposed changes make the assumption that they even want anyone that can clean less than 325 at regionals. I think single modality events like that are to prevent cardio body weight ninjas from breezing through the open.

The HSPU workout was unfortunate for Heppner, but it opened up a glaring weakness. No matter what anyone says, he was obviously able to execute the standard, but doing so consistently was his problem. Commendable for not accepting broreps like many of his competitors likely did, but he also wasn’t the only one who had to reevaluate the way they execute HSPUs through that weekend. That seems to be the only example where a single non 1RM performance is keeping anyone out of regionals.

 

[spinny]

@aimee3915 I completely agree that maybe there is some secret pre-requisite strength to make it to the next level. If that's the case, then they should make that the minimum work requirement for rx. Or they should increase the weights of the other events to further separate the athletes if they don't want body weight ninjas from destroying those workouts.

If you are already crushing rx+ workouts in your box, but don't win the 1RM scores because there is some beastly lifter, it's kind of the same thing. If you have the requisite strength to crush rx+ workouts with high intensity, where do you go from there?

 

[georgearriolamusic]

@aimee3915

However, any of your proposed changes make the assumption that they even want anyone that can clean less than 325 at regionals. I think single modality events like that are to prevent cardio body weight ninjas from breezing through the open.

Yes, crossfit HQ values weightlifting more than gymnastic and endurance.

 

[abdelas]

@spinny Very cool! I doubt HQ would consider changing the scoring of the open to reflect all of this but they might be convinced to drop individuals who don’t submit scores for certain workouts. That seems fair especially when weightlifters will often do just the weightlifting workout and don’t submit any other Open workouts.

 

[dawn16]

@spinny Fantastic, thank you for posting this.

 

[seventhtrumpetnow]

@spinny This is so great. And I'm not just saying this because I go from 114 to 63 in my region

 

[tryallthings]

@spinny Wow. This is awesome. I love it! It actually drops my rank by 6 positions, but I love it because I thought the 1RM gave me a huge advantage and was the main reason I was in the top 50 this year. But even with the normalized rankings, I would still be in the top 50. My goal going into the open was to be top 80 in the region, trying to climb to an individual spot by next year

 

[dawn16]

@spinny This is nicey done. Bravo. Now if you can figure out how to apply confidence intervals to account for likely judging and standard errors in this population and in our own scores, I think we'll actually get closer to a true measure of excercise racing ability.

 

[lyliarose]

@spinny How/why do some athletes have the same score using the JM method?

An example is West Cost, Sameul Kwant and Jason Carroll are both JM ranked 1 for 18.1, though Carroll scored 471 reps and Kwant scored 448.

 

[spinny]

@lyliarose When we get down to the 2 fittest athletes, Jason Carroll would have a 1 in 18.1 with 471 reps, Samuel Kwant would have a 2 with 448 reps. This is because there are only 2 athletes to compare that that point, and Samuel is worse than Jason for that one workout.

But then Jason Carroll has a higher total points (8 total points), so we drop him out to 2nd place overall and re-calculate Samuel Kwant. I'm not sure what Samuel's total would have been exactly at that point, either a 7 or 6.

Since Samuel himself at the lowest total points with nobody to re-rank individual events against, he gets a 1 in everything for a total points of 5.

Sorry, that was the math version of the explanation. The human explanation would be something like: Samuel is 1st place in that workout for people sharing his same overall fitness level, and Jason is 1st place in that workout for people sharing his same overall fitness level.

 

[autisticchristianknight]

@spinny hi not sure what Samuel's total would have been exactly at that point

 

[5teve]

@autisticchristianknight bad bot

 

[littlefox]

@spinny Nice post, but if I may suggest something:

For your last example it would have been helpful if you ordered the participants in alphabetical order in the first part, and then ordered them in their new order in the next part to help the reader more easily distinguish the changes in the ranking system.

Right now the order of entrants doesn't make a whole lot of sense when comparing before and after.

 

[spinny]

@littlefox Sorry, I was trying to order them by overall total points.

At one point I was going to say they go from AJHEFGCDIB to AHJEFGCDIB but that sounded like some weird crytpic code, and it's hard to see the ordering change. Since I was making up the examples as I went, and going from simple to complex the identification of each athlete was a little hard to keep track of and I apologize for that. Thanks for the suggestion.

 

[littlefox]

@spinny Very cool workup!

I'm still trying to wrap my head around this method. If I may ask you to generalize, what would categorize athletes that receive large rank increases vs. large rank decreases under your method?

 

[spinny]

@littlefox Someone who improves overall is a beast in all 5 workouts with a bad score in one workout. But the bad score is not actually that bad if you compare it to people of the same overall fitness level as them. It's a mis-represented score of their overall fitness.

People whose overall rank worsened were consistent in the 5 workouts, but weren't the absolute best in any single workout. They had just enough strength to do well in the 1RM workouts, and they had just enough speed to be do well in the speed workouts. That being said, there were other athletes who had a higher overall combination of strength and speed, but were held back by the current ranking system.

 

[uncertainthomas]

@spinny

And since they have already demonstrated fitness over him, can we also agree that his placement is no longer relevant to their scores?

First of all, I disagree that other scores aren't relevant because that's how CFHQ has decided to rank performance (as opposed to using the raw reps or times).

In your CEF example, why are you just dropping C's scores and not any of the other scores which are "no longer relevant"? By that same line of thinking, wouldn't all other scores also need to be removed (or at least all scores above either athlete)?

 

[spinny]

@uncertainthomas Sorry, maybe the wording is a bit tricky. Their "scores" are static (raw reps, lbs, time). The "placement" is how those scores relate to each other per workout (higher "score" means lower "placement").

In the

CEF

example, assume that we have already dropped all other athletes who have a higher total points than

[C]

, and this is our starting point. Also for the sake of that example, assume that these are the only 3 athletes being compared.

When we drop

[C]

, then the "scores" of

[E]

and

[F]

do not change. Consider workout 1:

[C] had 200 reps, placing 2nd

in workout 1

[F] had 150 reps, placing 15th

in workout 1

[E] had 100 reps, placing 20th

in workout 1

After dropping

[C]

[F]* had 150 reps, placing 14th

in workout 1

[E]* had 100 reps, placing 19th

in workout 1

Their absolute scores never change (number of reps), and their relative position to each other does not change (for this example). But their relative position to the next fittest athlete has changed.

[E]

used to be 2 points behind 13th, but now is only 1 point behind him.

[F]

used to be 2 points behind 18th, but now is only 1 point behind him.



If their placements were not equal for the other four workouts, then their overall points total might change and affect their overall ranking (as shown in the following example

GEF

).