The purpose of the comprehensive ranking is to predict how well an athlete might perform in a future competition.
Last fall, CrossFit held a one-day competition in London, England. The CrossFit Invitational pitted six American athletes against six European athletes.
Rich Froning, Matt Chan, Jason Khalipa, Julie Foucher, Kristan Clever and Rebecca Voigt represented the United States, while Numi Snaer Katrinarson, Mikko Aronpaa, Tuomas Vainio, Annie Thorisdottir, Sam Briggs and Katrin Davidsdottir represented Europe.
To the surprise of some Games fans, the athletes chosen for the CrossFit Invitational didn’t match the finishing order at the 2012 Reebok CrossFit Games.
While the two-time CrossFit Games champions, Rich Froning Jr. and Annie Thorisdottir, were obvious picks, others caused some viewers to pause. Why, for example, would Rebecca Voigt (10th in 2012) and Kristan Clever (fourth in 2012) receive invites while Talayna Fortunato (third in 2012) stayed home?
Two words for you: comprehensive ranking.
Instead of just taking the top finishers from the Games, the teams were selected by comparing past performances in standardized programming from recent years. Namely, how they performed in the Open, Regionals and Games for the last three seasons.
The CrossFit Games: Fittest on Earth
The CrossFit Games are the definitive test of fitness. Each season, the CrossFit Games take the concept of fitness and test athletes over three stages of competition: the Open, the Regionals and the Games.
“This is more than just a competition of athletes, this is a competition of training methodologies. It is the sport of fitness … The men and women competing (in the CrossFit Games) have legitimate claim to title fittest men and women on Earth, by the same rights a logic that the Lakers, Steelers, Penguins and Phillies are the best basketball team, football team, hockey and baseball teams on Earth,” Greg Glassman, CEO and Founder of CrossFit, said at the 2009 CrossFit Games.
CrossFit’s contention is that the fittest will perform statistically best across any set of measurable physical challenges (see Hopper Model in L1 participant handbook). The competitor’s fitness must be broad and general enough to allow them to perform well at anything thrown at them. Just like inside your affiliate, no one workout can accurately determine the fittest because of the variety of movements, loads and time domains that must be tested. For that reason, multiple workouts are required in order to really test the breadth and depth of an athlete's true fitness.
Throughout the CrossFit Games season, the true portrait of the Fittest on Earth becomes clearer with each stage of competition. The 2012 Open featured five scored workouts, with one workout posted per week. As each each week passed, the overall Open Leaderboard shifted and the best rose near the top. The top athletes advanced to one of 17 Regionals, where the slate was wiped clean and they faced six scored workouts within three days. At the Games, the final pool of athletes once again started evenly, and then tackled up to 14 scored workouts — more than any previous Games.
Few will dispute that Froning Jr. and Thorisdottir earned the title Fittest on Earth (both are repeat and defending champions of the Games). However, just looking at the final Leaderboard would miss the level of dominance both have risen to. Froning actually pulled off a hat trick by winning the 2012 Open, the Central East Regional and the Games. An impressive feat, which only one other competitor has ever accomplished: Thorisdottir in 2011. Froning and Thorisdottir’s ability to win at each stage clearly separates them from their peers, and their growing body of work suggests they will remain in the upper echelon in the near future.
Developing a comprehensive ranking
The Games performs the task of crowning the Fittest on Earth admirably. However, for selecting teams for exhibitions, we generally compare more than just their most recent position on the Games Leaderboard. In selecting for specialty events, CrossFit is not unique in looking back at past performances.
In men’s tennis, the ATP uses a rolling 52-week window to capture the performance of its best players. Professional golf uses a rolling two-year window to determine the Official World Golf Rankings. In both sports, major events are weighted more heavily and those fields of athletes are much more competitive.
It is important to note that the comprehensive rankings are statistical comparisons of athletes across multiple competitions, and they typically exist to offer some utility to the sport. For instance, the ATP uses their rankings to seed players in their tournaments, and the World Golf Rankings are used to determine entrance into golf’s four Majors and select teams for the Ryder Cup (other factors contribute, too). For international sports, this is particularly important since athletes may compete in different tours and have little head-to-head competition. These rankings don’t determine a champion, but rather they determine who gets to compete in select events and who gets the best starting position.
For the CrossFit Invitational, the teams were decided by selecting individuals who performed the best over the past three years of standardized CrossFit Games programming. This includes the Open, Regionals and Games from 2011 and 2012, and the Games from 2010. The Regionals and Sectionals from 2010 were excluded because they had different programming at each competition.
The formula we used to select the athletes was a preliminary one. As the sport matures and the top athletes compete in more shared events, we’ll continue to refine the formula to improve the ranking.
The first requirement was that an athlete must have competed in at least one CrossFit Games final in the last three years. Athletes received a maximum of 400 points for winning the Games in 2012, 200 points for winning in 2011 and 100 points for winning in 2010. Different points were awarded for each finishing position for each year they competed.
Athletes were awarded a maximum of 100 points for winning a 2012 Regional. Different points were awarded for first through fifth positions, but placing in sixth through 10th earned a common four points regardless of exact place. Finishing in 11th through 20th place earned a common two points. Finishing outside of the top 20 in Regionals earned no points for that stage. Points for 2011 were one half of those available for the present year.
Athletes earned a maximum of 100 points in the Open by finishing in the top 10 worldwide in 2012. They earned 50 points by finishing in 11th through 50th and 20 points for finishing in the top 100. Points were awarded all the way down to 2,500th place in the Open. There were half as many points available for 2011.
Beyond the numbers
The purpose of the comprehensive ranking is to predict how well an athlete might perform in a future competition. This prediction is based on past performance. The assumption, of course, is that past performances correlate to future performances. All sports fans know that the outcomes of any given competition are determined on the field of play, and upsets are common occurrences. So, while there’s no better way to predict, we hold no pretenses that these rankings are somehow fundamentally true.
What is true is that there are some athletes in our sport who perform near the top year after year. They are more likely to continue to perform at or near the top than a newcomer whose current finish is dramatically better than previous years. This doesn’t mean the newcomer won’t outperform; it just means there’s a greater chance they won’t repeat than someone with years of consistency.
One thing we’re not going to predict is exactly how this comprehensive ranking will be used in the future. There will be special events like the CrossFit Invitational, and some method will need to be used to select athletes. At some point, there might even be sponsorship opportunities or professional series based on them.
In the meantime, we’ll continue to refine the formula. Expect us to publish the formula and rankings shortly after the conclusion of this year’s CrossFit Games. The men and women at the top of the rankings will be invited to this year’s Invitational or other event(s) featuring the best of our sport. While each year’s Games performances will always carry the most weight, those on the bubble now have reason to push harder in each qualifying round.