Predicting which team will win in elite CrossFit competition is a tricky business.
Sometimes it seems a sure thing, as with Hack’s Pack dominating Games wins in 2012 and 2013. Sometimes you’re left slack-jawed, as with Team World’s upset of Team USA in last year’s CrossFit Invitational, or with Team Reebok East’s takedown of Rogue Fitness Red and Black in the first-ever CrossFit Team Series.
One reason these defeats surprise us is our intuition that fitter teammates make fitter teams.
But is it really that simple?
In this post, we use data from the 2014 Reebok CrossFit Games team competition to see whether our intuition is accurate. We then turn our findings to the near future, and predict the winner of this weekend’s CrossFit Invitational.
Do Fitter Teammates Make Fitter Teams?
To test our intuition, we’ve taken the 43 CrossFit Games teams, and sorted them based on the individual worldwide Open ranks of their six teammates. If the teams with the fittest teammates in the Open also did the best at the Games, and the least fit did the worst, then we can say that a team is simply the sum of its parts.
Let’s take a look.
On the x-axis we have each team’s expected rank at the CrossFit Games based on the fitness of its competitors in the Open. On the y-axis we have each team’s actual CrossFit Games rank.
Teammate fitness does have some relationship with team performance at the Games. CrossFit Invictus, CrossFit Conjugate and CrossFit Marysville finished on the podium at the CrossFit Games this past summer, and they were predicted to finish within the top 6. You can see them in the lower left, close to the diagonal line.
Yet there is also a fair degree of deviation from the line, meaning some teams did much better than expected, and some did much worse. It appears that there’s more to team performance than teammate fitness. But how much more?
To What Extent Does Teammate Fitness Affect Team Performance?
To answer that question, we built a statistical model. The idea of this model is that Games team performance is affected by several factors, perhaps like this:
Games Finish = Teammate Fitness + Other Factors (Strategy? + Chance? + Camaraderie? + Ambition?)
Over the whole Games field, each of those factors accounts for some percentage of the variation in Games finish, and their percentages add to 100 percent. Kind of like in the Fort Minor song: "This is 10 percent luck, 20 percent skill, 15 percent concentrated power of will ..."
How well our model fits the Open data allows us put a percentage on that “Teammate Fitness” portion.
We built models using Open data for the teams that competed in the 2012, 2013 and 2014 Reebok CrossFit Games.* For comparison, we also built models for those years predicting Games performance based on Open performance for the individual competitors at the CrossFit Games, like Rich Froning and Julie Foucher.
We found that 21 - 25 percent of Games team rank can be explained by teammate fitness, which suggests that the "other factor" plays a big role in team competition (75 - 79 percent). This is consistent with the spread-out pattern we saw in the plot above; if the points were tightly arranged around the line, that 25 percent would grow much closer to 100 percent.
By comparison, we found that 19 - 44 percent of Games individual rank can be explained by athlete fitness, which suggests that the "other factor" plays a smaller role in the individual competition (56 - 81 percent).
The Open is probably not a perfect measure of athlete or teammate fitness, since it’s months away from when the Games are contested, Open events have a different character than Games events, and many elite athletes are known to train through the Open. Because of this, the true percent contribution of teammate fitness to their team is likely somewhat higher than what we see here.
Our takeaway is that to a limited extent, fitter teammates do make fitter teams, but that their performance as a unit may be largely shaped by other, yet-to-be-determined, factors. Games teams appear to be considerably more than the sum of their parts.
What Does This Mean For the CrossFit Invitational?
Now that we’ve built this model relating teammate fitness to team performance, we can have some fun with it, turning it toward this weekend’s CrossFit Invitational.
We gathered the 2014 Open, Regional and Games finishes of each of the 16 competitors from the four CrossFit Invitational teams, including Australia, Canada, Europe and the USA.
For example, Jason Khalipa took second worldwide in the Open, ninth worldwide in the Cross-Regional Comparison, and third at this year's Games.
We averaged those finishes together for each athlete, and plugged them into our model. Now we could simulate virtual Invitationals, as many as we like, and look at the finish order.
We ran 1 million simulations, and here’s what we saw.
Out of the 1 million simulations, USA won approximately 540,000 times (54 percent). In the other 460,000 simulations, Europe, Australia or Canada won. So while USA is the favorite, there's still plenty of room for the other teams to take the title.
The European and Australian estimates might be a touch low, because Sam Briggs and Kara Webb’s numbers are based only on their Open and regional finishes--if they had finished in the top five at the Games, say, the gap between the teams would be narrower.
These results reflect an interesting signal in the data: all eight women in this year’s Invitational are very strong, with every single one having posted a worldwide top 10 in either the Open, regionals or the Games in 2014. So the teams are fairly evenly matched on the women’s side.
The men’s side is a different story. Froning and Khalipa are arguably the top two male CrossFit athletes on the planet, and while the six men on the other three teams are incredible, they have not performed at the ultra-elite level of the Americans this year. If the historical pattern we see from Games teams data is a guide, the Froning-Khalipa edge translates to about a 3:1 edge for USA over each of the other three teams, and even odds that USA beats all three teams for the overall win.
Will these teams act as the sum of their parts, or will some of those parts defy the numbers?
We'll have to watch to find out.
To keep this post readable, some of the gory details of the analysis have been omitted. For those who’d like to read more, a more detailed technical account of the analysis is provided here.
About the Authors
Mike Macpherson (@datawod) has been doing CrossFit since 2010, and analyzing Games data for about as long. He teaches genetics and statistics at Chapman University.
Megan Mitchell is a staff writer and editor at CrossFit.