How Variation Affects Outcomes

By Michael Gertz
Thursday, July 5, 2012

Random variation and luck in sports are concepts that everyone can relate to. We all understand that a bad team having a good day will beat a good team having a bad day. Sometimes fans and media try to assign reasons behind these variations (usually incorrectly) after the fact such as "they were more motivated after the loss last week," but we all agree that there are a number of factors affecting games that can not be predicted prior to games. At ProFootballLogic, we group these factors together as random variation. To estimate the odds that such factors will affect games to any specific extent, we use a normal probability distribution based on past game results.

As fans, we are inclined to exlaim that our favorite team got robbed by the refs or got unlucky when they lose. When they win, we like to act as though skill was the only factor. The truth is somewhere in between. Random variation and luck occur to some extent whether or not our favorite team wins. So where does the variation or luck come from? It comes from players' performance and decisions, coaching decisions or play calls, and referees' decisions.

No quarterback can throw a football to the same exact same location every time. The better QBs typically are more accurate on average, but all QBs have some games in which they are more accurate than others. Similar concepts exist at every position on the field. Player variations can also be mental. An offensive lineman will make better blocking decisions during one game than another. Often these slight variations will not impact a game greatly, but sometimes the combination of 22 players all showing minor variations can result in surprising outcomes.

Further adding to player variation is coaching variation. The belief that coaches can have a reliably positive or negative impact on a team is probably rather overblown in football. But coaching play calls can have an unpredictable impact on games. Often offensive or defensive play calls are generally more effective depending on an opponent's play call. Certain plays better attack or defend specific areas of the field or are more or less effective depending on the type of pass rush the defense sends. Coaches may be able to estimate how often their opponent will call certain types of plays, but they usually can't tell whether it will come on a certain play or in a certain quarter. Due to random combinations of play calls, outcomes can vary greatly in a game.

Finally, referees are certainly not perfect. They make incorrect calls at times, which means that during a game one team may receive more incorrect calls in their favor. Usually these incorrect calls are impossible to predict, but can still have an impact on the game. The exception of course is home field advantage. Many studies have shown that referees play a big part in this by favoring the home team. Because this aspect is predictable, home field advantage is factored into our ratings and projections as separate from random variation and luck.

How Bad Teams Beat Good Teams and Have Good Seasons

The blueprint for a bad team to upset a good team is simple. They simply need a bit of variation or luck from any or all of the forms listed above. If the overall difference in random variation can make up for the difference in quality, the worse team will win. We prefer to use the word variation because several of the factors above do not fall into the classic description of luck. If a team performs better than their average, we prefer to say that they played well that day rather than that they got lucky to have played well that day. That said, the latter description is equally valid because playing well during a more important game or drive usually is a lucky coincidence rather than evidence of clutchness.

This variation can show up in most aspects or play types of a football game. Teams can have a good day throwing the ball or running it, punting or kicking field goals. Certain types of plays are especially prone to be the result of variation rather than skill. In general, offensive success or failure is more than 50% the result of the offense than of bad or good defensive play by the opponent. Other plays such as fumbles and special teams touchdowns are usually more an aberration than result of skill as well, and are not likely to be repeated at the same level as other success. More about these scenarios can be found in our General Play Type Analysis article.

The most common way for a bad team to have a good season is, ironically, for them to be lucky at how they got lucky. The reverse is also true for good teams having bad seasons. Because variation or luck from one game to the next will often cancel out, it is very rare for a team to experience a large amount of positive variation or luck over the course of an entire season. The much more common way that variation affects season outcomes is that it happens to come at the most important times. Playing your best or getting lucky is much more valuable during close games than during blowouts because it is more likely to affect the outcome of the game. The odds that a bad team will happen to play well or get lucky during a few crucial games are much more than the odds that they will simply play well or get lucky in almost every game. Yet the impact of these scenarios is still close to equal.

The combination of playing a little above true quality, getting a little lucky, and happening to do those things during the most crucial games can lead teams to end a season with several more wins than they would acquire on average. The opposite also applies for underachieving teams. At ProFootballLogic, this idea is a fundamental product of our ratings model, and we are at the forefront of separating variation from true quality to better analyze teams in the NFL. Visit our Ratings and Standings to see these ideas in action.