General Play Type Analysis

By Michael Gertz
Thursday, July 5, 2012

Using standard box scores, it is difficult to judge many of the underlying principles behind why teams win games. Does defense really win championships? Is a high flying pass game the key to success in the modern era, or is the passing strategy too risky due to sacks and interceptions? Is an elite quarterback really worth 1/6th of a team's entire salary cap? The simplest way to answer these questions is to assume that the people being paid to make these decisions are making the right decisions. But general managers, coaches, and players all have routinely become media analysts, and when forced to use their opinions to make predictions, almost universally fail to compare to the accuracy of statisticians and oddsmakers.

A brief glance at ProFootballLogic's stats and ratings can answer a myriad of questions that insiders have failed to answer with certainty for years. Using our EPA stats, we can see exactly how important things such as passing, running, and turnovers are relative to each other. Utilizing our ratings, we can see which types of success are more likely to continue going forward.

Analysis from the Stats

Much of this analysis can be done even by ignoring individual teams and looking at league averages to get a sense of how the league as a whole operates. During the average NFL game, each offense nets about -3.5 points from interceptions, -3 points from all sacks including those with fumbles, and 10 points from all other pass plays without fumbles. They gain only about a half of a point on run plays that are not fumbled. They lose about a point due to pre-snap penalties and fumbled snaps, and another 2 points due to fumbles on run plays or after completions. Most other plays such as kickoffs, punts, and field goals average about neutral, as they are simply plays with basic outcomes and little decision making. A final point lost comes from running out of time to take advantage of field positions at the end of the half or game.

All negative points except those due to interceptions and sacks can be about equally assigned to both passing and running plays. The final tally leaves pass plays netting about 1.5 points per game while run plays on average lose 1.5 points per game. The 3 point difference between the two options may seem small, but keep in mind this is about equal to home field advantage and a 10% increase in the chance of winning a game. So teams should just pass the ball more than they do, right? Probably, but not for certain. The logic here gets rather complex, but in short it is proof that something is amiss. If both offenses and defenses were maximizing their percentages, passing and running would indeed both show equal returns, and net 0 points on average.

The classic excuse for rushing plays netting less yards is that teams run more often in short yardage situations. This is true, but this effect disappears when considering EPA instead of yards because the values of different situations account for this. Another excuse is that teams run to waste the clock when ahead late in the game. However, this is often cancelled out by the fact that their opponents will also be less efficient while passing against a defense geared to defend passes. Perhaps it is easier for a defense to sell out against the run than the pass, but our analysis suggests the inefficiencies of rushing hold up to about the same level even when only taking data from the first half of games.

An explanation for the offensive inefficiency could be risk aversion, since passing is more likely to create negative plays. Risk aversion in a classic economical sense is logical because many decisions must be made assuming a certain level of future wealth. In the NFL, decisions can not be made under assumptions of a future state of a game. Risk aversion only makes sense if a team is winning, and a losing team should favor added risk, meaning as a whole it cancels out. Even though it is illogical, risk aversion on average in the NFL probably does exist. Analysis has shown that even in sports, people are more motivated to avoid losing a game rather than try winning it. The public and media multiply this effect, as seen any time a coach decides to go for 4th and short, even if the odds are in his favor.

If teams begin to pass more, in the short term they will probably have more success. But what if defenses catch on and play more pass-oriented defense? In theory, this could actually create a scenario where offenses are less efficient. But it turns out the only way that scenario is possible is if defenses are already too run-oriented given the current offensive play selection ratios. If defenses are fundamentally mistaken in their strategy, in theory offenses could already be maximizing their long term gains.

An analogy for this could be a an offensive basketball player who is better at driving to their right. A defender in this case would be wise to cover him by staying slightly to the offensive player's right, to make the relative success rates of driving right and driving left equal. If met with a defender who plays him straight on, the offensive player will have more success going right, and thus would be better off going right more often, especially in critical situations. However, if the player exploits the right side too often, it may cause the defender (or other teams' scouts) to see the error in their ways.

Therefore, in theory, there is a possible scenario where given a flawed opponent, the offense could be best off by not exploiting his advantage to the point of equilibrium. While flawed defenses may be common in the NFL, it's hard to imagine that the majority of the blame doesn't fall on offenses that are afraid to go too pass heavy. The current success of the two-minute drill in the NFL is certainly an indication that a more heavy passing attack does not lead to diminishing returns. Considering that offensive play selection has slowly trended toward more passing in recent history, it will likely continue to going forward as teams inch toward the optimum strategy.

Analysis from the Ratings

We have established that passing is more efficient for offenses. But this is a league average that should not benefit one team over the next all that much. Better passing teams will be inclined to pass more, which can exploit the situation to an extent, but the much larger factor is the relative quality of teams at each play type. Even if passing was more efficient, if all teams passed about equally successfully, but had a large variation in run success, then a relatively strong run game would be the key to winning. So what really matters are the play types that lead to a large variation in success from one team to another.

In looking at our stats, it is possible to spot the play types that have the greater variations in net points generated among teams. For a better understanding though, we can look at our ratings, which account for how much past performance leads to future success in each play type. Some play types such as fumbles can have a big impact on a game, but not be indicative of future success due to a smaller sample size. Others such as many defensive categories can also create a wide gap between past performance and true quality, as defensive success is often the result of playing bad or underperforming opponents.

Our ratings are an indication of true quality going forward. Therefore, larger variations in them among teams for a certain play type are a sign of the more important aspects of NFL football. If being near the top of the league in one category can more than make up for being near the bottom in multiple other categories, then the first category is clearly the more valuable. It comes as no surprise that play types such as field goals and punts are less important than passing and running. But some results are rather striking. For instance, the age old cliche that defense and a strong running game win championships is dead wrong.

The truth is the exact opposite: a strong passing game is key. This should not be a great surprise, after all a look at recent Super Bowl champions reveals many more great quarterbacks than great running backs or defenses. Great defenses can win Super Bowls, but are usually complimented by good offenses. Meanwhile, great quarterbacks have won even with poor defenses. The more accurate cliche is that traditions die slowly. Most important decision makers in the NFL grew up during a time when passing was probably not as important, and struggle to adjust.

How exactly do the play types break down in terms of importance? Below is a list of average deviations from the mean for the main play types. This is a measure of how much quality among teams varies, meaning being good in the play types with more variation is more valuable than those with less variation. Average deviation here essentially represents how much better a team in the middle of the top half of the league (ranked 8th) is than an average team (ranked 16th) in terms of points per game. For instance, a team would rather be ranked 8th in passing offense and averge otherwise than ranked 8th in rushing offensive, passing defense, and rushing defense all, while average otherwise.

Passing offense - 2.7
Passing defense - 0.6
Rushing offense - 0.6
Rushing defense - 0.4
Interceptions offense - 0.4
Sacks offense - 0.3
Interceptions defense - 0.3
Punts kicking team - 0.2
Punts receiving team - 0.2
Kickoffs receiving team - 0.1
Sacks defense - 0.1
Kickoffs kicking team - 0.1
Field goals offense - 0.1

Like stated above, the passing categories do not include interceptions and sacks, and fumbles on run plays or after receptions are separated out. Those fumbles have a negligible average deviation, like the rest of the play types, meaning success in those situations is more luck than skill. We can see that a strong passing offense is by far key to being a great team. It does not always guarantee a great record every season because of the high amount of random variation in the NFL and the small number of games, but in the long run it is a huge advantage. It is possible for average passing teams to be good teams if they excel at several other play types. But bad passing teams usually require a large amount of luck over the course of the season just to make the playoffs. The sack defense number is surprisingly low. An explanation may be that good pass rushing pays off more from incompletions from hurries and teams having to use extra blockers than from actual sacks. Additionally, forced fumbles from sacks (the most effective sacks) may be more luck than skill on average.

Overall average deviations for offense and defense are 3.2 and 0.9 respectively, with overall team quality at 3.1. These average values are not simply a matter of addition of the categories within them, as for instance teams that are good at passing are not much more likely to also be good at other offensive play types. However for a team that is above the mean in multiple categories, the benefit is a sum of each.

Salary Cap Analysis

These values have a huge implication on how teams utilize their salary cap. It is very difficult to calculate similar values for each position rather than by team, but we can see in general which units of teams should be paid accordingly. A top 8 passing offense can make up for a bottom 4 overall defense and rushing offense, but most coaches and general managers probably would not believe it. Considering the above proof that coaches are not making optimal play calling decisions, and other statistical analyses showing suboptimal play calls on 4th downs, it is not a stretch to assume that perhaps GMs are not perfect either.

Further evidence lies in a basic correlation test using our player ratings and team ratings. Quarterbacks are the only position in the league that correlate to the overall quality of their team. If players were being compensated correctly, no position would show a correlation as teams acquiring stars at that position would have to spend less at other positions. It appears that quarterbacks, or at least great quarterbacks, are severely underpaid, while the rest of the positions are all resultingly slightly overpaid. Quarterback contracts appear to have continued on being based on past contracts and have not adjusted quickly enough to the importance of the QB in the modern NFL. Other positions may also very well be paid incorrectly, because our correlation tests are only effective enough to point out the most blatant scenarios such as QBs.

Just because quarterback is the most important position does not mean that teams with great QBs should always be the best. If teams were paying the offensive passing units appropriately, we would see that teams with great passing offenses would have terrible other units such that on average overall they were still an average team. But this is not currently the case. If teams were managing salaries properly, quarterback would still be the most important position, but having a great QB would not make a team more likely to be good because they would have so much less money to pay other players. As the NFL currently stands though, teams with the best QBs are usually the best teams. This is blatantly apparent to even the most novice football fan.