Writers of Pro Football Prospectus 2008

Methods To Our Madness

THE ULTRA-SHORT VERSION:

DVOA measures a team's efficiency by comparing success on every single play to a league average based on situation and opponent.

THE SHORT VERSION:

DVOA is a method of evaluating teams, units, or players. It takes every single play during the NFL season and compares each one to a league-average baseline based on situation. DVOA measures not just yardage, but yardage towards a first down: Five yards on third-and-4 are worth more than five yards on first-and-10 and much more than five yards on third-and-12. Red zone plays are worth more than other plays. Performance is also adjusted for the quality of the opponent. DVOA is a percentage, so a team with a DVOA of 10.0% is 10 percent better than the average team, and a quarterback with a DVOA of -20.0% is 20 percent worse than the average quarterback. Because DVOA measures scoring, defenses are better when they are negative. For more detail, read below.

Please feel free to contact us with questions and comments about our original statistics using the contact form.

DVOA EXPLAINED

One running back runs for three yards. Another running back runs for three yards. Which is the better run? This sounds like a stupid question, but it isn’t. In fact, this question is at the heart of nearly all of the analysis on Football Outsiders.

Several factors can differentiate one three-yard run from another. What is the down and distance? Is it third-and-2 or second-and-15? Where on the field is the ball? Does the player get only three yards because he hits the goal line and scores? Is the player’s team up by two touchdowns in the fourth quarter, and thus running out the clock; or down by two touchdowns, and thus facing a defense that is playing purely against the pass? Is the running back playing against the porous defense of the Raiders, or the stalwart defense of the Bears?

Conventional NFL statistics value plays based solely on their net yardage. The NFL determines the best players by adding up all their yards no matter what situations they came in or how many plays it took to get them. Now, why would they do that? Football has one objective -- to get to the end zone -- and two ways to achieve that -- by gaining yards and achieving first downs. These two goals need to be balanced to determine a player’s value or a team’s performance. All the yards in the world won’t help a team win if they all come in six-yard chunks on third-and-10.

The popularity of fantasy football only exacerbates the problem. Fans have gotten used to judging players based on how much they help fantasy teams win and lose, not how much they help real teams win and lose. Typical fantasy scoring further skews things by counting the yard between the one and the goal line as 61 times more important than all the other yards on the field (each yard worth 0.1 points, a touchdown worth 6). Let’s say Larry Fitzgerald catches a pass on third-and-15 and goes 50 yards but gets tackled two yards from the goal line, and then Andre Ellington takes the ball on first-and-goal from the two-yard line and plunges in for the score. Has Ellington done something special? Not really. When an offense gets the ball on first-and-goal at the two-yard line, they're expected to score a touchdown five out of six times. Ellington is getting credit for the work done by the passing game.

Doing a better job of distributing credit for scoring points and winning games is the goal of DVOA, or Defense-adjusted Value Over Average. DVOA breaks down every single play of the NFL season, assigning each play a value based on both total yards and yards towards a first down, based on work done by Pete Palmer, Bob Carroll, and John Thorn in their seminal book, The Hidden Game of Football. On first down, a play is considered a success if it gains 45 percent of needed yards; on second down, a play needs to gain 60 percent of needed yards; on third or fourth down, only gaining a new first down is considered success.

We then expand upon that basic idea with a more complicated system of “success points,” improved over the past few years with a lot of mathematics and a bit of trial and error. A successful play is worth one point; an unsuccessful play, zero points with fractional points in between (e.g., eight yards on third-and-10 is worth 0.54 “success points”). Extra points are awarded for big plays, gradually increasing to three points for 10 yards (assuming those yards result in a first down), four points for 20 yards, and five points for 40 yards or more. Losing three or more yards is -1 point. Interceptions occurring on fourth down during the last two minutes of a game incur no penalty whatsoever, but all others average -6 points, with an adjustment for the length of the pass and the location of the interception (since an interception tipped at the line is more likely to produce a long return than an interception on a 40-yard pass). A fumble is worth anywhere from -1.7 to -4.0 points depending on how often a fumble in that situation is lost to the defense -- no matter who actually recovers the fumble. Red zone plays get a bonus: 20 percent for team offense, five percent for team defense, and 10 percent for individual players. There is a bonus given for a touchdown, which acknowledges that the goal line is significantly more difficult to cross than the previous 99 yards (although this bonus is nowhere near as large as the one used in fantasy football).

(Our system is a bit more complex than the one in Hidden Game thanks to our subsequent research, which added larger penalties for turnovers, the fractional points, and a slightly higher baseline for success on first down. The reason why all fumbles are counted, no matter whether they are recovered by the offense or defense, is explained in FO Basics.)

Every single play run in the NFL gets a “success value” based on this system, and then that number gets compared to the average success values of plays in similar situations for all players, adjusted for a number of variables. These include down and distance, field location, time remaining in game, and the team’s lead or deficit in the game score. Teams are always compared to the overall offensive average, as the team made its own choice whether to pass or rush. When it comes to individual players, however, rushing plays are compared to other rushing plays, passing plays to other passing plays, tight ends to tight ends, wideouts to wideouts, and so on.

Going back to our example of the three-yard rush, if Player A gains three yards under a set of circumstances in which the average NFL running back gains only one yard, then Player A has a certain amount of value above others at his position. Likewise, if Player B gains three yards on a play on which, under similar circumstances, an average NFL back gains four yards, that Player B has negative value relative to others at his position. Once we make all our adjustments, we can evaluate the difference between this player’s rate of success and the expected success rate of an average running back in the same situation (or between the opposing defense and the average defense in the same situation, etc.). Add up every play by a certain team or player, divide by the total of the various baselines* for success in all those situations, and you get VOA, or Value Over Average.

The biggest variable in football is the fact that each team plays a different schedule against teams of disparate quality. By adjusting each play based on the opposing defense’s average success in stopping that type of play over the course of a season, we get DVOA, or Defense-adjusted Value Over Average. Rushing and passing plays are adjusted based on down and location on the field; passing plays are also adjusted based on how the defense performs against passes to running backs, tight ends, or wide receivers. Defenses are adjusted based on the average success of the offenses they are facing. (Yes, technically the defensive stats are actually “offense-adjusted.” If it seems weird, think of the “D” in “DVOA” as standing for “opponent-Dependent” or something.)

The final step in calculating DVOA involves normalizing each year's ratings. As you may know, offensive levels in the NFL have gone up and down over the years. Right now, the overall level of offense in the league is probably at its highest level of all time. Therefore, we need to ensure that DVOA in a given season isn't skewed by league environment.

For teams, DVOA is normalized so that league averages for offense and defense are 0%. (However, because pass plays are more efficient than run plays, league averages for team passing and team rushing are not zero.) For players, DVOA is normalized separately for individual passing, individual rushing, and the three individual receiving groups (wide receivers, tight ends, and running backs) so that the league average for each is 0%.

Of course, one of the hardest parts of understanding a new statistic is interpreting its scale. To use DVOA, you have to know what numbers represent good performance and what numbers represent bad performance. We’ve made that easy. In all cases, 0% represents league-average. A positive DVOA represents a situation that favors the offense, while a negative DVOA represents a situation that favors the defense. This is why the best offenses have positive DVOA ratings (last year, Denver led the league at +33.5%) and the best defenses have negative DVOA ratings (with Seattle number one in 2013 at -25.9%). In most years, the best and worst offenses tend to rate around ± 30%, while the best and worst defenses tend to rate around ± 25%. For starting players, the scale tends to reach roughly ± 40% for passing and receiving, and ± 30% for rushing. As you might imagine, some players with fewer attempts will surpass both extremes.

DVOA has three main advantages over more traditional ways to judge NFL performance. First, by subtracting defense DVOA from offense DVOA (and adding in special teams DVOA, which is described below), we can create a set of team rankings that's based on play-by-play efficiency rather than total yards. Because DVOA does a better job of explaining past wins and predicting future wins than total yards, it gives a more accurate picture of how much better (or worse) a team really is relative to the rest of the league.

Because it compares each play only to plays with similar circumstances, this advantage also applies vis-a-vis situational team rankings. The list of top DVOA offenses on third down, for example, is more accurate than the conventional NFL conversion statistic because it takes into account that converting third-and-long is more difficult than converting third-and-short, and that a turnover is worse than an incomplete pass because it eliminates the opportunity to move the other team back with a punt on fourth down. The same could be said about plays on fourth down or in the red zone.

Second, unlike formulas based on comparing drives rather than individual plays, DVOA can be separated into a myriad of splits (e.g., by down, by week, by distance needed for a first down, etc.). Therefore, we're able to break teams and players down to find strengths and weaknesses in a variety of situations. All Pittsburgh third downs can be compared to how an average team does on third down. Kevin Kolb and John Skelton can each be compared to how an average quarterback performs in the red zone, or with a lead, or in the second half of the game. This doesn't just give us a better idea of which team or player is better. More importantly, it helps us understand why they're better, and therefore allows us to offer prescriptions for improvement in the future.

Finally, a third advantage of DVOA is that normalization makes our comparisons of current teams and players to past teams and players (going back to 1989) more accurate than those based on traditional statistics like wins or total yards, as well as those based on more sophisticated metrics that aren't normalized (e.g., expected points added, passer rating differential, etc.). For instance, which Denver Broncos team had the better offense: the 2013 edition with Peyton Manning, or the 1998 club led by Terrell Davis? Going by total yardage (7,317 vs. 6,092) or even yards per play (6.3 vs. 5.9), it's not even a contest. The 2013 team were clearly better. However, this ignores the fact that the average NFL offense was much more pass-oriented, and thus more efficient, in 2013 than in 1998. If we take the difference in offensive environment into account by using DVOA, it turns out that the 1998 Broncos offense was slightly better relative to the rest of the league (34.5% to 33.5%).

*It should be noted that certain plays are included in DVOA for offense but not for defense. Other plays are included for both, but scored differently. This leads to separate baselines on each side of the ball. For instance

  • Only four total penalties are included. Two penalties count as pass plays on both sides of the ball: intentional grounding and defensive pass interference. The other two penalties are included for offense only: false starts and delay of game. Because the inclusion of these penalties means a group of negative plays that don’t count as either passes or runs, the league averages for pass offense and run offense are higher than the league averages for pass defense and run defense.
  • Aborted snaps and incomplete backwards lateral passes are only penalized on offense, not rewarded on defense.
  • Adjustments for playing from behind or with a lead in the fourth quarter are different for offense and defense, as are adjustments for the final two minutes of the first half when the offense is not near field-goal range.
  • Offense gets a slight penalty and defense gets a slight bonus for games indoors.

DYAR EXPLAINED

After using DVOA for a few months, we came across a strange phenomenon: well-regarded players, particularly those known for their durability, had DVOA ratings that came out around average. The reason is that DVOA, by virtue of being a percentage or rate statistic, doesn’t take into account the cumulative value of having a player producing at a league-average level over the course of an above-average number of plays. By definition, an average level of performance is better than that provided by half of the league and the ability to maintain that level of performance while carrying a heavy workload is very valuable indeed. In addition, a player who is involved in a high number of plays can draw the defense’s attention away from other parts of the offense, and, if that player is a running back, he can take time off the clock with repeated runs.

Let’s say you have a running back who carries the ball 300 times in a season. What would happen if you were to remove this player from his team’s offense? What would happen to those 300 plays? Those plays don’t disappear with the player, though some might be lost to the defense because of the associated loss of first downs. Rather those plays would have to be distributed among the remaining players in the offense, with the bulk of them being given to a replacement running back. This is where we arrive at the concept of replacement level, borrowed from our partners at Baseball Prospectus. When a player is removed from an offense, he is usually not replaced by a player of similar ability. Nearly every starting player in the NFL is a starter because he is better than the alternative. Those 300 plays will typically be given to a significantly worse player, someone who is the backup because he doesn’t have as much experience and/or talent. A player’s true value can then be measured by the level of performance he provides above that replacement level baseline, totaled over all of his run or pass attempts.

Of course, the real replacement player is different for each team in the NFL. In 2013, the second-string running back in Cleveland (Chris Ogbannaya) had a higher DVOA than the alleged starter (Willis McGahee), and the third-string running back (Edwin Baker) had a higher DVOA than either of them. Sometimes a player like Ryan Grant or Danny Woodhead will be cut by one team and turn into a star for another. On other teams, the drop from the starter to the backup can be even greater than the general drop to replacement level. (The Indianapolis Colts of 2011--the dark year between the Manning and Luck eras--will be the hallmark example of this until the end of time.) The choice to start an inferior player or to employ a sub-replacement level backup, however, falls to the team, not the starter being evaluated. Thus, we generalize replacement level for the league as a whole, as the ultimate goal is to evaluate players independent of the quality of their teammates.

Our estimates of replacement level were re-done during the 2008 season and are computed differently for each position. For quarterbacks, we analyzed situations where two or more quarterbacks had played meaningful snaps for a team in the same season, then compared the overall DVOA of the original starters to the overall DVOA of the replacements. We did not include situations where the backup was actually a top prospect waiting his turn on the bench, since a first-round pick is by no means a "replacement-level" player.

At other positions, there is no easy way to separate players into "starters" and "replacements," since unlike at quarterback, being the starter doesn't make you the only guy who gets in the game. Instead, we used a simpler method, ranking players at each position in each season by attempts. The players who made up the final 10 percent of passes or runs were split out as "replacement players" and then compared to the players making up the other 90 percent of plays at that position. This took care of the fact that not every non-starter at running back or wide receiver is a freely available talent. (Think of Jonathan Stewart or Randall Cobb, for example.)

As noted earlier, the challenge of any new stat is to present it on a scale that’s meaningful to those attempting to use it. Saying that Tony Romo's passes were worth 40 success value points over replacement in 2013 has very little value without a context to tell us if 40 is good total or a bad one. Therefore, we translate these success values into a number called "Defense-adjusted Yards Above Replacement, or DYAR. Thus, Romo was seventh among quarterbacks with 839 passing DYAR. It is our estimate that a generic replacement-level quarterback, throwing in the same situations as Romo, would have been worth 839 fewer yards. Note that this doesn’t mean the replacement level quarterback would have gained exactly 839 fewer yards. First downs, touchdowns, and turnovers all have an estimated yardage value in this system, so what we are saying is that a generic replacement-level quarterback would have fewer yards and touchdowns (and more turnovers) that would total up to be equivalent to the value of 839 yards.

(Note: Prior to the 2008 season, DYAR was translated in terms of points rather than yardage, and old articles will refer to these stats as "DPAR" instead.)

HOW CAN A 16-GAME SEASON BE SIGNIFICANT?

Football statistics can't be analyzed in the same way baseball statistics are. After all, there are only 16 games in a season. Baseball has over ten times more, and even the NBA and NHL offer over five times more. The more games, the more events to analyze, and the more events to analyze, the more statistical significance.

That is true, but the trick is to consider each play in an NFL game as a separate event. For example, Drew Brees played only 16 games in 2013, but in those 16 games he had 687 passing plays (including sacks) and 35 rushing plays (including scrambles) for a total of 722 events. Nick Markakis in 2013 played in 160 games and had 700 plate appearances. For the most part, a quarterback who plays a full season will have almost the same number of plays as a baseball hitter who plays in most of his team's games.

A running back will have fewer plays than a quarterback, and wide receivers and tight ends will have even fewer. But there should still be enough plays with most starting running backs and receivers to allow for analysis with some significance. As an example, LeSean McCoy ran the ball 314 times in 2013, and was the target of 64 passes (including incompletes), for a total of 378 plays. In general, a starting running back will have 375-450 plays over 16 games. Receivers are used a bit less, and therefore their stats are likely not as accurate. In general, starting wide receivers have 75-150 pass targets over a full season.

ISSUES WITH DVOA/DYAR

DVOA is limited by what’s included in the official NFL play-by-play or tracked by the Football Outsiders game charting project. Because we need to have the entire play-by-play of a season in order to compute DVOA and DYAR, these metrics are not yet ready to compare players of today to players throughout the league’s history. As of this writing, we have processed 25 seasons, 1989 through 2013, and we add seasons at a rate of roughly two per year (the most recent season, plus one season back into history.)

Football is a game in which nearly every action requires the work of two or more teammates -- in fact, usually 11 teammates all working in unison. Unfortunately, when it comes to individual player ratings, we are still far from the point at which we can determine the value of a player independent from the performance of his teammates. That means that when we say, "In 2013, Marshawn Lynch had a DVOA of 5.9%, what we are really saying is “In 2013, Marshawn Lynch, playing in Darrell Bevell’s offensive system with the Seattle offensive line blocking for him and Russell Wilson selling the fake when necessary, had a DVOA of 5.9%."

With fewer situations to measure, the numbers spread out a bit more, so you'll see more extreme DVOA ratings for part-time players and for measurements of teams in more specific situations (for example, passing on third downs). The charts listing players in order of DVOA have cut-offs for number of attempts, because players with just a handful of plays end up with absurd VOA and DVOA numbers. (In 2009, for instance, Tarvaris Jackson had a 75.7% passing DVOA in 21 plays.)

Passing statistics include sacks as well as fumbles on aborted snaps. Receiving statistics include all passes intended for the receiver in question, including those that are incomplete or intercepted. At some point, we hope to be able to determine just how much impact different receivers have on completes vs. incomplete passes, but various regression analyses make it clear that both quarterback and receiver have an impact on whether a pass is complete or not. The word passes refers to both complete and incomplete pass attempts.

Unless we say otherwise, all references to third down also include the handful of rushing and passing plays that take place on fourth down (primarily fourth-and-1).

DVOA FOR SPECIAL TEAMS

The problem with a system based on measuring both yardage and yardage towards a first down, of course, is what to do with plays that don't have the possibility of a first down. Special teams are an important part of football and we needed a way to add that performance to the team DVOA ranking. Our special teams metric includes five separate measurements: field goals (and extra points), net punting, punt returns, net kickoffs, and kick returns.

The foundation of most of these special teams ratings is the concept that each yard line has a different value based on how the likelihood of scoring changes with better field position. In Hidden Game, the authors suggested that the value of field position for the offense existed on a straight line with your own goal line being worth -2 points, the 50-yard line 2 points, and the opposing goal line 6 points. (-2 points isn't just the value of a safety; it also reflects the fact that when you are backed up in your own zone, you are likely going to see your drive stall, and you'll need to punt and give the ball to the other team in good field position. Thus, the defense is more likely to score next.) We use a more refined set of values based on our research, but the idea is the same.

Our special teams ratings compare each kick or punt to the league average for based on the point value of field position at the position of each kick, catch, and return. We've determined a league average for how far a kick goes based on the yard line from where the kick occurs (almost always the 35-yard line for kickoffs, variable for punts) and a league average for how far a return goes based on both the yard line where the ball is caught and the distance that it traveled in the air.

The kicking or punting team is rated based on net points compared to average, taking into account both the kick and the return if there is one. Because the average return is always positive, punts that are not returnable (touchbacks, out of bounds, fair catches, and punts downed by the coverage unit) will rate higher than punts of the same distance which are returnable. (This is also true of touchbacks on kickoffs.) There are also separate individual ratings for kickers and punters that are based only on distance and whether the kick is returnable, otherwise assuming an average return in order to judge the kicker separate from the coverage.

For the return team, the rating is based on how many points the return is worth compared to average, based on the location of the catch and the distance the ball traveled in the air. Return teams are not judged on the distance of kicks, nor are they judged on kicks that cannot be returned. As explained below, blocked kicks are so rare as to be statistically insignificant as predictors for future performance and are thus ignored.

Field goal kicking is measured differently. Measuring kickers by field goal percentage is a bit absurd, as it assumes that all field goals are of equal difficulty. In our metric, each field goal is compared to the average number of points scored on all field goal attempts from that distance over the past 15 years. The value of a field goal increases as distance from the goal line increases.

Kickoffs, punts, and field goals are then adjusted based on weather and altitude. It will surprise no one to learn that it is easier to kick the ball in Denver or a dome than it is to kick the ball in Buffalo in December. Because we do not yet have enough data to tailor our adjustments specifically to each stadium, each one is assigned to one of four categories: Cold, Warm, Dome, and Denver. There is also an additional adjustment dropping the value of field goals in Florida (because the warm temperatures allow the ball to carry better) and raising the value of punts in San Francisco (because of those infamous winds).

The baselines for special teams are adjusted in each year for rule changes such as the introduction of the special-teams-use-only “k-ball” in 1999 as well as the move of the kickoff line from the 35 to the 30 in 1994 and then back to the 35 in 2011. Baselines have also been adjusted each year to make up for the gradual improvement of kickers over the last two decades.

Once we’ve totaled how many points above or below average can be attributed to special teams, we translate those points into DVOA so the ratings can be added to offense and defense to get total team DVOA. As a final step, we then normalize special teams DVOA to reflect the league environment in a given year. It should be noted, however, that we don’t yet have a method to perfectly normalize each year of special teams, so the league average for special teams deviates from 0.0% in certain years, although not by more than 0.5%.

There are three aspects of special teams that have an impact on wins and losses, but don’t show up in the standard special teams rating because a team has little or no influence on them. The first is the length of kick¬offs by the opposing team, with an asterisk. Obviously, there are no defenders standing on the 35-yard line, ready to block a kickoff after the whistle blows. However, over the past few years, some teams have delib¬erately kicked short in order to avoid certain top return men, such as Devin Hester and Josh Cribbs. The special teams formula now includes adjustments to give teams extra credit for field position on kick returns if kickers are deliberately trying to avoid a return.

The other two items that special teams have little control over are field goals against your team, and punt distance against your team. Research shows no indication that teams can influence the accuracy or strength of field-goal kickers and punters, except for blocks. As mentioned above, although blocked field goals and punts are definitely skillful plays, they are so rare that they have no correlation to how well teams have played in the past or will play in the future, thus they are included here as if they were any other missed field goal or botched punt, giving the defense no additional credit for their efforts. The value of these three elements is listed separately as “Hidden” value.

Special teams ratings also do not include two-point conversions or onside kick attempts, both of which, like blocks, are so infrequent as to be statistically insignificant in judging future performance.

ADJUSTED LINE YARDS EXPLAINED

One exception to the use of DVOA/DYAR, and the use of "play success" instead of raw yardage, is the rating system for offensive and defensive lines. Actually, these are only measures of running plays, and of course the defensive numbers don't measure just the defensive line, but the whole front seven against the run.

One of the most difficult goals of statistical analysis in football is somehow isolating how much responsibility for a play lies with each of the 22 men on the field. Nowhere is this as obvious as the running game, where one player runs while up to nine other players -- including wideouts, tight ends, and fullback -- block in different directions. None of the statistics we use for measuring rushing -- yards, touchdowns, yards per carry -- differentiate between the contribution of the running back and the contribution of the offensive line. Neither do our advanced metrics DVOA and DYAR.

We have enough data amassed that we can try to separate the effect that the running back has on a particular play from the effect of the offensive line (and other offensive blockers) and the effect of the defense. A team might have two running backs in its stable: RB A, who averages 3.0 yards per carry, and RB B, who averages 3.5 yards per carry. Who is the better back? Imagine that RB A doesn't just average 3.0 yards per carry, but gets exactly 3 yards on every single carry, while RB B has a highly variable yardage output: sometimes 5 yards, sometimes -2 yards, sometimes 20 yards. The difference in variability between the runners can be exploited to not only determine the difference between the runners, but the effect the offensive line has on every running play.

We know that at some point in every long running play, the running back has gotten past all of his offensive line blocks. From here on, the rest of the play is dependent on the runner's own speed and elusiveness, combined with the speed and tackling ability of the defensive players. If Frank Gore breaks through the line for 50 yards, avoiding tacklers all the way to the goal line, his offensive line has done a great job -- but they aren't responsible for most of that run. How much are they responsible for?

For each running back carry, we calculated the probability that the back involved would run for the specific yardage on that play, based on that back's average yardage per carry and the variability of their yardage on every play. We also calculated the probability that the offense would get the yardage based on the team's rushing average and variability without the back involved in the play, and the probability that the defense would give up the specific amount of yardage based on its average rushing yards allowed per carry and variability. For example, based on his rushing average and variability, the probability in 2004 that Tiki Barber would have a positive carry was 80% while the probability that Giants would have a positive carry without Barber running was only 73%.

Yardage ends up falling into roughly the following combinations: Losses, 0-4 yards, 5-10 yards, and 11+ yards. In general, the offensive line is 20% more responsible for lost yardage than it is for yardage gained up to four yards, but 50% less responsible for yardage gained from 5-10 yards, and not responsible for yardage past that. Thus, the creation of Adjusted Line Yards.

Adjusted Line Yards take every carry by a running back and apply those percentages. (We don’t include carries by receivers, which are usually based on deception rather than straight blocking, or carries by quarterbacks, which are almost always busted passing plays unless they involve someone like Tim Tebow or Cam Newton.) Those numbers are then adjusted based on down, distance, situation, opponent and whether or not a team is in the shotgun. (Because defenses are generally playing pass when the quarterback is in shotgun, the average running back carry from shotgun last year gained 5.36 yards, compared to just 4.16 yards on other carries.) The adjusted numbers are then normalized so that the league average for Adjusted Line Yards per carry is the same as the league average for RB yards per carry (in 2013, 4.10 yards).

Runs are listed by the NFL in seven different directions: left/right end, left/right tackle, left/right guard, and middle. Further research showed no statistically significant difference between how well a team performed on runs listed middle, left guard, and right guard, so we also list runs separated into five different directions. Note that there may not be a statistically significant difference between right tackle and middle/guard either, but until we can research further (and for the sake of symmetry) we do still split out runs behind the right tackle separately.

These splits allow us to evaluate subsections of a team’s offensive line, but not necessarily individual linemen, as we can’t account for blocking assignments. We don't know when a guard is pulling and when a guard is blocking straight ahead. We know that some runners are just inherently better going up the middle, and some are better going side to side, and we can't measure how much that impacts these numbers. We have no way of knowing the blocking contribution made by fullbacks, tight ends, or wide receivers.

Other numbers we use to measure the running game:

  • Open Field Yards gives the portion of the team's rushing average gained after the first 10 yards of each run. So for a 10-yard run, no yards are counted; for a 15-yard run, five yards are counted; for an 80-yard run, 70 yards are counted. This number gives you an idea of how much of a team's running game was based on the breakaway speed of the running backs. A team with a low ranking in Adjusted Line Yards but a high ranking in Open Field Yards is heavily dependent on its running back breaking long runs to make the running game work. This number is not adjusted in any way.
  • Second Level Yards gives the portion of the team's rushing average gained 5-10 yards past the line of scrimmage. So for a five-yard run, no yards are counted; for a 10-yard run, five yards are counted; for an 80-yard run, still only five yards are counted. This number represents the midpoint between the contributions of a team's offensive line and its running back. A low ranking in Open Field Yards coupled with a high ranking in Second Level Yards indicates that the team's offensive line is opening holes, but its backs aren't proficient open-field runners. This number is not adjusted in any way.

  • Power success measures the success of specific running plays rather than the distance. This number represents how often a running attempt on third or fourth down, with two yards or less to go, achieved a first down or touchdown. Since quarterback sneaks, unlike scrambles, are heavily dependent on the offensive line, this percentage does include runs by all players, not just running backs. This is the only stat given that includes quarterback runs. It is not adjusted based on game situation or opponent.
  • Stuffed measures the percentage of runs that are stopped at or before the line of scrimmage.

DRIVE STATS

The stats section of our website also features drive stats compiled by Jim Armstrong. These stats are computed from NFL Drive Charts and are not adjusted for strength of schedule or situation. Take-a-knee drives at the end of halves are discarded. Drive stats are generally self-explanatory, giving each team's total number of drives as well as average yards per drive, points per drive, touchdowns per drive, punts per drive, and turnovers per drive, interceptions per drive, and fumbles lost per drive. LOS/Drive represents average starting field position (line of scrimmage) per drive from the offensive point of view. Drive stats are given for offense and defense, with NET representing simply offense minus defense.

A NOTE ON PLAY-BY-PLAY DATA

Our data may differ slightly from official NFL numbers due to discrepancies in different play-by-play reports. In addition, we've adjusted clock plays, with kneels no longer counting as rush attempts and spikes no longer counting as pass attempts. We also count most aborted snaps as passing plays, not rushing plays, unless the play-by-play specifies that the play was an aborted handoff.