by Aaron Schatz
Football Outsiders Almanac 2012 introduced the latest iteration of our DVOA formula, which we are calling DVOA v7.0. As promised, for those of you who may not have bought the book, we want to explain the changes in the new rating and look at what the changes mean for historical ratings.
The biggest change in DVOA v7.0 is the normalization of DVOA ratings so that the league average for all years is now 0%. As you may know, offensive levels in the NFL have gone up and down over the years. Right now, the overall level of offense in the league is probably at its highest level of all-time. This had created a big problem in recent years. Because DVOA baselines were created using the years 2002-2007, recent years were significantly slanted towards offense, and most of the years we have broken down from the 1990s were slanted towards defense.
For example, in a league where the average is 0%, roughly the half the teams should be positive and the other half should be negative. However, using the previous version of DVOA, 18 teams in 2011 had positive offensive ratings, and 22 teams had positive (i.e. worse than average) defensive ratings. On the other hand, in 1993, only nine out of 28 teams had positive offensive ratings, and only nine teams had positive defensive ratings.
With the new DVOA v7.0, each year's ratings have been normalized so that the league average for team offense and team defense is 0%. These ratings include all plays, however, so the league averages for team passing and team rushing are not 0%. Because passing is more efficient than rushing, the average for team passing is almost always positive and the average for team rushing is almost always negative. The variables for normalization were created by looking at how the league average compared to our multi-year baseline by different down-and-distance combinations. To give an example, in a season where teams had trouble gaining yardage on first down but were better than usual converting short third downs, teams would see their first-down performance adjusted upwards but their third-and-short performance adjusted downwards.
Individual DVOA ratings have also been normalized using the same methods. Unlike with team ratings, the individual ratings only compare passes to other passes and runs to other runs, so the league average for individual passing is 0%, as are the league averages for rushing and the three separate league averages for receiving by wide receivers, tight ends, and running backs. We've also worked to normalize each year's special teams numbers so that the league total value for each element of special teams is close to 0.0 points. However, we don't yet have a method to perfectly normalize each year of special teams, so the league totals for special teams are generally off by somewhere between 0.0% and 0.5%. (Nevertheless, this is much lower than the previous year-to-year differences in the league average for special teams.)
We have kept "raw VOA" numbers so that we can still track how strong the overall offensive levels are in a particular season. Here's a closer look at how offense has developed over the past 21 seasons. This graphic shows you the "raw VOA" numbers for total defense, pass defense, and run defense over the last 21 seasons. (We've used defensive VOA for this graphic so we wouldn't need to worry about non-passing/rushing plays.) The dotted line shows you the gap between rushing and passing.
As you can see, offensive levels have been gradually rising for the last two decades, although there's a blip where offense jumped up a little extra in 1994-1995 and then came back down again. You can also see that passing has always been more successful than rushing, except in 1992 when they were virtually tied, and in general the gap between rushing and passing has been rising since about 2002.
Another interesting find is that the standard deviation of offensive DVOA is higher than the standard deviation of defensive DVOA in every season, although the ups and downs of the two aren't necssarily connected. Last year had a particularly high standard deviation on the offensive side of the ball -- that's what happens when you have three of the best offenses in NFL history in one season, far ahead of everyone else -- but a very low standard deviation on the defensive side of the ball.
Normalization of the ratings does not really improve the predictive ability of DVOA, or its correlation to wins. However, it makes our comparisons of current teams and players to past teams and players much more accurate. Very little changes when we're talking about total DVOA, since both offense and defense move up and down. Here's a list of the top 10 teams in DVOA history with both the previous and current versions of DVOA:
|Best Total DVOA, 1991-2011|
|DVOA v6.0||DVOA v7.0|
However, you can see the changes when we look at offense and defense. On the top offenses list, teams from the high-octane seasons of 2004 and 2011 move down a bit, and 2007 Patriots are now ahead of the 2010 Patriots as the best offense ever.
|Best Offensive DVOA, 1991-2011|
|DVOA v6.0||DVOA v7.0|
While on defense, the new version of DVOA boosts recent teams such as the 2008 Ravens and 2009 Jets while moving down teams from the late '90s.
|Best Defensive DVOA, 1991-2011|
|DVOA v6.0||DVOA v7.0|
[ad placeholder 3]
The new DVOA v7.0 also improves our ratings of players by properly moving replacement level around from year to year, in conjunction with our calculation of the league average. To give an example from last year, Jermaine Gresham in 2011 had a season which came very close to the baseline for tight ends. With our old method, this meant he had value above replacement level, 42 DYAR to be exact. However, tight ends had more success in 2011 than in any season we have tracked. The league-wide raw VOA for tight ends was 7.3%. Once we normalize, Gresham is now a below-average tight end, and in fact his total value comes in slightly below replacement level, at -8 DYAR. Because Gresham is targeted more than most other tight ends, this means his rank among tight ends in DYAR has dropped from 26th to 31st.
This change impacts our lists of the best and worst seasons of all-time. Recent passing seasons drop because passing is so strong overall right now. With DVOA v6.0, the 2011 Aaron Rodgers, Drew Brees, and Tom Brady all ranked in the all-time top ten for passing DVOA in a season (minimum 400 passes). With the new ratings, Brees' 2011 season has dropped to 17th and Brady's 2011 season has dropped to 25th. (They are still ranked fourth, fifth, and seventh in passing DYAR because quarterbacks now -- Brees and Brady more than Rodgers -- throw so many more passes than they did 15 years ago.) You can alo see some strong effects in rushing, where league-wide rushing averages were very low from 1994 to 1999. Here's a look at the top rushing DYAR seasons using both the old and new versions of DVOA.
|Best Running Back Seasons by Rushing DYAR, 1991-2011|
|DVOA v6.0||x||DVOA v7.0|
|2002||Priest Holmes||KC||538||x||1998||Terrell Davis||DEN||602|
|1998||Terrell Davis||DEN||535||x||1999||Stephen Davis||WAS||526|
|2003||Priest Holmes||KC||480||x||1997||Terrell Davis||DEN||526|
|1997||Terrell Davis||DEN||478||x||1995||Emmitt Smith||DAL||505|
|2000||Marshall Faulk||STL||473||x||2000||Marshall Faulk||STL||501|
|2005||Larry Johnson||KC||468||x||2002||Priest Holmes||KC||497|
|1995||Emmitt Smith||DAL||463||x||2005||Larry Johnson||KC||488|
|2006||LaDainian Tomlinson||SD||453||x||2003||Priest Holmes||KC||485|
|1999||Stephen Davis||WAS||450||x||1994||Emmitt Smith||DAL||461|
|2005||Shaun Alexander||SEA||449||x||2006||LaDainian Tomlinson||SD||460|
|2002||Clinton Portis||DEN||430||x||2005||Shaun Alexander||SEA||453|
|2004||Curtis Martin||NYJ||415||x||1997||Barry Sanders||DET||447|
I'll be running a lot more best and worst lists over the next couple weeks, highlighting where the new DVOA v7.0 has changed the rankings to give a more accurate balance between players before and after the turn of the century.
In addition to the normalization of each year's ratings, a few other changes were made as part of DVOA v7.0, as follows:
- Based on research showing that red-zone performance is much less consistent from year-to-year than overall performance, the red zone bonus for team DVOA was dropped from 25 percent on both sides of the ball to 20 percent on offense and five percent on defense. We considered removing the red zone bonus entirely, but found that these rates made DVOA slightly more predictive than no red zone bonus at all.
- The coefficient used to translate special teams "points over average" into DVOA was increased slightly.
- A fix was added so that fourth-down interceptions now have no penalty whatsoever in the final two minutes of a game. Other fourth-down interceptions are still penalized based on the distance of the pass, although fourth-down interceptions have a much smaller penalty than interceptions on first through third down.
- A number of fixes were made to older years of play-by-play, including a change where we have decided to count lateral passes as pass plays rather than running plays where we have that information.
We've also taken this opportunity to make a few changes on our tables:
[ad placeholder 4]
- The formulas for "estimated wins" and "Pythagorean wins" have each been changed slightly. The new "estimated wins" formula adjusts variance for team quality so that bad teams have more estimated wins if they have a high variance, while good teams have more estimated wins if they have a low variance. Pythagorean wins now use the "Pythagenport" equation which adjusts the exponent in the formula based on each team's offensive environment. You'll find an explanation of that new equation in this article.
- The team offense and team defense pages now include specific strength of schedule ratings for each unit. Team offense is ranked by the average defensive DVOA of opponent, and team defense is ranked by the average offensive DVOA of opponent.
- DPI plays and yards will be listed in the tables separately from other plays. Previously, we had a weird inconsistency where we were adding DPI plays and yards into the totals for quarterbacks, but not for receivers. (I'm talking here about the totals on stat pages; DPI counted for DVOA for everyone.) With the new tables, DPI will be listed separately. We still need to update this on most pages but you can see an example from the 2011 wide receivers.
- The Phoenix Cardinals from 1988-1993 are now listed as "PHX" rather than "ARI."
Another piece of good news is that we've finally solved the problems with our player page database. All player pages are now updated with 1991 and 2011 stats, and all stats from 1992-2010 are updated to the new DVOA v7.0 numbers. Defensive players now have charting stats updated for 2011. In addition, similarity scores for current players are updated so that the pages feature lists of the top ten similar players through 2011 rather than through 2010. Remember, FO Premium subscribers can see all historical player stats on player pages, and all ten most similar players for each one-, two-, and three-year span. Non-subscribers can only see the last three years of player stats and the top most similar player for each span.
The bad news is that not all the numbers on FO are fully updated to the new DVOA v7.0 yet. It takes a lot of work to run off lots and lots of ratings and make sure they all get properly updated on the site. Here are the changes we still need to make:
- The team DVOA splits in FO Premium are not yet updated to DVOA v7.0 and do not yet include 1991, partly so we could adjust the database to account for the new offensive/defensive schedule columns. Premium should be fully updated by the beginning of next week.
- Players whose careers ended in 1991 are not yet added to the FO player database and do not yet have player pages.
- On some positional stat pages, players are not all linked to their player pages because some links need to be added manually.
- Because of an early error in the table builder, some VOA numbers (the non-adjusted ratings) are listed incorrectly and need to be fixed. VOA ratings for 1992-1998 need to be fixed both on the positional stat pages and on player pages. VOA ratings for 1999-2011 are correct on positional stat pages but need to be fixed on player pages. However, all DVOA, DYAR, and YAR listings are correct.
- The new DPI columns still need to be added for all quarterback tables and most wide receiver and tight end tables.
- We haven't yet had a chance to run any DVOA numbers for the postseason, either for players or teams, except for 2011.
We'll let everyone know when each of these issues is resolved. And if you see any other errors that snuck through on the FO player pages, the team stat pages, or the positional stat pages, let us know.