On the heels of Andrew Luck's new megadeal, we look back at 2015 to see which quarterbacks earned their money, and which were overpaid.
01 Aug 2012
by Aaron Schatz
Football Outsiders Almanac 2012 introduced the latest iteration of our DVOA formula, which we are calling DVOA v7.0. As promised, for those of you who may not have bought the book, we want to explain the changes in the new rating and look at what the changes mean for historical ratings.
The biggest change in DVOA v7.0 is the normalization of DVOA ratings so that the league average for all years is now 0%. As you may know, offensive levels in the NFL have gone up and down over the years. Right now, the overall level of offense in the league is probably at its highest level of all-time. This had created a big problem in recent years. Because DVOA baselines were created using the years 2002-2007, recent years were significantly slanted towards offense, and most of the years we have broken down from the 1990s were slanted towards defense.
For example, in a league where the average is 0%, roughly the half the teams should be positive and the other half should be negative. However, using the previous version of DVOA, 18 teams in 2011 had positive offensive ratings, and 22 teams had positive (i.e. worse than average) defensive ratings. On the other hand, in 1993, only nine out of 28 teams had positive offensive ratings, and only nine teams had positive defensive ratings.
With the new DVOA v7.0, each year's ratings have been normalized so that the league average for team offense and team defense is 0%. These ratings include all plays, however, so the league averages for team passing and team rushing are not 0%. Because passing is more efficient than rushing, the average for team passing is almost always positive and the average for team rushing is almost always negative. The variables for normalization were created by looking at how the league average compared to our multi-year baseline by different down-and-distance combinations. To give an example, in a season where teams had trouble gaining yardage on first down but were better than usual converting short third downs, teams would see their first-down performance adjusted upwards but their third-and-short performance adjusted downwards.
Individual DVOA ratings have also been normalized using the same methods. Unlike with team ratings, the individual ratings only compare passes to other passes and runs to other runs, so the league average for individual passing is 0%, as are the league averages for rushing and the three separate league averages for receiving by wide receivers, tight ends, and running backs. We've also worked to normalize each year's special teams numbers so that the league total value for each element of special teams is close to 0.0 points. However, we don't yet have a method to perfectly normalize each year of special teams, so the league totals for special teams are generally off by somewhere between 0.0% and 0.5%. (Nevertheless, this is much lower than the previous year-to-year differences in the league average for special teams.)
We have kept "raw VOA" numbers so that we can still track how strong the overall offensive levels are in a particular season. Here's a closer look at how offense has developed over the past 21 seasons. This graphic shows you the "raw VOA" numbers for total defense, pass defense, and run defense over the last 21 seasons. (We've used defensive VOA for this graphic so we wouldn't need to worry about non-passing/rushing plays.) The dotted line shows you the gap between rushing and passing.
As you can see, offensive levels have been gradually rising for the last two decades, although there's a blip where offense jumped up a little extra in 1994-1995 and then came back down again. You can also see that passing has always been more successful than rushing, except in 1992 when they were virtually tied, and in general the gap between rushing and passing has been rising since about 2002.
Another interesting find is that the standard deviation of offensive DVOA is higher than the standard deviation of defensive DVOA in every season, although the ups and downs of the two aren't necssarily connected. Last year had a particularly high standard deviation on the offensive side of the ball -- that's what happens when you have three of the best offenses in NFL history in one season, far ahead of everyone else -- but a very low standard deviation on the defensive side of the ball.
Normalization of the ratings does not really improve the predictive ability of DVOA, or its correlation to wins. However, it makes our comparisons of current teams and players to past teams and players much more accurate. Very little changes when we're talking about total DVOA, since both offense and defense move up and down. Here's a list of the top 10 teams in DVOA history with both the previous and current versions of DVOA:
|Best Total DVOA, 1991-2011|
|DVOA v6.0||DVOA v7.0|
However, you can see the changes when we look at offense and defense. On the top offenses list, teams from the high-octane seasons of 2004 and 2011 move down a bit, and 2007 Patriots are now ahead of the 2010 Patriots as the best offense ever.
|Best Offensive DVOA, 1991-2011|
|DVOA v6.0||DVOA v7.0|
While on defense, the new version of DVOA boosts recent teams such as the 2008 Ravens and 2009 Jets while moving down teams from the late '90s.
|Best Defensive DVOA, 1991-2011|
|DVOA v6.0||DVOA v7.0|
The new DVOA v7.0 also improves our ratings of players by properly moving replacement level around from year to year, in conjunction with our calculation of the league average. To give an example from last year, Jermaine Gresham in 2011 had a season which came very close to the baseline for tight ends. With our old method, this meant he had value above replacement level, 42 DYAR to be exact. However, tight ends had more success in 2011 than in any season we have tracked. The league-wide raw VOA for tight ends was 7.3%. Once we normalize, Gresham is now a below-average tight end, and in fact his total value comes in slightly below replacement level, at -8 DYAR. Because Gresham is targeted more than most other tight ends, this means his rank among tight ends in DYAR has dropped from 26th to 31st.
This change impacts our lists of the best and worst seasons of all-time. Recent passing seasons drop because passing is so strong overall right now. With DVOA v6.0, the 2011 Aaron Rodgers, Drew Brees, and Tom Brady all ranked in the all-time top ten for passing DVOA in a season (minimum 400 passes). With the new ratings, Brees' 2011 season has dropped to 17th and Brady's 2011 season has dropped to 25th. (They are still ranked fourth, fifth, and seventh in passing DYAR because quarterbacks now -- Brees and Brady more than Rodgers -- throw so many more passes than they did 15 years ago.) You can alo see some strong effects in rushing, where league-wide rushing averages were very low from 1994 to 1999. Here's a look at the top rushing DYAR seasons using both the old and new versions of DVOA.
|Best Running Back Seasons by Rushing DYAR, 1991-2011|
|DVOA v6.0||x||DVOA v7.0|
|2002||Priest Holmes||KC||538||x||1998||Terrell Davis||DEN||602|
|1998||Terrell Davis||DEN||535||x||1999||Stephen Davis||WAS||526|
|2003||Priest Holmes||KC||480||x||1997||Terrell Davis||DEN||526|
|1997||Terrell Davis||DEN||478||x||1995||Emmitt Smith||DAL||505|
|2000||Marshall Faulk||STL||473||x||2000||Marshall Faulk||STL||501|
|2005||Larry Johnson||KC||468||x||2002||Priest Holmes||KC||497|
|1995||Emmitt Smith||DAL||463||x||2005||Larry Johnson||KC||488|
|2006||LaDainian Tomlinson||SD||453||x||2003||Priest Holmes||KC||485|
|1999||Stephen Davis||WAS||450||x||1994||Emmitt Smith||DAL||461|
|2005||Shaun Alexander||SEA||449||x||2006||LaDainian Tomlinson||SD||460|
|2002||Clinton Portis||DEN||430||x||2005||Shaun Alexander||SEA||453|
|2004||Curtis Martin||NYJ||415||x||1997||Barry Sanders||DET||447|
I'll be running a lot more best and worst lists over the next couple weeks, highlighting where the new DVOA v7.0 has changed the rankings to give a more accurate balance between players before and after the turn of the century.
In addition to the normalization of each year's ratings, a few other changes were made as part of DVOA v7.0, as follows:
We've also taken this opportunity to make a few changes on our tables:
Another piece of good news is that we've finally solved the problems with our player page database. All player pages are now updated with 1991 and 2011 stats, and all stats from 1992-2010 are updated to the new DVOA v7.0 numbers. Defensive players now have charting stats updated for 2011. In addition, similarity scores for current players are updated so that the pages feature lists of the top ten similar players through 2011 rather than through 2010. Remember, FO Premium subscribers can see all historical player stats on player pages, and all ten most similar players for each one-, two-, and three-year span. Non-subscribers can only see the last three years of player stats and the top most similar player for each span.
The bad news is that not all the numbers on FO are fully updated to the new DVOA v7.0 yet. It takes a lot of work to run off lots and lots of ratings and make sure they all get properly updated on the site. Here are the changes we still need to make:
We'll let everyone know when each of these issues is resolved. And if you see any other errors that snuck through on the FO player pages, the team stat pages, or the positional stat pages, let us know.
133 comments, Last at 06 Aug 2012, 5:45pm by Jerry