by Aaron Schatz

Have you been wondering, "What does an unemployed pop culture analyst turned aspiring football analyst do with his days?"  Well, Football Outsiders today presents the answer to this oft-asked question.  Powered by the new Mission of Burma album and too much coffee, we're proud to unveil the latest upgrade to our DVOA (Defense-adjusted Value Over Average) ratings system that breaks the entire NFL season down play-by-play to determine how teams perform compared to the league average.  (Need the basics?  Read here.)

For those who are keeping track, this is the fourth major iteration of the system.

• v1.0 was the original, developed in early 2003 and presented on this site when we went online in July.
• v2.0 was introduced in Week 3 of 2003.  The major changes were moving the threshold for a "successful" play from 40% to 50% of needed yards on first down, and from 60% to 65% of needed yards on second down.
• v3.0 was introduced in February 2004 and created new baselines for league average that were based on both 2002 and 2003 instead of just 2002.  It also doubled the importance of special teams.

I'll explain the nuts-and-bolts of how this new version differs from the previous versions at the end of this article.  But for those more concerned with results than with process, let's look at how the new version has changed the ratings of teams for 2003.  Here are the top ten teams in offense, defense, and overall with both the new version (version 4.0) and the version that's been on the website since February (version 3.0):

In general, the new version of DVOA moves teams that you may have felt were ranked a bit too high or low in the previous version.  The 49ers, Cowboys, and Raiders have all moved down.  The Patriots, Packers, and Eagles have all moved up -- in fact, the Patriots now become the top team in WEIGHTED VOA, the stat that makes late-season games worth more than early-season games.  "Packer Fan" will be happy to see that the Packer offense has had a significant boost in DVOA, and unhappy to see that the Packer defense still rates better than the offense, although now by a slimmer margin.

In an effort to frustrate me completely, Tampa Bay has moved even higher as well; no matter how much better I make the correlation between VOA and wins, the Bucs still "buck" the system and come out as one of the league's best teams last year.  And Carolina hasn't really gotten any better -- there still is nothing in their early-season performance to predict the power of their playoff run.

The changes that the new version makes to 2002's ratings also make sense.  One of the best aspects of this new version is that DVOA makes an even better predictor of future performance, with a better correlation from year-to-year.  Good teams from 2002 that had bad years in 2003 have dropped in the 2002 ratings (Giants, Falcons, Saints).

Here's a look at how much more accurate the new version of DVOA is compared to version 3.0, the one we've been using since February.  Here are a variety of correlation coefficients for the two versions of DVOA; remember, a correlation coefficient closer to 1.0 means that the two variables are more closely related.  Also, I'll point out that when I'm checking the correlation of my ratings to points or wins, I'm using VOA, which is not adjusted for opponent, not DVOA -- after all, points and wins aren't adjusted for opponent, so they can't be compared to a number that is:

The formula also correlates very well with actual success when you include 2001.  (Yes, we now have 2001 DVOA numbers, and they will be appearing at some point -- though where is going to be a bit of a surprise.)  The baselines for DVOA are still based on only 2002-2003, so the correlation isn't quite as strong when you include 2001.  Oddly, however, the year-to-year correlation is actually better when you look at 2001-2002 along with 2002-2003.  To show how DVOA is a better predictor of future performance than points or wins, I'll toss on the year-to-year correlation of point differential as well as wins:

Houston is left out of the 2001-2002 correlations, of course.  The next project is an overhaul of the special teams system, followed by an attempt to include 2001 numbers in the baselines -- although that might skew things a bit, because 2001 seems to be a less offense-driven year than 2002 or 2003.

If you wish to look at the new numbers, and comment on them, you'll find new tables for total team efficiency, as well as offense and defense.  I'll be updating the player stats with the new formulas gradually over the next month or two.

For those who are interested in the specifics, here is a list of changes made for DVOA version 4.0:

• The red zone is now split in half, with goal-to-go plays compared to a different baseline than other red zone plays.
• In order to account for the difficulty of gaining that one last yard before the goal line, touchdowns now receive a bonus of 1.5 "success points."
• Turnovers are penalized an additional "success point" if they take place within 20 yards of either goal line.
• On top of all those changes that make the red zone more important, all red zone plays have their value (negative or positive) increased by 20 percent.
• All plays in the back zone (when a team is on its own 20-39 yard lines) have their value increased by 10 percent.  (Huh?  I'll explain the reason for this in an article next season which will update last year's "Maroon Zone" article.)
• Plays from a team's own 20-yard line are now included as part of the "back" zone instead of the "deep" zone.  As much as I wanted the red zone and the deep zone to be the same, it was silly to include the first down after a touchback in the same category as a down where the offense is pinned to its own goal line.
• First down threshold was moved back to 45%, which means nothing on 1st-and-10 but gives more value to moderate gains on 1st-and-15 or 1st-and-20.
• Second down threshold for success was moved back to 60%.
• Slightly more value for third down plays that gain significant yardage but fail to convert.
• Value of special teams was dropped back again, though they are still worth more than they were originally.
• Fixed an error that skewed results for plays with over 25 yards to go.

Posted by: Aaron Schatz on 13 May 2004

comments