Is Kurt Warner a Hall of Fame quarterback? We dissect both sides of the case from multiple angles.
13 May 2004
by Aaron Schatz
Have you been wondering, "What does an unemployed pop culture analyst turned aspiring football analyst do with his days?" Well, Football Outsiders today presents the answer to this oft-asked question. Powered by the new Mission of Burma album and too much coffee, we're proud to unveil the latest upgrade to our DVOA (Defense-adjusted Value Over Average) ratings system that breaks the entire NFL season down play-by-play to determine how teams perform compared to the league average. (Need the basics? Read here.)
For those who are keeping track, this is the fourth major iteration of the system.
I'll explain the nuts-and-bolts of how this new version differs from the previous versions at the end of this article. But for those more concerned with results than with process, let's look at how the new version has changed the ratings of teams for 2003. Here are the top ten teams in offense, defense, and overall with both the new version (version 4.0) and the version that's been on the website since February (version 3.0):
In general, the new version of DVOA moves teams that you may have felt were ranked a bit too high or low in the previous version. The 49ers, Cowboys, and Raiders have all moved down. The Patriots, Packers, and Eagles have all moved up -- in fact, the Patriots now become the top team in WEIGHTED VOA, the stat that makes late-season games worth more than early-season games. "Packer Fan" will be happy to see that the Packer offense has had a significant boost in DVOA, and unhappy to see that the Packer defense still rates better than the offense, although now by a slimmer margin.
In an effort to frustrate me completely, Tampa Bay has moved even higher as well; no matter how much better I make the correlation between VOA and wins, the Bucs still "buck" the system and come out as one of the league's best teams last year. And Carolina hasn't really gotten any better -- there still is nothing in their early-season performance to predict the power of their playoff run.
The changes that the new version makes to 2002's ratings also make sense. One of the best aspects of this new version is that DVOA makes an even better predictor of future performance, with a better correlation from year-to-year. Good teams from 2002 that had bad years in 2003 have dropped in the 2002 ratings (Giants, Falcons, Saints).
Here's a look at how much more accurate the new version of DVOA is compared to version 3.0, the one we've been using since February. Here are a variety of correlation coefficients for the two versions of DVOA; remember, a correlation coefficient closer to 1.0 means that the two variables are more closely related. Also, I'll point out that when I'm checking the correlation of my ratings to points or wins, I'm using VOA, which is not adjusted for opponent, not DVOA -- after all, points and wins aren't adjusted for opponent, so they can't be compared to a number that is:
|VOA version 3.0||VOA version 4.0|
|correlation of VOA to wins (02-03)||.856||.870|
|correlation of VOA to point differential (02-03)||.948||.960|
|correlation of 2002 VOA to 2003 VOA||.412||.454|
|correlation of 2002 DVOA to 2003 DVOA||.434||.484|
The formula also correlates very well with actual success when you include 2001. (Yes, we now have 2001 DVOA numbers, and they will be appearing at some point -- though where is going to be a bit of a surprise.) The baselines for DVOA are still based on only 2002-2003, so the correlation isn't quite as strong when you include 2001. Oddly, however, the year-to-year correlation is actually better when you look at 2001-2002 along with 2002-2003. To show how DVOA is a better predictor of future performance than points or wins, I'll toss on the year-to-year correlation of point differential as well as wins:
|VOA version 3.0||VOA version 4.0|
|correlation of VOA to wins (01-03)||.839||.861|
|correlation of VOA to point differential (01-03)||.907||.925|
|year-to-year correlation VOA (01-03)||.465||.485|
|year-to-year correlation DVOA (01-03)||.490||.518|
|year-to-year correlation wins (01-03)||.278|
|year-to-year correlation point differential (01-03)||.351|
Houston is left out of the 2001-2002 correlations, of course. The next project is an overhaul of the special teams system, followed by an attempt to include 2001 numbers in the baselines -- although that might skew things a bit, because 2001 seems to be a less offense-driven year than 2002 or 2003.
If you wish to look at the new numbers, and comment on them, you'll find new tables for total team efficiency, as well as offense and defense. I'll be updating the player stats with the new formulas gradually over the next month or two.
For those who are interested in the specifics, here is a list of changes made for DVOA version 4.0: