You've just been awarded an NFL expansion team and must build your personnel department. How would you do it? Matt Waldman takes on the exercise.
08 Jul 2009
by Aaron Schatz
One of this offseason's big projects at Football Outsiders was another round of improvements on our DVOA formula. Today, with the release of Football Outsiders Almanac 2009, we also switch over the stats pages on our site from the old version of DVOA to the new version.
The biggest change in DVOA v6.0 is that offense and defense are no longer judged on the same baseline. This is related to the specific adjustments and additions made to the formula, particularly these three:
1) DVOA now includes more penalties. Originally, DVOA only included passes and runs. In 2007, with DVOA v5.0, we added defensive pass interference and began to count intentional grounding with yards lost (rather than just as a regular incomplete pass.) Now, we add in the two offensive penalties which have both the highest correlation to wins and the most consistency from year-to-year: false starts and delays of game. Yes, that's right, delay of game really has a higher effect on wins and losses than holding, offensive pass interference, or any number of other penalties. DVOA only counts false starts and delays of game on offense, so fourth down delays when a team is trying to draw the defense don't count, and neither do those lame penalties for when a guy spikes the ball after the play or breathes on the official wrong or whatever.
However, there is no correlation between the defense on the field and the frequency of false starts and delays of game. Therefore, we could only count these changes on the offensive side of DVOA. That required a big change: separate baselines for offense and defense. The average for "success points" in any given situation is now lower on offense because of the possibility of a penalty. This also means that offensive plays come in three categories: instead of just rushing and passing, there are also penalties. So on defense, all pass plays and run plays from 2002-2007 total to 0% DVOA, but on offense they do not, because you have to add in false starts and delays of game (which are all negative plays) to get 0% DVOA as the average. Therefore, you may notice that team DVOA for both pass offense and run offense will seem higher than in past years. (This is not an issue with defensive pass interference or intentional grounding, since we count both of those as pass plays.)
2) Once we split offense and defense, we were able to change the way we counted aborted plays such as dropped snaps and incomplete backwards lateral passes. These plays now only count as negative for the offense, and defenses do not receive a "bonus" just because the quarterback can't hold onto the snap. Defenses still get credit for fumbles when they actually strip the ball.
3) DVOA v6.0 improves the way we adjust for teams playing from behind or with a lead in the fourth quarter. These adjustments are different for offense and defense. We also added an adjustment for the final two minutes of the first half when the offense is not near field-goal range.
Other improvements in DVOA v6.0 include:
I should point out that there were also a number of adjustments that I tried but didn't include because they didn't improve the correlation of DVOA to wins in the current year or DVOA the following year. That includes adjustments based on climate rather than just indoors/outdoors, "blowout" adjustments for plays before the fourth quarter, and adjustments for home-field advantage. That last one really surprised me when it didn't actually improve correlation from year to year or from the first half of the season to the second half, but it didn't, so it doesn't go into the new DVOA.
For most teams, the new version of DVOA only makes a small change to their rating. The average team since 1995 saw its rating change by 2.7 percent. However, a handful of teams did see their ratings change significantly, and the biggest improvement belonged to a team from the 2008 season. The Jacksonville Jaguars had a total DVOA of -8.5% in the final ratings posted on FootballOutsiders.com in 2008. In the new DVOA v6.0, their total DVOA is 1.9%, more than 10 percentage points higher -- although because the old version of DVOA had a big gap between the 19th and 20th best teams of 2008, their rating actually only goes up two spots, from 22nd to 20th overall. Nonetheless, this higher DVOA rating helps to explain why our projection for Jacksonville in 2009 may be better than most fans expect.
Here's a look at which teams saw their DVOA ratings change the most with the new version of the system:
|DVOA Improved in v6.0||ccc||DVOA Declined in v6.0|
Some other changes in the new ratings that might interest you:
Along with the changes in DVOA, there are also a couple of changes in the offensive line/defensive front seven stats.
First, we've changed the definition of "Stuffed" to make it simpler. "Stuffed" now means exactly that -- it is simply the percentage of running back carries which are stopped at or before the line of scrimmage, no matter what the situation.
The second change is related to the increased usage of shotgun formations over the past couple years. Adjusted Line Yards are supposed to measure the quality of run-blocking. The New England Patriots led the league in ALY in 2007 and finished second in 2008, but does anyone think that the Patriots have the best run-blocking line in the league? No, part of the reason they run so well is that the defense always has to expect pass against their pass-heavy shotgun spread offense.
It turns out this is a general problem. No matter the team or the down-and-distance situation, rushing plays gain more yardage from shotgun. Obviously, the way teams defend the shotgun is leaving more room for runners, which leads us to overvalue the offensive lines.
|Runs in 2008, Shotgun vs. Other|
|3rd/4th with 1-3 to go||3.40||4.88|
|3rd/4th with 4+ to go||4.71||7.52|
As I said before, this shotgun advantage has nothing to do with the quality of the offensive line. As an example, here are the NON-ADJUSTED Line Yards numbers for five different offenses in 2008 -- good lines and bad ones, teams that used a lot of shotgun (Denver, New England, Kansas City) and teams that did not (New York Giants, Cincinnati).
|Non-Adjusted Line Yards in 2008,
Shotgun vs. Other
Therefore, Adjusted Line Yards now adjust for whether a team is in shotgun or standard formation, moving offenses that use a lot of shotgun down and moving offenses that don't use a lot of shotgun up. The changes are small, but -- to give a couple examples -- the Giants pass the Patriots in last year's Adjusted Line Yards standings, and while the Patriots still lead the league for 2007, their lead over the other teams is about one-third as large as it used to be.
There are also improvements in the way the ALY stats handle running out the clock in the fourth quarter, to match the changes in DVOA. With a fourth-quarter lead, the baseline for ALY drops each minute, starting when there are ten minutes left in the game (two-score lead) or five minutes left in the game (one-score lead). When a team is behind, the baseline goes up with each successive minute, with a steeper climb the farther the team is behind.
70 comments, Last at 22 Sep 2009, 12:22am by kcurtis83