Looking back at FEI's preseason projections, we find that most teams did about what they were supposed to do -- but not in the Big Ten, where things got screwy.
08 Jun 2004
by Aaron Schatz
We've almost reached Football Outsiders' one-year anniversary, and in the year we've been doing this website there have been a few themes that return again and again. Strength of schedule is important. Field position is fluid. Rushing is better than passing in short-yardage situations unless your name is "McNair." But it looks like the idea we're going to have to mention the most, our version of Bill James' crusade against empty batting averages, is this: Pass defense is more important than rush defense.
You would think people would have some idea about this by now, considering that Bud Goode, the godfather of NFL statistical analysis, has been hammering home the importance of net yards per pass attempt since 1959. And yet, you still get articles insisting on the importance of the running game, and preventing the running game. The latest comes from Pro Football Weekly's Mike Wilkening:
One of the reasons the Raiders should be considered contenders for the AFC West title is a big factor in why the Dolphins will continue to be competitive in the AFC East, no matter what off-field drama the team endures.
It is a major discussion point for anyone who wants to argue the Cowboys will be in the mix in an improved NFC East, or that the Steelers and Jaguars will compete for division titles.
What do these teams have in common? All should be among the league's better teams at stopping the run in the upcoming season. The Cowboys, Steelers, Jaguars and Dolphins all finished in the top 12 vs. the run in 2003, and the Raiders have made major improvements to their run defense in the offseason. In a league where stopping the run has become more difficult of late, it will give them a fighting chance to make the playoffs.
Seven of the 11 best run-stuffing teams made the postseason in 2003, including Super Bowl entrants New England (fourth vs. the run) and Carolina (11th). Two seasons ago, Oakland and Tampa Bay played strong run defense, and it helped propel both teams to the Super Bowl.
It's true that seven of the 11 best run-stuffing teams made the postseason in 2003, if you accept the NFL's conventional idea of what counts as "run-stuffing." The problem is, which gives a better example of a team's ability to stop the run: total rushing yards, or rushing yards per attempt? The NFL's "official" ranks for run defense, the ones you find listed in the Sunday paper, are based on total rushing yards. But of course the best teams give up fewer rushing yards; teams aren't going to run as much against them because they're playing catch-up in the second half.
A good way to show how much the number of carries influences total rushing yards allowed is to look at correlations between rush defense and wins over the past three years. Correlation coefficient, for those not familiar with the math, evaluates how much one variable influences another variable. A correlation of 1 means the two variables are completely connected; 0 means they have no connection. Since wins get higher as yardage against a defense gets smaller, these correlations will be negative; the closer to -1, the more often a better defense results in more wins.
For 2001-2003, the correlation between rushing yards allowed per game and wins is -0.45. That means there is a strong relationship between a team's place in the NFL's official run defense rankings and its chances of making the playoffs. But do you want to see a better relationship? The correlation between rushing attempts against a defense and wins is -0.70. The number of rushing attempts your defense faces is a better indicator of how many games you have won than the number of yards you allow!
Rushing yards per attempt is a far better indicator of the true quality of a run defense, and it turns out that the correlation between rushing yards allowed per attempt and wins is pretty much non-existent: -0.02. Five of the ten best teams in this stat made the playoffs last year -- but so did four of the ten worst teams. Over the past three years, playoff teams have averaged 4.12 rushing yards allowed per attempt. Non-playoff teams have averaged a whopping 4.13 rushing yards allowed per attempt.
Does that mean that rushing defense is totally meaningless? No, of course not. Rushing yards allowed per attempt is a better indicator of defensive ability than total rushing yards, but our statistic VOA is an even better indicator because it takes game situation into account. The correlation between rushing defense VOA and wins from 2001-2003 is -0.29. So there is definitely value to rush defense. And let's not belittle the importance of the perception of a good rush defense, which can itself scare teams away from running the ball even if the defense isn't the league's best when judged on yards allowed per attempt. This is what I call "The Tennessee Run Defense Problem."
But a far, far better indicator of how many games a team is going to win is Bud Goode's ol' favorite statistic, net yards per pass attempt. Even before taking interceptions into account, the correlation between net yards allowed per pass attempt and team wins is -0.60. Whereas there is no difference between playoff teams and non-playoff teams when it comes to rushing yards allowed per attempt, there's a big difference in net yards per pass. From 2001-2003, playoff teams have allowed 5.88 net yards per pass; non-playoff teams have allowed 6.52 net yards per pass. And passing defense VOA, as you might expect from reading this site, is one step better. The correlation between passing defense VOA and wins is -0.67.
I don't mean to pick on Wilkening. Almost all NFL journalists overstate the importance of run defense at the expense of pass defense. But no matter how much Warren Sapp and Ted Washington help the Raiders improve against the great running backs of the AFC West, that team is going nowhere unless Sapp and Washington can also help a pass defense that last year allowed 7.3 net yards per pass, worse than 29 other NFL teams.
The table for those interested in a summary, here are all the correlations of stats with winning from 2001-2003:
|RUSH DEFENSE||PASS DEFENSE|
|Yards allowed per game||-0.45||Yards allowed per game||-0.28|
|Yards allowed per rush||-0.02||Yards allowed per pass||-0.60|
|Total Rushes against||-0.70||Total passes against||0.35|
|Rush defense VOA||-0.29||Pass defense VOA||-0.67|