Guest columnist Zachary O. Binney fact-checks a story in a national publication and finds that everyone makes mistakes.
15 Jun 2004
by Aaron Schatz
There were so many good comments in the discussion thread for last week's article on rush defense, and they posed so many good questions, that I decided that the answers deserved their own article instead of getting lost at the end of a comment thread. If you would like to read the original article and the comments, click here. I'm going to warn that this gets really geeky and number-heavy, but I'll toss a little summation at the end for those who want the quick version. On to the questions...
"The article states that rushing attempts against a defense is a good indicator of success. Does the same hold for rushing attempts by an offense? And how does this relate to the origins of this site?" - B Money
Hey, B Money, are you related to Money B from Digital Underground? I love those guys. I think I'm the only white man in America who owns all five DU albums. Anyway, yes, over the past four years, the number of rushing attempts by a team has been a better indicator of success than the actual number of rushing yards, or yards per rush. The correlation of wins to carries is .51, but the correlation of wins to rush yards per carry is only .23. Rushing yards correlate much better with actual scoring, but when you look at point differential (i.e. points scored minus points allowed), actual carries are a better indicator. It makes logical sense; that's the whole point of running out the clock. Here are the numbers for 2000-2003:
|RUSH OFFENSE||RUSH DEFENSE|
|Correlation to...||Wins||Pts for||Pts dif||Correlation to...||Wins||Pts vs||Pts dif|
|Yards per game||.43||.46||.48||Yards allowed per game||-.49||.52||-.51|
|Yards per rush||.23||.47||.30||Yards allowed per rush||-.06||.38||-.09|
|Total rushes||.51||.30||.51||Total rushes against||-.71||.45||-.70|
|Rush offense VOA||.40||.64||.47||Rush defense VOA||-.32||.59||-.37|
What about passing offense? Just like a better pass defense leads to more wins than a better rush defense, so does a better pass offense lead to more wins than a better rush offense. The correlation of net yards per pass on offense to wins is .55, compared to .23 for rush yards per carry. The correlation of pass offense VOA to wins is .62, compared to .40 for rush offense VOA. (VOA, for those new to the site, is our proprietary Value Over Average statistic.)
|Best Run Defense VOA, 2000-2003|
Note that the numbers repeated from the last article on rushing defense are slightly different, because we've added 200. 2000 is going to change things a little because the 2000 Ravens had one of the most ridiculous rushing defenses of all time. The table to the right shows the best run defenses of the past four years by VOA (not adjusted for schedule strength), and you can see that the 2000 Ravens lap the field. They are actually changing that line in the Princess Bride; it is now "never start a land war in Asia, never challenge a Sicilian when death is on the line, and never run against the 2000 Baltimore Ravens." Oh, and they also had the second-best pass defense of the past four years (only the 2002 Bucs were better).
"Has anyone ever looked into whether teams that have a large lead give up more large rushing gains (on fewer attempts) because they are concentrating on the pass?" -- M
Yes, we can look at that, and the results are just what you expect. Not only do teams with the lead run more often and for fewer yards per carry, the impact becomes even more clear when you look only at the fourth quarter. Here are the numbers for 2000-2003:
|Score Gap||<-21||-21 to -15||-14 to -8||-7 to -1||0 to 7||8 to 14||15 to 21||>21|
|Yd/Carry, Q4 only||4.79||5.51||4.69||4.31||4.09||3.93||3.99||4.04|
|Total Carries, Q4 only||615||648||1062||2139||4068||2415||1678||1103|
I'm trying to say is that if you have a good-enough offense (reputationally), teams will try to run on you less, and the quality of your run defense has less and less correlation w/ your record. If you are below a threshold "X", then run defense quality should have more of a correlation w/ your record. This would best be illustrated by the difference between Oakland & Kansas City.
Conversely, I think pass Defense would have a positive correlation with record at all levels of proficiency.
Is there any way to test this hypothesis? Also, is the converse true? (If your defense is good enough, the quality of your running has little to no correlation w/ your wins)." -- M
In order to test this idea, I sorted all 126 teams of the past four years by offensive VOA and then separated them into six groups of 21 teams apiece. Then I ran the numbers to see what was the correlation between rush defense and wins in each group. Obviously, with only 21 teams in each group, there is going to be a sample size issue, but I was worried that fewer groups would mean that the numbers didn't really represent the difference between a good offense and a great offense, or a good one and an average one.
The results didn't seem to bear out M's hypothesis. The correlation between yards per carry and wins was pretty much the same for each group, except for Group 3 (offenses from -1% to 5% VOA) where there was actually a positive correlation, meaning that for these teams allowing more yards per rush actually meant more wins. There's that small sample size fluke I was talking about. Furthermore, it seems a low number of carries against a team is actually a better indicator of winning when a team has an average offense than when it has a good offense. That would mean that a team with a good offense could generally get away with facing more rush attempts from the opposition. By "generally," I mean "not the 2003 Chiefs." By the way, I ran the numbers for point differential as well, and they were basically the same. Here's your handy-dandy table:
|Correlation of defense with wins||Yd/rush||Yd/game||Carries||Rush VOA|
|Group 1 (best offenses)||-.28||-.35||-.33||-.54|
|Group 6 (worst offenses)||-.26||-.33||-.36||-.27|
The next thing to do was to analyze only the teams which reached the playoffs. 15 different teams have reached the playoffs over the past four years despite a negative offensive VOA (once again, not adjusted for schedule strength). I created three groups: one of these 15 teams, one of the other 33 playoff teams, and one of the 78 non-playoff teams and ran the correlations again. This time, there seemed to be something. The correlation between winning and rush yards allowed per carry was stronger for the 15 playoff teams with below-average offenses than for the 33 playoff teams with above-average offenses, and the correlation was stronger for both groups of playoff teams than it was for the non-playoff teams. Furthermore, while the correlation of rush defense and wins was about the same as the correlation of pass defense and wins for both groups of playoff teams, the correlation to wins was much better for pass defense for the non-playoff teams.
|Correlation of defense with wins||Yd/rush||Yd/pass||Rush VOA||Pass VOA|
|Playoffs, offense VOA <0%||-.35||-.37||-.25||-.32|
|Playoffs, offense VOA >0%||-.23||-.24||-.41||-.39|
One thing that should be noted is that 15 teams is a really small sample, and you're talking about the same franchises over and over. The teams that have made the playoffs with a below-average offense include three years of the Ravens (2000, 2001, 2003) and two each of the Dolphins (2000, 2001), the Patriots (2001, 2003), and the Eagles (2000, 2001).
It is also worth noting that only eight teams during the past four seasons have made the playoffs despite a below-average pass defense VOA. Twice as many teams, however, have made the playoffs despite a below-average rush defense VOA (15 teams, actually, plus the 2001 Packers who were at 0%).
The second part of this question was a request to look at the converse, comparing teams with different quality defenses to see if a better defense means your rush offense doesn't need to be as good. I'll be honest, I'm not going to bother running the numbers here breaking the teams down into six groups. Frankly, they are all over the place. Even stranger, the correlations for wins and the correlations for point differential seem to be totally disconnected -- remember, when it came to looking at the importance of defense, they were very close.
I did run numbers for playoff and non-playoff teams as well. Since only 10 teams have made the playoffs despite below-average defenses, I lowered the threshold slightly (actually raised the threshold, since defense is better when it is negative). Once again, however, the correlations with point differential are not necessarily the same as the correlations with wins. It seems that for the teams that make the playoffs with better defenses, rush offense isn't as important for winning but it is more important for scoring more points than the other team. Of course, that doesn't really make logical sense, so I'm afraid I can't really provide any answers at this point. Here are the numbers:
|Correlation of offense...||Yd/rush||Yd/pass||Rush VOA||Pass VOA|
|Playoffs, defense VOA >-2% with wins||.21||.11||.39||.23|
|Playoffs, defense VOA >-2% with point dif.||.03||.35||.29||.55|
|Playoffs, defense VOA <-2% with wins||-.10||.39||.01||.33|
|Playoffs, defense VOA <-2% with point dif.||.32||.28||.36||.28|
|Miss playoffs with wins||.25||.46||.41||.54|
|Miss playoffs with point dif.||.30||.59||.50||.68|
Unlike with the list of teams making the playoffs despite poor offense, this list doesn't repeat the same teams over and over, except that it includes three versions of the Colts (2000, 2002, 2003). One interesting note: the 2000 Vikings made the playoffs despite having the worst (unadjusted) defense of the past four seasons, +26.4%.
1 comment, Last at 05 Aug 2005, 8:46pm by David Brude