FO Mailbag: Do FO's Opponent Adjustments Matter?
Jason Martin: I recently read an article discussing the defense-adjustments you make in DVOA. If you look specifically at this graph, you'll there is a definite correlation(-.66), where ideally there would be none. The original article goes into a few possible explanations, but I was wondering if you had any thoughts or could expand on them. The current explanation for how it is DVOA is defense-adjusted seems like it would eliminate any correlation with strength of schedule.
Jason, your e-mail specifically addresses the question of how strength of schedule correlates to opponent adjustments. (Remember, DVOA is adjusted for both offensive and defensive opponent strength, even though the acronym just has "D.") However, the article on igglesblog makes two points.
First, it argues that the opponent adjustments in DVOA don't mean anything because there's a ridiculously high .985 correlation between DVOA and VOA. Well, of course there's a high correlation! DVOA essentially is the same thing as VOA with some small adjustments. The quality of play is a lot more important than the opponent adjustments. The opponent adjustments tend to even out over the course of the season -- opponent adjustments mean a lot more when looking at one week's performance in Quick Reads than they do looking at a full-season performance.
In addition, opponent adjustments are generally smaller than conventional wisdom might expect. Even the best defenses have bad days. Even the worst defenses have good days. When you average those in with the regular season, the penalty for playing a bad defense is going to be a little smaller than you would expect, and the bonus for playing a good defense is also going to be a little smaller.
The second argument is that there is a really high correlation between strength of schedule and DVOA in 2011. And there is! Igglesblog graphed this using just the ranks and got a -.66 correlation, but the correlation is even bigger if you use the actual ratings. The correlation between strength of schedule (i.e. average DVOA of opponent) and a team's total DVOA in 2011 was -.72. That's just absurdly huge.
It's also a one-time thing.
If we look at the last ten years, there is only one year where the correlation between strength of schedule and DVOA is even HALF as strong as 2011.
Yes, it is negative more often than it is positive. Perhaps we need to look into improving our opponent adjustments this offseason by using third-order adjustments rather than second-order (or fourth-order, or fifth-order -- honestly, I'm not sure how complex my Excel spreadsheet can get before it starts taking three hours to run each week). However, the massive (negative) correlation between schedule strength and DVOA is entirely unique to the 2011 season. I have no idea why things ended up that way. But they did.