2004 DVOA Projections

by Aaron Schatz

Well, here they are, the fruit of much labor, the 2004 DVOA projections. This system will get better each season as I have more data to work with, but I feel pretty good about its accuracy and it should do a good job of spotting teams that are not expected by most to improve or decline.

(Here's the requisite link [1] to an explanation of DVOA, which stands for Defense-adjusted Value Over Average and measures a team's performance on every play of the season compared to league average in the same situation, adjusted for opponent. I know a lot of people are coming here from various message boards and this is just going to look like a jumble of pointless numbers. Trust me, there is a method to the madness, and over the past four seasons DVOA has been a far more accurate predictor of future performance than wins or points. If you want to get past the nuts and bolts and just see the projections, click here [2]. A number of questions asked in the discussion below or in email are now answered here [3].)

Offense, defense, and special teams DVOA were all projected separately based on 2000-2003 numbers. I tried more variables than you can possibly imagine to get the most accurate projections. Variables include last year's DVOA on specific downs, on rushing plays vs. passing plays, and in the red zone, as well as improvement or decline from September to December of last season, points scored and allowed over the previous two seasons, tenure of the offensive and defensive coordinators, number of previous year Pro Bowl players, draft value of picks over the past four seasons, and (on offense) starting experience for the quarterback. The correlation between projected DVOA and actual DVOA for the past three seasons is .776 for offense, .663 for defense, and .515 for special teams.

I hope people don't mind if I refrain from sharing all the specific coefficients, but here are a couple of comments about how some of the variables effect the ratings:

Extreme offensive values on passing on third down or passing in the red zone, whether good or bad, don't carry over as strongly the next season as general passing ability or passing on first down, so teams that whose offensive ratings in 2003 were significantly affected by very good or bad performances in these areas are projected closer to average. You'll see below that with a couple of teams this effect was so strong that I ended up making a subjective adjustment.
On the other hand, better third down defense does seem to indicate a better defense the following year.
Defensive improvement over the course of the 2003 season affects the 2004 defensive projection more than twice as much as offensive improvement over the course of the 2003 season affects the 2004 offensive projection. That's good news for teams whose defenses got better later in the year (Chicago, Green Bay, Tennessee) and bad news for teams whose defenses got worse later in the year (Kansas City, Minnesota). It also means the projection is not as strong as you might expect for offenses that improved over 2003, like Cincinnati and Philadelphia.
The carryover from year to year is much higher for rushing offense than for rushing defense.
The variables having to do with tenure are capped at a certain point; I refer to this as the "Gary Kubiak rule" because otherwise Denver would be projected with an offense around 1000%.
Yes, at first I thought I would just project all special teams as 0%, but it turns out that certain aspects of special teams -- field goal kicking, kick returns -- are better for predicting the following year's performance than others, and considering each part of special teams play separately allowed me to come up with a mildly reliable projection.
The first offensive variable I tried for draft value came out as negative. So did the second and the third. I tried this year's draft, last year's draft, even two and three years ago. I tried adding draft values together, squaring them, and square rooting them. I tried counting only the offensive line picks and only the skill position picks. I tried basing the regressions on the Scramble values of the names of the draft picks. No matter how I measured things, the resulting equations said that the more draft value from your recent offensive draft picks, the worse your offense. This is totally counterintuitive, but it was so unavoidable (with, for you stats geeks, a P-value of .001) that I finally gave up and left it in the system. The defensive variable, on the other hand, works in the way you would expect -- the more defensive draft value over the 2003 and 2004 drafts, the better your defensive projection.

Strength of schedule was then figured based on the average projected total DVOA of all 16 opponents for 2004 (yes, projected performance, not 2003 performance), and then wins were projected based on offense, defense, special teams, and strength of schedule. Unfortunately, as you would expect from a regression analysis with a lot of variation, the projected wins were all grouped around the middle of the range, with every team but four projected between 6.2 and 10.1 wins and half the teams in the league projected between 6.9 and 7.9 wins. Based on some advice (thanks, Ben Alamar) I did a second regression with above and below average teams separately, averaged the two projections, and then adjusted things a bit more so that the scale of win-loss records looked like the average scale of win-loss records over the past four seasons, except that I created a limit where wins or losses only go to 11 (which is one louder [4]).

The problem with a purely objective prediction system, of course, is that it can't take into account major changes that aren't reflected in statistics, or major issues that aren't represented by the variables in the system. Player movement is already factored into the system with a variable that represents players who made the Pro Bowl the prior year (i.e., the Redskins offense gets credit for Clinton Portis, the Broncos defense for Champ Bailey). But there is no variable to represent that the Dallas offense has a higher average age than the Supreme Court, or the fact that San Francisco's 2003 offense and 2004 offense have the same relationship as the Velvet Underground with Lou Reed and the Velvet Underground that made Squeeze [5]. There were also a couple of predictions so extreme that I had to restrain them, lest the projection system strain credibility. But since the goal here is to find out what the numbers tell us, not to tell you my personal picks (that article comes next week), I tried to keep the subjective changes to a minimum. The changes are based on altered 2003 numbers instead of just scribbled randomly because I thought "this team's offense should be 10% higher."

The ratings that have been adjusted are in blue, and the reasons for the adjustments are given below the table. Schedule is ranked from hardest (Houston) to easiest (Dallas). Remember, a positive number means more scoring so defenses are better when they are negative, and total DVOA is offense plus special teams plus the opposite of defense.

Some of these projections will be unexpected. Some of these projections you will disagree with. Hey, some of these projections I disagree with. Here they are anyway. If these projections prove to be accurate, the breakout team of 2004 will be the New York Jets.

TEAM	W-L	TOTAL DVOA	TOTAL RANK	OFFENSE DVOA	OFF. RANK	DEFENSE DVOA	DEF. RANK	SPECIAL DVOA	S.T. RANK	SCHED	SCHED RANK
NWE	11-5	23.6%	1	3.7%	9	-19.7%	2	0.2%	15	0.1%	17
TEN	11-5	18.7%	2	10.5%	4	-8.7%	5	-0.6%	21	-2.5%	28
GNB	11-5	14.7%	3	6.8%	7	-6.1%	9	1.8%	6	-2.5%	29
NYJ	10-6	14.2%	4	12.3%	3	0.6%	13	2.5%	3	-1.7%	25
TAM	11-5	13.9%	5	-0.3%	19	-17.3%	3	-3.1%	31	-2.0%	26
KAN	10-6	13.2%	6	20.2%	1	9.9%	28	3.0%	1	2.6%	4
DEN	10-6	10.1%	7	2.5%	10	-7.8%	7	-0.3%	18	0.3%	16
DAL	10-6	10.0%	8	-2.7%	23	-13.5%	4	-0.8%	22	-5.0%	32
PHI	10-6	9.8%	9	16.7%	2	8.7%	27	1.8%	5	-4.1%	31
IND	9-7	8.3%	10	8.6%	5	2.6%	17	2.3%	4	1.6%	10
STL	9-7	7.9%	11	-0.3%	18	-8.0%	6	0.3%	14	1.1%	12
MIA	8-8	2.3%	12	0.3%	16	-1.4%	11	0.6%	13	3.2%	3
BAL	9-7	2.0%	13	-27.8%	32	-27.0%	1	2.9%	2	-1.0%	22
SDG	6-10	-1.6%	14	8.1%	6	8.4%	25	-1.3%	25	3.5%	2
OAK	7-9	-1.7%	15	2.2%	14	5.4%	23	1.6%	8	2.2%	7
SEA	8-8	-1.9%	16	2.5%	11	3.8%	20	-0.6%	20	1.1%	11
TEAM	W-L	TOTAL DVOA	TOTAL RANK	OFFENSE DVOA	OFF. RANK	DEFENSE DVOA	DEF. RANK	SPECIAL DVOA	S.T. RANK	SCHED	SCHED RANK
ATL	8-8	-2.1%	17	4.7%	8	4.5%	22	-2.3%	28	-0.8%	20
CAR	7-9	-2.6%	18	-2.1%	20	1.7%	16	1.2%	10	2.5%	5
BUF	7-9	-3.5%	19	-7.5%	28	-4.9%	10	-0.8%	23	1.8%	8
JAC	7-9	-3.7%	20	0.1%	17	0.4%	12	-3.4%	32	0.8%	13
MIN	8-8	-4.6%	21	2.4%	12	4.4%	21	-2.6%	29	-1.0%	21
CLE	8-8	-4.7%	22	-2.2%	21	2.7%	19	0.2%	16	-3.1%	30
CHI	7-9	-4.8%	23	-13.3%	30	-7.7%	8	0.8%	12	-0.1%	18
NOR	6-10	-6.2%	24	-4.0%	24	0.7%	14	-1.6%	26	1.8%	9
SFO	6-10	-7.2%	25	1.7%	15	5.8%	24	-3.0%	30	0.5%	15
PIT	8-8	-7.7%	26	-6.3%	26	0.9%	15	-0.5%	19	-1.5%	23
DET	7-9	-10.0%	27	-9.1%	29	2.6%	18	1.8%	7	-0.8%	19
WAS	6-10	-10.7%	28	2.4%	13	14.1%	32	1.1%	11	-1.5%	24
ARI	5-11	-14.4%	29	-2.6%	22	10.8%	29	-1.0%	24	0.6%	14
NYG	6-10	-17.9%	30	-4.5%	25	11.3%	30	-2.1%	27	-2.0%	27
CIN	5-11	-20.3%	31	-6.6%	27	13.6%	31	-0.1%	17	2.2%	6
HOU	5-11	-23.6%	32	-16.3%	31	8.6%	26	1.4%	9	4.5%	1

We may get a bit technical in the commentary here, so bear with me. There are probably a couple of projections here that jump out at you, but I can pretty much guess what the first one is: San Diego with the #6 offense. Exhibit A of how statistics are useless, right? I tried a ton of different variables that would knock down the San Diego offensive projection, but nothing lowered it significantly without completely destroying the reliability of the projection system as a whole.

Remember above, when I said that one of the variables in the offensive projection takes into account the fact that poor passing performance on third down does not carry over from year to year? San Diego was so much worse when passing on third downs, compared to first and second down, that the adjustment ended up too strong to pass what we'll call "the laugh test." Yes, if you think projecting them as the sixth-best offense in the league makes no sense, understand that San Diego's offense was originally projected even higher. The same problem existed for Miami. I replaced the offensive projection for San Diego and Miami with the projection from what I call the "simple formula" which only considers a few variables: DVOA for each down, for all passing and rushing, weighted, and year-to-year change in points scored. This weakens the San Diego offense from 14.3% to 8.1% and the Miami offense from 6.6% to 0.3%. Still higher than you expect, I would feel uncomfortable dropping the projection any further. (By the way, this third down passing variable is also why the Seattle and New Orleans offenses project as declining significantly from last season.)

Don't get excited, San Diego fans. Because you play the second-hardest schedule in the league, you get projected to finish 6-10, last in the division. Oakland, which actually projects as a worse team, has an easier schedule because they play Buffalo instead of the Jets.

Speaking of those Jets, yes, they are projected as one of the five best teams in the league, before even taking into account the easy schedule that every AFC East team is getting this year. The #3 offensive projection is a combination of the fact that they were already a very good offense last year, that they were better on first down (which, again, is a better predictor for next year's success than third down), and that they score very low on the strange "drafting offensive players recently is bad" variable. The flipside is that no team scores better on the defensive "recent draft value" variable, and the 2003 Jets were better defensively on third down compared to first and second down.

Finally, I should address Baltimore. Very early on, when I was first building this system and Al and I wrote the first of the four Scramble for the Ball columns previewing the season [6], I mentioned that Baltimore was projected as the worst team in the NFL. Now they project as division champions again. What the heck happened? The league-worst offensive projection hasn't changed. Adding on a special teams projection improved them a little bit. But the major change is defense, as a number of new variables (like Pro Bowlers and WEIGHTED DVOA) caused the top defensive projections to move farther and farther way from the rest of the league. When I wrote that Baltimore was projecting as the worst team in the league, their defensive DVOA projection was -15.5%, and the Patriots were second at -14.7%. By the end of the projections, those two defenses were projected even better than what you see in the table above. Baltimore was projecting as -30.4%, and New England was projecting as -22.6%. These two projections were totally out of line with the other 30 teams, mostly because the Ravens and Patriots were the best defenses and got better as the season went along. To bring them back to the pack, I changed the variable for WEIGHTED DVOA to be the same as total 2003 defensive DVOA, which brought the two teams closer to the rest of the league: -27.0% for Baltimore, -19.7% for New England.

Baltimore's offense also got bumped up a bit, for the same reason. Like the two best defenses, the two worst offenses were miles away from the other 30 offenses, to the point where the difference in DVOA between the worst offense (Baltimore) and the #30 offense (Chicago) was the same as the distance between the #30 offense and the #7 offense (Green Bay). In order to blunt this effect a bit, Baltimore and Houston had their projections changed to the average between their "complex" and "simple" projections (improves Houston offense from -26.0% to -16.3%, Baltimore offense from -33.2% to -27.8%).

Strangely enough, I felt more comfortable adjusting offensive ratings than defensive ratings precisely because the offensive projection is more accurate. I think this makes the peculiar projections stand out more.

Poor Houston. Not only does the projection system think their offense is nowhere near a breakout, and their defense will still be awful, but they also project as having the hardest schedule in the league for the second straight season. That's just not fair.

Here is a list of the other subjective changes made:

Atlanta's offensive projection is based on their 2002 statistics with Michael Vick at quarterback (improves offense from -2.9% to 4.7%).
There are adjustments for an offense that is too young, but not for an offense that is too old, and I felt that projecting a positive offense for Dallas caused a bit of a problem. The projection here represents the average of their "complex" and "simple" projections, which I felt was a good compromise since it lowered the offense while keeping their total projection in the top ten (weakens offense from 2.0% to -2.7%).
Chicago special teams projected based on 2003 kick returns dropped to zero due to injury to Jerry Azumah (weakens special teams from 1.8% to 0.8%).
In an attempt to factor in the remarkable improvement they showed in the playoffs, Carolina's 2003 pass offense was raised to league average before the projection was run (improves offense from -13.2% to -2.1%)
Because of the massive turnover of offensive personnel, San Francisco's 2003 pass offense was lowered to league average before the projection was run (weakens offense from 4.9% to 1.7%).

In both cases, I could have adjusted further, but I preferred to err on the side of adjusting things too little rather than too much. So if both teams were projected based on the same passing offense, why are the 49ers listed with a better 2004 offense? Kevan Barlow.

Finally, physicists have theorized that if this year's Jacksonville kicker were to suck as much as Seth Marler did last year, it would create a cosmic singularity in the middle of North Florida, twisting the entire Earth inside out and basically making a mess of the entire NFL season, not to mention the Super Bowl. Under the assumption that this won't happen -- besides, the presidential election is going to make enough of a mess in Florida -- I raised Jacksonville's special teams projection from -4.6% to -3.4%.

Note that these are not the "official" predictions, per se. There really aren't "official" Football Outsiders predictions, but there will be an article next week with predictions and commentary from the entire Football Outsiders staff to go with these statistical projections.