DVOA Analysis
Football Outsiders' revolutionary metrics that break down every single play of the NFL season

Week 9 DVOA Ratings

by Aaron Schatz

Perhaps the New York Giants were just miffed about last week's DVOA ratings, where they dropped five spots based entirely on changes in opponent adjustments. They took their frustrations out on the Seattle Seahawks, and now they are back on top. Their division rivals in Philadelphia are still close behind thanks to a win over the always-tough Indianapolis Colts, another one of those rare games where both teams emerge with positive DVOA. Philadelphia got 30.1% for the win, Indianapolis 20.2% for the loss. That's how close the game was, and how strong the opponent adjustments are for these two teams overall.

This week was the most upset-friendly of the NFL season so far, at least judged by DVOA ratings. (One piece of evidence: Our premium picks against the spread, which had been hitting at over 60 percent so far this season, went just 4-8-1 this week.) It wasn't just the surprising wins, like Cleveland over New England and Oakland over Kansas City. This week also had surprisingly close wins, like the Jets barely beating Detroit and Cincinnati coming within a few yards of edging past Pittsburgh. As a result, we've got a lot of movement in the rankings this week. Six different teams move by four or five spots, which is par for the course in subjective power rankings around the Internet but somewhat rare for the DVOA ratings once we've got a few games in the books.

Overall, the theme of parity still applies to the 2010 season, although the Carolina Panthers are starting to really stand out like a sore thumb. At least Buffalo and Arizona are playing some close games, but Carolina looks horrible. Still, the Panthers are far from "worst DVOA ever" status. Last year at this time, Oakland and Detroit were each below -50%, much worse than what Carolina has this year.

A couple of other interesting notes I noticed when going through this week's numbers:

  • As I noted on Twitter Sunday afternoon, this is the first season since 1971 where no team is either 7-1 or 8-0 after eight games. In 1971, the Redskins were 6-1-1 after eight games, but I can't find a season where the best team after eight games was only 6-2.
  • Although only four NFC teams rank in the top dozen for DVOA, the extreme stratification of the NFC causes our playoff odds report to list those four teams as four of the top five possibilities to win the Super Bowl. After all, those good AFC teams all have to go through each other -- not only to get to the Super Bowl, but to make it to the postseason in the first place. How ironic it would be if the unfortunate fans of Arlington, Texas had to watch not one but two Giants championship celebrations in a four-month period.
  • You may have noticed that the Innovative Stats box on the FO front page now shows the top teams in the last playoff odds, rather than the top teams in DVOA. Don't forget you can use the tabs in that box to get a quick look at the top teams and players this season without having to click through to individual stats pages.
  • The San Diego Chargers special teams not only is the worst of the DVOA Era, but they haven't even had one good week. San Diego's special teams DVOA has been below 0% in all nine games this season.
  • The Oakland Raiders story is swell and all, but when you look at the schedules it becomes clear how difficult it will be for them to get past Kansas City -- and stay ahead of San Diego -- to win the division. The Raiders still have to play at Kansas City and at San Diego. Their other five games include Pittsburgh, Indianapolis, and Miami, plus the inscrutable Jaguars, who honestly could beat Oakland 45-0 or lose to Oakland 45-0. Neither result would really surprise me.
  • Seattle has completely melted down over the past two weeks with DVOA of -98.3% against Oakland and then -108.9% against the Giants. Because of this, the 2-6 San Francisco 49ers actually have the best DVOA in the NFC West right now. Maybe they really could pull off a division title. The current DVOA playoff odds report gives all four teams in the NFC West an average projection between 6.5 and 7.1 wins, and all four teams are listed with between 19 percent and 29 percent chance of winning the division.

I think I've given my solution to the problem before, but if not, here it is: The NFL should change the rules to state that when no team in a division has at least eight wins, that team forfeits its postseason spot to a third wild card team, with the top wild card team now hosting a first-round playoff game. The solution to the NFC West problem isn't a wholesale change in the playoff structure, as some people have suggested. Even if we get a 7-9 division champion this year, it's not something we need to worry about more than once per decade. In general, I think it is important to maintain the integrity of the divisions because it makes those divisional games more important and thus strengthens rivalries. But there really should be something in place to prevent a losing team from making the playoffs in those rare years like 2010.

* * * * *

This week actually sees a small fix to the DVOA formula for both teams and quarterbacks. When I made my last big overhaul to DVOA a couple of years ago, it included a fix that lowered the penalty for interceptions on fourth down. However, fourth-down interceptions still do have some negative value, because of the possibility of a return. If the ball gets picked off on the line of scrimmage, that's obviously going to be worse than a simple incomplete pass, even if both plays result in the defense taking over control of the ball.

The exception, of course, is in the last two minutes of the game when a team is driving in an attempt to tie the game. At that point, any throw on fourth-down is basically a "what the hell" kind of play, where there are only two results: a complete pass that gets past the sticks, and anything else. Unfortunately, the DVOA formula was still giving a small amount of negative value to fourth-down interceptions at the end of a game. I have now changed this so that any fourth-down interception in the final two minutes is translated as an incomplete pass. This is the same way we treat passes which we manually mark as "Hail Mary," like the 50-yard downfield interception on second down with three seconds left. The change gives a small boost to a handful of quarterbacks who threw last-gasp fourth-down picks this season: Sam Bradford, Bruce Gradkowski, Philip Rivers, and Chad Henne (twice!). However, since the change only affects interceptions in the final two minutes, Trent Edwards is still penalized for his goal-line interception against Tennessee with 6:06 left in that boring Week 6 Monday Night Football game. (That one's a great example of the difference between an incomplete and a pick, actually -- an incomplete pass gives the Titans the ball back at their own one-yard line, but the pick gave them the ball at their own 20 thanks to the touchback.)

I won't have time to go back and make this change in previous seasons until after 2010 is finished, but the change is now made for both 2010 team stats and quarterback passing stats.

* * * * *

These are the Football Outsiders team efficiency ratings through nine weeks of 2010, measured by our proprietary Defense-adjusted Value Over Average (DVOA) system that breaks down every single play and compares a team's performance to the league average based on situation in order to determine value over average. (Explained further here.)

OFFENSE and DEFENSE DVOA are adjusted to consider all fumbles, kept or lost, as equal value. SPECIAL TEAMS DVOA is adjusted for type of stadium (warm, cold, dome, Denver) and week of season. WEIGHTED DVOA represents an attempt to figure out how a team is playing right now, as opposed to over the season as a whole, by making recent games more important than earlier games.

Opponent adjustments are currently at 90 percent strength and will steadily grow stronger until Week 10. As always, positive numbers represent more points so DEFENSE is better when it is NEGATIVE.

To save people some time, please use the following format for all complaints:

<team> is clearly ranked <too high/too low> because <reason unrelated to DVOA>. <subjective ranking system> is way better than this. <unrelated team-supporting or -denigrating comment, preferably with poor spelling and/or chat-acceptable spelling>

TEAM TOTAL
DVOA
LAST
WEEK
WEIGHTED
DVOA
RANK W-L OFFENSE
DVOA
OFF.
RANK
DEFENSE
DVOA
DEF.
RANK
S.T.
DVOA
S.T.
RANK
1 NYG 30.9% 6 31.2% 1 6-2 16.3% 6 -21.4% 1 -6.8% 31
2 PHI 29.5% 2 30.3% 2 5-3 26.6% 3 -4.9% 7 -1.9% 25
3 TEN 27.4% 4 26.7% 3 5-3 6.3% 13 -16.7% 3 4.5% 5
4 PIT 25.1% 5 24.9% 4 6-2 3.4% 17 -19.8% 2 2.0% 12
5 KC 21.2% 1 19.8% 6 5-3 12.1% 9 -9.6% 6 -0.4% 17
6 GB 21.1% 10 20.2% 5 6-3 14.5% 8 -10.8% 4 -4.3% 28
7 SD 18.4% 7 19.6% 7 4-5 23.3% 4 -10.7% 5 -15.5% 32
8 NE 17.8% 3 16.4% 9 6-2 28.8% 1 14.5% 27 3.5% 9
9 IND 14.9% 8 16.7% 8 5-3 23.1% 5 3.6% 20 -4.5% 29
10 ATL 13.3% 12 12.6% 10 6-2 16.3% 7 1.2% 17 -1.7% 24
11 BAL 11.0% 13 11.5% 11 6-2 9.8% 10 1.1% 16 2.4% 11
12 NYJ 9.9% 9 8.9% 14 6-2 2.4% 18 -2.5% 12 5.0% 4
13 NO 9.8% 14 9.8% 12 6-3 6.3% 12 -4.5% 8 -1.0% 20
14 CLE 7.6% 16 9.0% 13 3-5 3.9% 16 0.7% 15 4.4% 6
15 MIA 5.6% 11 6.8% 15 4-4 8.5% 11 2.3% 18 -0.6% 19
16 HOU 2.0% 15 1.4% 16 4-4 28.5% 2 25.4% 31 -1.0% 22
TEAM TOTAL
DVOA
LAST
WEEK
WEIGHTED
DVOA
RANK W-L OFFENSE
DVOA
OFF.
RANK
DEFENSE
DVOA
DEF.
RANK
S.T.
DVOA
S.T.
RANK
17 SF -4.2% 17 -0.4% 17 2-6 -7.2% 26 -3.3% 10 -0.3% 16
18 CIN -4.7% 20 -3.5% 18 2-6 4.3% 15 4.9% 22 -4.1% 27
19 DET -5.4% 19 -4.6% 20 2-6 -4.4% 23 5.2% 23 4.2% 8
20 OAK -6.0% 22 -3.6% 19 5-4 -8.1% 27 -2.0% 13 0.1% 14
21 MIN -8.3% 18 -7.8% 21 3-5 -5.9% 25 2.3% 19 -0.1% 15
22 WAS -9.5% 21 -8.4% 22 4-4 -5.1% 24 4.0% 21 -0.4% 18
23 TB -9.9% 23 -10.8% 23 5-3 -1.1% 19 9.7% 25 0.9% 13
24 JAC -13.6% 25 -14.6% 24 4-4 -1.2% 20 19.7% 29 7.3% 2
25 STL -18.0% 28 -16.2% 25 4-4 -14.1% 28 0.5% 14 -3.5% 26
26 CHI -19.9% 26 -21.8% 26 5-3 -29.2% 30 -3.5% 9 5.8% 3
27 DAL -23.4% 27 -25.6% 28 1-7 -4.1% 22 18.3% 28 -1.0% 21
28 DEN -24.7% 29 -24.6% 27 2-6 4.3% 14 24.4% 30 -4.7% 30
29 SEA -25.4% 24 -28.9% 29 4-4 -22.1% 29 10.8% 26 7.5% 1
30 BUF -28.7% 30 -29.0% 30 0-8 -2.6% 21 29.2% 32 3.1% 10
31 ARI -35.5% 32 -36.5% 31 3-5 -32.1% 31 7.6% 24 4.3% 7
32 CAR -42.9% 31 -43.4% 32 1-7 -44.7% 32 -3.3% 11 -1.4% 23
  • NON-ADJUSTED TOTAL DVOA does not include the adjustments for opponent strength or the adjustments for weather and altitude in special teams, and only penalizes offenses for lost fumbles rather than all fumbles.
  • ESTIMATED WINS uses a statistic known as "Forest Index" that emphasizes consistency as well as DVOA in the most important specific situations: red zone defense, first quarter offense, and performance in the second half when the score is close. It then projects a number of wins adjusted to a league-average schedule and a league-average rate of recovering fumbles. Teams that have had their bye week are projected as if they had played one game per week.
  • PAST SCHEDULE lists average DVOA of opponents played this season, ranked from hardest schedule (#1, most positive) to easiest schedule (#32, most negative). It is not adjusted for which games are home or road.
  • FUTURE SCHEDULE lists average DVOA of opponents still left to play this season, ranked from hardest schedule (#1, most positive) to easiest schedule (#32, most negative). It is not adjusted for which games are home or road.
  • VARIANCE measures the statistical variance of the team's weekly DVOA performance. Teams are ranked from most consistent (#1, lowest variance) to least consistent (#32, highest variance).



TEAM TOTAL
DVOA
W-L NON-ADJ
TOT VOA
ESTIM.
WINS
RANK PAST
SCHED
RANK FUTURE
SCHED
RANK VAR. RANK
1 NYG 30.9% 6-2 34.0% 6.9 1 -9.1% 29 2.0% 14 21.0% 26
2 PHI 29.5% 5-3 27.9% 6.9 2 5.5% 5 -2.6% 20 7.3% 6
3 TEN 27.4% 5-3 23.8% 6.4 3 4.5% 8 4.7% 10 9.9% 11
4 PIT 25.1% 6-2 21.9% 6.1 4 7.5% 3 -4.5% 22 10.0% 12
5 KC 21.2% 5-3 20.1% 5.9 7 -1.2% 20 -11.1% 29 13.2% 16
6 GB 21.1% 6-3 25.0% 6.1 5 -5.6% 26 3.5% 11 16.8% 20
7 SD 18.4% 4-5 15.8% 6.0 6 -3.3% 24 -4.0% 21 16.9% 21
8 NE 17.8% 6-2 15.9% 5.8 9 1.3% 15 2.8% 12 15.3% 19
9 IND 14.9% 5-3 13.0% 5.6 11 4.7% 7 5.4% 9 11.9% 13
10 ATL 13.3% 6-2 10.8% 5.7 10 2.2% 13 -12.1% 30 8.3% 9
11 BAL 11.0% 6-2 9.9% 5.8 8 1.0% 16 0.1% 17 5.5% 3
12 NYJ 9.9% 6-2 20.9% 5.2 14 -1.5% 21 0.6% 15 5.4% 2
13 NO 9.8% 6-3 13.9% 5.3 13 -10.8% 31 -8.2% 27 8.3% 8
14 CLE 7.6% 3-5 5.0% 4.8 15 10.5% 1 -4.8% 23 13.6% 17
15 MIA 5.6% 4-4 4.4% 5.4 12 5.4% 6 0.3% 16 20.8% 24
16 HOU 2.0% 4-4 -3.0% 4.5 17 7.7% 2 6.7% 7 21.0% 25
TEAM TOTAL
DVOA
W-L NON-ADJ
TOT VOA
ESTIM.
WINS
RANK PAST
SCHED
RANK FUTURE
SCHED
RANK VAR. RANK
17 SF -4.2% 2-6 -3.5% 4.5 16 -3.1% 22 -12.8% 31 31.1% 29
18 CIN -4.7% 2-6 -5.0% 3.8 22 3.5% 12 8.5% 2 7.1% 5
19 DET -5.4% 2-6 -7.1% 4.1 19 4.5% 9 -5.7% 24 7.8% 7
20 OAK -6.0% 5-4 4.2% 3.8 21 -4.3% 25 6.7% 6 33.2% 31
21 MIN -8.3% 3-5 -10.1% 3.9 20 0.0% 18 -0.2% 19 4.4% 1
22 WAS -9.5% 4-4 -4.1% 3.7 24 0.1% 17 8.0% 4 8.6% 10
23 TB -9.9% 5-3 -7.7% 4.4 18 -5.6% 27 -6.7% 26 12.4% 15
24 JAC -13.6% 4-4 -17.6% 3.7 23 4.3% 10 8.7% 1 22.2% 27
25 STL -18.0% 4-4 -7.2% 3.2 25 -14.5% 32 -6.2% 25 13.9% 18
26 CHI -19.9% 5-3 -11.1% 2.8 27 -10.4% 30 7.7% 5 19.0% 23
27 DAL -23.4% 1-7 -26.2% 2.7 28 3.8% 11 8.0% 3 32.6% 30
28 DEN -24.7% 2-6 -23.1% 2.4 29 1.8% 14 2.7% 13 17.0% 22
29 SEA -25.4% 4-4 -21.2% 3.0 26 -7.4% 28 -8.3% 28 38.4% 32
30 BUF -28.7% 0-8 -27.3% 2.4 30 6.6% 4 6.0% 8 5.7% 4
31 ARI -35.5% 3-5 -31.2% 2.3 31 -3.3% 23 -15.2% 32 23.9% 28
32 CAR -42.9% 1-7 -44.4% 1.5 32 -0.8% 19 0.0% 18 12.0% 14

Comments

162 comments, Last at 12 Nov 2010, 9:09pm

1 Re: Week 9 DVOA Ratings

The amazing thing is, almost all national pundits claim the cream of the AFC is better than the cream of the NFC, and yet DVOA claims the opposite is true.

3 Re: Week 9 DVOA Ratings

The pundits do so because they care about wins, not who plays who and how they looked. A 6-1 NE team is clearly the best team in sports no matter what, because oooh, one whole loss!

Fortunately we don't have to look at things like that all the time.

I'm surprised how much lower NE went after losing to the Browns.

7 Re: Week 9 DVOA Ratings

I have nothing to add, I just thought of a good (not verified) joke. Apologies if someone's already used it.

157 Re: Week 9 DVOA Ratings

It's a shame, then, that all that matters is wins.

The rankings make much more sense for individual players opposed to teams, you can't really say that the chargers, who are 4-5, third in division and unlikely to make the play-offs, are better than the patriots, tied 1st in division and will, most likely, make the play-offs.

Also, putting the giants 1st, when the best team they have beaten, by these rankings, is the 16th ranked Texans? these stats are nice, but use some common sense.

4 Re: Week 9 DVOA Ratings

Depends on how you look at it, 6 out of the top 10 are AFC and 10 out of the top 15 are AFC. In that respect, the AFC is stronger.

21 Re: Week 9 DVOA Ratings

I'd side with the 'experts' on this.

I don't think DVOA handles really good teams playing really bad teams all that well, and thats pretty much what the entire NFC is.

48 Re: Week 9 DVOA Ratings

I don't subscribe, so I don't know what the official pick was, but by my calculations, DVOA favored KC by about 4.5 points:

prediction = [KCweightedDVOA - OAKweightedDVOA - HFA] * avePts/Gm
... = [29.1 - (-9.1) - 17.5] * 21.7
... = +4.5

I don't know if I got HFA exactly right, but I think I remember hearing 17.5 once upon a time. And I think the time period that DVOA is scaled to had a lower scoring average (I used this year's). But plugging in slightly different numbers won't make a huge difference. The Chiefs were solidly favored, but a snot-inducing beating would have been as big of a surprise as what actually happened.

55 Re: Week 9 DVOA Ratings

I remember HFA being 17%, from what Aaron posted several years ago. I don't know if he's adjusted it since then, but using 17.5% should be fine.

In my pick 'ems league I used DVOA comparisons pretty exclusively, though not to the point of making exact calculations for every matchup, and I only gave KC 5 confidence points, fairly low out of 13 total games.

60 Re: Week 9 DVOA Ratings

This formula limits the largest possible point differential to 14 points between the best and worst team on an average year. Each week there are multiple games that end with 20+ point differential.

Not only that, it also creates a 17% dead zone which means the DVOA does not favor one team over another if they are within 8 or so teams within the rankings.

This is useless.

79 Re: Week 9 DVOA Ratings

This formula limits the largest possible point differential to 14 points between the best and worst team on an average year. Each week there are multiple games that end with 20+ point differential.

Yep. And how many times per year is a team favored in Vegas by 20+ points? Without looking it up, my guess is zero. I fail to see how this is a black mark on the formula. Are you claiming that a formula which does not predict outliers correctly is useless? Should DVOA have had Cleveland favored by 20 over New England heading into last week?

Not only that, it also creates a 17% dead zone which means the DVOA does not favor one team over another if they are within 8 or so teams within the rankings.

Did you not understand that "HFA" means "home field advantage?" Because your claim here is wrong. What is correct is that a team must be be 17% better than their opponent in order to be favored everywhere, even on the opponent's home field.

This is useless.

If you're talking about your comment, I agree.

125 Re: Week 9 DVOA Ratings

The only time I remember a team being favoured by more than 20 points in recent times was the '07 Patriots at home to the AJ Feely led Eagles about 2/3rds the way through that season.

I think the Patriots won the game by 5 points, and could easily have lost.

134 Re: Week 9 DVOA Ratings

The Patriots won by three points, actually, and IIRC they were still favored by 20 points the next week against the Ravens, who were starting Kyle Boller. They also won that game by only three, and they were quite lucky just to win it. (I believe the line in the Eagles game reached 24 points, but I'm not sure if it moved back a bit before the game started. Also, who the heck is that in the photo for A.J. Feeley?)

132 Re: Week 9 DVOA Ratings

I am not sure why you are comparing this to Vegas? When did I say Vegas has better estimates? Does Vegas lines are an estimate of the result or the line that draws the most gamblers to gamble? I find Vegas lines yo be useless too as an estimate of final point differential.

Yes I did not understand HFA means "home field advantage". So they give ~3 points to home team. I guess that is what Vegas do. Is there really any study says that HFA is 3 points?

I am glad you find that formula useful. Whatever floats your boat. I on the other hand question a supposedly scientific prediction that puts just 4.5 points between Chiefs and Raiders after the same system suggests one is the best team in the league and the other is #22.

146 Re: Week 9 DVOA Ratings

"Overall, home teams score 52.5% of all points through the entire game. " For that to be 3 points of HFA, 60 points must be scored on the average. It seems like HFA is ~2 points.

136 Re: Week 9 DVOA Ratings

I am not sure why you are comparing this to Vegas?

Because Vegas is a commonly used standard with readily available historical line info. It's often used as a barometer to gauge how well a prediction system does, and I don't know of any systems that are consistently much better. If you do, I'd honestly love to know about them (not being facetious here).

When did I say Vegas has better estimates?

You didn't, and I didn't say you did. I was trying to point out that your complaint also applied to something usually accepted as a relatively accurate standard (not knowing your position was so far outside of mainstream).

Does Vegas lines are an estimate of the result or the line that draws the most gamblers to gamble?

From my understanding, they are closer to the former than the latter (the line that drew the most gamblers would be something totally wrong, that everyone could bet against and win). But what might be more accurate is the idea that they attempt to balance the amount wagered on either side of the line. Though there seem to be instances where they do NOT move the line, despite a high proportion of bets coming in on one side. This is taken by some to be evidence that Vegas does indeed shade the line a bit towards what they think is an accurate estimate of the result (and away from the number that balances the bets).

I find Vegas lines yo be useless too as an estimate of final point differential.

I'm surprised you feel that way. I seem to remember seeing historical data showing that for any given spread, the actual resulting score margins cluster around the spread. Unfortunately, a Google search turns up nothing, so maybe I'm misremembering. Again, if you know of a significantly more useful system, I'd love to hear about it. Or, if you know of data showing that Vegas has some kind of bias, that would be interesting as well.

Yes I did not understand HFA means "home field advantage". So they give ~3 points to home team. I guess that is what Vegas do. Is there really any study says that HFA is 3 points?

Sorry about the confusion. Yes, there have been plenty of studies on home field advantage, which generally come up with an average value in the neighborhood of 3 to 3.5 points. However, I have seen studies that seem to indicate that HFA shouldn't be represented by a constant value, but should depend on the disparity in team quality, the actual teams in question, the time zone differences between the teams, etc. In the end, most formulas I've seen just throw their hands in the air and award a constant point value to the home team.

I am glad you find that formula useful. Whatever floats your boat. I on the other hand question a supposedly scientific prediction that puts just 4.5 points between Chiefs and Raiders after the same system suggests one is the best team in the league and the other is #22.

Well, first of all, that 4.5 becomes 8 on a neutral field, or 11.5 in Kansas City. Second, focus on the weighted DVOA values, not the rankings. Teams #1 through #6 were tightly clustered that week, and I believe had lower values than the top team would usually have. Third, and most importantly, I think you're looking for a degree of certainty that just doesn't exist in the NFL. When the eventual Super Bowl winning team can lose 31-0 against a team that ends up 6-10, followed the very next week by winning 31-10 against an eventual 12-4 team (see http://www.pro-football-reference.com/teams/nwe/2003.htm), it would be foolish to be too sure of yourself.

142 Re: Week 9 DVOA Ratings

"From my understanding, they are closer to the former than the latter (the line that drew the most gamblers would be something totally wrong, that everyone could bet against and win). But what might be more accurate is the idea that they attempt to balance the amount wagered on either side of the line. Though there seem to be instances where they do NOT move the line, despite a high proportion of bets coming in on one side. This is taken by some to be evidence that Vegas does indeed shade the line a bit towards what they think is an accurate estimate of the result (and away from the number that balances the bets)."

The "accurate" line usually winds up being the same as the "even number of bets" line. Why?

Well, while the general public betting might be misinformed, or tend to favor teams with large fan bases (Cowboys, Steelers, Bears, etc.), the "professional" gamblers all have systems that, for the most part, work fairly well. Therefore, if a line is too far to one side, the professionals, who bet larger amounts of money that the public, will hammer the other side, evening out the line.

Overall, you're right; Vegas does a very good job at setting lines.

145 Re: Week 9 DVOA Ratings

"Again, if you know of a significantly more useful system, I'd love to hear about it."
I don't know a better prediction. But I do not search one for either. I don't really care about guessing the results of the games correctly. I am not a gambler either.

Here is what DVOA predicted based on the formula for week 9: (difference, DVOA prediction, DVOA error, sorry for the formatting)
NYJ-DET 3 :0.5:2.5
MIA-BAL -16:-2.8:-13.2
NE -CLE -20:2.8:-22.8
SD -HOU 6:-0.7: 6.7
CHI-BUF 3,:-0.8:3.8
ARI-MIN -3:-11.0 :8.0
NY -SEA 34:4.3:29.7
IND-PHI -2:-6.4:4.4
KAN-OAK -3:4.8:-7.8
DAL- GB -38:-10.2:-27.8
PIT-CIN 6: 3.3: 2.7

On the average teams score ~22 points. And DVOA erred more than that number on 3 of 11 games. It got within a field goal of only 2 games.

"Third, and most importantly, I think you're looking for a degree of certainty that just doesn't exist in the NFL."
You know, if it is not predictable, then they should not make predictions.

150 Re: Week 9 DVOA Ratings

I don't really care about guessing the results of the games correctly.

Then why are you complaining that DVOA didn't guess the results of the games correctly?

You know, if it is not predictable, then they should not make predictions.

It is predictable. It's just that the variation around those predictions is high. When the weatherman says there is a 60% chance of rain tomorrow, that still means that 40% of the time it won't rain.

152 Re: Week 9 DVOA Ratings

"Then why are you complaining that DVOA didn't guess the results of the games correctly?"
I am complaining that it does not work yet people like you treat it as if it works.

If the results are predictable DVOA is not doing a good job. It does not matter if it is slightly better than Vegas lines. The definition of good is not "slightly better than one other thing". If you think DVOA is doing a good job, please prove it. I bet you can't. I have never seen FO come up with anything that proves the predictive power of DVOA. Had it predictive power, they would have been ridiculously rich by now.

153 Re: Week 9 DVOA Ratings

I am complaining that it does not work yet people like you treat it as if it works.

When I read this, I thought "What? I've acknowledged repeatedly in this thread that the predictions have variation, and that one shouldn't take them as gospel." But then I went back and read my comments, and realized I wasn't too clear about that:

"The Chiefs were solidly favored, but a snot-inducing beating would have been as big of a surprise as what actually happened." ... I should have added something like "i.e. not a very big one." What I meant was that DVOA wasn't taking a very forceful position, and that either of those results would have been not all that surprising. What would have been surprising was an Oakland blowout.

"Are you claiming that a formula which does not predict outliers correctly is useless? Should DVOA have had Cleveland favored by 20 over New England heading into last week?" ... My point here was that good teams lose to bad teams all the time, i.e. there is a ton of variation. Again, though, that doesn't change the fact that the expected result of this game (if played a million times) is NOT centered around a final margin of 0. So the best prediction was probably something like New England by a few points.

"I think you're looking for a degree of certainty that just doesn't exist in the NFL. When the eventual Super Bowl winning team can lose 31-0 against a team that ends up 6-10, followed the very next week by winning 31-10 against an eventual 12-4 team (see http://www.pro-football-reference.com/teams/nwe/2003.htm), it would be foolish to be too sure of yourself."

"the variation around those predictions is high."

I think the last two don't really need more explanation. But to be clear, I take the following positions:

1) Some NFL teams are better than others.
2) There is a lot of variation in NFL results.
3) Given 1 & 2, the best prediction for a specific game will often be one that favors one team, but not strongly.
4) A prediction system that, over the long run, is more accurate than guessing random numbers (or always guessing a margin of 0, or always guessing +3 for the team you think is better) is useful.
5) A prediction system is extremely useful if it can consistently beat Vegas.

I believe DVOA satisfies #4. I would guess it fails to satisfy #5.

Look, this whole thing started when I used the formula to try to show that even the difference between the "best" team and a below average team is not that large. I think the biggest things we're disagreeing on here are the definitions of "predictable" and "useful."

156 Re: Week 9 DVOA Ratings

Thanks for taking the time to explain your ideas further in a very respectful manner. I appreciate it.

I understand that the prediction of point differentials is extremely hard and within that complexity you think that DVOA is doing a decent job.

I believe DVOAs value is not predictive but it is very valuable as giving insight to what happened.

If you look at the DVOA at week X and predict the results for week X+1 you will find out that most of the time the results are out of place. You can explain that with variance but, though a valid excuse, it does not change the fact the tool failed to do its job. You can use DVOA to look into playoffs mid season and it may be slightly better than win-loss information on who would end up in playoffs, but not much more than win-loss adjusted for strength of schedule. My point is, until someone shows me the DVOA predicting things in a fairly consistent manner, I will continue to dismiss it as a predictive tool. As you pointed out, we disagree on what can be considered "useful" within this context. Not surprising given that useful is not a quantitative adjective.

Getting back to the non-predictive value of DVOA. If you look at the DVOA at the end of the year and then generate the expected point differential for each game and compare the expected results with the actual ones, you would see that they are much more closer. That is not very surprising as DVOA uses an iterative fitting model based on first down/yardage success, which has high correlation with points scored.

IMO, the value of DVOA comes from that it explains what contributed the success and failure . If it was run offense, run defense, special teams, etc. and how much of each.

As a football fan I cannot follow all the teams and DVOA provides me an approximate analysis of the teams I do not/cannot follow. So it is good to have DVOA. But, had I been a gambler, I wouldn't have used it to make bets when I am in Vegas.

160 Re: Week 9 DVOA Ratings

IMO, the value of DVOA comes from that it explains what contributed the success and failure . If it was run offense, run defense, special teams, etc. and how much of each. As a football fan I cannot follow all the teams and DVOA provides me an approximate analysis of the teams I do not/cannot follow.

Totally. I think we are much more in agreement on this than we are at odds about the predictive value of DVOA.

Politicians: please take note of how we handled this.

115 Re: Week 9 DVOA Ratings

Considering they are saying that this is a year where the highest and lowest DVOAs are less extreme than typical, if you put Giants at home against the Panthers it would be about a 20 point spread according to this formula. Where does the 14 come from. I am thinking that the average/game is a constant 21.7 since both KC and OAK average higher than that per game. Where does the 14 point max spread come from?

118 Re: Week 9 DVOA Ratings

I think maybe he didn't understand that HFA was a home field advantage adjustment. If you assume that you always subtract 17% from the better team, then his assertion makes more sense.

86 Re: Week 9 DVOA Ratings

audacity, i'd be interested to see what their breakdown is by confidence group. i use a simpler formula using their estimated wins and have found the closer games seem to be better picks than the ones where the formula predicts a big spread knockout (case in point KC the last two weeks). But NeverSurrender is right, DVOA isnt designed to have predictive value.

74 Re: Week 9 DVOA Ratings

I always keep in mind that DVOA is, and cannot be other than, an essentially backwards-looking rating system. It's very interesting and is useful if you want to get deeper than the typical water cooler talk. It gives us lots of ways to categorize and describe performance. However, when we start talking about next Sunday's games, it quickly loses its potency . . . not that there's anything wrong with that.

93 Re: Week 9 DVOA Ratings

I disagree with this completely. I think this is a common misunderstanding of predictive systems. They are not designed to and cannot possibly see the future. However taking an informed view on what is likely to happen is the exact purpose of any predictive system. The issue is that at the beginning of Sunday there are many potential outcomes for each game, but by the end of Sunday there is only the realized outcome for each game. This is not a failing of DVOA but a misunderstanding of the nature of forecasting. This is why weather guys are the butt of a lot of jokes too. Well that and the ridiculous stage names. DVOA attempts to capture the actions of teams which are repeatable and more predictive of a given level of success to better understand who will perform well and who will perform poorly from week to week.

96 Re: Week 9 DVOA Ratings

You bring up a very good point. Brian Burke at Advanced NFL Stats (yeah, he's smug at times, and I don't love his work, but he has some good stuff) said recently that, when he forecasts a game as 60% in favor of one team, "getting it right" doesn't mean that he picked the correct winner; rather, it means that he picked the correct winner 60% of the time.

101 Re: Week 9 DVOA Ratings

In reply to by RowdyRoddyPiper

I don't misunderstand predictive systems, and I don't disagree with a lot of what you've written . . . in fact, it seems fairly complementary to my point, as far as I can tell. You say you "disagree completely" but I'm not sure where the disagreement is supposed to be. I think you might be reading a little too much into what I wrote above.

106 Re: Week 9 DVOA Ratings

Aaron stated explicitly in FO's pre-season primer that DVOA is designed to balance analysis of past performance with predictions for future performance.

You stated that DVOA is basically not useful for predicting future games. This is not true, I use if for predicting games every week, and it usually does a very good job (at least as good as, and probably a bit better than the Vegas betting lines this season).

In any case, as long as past performance is somewhat predictive of future performance (more true than not in almost any field or endeavor, at least of the long haul), then evaluations of past performance will have useful predictive value.

116 Re: Week 9 DVOA Ratings

at the end of the day, if the top dvoa picks can be between 55-60% accurate against the spread then it has some useful value. the only thing we can really use dvoa, or any rankings system for that matter, when it comes to betting is whether or not a vegas line has value. I've used this model for five years, to varying degrees of success (2006 was especially good, 2009 not so much). The reason i trust the model is that it rarely gives me more than 2-3 games per week that are greater than 4.5 points from the vegas line (i'd be scared if i was getting 8 games per week)

arkaein, i would like to hear your definiton of "very good," and the model you use. My concern comes when dvoa falls in love with a team that underperforms on the scoreboard. The last few weeks, that team has been kansas city, and before that new orleans. That said though dvoa properly pegged one early this year (Cleveland, of which Vegas has caught up), and properly faded two others (Chicago and Minny).

126 Re: Week 9 DVOA Ratings

I don't subscribe to the picks, so i don't know for sure, but it also correctly pegged that Detroit were considerably improved early on. Indeed Detroit are the 'Against The Line' champions so far this year, covering in 7 out of 8 games to date.

130 Re: Week 9 DVOA Ratings

Well, I'd argue that any prediction system that is better than a coin flip has value. Requiring that a system be better than the Vegas lines seems fairly arbitrary to me, since going even 50% against the spread requires picking winners about 65% of the time. I take it you are thinking of value in a wagering sense, where any system would have to at least be as good as what Vegas is using.

As for DVOA this year, I just checked, and although I don't bet on games for money in my pick 'ems league I have missed on 40 games this year. I don't know the total number of games played, but it's probably about 130 (average of 14.5 games per week for 9 weeks). That means I'm picking winners at a rate of about 69%. My entire method is taking the two teams Weighted DVOA (or DAVE early in the season), adding 17% to the home team, and picking the team with the higher total. Once in a while I will tweak my picks based on extra knowledge that I know DVOA will not account for, such as a recent QB injury.

And FO's preseason predictions were brilliant for week 1. Using this formula I would have gone 14-2 in my picks, but I actually picked GB as slight DVOA underdogs over the Eagles and ended up 15-1. Since then my picks have not reached that spectacular level, but have still been very solid. Even if teams like KC seem to be underperforming DVOA right now, DVOA was really ahead of the curve to start the season.

I think that if DVOA seems to have a flaw it's that it has been developed to reach a bit more than the Vegas oddsmakers do. Each year it usually manages to predict a few teams exceeding or falling short of expectations that most conventional analysts miss, and it does it while still posting solid overall predictive values. Vegas oddsmakers are good at not buying into unwarranted hype, but stick closer to conventional wisdom than DVOA.

141 Re: Week 9 DVOA Ratings

Yes, this was my only point, as I reiterate below. I do not claim or think that DVOA is of no use for predicting future events. I just think that those predictions, and more pertinently all of the discussion that they generate, need to be tempered by the realization of where they come from, and what their limitations are.

128 Re: Week 9 DVOA Ratings

I wasn't specifically accusing you of misunderstanding how predictive systems work. I meant it generally. The element I disagree with is that you state DVOA can be no more than a backward looking ranking system. If I want a backward looking ranking system of teams I can use aggregate yardage stats, TDs thrown, etc. Those can also be used to predict the future expectation but I think without understanding why that happens your predictions are less than accurate.

140 Re: Week 9 DVOA Ratings

In reply to by RowdyRoddyPiper

I don't think that a DVOA-like predictive system "can be no more than" backward-looking. When I say that it's "essentially" backward-looking, what I mean is that it can (by definition) only make future predictions on the basis of past performances. In situations like football games, which involve complex variables that we either cannot or likely will not uncover, my view is that the past isn't especially predictive.

That doesn't mean DVOA doesn't get you anywhere. As others are pointing out, it improves on a coin flip by 5-10%. That's not negligible, but neither is it something I get excited about. So I think it's far more interesting and useful as a tool for talking about past performance.

So that's all I meant. I agree with your response to my original post, and I think (as I suspected) perhaps you were reading a little too much into my "essential" statement.

159 Re: Week 9 DVOA Ratings

Alles klar. I tend to believe that in the context of a team sport, with a relatively stable set of actors that past tends to be prologue but reasonable people can disagree. As with all models, it's essential to understand what the model simplifies or doesn't account for to properly utilize it. Honestly if you were able to improve a heads or tails result by 5-10% in the financial markets you'd be incredibly wealthy. Hell, shifting the edge in black jack 1-2% is pretty hard to do but gives you a fighting chance to at least stay on the table for a while. Also, do you have an engineering background? I've worked with a lot of people with that type of background and when they are used to tolerances in the 1/10,000th or smaller scale picking up a point or three of edge seems really unimpressive to them.

2 Re: Week 9 DVOA Ratings

Kolb also threw a last second Hail Mary interception, like the other QBs you listed.

Dallas has really let down the NFC East this year. If it weren't for their hilarious underperformance, the NFC East could lay claim to legitimately being the toughest division in football.

11 Re: Week 9 DVOA Ratings

Improved performance by Dallas would likely lower some of the other NFC east DVOA values. I have no idea how much, but it's sort of (though not entirely) a zero sum game. For some teams to look good, others have to look bad.

17 Re: Week 9 DVOA Ratings

Actually the opposite. Dallas' terrible opponent adjustments (NEW! IMPROVED! Now powered by Jon Kitna!) would be reduced by improved play, and since each division opponent plays Dallas twice, this would improve the NFC East numbers.

22 Re: Week 9 DVOA Ratings

Actually, shouldn't your idea and the previous one cancel each other out, assuming the opponent adjustments work correctly? That's the whole point of the D in DVOA. If the Cowboys were better, the other NFC East teams would perform worse against them (lower VOA), but the downward opponent adjustments would not be as severe - and the change in opponent adjustment would in theory counteract the worse performance (no change in DVOA).

Of course, that is over the long run. In a sample of only a few games, who knows what would happen.

33 Re: Week 9 DVOA Ratings

Not if they played better vs. non-division teams relative to in-division. If they'd put up a semblance of a fight the last 2 weeks, eg, the DVOA of teams that hadn't previously beaten them would improve slightly.

51 Re: Week 9 DVOA Ratings

True, but I didn't realize that's what we were talking about. Obviously if Dallas plays better against non-division teams and maintains their level of play against division teams, the division teams will look better.

But that seems kind of trivial. I'm more curious about the hypothetical situation where Dallas plays equally better in every game. And after thinking about it, I guess it is essentially a zero-sum game (ignoring some fluky stuff like missed kicks, weather, etc), so the rest of the league (minus Dallas) would have to be rated slightly lower. It makes intuitive sense that their most frequent opponents might take the brunt of that reduction.

36 Re: Week 9 DVOA Ratings

Yeah, the fact that the NFC east teams play each other twice is why I was assuming an uptick in the Cowboys DVOA would have some small effect at lowering the other NFC east teams' DVOA values.

I guess I was thinking of it something like this....

Actual DVOA :
Giants...30.9
Eagles...29.5
Redskins...-9.5
Cowboys...-23.4

Assumed "If the Cowboys were average" DVOA :

Giants...28
Eagles...27
Redskins...-12
Cowboys...0

Does anyone know for sure if it would really work this way? Does the "D" really only mean "Defense Adjusted" (i.e. is defensive or ST DVOA affected by the team being played)?

40 Re: Week 9 DVOA Ratings

Given that the Eagles haven't played the Cowboys yet, I doubt there would be all that much of an effect on their DVOA total regardless of how the Cowboys played - any effects would be indirect at this point.