Writers of Pro Football Prospectus 2008

Most Recent FO Features

HundleyBre15.jpg

» Futures: UCLA QB Brett Hundley

Beyond the immediate considerations of Hundley's potential, the quarterback's tape raises larger questions about the position.

09 Nov 2010

Week 9 DVOA Ratings

by Aaron Schatz

Perhaps the New York Giants were just miffed about last week's DVOA ratings, where they dropped five spots based entirely on changes in opponent adjustments. They took their frustrations out on the Seattle Seahawks, and now they are back on top. Their division rivals in Philadelphia are still close behind thanks to a win over the always-tough Indianapolis Colts, another one of those rare games where both teams emerge with positive DVOA. Philadelphia got 30.1% for the win, Indianapolis 20.2% for the loss. That's how close the game was, and how strong the opponent adjustments are for these two teams overall.

This week was the most upset-friendly of the NFL season so far, at least judged by DVOA ratings. (One piece of evidence: Our premium picks against the spread, which had been hitting at over 60 percent so far this season, went just 4-8-1 this week.) It wasn't just the surprising wins, like Cleveland over New England and Oakland over Kansas City. This week also had surprisingly close wins, like the Jets barely beating Detroit and Cincinnati coming within a few yards of edging past Pittsburgh. As a result, we've got a lot of movement in the rankings this week. Six different teams move by four or five spots, which is par for the course in subjective power rankings around the Internet but somewhat rare for the DVOA ratings once we've got a few games in the books.

Overall, the theme of parity still applies to the 2010 season, although the Carolina Panthers are starting to really stand out like a sore thumb. At least Buffalo and Arizona are playing some close games, but Carolina looks horrible. Still, the Panthers are far from "worst DVOA ever" status. Last year at this time, Oakland and Detroit were each below -50%, much worse than what Carolina has this year.

A couple of other interesting notes I noticed when going through this week's numbers:

  • As I noted on Twitter Sunday afternoon, this is the first season since 1971 where no team is either 7-1 or 8-0 after eight games. In 1971, the Redskins were 6-1-1 after eight games, but I can't find a season where the best team after eight games was only 6-2.
  • Although only four NFC teams rank in the top dozen for DVOA, the extreme stratification of the NFC causes our playoff odds report to list those four teams as four of the top five possibilities to win the Super Bowl. After all, those good AFC teams all have to go through each other -- not only to get to the Super Bowl, but to make it to the postseason in the first place. How ironic it would be if the unfortunate fans of Arlington, Texas had to watch not one but two Giants championship celebrations in a four-month period.
  • You may have noticed that the Innovative Stats box on the FO front page now shows the top teams in the last playoff odds, rather than the top teams in DVOA. Don't forget you can use the tabs in that box to get a quick look at the top teams and players this season without having to click through to individual stats pages.
  • The San Diego Chargers special teams not only is the worst of the DVOA Era, but they haven't even had one good week. San Diego's special teams DVOA has been below 0% in all nine games this season.
  • The Oakland Raiders story is swell and all, but when you look at the schedules it becomes clear how difficult it will be for them to get past Kansas City -- and stay ahead of San Diego -- to win the division. The Raiders still have to play at Kansas City and at San Diego. Their other five games include Pittsburgh, Indianapolis, and Miami, plus the inscrutable Jaguars, who honestly could beat Oakland 45-0 or lose to Oakland 45-0. Neither result would really surprise me.
  • Seattle has completely melted down over the past two weeks with DVOA of -98.3% against Oakland and then -108.9% against the Giants. Because of this, the 2-6 San Francisco 49ers actually have the best DVOA in the NFC West right now. Maybe they really could pull off a division title. The current DVOA playoff odds report gives all four teams in the NFC West an average projection between 6.5 and 7.1 wins, and all four teams are listed with between 19 percent and 29 percent chance of winning the division.

I think I've given my solution to the problem before, but if not, here it is: The NFL should change the rules to state that when no team in a division has at least eight wins, that team forfeits its postseason spot to a third wild card team, with the top wild card team now hosting a first-round playoff game. The solution to the NFC West problem isn't a wholesale change in the playoff structure, as some people have suggested. Even if we get a 7-9 division champion this year, it's not something we need to worry about more than once per decade. In general, I think it is important to maintain the integrity of the divisions because it makes those divisional games more important and thus strengthens rivalries. But there really should be something in place to prevent a losing team from making the playoffs in those rare years like 2010.

* * * * *

This week actually sees a small fix to the DVOA formula for both teams and quarterbacks. When I made my last big overhaul to DVOA a couple of years ago, it included a fix that lowered the penalty for interceptions on fourth down. However, fourth-down interceptions still do have some negative value, because of the possibility of a return. If the ball gets picked off on the line of scrimmage, that's obviously going to be worse than a simple incomplete pass, even if both plays result in the defense taking over control of the ball.

The exception, of course, is in the last two minutes of the game when a team is driving in an attempt to tie the game. At that point, any throw on fourth-down is basically a "what the hell" kind of play, where there are only two results: a complete pass that gets past the sticks, and anything else. Unfortunately, the DVOA formula was still giving a small amount of negative value to fourth-down interceptions at the end of a game. I have now changed this so that any fourth-down interception in the final two minutes is translated as an incomplete pass. This is the same way we treat passes which we manually mark as "Hail Mary," like the 50-yard downfield interception on second down with three seconds left. The change gives a small boost to a handful of quarterbacks who threw last-gasp fourth-down picks this season: Sam Bradford, Bruce Gradkowski, Philip Rivers, and Chad Henne (twice!). However, since the change only affects interceptions in the final two minutes, Trent Edwards is still penalized for his goal-line interception against Tennessee with 6:06 left in that boring Week 6 Monday Night Football game. (That one's a great example of the difference between an incomplete and a pick, actually -- an incomplete pass gives the Titans the ball back at their own one-yard line, but the pick gave them the ball at their own 20 thanks to the touchback.)

I won't have time to go back and make this change in previous seasons until after 2010 is finished, but the change is now made for both 2010 team stats and quarterback passing stats.

* * * * *

These are the Football Outsiders team efficiency ratings through nine weeks of 2010, measured by our proprietary Defense-adjusted Value Over Average (DVOA) system that breaks down every single play and compares a team's performance to the league average based on situation in order to determine value over average. (Explained further here.)

OFFENSE and DEFENSE DVOA are adjusted to consider all fumbles, kept or lost, as equal value. SPECIAL TEAMS DVOA is adjusted for type of stadium (warm, cold, dome, Denver) and week of season. WEIGHTED DVOA represents an attempt to figure out how a team is playing right now, as opposed to over the season as a whole, by making recent games more important than earlier games.

Opponent adjustments are currently at 90 percent strength and will steadily grow stronger until Week 10. As always, positive numbers represent more points so DEFENSE is better when it is NEGATIVE.

To save people some time, please use the following format for all complaints:

<team> is clearly ranked <too high/too low> because <reason unrelated to DVOA>. <subjective ranking system> is way better than this. <unrelated team-supporting or -denigrating comment, preferably with poor spelling and/or chat-acceptable spelling>

TEAM TOTAL
DVOA
LAST
WEEK
WEIGHTED
DVOA
RANK W-L OFFENSE
DVOA
OFF.
RANK
DEFENSE
DVOA
DEF.
RANK
S.T.
DVOA
S.T.
RANK
1 NYG 30.9% 6 31.2% 1 6-2 16.3% 6 -21.4% 1 -6.8% 31
2 PHI 29.5% 2 30.3% 2 5-3 26.6% 3 -4.9% 7 -1.9% 25
3 TEN 27.4% 4 26.7% 3 5-3 6.3% 13 -16.7% 3 4.5% 5
4 PIT 25.1% 5 24.9% 4 6-2 3.4% 17 -19.8% 2 2.0% 12
5 KC 21.2% 1 19.8% 6 5-3 12.1% 9 -9.6% 6 -0.4% 17
6 GB 21.1% 10 20.2% 5 6-3 14.5% 8 -10.8% 4 -4.3% 28
7 SD 18.4% 7 19.6% 7 4-5 23.3% 4 -10.7% 5 -15.5% 32
8 NE 17.8% 3 16.4% 9 6-2 28.8% 1 14.5% 27 3.5% 9
9 IND 14.9% 8 16.7% 8 5-3 23.1% 5 3.6% 20 -4.5% 29
10 ATL 13.3% 12 12.6% 10 6-2 16.3% 7 1.2% 17 -1.7% 24
11 BAL 11.0% 13 11.5% 11 6-2 9.8% 10 1.1% 16 2.4% 11
12 NYJ 9.9% 9 8.9% 14 6-2 2.4% 18 -2.5% 12 5.0% 4
13 NO 9.8% 14 9.8% 12 6-3 6.3% 12 -4.5% 8 -1.0% 20
14 CLE 7.6% 16 9.0% 13 3-5 3.9% 16 0.7% 15 4.4% 6
15 MIA 5.6% 11 6.8% 15 4-4 8.5% 11 2.3% 18 -0.6% 19
16 HOU 2.0% 15 1.4% 16 4-4 28.5% 2 25.4% 31 -1.0% 22
TEAM TOTAL
DVOA
LAST
WEEK
WEIGHTED
DVOA
RANK W-L OFFENSE
DVOA
OFF.
RANK
DEFENSE
DVOA
DEF.
RANK
S.T.
DVOA
S.T.
RANK
17 SF -4.2% 17 -0.4% 17 2-6 -7.2% 26 -3.3% 10 -0.3% 16
18 CIN -4.7% 20 -3.5% 18 2-6 4.3% 15 4.9% 22 -4.1% 27
19 DET -5.4% 19 -4.6% 20 2-6 -4.4% 23 5.2% 23 4.2% 8
20 OAK -6.0% 22 -3.6% 19 5-4 -8.1% 27 -2.0% 13 0.1% 14
21 MIN -8.3% 18 -7.8% 21 3-5 -5.9% 25 2.3% 19 -0.1% 15
22 WAS -9.5% 21 -8.4% 22 4-4 -5.1% 24 4.0% 21 -0.4% 18
23 TB -9.9% 23 -10.8% 23 5-3 -1.1% 19 9.7% 25 0.9% 13
24 JAC -13.6% 25 -14.6% 24 4-4 -1.2% 20 19.7% 29 7.3% 2
25 STL -18.0% 28 -16.2% 25 4-4 -14.1% 28 0.5% 14 -3.5% 26
26 CHI -19.9% 26 -21.8% 26 5-3 -29.2% 30 -3.5% 9 5.8% 3
27 DAL -23.4% 27 -25.6% 28 1-7 -4.1% 22 18.3% 28 -1.0% 21
28 DEN -24.7% 29 -24.6% 27 2-6 4.3% 14 24.4% 30 -4.7% 30
29 SEA -25.4% 24 -28.9% 29 4-4 -22.1% 29 10.8% 26 7.5% 1
30 BUF -28.7% 30 -29.0% 30 0-8 -2.6% 21 29.2% 32 3.1% 10
31 ARI -35.5% 32 -36.5% 31 3-5 -32.1% 31 7.6% 24 4.3% 7
32 CAR -42.9% 31 -43.4% 32 1-7 -44.7% 32 -3.3% 11 -1.4% 23
  • NON-ADJUSTED TOTAL DVOA does not include the adjustments for opponent strength or the adjustments for weather and altitude in special teams, and only penalizes offenses for lost fumbles rather than all fumbles.
  • ESTIMATED WINS uses a statistic known as "Forest Index" that emphasizes consistency as well as DVOA in the most important specific situations: red zone defense, first quarter offense, and performance in the second half when the score is close. It then projects a number of wins adjusted to a league-average schedule and a league-average rate of recovering fumbles. Teams that have had their bye week are projected as if they had played one game per week.
  • PAST SCHEDULE lists average DVOA of opponents played this season, ranked from hardest schedule (#1, most positive) to easiest schedule (#32, most negative). It is not adjusted for which games are home or road.
  • FUTURE SCHEDULE lists average DVOA of opponents still left to play this season, ranked from hardest schedule (#1, most positive) to easiest schedule (#32, most negative). It is not adjusted for which games are home or road.
  • VARIANCE measures the statistical variance of the team's weekly DVOA performance. Teams are ranked from most consistent (#1, lowest variance) to least consistent (#32, highest variance).
TEAM TOTAL
DVOA
W-L NON-ADJ
TOT VOA
ESTIM.
WINS
RANK PAST
SCHED
RANK FUTURE
SCHED
RANK VAR. RANK
1 NYG 30.9% 6-2 34.0% 6.9 1 -9.1% 29 2.0% 14 21.0% 26
2 PHI 29.5% 5-3 27.9% 6.9 2 5.5% 5 -2.6% 20 7.3% 6
3 TEN 27.4% 5-3 23.8% 6.4 3 4.5% 8 4.7% 10 9.9% 11
4 PIT 25.1% 6-2 21.9% 6.1 4 7.5% 3 -4.5% 22 10.0% 12
5 KC 21.2% 5-3 20.1% 5.9 7 -1.2% 20 -11.1% 29 13.2% 16
6 GB 21.1% 6-3 25.0% 6.1 5 -5.6% 26 3.5% 11 16.8% 20
7 SD 18.4% 4-5 15.8% 6.0 6 -3.3% 24 -4.0% 21 16.9% 21
8 NE 17.8% 6-2 15.9% 5.8 9 1.3% 15 2.8% 12 15.3% 19
9 IND 14.9% 5-3 13.0% 5.6 11 4.7% 7 5.4% 9 11.9% 13
10 ATL 13.3% 6-2 10.8% 5.7 10 2.2% 13 -12.1% 30 8.3% 9
11 BAL 11.0% 6-2 9.9% 5.8 8 1.0% 16 0.1% 17 5.5% 3
12 NYJ 9.9% 6-2 20.9% 5.2 14 -1.5% 21 0.6% 15 5.4% 2
13 NO 9.8% 6-3 13.9% 5.3 13 -10.8% 31 -8.2% 27 8.3% 8
14 CLE 7.6% 3-5 5.0% 4.8 15 10.5% 1 -4.8% 23 13.6% 17
15 MIA 5.6% 4-4 4.4% 5.4 12 5.4% 6 0.3% 16 20.8% 24
16 HOU 2.0% 4-4 -3.0% 4.5 17 7.7% 2 6.7% 7 21.0% 25
TEAM TOTAL
DVOA
W-L NON-ADJ
TOT VOA
ESTIM.
WINS
RANK PAST
SCHED
RANK FUTURE
SCHED
RANK VAR. RANK
17 SF -4.2% 2-6 -3.5% 4.5 16 -3.1% 22 -12.8% 31 31.1% 29
18 CIN -4.7% 2-6 -5.0% 3.8 22 3.5% 12 8.5% 2 7.1% 5
19 DET -5.4% 2-6 -7.1% 4.1 19 4.5% 9 -5.7% 24 7.8% 7
20 OAK -6.0% 5-4 4.2% 3.8 21 -4.3% 25 6.7% 6 33.2% 31
21 MIN -8.3% 3-5 -10.1% 3.9 20 0.0% 18 -0.2% 19 4.4% 1
22 WAS -9.5% 4-4 -4.1% 3.7 24 0.1% 17 8.0% 4 8.6% 10
23 TB -9.9% 5-3 -7.7% 4.4 18 -5.6% 27 -6.7% 26 12.4% 15
24 JAC -13.6% 4-4 -17.6% 3.7 23 4.3% 10 8.7% 1 22.2% 27
25 STL -18.0% 4-4 -7.2% 3.2 25 -14.5% 32 -6.2% 25 13.9% 18
26 CHI -19.9% 5-3 -11.1% 2.8 27 -10.4% 30 7.7% 5 19.0% 23
27 DAL -23.4% 1-7 -26.2% 2.7 28 3.8% 11 8.0% 3 32.6% 30
28 DEN -24.7% 2-6 -23.1% 2.4 29 1.8% 14 2.7% 13 17.0% 22
29 SEA -25.4% 4-4 -21.2% 3.0 26 -7.4% 28 -8.3% 28 38.4% 32
30 BUF -28.7% 0-8 -27.3% 2.4 30 6.6% 4 6.0% 8 5.7% 4
31 ARI -35.5% 3-5 -31.2% 2.3 31 -3.3% 23 -15.2% 32 23.9% 28
32 CAR -42.9% 1-7 -44.4% 1.5 32 -0.8% 19 0.0% 18 12.0% 14

Posted by: Aaron Schatz on 09 Nov 2010

162 comments, Last at 12 Nov 2010, 10:09pm by nat

Comments

1
by NJBammer :: Tue, 11/09/2010 - 5:46pm

The amazing thing is, almost all national pundits claim the cream of the AFC is better than the cream of the NFC, and yet DVOA claims the opposite is true.

3
by Kal :: Tue, 11/09/2010 - 5:54pm

The pundits do so because they care about wins, not who plays who and how they looked. A 6-1 NE team is clearly the best team in sports no matter what, because oooh, one whole loss!

Fortunately we don't have to look at things like that all the time.

I'm surprised how much lower NE went after losing to the Browns.

7
by Stats are for losers (not verified) :: Tue, 11/09/2010 - 6:08pm

I have nothing to add, I just thought of a good (not verified) joke. Apologies if someone's already used it.

147
by the cat in the box is dead (not verified) :: Thu, 11/11/2010 - 5:04am

Fair play to you, sir.

157
by Randy moss 1 (not verified) :: Fri, 11/12/2010 - 7:20am

It's a shame, then, that all that matters is wins.

The rankings make much more sense for individual players opposed to teams, you can't really say that the chargers, who are 4-5, third in division and unlikely to make the play-offs, are better than the patriots, tied 1st in division and will, most likely, make the play-offs.

Also, putting the giants 1st, when the best team they have beaten, by these rankings, is the 16th ranked Texans? these stats are nice, but use some common sense.

4
by Sundar (not verified) :: Tue, 11/09/2010 - 5:58pm

Depends on how you look at it, 6 out of the top 10 are AFC and 10 out of the top 15 are AFC. In that respect, the AFC is stronger.

21
by RichC (not verified) :: Tue, 11/09/2010 - 6:46pm

I'd side with the 'experts' on this.

I don't think DVOA handles really good teams playing really bad teams all that well, and thats pretty much what the entire NFC is.

44
by RickD :: Tue, 11/09/2010 - 8:34pm

Don't overrate DVOA. According to DVOA, the Chiefs should have beaten the snot out of the Raiders on Sunday.

48
by AudacityOfHoops :: Tue, 11/09/2010 - 9:10pm

I don't subscribe, so I don't know what the official pick was, but by my calculations, DVOA favored KC by about 4.5 points:

prediction = [KCweightedDVOA - OAKweightedDVOA - HFA] * avePts/Gm
... = [29.1 - (-9.1) - 17.5] * 21.7
... = +4.5

I don't know if I got HFA exactly right, but I think I remember hearing 17.5 once upon a time. And I think the time period that DVOA is scaled to had a lower scoring average (I used this year's). But plugging in slightly different numbers won't make a huge difference. The Chiefs were solidly favored, but a snot-inducing beating would have been as big of a surprise as what actually happened.

55
by Arkaein :: Tue, 11/09/2010 - 10:25pm

I remember HFA being 17%, from what Aaron posted several years ago. I don't know if he's adjusted it since then, but using 17.5% should be fine.

In my pick 'ems league I used DVOA comparisons pretty exclusively, though not to the point of making exact calculations for every matchup, and I only gave KC 5 confidence points, fairly low out of 13 total games.

60
by greybeard :: Wed, 11/10/2010 - 12:51am

This formula limits the largest possible point differential to 14 points between the best and worst team on an average year. Each week there are multiple games that end with 20+ point differential.

Not only that, it also creates a 17% dead zone which means the DVOA does not favor one team over another if they are within 8 or so teams within the rankings.

This is useless.

79
by AudacityOfHoops :: Wed, 11/10/2010 - 11:13am

This formula limits the largest possible point differential to 14 points between the best and worst team on an average year. Each week there are multiple games that end with 20+ point differential.

Yep. And how many times per year is a team favored in Vegas by 20+ points? Without looking it up, my guess is zero. I fail to see how this is a black mark on the formula. Are you claiming that a formula which does not predict outliers correctly is useless? Should DVOA have had Cleveland favored by 20 over New England heading into last week?

Not only that, it also creates a 17% dead zone which means the DVOA does not favor one team over another if they are within 8 or so teams within the rankings.

Did you not understand that "HFA" means "home field advantage?" Because your claim here is wrong. What is correct is that a team must be be 17% better than their opponent in order to be favored everywhere, even on the opponent's home field.

This is useless.

If you're talking about your comment, I agree.

125
by BJR :: Wed, 11/10/2010 - 3:30pm

The only time I remember a team being favoured by more than 20 points in recent times was the '07 Patriots at home to the AJ Feely led Eagles about 2/3rds the way through that season.

I think the Patriots won the game by 5 points, and could easily have lost.

134
by Semigruntled Eagles fan (not verified) :: Wed, 11/10/2010 - 6:00pm

The Patriots won by three points, actually, and IIRC they were still favored by 20 points the next week against the Ravens, who were starting Kyle Boller. They also won that game by only three, and they were quite lucky just to win it. (I believe the line in the Eagles game reached 24 points, but I'm not sure if it moved back a bit before the game started. Also, who the heck is that in the photo for A.J. Feeley?)

132
by greybeard :: Wed, 11/10/2010 - 5:33pm

I am not sure why you are comparing this to Vegas? When did I say Vegas has better estimates? Does Vegas lines are an estimate of the result or the line that draws the most gamblers to gamble? I find Vegas lines yo be useless too as an estimate of final point differential.

Yes I did not understand HFA means "home field advantage". So they give ~3 points to home team. I guess that is what Vegas do. Is there really any study says that HFA is 3 points?

I am glad you find that formula useful. Whatever floats your boat. I on the other hand question a supposedly scientific prediction that puts just 4.5 points between Chiefs and Raiders after the same system suggests one is the best team in the league and the other is #22.

135
by ammek :: Wed, 11/10/2010 - 7:04pm

Home teams average 3 points more than road teams, but the split is more complicated than that.

137
by AudacityOfHoops :: Wed, 11/10/2010 - 7:13pm

Sweet article.

146
by greybeard :: Thu, 11/11/2010 - 3:08am

"Overall, home teams score 52.5% of all points through the entire game. " For that to be 3 points of HFA, 60 points must be scored on the average. It seems like HFA is ~2 points.

136
by AudacityOfHoops :: Wed, 11/10/2010 - 7:10pm

I am not sure why you are comparing this to Vegas?

Because Vegas is a commonly used standard with readily available historical line info. It's often used as a barometer to gauge how well a prediction system does, and I don't know of any systems that are consistently much better. If you do, I'd honestly love to know about them (not being facetious here).

When did I say Vegas has better estimates?

You didn't, and I didn't say you did. I was trying to point out that your complaint also applied to something usually accepted as a relatively accurate standard (not knowing your position was so far outside of mainstream).

Does Vegas lines are an estimate of the result or the line that draws the most gamblers to gamble?

From my understanding, they are closer to the former than the latter (the line that drew the most gamblers would be something totally wrong, that everyone could bet against and win). But what might be more accurate is the idea that they attempt to balance the amount wagered on either side of the line. Though there seem to be instances where they do NOT move the line, despite a high proportion of bets coming in on one side. This is taken by some to be evidence that Vegas does indeed shade the line a bit towards what they think is an accurate estimate of the result (and away from the number that balances the bets).

I find Vegas lines yo be useless too as an estimate of final point differential.

I'm surprised you feel that way. I seem to remember seeing historical data showing that for any given spread, the actual resulting score margins cluster around the spread. Unfortunately, a Google search turns up nothing, so maybe I'm misremembering. Again, if you know of a significantly more useful system, I'd love to hear about it. Or, if you know of data showing that Vegas has some kind of bias, that would be interesting as well.

Yes I did not understand HFA means "home field advantage". So they give ~3 points to home team. I guess that is what Vegas do. Is there really any study says that HFA is 3 points?

Sorry about the confusion. Yes, there have been plenty of studies on home field advantage, which generally come up with an average value in the neighborhood of 3 to 3.5 points. However, I have seen studies that seem to indicate that HFA shouldn't be represented by a constant value, but should depend on the disparity in team quality, the actual teams in question, the time zone differences between the teams, etc. In the end, most formulas I've seen just throw their hands in the air and award a constant point value to the home team.

I am glad you find that formula useful. Whatever floats your boat. I on the other hand question a supposedly scientific prediction that puts just 4.5 points between Chiefs and Raiders after the same system suggests one is the best team in the league and the other is #22.

Well, first of all, that 4.5 becomes 8 on a neutral field, or 11.5 in Kansas City. Second, focus on the weighted DVOA values, not the rankings. Teams #1 through #6 were tightly clustered that week, and I believe had lower values than the top team would usually have. Third, and most importantly, I think you're looking for a degree of certainty that just doesn't exist in the NFL. When the eventual Super Bowl winning team can lose 31-0 against a team that ends up 6-10, followed the very next week by winning 31-10 against an eventual 12-4 team (see http://www.pro-football-reference.com/teams/nwe/2003.htm), it would be foolish to be too sure of yourself.

142
by Eddo :: Wed, 11/10/2010 - 10:28pm

"From my understanding, they are closer to the former than the latter (the line that drew the most gamblers would be something totally wrong, that everyone could bet against and win). But what might be more accurate is the idea that they attempt to balance the amount wagered on either side of the line. Though there seem to be instances where they do NOT move the line, despite a high proportion of bets coming in on one side. This is taken by some to be evidence that Vegas does indeed shade the line a bit towards what they think is an accurate estimate of the result (and away from the number that balances the bets)."

The "accurate" line usually winds up being the same as the "even number of bets" line. Why?

Well, while the general public betting might be misinformed, or tend to favor teams with large fan bases (Cowboys, Steelers, Bears, etc.), the "professional" gamblers all have systems that, for the most part, work fairly well. Therefore, if a line is too far to one side, the professionals, who bet larger amounts of money that the public, will hammer the other side, evening out the line.

Overall, you're right; Vegas does a very good job at setting lines.

145
by greybeard :: Thu, 11/11/2010 - 2:02am

"Again, if you know of a significantly more useful system, I'd love to hear about it."
I don't know a better prediction. But I do not search one for either. I don't really care about guessing the results of the games correctly. I am not a gambler either.

Here is what DVOA predicted based on the formula for week 9: (difference, DVOA prediction, DVOA error, sorry for the formatting)
NYJ-DET 3 :0.5:2.5
MIA-BAL -16:-2.8:-13.2
NE -CLE -20:2.8:-22.8
SD -HOU 6:-0.7: 6.7
CHI-BUF 3,:-0.8:3.8
ARI-MIN -3:-11.0 :8.0
NY -SEA 34:4.3:29.7
IND-PHI -2:-6.4:4.4
KAN-OAK -3:4.8:-7.8
DAL- GB -38:-10.2:-27.8
PIT-CIN 6: 3.3: 2.7

On the average teams score ~22 points. And DVOA erred more than that number on 3 of 11 games. It got within a field goal of only 2 games.

"Third, and most importantly, I think you're looking for a degree of certainty that just doesn't exist in the NFL."
You know, if it is not predictable, then they should not make predictions.

150
by AudacityOfHoops :: Thu, 11/11/2010 - 1:03pm

I don't really care about guessing the results of the games correctly.

Then why are you complaining that DVOA didn't guess the results of the games correctly?

You know, if it is not predictable, then they should not make predictions.

It is predictable. It's just that the variation around those predictions is high. When the weatherman says there is a 60% chance of rain tomorrow, that still means that 40% of the time it won't rain.

152
by greybeard :: Thu, 11/11/2010 - 2:29pm

"Then why are you complaining that DVOA didn't guess the results of the games correctly?"
I am complaining that it does not work yet people like you treat it as if it works.

If the results are predictable DVOA is not doing a good job. It does not matter if it is slightly better than Vegas lines. The definition of good is not "slightly better than one other thing". If you think DVOA is doing a good job, please prove it. I bet you can't. I have never seen FO come up with anything that proves the predictive power of DVOA. Had it predictive power, they would have been ridiculously rich by now.

153
by AudacityOfHoops :: Thu, 11/11/2010 - 3:33pm

I am complaining that it does not work yet people like you treat it as if it works.

When I read this, I thought "What? I've acknowledged repeatedly in this thread that the predictions have variation, and that one shouldn't take them as gospel." But then I went back and read my comments, and realized I wasn't too clear about that:

"The Chiefs were solidly favored, but a snot-inducing beating would have been as big of a surprise as what actually happened." ... I should have added something like "i.e. not a very big one." What I meant was that DVOA wasn't taking a very forceful position, and that either of those results would have been not all that surprising. What would have been surprising was an Oakland blowout.

"Are you claiming that a formula which does not predict outliers correctly is useless? Should DVOA have had Cleveland favored by 20 over New England heading into last week?" ... My point here was that good teams lose to bad teams all the time, i.e. there is a ton of variation. Again, though, that doesn't change the fact that the expected result of this game (if played a million times) is NOT centered around a final margin of 0. So the best prediction was probably something like New England by a few points.

"I think you're looking for a degree of certainty that just doesn't exist in the NFL. When the eventual Super Bowl winning team can lose 31-0 against a team that ends up 6-10, followed the very next week by winning 31-10 against an eventual 12-4 team (see http://www.pro-football-reference.com/teams/nwe/2003.htm), it would be foolish to be too sure of yourself."

"the variation around those predictions is high."

I think the last two don't really need more explanation. But to be clear, I take the following positions:

1) Some NFL teams are better than others.
2) There is a lot of variation in NFL results.
3) Given 1 & 2, the best prediction for a specific game will often be one that favors one team, but not strongly.
4) A prediction system that, over the long run, is more accurate than guessing random numbers (or always guessing a margin of 0, or always guessing +3 for the team you think is better) is useful.
5) A prediction system is extremely useful if it can consistently beat Vegas.

I believe DVOA satisfies #4. I would guess it fails to satisfy #5.

Look, this whole thing started when I used the formula to try to show that even the difference between the "best" team and a below average team is not that large. I think the biggest things we're disagreeing on here are the definitions of "predictable" and "useful."

156
by greybeard :: Fri, 11/12/2010 - 12:37am

Thanks for taking the time to explain your ideas further in a very respectful manner. I appreciate it.

I understand that the prediction of point differentials is extremely hard and within that complexity you think that DVOA is doing a decent job.

I believe DVOAs value is not predictive but it is very valuable as giving insight to what happened.

If you look at the DVOA at week X and predict the results for week X+1 you will find out that most of the time the results are out of place. You can explain that with variance but, though a valid excuse, it does not change the fact the tool failed to do its job. You can use DVOA to look into playoffs mid season and it may be slightly better than win-loss information on who would end up in playoffs, but not much more than win-loss adjusted for strength of schedule. My point is, until someone shows me the DVOA predicting things in a fairly consistent manner, I will continue to dismiss it as a predictive tool. As you pointed out, we disagree on what can be considered "useful" within this context. Not surprising given that useful is not a quantitative adjective.

Getting back to the non-predictive value of DVOA. If you look at the DVOA at the end of the year and then generate the expected point differential for each game and compare the expected results with the actual ones, you would see that they are much more closer. That is not very surprising as DVOA uses an iterative fitting model based on first down/yardage success, which has high correlation with points scored.

IMO, the value of DVOA comes from that it explains what contributed the success and failure . If it was run offense, run defense, special teams, etc. and how much of each.

As a football fan I cannot follow all the teams and DVOA provides me an approximate analysis of the teams I do not/cannot follow. So it is good to have DVOA. But, had I been a gambler, I wouldn't have used it to make bets when I am in Vegas.

160
by AudacityOfHoops :: Fri, 11/12/2010 - 2:48pm

IMO, the value of DVOA comes from that it explains what contributed the success and failure . If it was run offense, run defense, special teams, etc. and how much of each. As a football fan I cannot follow all the teams and DVOA provides me an approximate analysis of the teams I do not/cannot follow.

Totally. I think we are much more in agreement on this than we are at odds about the predictive value of DVOA.

Politicians: please take note of how we handled this.

115
by Michael (not verified) :: Wed, 11/10/2010 - 2:38pm

Considering they are saying that this is a year where the highest and lowest DVOAs are less extreme than typical, if you put Giants at home against the Panthers it would be about a 20 point spread according to this formula. Where does the 14 come from. I am thinking that the average/game is a constant 21.7 since both KC and OAK average higher than that per game. Where does the 14 point max spread come from?

118
by AudacityOfHoops :: Wed, 11/10/2010 - 2:54pm

I think maybe he didn't understand that HFA was a home field advantage adjustment. If you assume that you always subtract 17% from the better team, then his assertion makes more sense.

86
by Jetspete :: Wed, 11/10/2010 - 11:41am

audacity, i'd be interested to see what their breakdown is by confidence group. i use a simpler formula using their estimated wins and have found the closer games seem to be better picks than the ones where the formula predicts a big spread knockout (case in point KC the last two weeks). But NeverSurrender is right, DVOA isnt designed to have predictive value.

74
by Never Surrender (not verified) :: Wed, 11/10/2010 - 9:44am

I always keep in mind that DVOA is, and cannot be other than, an essentially backwards-looking rating system. It's very interesting and is useful if you want to get deeper than the typical water cooler talk. It gives us lots of ways to categorize and describe performance. However, when we start talking about next Sunday's games, it quickly loses its potency . . . not that there's anything wrong with that.

93
by RowdyRoddyPiper :: Wed, 11/10/2010 - 11:58am

I disagree with this completely. I think this is a common misunderstanding of predictive systems. They are not designed to and cannot possibly see the future. However taking an informed view on what is likely to happen is the exact purpose of any predictive system. The issue is that at the beginning of Sunday there are many potential outcomes for each game, but by the end of Sunday there is only the realized outcome for each game. This is not a failing of DVOA but a misunderstanding of the nature of forecasting. This is why weather guys are the butt of a lot of jokes too. Well that and the ridiculous stage names. DVOA attempts to capture the actions of teams which are repeatable and more predictive of a given level of success to better understand who will perform well and who will perform poorly from week to week.

96
by Eddo :: Wed, 11/10/2010 - 12:15pm

You bring up a very good point. Brian Burke at Advanced NFL Stats (yeah, he's smug at times, and I don't love his work, but he has some good stuff) said recently that, when he forecasts a game as 60% in favor of one team, "getting it right" doesn't mean that he picked the correct winner; rather, it means that he picked the correct winner 60% of the time.

101
by Never Surrender (not verified) :: Wed, 11/10/2010 - 1:05pm

I don't misunderstand predictive systems, and I don't disagree with a lot of what you've written . . . in fact, it seems fairly complementary to my point, as far as I can tell. You say you "disagree completely" but I'm not sure where the disagreement is supposed to be. I think you might be reading a little too much into what I wrote above.

106
by Arkaein :: Wed, 11/10/2010 - 1:41pm

Aaron stated explicitly in FO's pre-season primer that DVOA is designed to balance analysis of past performance with predictions for future performance.

You stated that DVOA is basically not useful for predicting future games. This is not true, I use if for predicting games every week, and it usually does a very good job (at least as good as, and probably a bit better than the Vegas betting lines this season).

In any case, as long as past performance is somewhat predictive of future performance (more true than not in almost any field or endeavor, at least of the long haul), then evaluations of past performance will have useful predictive value.

116
by Jetspete :: Wed, 11/10/2010 - 2:43pm

at the end of the day, if the top dvoa picks can be between 55-60% accurate against the spread then it has some useful value. the only thing we can really use dvoa, or any rankings system for that matter, when it comes to betting is whether or not a vegas line has value. I've used this model for five years, to varying degrees of success (2006 was especially good, 2009 not so much). The reason i trust the model is that it rarely gives me more than 2-3 games per week that are greater than 4.5 points from the vegas line (i'd be scared if i was getting 8 games per week)

arkaein, i would like to hear your definiton of "very good," and the model you use. My concern comes when dvoa falls in love with a team that underperforms on the scoreboard. The last few weeks, that team has been kansas city, and before that new orleans. That said though dvoa properly pegged one early this year (Cleveland, of which Vegas has caught up), and properly faded two others (Chicago and Minny).

126
by BJR :: Wed, 11/10/2010 - 3:36pm

I don't subscribe to the picks, so i don't know for sure, but it also correctly pegged that Detroit were considerably improved early on. Indeed Detroit are the 'Against The Line' champions so far this year, covering in 7 out of 8 games to date.

130
by Arkaein :: Wed, 11/10/2010 - 4:44pm

Well, I'd argue that any prediction system that is better than a coin flip has value. Requiring that a system be better than the Vegas lines seems fairly arbitrary to me, since going even 50% against the spread requires picking winners about 65% of the time. I take it you are thinking of value in a wagering sense, where any system would have to at least be as good as what Vegas is using.

As for DVOA this year, I just checked, and although I don't bet on games for money in my pick 'ems league I have missed on 40 games this year. I don't know the total number of games played, but it's probably about 130 (average of 14.5 games per week for 9 weeks). That means I'm picking winners at a rate of about 69%. My entire method is taking the two teams Weighted DVOA (or DAVE early in the season), adding 17% to the home team, and picking the team with the higher total. Once in a while I will tweak my picks based on extra knowledge that I know DVOA will not account for, such as a recent QB injury.

And FO's preseason predictions were brilliant for week 1. Using this formula I would have gone 14-2 in my picks, but I actually picked GB as slight DVOA underdogs over the Eagles and ended up 15-1. Since then my picks have not reached that spectacular level, but have still been very solid. Even if teams like KC seem to be underperforming DVOA right now, DVOA was really ahead of the curve to start the season.

I think that if DVOA seems to have a flaw it's that it has been developed to reach a bit more than the Vegas oddsmakers do. Each year it usually manages to predict a few teams exceeding or falling short of expectations that most conventional analysts miss, and it does it while still posting solid overall predictive values. Vegas oddsmakers are good at not buying into unwarranted hype, but stick closer to conventional wisdom than DVOA.

141
by Never Surrender (not verified) :: Wed, 11/10/2010 - 9:46pm

Yes, this was my only point, as I reiterate below. I do not claim or think that DVOA is of no use for predicting future events. I just think that those predictions, and more pertinently all of the discussion that they generate, need to be tempered by the realization of where they come from, and what their limitations are.

128
by RowdyRoddyPiper :: Wed, 11/10/2010 - 4:16pm

I wasn't specifically accusing you of misunderstanding how predictive systems work. I meant it generally. The element I disagree with is that you state DVOA can be no more than a backward looking ranking system. If I want a backward looking ranking system of teams I can use aggregate yardage stats, TDs thrown, etc. Those can also be used to predict the future expectation but I think without understanding why that happens your predictions are less than accurate.

140
by Never Surrender (not verified) :: Wed, 11/10/2010 - 9:40pm

I don't think that a DVOA-like predictive system "can be no more than" backward-looking. When I say that it's "essentially" backward-looking, what I mean is that it can (by definition) only make future predictions on the basis of past performances. In situations like football games, which involve complex variables that we either cannot or likely will not uncover, my view is that the past isn't especially predictive.

That doesn't mean DVOA doesn't get you anywhere. As others are pointing out, it improves on a coin flip by 5-10%. That's not negligible, but neither is it something I get excited about. So I think it's far more interesting and useful as a tool for talking about past performance.

So that's all I meant. I agree with your response to my original post, and I think (as I suspected) perhaps you were reading a little too much into my "essential" statement.

159
by RowdyRoddyPiper :: Fri, 11/12/2010 - 11:40am

Alles klar. I tend to believe that in the context of a team sport, with a relatively stable set of actors that past tends to be prologue but reasonable people can disagree. As with all models, it's essential to understand what the model simplifies or doesn't account for to properly utilize it. Honestly if you were able to improve a heads or tails result by 5-10% in the financial markets you'd be incredibly wealthy. Hell, shifting the edge in black jack 1-2% is pretty hard to do but gives you a fighting chance to at least stay on the table for a while. Also, do you have an engineering background? I've worked with a lot of people with that type of background and when they are used to tolerances in the 1/10,000th or smaller scale picking up a point or three of edge seems really unimpressive to them.

2
by Pass to Set Up ... :: Tue, 11/09/2010 - 5:52pm

Kolb also threw a last second Hail Mary interception, like the other QBs you listed.

Dallas has really let down the NFC East this year. If it weren't for their hilarious underperformance, the NFC East could lay claim to legitimately being the toughest division in football.

11
by dbostedo :: Tue, 11/09/2010 - 6:25pm

Improved performance by Dallas would likely lower some of the other NFC east DVOA values. I have no idea how much, but it's sort of (though not entirely) a zero sum game. For some teams to look good, others have to look bad.

17
by commissionerleaf :: Tue, 11/09/2010 - 6:37pm

Actually the opposite. Dallas' terrible opponent adjustments (NEW! IMPROVED! Now powered by Jon Kitna!) would be reduced by improved play, and since each division opponent plays Dallas twice, this would improve the NFC East numbers.

22
by AudacityOfHoops :: Tue, 11/09/2010 - 6:47pm

Actually, shouldn't your idea and the previous one cancel each other out, assuming the opponent adjustments work correctly? That's the whole point of the D in DVOA. If the Cowboys were better, the other NFC East teams would perform worse against them (lower VOA), but the downward opponent adjustments would not be as severe - and the change in opponent adjustment would in theory counteract the worse performance (no change in DVOA).

Of course, that is over the long run. In a sample of only a few games, who knows what would happen.

32
by Bjorn Nittmo (not verified) :: Tue, 11/09/2010 - 7:21pm

Not if they played better vs. non-division teams relative to in-division. If they'd put up a semblance of a fight the last 2 weeks, eg, the DVOA of teams that hadn't previously beaten them would improve slightly.

51
by AudacityOfHoops :: Tue, 11/09/2010 - 9:19pm

True, but I didn't realize that's what we were talking about. Obviously if Dallas plays better against non-division teams and maintains their level of play against division teams, the division teams will look better.

But that seems kind of trivial. I'm more curious about the hypothetical situation where Dallas plays equally better in every game. And after thinking about it, I guess it is essentially a zero-sum game (ignoring some fluky stuff like missed kicks, weather, etc), so the rest of the league (minus Dallas) would have to be rated slightly lower. It makes intuitive sense that their most frequent opponents might take the brunt of that reduction.

34
by Kurt :: Tue, 11/09/2010 - 7:21pm

Dallas has played two games against NFC East opponents, and IIRC in one of them they "beat" the Redskins according to DVOA.

36
by dbostedo :: Tue, 11/09/2010 - 7:34pm

Yeah, the fact that the NFC east teams play each other twice is why I was assuming an uptick in the Cowboys DVOA would have some small effect at lowering the other NFC east teams' DVOA values.

I guess I was thinking of it something like this....

Actual DVOA :
Giants...30.9
Eagles...29.5
Redskins...-9.5
Cowboys...-23.4

Assumed "If the Cowboys were average" DVOA :

Giants...28
Eagles...27
Redskins...-12
Cowboys...0

Does anyone know for sure if it would really work this way? Does the "D" really only mean "Defense Adjusted" (i.e. is defensive or ST DVOA affected by the team being played)?

40
by Semigruntled Eagles fan (not verified) :: Tue, 11/09/2010 - 7:41pm

Given that the Eagles haven't played the Cowboys yet, I doubt there would be all that much of an effect on their DVOA total regardless of how the Cowboys played - any effects would be indirect at this point.

50
by AudacityOfHoops :: Tue, 11/09/2010 - 9:11pm

Does the "D" really only mean "Defense Adjusted" (i.e. is defensive or ST DVOA affected by the team being played)?

No (yes).

5
by ChicagoRaider :: Tue, 11/09/2010 - 6:01pm

Does DVOA have a correction for games played in the rain? It appeared that a slippery ball in Oakland affected both offenses. That made the offenses look worse than they are, and the defenses better.

46
by Vincent Verhei :: Tue, 11/09/2010 - 8:55pm

It does, but it's smaller than you might think. The wet ball is usually accompanied by a wet field, which typically benefits the offense -- you're a lot more likely to slip and fall if you're reacting to somebody else than if you're acting on your own.

The biggest adjustment is for wind, which does play havoc with the passing and kicking games.

53
by AudacityOfHoops :: Tue, 11/09/2010 - 9:23pm

Whoa, what? I don't remember reading about weather adjustments. I'm going to have to dig into the "About DVOA" pages again to answer questions like:

Does the weather adjustment affect passing and rushing differently?
How do you determine the weather (weather.com? game book?)
Is the weather treated as a constant through the whole game?
Is it actually a weather "adjustment," or is it more that the "similar situations" that every play gets compared against are also binned by weather (besides down/distance/opponent strength/etc)?

EDIT: I looked in the linked "DVOA Explained" page (http://www.footballoutsiders.com/info/methods#dvoa), and the only weather adjustment I saw was for special teams. Can you point me towards some info about the weather adjustment for offense and defense?

6
by Karl Cuba :: Tue, 11/09/2010 - 6:06pm

Does FO do a weighted DVAO for units? I'm asking because I'm curious to see if there has been any change to the 49ers' offensive rating since Jimmy Raye left the building.

8
by observer (not verified) :: Tue, 11/09/2010 - 6:09pm

I understand that recovering lost fumbles is not a predictable skill. Still, having seen a lot of them, many of these jets fumbles that they recovered were fluky and furthermore many of them were under circumstances where there was no chance of the other team recovering.
If that accounts for a large part of the reason that they have the biggest adjustment, then I think we can expect them to play closer to that unadjusted figure going forward than the adjusted one.

Just a prediction.

14
by Kal :: Tue, 11/09/2010 - 6:27pm

Recovering fumbles is not a skill, but it can be predictive. It's just not something that can be trained easily. For instance, there's good statistical backing that when the QB fumbles the snap the offense will recover it most of the time. When a receiver fumbles the defense recovers more of the time. FO often talks about fumble luck in very simple terms (4 of 5 fumbles recovered) but I believe that they look into it a lot more in detail in terms of where the ball was fumbled and how it was fumbled.

9
by Sean McCormick :: Tue, 11/09/2010 - 6:20pm

The Giants have certainly been putting the hurt on people, but they've also played an unusually high number of games against backup quarterbacks. (Though of course, it must be mentioned that they pulled that trick by consistently knocking out the starter.) There are elements about their current run that remind me of last year's 5-0 start, in that they have run up against very few offenses that were in a position to attack the back seven successfully.

23
by starzero :: Tue, 11/09/2010 - 6:52pm

And one of the Giants' big losses was against Indy--back when the Colts could attack the back seven.

--
hail damage

62
by Bobman :: Wed, 11/10/2010 - 1:59am

But they didn't really, did they? That was when NYG played dime all game with 3-4 DEs rushing, and Indy just ran ran ran all night long. Right?

72
by ChaosOnion :: Wed, 11/10/2010 - 9:23am

NYG did that *specifically* to protect the back 7. They were afraid Peyton Manning would pick their secondary apart, hence the dime to shore up their pass coverage. Peyton was able to audibled to the run and take advantage of NYG's overcompensation.

So in a way, they did and they did not. NYG refused to let Peyton take advantage of the secondary, so they created a gaping disadvantage on run defense.

77
by Mark S. (not verified) :: Wed, 11/10/2010 - 10:53am

They have played 1 team with a really good passing attack - Houston. they held them to 16/34 for 171 (while also shutting down Foster completely).

The G-Men's secondary is, while I'm sure not as dominant as the D-Line, a solid unit. This isn't last year with Aaron Rouse and CC Brown manning the safety positions.

83
by Bobby Wommack (not verified) :: Wed, 11/10/2010 - 11:31am

I'm sorry, but Houston does not have a "really good" passing attack anymore. Last year, yes. This year, no.

Schaub has been average for most of the year. He has 10 TDs and 7 picks. He's thrown for under 200 yards three times, and only has two 300 yard games. His only real big game was against the Skins and their terrible secondary when he threw for over 400 yards.

89
by chemical burn :: Wed, 11/10/2010 - 11:45am

Backing up Bobby Wommack: Houston's DVOA is 11th in passing (in a mediocre year overall.) That's good, but that's not "really good" by any stretch. Plus, didn't the Giants plays them right when Johnson was really starting to tank because of his injuries?

I'm not saying the Giants aren't a great defense, but they really haven't been tested through the air, other than against the Colts. That game was very strange, of course, because the Giants came out with a very specific idiosyncratic plan for defending against the pass and the plan completely bombed. So I'm not sure that game says much one way or the other about their team or overall defensive ability, just that suffered from massive coaching failure (against a QB who will make you pay dearly for such failure.)

Another weird game where the Giants defense didn't exactly dominate was the Cowboys game, where Dallas was actually up 20-7 at one point, but because of turnovers and special teams. Then the Giants blew it open, the Cowboys scored a bunch of garbage time TD's, but had the game within a TD at the end. And that's with Kitna engineering the comeback.

I really don't know what to make of the Giants defense: they've undeniably looked awesome at times, but almost exclusively against chumps. I agree with FO that there's a lot to be said for stomping down chumps, but they've had such a weird season so far I think saying too much about their greatness is probably premature...

98
by thendcomes :: Wed, 11/10/2010 - 12:27pm

The points put up by the Cowboys was not the defense's fault, and I wouldn't be surprised if the defense DVOA reflected that.

All 20 of those points came off turnovers (Cowboys' drives started at NYG 5, 18, and 43, resulting in TD, FG, FG respectively) and a punt return for a touchdown.

The first garbage time touchdown, when NY was leading 38-20, was also off a turnover, Cowboys' drive started at NYG 15 and led to a touchdown.

The Cowboys' last drive started at the DAL 27, under 2 minutes and down 2 scores, and led to a touchdown.

121
by Mark S. (not verified) :: Wed, 11/10/2010 - 3:08pm

<>

Since Johnson had 5 for 95 in the game, followed the next week with an 8 for 138 against KC, I'm going to say probably not.

Also: Granted, they lost vs. TEN (8th ranked passing DVOA), but held them to 271 yards (110 passing) in a game they sorta dominated but threw away on penalties, turnovers, and ST.

They really only have 3 games remaining with teams higher than HOU in passing DVOA: the 2 Philly games and @GB, so I guess we will have to wait and see what they do in those.

123
by chemical burn :: Wed, 11/10/2010 - 3:23pm

Don't give me raw numbers on Johnson - they were forcing the ball to him even when he was clearly struggling with injuries. If he's going boom and bust with a low catch % but still amassing raw totals then that's actually to the defense's favor. The gulf between his DYAR and DVOA seems to indicate there's some of that going on. (look at some of the other guys with similar gulfs, dudes that are having the ball forced to them on struggling offenses: Calvin Johnson & Miles Austin.)

133
by Mark S. (not verified) :: Wed, 11/10/2010 - 5:41pm

He was targeted 13 times for 5 receptions and 95 yards against the Giants in Week 5, so yeah, low catch percentage but good yards. Maybe he wasn't 100%, maybe he was well-defended. Whatever the case, the following week he was targeted 13 times for 8 catches and 138 yards against KC, who by the way rank #2 in the league vs. the opponents' top wideout. So no, his season was performance was not in the tank due to injuries when he played the Giants.

120
by TomC :: Wed, 11/10/2010 - 3:00pm

An NFL scheduling quirk works out in our favor this year: we get two Philly/NYG games in the next six weeks. If Vick stays healthy, that Philly pass O vs. Giants pass D will be a fascinating matchup.

10
by Matty D (not verified) :: Tue, 11/09/2010 - 6:21pm

Remember waaaaay back in 2008, when everybody was complaining about how a 7-9 team would probably win the NFC West, and it would be terrible for football, and what are we going to do?

And then Kurt and Larry led the Cards to a 9-7 finish, went on a legendary barnstorming playoff roadshow, set a few postseason records, and nearly beat the favored Steelers?

Let's save the crying about a 7-9 division winner for when, you know, it *actually happens*.

15
by Spielman :: Tue, 11/09/2010 - 6:28pm

Shhh. Aaron's still not over that.

122
by Mark S. (not verified) :: Wed, 11/10/2010 - 3:21pm

Damn, I'm still not over that. They went 6-0 against their own crappy division, and 3-7 against everyone else. Then they beat a rookie QB making his first playoff start (vs. ATL) in the WC round. They were the beneficiary of one of Jake Delhomme's most epic of failures (5 INTs!) in the Divisional round, and through the quirk of NFL playoff seeding got a home game against the Eagles (who had a better record and played a tougher schedule) in the Championship game, where they jumped out to a 24-6 lead and barely survived the Iggles' 2nd half comeback to make the Super Bowl. Yes, they gave PIT a hell of game, but that doesn't mean they deserved to be there.

124
by chemical burn :: Wed, 11/10/2010 - 3:24pm

No - winning those games does mean they deserved to be there. It doesn't mean they were a great team.

143
by Mark S. (not verified) :: Wed, 11/10/2010 - 10:46pm

Um, No. Winning those games means that they were there, not that they deserved to be. Just like the strong possibility that this year's NFCW winner will be in the playoffs, despite not deserving to be.

144
by Raiderjoe :: Wed, 11/10/2010 - 11:07pm

Deserve aint got nothgin to do with it

148
by tuluse :: Thu, 11/11/2010 - 5:24am

Donovan McNabb was building a house!

151
by Spielman :: Thu, 11/11/2010 - 1:57pm

The NFL's playoffs exist to crown a champion in an entertaining manner, not to validate DVOA or even to reward the very best team with a championship.

You play the game according to the rules as they are set down, not according to whatever idealized rules you might prefer. By the rules of the NFL, the Cardinals deserved to be there. Period. You can whine about it, you can try to say that beating Matt Ryan and Jake Delhomme shouldn't count, and you can say that points scored in the first half shouldn't count, or whatever. It doesn't change reality.

154
by chemical burn :: Thu, 11/11/2010 - 4:11pm

yeah, that's all I mean. Their playoffs wins weren't from botched extra points or crazy tipped passes or penalty flubs or kick-off return lateral miracles that might have been forward passes or anything highly questionable of that sort. They had an unimpressive season but then went on a run in the playoffs. The playoff run was as legit as a playoff run can be.

158
by nat :: Fri, 11/12/2010 - 9:34am

FYI, there's a growing movement to mock people who "punctuate" their arguments with "Period". The rest of your post is clear and at least mildly persuasive. Do you really want to weaken it? Question Mark?

At Pound Dollar Asterisk Percent Pound Exclamation Point! Now you've got me doing it!

161
by Spielman :: Fri, 11/12/2010 - 7:16pm

There's also a growing movement to set fire to punctilious wannabe rhetorical critics. Do you really want that?

162
by nat :: Fri, 11/12/2010 - 10:09pm

Hey, chill there. I'm just having fun at ya! Colon right parenthesis.

19
by cfn_ms :: Tue, 11/09/2010 - 6:44pm

It's also worth noting that they're projecting that the mean result of the four teams is between 6.5 and 7.1, which I think means it's VERY likely that one of them hits eight wins. If the standard deviation on win totals for each team is something like 1.5 (just a guess), then each team is within a standard deviation of eight wins.

It's even more likely that at least one hits eight wins since they've all got a bunch left against each other; the Niners have yet to play either the Rams or Cardinals, and every other division matchup has happened exactly once.

So unless you think that the NFC West's non-division records going forward will just be atrocious (and that definitely hasn't happened yet), or that everyone's just going to simply split their remaining division games (also unlikely), I think that a 7 win or less division champ is VERY unlikely.

Of course, if the NFC West had been playing the AFC South or AFC East instead of the AFC West, a 7-win champ might have happened given the quality of the teams this year.

12
by Dan :: Tue, 11/09/2010 - 6:26pm

Which team is 6th in the NFC in DVOA?

San Francisco. So maybe the NFC West does deserve or playoff spot. Or perhaps the NFC should forfeit its last playoff spot to the AFC?

57
by Quincy :: Wed, 11/10/2010 - 12:14am

Excellent point. A 7-9 NFC West winner is only a crime if there's a more deserving team from another division getting left out. If one of Minnesota, Tampa Bay, Washington, or Chicago finishes with a winning record and looks like a good team by the end of the year, they can start complaining. Right now, I'm not sure which of the above we'd be shedding a tear for.

13
by Mostly Anonymous (not verified) :: Tue, 11/09/2010 - 6:26pm

This hasn't happened in 2010...

"But there really should be something in place to prevent a losing team from making the playoffs in those rare years like 2010."

... yet.

Out of the 10,000 playoff scenarios, I wonder how many had a sub-500 NFC West champ?

16
by RowdyRoddyPiper :: Tue, 11/09/2010 - 6:31pm

If you look at teams in the top 10 in DVOA, 6 out of 10 of them are in the bottom 10 of special teams performance. Conversely if you look at the bottom 10 teams by DVOA in the league, 5 out of 10 of them are in the top 10 for special teams performance including the # 1, 2 and 3 rated units. I did adjust out ST and used only OFF DVOA - DEF DVOA and reranked but it only impacted the order of the top and bottom 10, not if a team was in either bucket. I can't remember either of the Giant's two losses but if I were a betting man, I'd wager special teams did them in on at least one occasion. Though I generally don't bet on things that have already happened and can be easily researched.

20
by KG (not verified) :: Tue, 11/09/2010 - 6:44pm

Giants loss to Indy:
Perry Fewell went with an unorthodox defensive gameplan. He dared Peyton to audible to the run all night. Problem is he sold out on this front to the tune of starting in a dime defense with one defensive tackle, and they only carried two defensive tackles on the roster that night. It was an all-in gamble. It backfired, as Peyton and the Colts ran the ball far more effectively than the Giants anticipated, got up big early and then the midget DEs went to town with a 3 score lead.

Giants loss to Tennessee:
It's no insult to Tennessee, as they played well enough, but the Giants shot themselves in the foot this game, essentially losing it. A left-handed INT on the goal line, a fumble in the red zone, 5 personal fouls, etc etc. Chris Johnson had something around 25 carries for 70 insignificant yards until the game was effectively out of reach (he tacked on some put-the-game-away yards).

So surprisingly, STs has NOT cost us a game yet. It will at some point.

25
by K (not verified) :: Tue, 11/09/2010 - 6:59pm

On the other hand, San Diego would very likely be 7-2 or even 8-1 if their special teams were just 30th in the league (to say nothing of league-average).

54
by Bob in Jax :: Tue, 11/09/2010 - 10:02pm

San Diego 2009 S.T. DVOA was .3%; Jacksonville 2009 S.T. DVOA was -1.3%. San Diego lets its Pro Bowl special teamer, Kassim Osgood, walk at the end of 2009 season. Jacksonville signs him. Now, San Diego S.T. DVOA is -15.5%, Jacksonville S.T. DVOA is 7.3%.

Yes, yes, I know, but it sure is funny!

82
by graywh :: Wed, 11/10/2010 - 11:28am

Sounds a lot like the Brett Kern situation. Broncos were 6-0 with him last year. Then he was released and the 0-6 Titans signed him.

37
by Bjorn Nittmo (not verified) :: Tue, 11/09/2010 - 7:36pm

Oh, don't discount the ST's contributions to that Tennessee loss: 45 yard kickoff return; delay of game in FG formation leading to missed FG; another missed FG, giving TN ball at their own 43; following safety, worst free kick I've ever seen, giving TN ball at midfield.

38
by KG (not verified) :: Tue, 11/09/2010 - 7:39pm

Jesus Christ I forgot ALL of that. I think it's an instance of subconscious selective memory, as that was a depressing sight to behold in person.

The DOG -> Missed FG -> Personal Foul sequence was particularly hilarious.

STs certainly contributed. My bad.

18
by jmaron :: Tue, 11/09/2010 - 6:42pm

Why the hell should we worry about a losing team making the playoffs when the schedules for teams aren't in any way fair to begin with? There isn't even a pretense of fair in that the league makes up 1st place vs 2nd, 3rd and 4th place schedules.

24
by K (not verified) :: Tue, 11/09/2010 - 6:54pm

Teams in the same division play 14 of their 16 games against common opponents (Technically 12 of 16, since a team can't play itself.)

The 2 games that differ based on last year's finish are pretty minor, especially when you factor in year-to-year variance. For instance, New Orleans' two variable games are Minnesota and Dallas.

(Incidentally, this is a good reason why division winners NEED to make the playoffs. There's vastly more schedule overlap for same-division teams, so at least we're sure they're the best of an (arbitrary) group of four teams. Really not very different that World Cup or Olympic pool play.)

The real problem with unbalanced schedules happens when divisions have wildly varying levels of competence. (Look at the past and remaining schedule ranks for the NFC west teams. They all get to play each other twice.)

Unfortunately, I don't really know what the solution is to the problem of unbalanced divisions, except to hope that it evens out over decade-long time scales.

30
by jmaron :: Tue, 11/09/2010 - 7:10pm

You are dead on about that. But that at least is by happenstance for the most part - the actual setting of 1st place schedules is direct attempt to give weaker teams a better chance to compete.

The solution if anyone is concerned about fairness is to get rid of divisions and get rid of playoffs. Have a 15 game schedule. Have 16 teams in the NFC play 15 games against the other teams in the NFC. Do the same in the AFC. Have the team with the best record in the AFC play the best in the NFC two games total points in each stadium.

That process would tell you who was the best team far more often than the current one. But it certainly wouldn't be as exciting for the majority of teams.

42
by Jerry :: Tue, 11/09/2010 - 7:59pm

I suspect that setting first place schedules is about providing attractive matchups to television. (You could, I suppose, argue that making a first place team play the other two teams that finished first makes up for not having two games against a first place in their own division, but that sounds less reasonable to me.)

45
by RickD :: Tue, 11/09/2010 - 8:40pm

The unbalanced schedules justify the fact that every division gets at least one playoff team. The lack of balance is not intra-divisional, but inter-divisional. The division that plays the NFC West will tend to have better records than the division that plays the NFC East.
What you see as a bug I see as a feature. The NFL likes rivalries and the divisional framework produces rivalries.

66
by jebmak :: Wed, 11/10/2010 - 3:21am

I think that it is pretty fair. Every single team has four games against teams that finshed 1st in their division. Same with 2nd 3rd and 4th.

That doesn't account for differences in division strength, but it is about as fair as you can get while keeping the division structure.

26
by jmaron :: Tue, 11/09/2010 - 7:00pm

Observationally I really don't see any difference in quality of the top 5 AFC teams vs the top 5 NFC teams.

NYG, Philly, GB, NO, Atl, sure seem the match of Tenn, Pitt, KC, SD and NE to me.

There's only been 4 games I believe among those 4 (Tenn-Phil, Tenn-NYG, Pitt-NO, Pitt-Atl) with the AFC winning 4. The two wins by Tenn sure as heck didn't look to me as if Tenn was a superior team.

The next 6 in the AFC certainly seems a hell of a lot better though,

Indy, Balt, NYJ, Cle, Mia, Hous
vs
SF, Det, Minn, Wash, TB, StL

32
by wr (not verified) :: Tue, 11/09/2010 - 7:21pm

That should be 3. The Saints beat Pittsburg.

27
by Jerry F. (not verified) :: Tue, 11/09/2010 - 7:01pm

I feel like we didn't have these conversations as much with three divisions. I much preferred that format. Could we have some division de-alignment?

29
by cfn_ms :: Tue, 11/09/2010 - 7:09pm

Not unless you wanted contraction. 32 teams with 8 divisions of 4 teams each is the current system and it doesn't make much sense to change it. 30 teams with 6 divisions of 5 teams each would also make sense... but good luck figuring out which teams to throw out (maybe Jacksonville? but who else? and does the NFL really want to jettison teams?).

69
by Theo :: Wed, 11/10/2010 - 6:16am

The league had uneven divisions before. The 6 team AFC Central for example.
I like this format though. Too bad that the Texans take about 10 years to get rid of the 'expansion franchise' tag.

28
by kevinM (not verified) :: Tue, 11/09/2010 - 7:02pm

Unless the 2010 Giants' secondary includes players like Aaron Rouse, Can't Cover Brown, Michael Johnson, and Bruce Johnson, I think this year's back 7 is more difficult to attack.

The special teams hasn't cost them a game yet, but it's (in addition to turnovers) created a LOT of short field opportunities for their opponents. The "Drive Stats" has only 2 teams with a worst average starting field position for their defense than the Giants... the 1-15 Panthers and Bills.

31
by Temo :: Tue, 11/09/2010 - 7:19pm

By the way, Aaron: the link in the glossary for a description of the Forest Index's methodology appears to be broken.

35
by BlueStarDude :: Tue, 11/09/2010 - 7:28pm

RE: 'At that point, any throw on fourth-down is basically a "what the hell" kind of play, where there are only two results: a complete pass that gets past the sticks, and anything else.'

49ers wish that were true (and if we go back a few years, so do the Chargers and Marty [and Wade]).

Just having some fun, though: your general point is well taken: throwing an interception and hoping to cause a fumble and recover it isn't a great game plan or predictive event.

39
by jmaron :: Tue, 11/09/2010 - 7:39pm

I'm highly skeptical of teams that rank highly in DVOA but have a negative yds/play differential. NE and Atl have a differential over -.5. They are even worse in yds per pass attempt differential (-.7 and -.8).

I think NO is a far better team than people think. They had some key injuries - but they look as good as anyone to me when healthy.

56
by Pat (filler) (not verified) :: Wed, 11/10/2010 - 12:12am

New England has a negative net yards/drive, but it's essentially zero: they've got a positive points/drive, though, and significantly negative TOs/drive (which is a good thing). New England's been scoring much better than you'd expect at 33 yards/drive (second in the league only to Indy, and about 0.4 points/drive more than you'd expect), and their defense is well above average in interceptions/fumbles per drive.

So yeah, it's fair to be skeptical: they're mainly surviving on turnovers and probably a higher red zone conversion rate. Since those things are all small-numbers stuff, they could easily drop fast. Both of those things have some predictivity, though, so they also could persist.

87
by Bobby Wommack (not verified) :: Wed, 11/10/2010 - 11:42am

I'm sorry, but the Saints troubles have not been because of injuries. They lost Bush and Thomas. They've struggled all year because of the o-line and Brees has made a lot of big mistakes this year. He hasn't played to the same level as the past two years. Did injuries make Brees throw pick 6's against the Browns or Cardinals to lose? What about struggling to beat the Vikings, 49ers and Panthers (1st time)?

The Saints offensive dominance of last year isn't sustainable. They've come back to earth. The lack of running game has played a part, but last year was the only time the Saints had a legitimate running game. The offense wasn't this bad in '08.

The Saints are still good enough to beat anybody, but they can also lose to anybody. They are in the middle of the "good teams" pack just like everyone else right now. They were struggling since the 1st game of the year, even before the RB injuries.

102
by BJR :: Wed, 11/10/2010 - 1:07pm

In 2008 the Saints had the #1 ranked weighted offence according to DVOA. Last year they were #4. In 2007 they were also #4. In 2006 #7. They have achieved sustained excellent offensive performance since Drew Brees and Sean Payton arrived - until this year. Last year the Saints had the #1 ranked running game according to DVOA, but injuries have decimated that backfield. Brees has struggled shouldering a huge load, and hasn't been able to use play-action like he has in the past. But it seems pretty clear to me that if Thomas and Bush return healthy this will once again be a dominant offence.

104
by mm (not verified) :: Wed, 11/10/2010 - 1:26pm

For whatever reason, the interior of the offensive line has not been playing as well this year (the tackles have always been overrated). I think that has affected the offense more than the running back injuries.

41
by ammek :: Tue, 11/09/2010 - 7:46pm

Here's a one-off: the Packers have enjoyed the most advantage from weather conditions so far.

Sadly, everything else about their special teams is still below average.

47
by DisplacedPackerFan :: Tue, 11/09/2010 - 8:59pm

Indeed but things are improving.

Week 6: -6.7%
Week 7: -6.4%
Week 8: -5.6%
Week 9: -4.3%

Tim Masthay got special teams player of the week vs the Jets (and I agree with that the Packers lose the game if he didn't punt like that) and then the only two punts in the meaningless opportunities vs Dallas were quite good.

Sure the special teams break down as everything is below average. But if the punting unit performs the way it has the last 15 or so punts (it was mostly good vs Minnesota too) it will finish above average for the year.

Punt return I expect to stay right about where it is, a bit below average. The unit is mediocre at blocking and Tramon is mediocre on returns. They will put a few together and they will screw up a few. I expect the FG/XP portion to end up being just above average based on past history.

Kick offs will hover right around where they are as well since they are right about where they have been in the Mason Crosby years (last year they were worse but the coverage units have improved this year)

Kick returns I don't expect to improve. I still wonder if Sam Shields has vision issues even with how he played vs Dallas, it's just the way he tries to catch things. They are going to keep trying him on kick returns and while he might be able to bring one back for a TD, he also might fumble one for a TD.

So I could actually see the GB special teams ending up in the -2% range. An area that hurts them but still a marked improvement over what they were last year.

I still think Slocum is a bad special teams coach and I think the Packers management loving youth is problematic as well for special teams. Those units do get better working together and experience in the NFL does help on them.

Of course I want the punting to be like the last couple of games for another 2 games or so before I stop worrying about it while watching, but I actually have hope, based on the eyeball test and not just stats too. :)

43
by Raiderjoe :: Tue, 11/09/2010 - 8:14pm

DVoa drinkign egg nog early trhis year. How else to expalin Raiders 20 and Cheisf 5 and chagres 7.

Raiders are true cream of crop AFC West. Beta both KC and Sd on field. Not in cpmputer world or dugneons and dragons fantasy world.

49
by AudacityOfHoops :: Tue, 11/09/2010 - 9:11pm

Yes, a home overtime win is certainly extremely good evidence of superiority over your opponent.

58
by greybeard :: Wed, 11/10/2010 - 12:38am

It was an overtime win because of the referees. When the opponent gets an extra possession at your 30 yard line and gets a touchdown out of it it is harder to win in regulation.
Also before this game KC was #1 in DVOA and Oakland was #22 with 40% difference.
And there is no evidence of superiority that is better than winning. They don't give the superbowl to the team with the highest DVOA.

61
by dbostedo :: Wed, 11/10/2010 - 1:00am

"...there is no evidence of superiority that is better than winning."

Sure there is for any single game. Otherwise the better team would always win every game. Now across many games, that's true, which is why the DVOA formulas are designed to achieve the best possible correlation with winning (within the chosen success points scheme).

Even then, a single football season isn't really long enough and has too much variation in schedule strength to definitively prove who the better team is when records are close - there's too much luck/randomness involved. So you can probably say that a 12-4 team is definitively better than a 4-12 team, but that's not true with a 9-7 team versus a 7-9 team.

DVOA tries to account for the schedule strength aspect as much as possible and ignore luck, but it can't really account for everything. Ignoring the luck aspect of a team's record doesn't make it go away.

Case in point, the Raiders won, but I'd hardly say that proves that they are a superior team. They were superior on the day in the result, yes, but perhaps not in the performance, and not in performance over the course of the season.

64
by Whatev :: Wed, 11/10/2010 - 2:58am

You know what MY problem with DVOA is? My problem with DVOA is that with the Giants at 30.9% with a variance of 21% and 8 games played, and assuming the distribution is normal, the variance of their mean is 7.4%, meaning that the difference between them and the Patriots in 8th at a little less than 18% is not statistically significant at the (admittedly arbitrary, but conventional) 5% level. To make matters worse, with such a low n, you don't even have enough separate observations to apply the central limit theorem, which means their confidence interval is probably even larger than that. And even THAT assumes that their DVOA measurements of individual games are perfectly precise, which is highly unlikely!

It's entirely conceivable that any team in the upper third of DVOA rankings could be the "real best team." And who knows what things are like in the middle? Of course, any other statistical method would have these problems too, and just as bad. But the seriousness with which everybody keeps taking numbers that have huge standard errors is pretty absurd.

67
by tuluse :: Wed, 11/10/2010 - 6:00am

DVOA is per play, not per game. Giants have played 9 games, times about 50 plays per game, gives an n of 450. That seems to be a reasonable sample size to me.

68
by Whatev :: Wed, 11/10/2010 - 6:15am

I know that, but you're not getting 450 samples to measure that one thing. You're getting 450 samples which each require multiple adjustments to place them in context, which adjustments depend on evaluating hundreds of other samples themselves what need to be placed into context with adjustments. In short, you have 450 samples and a metric fuckton of degrees of freedom. I don't have all their data and don't have time to run a ton of regressions anyway, but it seems far more plausible that I'm underestimating their true variance by assuming that they can determine each game's "true rating" perfectly and then calculating their variance based on that as the unit of observation than that I'm overestimating it.

113
by asaltz :: Wed, 11/10/2010 - 2:21pm

I'd love to see the phrase "metric fuckton of degrees of freedom" get more use (on this site, academic literature, etc).

70
by Semigruntled Eagles fan (not verified) :: Wed, 11/10/2010 - 6:19am

You're actually understating this a bit - firstly, although the Giants have only played 8 games the average number of plays per game is higher - the Giants offense has run 542 plays from scrimmage. DVOA also takes into account their defense, however, which has defended 460 plays, for a total of a bit more than 1000 plays included in the DVOA analysis (as I don't feel like tracking down the number of plays on special teams and the non-plays due to penalties that would be included in the analysis).

Total plays taken from the NFL's official site:
Offense
Defense

71
by Whatev :: Wed, 11/10/2010 - 6:55am

Again, true, but once again, I'm telling you that those 1000 plays don't all measure the same thing. Not only do you have the problem of calculating adjustments based on the context of the plays, you also have the problem that the target itself doesn't sit still. Teams change from week to week, not only between seasons, and it's not clear what the degree of autocorrelation is between weeks, especially as the thing you're trying to measure receives regular shocks in the form of injuries.

I want to emphasize that I'm not saying that DVOA doesn't do anything. If it can beat the spread by even a small margin, they must be onto something. What I'm annoyed by is the fact that people, when arguing about what it all means, never take into account that it simply can't be a very precise number. There's not enough data and the data they have isn't sufficiently well-behaved and free of measurement error to get anything of the sort.

97
by RowdyRoddyPiper :: Wed, 11/10/2010 - 12:26pm

I agree with you that people tend to underestimate the issues with DVOA, but this is a common problem with statistical modeling of many things. It's not unique to modeling football team performance. Modeling economic systems has many of the same issues you've pointed out. Same for weather systems, epidemiolgy, etc. Actually football is a sight easier to model as it's a closed system with well known rules.

I'd argue that your contention the 1000 plays don't measure the same thing is completely wrong. The measurement is the success of the play. The fact that you put context around it changes nothing about what you are ultimately trying to measure, the success of a single play. Without looking at the model, I can't definitively say that it is well specified but the intuition behind it makes perfect sense to me.

139
by Whatev :: Wed, 11/10/2010 - 8:38pm

If you're arguing that it's just measuring the success of a single play, why bother come up with DVOA? You already have the yardage of the play, whether it got a first down, whether it scored. If you only want to see whether one play is successful, those measures are more than adequate. The question DVOA wants to answer is how successful teams are likely to be in the future, and the plays are the data they have to do that with. If DVOA does what it claims, the defensive plays measure primarily how successful they are on defense. The offensive plays measure primarily how successful they are on offense. Those are ALREADY not the same thing, and you don't get to claim that they have 1000 plays to measure one thing. They have ~500 plays each to measure TWO things, which they then aggregate together. When you do this the variance goes UP (additively), not DOWN. Moreover, as I pointed out last post, the players and schemes are different from play to play and game to game. Selection bias and autocorrelation are absolutely RAMPANT in this system.

At least in epidemiology, 100000 copies of a virus are largely fungible and you can run the same experiment on each, which is more than you can say about football players.

149
by DGL :: Thu, 11/11/2010 - 10:30am

"If you're arguing that it's just measuring the success of a single play, why bother come up with DVOA? You already have the yardage of the play, whether it got a first down, whether it scored. If you only want to see whether one play is successful, those measures are more than adequate."

Really?

Team A runs a play that picks up three yards and doesn't get a first down or score.

Team B runs a play that picks up 12 yards and doesn't get a first down or score.

If that information is more than adequate, then clearly Team B was four times as successful as Team A.

But if Team A's play was run on second and 4 from their opponent's 25 yard line leading 20-14 with three minutes left in the game, while Team B's play was run on third and 25 from their own 15 yard line trailing 14-0 at the start of the second quarter, I think most people would agree that Team A's play was far more successful.

155
by Whatev :: Thu, 11/11/2010 - 9:51pm

And I think most people would agree that a recovered fumble is a far more successful play than a lost one, but DVOA says that's not a predictive event.

111
by Still Alive (not verified) :: Wed, 11/10/2010 - 2:11pm

Exactly what I am thinking each week as I watch people wail and gnash their teeth about teams being 1 vs 10 when it means so very little. The numbers are interesting and useful, but they have very very large error brackets. If you graphed it scientifically it would be very unimpressive (of course as mentioned so would most economics et cetera).

112
by Still Alive (not verified) :: Wed, 11/10/2010 - 2:12pm

Exactly what I am thinking each week as I watch people wail and gnash their teeth about teams being 1 vs 10 when it means so very little. The numbers are interesting and useful, but they have very very large error brackets. If you graphed it scientifically it would be very unimpressive (of course as mentioned so would most economics et cetera).

138
by MainerRaider (not verified) :: Wed, 11/10/2010 - 8:16pm

Watching that game, I'd say neither team performed consistently well--possibly because sloppy conditions led to things like fumbles and a botched snap, and therein lies that luck limitation of DVOA. I agree that Oakland was slightly superior Sunday, but KC will probably prove the better team this season.

52
by Jonadan :: Tue, 11/09/2010 - 9:20pm

This is one of those times I'm really hoping the incomprehensibility is an act, because I'm tempted to agree with the sentiment behind it.

59
by tuluse :: Wed, 11/10/2010 - 12:46am

In the past 3 weeks Oakland's DVOA has increased over 30%. The computers do see them as a far improved team, you still can't ignore all those other games though.

http://footballoutsiders.com/dvoa-ratings/2010/week-6-dvoa-ratings

65
by Whatev :: Wed, 11/10/2010 - 3:00am

No problem, you know, if the Raiders are "fuhreal" they'll continue to win and fix the rankings.

75
by ChicagoRaider :: Wed, 11/10/2010 - 9:45am

I think that will be the case. Weighted DVOA is much more favorable than DVOA itself. The Raiders are still figuring out how their pieces fit together the best.

They changed QB and LT on offense, and McFadden is a totally changed RB this year. It looks like even more WR change may be coming now with injuries giving Ford an opportunity to advance himself. The change in McFadden is opening up more deep routes.

Defense has even bigger changes with changes at DE, Sam, Will, Mike and CB. And there are some really serious blitz/cover-0 packages.

Is there a team that has visibly changed more from last year? I can't think of one. And it can still get better.

85
by MilkmanDanimal :: Wed, 11/10/2010 - 11:34am

Tampa, perhaps? There's been significant improvement at QB, a rookie WR who has completely opened up the passing game, there's been some semblance of RB, and the defense is no longer made of wet paper, at least some of the time.

Could possibly also argue Cleveland based on the last few games.

63
by RickD :: Wed, 11/10/2010 - 2:49am

I'm wonder what would have to happen for the Pats to escape the 27 slot on defense. They have a good game on defense? They're still at 27. They have a terrible game? Still at 27.

84
by Led :: Wed, 11/10/2010 - 11:34am

I noticed that, too. It's pretty funny.

91
by Pat (filler) (not verified) :: Wed, 11/10/2010 - 11:51am

Yeah, that 27 slot is pretty wide: like, 6% in DVOA, about twice as wide as the gaps around it. There's a similar huge gap after the top 3 defenses, too.

73
by DeltaWhiskey :: Wed, 11/10/2010 - 9:39am

For those who are interested clustering by Standard Deviations:

OVERALL DVOA

ELITE (> 2SD)
NONE

GOOD (>1 SD, <2SD)
NYG
PHI
TEN
PIT
KC
GB

AVG (<1SD, >-1SD)
SD
NE
IND
ATL
BAL
NYJ
NO
CLE
MIA
HOU
SF
CIN
DET
OAK
MIN
WAS
TB
JAC
STL
CHI

BAD (<-1SD, >-2SD)
DAL
DEN
SEA
BUF
ARI

HORRID (< 2SD)
CAR

OFF DVOA

ELITE (>2SD)
NONE

GOOD (>1 SD, <2SD)
NE
HOU
PHI
SD
IND

AVG (<1SD, >-1SD)
NYG
ATL
GB
KC
BAL
MIA
TEN
NO
CIN
DEN
CLE
PIT
NYJ
TB
JAC
BUF
DAL
DET
WAS
MIN
SF
OAK
STL

BAD (<-1SD, >-2SD)
SEA
CHI
ARI

HORRID (< 2SD)
CAR

DEF DVOA

ELITE (>2SD)
NONE

GOOD (>1 SD, <2SD)
NYG
PIT
TEN
GB
SD

AVG (<1SD, >-1SD)
KC
PHI
NO
CHI
SF
CAR
NYJ
OAK
STL
CLE
BAL
ATL
MIA
MIN
IND
WAS
CIN
DET
ARI
TB
SEA
NE

BAD (<-1SD, >-2SD)
DAL
JAC
DEN
HOU

HORRID (< 2SD)
BUF

STDVOA
ELITE (>2SD)
NONE

GOOD (>1 SD, <2SD)
SEA
JAC
CHI
NYJ

AVG (<1SD, >-1SD)
TEN
CLE
ARI
DET
NE
BUF
BAL
PIT
TB
OAK
MIN
SF
KC
WAS
MIA
NO
DAL
HOU
CAR
ATL
PHI
STL
CIN
GB
IND

BAD (<-1SD, >-2SD)
DEN
NYG

HORRID (< 2SD)
SD

COMMENTS:
- Defense may/may not win championships, but so far this year, it's what makes GOOD DVOA team. Four out of the six GOOD teams have defenses that are GOOD.
- Take SD out of the S.T. calculations and the following teams have GOOD S.T.:
SEA
JAC
CHI
NYJ
TEN
CLE
ARI

and the following are bad:

STL
CIN
GB
IND
DEN
NYG

76
by bigtencrazy (not verified) :: Wed, 11/10/2010 - 9:55am

Chicago is such an annoyance. They parlay luck on defense and special teams into a winning record on a semi-regular basis. It's cr*ppy football to watch and worse yet to see be rewarded.

I relish Lady Luck giving this team the bum's rush and the beatings to follow.

78
by Independent George :: Wed, 11/10/2010 - 10:59am

Except it's not luck, really; Lovie has produced consistently good defenses and consistently excellent special teams ever since he's been coaching in Chicago.

ST
2010: 5.8% (3) [8 games]
2009: 4.1% (3)
2008: 4.2% (5)
2007: 9.1% (1)
2006: 7.6% (1)
2005: -2.6% (27)
2004: 1.9% (11)

DEF:
2010: -3.5% (9) [8 games]
2009: 7.1% (22)
2008: -6.8% (7)
2007: -6% (8)
2006: -19.7% (2)
2005: -21.6% (1)
2004: -7.2% (9)

If these numbers hold, it would be five consecutive Top-5 special teams, and six Top-10 defenses, in seven years with only a single below-average season for either in his entire tenure. That's a pretty good track record for a pretty good sample size, and it's why I continue to insist he's an excellent coach.

81
by DeltaWhiskey :: Wed, 11/10/2010 - 11:21am

The historical Standard Deviation for ST DVOA is %3.46, therefore in 7-years, his team has delivered two ELITE performances, three GOOD performanes, and 2 AVG performances.

The historical Standard Deviation for DEF DVOA is %10.96, therefore in 7-years, his team has delivered no ELITE performances, two GOOD performances (one of which does approach ELITE), and five average performances.

I don't think the data support Lovie Smith as an excellent coach given that 50% of the markers used above are in the average range and we haven't even looked at the offensive side of the ball, I can only assume (fairly safely I suspect) that it does not support the argument in any way shape or form.

To me, this is another example of how using ordinal rankings to make an argument is a bad idea, especially using "Top 10," as "Top 10" essentially means the to 1/3 of the league.

88
by AudacityOfHoops :: Wed, 11/10/2010 - 11:42am

I think once you start talking about repeating a performance, your ELITE/GOOD/AVG calculations become less useful. If a team is on the better end of average over and over again, you still might not be able to assume that any specific year is better than average, but you can infer that their combined performance over the whole period is above average.

100
by DeltaWhiskey :: Wed, 11/10/2010 - 1:00pm

I think it works better than the ordinal rankings. If the argument was he has been a good coach, I might buy it, but I find it hard to say he was excellent coach.

That being said, I agree in premise with what you're saying. Unless we look at a lot of performances, this one look means little. We need a sense of what the norm is for teams over this period.

94
by Independent George :: Wed, 11/10/2010 - 11:59am

I only included ST and Def because that's what the original comment talked about. His offenses have been horrendous - there's no disputing that. The point I'm making about Lovie is that his teams have been consistently good in 2 of 3 phases of the game for a pretty long period of time - and that's better most of the other coaches in the NFL.

95
by Pat (filler) (not verified) :: Wed, 11/10/2010 - 12:04pm

"I don't think the data support Lovie Smith as an excellent coach given that 50% of the markers used above are in the average range"

Consistently falling in the "average" range is pretty much the definition of a good coach: if *his* team's standard deviation on defense is on the order of the league's, ending up with 5 average and 2 good performances in 7 tries would indicate a 'true' level on the order of +1/3 standard deviations above average for defense, and +1 standard deviation in special teams.

Is that good? Dunno, you'd have to do something similar for all other coaches, but my guess is that it's probably pretty high.

"I don't think the data support Lovie Smith as an excellent coach given that 50% of the markers used above are in the average range"

Yeah, that's a different point. There's some weird behavior in the league where you've basically got two "styles" of "good" teams - great defense, mediocre/bad offense, and good defense, good offense. It's almost like if a coach focuses on offense, he gets a good offense and good defense, but if he focuses on defense, he gets a great defense and good offense (A scatterplot of offensive DVOA vs. defensive DVOA shows this pretty darn clearly). It's pretty clear that Lovie falls into the "focus on defense" side.

129
by DeltaWhiskey :: Wed, 11/10/2010 - 4:38pm

Okay, so I looked at the data (2000-2008, that's what I have right now). For inclusion on this list, in at least one category, there had to be 4 instances of being GOOD or better. I arbitrarly chose 4. Now I know the question is about coaches, and I just don't have the time to look it up and figure out which coaches go with what, but I think this gives an interesting look at organizations.

First thing to note, is that only 2-teams in that era had both OFF and DEF DVOA that were above average (GOOD): 2001 STL, 2004 PIT.

BAL: 4/9 years of GOOD TOTDVOA, 6/9 years of GOOD or ELITE DEF
BUF: 1/9 years of GOOD TOTDVOA, 5/9 years of GOOD or ELITE S.T.
CHI: 1/9 years of GOOD TOTDVOA, 3/9 years of Good DEF, 4/9 years GOOD or ELITE S.T.
DEN: 2/9 years of GOOD TOTDVOA, 5/9 years of GOOD OFF
IND: 6/9 years of GOOD TOTDVOA, 7/9 years of GOOD or ELITE OFF
KC: 3/9 years of GOOD TOTDVOA, 4/9 years of GOOD or ELITE OFF, 1/9 years GOOD S.T.
NE: 4/9 years of GOOD or ELITE TOTDVOA, 5/9 years of GOOD or ELITE OFF, 1/9 year of GOOD DEF, 1/9 year of GOOD S.T.
PHI: 6/9 years of GOOD TOTDVOA, 3/9 years of GOOD OFF, 2/9 years of GOOD DEF, 5/9 years of GOOD to ELITE S.T.
PIT: 5/9 years of GOOD TOTDVOA, 1/9 years of GOOD OFF, 4/9 years of GOOD to ELITE DEF
SD: 2/9 years of GOOD TOTDVOA, 4/9 years of GOOD OFF, 2/9 years of GOOD S.T.
TB: 2/9 years of GOOD TOTDVOA, 4/9 years of GOOD to ELITE DEF,

This snapshot validates the arguments that BAL, IND, NE, PIT, and PHI have been the cream of the crop in the NFL.

80
by Big Frank :: Wed, 11/10/2010 - 11:15am

You can't call it luck if that's what they're trying to do. They are specifically trying to cause fumbles and return kicks for TD's. When they find success in the things they're deliberately trying to do, by definition it can't be luck.

99
by Spielman :: Wed, 11/10/2010 - 12:47pm

Why not? If I buy a lottery ticket, it's because I'm trying to win the lottery. If I do win the lottery, does that mean that my winning wasn't luck?

90
by Jetspete :: Wed, 11/10/2010 - 11:49am

i'm surprised you dont have a good read on the Jags. They seem to be a classic mediocre team. They beat 3 putrid squads (Denver, Buffalo and post-Romo Dallas), and got murdered by 4 top seven squads. The only outlier being a victory in their annual super bowl home game against Indy. This week has Houston 38 Jacksonville 10 written all over it

92
by bigtencrazy (not verified) :: Wed, 11/10/2010 - 11:56am

Appreciate the feedback. And a poor choice of words on my part.

I just do not enjoy watching the Bears play. Everything has to go 'just so' for them to win and would seem to play better in college than the pros where mistakes are far more common. They seem like the Va Tech of pro football.

117
by TomC :: Wed, 11/10/2010 - 2:54pm

As a Bears fan, I (mostly) agree with you. The pass D in particular is like Chinese water torture (10 yards, drip, 12 yards, drip, 8 yards, drip, ...). On the other hand, the excellent blocking on special teams and the emphasis on returns when they finally do take away the ball on defense results in a higher-than-average rate of big, fun plays.

103
by Julio (not verified) :: Wed, 11/10/2010 - 1:19pm

How can the Pats be ranked #1 in offense after the
performance in Cleveland where they (barely) scored
14 points and couldn't maintain possession? They
have dropped to second behind Tennessee in pts/game
but Tennessee is ranked 13th!

105
by Jonadan :: Wed, 11/10/2010 - 1:37pm

Taking a wild guess on this particular comparison, I would say it's because the Patriots' defense is bad, and the Titans' is good, or at least decent. If the Patriots have less to work with, and are still putting up yards/downs/points, then they're going to get the respect offensively, at least from DVOA.

108
by Led :: Wed, 11/10/2010 - 1:47pm

Pats have had very few interceptions (tied for 2nd least in the league) and fumbles (least in the league) with slightly bad fumble luck. They've otherwise been efficient, although not overwhelmingly so since they traded Moss, but the relative lack of fumbles and picks is big. The fumble number is probably unsustainably low, however.

110
by Arkaein :: Wed, 11/10/2010 - 1:57pm

Following up on this and looking an NE vs. Tennessee directly, comparing QBs alone the Titans QBs have been responsible for about 5 more DVOA turnovers (counting all fumbles as half a turnover) than Brady.

Also, DVOA sees Chris Johnson as having a horrible year. He's barely above 4.0 YPC, which is quite bad for a boom-or-bust runner like him. He's also rated as one of the worst receiving RBs in the league this year. BJGE has nearly the same average, but with presumable far fewer long runs it means he's getting a lot more successful short runs.

Woodhead is averaging over 5.0 YPC, and is also a very efficient receiver by DVOA.

So overall, it's not hard to see why NE is ranked higher. This is really just a perfect example of DVOA preferring offenses that are consistently successful and can avoid negative plays over teams that mix spectacular gains with a lot of failures.

107
by AudacityOfHoops :: Wed, 11/10/2010 - 1:45pm

The Tennessee offense is clearly ranked too low because they get handed free points and great field position by their defense. Points scored is way better than this. The Pats (barely) scored 14 points and couldn't maintain possession against Cleveland!

Fixed that for ya.

EDIT: Oops, this was supposed to be in reply to #103.

114
by graywh :: Wed, 11/10/2010 - 2:32pm

Except Tennessee has a bend-but-don't-break mentality on defense, giving up yards, but not points.

119
by AudacityOfHoops :: Wed, 11/10/2010 - 2:58pm

I don't really care what their mentality is, just what happens on the field. Look at FO's drive stats. Tennessee's defense has the 3rd highest TO rate, and their offense has the 2nd best average starting position.

109
by PatsFan :: Wed, 11/10/2010 - 1:57pm

Gostkowski's been IR'd and replaced by Shayne Graham. How much does that zap them by?

127
by Richie :: Wed, 11/10/2010 - 4:01pm

Our premium picks against the spread, which had been hitting at over 60 percent so far this season, went just 4-8-1 this week.

That's interesting. I used DAVE to make my picks in my straight-up confidence pool last week, and won with a 10-3 record. DAVE only sent me wrong on Oak, Cle and Bal.

131
by ammek :: Wed, 11/10/2010 - 5:01pm

Overall, the theme of parity still applies to the 2010 season………

Yes, but it might not last. Teams with below average DVOA have the six hardest future schedules (and the #8). Could 2010's parity be, in part, down to the luck of the draw?

Also, that's not a misprint: Arizona's average opponent the rest of the way has -15% DVOA? Is slightly worse than Jacksonville? If the Cards manage to recover a few more of their own fumbles in the end zone, win the NFC West, and get the Bucs at home in the wildcard round, they might become one of the worst teams to win a playoff game, along with the 1998 Cardinals, the 2008 Cardinals …………