Week 12 DVOA Ratings
by Aaron Schatz
All is right with the world after New Orleans' nationally-televised crush-tacular over the New England Patriots. Our two undefeated teams now stand 1-2 on top of the Football Outsiders DVOA ratings, with the Saints on top, the Colts second, and the Patriots dropped into third place. According to the DVOA playoff odds simulator, we now have a 32.3 percent chance of a team (either New Orleans, or Indianapolis, or both) going 16-0 during the regular season.
That being said, can we hold off a little bit on the coronation ceremony for the Saints as one of the best teams in NFL history? Based on DVOA, this isn't even one of the best ten teams since 1994. Their amazing "perhaps the best ever" offense is tenth in DVOA history, behind such teams as the 1995 Cowboys, the 1998 Broncos, the 2004 Colts, and three straight years of Kansas City Chiefs, 2002-2004. The Saints definitely look like the best team of the season, but this team hasn't yet shown it is historically awesome on a weekly basis. Last night's game was spectacular, but it counts just as much as the game two weeks ago where the Saints barely beat the St. Louis Rams, 28-23.
If the Saints make it all the way through the season and finish 19-0, they will deserve all the accolades that are thrown at them (not to mention our sincere gratitude for shutting up Mercury Morris and preventing two weeks of Brett Favre Super Bowl hype). That team from Monday night is certainly good enough to make that run. The only problem? That team was missing for most of November. From Week 7 to Week 10, the Saints had four straight wins over Miami (5-6), Atlanta (6-5), Carolina (4-7), and St. Louis (1-10) by an average of just 8.8 points. If that team shows up in the NFC Championship game, we aren't going to get Saints dominance. We're going to get two weeks of non-stop Brett Favre Super Bowl hype.
While the two unbeaten teams are now the top two in DVOA, the rest of the ratings still differ from the regular NFL standings in some big ways. One team that stands out is Cincinnati, which is 15th in DVOA despite eight wins -- and two losses (Denver, Oakland) that would have been wins if not for fluke end-of-game plays. Back in Week 5, I wrote that Cincinnati was a bit of a mirage -- a surprise team, sure, but mediocre according to DVOA. I expected that either the Bengals would start losing some games, or their rating would rise to meet their record. Instead, they keep on winning most of their games yet they don't climb any higher in DVOA.
Many of the usual reasons why DVOA disagrees with conventional wisdom don't apply here. The Bengals' schedule hasn't been particularly easy; it ranks 20th in the league so far. They're average when it comes to fumble recovery rate and they haven't been particularly lucky when it comes to "hidden" aspects of special teams.
The basic problem with the Bengals is simply that all their games are close. Every Cincinnati game, win or lose, has had a score margin of 11 points or fewer, with one exception -- their 45-10 dismantling of Chicago in Week 7. If we look at the week-to-week DVOA graph, we've got three impressive wins, all in a row: Chicago, Baltimore, and Pittsburgh. However, we also have two games that seriously drag down the team's overall rating.
- First, the 23-20 win over Cleveland in Week 4. If we look purely at unadjusted offense and defense, the Bengals and Browns come out about equal, and the opponent adjustments from playing Cleveland are only responsible for about half of that -75% DVOA rating. Somehow the Bengals managed to win that game despite completely abysmal special teams, the worst by any one team in any one game this season. Shayne Graham had an extra point and a 23-yard field goal blocked, and the Bengals gave up 212 total return yards to Josh Cribbs. Cleveland started four different drives in Cincinnati territory -- but only got 13 points on those four drives, with two field goals, a touchdown, and a fumble. The other important issue with this game is that DVOA, as a play-by-play metric, gives every play from the entire season equal weight. The Bengals have an extra quarter worth of plays on one of their worst days of the season, which does even more to drag down their season-long rating.
- Second, the 20-17 last-second loss to Oakland from Week 11. Again, we have a close game that DVOA thinks was much more one-sided, only this time special teams aren't as much of an issue. The system basically thinks the Bengals were lucky to get away with a close game given the inconsistency of their running game against Oakland. Cincinnati averaged 4.1 yards per carry, while the Raiders averaged just 3.8 yards per carry -- but the Bengals had 10 runs tackled for a loss, while the Raiders got back to the line of scrimmage on every carry. Cincinnati's overall numbers look respectable primarily because of one 61-yard run by Bernard Scott -- which didn't even end up leading to a score because Carson Palmer took an 18-yard sack and then Shayne Graham honked a 37-yard field goal attempt.
This doesn't take away from what the Bengals have done, of course, but the stats agree with the general conventional wisdom that the Bengals just don't "feel" like a dominant team. They haven't been one. They've mostly been a slightly good team, with a couple of very good weeks, and it just so happens that adds up to 8-3. Based on the Universal Laws of What Makes a Super Bowl Team (1966-2004), this team simply isn't a major Super Bowl contender. Then again, after the last couple seasons, who the hell knows.
* * * * *
I said last week that I planned on answering questions from the comments thread in this week's commentary, but I have to admit -- there actually weren't many questions in there. I expected a lot more "why does team X have such and such a DVOA" type questions, but they just weren't there. Here are some quick answers to the questions I did find.
Q: There's a lot of talk about the Patriots 1st Half/2nd Half productivity. Does their DVOA change significantly between the halves? If so, does the DVOA indicate why? How about by quarter?
A: If you got a feeling that the New England Patriots defense was lousy in the second half, well, you're right. Here are the breakdowns by quarter, including Monday night. The real problem seems to be the third quarter.
| Off/Def | 1st | Rank | 2nd | Rank | 3rd | Rank | 4th/OT | Rank |
| Offense | 28.3% | 4 | 43.1% | 1 | 25.3% | 2 | 18.1% | 7 |
| Defense | -21.9% | 2 | -0.7% | 13 | 18.4% | 29 | 3.6% | 15 |
Q: How is variance calculated? Is it based on the total DVOA for each game played, or averaging the variance of the three individual units. In other words, would a team that has a decent game with poor defense and great offense following another decent overall game with great defense and poor offense have low or high variance? To me, it would make sense to count that as higher variance, since the units are independent.
A: Honestly, it's really simple, just the VARIANCE function in Excel for each single-game DVOA rating for that team. You make a good point, perhaps I should look into a better rating of consistency in the offseason. I'm also exploring a couple of different new schedule ratings, because "average opponent DVOA" doesn't account for the fact that two games against bad teams and one game against a great team is different than two games against good teams and one game against a horrible team, even if the average DVOA of opponents is the same.
Q: The top variance teams, except for the Eagles, are have Defensive DVOAs in the lower half. Does having a poor defensive team lend to more variance, historically?
A: Not really. There's no correlation between variance overall and defensive DVOA, and there's actually a slight negative correlation between defensive variance and defensive DVOA. (In other words, historically, bad defenses are more likely to be consistently bad than good defenses are to be consistently good.)
* * * * *
These are the Football Outsiders team efficiency ratings through 12 weeks of 2009, measured by our proprietary Defense-adjusted Value Over Average (DVOA) system that breaks down every single play and compares a team's performance to the league average based on situation in order to determine value over average. (Explained further here.)
OFFENSE and DEFENSE DVOA are adjusted to consider all fumbles, kept or lost, as equal value. SPECIAL TEAMS DVOA is adjusted for type of stadium (warm, cold, dome, Denver) and week of season. WEIGHTED DVOA is adjusted so that earlier games in the season become gradually less important. It better reflects how well the team is playing right now.
As always, positive numbers represent more points so DEFENSE is better when it is NEGATIVE.
To save people some time, please use the following format for all complaints: <team> is clearly ranked <too high/too low> because <reason unrelated to DVOA>. <subjective ranking system> is way better than this. <unrelated team-supporting or -denigrating comment, preferably with poor spelling and/or chat-acceptable spelling>
| TEAM | TOTAL DVOA |
LAST WEEK |
WEIGHTED DVOA |
RANK | W-L | OFFENSE DVOA |
OFF. RANK |
DEFENSE DVOA |
DEF. RANK |
S.T. DVOA |
S.T. RANK |
|
| 1 | NO | 38.2% | 3 | 36.8% | 1 | 11-0 | 31.1% | 1 | -9.2% | 5 | -2.1% | 25 |
| 2 | IND | 31.8% | 2 | 31.0% | 3 | 11-0 | 26.8% | 3 | -4.8% | 9 | 0.2% | 18 |
| 3 | NE | 31.3% | 1 | 32.5% | 2 | 7-4 | 29.9% | 2 | 0.0% | 15 | 1.5% | 14 |
| 4 | PHI | 30.6% | 4 | 28.6% | 5 | 7-4 | 14.0% | 12 | -11.0% | 3 | 5.7% | 4 |
| 5 | MIN | 29.7% | 5 | 30.2% | 4 | 10-1 | 19.3% | 6 | -2.6% | 10 | 7.8% | 1 |
| 6 | BAL | 27.2% | 6 | 25.4% | 7 | 6-5 | 17.0% | 9 | -9.5% | 4 | 0.7% | 16 |
| 7 | GB | 26.6% | 7 | 27.3% | 6 | 7-4 | 18.1% | 8 | -16.4% | 1 | -7.9% | 32 |
| 8 | DAL | 18.4% | 9 | 19.7% | 9 | 8-3 | 22.3% | 4 | 6.8% | 18 | 2.9% | 8 |
| 9 | ARI | 17.9% | 8 | 20.4% | 8 | 7-4 | 16.2% | 10 | 1.1% | 16 | 2.8% | 9 |
| 10 | PIT | 17.9% | 10 | 18.8% | 10 | 6-5 | 15.8% | 11 | -8.4% | 6 | -6.3% | 31 |
| 11 | HOU | 12.9% | 14 | 15.8% | 11 | 5-6 | 19.5% | 5 | 10.0% | 24 | 3.4% | 7 |
| 12 | DEN | 12.1% | 16 | 9.3% | 14 | 7-4 | 6.5% | 19 | -7.6% | 8 | -2.0% | 24 |
| 13 | SD | 11.6% | 13 | 14.9% | 12 | 8-3 | 18.7% | 7 | 7.1% | 19 | -0.1% | 20 |
| 14 | NYG | 9.8% | 11 | 7.5% | 15 | 6-5 | 10.7% | 15 | -1.5% | 13 | -2.4% | 27 |
| 15 | CIN | 8.9% | 15 | 10.7% | 13 | 8-3 | 11.7% | 14 | 2.0% | 17 | -0.9% | 22 |
| 16 | MIA | 4.9% | 12 | 5.2% | 16 | 5-6 | 10.2% | 16 | 9.2% | 23 | 3.8% | 5 |
| TEAM | TOTAL DVOA |
LAST WEEK |
WEIGHTED DVOA |
RANK | W-L | OFFENSE DVOA |
OFF. RANK |
DEFENSE DVOA |
DEF. RANK |
S.T. DVOA |
S.T. RANK |
|
| 17 | NYJ | 4.2% | 19 | 1.6% | 17 | 5-6 | -10.8% | 23 | -13.1% | 2 | 1.9% | 10 |
| 18 | ATL | -1.3% | 17 | -2.8% | 19 | 6-5 | 9.8% | 17 | 11.1% | 25 | 0.0% | 19 |
| 19 | SF | -1.6% | 20 | -2.5% | 18 | 5-6 | -10.0% | 21 | -7.6% | 7 | 0.8% | 15 |
| 20 | JAC | -3.1% | 18 | -6.7% | 20 | 6-5 | 12.3% | 13 | 14.4% | 28 | -1.0% | 23 |
| 21 | WAS | -10.3% | 22 | -9.2% | 23 | 3-8 | -10.9% | 24 | -1.3% | 14 | -0.7% | 21 |
| 22 | TEN | -10.5% | 23 | -8.6% | 22 | 5-6 | 6.9% | 18 | 14.3% | 27 | -3.1% | 28 |
| 23 | CAR | -10.9% | 21 | -6.9% | 21 | 4-7 | -8.6% | 20 | -1.7% | 12 | -3.9% | 29 |
| 24 | BUF | -15.1% | 26 | -14.9% | 24 | 4-7 | -18.8% | 28 | -2.2% | 11 | 1.5% | 13 |
| 25 | SEA | -16.9% | 25 | -17.9% | 25 | 4-7 | -10.4% | 22 | 8.1% | 20 | 1.6% | 12 |
| 26 | CHI | -20.9% | 24 | -23.3% | 26 | 4-7 | -18.5% | 27 | 8.4% | 21 | 6.0% | 3 |
| 27 | TB | -29.5% | 28 | -28.7% | 27 | 1-10 | -15.2% | 26 | 16.0% | 29 | 1.7% | 11 |
| 28 | KC | -30.0% | 27 | -30.1% | 28 | 3-8 | -21.2% | 29 | 12.5% | 26 | 3.6% | 6 |
| 29 | CLE | -37.8% | 31 | -39.4% | 31 | 1-10 | -25.9% | 31 | 19.1% | 30 | 7.2% | 2 |
| 30 | OAK | -37.9% | 29 | -36.6% | 30 | 3-8 | -29.3% | 32 | 9.0% | 22 | 0.4% | 17 |
| 31 | STL | -37.9% | 30 | -36.5% | 29 | 1-10 | -14.0% | 25 | 21.6% | 31 | -2.2% | 26 |
| 32 | DET | -51.2% | 32 | -51.9% | 32 | 2-9 | -24.7% | 30 | 22.6% | 32 | -4.0% | 30 |
- ESTIMATED WINS uses a statistic known as "Forest Index" that emphasizes consistency as well as DVOA in the most important specific situations: red zone defense, first quarter offense, and performance in the second half when the score is close. It then projects a number of wins adjusted to a league-average schedule and a league-average rate of recovering fumbles. Teams that have had their bye week are projected as if they had played one game per week.
- PAST SCHEDULE lists average DVOA of opponents played this season, ranked from hardest schedule (#1, most positive) to easiest schedule (#32, most negative). It is not adjusted for which games are home or road.
- FUTURE SCHEDULE lists average DVOA of opponents still left to play this season, ranked from hardest schedule (#1, most positive) to easiest schedule (#32, most negative). It is not adjusted for which games are home or road.
- VARIANCE measures the statistical variance of the team's weekly DVOA performance. Teams are ranked from most consistent (#1, lowest variance) to least consistent (#32, highest variance).
| TEAM | TOTAL DVOA |
W-L | NON-ADJ TOT VOA |
ESTIM. WINS |
RANK | PAST SCHED |
RANK | FUTURE SCHED |
RANK | VAR. | RANK | |
| 1 | NO | 38.2% | 11-0 | 43.7% | 9.2 | 1 | -5.4% | 29 | -8.4% | 26 | 13.4% | 16 |
| 2 | IND | 31.8% | 11-0 | 34.9% | 8.7 | 2 | 3.1% | 14 | -3.1% | 21 | 10.1% | 11 |
| 3 | NE | 31.3% | 7-4 | 26.8% | 7.5 | 6 | 5.5% | 4 | -2.8% | 20 | 22.9% | 30 |
| 4 | PHI | 30.6% | 7-4 | 36.7% | 7.5 | 7 | -6.0% | 30 | 9.4% | 7 | 19.9% | 27 |
| 5 | MIN | 29.7% | 10-1 | 37.2% | 8.4 | 3 | -9.9% | 31 | 1.2% | 14 | 5.0% | 1 |
| 6 | BAL | 27.2% | 6-5 | 26.9% | 7.9 | 4 | 3.9% | 9 | -16.4% | 32 | 8.8% | 10 |
| 7 | GB | 26.6% | 7-4 | 37.7% | 7.3 | 8 | -11.9% | 32 | 6.3% | 11 | 11.3% | 14 |
| 8 | DAL | 18.4% | 8-3 | 23.5% | 7.3 | 9 | -4.8% | 26 | 20.0% | 1 | 7.1% | 4 |
| 9 | ARI | 17.9% | 7-4 | 22.8% | 7.7 | 5 | -5.3% | 28 | -8.6% | 27 | 15.7% | 21 |
| 10 | PIT | 17.9% | 6-5 | 23.7% | 7.3 | 10 | -4.3% | 24 | -4.3% | 22 | 10.3% | 12 |
| 11 | HOU | 12.9% | 5-6 | 11.6% | 6.8 | 11 | 1.3% | 19 | -5.4% | 24 | 8.7% | 9 |
| 12 | DEN | 12.1% | 7-4 | 10.9% | 6.6 | 13 | 4.2% | 7 | -8.9% | 28 | 19.7% | 26 |
| 13 | SD | 11.6% | 8-3 | 14.6% | 6.6 | 12 | -1.8% | 22 | -7.8% | 25 | 8.1% | 7 |
| 14 | NYG | 9.8% | 6-5 | 6.8% | 5.7 | 15 | 1.7% | 17 | 14.4% | 3 | 18.5% | 25 |
| 15 | CIN | 8.9% | 8-3 | 14.1% | 6.5 | 14 | 0.6% | 20 | -8.9% | 29 | 24.2% | 31 |
| 16 | MIA | 4.9% | 5-6 | 2.3% | 5.7 | 16 | 4.1% | 8 | 12.1% | 4 | 13.6% | 18 |
| TEAM | TOTAL DVOA |
W-L | NON-ADJ TOT VOA |
ESTIM. WINS |
RANK | PAST SCHED |
RANK | FUTURE SCHED |
RANK | VAR. | RANK | |
| 17 | NYJ | 4.2% | 5-6 | 7.0% | 5.6 | 17 | 3.8% | 10 | -1.3% | 17 | 15.6% | 20 |
| 18 | ATL | -1.3% | 6-5 | -0.6% | 5.5 | 18 | 1.6% | 18 | 7.1% | 10 | 11.0% | 13 |
| 19 | SF | -1.6% | 5-6 | -1.8% | 5.2 | 20 | 2.4% | 15 | -14.4% | 31 | 8.4% | 8 |
| 20 | JAC | -3.1% | 6-5 | -2.5% | 5.5 | 19 | -4.6% | 25 | 10.8% | 5 | 20.5% | 28 |
| 21 | WAS | -10.3% | 3-8 | -4.5% | 4.8 | 22 | -4.9% | 27 | 10.1% | 6 | 7.6% | 6 |
| 22 | TEN | -10.5% | 5-6 | -21.2% | 4.8 | 21 | 8.8% | 2 | -1.6% | 19 | 38.5% | 32 |
| 23 | CAR | -10.9% | 4-7 | -17.7% | 4.4 | 24 | 4.7% | 6 | 19.9% | 2 | 18.4% | 24 |
| 24 | BUF | -15.1% | 4-7 | -11.6% | 4.4 | 23 | 0.4% | 21 | 9.0% | 8 | 17.3% | 22 |
| 25 | SEA | -16.9% | 4-7 | -15.8% | 3.6 | 26 | -3.1% | 23 | -0.5% | 15 | 17.5% | 23 |
| 26 | CHI | -20.9% | 4-7 | -21.2% | 3.9 | 25 | 1.9% | 16 | -1.4% | 18 | 11.9% | 15 |
| 27 | TB | -29.5% | 1-10 | -32.3% | 3.0 | 27 | 10.2% | 1 | 3.3% | 12 | 13.5% | 17 |
| 28 | KC | -30.0% | 3-8 | -28.1% | 2.9 | 28 | 3.2% | 13 | -5.0% | 23 | 7.2% | 5 |
| 29 | CLE | -37.8% | 1-10 | -46.4% | 2.4 | 31 | 5.9% | 3 | -10.4% | 30 | 14.7% | 19 |
| 30 | OAK | -37.9% | 3-8 | -42.9% | 2.7 | 29 | 5.0% | 5 | 2.3% | 13 | 20.8% | 29 |
| 31 | STL | -37.9% | 1-10 | -39.7% | 2.5 | 30 | 3.7% | 12 | -0.5% | 16 | 5.7% | 2 |
| 32 | DET | -51.2% | 2-9 | -53.2% | 1.8 | 32 | 3.7% | 11 | 7.9% | 9 | 6.2% | 3 |
Comments
294 comments, Last at 07 Dec 2009, 6:52pm
#1 by zip.4chan.org/sp/ (not verified) // Dec 01, 2009 - 7:32pm
Aaron, the week-on-week breakdowns of performance are far more enlightening than a single unified DVOA number. Think about including a sparkline of a team's per-game-DVOA in these charts, or making full charts a regular feature.
#20 by Ben // Dec 01, 2009 - 8:09pm
I'll second that request. Sparklines would be a very interesting addition to the charts.
#23 by peterplaysbass… (not verified) // Dec 01, 2009 - 8:17pm
I'll third that. I get excited when I see those graphs, then a little deflated when there is no graph for any of the teams I follow closely.
#97 by EaglesFan (not verified) // Dec 01, 2009 - 11:56pm
Fourthed!
I'd really love to know the game by game DVOA and not need to wait until it shows up in a graph for my Eagles.
#2 by Raiderjoe // Dec 01, 2009 - 7:34pm
Computer is like blind man. No way Raiders worse than Chiefs, browns and Buccaneers. can make strong case Raiders bvetter than all 4-7 temas and redkins too . When Raiders beat Steellers, what's computer going to do then? Going to have to respect Raiders and mnove them up. So check back here same time next week
#70 by J.D. (not verified) // Dec 01, 2009 - 10:31pm
Raiders logo is like half-blind man.
#72 by Anonymouse (not verified) // Dec 01, 2009 - 10:33pm
*remembers Thanksgiving*
Nah, I think DVOA's got it right on Oakland.
#126 by Danish Denver-Fan // Dec 02, 2009 - 3:27am
I would guess, that the system is constructed in a way, so that the computer will do an emergency reboot.
#149 by Kevin from Philly // Dec 02, 2009 - 9:43am
I have a feeling that, if the FO computers spell check can handle Raiderjoe, its Excel can handle Oakland beating Pittsburgh. On the other hand, if it starts singing "Daisy" or building robots that look like Governor Arnold, I recommend that the FO staff abandon the building.
#294 by Spielman // Dec 07, 2009 - 6:52pm
Looks like Raiderjoe gets the last laugh this week.
#3 by Keith (1) (not verified) // Dec 01, 2009 - 7:36pm
Go awesome Packers defense (minus Harris/Kampman) against Detroit. Still number 1!
#5 by Keith (1) (not verified) // Dec 01, 2009 - 7:38pm
In all seriousness -- I keep expecting them to fall and they never do. Maybe I just need to ride the wave finally.
#52 by BadgerDave // Dec 01, 2009 - 9:34pm
Didn't realize they would still be #1 in DVOA, saw they were #1 in NFL (total yards Defense) I wonder how often the NFL yardage rankings actually correspond with DVOA?
#242 by Nick Wells (not verified) // Dec 03, 2009 - 12:06am
Damn. I so wanted to be the 1st one to comment this week about how overrated the Packers defense is, but I was about 150 posts too late. Oh well, maybe next week. Until then, I'll just go with this: Aaron Rodgers is, like, really overrated too. If he played for the Vikings, he couldn't even beat out Tavaris Jackson or Sage Rosenfels for the backup job behind St. Bret.
#243 by justanothersteve // Dec 03, 2009 - 1:13am
No troll like a late troll.
#245 by Nick Wells (not verified) // Dec 03, 2009 - 1:54am
Just to add to my previous comment directed toward Keith (1), which was entirely sarcastic, in case that wasn't obvious: Last Thursday, the Packer D forced 5 turnovers, had a pick-six, limited Stafford to less than 5 yds./pass (despite a late 47 yd. completion against new nickleback/disaster-in-waiting Jarrett Bush), and didn't allow a TD after Detroit's opening drive, which started at the Packer 20-yd. line. Please explain to me why you expected this performance to result in a drop in the rankings? Look, I'm not likening the Packers' D to the '85 Bears, but there really isn't a dominant defense in the league this year, and Green Bay's has played as well as anyone's, especially recently. I believe this is the third week in a row that the Packers have had the #1 ranked defense, and most of the dissenting opinions have been along the lines of, "I just really don't think they're that good." Correct me if I'm wrong, but isn't it pretty much the entire purpose of DVOA to create a sophisticated OBJECTIVE method of evaluating teams and players, as opposed to the good ole eye test. (Sigh) I should really just stop reading the comments.
#4 by Hari-Kiri Beng… (not verified) // Dec 01, 2009 - 7:38pm
Cincinnati is clearly ranked too low because DVOA has magically gained sentience and witnessed our more-conservative-than-Dick-Cheney gameplan. Basing stats on Chad Ochocinco's Twitter stream is way better than this. We shall rename ourselves the Cincinnati Ceiling Cats and wield the godlike power that comes with that title!
#24 by jebmak // Dec 01, 2009 - 8:18pm
Basement Cats seems like a lot more likely sports team name.
#71 by J.D. (not verified) // Dec 01, 2009 - 10:31pm
Aren't the Basement Cats a NBDL team?
#150 by Kevin from Philly // Dec 02, 2009 - 9:44am
Basement Cats are GO!!
#8 by t.d. // Dec 01, 2009 - 7:43pm
I would love to get a pick 'em line to take the Vikings over Philly. Baltimore hasn't been a good team for a while. And once again, Dallas faces a murderers row of a December schedule. I'm really surprised that Denver is ahead of San Diego, but they were very impressive on Thanksgiving.
#36 by Xeynon (not verified) // Dec 01, 2009 - 8:58pm
I would love to get a pick 'em line to take the Vikings over Philly. Baltimore hasn't been a good team for a while. And once again, Dallas faces a murderers row of a December schedule. I'm really surprised that Denver is ahead of San Diego, but they were very impressive on Thanksgiving.
As an Eagles fan I wouldn't be afraid to take that bet. Barring injury I don't think this year's Eagles team has much chance to beat the Saints in New Orleans. The Vikings, on the other hand, I am reasonably confident they could beat, even in Minnesota. It certainly won't be a pick 'em but I would take the Eagles to cover, say, a 7 point spread, and think there's a not-insubstantial possibility they win the game outright.
#57 by crack (not verified) // Dec 01, 2009 - 9:55pm
True, I mean it's not like the Vikings are Oakland or anything.
#6 by Raiderjoe // Dec 01, 2009 - 7:42pm
Saints offense great no matter what DVOA and SAVE said this year. Going to take colosonal effrt for any etam to beat Saints. Vikins and Cardinals got shot. Cowobys no. Eagles no. Falcons maybe. Packers could. But that it in NFC. In Afc Colts would give Saimnts good Superbwl compeition. Chargers no. Steelers not making pklayoff so forget them. Patrios no. Ravens could (good pjhsyical team might beat up Saints skill playters rough up). To rank them as best chance to beat Saints-
1. Vikings
2 Colts
3 Cardinals
4 Ravens
5 some other teams (but if bettibng man don't bother to bet on anyoen except four above team
#28 by Whatev (not verified) // Dec 01, 2009 - 8:37pm
And it really depends on which Cardinals show up that day.
#147 by dbostedo // Dec 02, 2009 - 9:34am
And, as Aaron pointed out above, which Saints team shows up.
#153 by Phil O'sopher (not verified) // Dec 02, 2009 - 10:03am
I love how you mix in quality posts on general NFL knowledge with the horse blinders you wear for your beloved Raiders.
Raiderjoe posts=highlight of my day
#201 by M // Dec 02, 2009 - 1:38pm
I'm very surprised to not see the Raiders at #3 on the list; after all, they've beaten Philadelphia and Cincinnati!
#7 by DEguy (not verified) // Dec 01, 2009 - 7:42pm
The Giants are 6-5, not 7-4.
#9 by t.d. // Dec 01, 2009 - 7:44pm
Jets are 5-6, not 4-7
#25 by Aaron Schatz // Dec 01, 2009 - 8:19pm
Grrr. Hate when I mess up W-L records. I'll go fix.
#10 by t.d. // Dec 01, 2009 - 7:46pm
Detroit is actually getting worse as the season goes on. Poor Jim Schwartz.
#11 by tunesmith (not verified) // Dec 01, 2009 - 7:48pm
Denver's out of the lowest quartile for special teams!!! Awright!!! Been a long time since that has happened, I think.
#12 by t.d. // Dec 01, 2009 - 7:48pm
I'm surprised Tennessee's weighted DVOA is still below 0%. Is it still being anchored by that New England game?
#31 by Tom Gower // Dec 01, 2009 - 8:42pm
Yes, and the first JAC game as well. The weights of both games will decrease over time.
#13 by Raiderjoe // Dec 01, 2009 - 7:51pm
how are steelers ranked 31 in specisl teams and GB packers rank 32?
#16 by theshadowj // Dec 01, 2009 - 7:54pm
The man has a good point. What happened to the historically bad special teams of Pittsburgh? Is that a mistake or did they improve that much in one game?
#32 by Steelers fan - ouch (not verified) // Dec 01, 2009 - 8:43pm
I believe the recent article you are referring to was noting the Steelers' record-setting worst kickoff cover rating, not full special teams rating. At the time, the Steelers were only approaching the "ingoble" record, but then allowed another KO return TD on the first play of the KC game and have now surpassed (sunk below?) the 2000 Bills...
#56 by justanothersteve // Dec 01, 2009 - 9:54pm
Yup. The Packers realize they can't outdo the Steelers kickoff rating. So they are managing by sucking at nearly all other aspects of special teams. Punting - last net by over a yard (Pats are next which may help explain 4th and 2). Crosby has a lower FG% than Reed and has missed an extra point. Preventing returns aren't great either; they're in the bottom third for kick and punt return average. They even managed to fumble one of the few KO returns last week to give Detroit a glimmer of hope early in the game.
#185 by Cliff Claven (not verified) // Dec 02, 2009 - 12:22pm
Just to be clear, the 2000 Bills are from the season AFTER the Music City Miracle, right?
Cliff
#202 by M // Dec 02, 2009 - 1:41pm
They had a good special teams coach who they made the fall guy for the Music City Miracle. But there had to be more things at work to create that level of inepitude in 2000.
#55 by RickD // Dec 01, 2009 - 9:51pm
Well, they didn't give up a kickoff return for a TD. That alone constitutes a massive improvement.
#14 by Raiderjoe // Dec 01, 2009 - 7:52pm
is Steelers specoal team dvoa adjusted beceaUSE OF jEFF rEED?
#15 by t.d. // Dec 01, 2009 - 7:52pm
I'm OK with Favre-hype if it's justified by incredible performance. He's been unbelievably good, and he can flip-flop on retiring all he wants next offseason for all I care, I'll just change the channel.
#120 by andrew // Dec 02, 2009 - 2:13am
Yeah not understanding the favre-hype hate... if he makes it to the superbowl, he'll be deserving.
I can totally understand the frustration with commentators, and even more so with networks who show his game over more deserving other games on national tv, but I'd have thought sunday ticket would be a near must in this business anyway... a business expense. (I shelled out for it & superfan while charting, and that's how I justified it anyway... dropped it this year but if money were better would definitely have it back...)
#17 by t.d. // Dec 01, 2009 - 7:54pm
Indy is really boosted by their spectacularly mediocre special teams. Too bad they waited until the offensive line started to decline to get that straightened out.
#19 by Ben // Dec 01, 2009 - 8:08pm
Most Colts fans are delighted with average special teams play. That's a big step up from where it's been for years. Especially the improvements in coverage. I'll certainly take it, despite the total lack of a return game.
#33 by Purds // Dec 01, 2009 - 8:44pm
Agreed. MacAffee's booming kickoffs have been a real boon. But, man, they might as well just take a knee on every kick off. They never move the ball out beyond the 25, and there's always a chance for a fumble. Whenever I watch them receive a kick off, I just pray they don't turn the ball over. It's like watching your favorite baseball team play defense -- nothing good can happen. Lots of bad can happen.
#65 by Bobman // Dec 01, 2009 - 10:13pm
I think that is exactly the point. I would not be surprised if the organization philosophy is "Our O and D can do what they need to, so all we need to do is give Peyton the ball and don't f-up." Fumbles are bad. Blocks in the back on desperation punt returns are bad. Just secure the ball, and flip it to the refs.
Kind of like our running game--does what's needed. Not flashy, won't kill the opponents but more importantly, won't kill the Colts, either.
#18 by Jeff Fogle // Dec 01, 2009 - 7:57pm
Meant to ask this last week. Wish I did since you were looking for more questions. What if you used the "median" of the weekly performances instead of the total? Eyeballing the chart above, it looks like Cincy's median performance is a shade under 25% in their game with Green Bay. I may be reading it wrong because I'm new to studying the details of how you come up with the ratings. Would Cincy show better in "a collection of medians?"
If there's a possibility that teams handle blowout games differently, or prioritize certain games differently (I've always felt some teams place less emphasis on non-conference games because those count least in the tie-breakers...a discussion for another time), something that throws out the highs and lows entirely might shuttle things around in a way that's more meaningful for a "typical" game.
Worth noting too I think that Cincy's two "off the charts" horrible games came the week after defeating Pittsburgh in close hardfought battles. They aren't normally in letdown spots...but those games sure drag down their total rating.
Using medians may not end up moving the teams around much. Just thought I'd ask. I know a lot of numbers guys prefer medians to averages or totals. Wondered what you thought about it, or if you've already incorporated that theory to some degree.
#22 by Jeff Fogle // Dec 01, 2009 - 8:16pm
Quick addendum...
Cincy seems to be very similar to the stat profile of Pittsburgh/Baltimore last year in terms of total offense, total defense, third down conversions on both sides of the ball, and yards-per-play on both sides of the ball. Pittsburgh had the better defense last year than either Baltimore or this year's Cincy. Palmer's at 86.7 in passer rating though, while Ben and Flacco were around 80 last year.
Wouldn't call Cincy a juggernaut by any means. They seem pretty close to the conservative offense/strong defense profile that has done well in postseasons and sprung an occasional Super Bowl upset. Weird to think of them that way. But, they're 6th in total defense and allowing just 34% on third downs. They fit that profile, and that profile does well in January weather...
#26 by SlantNGo (not verified) // Dec 01, 2009 - 8:21pm
Medians are pretty bad for small sample sizes. Can you imagine how far off a median for 4 data points would be?
#30 by Jeff Fogle // Dec 01, 2009 - 8:39pm
Extremes are bad for small sample sizes too though. Is it better to include them or not? Small sample sizes are a bit of a pain no matter how you go about it obviously. Personally prefer the Olympic diving model where you throw out the Cold War judges and use the people in the middle (lol).
#74 by DaveRichters // Dec 01, 2009 - 10:38pm
How far off from what? The mean? This is true if there is one outlier in the bunch, but that is precisely the reason to use the median instead.
#94 by SlantNGo (not verified) // Dec 01, 2009 - 11:37pm
Well, I'm assuming that the distribution of a team's DVOA is normal (or at least unimodal symmetric) so that the true mean and median are the same. In this case, I expect the sample mean to give a better estimate of the true mean/median than the sample median. I ran a quick simulation to verify this.
The median is useful when the distribution is skewed--for example with grades in school. A class average is probably something like 75%, so a really low grade can pull down the mean significantly, and a median can counteract the skew. I have no reason to believe that DVOA is a skewed distribution though, so I think the mean is better.
#99 by DaveRichters // Dec 02, 2009 - 12:01am
I'm trying to understand your mostly nonsensical argument, so bear with me if I paraphrase you incorrectly.
You say that with skewed distributions, the median would be better. With normal distributions they would be the same. But you conclude that the mean is better? Please explain, and I would prefer you use "layman's terms" or define what you mean when you say "the true mean/median".
#131 by SlantNGo (not verified) // Dec 02, 2009 - 7:34am
Sorry, I didn't make a clear enough distinction between the true and sample means and medians. DVOA right now is calculated based on a sample mean--that is, taking the mean of all of the observed data. What has been suggested in this thread is to use a sample median--to take the median of the observed data.
The goal is really to estimate the TRUE mean or median, both of which are unobservable: the true mean you can think of as the point the sample mean (standard average) converges to as the number of games approaches infinity. The true median is the point at which the probability of an individual game's DVOA is equally likely to be above or below. For a normal distribution or any distribution that is symmetric and unimodal, the true mean and median are the same.
If you buy the symmetric and unimodal assumption, then what we're really trying to do is to estimate the true mean/median, and the sample mean does a better job than the sample median. Whether the symmetric and unimodal assumption is really true though (as one poster noted below) is the tough question. I'm not sure I buy it so much myself. Is a team equally likely to play above and below their "true ability"?
#259 by DaveRichters // Dec 03, 2009 - 11:46am
I don't think the the sample mean does a better job than the sample median, but it is trivial to prove that they must be within a standard deviation of each other. Really the only debate is how much to count the outliers. I think that the difference between a large and very large DVOA for a game mostly reflects the teams approach and not ability.
#263 by Eddo // Dec 03, 2009 - 12:53pm
The problem with the median is this:
Let's say you have a team that has the following distribution of success points over 10 plays, and it is perfectly repeated over and over for the season:
5, 2, 1, 0, 0, 0, 0, -5, -10, -25
The median would be 0, showing them to be a perfectly average team.
The mean, however, takes into account that their failures are much worse than their successes, and would be -3.2. That seems more correct for a team with this profile; they succeed as much as they fail, but when they fail, boy howdy!
This is a fabricated example, of course, but in real life, most NFL teams get 0 success points on a given play, which can skew the median in this manner.
#265 by Jeff Fogle // Dec 03, 2009 - 1:28pm
Note that if Vegas posted a line of "0" for the next game every time out, there wouldn't be an edge. But, if Vegas used the average of -3.2, you'd get rich betting on the team to be better than that because you'd be going 6-4 perpetually over every 10 games.
That's why a lot of gamblers use medians!
That's really at the heart of the old time football gambling strategies that focus on underdogs. Lines are influenced by the outliers (big wins or big losses influence perceptions), but the "typical" result is more in the middle of the potential results.
With team ratings, there's obviously an attempt to account for all the positives and negatives. The danger is that the system ends up measuring choices rather than quality...as has been mentioned by others. If "superior" teams are going to be up 27-10 in the third quarter...but a few of those decide to keep running up the score while others decided to run out the clock...what's being measured is the choice rather than the inherent quality of the teams.
If there's a spot on the schedule where it just makes more sense to live with a loss and save yourself for the next game (Southern team playing in a snowstorm, team playing way out of time zone, team in a huge letdown spot after a big win over a divisional rival), then the choice to accept the loss might create too big a negative outlier.
Can see all sides of the issue. I think the heart of many of the disagreements in these threads end up revolving around teams who win big in their blowouts getting rated too highly. The "stomps and guts" theme that Super Bowl champions win a lot of stomps has led to counting all stomps as meaningful. Bullies are ranked like they're Super Bowl contenders. When the "superior" teams are matched against each other, a different picture gets painted than the one suggested by the polluted math. Philadelphia goes 1-3 against the top 15 this year, 3-5 against the top 15 last year, while ranking 4th right now and 1st last year. That's 4-8 their last 12 games against top 15 opposition
Philly is 12-2 this year and last in games decided by double digits though, which looks like a Super Bowl champion.
Which is the best way to consider them? You answer there determines if you want the outliers in the mix or not. Personally, I don't think the fact that Super Bowl champs win a lot of stomps should lead to all stomps getting respected. That 4-8 record vs. top 15 opposition carries more weight with me. Doesn't mean I'm right, lol.
#266 by Eddo // Dec 03, 2009 - 1:42pm
Well, I agree with you that if the mean DVOA was calculated by averaging the single-game DVOAs, then the median would be better.
It's not, though; a team's DVOA is the average of all plays. Chances are, if you took the median value of every team's plays, you'd wind up with every team right around 0. That's my point.
#269 by Jeff Fogle // Dec 03, 2009 - 2:35pm
Okay...so I guess I'm lobbying for a second rundown that's based on using the median of single game DVOA's...then seeing if that methodology causes less debate. It might, it might not. (Why is the stomps/guts article referred to so often when the methodology is about plays?).
Then, I should say that the extra high quality PLAYS that bullies get when "not calling off the dogs" could warp the full season DVOA numbers that are a sum of the seasons plays. The DVOA isn't potentially warped in a "full-game" basis because of the differences in choice. It's potentially warped because those teams have so many more successful plays in their games vs. weak opposition than others who call off the dogs.
Eddo, is that what I'm trying to say? (lol). I'll speak the language eventually.
Cluster the plays into game-units because teams are playing games each week with a start and end point rather than just stringing together one long season of plays. Find a way to throw out the outliers. Find a way to best characterize the team's quality numerically once the outliers are thrown out. Don't let the fact that Super Bowl teams win a lot of stomps influence the overall attempt...a lot of stompers don't win Super Bowls.
Can you say what I'm trying to say better than I'm trying to say it?
#271 by Eddo // Dec 03, 2009 - 3:10pm
Well, the Guts and Stomps article is being brought up by commenters, though no one has really said that DVOA should use the premise in that (rather weak, in my opinion) article. You keep bringing it up to point out that having Stomps doesn't mean you will win the Super Bowl, but that's a strawman, as I haven't heard anyone argue that having Stomps leads to playoff success.
A median single-game DVOA would be an interesting addition to the table, I agree with you there.
Using the season as one long string of plays (which is a nice description, by the way), is what makes DVOA so powerful, as that is what brings the sample size up to a high level. You can then group those plays by game, down, quarter, etc.
I'm also not sure that, for example, New England's plays once the Titan game got out of hand are skewing things all that much. How many plays did they run once the game was essentially over? 20? And it's not like every single one of those was a crazy success; there were maybe 5 plays that fit that bill.
#272 by Thomas_beardown // Dec 03, 2009 - 3:11pm
I might argue they ran all their plays after the game was over.
#273 by nat // Dec 03, 2009 - 3:25pm
Stomps/Guts demonstrated that Stomps are useful in evaluating a team's quality. It follows that the plays in a Stomp game are also useful in evaluating a team's quality. That's why people refer to the Stomps/Guts article.
You've missed a very obvious point, regarding those "extra high quality plays". They don't exist.
Consider the Colts-Titans game and the Pats-Titans game. In the Colts game, Manning played the entire game, and ran a pass-heavy attack to the midpoint of the 4th quarter, and continued to pass in his final drive. In essence, it was a full game of normal offense for the Colts.
In the Pats game, Brady left after 5 minutes of the second half. After that, the Pats were running their scrubs offense. I'm pretty sure that DVOA "garbage time" doesn't kick in until much later, so the Pats DVOA for that game includes at least one Hoyer drive, probably several.
If anything, the DVOA for that game under-reported the quality the Patriots demonstrated that day. I think you are trying to solve a problem that doesn't exist.
#274 by Jeff Fogle // Dec 03, 2009 - 3:50pm
Guts/stomps looked to be referencing Super Bowl champions from a looking back view, rather than seeing what happens to all stompers. It didn't discuss the clear tendency for some teams to run up the score vs. patsies, but then not play very well within their own perceived class. This I think may be at the heart of the Philadelphia problem that people have referred to for awhile. It's not my perception that the Philadelphia problem is one that doesn't exist, unless you're convinced they were the best team in the NFL last year and are currently the 4th best...even though they have a 4-8 record against teams in the top 15 in DVOA during that period.
Someone will have to show the DVOA from New England/Tennessee and make the case that it under-reported their quality that day...and then suggest that the overwhelmingly clear possibility that Tennessee no-showed in terms of intensity shouldn't be considered when evaluating NE in the big picture. Is NE good enough on command to jump to 45-0 halftime leads with yardage edges of about 440-85 (eyeballing the drive charts in ESPN's boxscore)?
NE ranked 1st in DVOA last week even though they hadn't won a road game all year, then proceeded to get squashed in a road game vs. a quality team. The view among some critics last week that 7-3 NE should be #1 was born out. The view among many that Philly ranks too high in this methodology looks to be born out on a regular basis, though I wouldn't suggest they were horrible or anything. Just not as good (particularly vs. quality opposition) as DVOA is suggesting.
There's not much debate about whether or not an apple will fall to the ground when you let go of it. There's not much debate about when high tide will be tomorrow at Jacksonville Beach. We're not at the point yet where DVOA's late season ratings are so clearly accurate at face value that debate disappears.
#275 by Jeff Fogle // Dec 03, 2009 - 4:15pm
The first half yards-per-play line is pretty amazing in that Tenn/NE game. Median gain is "7" out of 40 plays run, with 14 of the plays going for double digits. Did it by hand so I might have missed a play here or there.
minus 6-0-0-0-0-1-1-3-3-4-4-5-5-5-5-6-6-6-7-7-7-7-
8-8-9-9-10-12-12-12-12-12-20-26-28-30-38-40-45-48
As to the point eddo was making...I don't think it's a strawman. The article was about how champions win blowouts. There was a reference to "Super Bowl profile" regarding Cincinnati in the article above. I've seen that as the defense for leaving in the outlier games..."stomps and guts shows that blowouts mean high quality" or variations thereof. It looks like a distance like FO is using "Super Bowl profile" to guide their thinking on leaving outliers in...but the danger from that is that the bullies are treated as Super Bowl contenders.
Now, NE is obviously a contender (or a mix of contender/bully if that makes sense). The question to me is...are the overall rankings shaded in a way that rewards stompers too much? The stomper that eventually wins the Super Bowl showed well in the weekly ratings...but other stompers who were just bullies may have been a few to several spots too high (Sagarin has Philly at 13th for example, not that I'm going to start defending Sagarin in everything, Las Vegas Sports Consultants has Cincy 4 spots ahead of Green Bay)...and one way to uncover who the bullies are is seeing who has unimpressive records when the top half (or whatever) are playing each other.
#276 by nat // Dec 03, 2009 - 4:27pm
Can we safely assume that you have completely abandoned the "bullies get extra plays" theory? That you've reverted to the "blowouts happen because bad teams give up in the second quarter" theory?
Man up.
#278 by Jeff Fogle // Dec 03, 2009 - 4:46pm
I've written, what, a couple thousand words in this thread..and that's your take?
What I'm trying to do is determine why a 7-4 team that struggles vs. quality competition ranks 4th in the league, while an 8-3 team that's 5-0 vs. the top 10 ranks 15th. The evidence suggests that blowout frequency vs. "bottom half" teams is a or the significant contributing factor. New England gets swept into that because they were part of the same type of discussion last week, before getting routed as the best team in the rankings at the time.
Haven't abandoned anything I don't think, lol...
#279 by Rick A. (not verified) // Dec 03, 2009 - 5:02pm
Exactly Jeff. You've asked and pointed out the most relevant questions regarding
the non-realism of DVOA. Why everyone else can't notice the obvious is also the question.
#283 by nat // Dec 03, 2009 - 11:39pm
Well, then. Man down, I guess.
#286 by Jeff Fogle // Dec 04, 2009 - 2:10pm
“I have a question about your chart here. You have Tom Cruise listed at 6’4” in height. I think he’s much shorter than that.”
“Tom Cruise…the action movie star from the “Mission Impossible” series, “War of the Worlds,” and “The Last Samuari?”
“Yes, obviously. He’s not 6’4””
“Let me refer you to an article we wrote about action stars. That should answer your questions. It’s called “Studs and Gumps.” You’ll find it here in the archives.”
“I read the article. It basically says that, over the last 40 years, action movie stars looked like they were 6’4” up on the big screen, so you’re going to list all action movie stars at 6’4””
“That’s an oversimplification.”
“Okay, it is. But…that really does sum it up pretty well. And, you DO have Cruise listed at 6’4” when he’s much shorter than that.”
“We have a very complicated, innovative, evaluation system. I suggest you read through the methodology.”
“I did that. But, whatever methodology you use, Cruise isn’t 6’4.” I also wanted to ask about Brad Garrett.”
“Who?”
“Brad Garrett, the guy from Ray Romano’s show.”
“You’re kidding.”
“You have him listed at 6’2.” The guy’s huge. He’s like 6’8” or something.”
“Based on the Universal Laws of What Makes an Action Star (1966-2004), Brad Garrett simply isn’t an action star.”
“Well, I can’t argue with that. But, he’s not 6’2.”
“Are you suggesting that Brad Garrett, a supporting character on “Everybody Loves Raymond” should be the lead in “Mission Impossible 4?” Is that what you’re thinking right now? Admit it. Man up.”
“Of course not. What does that even mean? Man up? I’m just saying that Tom Cruise isn’t 6’4” and Brad Garrett isn’t 6’2.” I’m trying to figure out why your listings show that.”
“Man down.”
“What? I still don’t know what that means. I never said Brad Garrett was an action star. Let me ask about another section at your site.”
“Yes.”
“You have Will Ferrell listed as the sixth best actor last year, when he's mostly made a bunch of flops in recent memory. This year has been a disaster.”
“Yes, we just wrote an article about that. Greg Cosell looks at a lot of film. He thinks it’s the supporting cast. Ferrell’s delivery is fixable. Cosell said “He doesn’t stink. He’s an unbelievable talent that needs to be harnessed.”
“Fine, but his box office stunk last year. It’s not just this year. Did you see “Semi Pro?” Ferrell was popular at one time. But, it’s like the “I didn’t do it kid” on “The Simpsons.” Everybody knew what to expect and it wasn’t funny any more.”
“You don’t understand our complicated, innovative, in-depth methodology at all if you’re comparing Will Ferrell to Bart Simpson.”
“Maybe I don’t. But, I do know Tom Cruise is reported to be about 5’7” rather than 6’4.” I do know Brad Garrett is about 6’8” rather than 6’2.” And, I know Will Ferrell wasn’t delivering at the box office last year or this year. I’m trying to figure out why YOU GUYS keep saying stuff like that.”
“Man overboard!”
#287 by Thomas_beardown // Dec 05, 2009 - 4:14am
I don't think stomps have anything to do with how DVOA is calculated. Don't know if that helps you.
#288 by Jeff Fogle // Dec 05, 2009 - 12:21pm
Appreciate the note tuluse. Looks from a distance like Stomps influences the decision not to make adjustments when the blowout teams are ranked high, and the grinders are ranked low. Blowout teams also have more "plays above average" or whatever the right way to phrase it is, that look to be rewarded by DVOA (Jay Cutler was ranked as 7th best QB DVOA last year even though he was a turnover disaster for an 8-8 team against a very weak schedule for example, how could he rank that high unless big plays are being valued very highly...and stompers have more big plays when running up the score on poor teams).
The thought that stomps equates with Super Bowl profile shows up in defenses of DVOA (and Cutler probably fits the profile of a Super Bowl QB very well if you're focused on his big plays). The author of this article mentioned that Cincinnati doesn't fit the profile of a Super Bowl champion (despite an 8-3 record and a 5-0 mark against DVOA Top 10), and Stomps is generally referred to in those types of discussions. There appears to be overlap in the thought process. And, ultimately, the fact that Tom Cruise is repeatedly listed at 6'4" (metaphorically) should be a red flag mystery that people are trying to solve rather than defend. If DVOA was referred to as a "Super Bowl profile," there might be less complaining about how things shake out every week.
So, stomps don't have anything to do with how DVOA is calculated. DVOA does reward stompers though. And, adjustments aren't made when rankings seem out of line with perceived reality because of a respect for stompers matching a Super Bowl profile. This is dangerous if there's a class of bullies who create the illusion of being championship material in their blowout games, but post below .500 records with poor stats vs. quality opposition. The Eagles squash weaklings (with a few exceptions). They don't have a good record vs. dvoa quality the past two seasons...yet they sit at #1 last year and #4 this year.
#290 by Sifter // Dec 05, 2009 - 2:58pm
I think you're overcomplicating it a bit. DVOA is calculated based with the future in mind, so it's really measuring the potential of the teams based on the traits they've shown thus far. That's why the Eagles are high, they SHOULD have a better record than they do and their record in close games and against good teams is far worse than it SHOULD be based on the potential they show. This year they've only really shown that potential in 3 games, but it's still been exhibited and that's why DVOA loves them. So DVOA says, 'the Eagles are likely to play well in the near future', not 'the Eagles have been successful this year'.
Plus DVOA is calculated purely with regular season wins in mind. It doesn't pick a Super Bowl champion, it picks teams who will win a lot of REGULAR SEASON games in the future. The guts and stomps article was focused on picking a Super Bowl winner, DVOA is not.
#291 by Jeff Fogle // Dec 05, 2009 - 3:33pm
Thanks sifter. Very helpful explanation. Maybe the paragraph that introduces the rankings each week should say something like "Here are the DVOA rankings for future expectation" or something similar. That way people won't wonder why New England at 7-3 ranks ahead of the 10-0 teams. Easy to grasp that NE could reasonably rank there if the emphasis is on the future. And, there would be less discussion I think about the "Philadelphia problem" from recent seasons if it was always clear that Philly is seen as having a bright future based on their production. Maybe not in that case, because there's skepticism about the head coach and McNabb in big games. At least the roar would be less loud.
And, to use my movie analogy from earlier...Tom Cruise is seen as having the future career of a 6'4" guy rather than a short guy, which makes sense given his track record...and Brad Garrett doesn't have the same future movie star potential. So, if the TITLE of the DVOA measurement emphasizes the future ("rankings" in all forms for as long as I can remember talk about "now" or "to this point," so it's easy to see why new visitors or casual readers would wonder about various teams), some of the confusion might be cleared up.
Then we could argue about whether or not the right indicators are being used for predicting the future...but everyone would be more on the same page.
That being said...I just went back and re-read the DVOA explanation here at the website, and it doesn't say anything like what you said. It doesn't say it's looking toward the future, or measuring potential. It's VERY clearly discussing how to rank teams NOW based on the quality they've shown in games that they've played in the past.
Example quotes:
"DVOA is a method of evaluating teams, units, or players."
"Since it compares each play only to plays with similar circumstances, it gives a more accurate picture of how much better a team really is compared to the league as a whole."
"The biggest advantage of DVOA is the ability to break teams and players down to find strengths and weaknesses in a variety of situations."
This is looking backward to compile a ranking, not "DVOA is calculated based with the future in mind." There's nothing in the website's explanation that talks about having the future in mind. It's about ranking teams based on their quality because DVOA is better at measuring quality.
If DVOA is really about the future, then I hope they'll re-write the website explanation to emphasize that...and they'll title the ranking emphasizing that element of future expectation. If it's about the past...New England #1 at 7-3 with no road wins and the stuff with Philly/Cincy suggests the plays from stomp games are being overly celebrated I think.
I greatly appreciate your clarification...but it's at odds with what the website is saying about its own stat.
#102 by Jeff Fogle // Dec 02, 2009 - 12:08am
Would be fun to compare the two lists (full season DVOA and the medians through 11 games). Might see things line up in a way that reflects some of the informed criticism (Philly drops...Cincy moves up). Might not. Seems that some of the discussions here talk about the impact of extreme games. One way to deal with that is to use medians...or throw out everybody's best two and worst two...whatever. How about a mean of the "middle 5" if you throw out best three and worst three?
Other options:
*Double count divisional games, half count non-conference games (reflecting priorities of teams as they try to win divisions and position themselves for tie-breakers within the conference).
*Double count road games to reduce the impact of home blowouts (some of the griping about New England reflects their high rating while not having won a road game all season).
Given the comments, there doesn't seem to be a consensus that DVOA represents the perfected light bulb Edison's trying to invent (or FEI in the colleges). If we're not there yet, nothing wrong with tinkering to see if something shines brighter. If it's dimmer we can eliminate it from consideration.
Saw something else matching Cincy for a Super Bowl profile. Think it was Tuesday Morning Quarterback at ESPN that pointed out that five of the last six teams to go 6-0 through their division either went to the Super Bowl, or won it. Going from memory and it's getting late (lol). Not endorsing the Bengals as an automatic lock through the playoffs or anything. Impressed with how much they've improved since last year. Had given up on the franchise.
#104 by narticus // Dec 02, 2009 - 12:17am
I'm not sure I'd buy either the unimodal or the symmetric assumptions, let alone both. I admit up front that I haven't crunched any data on the distribution of DVOA over the years to validate my doubt, and thus I also admit up front that it's entirely possible that you're right about the distribution. It seems to me however that there are many consistently bad teams and several consistently very good teams. Teams in the middle don't stay right in the middle, but rather jump back and forth between good and bad (e.g. Atlanta and Miami in '07 vs. '08, Cinci in '08 vs. '09) and that a bimodal distribution might therefore not be out of the question. I'm also not convinced that the bad teams are exactly as bad compared to the average as the good teams are good compared to the average. I'd be more surprised by symmetry than by skewness. Please don't take this is a vehement argument against you, because I think it's certainly possible that you're right, but rather consider this as a jumping off point for discussion. I will have to ponder this further. Perhaps someone has the time and patience to do the math with all the historical DVOA data.
#121 by Sifter // Dec 02, 2009 - 2:17am
I am a fan of median usage, but I don't think I'd use it outright. I think a mid-point between the median performance and the average performance seems right and I use that a lot in my number crunching. For Cincy, their median performance thus far: week 2 v Green Bay @ approx. 23-24%DVOA and their current average is 8.9% DVOA. I'd combine those figures to move them up to approx. 16% DVOA which would sit them in 11th instead of 15th. It doesn't have to be a 50-50 weight, you might weight average more, but I think adding a median figure into the formula somehow would lessen the drag those outliers have on the numbers. Whether it makes the numbers more predictive, well...who knows??
I also looked at Aaron's charts from the week 10 DVOA article where he showed Indy, Philly, New England, New Orleans and Tennessee. All readings from the charts are estimates and the DVOA averages I quote are from the week 10 article ie. not the current ones. So...
*For Indy, median performance through 9 games was approx. 35% DVOA, about the same as their 34.5%DVOA average
*Philly had a median performance of 5% DVOA (quite low!) compared to their average of 30.4%. They are propped up by 3 big games against Carolina, Tampa and the Giants. Using a median to soften that average would help bring their numbers back to reality, well supposed numerical reality, if that exists.
*New England had a median performance of 45% DVOA compared to their average of 39.8%. So the addition of a median would drag those lovable Pats even higher, which I'm sure would delight the posters...
*New Orleans had a median performance of 25% DVOA compared to their average of 30.4%. So they'd head downwards a little.
*Tennessee had a median performance of around 5% DVOA compared to their average of -12.7%. That shocker against the Pats kills their average making it almost 20% lower than their median performance.
Of course, if I had Premium I could have the exact figures for everybody. Maybe I should look into that...
#129 by jedmarshall // Dec 02, 2009 - 7:01am
This is certainly an interesting discussion if nothing else. I've thought for a while now, there should be some kind of DVOA Cap although where that threshhold should be set would be up to debate. Obviously destroying teams is better than squeaking out wins, but there's a point of diminishing gains. You still only beat (or lose) a single game once. I haven't really done analysis of too many other teams, but it's pretty clear the Eagles are a solid above average team, but far from a juggernaut, however they have 3 games that are greatly skewing their ratings. If those 3 games were capped at a certain level, their ratings would look a bit more realistic.
#145 by SlantNGo (not verified) // Dec 02, 2009 - 9:23am
A cap is an interesting idea, but I think a better way to do it is to use a weighted average, where the weight of a particular game is based on how close it is to the mean. We can do this using only the game-by-game DVOA by fitting a normal distribution with mean and variance equal to the sample mean and sample variance of the game-by-game DVOA. Then to get the weight of a particular game, evaluate the normal density at that game's DVOA. For example, if a particular game's DVOA is 18%, the weight is giving you roughly the probability that such a performance would occur given how the team has performed in the other games. Since we also use the sample variance when we fit the normal distribution, this prevents from assigning too small weights to outlier games from teams that have high variance.
For example, take the Bengals. From the graph, their weekly DVOA looks as follows:
24% 23% 25% -75% 12% -2% 5% 71% 75% -73% -2%
The sample mean is 11.6% (different from the listed DVOA since this assumes an equal number of plays in each game) and the sample variance is 24.5%. The following weights are assigned for each game:
78% 78% 78% 17% 81% 76% 60% 39% 36% 19% 78%
So basically the two really bad games are treated as significant outliers, and the two really good games also as outliers but not quite as significant. Using this weighted system, the Bengals DVOA would come out to be 18.8%.
#247 by Jerry // Dec 03, 2009 - 5:15am
Keep in mind that the DVOA number in the chart is the result of all plays, not games. Obviously, the results of each game's plays can be aggregated to provide a game score, but games aren't the basis of DVOA.
I find it hard to believe that using the median of all plays would be an improvement over the mean.
#122 by Oldcat (not verified) // Dec 02, 2009 - 2:18am
Well, using the median would have the virtue that at least on one week, the team played at that level in a game. In distributions like Philadelpha a week or so ago, the average of their three 100 games and three or four 10 percent games was no where near any actual game performance.
#133 by SlantNGo (not verified) // Dec 02, 2009 - 7:58am
To be honest, I'm not sure I buy those assumptions either, but you're thinking of a different distribution than I am. You're thinking of the distribution of the performance of all teams--this I think would definitely have more than one mode since, as you mentioned, there seem to be clusters of really good and really bad teams.
On the other hand, consider the distribution of a single team. If you believe this to be bimodal, then loosely speaking, you believe that a team is more likely to play well above or well below its true ability than close to its true ability. I don't believe this is true--if it was then "common sense" prediction would be entirely useless.
The symmetric assumption is the sticking point--is a team equally likely to perform above and below its true ability? Or is it more likely that a team underperforms than overperforms? As you mentioned, a good way to verify this is to go back in the DVOA database, look at the distribution of DVOA of all of the games of ONE team, and see if the unimodal and symmetric assumptions hold. Then repeat this for all of the teams and see if they still hold.
#205 by Jeff Fogle // Dec 02, 2009 - 2:23pm
Thanks to everyone who commented in this "discussion within the discussion." Hope somebody can guide me to past articles/comments where the author discussed outliers and "predictive value." Not everyone has been here since the beginning.
Don't think we're at a point yet where the majority is thinking "Oh my god, DVOA has completely solved the riddle." A great tool to be sure, and certainly better than many other computer projection systems. Oddities do pop up. Think we all agree it's a bit odd for an 8-3 team that's 5-0 against top 10 opposition (3-0 on the road) to rank 15th. The author felt it was important enough to put in the headline.
Regarding medians "excluding" results...that's probably why they work when they work. It's probably why they don't when they don't. What's pollution and what isn't? What fogs up the lens and what sharpens the view? A lot of riddles still to be solved. I've seen a lot of riddles solved with medians, or at least conundrums better understood with medians. Not always the best tool obviously. Glad many of you piped in on the issue...
#209 by nat // Dec 02, 2009 - 2:38pm
Have you read FO Basics, FO FAQ, and Our New Stats Explained on the "About" menu? They may not address your exact questions, but they do lay a foundation for discussing DVOA, and some of them have links to pivotal articles from past FO seasons.
#232 by Jeff Fogle // Dec 02, 2009 - 6:13pm
Appreciate the suggestion Nat. Read through the foundations for DVOA, and the stomps and guts article everyone's been talking about.
I think the heart of what's at issue in a lot of the discussions comes from that stomps/guts piece. I agree that championship caliber teams win a lot of blowouts, and that this is what helps differentiate them from the field. I don't think that's reciprocal though...that winning blowouts MEANS you're a championship caliber team.
"Super Bowl champions win blowouts" doesn't mean "Blowout winners are Super Bowl champions" if that makes sense.
Imagine there's a class of team we'll call "bullies," who run up the score vs. bad teams but don't fare well vs. other top competition. The bullies will create the illusion that they're Super Bowl caliber until they run into that string of games in the playoffs where they have to produce vs. top competition.
The playoffs will yield a champion who had a lot of stomps...AND expose bullies whose regular season stomps didn't mean anything.
If you use a system that rewards the stomps, then the bullies are going to look like Super Bowl contenders through the whole regular season even though they're not. The "rankings" are going to be measuring something more like a "Super Bowl profile"...which is something I've seen referenced a few times. It was done in this article/thread with the Bengals. The DVOA chart is arguably "who best fits the profile of what we've determined Super Bowl champions look like." That's fine. Just say that at the top. It's not necessarily a ranking of how the 32 teams will fare against each other head-to-head over 16 games. It's a ranking of the profiles that best match previous Super Bowl winners.
I think it's debatable that "best Super Bowl profiles" is the same as inherent quality when you're talking about how the 32 teams rank against each other, particularly when you're talking about how the top third or half are going to fare when they play each other.
Is Philadelphia a real league championship threat? Or, are they a bully? They've only played two games against the top dozen in DVOA, so it's hard to tell. They're 0-2 in those games (losses to New Orleans and Dallas). Is Green Bay a real threat or a bully? They've won one game against anybody in the top 18 but lost three times.
Cincinnati is 5-0 against the top ten. The lack of stomps could well be an indicator that they're not Super Bowl material. I could see them ranking 15th on a chart of "Who fits the best profile of a Super Bowl champion." I think it's tougher to make the case that they rank 15th in the league in quality.
What do the DVOA ratings represent? What does "team efficiency" mean. Can the thing at the top say "Here's how they rank in team efficiency with an eye toward who best fits the profile of a Super Bowl champion?" That might lower the complaints. New England may have been 7-3 last week, but it's hard to argue that a Belichick coached team wouldn't rank high in fitting a Super Bowl profile.
The eventual Super Bowl champion is probably going to have a lot of stomps. All the teams getting stomps aren't necessarily Super Bowl material. I don't think using medians, or an approach that trims away the outliers, is going to hide the real Super Bowl contenders from us. I do think the approach might help find some of the bullies and lower them to a slot that makes more sense for how they play within the top third or half of the league.
Maybe the current format is best for finding the next Super Bowl champion in terms of predictive value...but one that trimmed off the outliers would yield more accurate internal rankings when the upper half or upper third are playing each other in the regular season and playoffs. This year in particular, with so many teams who are blowout fodder, there's a chance that there are 3-4 "best" Super Bowl threats and double that number of bullies.
Much of the "naked eye" disagreements with DVOA may simply be educated fans knowing a bully when they see one, then complaining when the bully is ranked like a championship threat.
#109 by joon // Dec 02, 2009 - 1:03am
this doesn't even begin to make sense. DVOA is an average over every play, not every game. the biggest reason that it works better than other computerized rankings (which only have game-level granularity) is that larger sample sizes are better. now you say you'd like to throw out 10/11ths of the data? because looking at all the data is somehow worse? um, no, it's not. aaron has said in the past that he ran the tests, and tossing the outliers makes the model less predictive.
and really, given that DVOA averages over every play, the reductio ad absurdum of your argument is that you should rank all the plays and only look at the middle one. now you're throwing out 99+% of your data instead of just 90%. even better!
#233 by Jeff Fogle // Dec 02, 2009 - 6:18pm
Don't think using medians is "throwing out" data. It's more using the data to help pinpoint something of value that best defines the topic at issue. Taking the pressures coming from both sides to create a diamond in the middle. Diamonds are valuable.
Medians don't work for everything. They do work for some things.
#112 by AnonymousCats (not verified) // Dec 02, 2009 - 1:16am
The other thing the two "off the charts" horrible games have in common is that the opposing team switched QBs that week.
Derek "I only play well against Cincy" Anderson and Bruce "I only beat the Bengals" Gradkowski were called on to add their names to the long list of scrub QBs who beat the Bengals like drums (The all-time king of that club is Browning Nagle, who managed to do it by going 8 for 24 with two INTs and no TDs)
I think the Bengals have a chance to do a bit better as the season winds down. I read somewhere that drafting a stud offensive lineman is supposed to do a lot for improving a team's fortunes the following year. Andre Smith saw his first action last week. He's got a chance to be the starting right tackle over Dennis Roland in a couple more games, assuming things go well for him. Getting better blocking from the offensive line might then give coach Lewis the kick in the rear that is needed to finally activate Chase Coffman, who could actually give the Bengals a tight end more dangerous than the two brick-hands schlubs they've been playing.
#127 by Aussie Bengal (not verified) // Dec 02, 2009 - 3:27am
Did you say "stud offensive lineman" and Andre Smith in the same sentence. Come on, that guy's gonna give us headaches for two years then be out.
I mean, I hope you're right but living through the 90's doesn't give me any confidence in Cincy's #1 picks...
#188 by AnonymousCats (not verified) // Dec 02, 2009 - 12:27pm
He's got the chance to turn into Big Fatty Dan Wilkinson in terms of disappointments, but they did pretty well with Levi Jones who was considered a reach at the time of the pick.
Of course this is the Bengals so all sorts of bad outcomes are always possible...
captcha is Adolph bullocks.
#21 by jebmak // Dec 01, 2009 - 8:15pm
Well the Lions are way out in the lead, but Cle, Oak, and Stl sure are neck and neck and neck for #2.
#27 by Arren (not verified) // Dec 01, 2009 - 8:34pm
This update brought to you from Bizarro World, home of the bizarroburger: a delicious bread patty nestled between two tasty ground-beef buns!
#35 by Raiderjoe // Dec 01, 2009 - 8:53pm
Yes to that bizraorburger.
What does jenmak mean?
Liosn not going to get 1st pick. Going to be bestwenn Browns and Bucs and ramms. Not being Raiders or Lons or Redskins. Just look at the scheudle.
#29 by Splat // Dec 01, 2009 - 8:38pm
Re: Cinci
What causes them to grade out as a below average defense, when in conventional stats they have allowed the least points of any team this year?
#38 by Key19 // Dec 01, 2009 - 9:00pm
Well, being in the same division as the Browns certainly doesn't help.
Also, Oakland is rated as a horrible, HORRIBLE offense, and you guys lost to that. Hence, DVOA smites your goals for a great defensive rating.
Other than that, I don't know. I hear you though, because I think Dallas is #2 is least points allowed on defense and yet we're like 18th or something. Just roll with it. I've doubted FO before and have been wrong many times.
#45 by bruddog // Dec 01, 2009 - 9:23pm
The only significant split I see in the advanced stats database is that cinci is ranked #3 and #6 in defensive DVOA respectively when winning big or winning small/tie. And they are below/near average in all other situations.
I haven't watched very little of the Bengals this season but maybe when the are winning it forces the other team to pass more which plays to their strength: the secondary.
I don't think the front 7 is all that great but then again I haven't watched many games so I don't know.
#128 by Vincent Verhei // Dec 02, 2009 - 5:12am
Looking at their week-to-week DVOA trends (available right now in the Premium Database!), two games stand out weirdly. In Week 7 they held the Bears to 10 points, but their defensive DVOA for the game was 5.8%. (Remember that positive DVOA is bad for defense.) Then last week, they held the Browns to 7 points, but their DVOA was 14.2%.
Opponent adjustments are obviously part of that, and the Chicago and Cleveland offenses are putrid, but the Bengals played well against them -- Chicago averaged 2.9 yards per rush and 6.1 yards per pass play, with three picks and sack.
The Browns averaged 3.2 yards per carry, with a long of just 9. They completed less than half their passes and averaged just 7.4 yards per completion. They had six third downs on 36 pass plays.
For whatever reason, DVOA was not impressed with those games, and is punishing the Cincinnati defense for them.
#136 by ammek // Dec 02, 2009 - 8:09am
The Browns game I can just about understand, but Chicago?
It's a weird one because the game was only competitive for 20 minutes, during which time the Bears only ran about a dozen plays. But up until the time the Bengals went up 28-0, the defense had given up I think five first downs on three drives, with no big plays, one forced fumble, and only one fleeting trip inside Cincinnati territory, which was stopped at the 40. Chicago's offense did, to a degree, defeat itself with false starts and an aborted snap. Even so.
Chicago then had a nice two-minute drive which was stopped only by the clock running out. In the third quarter, its two possessions both resulted in picks — one after a longish drive, the other on the fourth play of a 20-yard drive.
I'd be tempted to write the fourth quarter off completely — hello, Caleb Hanie — but the Bears only had two drives: one for a touchdown, the other a three-and-out in mega-garbage time.
Regardless of the general putridity of the Chicago offense, that doesn't feel like a below average game to me. Cincy was even unfortunate with fumble luck, recovering just one of three. Pray tell, what does DVOA not like about that performance?
#139 by andrew // Dec 02, 2009 - 8:25am
Hrm. I know game situation and stuff is already factored into the metric...
but I was wondering if it would make sense to include some kind of drive success variable into evaluating a play... e.g., in two identical games versus identical opponents, a team rushes for 5 yards on first down from the 50. In one game this drive ends in a touchdown, converting a 4th and goal from the 1. In the other it ends at the 1 when time expires.
Could we somehow rate them differently, or would that violate the goal of evaluating each play independently? Because the fact that you did run might be why the clock ran out... also, I'm not sure if anything in DVOA accounts for say stupid challenges that cost timeouts or just poor clock management in general.
e.g., you could have two identical plays, but in one the team allows 40 critical seconds to tick off before the next play while in the other they properly get the next play off without wasting time. That time wasted could have a big impact on who wins or loses, but not sure if thats reflected in any way in DVOA.... I know DVOA is supposed to compare plays, but one of the overall goals is trying to figure out why teams win and lose, right?
#34 by funtime42 // Dec 01, 2009 - 8:49pm
I'd like to see the same Cinci chart with another graph point for their opponent's DVOA the week Cinci played them. Perhaps they're simply the luckiest (both good and bad) team out there.
#37 by John (not verified) // Dec 01, 2009 - 8:59pm
I'm a little disappointed, as I was hoping that the Patriots would STILL be No. 1, despite four losses, including a close one to the Colts and a beat-down at the hands of the Saints. Sadly, it didn't happen. Maybe if that game last night was closer, it would have exposed just how ridiculous the DVOA stats are this year.
I like Football Outsiders, and I like the statistics like DVOA. With that said, DVOA is not the be-all and end-all of all arguments about where teams ranks either in one season or especially historically. And honestly, we have to wait until the Super Bowl to make these arguments anyway. Does anyone think that the '98 Vikings, '04 Steelers and '07 Patriots are among the best teams of all time anymore? Of course not, because they all failed to win the game that mattered most. Best teams that didn't win the Super Bowl, sure. But not among the best teams ever.
The statistics are very interesting, but ultimately, you have to watch the games (Bill Simmons had an excellent column about this in the aftermath of the Fourth-and-2 call a couple weeks back). Frankly, to have two 7-4 teams ranked above a 10-1 Vikings team whose only loss came on the road to the defending champs (back when the Steelers were actually pretty healthy) is ridiculous, and all the stats in the world won't change that.
I, for one, was not at all surprised by what happened on Monday. In fact, that's exactly what I expected. I wasn't at all concerned that the Saints had played close games against mediocre competition. To me, it was fairly obvious that they were bored, and I was convinced that they would get up for the Patriots at home on Monday Night Football. The Saints are better than the Patriots (as it turns out, much better), and they proved that on Monday. Now, if you blindly follow DVOA, you would think that the Patriots should win that game. If you actually use your eyes and your brain, you'd know that the stats don't tell you everything and that the Saints were more likely to win. The Patriots got a lot of credit for annihilating the Titans several weeks ago, but the computer can't account for things like a warm-weather team that was already playing out the string (at the time) quitting because they're miserable in the cold. It also doesn't measure an undefeated team that is doing just enough to win against bad teams because it's bored.
Again, I like Football Outsiders and the statistics like DVOA. But these stats are not gospel, and they don't tell us everything. I think it's a useful tool for identifying strengths and weaknesses of teams that are close in record and perception, but they should not be taken seriously when they're obviously wrong.
#39 by Key19 // Dec 01, 2009 - 9:04pm
"Saints got bored, so they didn't play as well" is quite possibly the worst "analysis" I've ever seen on this site.
Saints can't dominate great running teams. All of their close wins came against teams that are great at running the ball. DVOA confirms this by having the Saints ranked 29th or something just abysmal like that against the run. Dallas and Carolina will give them a run for their money. Not because they are better teams, but because they are built to play the Saints. If Vikings play Saints, Saints will lose, because Vikings are the ultimate "Saints-killers."
#46 by t.d. // Dec 01, 2009 - 9:25pm
I keep hearing Cowboys fans saying their running game is no longer that good, that everyone is relying on impressions formed earlier in the season when all three backs and the o-line were healthier. I'm also scarred by the last time the Cowboys got torched by Brees and Payton. At the time, it was right after Payton had left Parcells' staff, and I attributed some of their advantage to his familiarity with scheme and personnel, but Brees played as well that night as he did against the Pats.
#83 by Key19 // Dec 01, 2009 - 11:07pm
That Cowboys fan you were hearing was probably me. I was very vocal about this team not being a great running team anymore. That said, I think they've finally got things worked out. Felix looks fresh again (which was the main reason I was skeptical of us returning to form, because Felix is so much of our success), Choice now has a solid role in the Razorback, and Marion is close to 100% again in my estimation. The dropoff from RT Marc Colombo to backup RT Doug Free was also less than expected. Hence, I am a believer in the running game again. That said, that may just be Oakland carry-over.
I remember that DAL/NO game as well. Dallas is MUCH better in the secondary now though. No more Anthony "Stone Shoes" Henry and whoever the hell else was back on that miserable defensive backfield. Those were the days before D-Ware and Ratliff had really emerged as well. Also, Wade's scheme is different from Parcells', even though both are 3-4. I think the Cowboys are the toughest matchup on the Saints' entire schedule, past games included. They are built to beat the Saints in my opinion.
Edit: This is not meant to say that the Cowboys will just come out and destroy the Saints, or anything like that. But I expect a very competitive game in which there is a decent chance that the Saints lose.
#113 by t.d. // Dec 02, 2009 - 1:22am
You have more faith that Garrett will gameplan effectively than I do. Just because the Cowboys have the personnel to give the Saints fits doesn't mean they will. Ware was already a stud by then, I thought, though I think Ratliff hadn't yet supplanted Ferguson. I'm more concerned that nobody has been able to get great pressure on Brees yet, he's just too quick.
#47 by Darrel Michaud // Dec 01, 2009 - 9:27pm
Philly and New England both have much better Rush DVOAs than Minnesota and both have similar overall DVOAs. New Orleans stole their lunch money and pantsed them.
#59 by t.d. // Dec 01, 2009 - 9:57pm
Minnesota's rush DVOA doesn't account for gameplanning, since teams don't gameplan to stop the Pats' and Eagles' running games.
#130 by jeff (not verified) // Dec 02, 2009 - 7:06am
pretty quick to forget that Kevin Kolb was making his first start ever in the game. Likewise, the Eagles run offense has only started to jell over the last few weeks. Morever, the game was close until HObbs fumbled the opening second half kick off.
#200 by Darrel Michaud // Dec 02, 2009 - 1:35pm
No, I remember it pretty well. This is the same Kevin Kolb who has a pretty good DVOA in a small sample size. Of course, the Saints also shut down the NE passing game for the most part.
#117 by nath // Dec 02, 2009 - 1:41am
It's not a coincidence that New Orleans' sluggish November stretch against teams that run the ball well coincided with the absence of Sedrick Ellis (and some of it with the loss of Jabari Greer, too).
At the same time, John has a point with his comments on DVOA-- since it's designed to measure performance over the course of the season, it doesn't always tell you how teams can perform at their peak, and it certainly doesn't tell you about matchups. Watching the Rams game, I got the sense that Payton really buttoned up the offense and didn't try to take too many chances. Combine that with a defense that couldn't stop the run, and you had a game that was much closer than it should have been.
When this team plays at its peak, I'm convinced it's the best in the league. That's not the same thing as being the best over the course of a full season, and even the best teams have sluggish stretches.
#42 by Still Alive (not verified) // Dec 01, 2009 - 9:14pm
I have never seen anyone seriously claim they are like gospel.
There is a pretty consistent mantra along the lines of, "here is some spice for your NFL opinion meal, remember to salt to taste". This is a model used to help show people things they are missing. It gives you a dispassionate view of what is happening, albeit an imperfect one.
There is a lot of bashing on other methods as compared to DVOA, rightfully so as DVOA generally outperforms them quite a bit.
Anyway you just seem to be setting up a straw man to beat up as a segue to your own ridiculous analysis.
#73 by Bobman // Dec 01, 2009 - 10:37pm
Still Alive, I beg to differ.
I clearly recall one night a few years back when Barnwell held my arms behind my back, Tanier had my kids bound and gagged in the next room, and Aaron held a loaded falafel to my head, ordering me to swear eternal fealty to DVOA. I refused and BAM! tahini all over my foil hat.
Worst.
Day.
Of.
My.
Life.
#78 by Will Allen (not verified) // Dec 01, 2009 - 10:53pm
Aaron really blew it when he didn't, for the cover of FO Almanac, photoshop Matt Millen's head, with a lemon meringue pie poised to strike, with the title, "Buy this book, or the clown gets it!"
#87 by Bobman // Dec 01, 2009 - 11:16pm
Damn, that is hitting at something buried in my memory... 1984, National Lampoon Magazine. Pretty sure there was a labrador retriever on the cover and a hand holding a gun to it's head with the caption: "buy this magazine or the dog gets it."
Yours is funnier, of course.
#90 by Will Allen (not verified) // Dec 01, 2009 - 11:24pm
Mine is theft, of course, but thanks anyways.
#91 by justanothersteve // Dec 01, 2009 - 11:25pm
It's far older than that - at least according to http://en.wikipedia.org/wiki/National_Lampoon_%28magazine%29. The quote from 1973 is "If You Don't Buy This Magazine, We'll Kill This Dog". My all-time favorite fake ad is there too, parodying the VW floats ads.
#95 by Raiderjoe // Dec 01, 2009 - 11:48pm
Yes Bobman correct
Before I was thinkign about Mad cover with dog from beverly Hills Cop on it.
#43 by Raiderjoe // Dec 01, 2009 - 9:17pm
Excellent post by John (not verfried). Maybe bets post ever wriiten on site. Only thing to argue about with it was comment about Ttennesse Titnas being warm waether team. Tennessee in south but not deep south like Georgia and Flirida. Weather known to get cold in Tennessee. If old enough to rmemeber 1999 Bills at Titnas posteason game, many fans wore wintr coats and hats and things like that. Same thing next year when ravens go there to play playoff fame.
Also check stats for 2000 final Monday night game. Dallas go to Nashbille and very cold game titans win game 31-0
#178 by Marcumzilla // Dec 02, 2009 - 11:54am
As a recent transplant to the Nashville area, let me chime in that while it's not GA or FL, the winters still tend to be very mild temperature-wise. Last year we only had ten days of what I'd classify as winter-weather (snow and/or low temp), and I heard complaints that it was the coldest winter in a while. Don't even get me started on school closures. I was happy for the cold spell so I could wear the new coat my wife bought me for Christmas.
#49 by Rick A. (not verified) // Dec 01, 2009 - 9:28pm
John-exactly. Thank you.
#60 by RickD // Dec 01, 2009 - 9:58pm
I don't know if I would say that the Saints are "much better" than the Pats, though that was certainly the case last night. The Pats usually don't play as poorly as they did last night. I'm all for giving credit to the victor, but when a WR runs downfield completely uncovered, it's a sign that the defense is playing atypically poorly.
The flip side is that I don't think we can rely upon the Saints on playing at that level for the next two months. They had definitely keyed on the Pats game as a way to prove themselves.
#64 by t.d. // Dec 01, 2009 - 10:10pm
38 points isn't an exceptional output for the Saints' offense. It's the 17 points for the Patriots that really stands out to me. And the fact that the Saints d-line spent the whole game in the Pats' backfield. Granted, Volmer was out for the Pats, but he isn't even a starter, and the Saints' starting secondary was depleted, too. It should have been a huge mismatch for the Pats, but they couldn't exploit it.
#93 by RickD // Dec 01, 2009 - 11:36pm
So...we're grading the Pats' defense on a curve?
No team can expect to give up 38 points against anyone and still win. What's worse was _how_ the team gave up 38 points. Huge yardage plays that led to quick TDs - about three in a row.
And once the Pats are down by more than 2 TDs, with their defense collapsing, they can hardly sit back and run the ball any more. The offense wasn't that bad, modulo the turnovers. They moved the ball, gaining 366 yards at a clip of 5.2 yards per offensive play. Yes, their season average is a bit higher, at 5.9 yards/play. But not by all that much.
In contrast, the defense gave up 9.6 yards/snap to New Orleans.
9.6 yards per play!!!!
Their season average is 5.5 yards/play. Even the league-worst Detroit Lions have given up only 6.2 yards/play.
9.6 yards/play is more than 50% worse than that!!!
5.2 yards/play is a reasonable clip. A defense that gives up 9.6 yards/play is not going to beat anybody.
And you may think it's easy to put up 38 points, but even the Saints (the league leader) only score 37 points/game.
#110 by t.d. // Dec 02, 2009 - 1:03am
So the Saints got one more point than their average. You seem to think the Patriots defense is some great unit that had an off day. They've been shredded twice in three weeks.
#114 by RickD // Dec 02, 2009 - 1:32am
I really have no idea where you get the idea that I think the Patriots defense is "some great unit". I thought you were the one saying that the Pats' offense was the unit to blame. You said:
"38 points isn't an exceptional output for the Saints' offense. It's the 17 points for the Patriots that really stands out to me." The implication being that a team should just expect to give up 38 points to the Saints, but if they don't score 38 themselves, something is wrong with the offense.
DVOA has the Pats' defense pretty much at the middle of the pack. But
a) they played awful on Monday, so blaming the Pats' offense is..just..weird..
b) they made a lot of mental errors on Monday that they do not usually make. As Mike Reiss said, blowing coverage on a CB blitz is something you do not even expect to see from a high school JV team.
Tell ya what - let me know whether you're blaming the loss on the Pats' d or on the Pats' o and then I'll be able to respond. You seem to have taken contradictory stances within the span of 3 hours.
#124 by t.d. // Dec 02, 2009 - 3:07am
I wasn't the one blaming the loss on blown coverages. Blown coverages seem to happen against the Saints a lot. The Pats weren't the ones starting two street free agents in their secondary. The Saints had an average offensive game, for them. The Pats lost because their great offense couldn't keep Brady upright and protected.
#246 by RickD // Dec 03, 2009 - 4:29am
I'm failing to make the connection between "blaming the loss on blown coverages" and "thinking the Pats have some great defense". If the Pats had a great defense, they wouldn't be blowing coverages like that, would they?
#67 by bravehoptoad // Dec 01, 2009 - 10:24pm
I, for one, was not at all surprised by what happened on Monday.
Prove it.
Post your predictions here for the upcoming weekend. We'll compare them to DVOA's and see if you're as smart as you say you are.
For some reason, this site brings out more omniscient-after-the-fact (not verified) blowhards than any other.
#80 by Key19 // Dec 01, 2009 - 10:57pm
lol I always love (not verified) jokes. Cheers to you.
#79 by Will Allen (not verified) // Dec 01, 2009 - 10:56pm
Please cite where it was ever claimed that the stats here "explain everything". Hey, DVOA didn't cure my halitosis, either! That's actionable!!!!
#81 by Key19 // Dec 01, 2009 - 11:05pm
If a team performance-rating system simply produced numbers that correlated perfectly with team record, said system would be utterly useless. Seeing things from a new perspective is not only healthy, but useful. I also feel that the non-human degree of objectivity (though we all know the machine is not truly objective, because it is not a gift of god, but merely a creation of Aaron, who is indeed a human, so it has some human biases, but not many) is very useful beyond what any human ranking system is.
DVOA is not perfect, but it's sure a hell of a lot better than resorting to other b/s stats and "analysis" out there.
If you want to beat up on analysis, turn on ESPN and yell at the TV all day. Even try heading over to their website if you so choose.
It never ceases to amaze me how many people come here simply to say "this place sucks." And then what do you know, there they are again next week, piping in to say "this place sucks" again. Maybe I'm old-fashioned, but I generally avoid places I hate.
This was not really a reply to your post, Will Allen (why are you not verified btw?), but merely a continuation.
#84 by JIPanick // Dec 01, 2009 - 11:08pm
"Does anyone think that the '98 Vikings, '04 Steelers and '07 Patriots are among the best teams of all time anymore?"
*Raises hand*
Two out three anyway, but I always thought Pitt '04 was overrated.
#123 by Jerry // Dec 02, 2009 - 2:54am
You're entitled to your opinion, but the two-week stretch where the Steelers hung emphatic first losses (with their rookie QB) on New England and Philly was pretty damned impressive.
#159 by JIPanick // Dec 02, 2009 - 10:29am
I think they were good, just not all-time great good.
#118 by 20938nva09ewn2[0t (not verified) // Dec 02, 2009 - 1:43am
Anyone who doesn't think the 2007 Patriots were one of the best football teams ever because they lost one game is an idiot, no matter how high the leverage of that game was for a particular trophy.
#154 by dbostedo // Dec 02, 2009 - 10:08am
I think I'm just repeating things others have said, but a few points :
1) Based on your first paragraph, you seem to want DVOA to not match win-loss records and/or public perception, just so you can claim that it's fallible. I'm not sure why.
2) I think those teams you mentioned are among the best of all time when that phrase is looked at objectively. A little more subjectively, you might argue that teams that don't win a championship can't be considered among the best of all time. But I'd then argue that the playoffs an super bowl are only somewhat related to proving who the best team is - the much larger sample size of the regular season actually does that better than the playoffs.
3) As for your perfect expectations on Monday, I'll echo the poster who said "prove it". I hear this from a lot of people (sometimes on here even), and it's very often just not right. What happens (just guessing here) is that people have a lot of "feelings" or thing they "know" about every big game. The times they are right, they remember and talk about and brag about. The times they are wrong, they simply forget or don't talk much about. Either way, it's mostly guessing. (I know I've had a vague notion that team A was better than team B, and when A destroys B I can get caught up in that and feel like "Hey, I KNEW that all along.")
4) Agreed that DVOA can't measure many things, and they do matter. But right now, there's nothing better out there, and no way to incorporate that stuff.
To elaborate, say you asked a lot of people to start with DVOA, make their own adjustments based on watching lots of games. The new ratings would probably fall in line more with what you're talking about - they'd be based more on big wins, overall record, etc. I'm willing to bet those ratings would be LESS predictive on average than DVOA. The point being that people would be wrong in how they change the ratings at least as often as they are right, and it would hurt predictive accuracy overall.
#162 by Temo // Dec 02, 2009 - 10:46am
With that said, DVOA is not the be-all and end-all of all arguments about where teams ranks either in one season or especially historically.
No one said it was.
Does anyone think that the '98 Vikings, '04 Steelers and '07 Patriots are among the best teams of all time anymore?
No one said that either. Especially about the '04 Steelers.
The statistics are very interesting, but ultimately, you have to watch the games
Seriously? And here I thought I could understand the game of football with spreadsheets alone.
Bill Simmons had an excellent column about this in the aftermath of the Fourth-and-2 call a couple weeks back
That was a terrible half-assed column. Like most of his columns these days.
Frankly, to have two 7-4 teams ranked above a 10-1 Vikings team whose only loss came on the road to the defending champs... is ridiculous
You just got done saying that teams with great regular season records (like the '04 Steelers) were not as good as their records indicate. And now you're upset because 7-4 teams are ranked above a 10-1 team. Besides, all those teams are very close in ranking and no one is saying for sure that the 7-4 teams are worse than the Vikings. It's way too close to call.
I, for one, was not at all surprised by what happened on Monday.
Congratulations. Now if you find a system that is as accurate as DVOA for every other game, I'll gladly listen further.
Now, if you blindly follow DVOA, you would think that the Patriots should win that game.
Actually, DVOA ranked both teams very close and the Saints were at home (3 point advantage), so DVOA thought the Saints would win a close game.
But these stats are not gospel, and they don't tell us everything
No one said they did.
I think it's a useful tool for identifying strengths and weaknesses of teams that are close in record and perception, but they should not be taken seriously when they're obviously wrong.
And you'd be absolutely right! Stop getting caught up in rankings when the DVOA between the top teams is so close.
If you believe that the Vikings are the 3rd best team in the league, and a better team than the Pats and Eagles, because the latter derive a lot of their value from huge days against quitting teams, then you can say that. The DVOA between these 3 teams is close enough to make that a sane argument. But you know, if you said Pittsburgh or Dallas were better than the Pats or Eagles, then DVOA would have a problem, because those teams are not close.
#40 by Alexander // Dec 01, 2009 - 9:05pm
How close is the Pat's romp over Tennessee to becoming one of DVOA's greatest games now that the team seems to be doing good things?
#41 by t.d. // Dec 01, 2009 - 9:14pm
I think the playoff odds report overstates the chances of Indianapolis going undefeated, and understates New Orleans' chances.
#44 by Still Alive (not verified) // Dec 01, 2009 - 9:17pm
That is nice, and I think kitties are horrible creatures, care to add some reasoning?
#58 by t.d. // Dec 01, 2009 - 9:55pm
The Saints' toughest opponents remaining, Dallas and Atlanta, are less formidable than a mathematical formula that doesn't take matchups or home field advantage into account can figure. Matt Ryan is out again this week, and who knows whether he and Michael Turner will be healthy in 12 days when they play. The last time Dallas played New Orleans, Brees absolutely torched them and the vaunted pass rush got nowhere near him. The Cowboys offensive line is also debilitated, and they will be in the midst of a brutal schedule, and their head coach has a history of his teams fading down the stretch. The toughest game, barring injuries, appears to be their finale against Carolina, when they could well have home field wrapped up, against a team whose coach might have his job on the line, and whose strengths match up well against the Saints' weaknesses.
The Colts, on the other hand, have four skates in their last five games, and they face four more consecutive games against teams of a similar profile (that is, neither good or terrible).
The Saints have been dominant far more often than the Colts, against a tougher schedule.
#62 by RickD // Dec 01, 2009 - 10:04pm
I don't think you need that much analysis.
In two weeks, the Colts will have the #1 seed wrapped up, and won't give a damn about going undefeated. (Or they will have already lost a game.) On January 3, if it's freezing out in Buffalo, are the Colts really going to care enough to try to win?
The Saints are only 1 game ahead of the Vikings and it's quite possible that they wouldn't be able to clinch home field advantage before Week 16 or 17.
I don't expect DVOA to take that kind of analysis into account. It's just a statistical tool. It's not an oracle.
If you want another analysis, the Colts beat the Pats by one and the Saints beat the Pats by 21. Q.E.D.
#132 by jedmarshall // Dec 02, 2009 - 7:53am
Your first analysis makes some sense, however I don't see it being the Buffalo game the Colts lose, but they will have homefield sewn up by at least the end of Week 15 and the Saints-Vikings may go down to the wire.
Your other analysis is just silly though. The Saints beat the Rams by 5 and the Colts beat the Rams by 36. Deciding who is better using a one game sample size is ridiculous.
#138 by Spoon // Dec 02, 2009 - 8:24am
Heh, but Saints beat the Rams by 5 and the Colts beat the Rams by 36. Q.E.D
#51 by Rick A. (not verified) // Dec 01, 2009 - 9:33pm
The odds in "football reality" are extreme that neither team will go undefeated so it really doesn't matter what the hypothesis by DVOA is.
#151 by tgt2 (not verified) // Dec 02, 2009 - 9:55am
I'm impressed by the proper use of quotes around football reality to show that it's a load of bull.
#48 by Big Johnson (not verified) // Dec 01, 2009 - 9:28pm
didnt i read here that good teams beat bad opponents by a lot?
ill bring once again whiney questions from san diego. san diego beats kc by 29 points and move up what? 1 spot? not unheard of but still not likely. the week before, they beat denver by 29 points and move up what? 1 spot? considering that denver is AHEAD of them that is very unlikely. then the two weeks before that they beat AGAIN two teams ranked ahead of them.... the eagles by 8 and the giants by 1. they beat the raiders by 8 and the chiefs by 30. i think i got it in reverse order and i dont think i missed any games. during that stretch they moved up like 5 spots. 3 absolute blowouts. a couple semi close 8 point games. 1 very close game. considering that 3 of the games were against teams rated higher than them and 3 were against lower its not like they faced an extremely easy schedule. and if beating the chiefs by 30 and 29 points doesnt move a team up then why do the steelers stay up when they lose to them?! im losing faith in dvoa. it seems to never predict winners. and also does this put to rest the norv hate since he has the chargers winning games that they shouldnt?
ive got one more question. why do certain teams seem to have a reserve spot at the top of dvoa and is there a type of preseason prediction that holds the ravens and steelers as good teams no matter how well they play?
#54 by Rick A. (not verified) // Dec 01, 2009 - 9:44pm
Good question about the seemingly pre-ordained slots for Balt and others. How about how bad New England's defense is in actual reality and yet this team is supposedly a more efficient, overall, squad than Minny ? No wonder Bellicheat went for it on 4th down in Indy, he knows how bad they are. Any team with good stickies and a QB that can get them the ball is going to take the measure of N.E.'s D down the stretch. They have no business being in anyone's power rating as one of the very top teams. As so many of the guys in this thread keep pointing out, the Tennessee non-game has skewed things horribly. That was a measurement that meant absolutely nothing because there was a team with the "blue-flu", wanting to get rid of their over-the-hill QB, who did what so many of these teams will do in today's league for varying reasons--purposefully not perform. In the meantime DVOA "measures" this robbery of the fans as if some quantification of the non-effort would actually say something. So New England likes to wrack up the score and stats, I'm real impressed...