Week 4 DVOA Ratings
by Aaron Schatz
A young, porous defense? No problem. Zero catches for Randy Moss? Easily identifiable fluke. While the New England press (and many fans) obsess over every last Patriots weakness, the Football Outsiders mainframe has proclaimed New England the best team in the NFL, albeit with a sizeable "so far" asterisk. Last night's big win over Miami vaults the Pats over Kansas City, Green Bay, and the New York Jets and into the top spot in our DVOA ratings. In fact, the Patriots zoom to number one even though two of the biggest plays of the night do not count in our special teams ratings. The Patriots get no credit for the blocked field goal, because blocks are rare and scoring on a missed field goal is rarer still. They get no credit for the punt block, because our punt return ratings for the receiving team only judge punt returns. The Dolphins do get penalized for both events, although the penalty for a blocked field goal is no different from the penalty for any other missed field goal.
Week 4 is when we introduce opponent adjustments into our formula for the first time. The hardest schedules so far have belonged to Buffalo, San Francisco, and Baltimore; the easiest have belonged to St. Louis, New Orleans, and Green Bay. The Jets have one of the biggest gaps between DVOA and non-adjusted VOA, but that's not because of schedule -- that's because of fumble recovery. The Jets have recovered four of five fumbles on defense and two of three fumbles on offense.
This year's parity -- with no dominant teams and lots of close games -- has also led to some strange results in the early DVOA rankings. Winless Detroit ranks three spots higher than 3-1 division rival Chicago. 2-2 Arizona is in last place, with two very close wins over below-average opponents far outweighed by two embarrassing blowout losses. The 1-2 Cowboys rank higher than six different teams with winning records. As for surprise teams, it's worth nothing that both Kansas City and Houston have played roughly average schedules so far. Based on current DVOA, however, Houston has the hardest remaining schedule in the league, while the Chiefs have the second-easiest.
There are also some strange goings-on with the player ratings so far, so let's take a look.
QUARTERBACKS
The pass-run balance in the NFL continues to tip towards the pass this year, with a number of quarterbacks off to historically ridiculous starts. Kyle Orton, Peyton Manning, and Philip Rivers are all on pace to break Dan Marino's record of 5,084 passing yards in a season. Obviously, we're used to hearing these kind of "on pace" stats early in the season, in every sport, and those players rarely stay on that pace. However, these three quarterbacks aren't on pace to barely top the record. All three are on pace to blow by it by at least 200 yards. Even with the inevitable slowdown, somebody is going to end up as just the third quarterback in NFL history with at least 4,900 passing yards. The other one, besides Marino, was Drew Brees two years ago, and that Brees season goes a long way towards explaining some of what we're seeing this year.
Drew Brees was very good in 2008, but he wasn't great. He put up great numbers because he threw the ball a ridiculous amount trying to keep up with the points given up by the Saints' terrible defense that year. He ended up with 635 attempts, the fifth highest season total in NFL history. This year, both Kyle Orton and Peyton Manning have been passing non-stop, stuck in close games because of poor defense. The Broncos rank 28th in defensive DVOA, the Colts 27th. Orton has the added albatross of a horrible running game, the worst in the league so far this season. Orton is a very good eighth in DVOA, but he's an even better second when it comes to DYAR, because he's thrown the ball 175 times. Manning is playing better on a per-play basis -- he leads the league in DVOA, not much of a surprise -- but he's also throwing the ball non-stop, with 173 attempts. Those numbers pro-rate to 700 attempts and 688 attempts over an entire season. The record is 691 attempts, by Drew Bledsoe in 1994, and no other season really comes close. Warren Moon (655 in 1991) and Drew Brees (652 in 2007) are the only other quarterbacks with more than 640 pass attempts in a season. Even if they slow down a little, it is hard to imagine Orton and Manning not hitting 640 pass attempts if they stay healthy.
Usually when teams throw this much, you don't expect a high level of accuracy, but that's just another way that the current offensive environment is so unique. So far this year, quarterbacks have completed 62 percent of passes, which would be an NFL record if it stays that high through 16 games.
RUNNING BACKS
Raise your hand if you expected the three most valuable running backs in the NFL to be Arian Foster, BenJarvus Green-Ellis, and Mike Tolbert? Those are the top three guys in rushing DYAR right now. LaDainian Tomlinson is fourth; Neil Paine of the P-F-R blog did a good write up this week about how historically unprecedented it is for a running back to have a career rebound like Tomlinson's after age 30.
Meanwhile, a series of strong veteran running backs have been below-average so far this year. Maurice Jones-Drew, Steven Jackson, Michael Turner, and Ricky Williams all rank 25th or lower in DYAR. Matt Forte's expected statistical rebound has so far only taken place in his receiving numbers -- he's dead last in rushing DYAR by a significant margin. But the strangest name in the strangest place is Chris Johnson. Johnson is currently 35th out of 37 running backs in DYAR. He's averaging just 3.8 yards per carry, a far cry from the 5.6 yards per carry he had a year ago. Even when he had 125 yards against the Giants in Week 3, it took him 32 carries. As we wrote in the book, Johnson was definitely going to face regression towards the mean, but he has to be better than this, right? The Titans line is 26th in Adjusted Line Yards but it isn't like Johnson had far better blocking last year, because the Titans were only 21st in ALY in 2009. You know Johnson's goal of 2,500 yards this season? Yeah, not happening.
WIDE RECEIVERS
If you think seeing Chris Johnson's name near the bottom of the running back rankings is odd, that's nothing compared to the receiving rankings, where Larry Fitzgerald currently ranks 74th in DYAR out of 86 wide receivers. It's not Fitzgerald's fault that Arizona can't find a starting-quality quarterback, but plenty of top receivers have put up good seasons with bad quarterbacks. (Hi, I'm Steve Smith.) Fitzgerald ranked in the DYAR top ten for four seasons, 2005-2008, and then ranked 11th last year. His catch rate, consistently over 60 percent until this year, is currently 42 percent for the season.
Another great receiver with bad numbers this year, partly because of the quarterback situation, is Lee Evans. Evans is 77th in DYAR, but what's even stranger is that the Bills quarterbacks have only thrown to him 22 times (including one Pass Interference). This guy is the best receiver on the team by a mile, and they barely throw him five passes a game? The Bills have thrown the same number of passes to Steve Johnson. Johnson is actually doing more with those passes -- 63 percent catch rate, 36th in DYAR -- but that's because no team is going to make sure they have a safety playing over the top to keep the defense from getting burned by Steve Johnson.
Meanwhile, at the top of the wide receiver rankings, the most valuable receiver this year has been... Brandon Lloyd. Yes, the guy who was run out of San Francisco, the guy who gave the Washington Redskins 25 catches and no touchdowns for $10 million guaranteed, the guy who barely could get on the field for Denver last year (two games, eight catches). Will Lloyd continue to produce for the Broncos? Or, as in every one of his previous career stops, will teammates and coaches gradually sour on him because he is, apparently, a total jackass?
Anyway, on to the ratings. All stat pages will be updated in the next few minutes. The FO Premium database of DVOA splits will be updated later tonight. Apparently, something went wrong with last week's update, but this week's update should get all the searchable splits fully updated through Week 4.
* * * * *
These are the Football Outsiders team efficiency ratings through four weeks of 2010, measured by our proprietary Defense-adjusted Value Over Average (DVOA) system that breaks down every single play and compares a team's performance to the league average based on situation in order to determine value over average. (Explained further here.)
OFFENSE and DEFENSE DVOA are adjusted to consider all fumbles, kept or lost, as equal value. SPECIAL TEAMS DVOA is adjusted for type of stadium (warm, cold, dome, Denver) and week of season.
Opponent adjustments are currently at 40 percent strength and will steadily grow stronger until Week 10. As always, positive numbers represent more points so DEFENSE is better when it is NEGATIVE.
DAVE is a formula which combines our preseason projection with current DVOA to get a more accurate forecast of how a team will play the rest of the season. Right now, the preseason projection makes up 40 percent of DAVE for teams that have played four games and 55 percent of DAVE for teams that have played three games.
To save people some time, please use the following format for all complaints:
<team> is clearly ranked <too high/too low> because <reason unrelated to DVOA>. <subjective ranking system> is way better than this. <unrelated team-supporting or -denigrating comment, preferably with poor spelling and/or chat-acceptable spelling>
| TEAM | TOTAL DVOA |
LAST WEEK |
TOTAL DAVE |
RANK | W-L | OFFENSE DVOA |
OFF. RANK |
DEFENSE DVOA |
DEF. RANK |
S.T. DVOA |
S.T. RANK |
|
| 1 | NE | 34.1% | 6 | 29.0% | 1 | 3-1 | 42.9% | 1 | 13.0% | 23 | 4.2% | 4 |
| 2 | KC | 33.6% | 2 | 14.7% | 9 | 3-0 | 11.5% | 10 | -16.3% | 5 | 5.8% | 3 |
| 3 | NYJ | 31.2% | 4 | 23.7% | 3 | 3-1 | 22.8% | 5 | -5.5% | 12 | 2.9% | 8 |
| 4 | GB | 27.6% | 1 | 22.8% | 4 | 3-1 | 27.0% | 4 | -7.9% | 8 | -7.3% | 29 |
| 5 | SD | 26.0% | 15 | 13.4% | 10 | 2-2 | 20.8% | 7 | -26.0% | 1 | -20.7% | 32 |
| 6 | PIT | 24.5% | 3 | 25.4% | 2 | 3-1 | -1.9% | 19 | -22.9% | 2 | 3.6% | 6 |
| 7 | PHI | 20.4% | 5 | 18.0% | 5 | 2-2 | 20.9% | 6 | -0.9% | 16 | -1.4% | 19 |
| 8 | DAL | 18.0% | 8 | 9.6% | 12 | 1-2 | 19.1% | 8 | 2.2% | 18 | 1.1% | 12 |
| 9 | ATL | 16.2% | 7 | 17.4% | 7 | 3-1 | 7.6% | 12 | -7.8% | 9 | 0.8% | 13 |
| 10 | NYG | 14.5% | 20 | 12.2% | 11 | 2-2 | 1.6% | 16 | -22.5% | 3 | -9.7% | 30 |
| 11 | IND | 13.9% | 10 | 16.5% | 8 | 2-2 | 32.6% | 2 | 15.0% | 27 | -3.7% | 25 |
| 12 | TEN | 11.4% | 13 | 4.1% | 13 | 2-2 | -9.2% | 21 | -20.2% | 4 | 0.4% | 14 |
| 13 | SEA | 8.8% | 9 | 2.6% | 15 | 2-2 | -14.3% | 24 | -6.2% | 11 | 16.9% | 1 |
| 14 | BAL | 8.6% | 18 | 17.8% | 6 | 3-1 | -1.6% | 18 | -10.9% | 6 | -0.7% | 17 |
| 15 | HOU | 4.1% | 21 | -4.9% | 20 | 3-1 | 31.2% | 3 | 26.1% | 31 | -0.9% | 18 |
| 16 | CIN | 2.3% | 16 | 0.3% | 17 | 2-2 | -0.2% | 17 | -4.6% | 13 | -2.1% | 22 |
| TEAM | TOTAL DVOA |
LAST WEEK |
TOTAL DAVE |
RANK | W-L | OFFENSE DVOA |
OFF. RANK |
DEFENSE DVOA |
DEF. RANK |
S.T. DVOA |
S.T. RANK |
|
| 17 | NO | 0.2% | 12 | 4.0% | 14 | 3-1 | 12.6% | 9 | 14.3% | 25 | 1.9% | 10 |
| 18 | TB | -1.4% | 17 | -7.0% | 21 | 2-1 | -10.3% | 22 | -6.9% | 10 | 2.0% | 9 |
| 19 | CLE | -5.1% | 19 | -11.3% | 23 | 1-3 | 2.9% | 15 | 8.0% | 21 | 0.0% | 16 |
| 20 | WAS | -7.3% | 25 | 0.4% | 16 | 2-2 | 8.5% | 11 | 13.6% | 24 | -2.1% | 21 |
| 21 | MIA | -11.0% | 11 | -1.4% | 19 | 2-2 | 3.2% | 14 | 4.1% | 20 | -10.2% | 31 |
| 22 | STL | -11.7% | 23 | -15.8% | 25 | 2-2 | -17.4% | 26 | -8.1% | 7 | -2.4% | 23 |
| 23 | DEN | -15.6% | 22 | -12.0% | 24 | 2-2 | 4.3% | 13 | 16.3% | 28 | -3.6% | 24 |
| 24 | DET | -17.4% | 26 | -23.8% | 27 | 0-4 | -11.6% | 23 | 9.3% | 22 | 3.5% | 7 |
| 25 | MIN | -18.9% | 24 | -1.1% | 18 | 1-2 | -21.8% | 28 | -2.9% | 14 | 0.1% | 15 |
| 26 | JAC | -20.3% | 30 | -16.7% | 26 | 2-2 | -2.7% | 20 | 25.1% | 30 | 7.6% | 2 |
| 27 | CHI | -21.9% | 14 | -7.5% | 22 | 3-1 | -22.8% | 29 | 0.2% | 17 | 1.2% | 11 |
| 28 | SF | -31.7% | 32 | -26.7% | 29 | 0-4 | -23.0% | 30 | 3.4% | 19 | -5.3% | 28 |
| 29 | CAR | -35.5% | 31 | -24.4% | 28 | 0-4 | -35.6% | 32 | -1.8% | 15 | -1.7% | 20 |
| 30 | BUF | -40.4% | 28 | -28.7% | 30 | 0-4 | -15.1% | 25 | 29.0% | 32 | 3.7% | 5 |
| 31 | OAK | -41.4% | 29 | -37.0% | 32 | 1-3 | -19.9% | 27 | 17.1% | 29 | -4.4% | 27 |
| 32 | ARI | -45.3% | 27 | -29.3% | 31 | 2-2 | -27.0% | 31 | 14.5% | 26 | -3.8% | 26 |
- ESTIMATED WINS uses a statistic known as "Forest Index" that emphasizes consistency as well as DVOA in the most important specific situations: red zone defense, first quarter offense, and performance in the second half when the score is close. It then projects a number of wins adjusted to a league-average schedule and a league-average rate of recovering fumbles. Teams that have had their bye week are projected as if they had played one game per week.
- PAST SCHEDULE lists average DVOA of opponents played this season, ranked from hardest schedule (#1, most positive) to easiest schedule (#32, most negative). It is not adjusted for which games are home or road.
- FUTURE SCHEDULE lists average DVOA of opponents still left to play this season, ranked from hardest schedule (#1, most positive) to easiest schedule (#32, most negative). It is not adjusted for which games are home or road.
- VARIANCE measures the statistical variance of the team's weekly DVOA performance. Teams are ranked from most consistent (#1, lowest variance) to least consistent (#32, highest variance).
| TEAM | TOTAL DVOA |
W-L | NON-ADJ TOT VOA |
ESTIM. WINS |
RANK | PAST SCHED |
RANK | FUTURE SCHED |
RANK | VAR. | RANK | |
| 1 | NE | 34.1% | 3-1 | 34.8% | 3.1 | 2 | -4.5% | 20 | 1.4% | 12 | 12.1% | 18 |
| 2 | KC | 33.6% | 3-0 | 31.6% | 3.1 | 1 | -3.6% | 16 | -12.9% | 31 | 8.6% | 12 |
| 3 | NYJ | 31.2% | 3-1 | 40.8% | 3.0 | 4 | -2.2% | 14 | -3.1% | 23 | 5.7% | 10 |
| 4 | GB | 27.6% | 3-1 | 35.0% | 3.0 | 3 | -14.8% | 30 | -1.1% | 18 | 10.6% | 14 |
| 5 | SD | 26.0% | 2-2 | 31.8% | 2.7 | 5 | -5.8% | 23 | -4.8% | 25 | 42.1% | 29 |
| 6 | PIT | 24.5% | 3-1 | 22.9% | 2.7 | 6 | 8.7% | 7 | -5.0% | 26 | 14.5% | 19 |
| 7 | PHI | 20.4% | 2-2 | 27.7% | 2.6 | 7 | -4.3% | 19 | 2.6% | 10 | 10.8% | 16 |
| 8 | DAL | 18.0% | 1-2 | 16.8% | 2.4 | 10 | -8.3% | 26 | 1.1% | 16 | 21.5% | 26 |
| 9 | ATL | 16.2% | 3-1 | 18.6% | 2.3 | 13 | -13.0% | 29 | -1.9% | 20 | 19.9% | 24 |
| 10 | NYG | 14.5% | 2-2 | 6.5% | 2.3 | 14 | -8.0% | 24 | 3.8% | 6 | 45.6% | 31 |
| 11 | IND | 13.9% | 2-2 | 16.4% | 2.5 | 9 | -4.3% | 18 | 7.7% | 2 | 19.0% | 22 |
| 12 | TEN | 11.4% | 2-2 | 12.8% | 2.5 | 8 | -4.5% | 21 | 6.3% | 4 | 18.6% | 21 |
| 13 | SEA | 8.8% | 2-2 | 6.3% | 1.8 | 20 | -8.2% | 25 | -14.1% | 32 | 19.8% | 23 |
| 14 | BAL | 8.6% | 3-1 | 0.9% | 2.4 | 11 | 13.2% | 3 | -2.3% | 22 | 5.2% | 8 |
| 15 | HOU | 4.1% | 3-1 | 7.1% | 2.2 | 15 | -4.2% | 17 | 9.6% | 1 | 16.9% | 20 |
| 16 | CIN | 2.3% | 2-2 | 3.5% | 2.0 | 17 | 0.5% | 13 | 7.3% | 3 | 10.4% | 13 |
| TEAM | TOTAL DVOA |
W-L | NON-ADJ TOT VOA |
ESTIM. WINS |
RANK | PAST SCHED |
RANK | FUTURE SCHED |
RANK | VAR. | RANK | |
| 17 | NO | 0.2% | 3-1 | 3.2% | 2.1 | 16 | -17.5% | 31 | -1.8% | 19 | 2.4% | 3 |
| 18 | TB | -1.4% | 2-1 | -0.2% | 2.4 | 12 | -5.4% | 22 | -7.4% | 28 | 30.4% | 27 |
| 19 | CLE | -5.1% | 1-3 | -2.4% | 2.0 | 18 | 10.8% | 5 | 2.9% | 9 | 1.8% | 2 |
| 20 | WAS | -7.3% | 2-2 | -8.6% | 1.8 | 21 | 7.7% | 8 | 3.4% | 7 | 3.5% | 5 |
| 21 | MIA | -11.0% | 2-2 | -6.6% | 1.8 | 19 | 1.5% | 12 | 1.1% | 14 | 10.6% | 15 |
| 22 | STL | -11.7% | 2-2 | 2.4% | 1.7 | 22 | -21.3% | 32 | -7.8% | 29 | 5.1% | 6 |
| 23 | DEN | -15.6% | 2-2 | -8.0% | 1.4 | 25 | 3.5% | 10 | -0.7% | 17 | 3.4% | 4 |
| 24 | DET | -17.4% | 0-4 | -18.7% | 1.4 | 26 | 1.8% | 11 | 1.1% | 15 | 1.8% | 1 |
| 25 | MIN | -18.9% | 1-2 | -14.0% | 1.5 | 24 | -9.4% | 27 | 1.5% | 11 | 5.2% | 9 |
| 26 | JAC | -20.3% | 2-2 | -23.6% | 1.6 | 23 | 11.2% | 4 | 1.4% | 13 | 39.1% | 28 |
| 27 | CHI | -21.9% | 3-1 | -18.4% | 1.1 | 28 | 10.7% | 6 | -2.3% | 21 | 49.1% | 32 |
| 28 | SF | -31.7% | 0-4 | -33.8% | 1.3 | 27 | 14.7% | 2 | -10.4% | 30 | 20.1% | 25 |
| 29 | CAR | -35.5% | 0-4 | -40.7% | 0.8 | 31 | 3.9% | 9 | -3.5% | 24 | 5.1% | 7 |
| 30 | BUF | -40.4% | 0-4 | -47.4% | 0.5 | 32 | 20.5% | 1 | 3.3% | 8 | 8.1% | 11 |
| 31 | OAK | -41.4% | 1-3 | -34.3% | 0.8 | 30 | -10.4% | 28 | 6.0% | 5 | 11.5% | 17 |
| 32 | ARI | -45.3% | 2-2 | -50.2% | 0.9 | 29 | -2.7% | 15 | -6.4% | 27 | 44.5% | 30 |
Comments
153 comments, Last at 09 Oct 2010, 11:39pm
#1 by Led // Oct 05, 2010 - 5:15pm
"Last night's big win over Miami vaults the Pats over Kansas City, Green Bay, and the New York Jets and into the top spot in our DVOA ratings."
That's a complete post hoc ergo propter hoc, no? The Miami game didn't "vault" them into first place. Special teams plays (of which only the TD return is measured by DVOA) aside, NE didn't play a particularly impressive game. Not a bad game, just not a great game. Applying the adjustments to the raw VOA has to be what "vaulted" NE to the top.
#3 by Aaron Schatz // Oct 05, 2010 - 5:21pm
Nope. Last night the Pats had the third-best game of this week, higher then the teams below them, so last night's game is in fact what vaulted them ahead. When I ran the ratings before MNF, the Pats were sixth.
#8 by Led // Oct 05, 2010 - 5:29pm
I'll take your word for it but it doesn't make much sense based on my understanding of the way DVOA works. I must be misunderstanding. NE had lower YPA and YPC than the Dolphins and a worse 3rd down percentage. Does not seem very efficient on a per play basis. The INTs are important but usually INTs are not enough to make up for the fact that the other team was marching up and down the field. There's obviously a nuance to DVOA that I'm missing.
#18 by morrongiello // Oct 05, 2010 - 5:55pm
I'm not defending their formula but I didn't think 3rd down percentage was that important (?). Has anyone been able to show that teams can maintain it consistently on a game-by-game basis? I'm looking at the first four weeks of this season and some of the best offensive teams have flopped around like crazy (at least with respect to their offensive yards per play).
#42 by nat // Oct 05, 2010 - 8:59pm
Back in 2003, Aaron wrote a nifty article about the value of turnovers. The short version is that they are worth about the same as giving up or getting about 55 yards of field position.
It's quite common for people to underestimate the impact of turnovers. Remember how Miami was "marching up and down the field" seemingly at will during the first half, all the while accumulating high YPA and YPC? They scored seven points.
So, no, you're not missing a nuance. You are missing one of the most important points for understanding football: ball security matters enormously.
#49 by Led // Oct 05, 2010 - 9:47pm
Right, and the Pats had 6 points in the first half with a sputtering offense and not much production on offense in the second half, either, although they didn't have the ball much. Against the 20th ranked defense. So why doesn't their superhigh offensive DVOA take a bigger hit? And on defense, yes, turnovers are very important, but giving up chunks of yardage on most of the other plays matters too. That's why Chad Henne had positive DYAR for the day, despite 3 picks. And that was just passing. Ricky Williams was the third most valuable running back. (In fairness, the Pats did do a great job on Ronnie Brown.) That doesn't scream out as dominating performance to me. Remember, the issue isn't whether the Pats played well overall. It's whether they dominated enough to cause a major increase in their already very high DVOA. Obviously, they did because that's what the numbers say, but I just don't see it.
#54 by Scott C // Oct 05, 2010 - 10:10pm
I agree with Led.
The big plays that turned it into a blowout were mostly not counted by DVOA. What is left did not look 'dominating' though it did look 'very solid'.
On the other hand DVOA values third down conversions (or more generally, first downs) VERY highly, and rightfully so. The research shows that although great third down success = great DVOA, it is not particularly predictive DVOA, especially season to season.
#76 by Kulko // Oct 06, 2010 - 2:05am
LED wrote: "Right, and the Pats had 6 points in the first half with a sputtering offense and not much production on offense in the second half, either, although they didn't have the ball much. Against the 20th ranked defense. So why doesn't their superhigh offensive DVOA take a bigger hit?"
A) They were marching up and down the field in the first half too, just ending in field goals. Miami appeared to play superior in the first half because they really carved up the NE Def, and because their was the feeling, that the field goals would not be enough, when the Def gives up TDs.
They were marchign up and down the field in the second half whenever they had the ball, evene converting first downs with the running game, when they were cleraly running out the clock.
B) They had fewer offensive snaps than in their first three games, and DVOA is an per play average, not per game. So the miami game values disproportionally low for their offensive DVOA when compared to common perception which is a per half average.
#85 by nat // Oct 06, 2010 - 8:18am
Do you legitimately want to learn, or are you just whining? If you're just whining, you should use the complaint format to save us all time.
DVOA is a per-play measure. When a team plays well for a sustained 16 play or 13 play drive, DVOA doesn't retroactively reduce the value of the first 15 or 12 plays because the last play was a chip-shot field goal. So, rather than saying "sputtering offense" in the first half say "offense with two long sustained drives out of four possessions". Not great, certainly, but not bad at all - on a per play basis.
As for the relative value of turnovers and yardage, you should really read the article I linked. It's not a matter of taste or at this point even much argument: An average turnover is worth about 55 yards of field position. Chad Henne achieved a value slightly higher than a replacement QB - a positive DYAR as you say. He did it by having decent YPA paired with 3 interceptions.
If you want to ignore special teams (why would you?) the Patriots' offense beat the Dolphins' offense 20-7, despite having only 7 drives (ignoring the kneel down drive) to the Dolphins' 10 drives.
So how did this excellent game result in an improved DVOA for the season? Primarily, it was the improved defensive play (average instead of awful) and the dominant special teams (even ignoring the blocked kick and punt, as DVOA must, the special teams were tremendous). The offense maintained its high per-play effectiveness, but that was already high so it wasn't the reason for the DVOA jump.
#118 by Led // Oct 06, 2010 - 8:43pm
What I'd really want is for you not be a condescending jerk. My comments have been perfectly reasonable and respectful, and not even close to whining. I have no dog in the race. (I even said I'm not surpised NE has the hightest DVOA, just that the Dolphins game -- rather than adjustments kicking in -- is what did it.) If you're not prepared to discuss things without being insulting, why post at all? Maybe you need to ask yourself why you are so irritated about a good natured discussion. And who is the "us" that you're purporting to speak for?
On the merits, and putting aside the ugliness, T-Ball makes some good points below, particularly with respect to the value of the KO return and all the touchbacks. Gotskowski was, indeed, a monster. Five points of ST DVOA goes a long way to explain the increase. And I confess I was probably discounting/ignoring the Thigpen plays in my assessment. Beating up on Tyler Thigpen isn't all that impressive, but the DVOA formula doesn't know that. As for the offense, it's still hard to see a performance of 6.1 YPA, 3.7 YPC, and 45% on 3rd downs against the 20th ranked defense (by DVOA) as anything more than mediocre.
#121 by nat // Oct 06, 2010 - 11:50pm
My apologies. The complaint format is an old FO tradition. People even use it when they have reasoned arguments. I thought you knew.
Did you read that article, though? Turnovers are really, really valuable. The article I linked explains why better than I could here. It's a classic, and ought to be required reading.
You keep repeating that the Dolphins had good YPA and YPC. I agree. The Patriots achieved a better than average defensive result by getting turnovers (3 against Henne, so it's not all or even mostly Thigpen), not by stopping the Dolphins from advancing the ball. It's a legitimate way of being good on defense, although I wonder whether it can be sustained. DVOA considers interceptions predictive of future performance, so perhaps it can.
Keep in mind that the Patriots D had been so bad in three games that merely getting a moderately better than average result (about -8% DVOA, I guess) is still a huge improvement, and enough in combination with their huge special teams play to move their total DVOA upwards.
I hear what you're saying about the offense, but you're forgetting turnovers again. Teams average about an interception per game. By avoiding that interception, the Patriots get about the same value as an extra 55 yards or so of passing. It's as if they were getting 8.4 YPA instead of 6.1. Their effectiveness shows up in their drive stats, too. 20 points on 7 drives (kneel down drives are ignored for this calculation) is 2.86 points per drive, which would lead the league if it could be sustained over a season. The drive success rate (% of sets of downs that lead to a first down or a TD) would lead the league, too. That's not mediocre at all, unless you think the entire league is bad on offense.
The take-away lessons are that turnovers matter a lot, YPA or YPC matters less than you might think, and sustaining drives matters more than you might think.
#86 by RichC (not verified) // Oct 06, 2010 - 8:29am
The Patriots had 4 drives in the 2nd half.
2 of them were touchdowns, one was a 3 and out, and the last one ran out the final 4.5 minutes of the game. Thats pretty damn efficient football.
#106 by TBall (not verified) // Oct 06, 2010 - 1:44pm
Led,
I'd profess to know less about DVOA than you, but I don't think New England's DVOA should be that surprising. The kickoff return for a touchdown still counts and Gostkowski was putting every kickoff in the endzone. The special teams plays that did count helped improve the Pats by about 5 points in ST over week 3. The offense's DVOA fell back 5 points from week 3 (47 to 42) and the defense improved by 7.
It may feel like the Dolphins were able to move up and down the field with abandon (gaining 275 yards on their first 36 plays) but after the Pat's last offensive TD (third TD of the 3rd quarter), the Dolphins managed 125 yards on 31 plays, which included a turnover on downs and a pick 6. It doesn't seem as important because much of it seemed to occur as the game was out of reach. Thigpen's 2/6 for 15 yards and 1 pick brought down the overall level of QB play for the Dolphins.
Would you rather have 265 yards on 59 plays with 0 TO's or 400 yards on 67 plays with 4 TO's?
#2 by >implying impl… (not verified) // Oct 05, 2010 - 5:17pm
Best 2-win team: Chargers
Best 1-win team: Cowboys
Best no-win team: Lions
Worst 2-win team: Cardinals
Worst 3-win team: Bears
#23 by AudacityOfHoops // Oct 05, 2010 - 6:13pm
Sticking with the theme, I converted Estimated Wins to an Estimated Win% (EW/4), then subtracted that from actual win%, to get a list of which teams are most over-/under-performing their Forest Index. Most of it is obvious by glancing at the DVOA ratings chart("I see 3-1 Chicago is listed near the bottom"), but I like to see it quantified anyway:
W%DIF TEAM
0.475 CHI
0.275 ARI
0.225 KC
0.225 NO
0.200 HOU
0.175 ATL
0.150 BAL
0.150 DEN
0.100 JAC
0.075 STL
0.075 PIT
0.067 TB
0.050 SEA
0.050 WAS
0.050 MIA
0.050 OAK
0.000 NYJ
0.000 GB
0.000 CIN
-0.025 NE
-0.042 MIN
-0.075 NYG
-0.125 IND
-0.125 TEN
-0.125 BUF
-0.150 PHI
-0.175 SD
-0.200 CAR
-0.250 CLE
-0.267 DAL
-0.325 SF
-0.350 DET
#120 by Jonadan // Oct 06, 2010 - 10:55pm
I know you're just going off DVOA, but I take issue with this, and I'd extend it a little. My own theory says it looks like this:
Best 2-win team: Chargers
Best 1-win team: Cowboys
Best 0-win team: Lions
Worst 3-win team: Texans
Worst 2-win team: Cardinals
Worst 1-win team: Raiders
I'm not convinced the Texans can really get it together if they keep giving up a bazillion points, and unlike the Pats (now) they haven't shut anybody down. "Worst 1-win team" is an unfortunate stat, but it amuses me.
#4 by Big Johnson // Oct 05, 2010 - 5:24pm
best defense according to dvoa is the chargers! wow that brings a smile to my face. i am pretty sure they wont end the year with the best defense but it seems pretty safe to say they arent anywhere near as bad as they were the last couple years. Someone on here also stated the chargers put 5 (or was it 6?) new players on special teams coverage this last week. If they can correct that at all then they just might be the best team in the league.
#32 by Mr Shush // Oct 05, 2010 - 7:53pm
Hmm. I think it mostly means that opponent adjustments remain highly inaccurate and are very low strength, and the Chargers have played half their schedule to date against the lousy Seahawks offense and the historically abysmal Cardinals offense, complete with undrafted rookie quarterback.
The best team in the league are the Steelers. I don't like it. I don't like them. But them's the breaks.
#5 by Neoplatonist B… (not verified) // Oct 05, 2010 - 5:25pm
Wow. San Diego is ranked 5th, with a -20% Special Teams DVOA. That's crazy.
#7 by Pass to Set Up… // Oct 05, 2010 - 5:28pm
Packers were ranked high last year, despite horrible special teams. I think Special Teams only account for 1/7th of a team's overall DVOA.
#17 by Scott C // Oct 05, 2010 - 5:50pm
The average value of DVOA for special teams is roughly 1/7 of the total. But that is already accounted for in the percentages. A -20% ST DVOA means it was so bad so far that it wiped out their offensive value so far.
Look at the above. SD has this line:
26.0% 15 13.4% 10 2-2 20.8% 7 -26.0% 1 -20.7%
26% = 20.8 -(-26.0) -20.7%.
So if their ST DVOA drops over time to -5%, the O stays at 20%, and the D moves down to -20%, then their overall DVOA will be 35%.
#39 by Chrisss (not verified) // Oct 05, 2010 - 8:38pm
The problem here is that while special teams may only count as 1/7th of the DVOA, special teams have cost the Chargers 2 games already this year. Special teams could derail SD regardless of their DVOA. That said, if they clean it up, they could be the best team in the league. Most yards per game, fewest allowed. They would be downright dangerous. Time will tell if they can clean that mess up.
#56 by Scott C // Oct 05, 2010 - 10:14pm
yes, a season long -20% ST DVOA would be devastating.
Not happening though.
#6 by n8 (not verified) // Oct 05, 2010 - 5:25pm
Packers - Bonamego back?
#9 by ST-less in San Diego (not verified) // Oct 05, 2010 - 5:30pm
So if San Diego could improve their league worst Special Teams DVOA to the level of the 2nd worst Special Teams DVOA, they might have the highest total DVOA. Come on Chargers Special Teams, rise from abhorrence to mere wretchedness!
#10 by Wonkothesane1 // Oct 05, 2010 - 5:33pm
I like the estimated wins of 3.1 for Kansas City. Nice touch.
#66 by Tundrapaddy (not verified) // Oct 05, 2010 - 11:10pm
'ESTIMATED WINS uses a statistic known as "Forest Index" that emphasizes consistency as well as DVOA in the most important specific situations: red zone defense, first quarter offense, and performance in the second half when the score is close. It then projects a number of wins adjusted to a league-average schedule and a league-average rate of recovering fumbles. Teams that have had their bye week are projected as if they had played one game per week.'
#11 by Led // Oct 05, 2010 - 5:34pm
It's like someone chopped a piece of the Colts off and the fragment regenerated itself into the Texans.
#33 by Mr Shush // Oct 05, 2010 - 7:58pm
Except that the Texans run arguably better than they pass, and stop the run parsecs better than they stop the pass. It's more as if Ian McDiarmid ordered a clone army of 2004 Chiefs, and they've just come ready for delivery.
#12 by B // Oct 05, 2010 - 5:34pm
Patriots the best team in the league? All right, I'll take it. Big "so far" asterisk? I'll just ignore that part.
#13 by jebmak // Oct 05, 2010 - 5:37pm
I don't get why you sort by DVOA instead of DAVE if DAVE is supposed to be more accurate.
#21 by BucNasty // Oct 05, 2010 - 6:08pm
That was my feeling when it was just DAVE and VOA, but now that there's opponent adjustments (however watered down) I prefer to give DAVE the could shoulder and just go by DVOA.
#28 by JSA (not verified) // Oct 05, 2010 - 7:04pm
It would be great if these tables were sortable
#63 by Karl Cuba // Oct 05, 2010 - 11:02pm
Agreed
#91 by billsfan // Oct 06, 2010 - 9:34am
Also agreed. But the FO web interface has always been buggy for me (for example, there's a big red error box that appears on the top of the page after I post a comment), and if I really want stuff sorted, I'll just import it into Excel.
(I also like the Eagles)
#14 by Will Allen // Oct 05, 2010 - 5:41pm
Now that the opponent adjustments are kicking in, the Vikings defensive rank is improving, as I expected, and the offensive rank seems about right. The schedule is brutal for the next four games, and unfortunately a split may be the worst they can survive. Then again, nobody else in the division looks all that great (the Packers may end up being the 4th best team in the league, but I don't think they are close to that now), so maybe the Vikings could survive a 2-5 start. However, that seems unlikely, to say the least.
#15 by Scott C // Oct 05, 2010 - 5:44pm
After replacing 50% of their coverage teams, I'm pretty sure the Chargers won't be playing at -20% ST the rest of the way. Its only one game, but the coverage units actually stayed in their lanes on Sunday and looked like professionals.
The ~20% DVOA offense is expected, and watching the D things are definitely more solid there than the last few years on the D-line and secondary. Injury issues at LB though. The pre-season projection for the Chargers D is looking to be off.
With all the talk of 'no dominant teams' so far, it would not surprise me if 4 games from now we're looking at the AFC with at least 3 of:
SD, BAL, NE, PIT, IND
looking very strong. IND has a tougher schedule than the others though.
There's going to be a lot of movement up and down these rankings in the next couple weeks, though that is usual. The median variance this year is 1% higher than last year but 2% lower than 2008.
#16 by Rivers McCown // Oct 05, 2010 - 5:49pm
Houston is clearly ranked too low because Matt Schaub will never get hurt again just to spite FO. Calling my uncle up and asking him to make power rankings by which teams have the funniest names is way better than this. Lol u guys r dummmmb, Trolololol Houston wins it all.
I always wanted to do that.
#19 by Chris84 (not verified) // Oct 05, 2010 - 5:57pm
San Diego has a better defense than the Steelers? Um, what???
#25 by Sjt (not verified) // Oct 05, 2010 - 6:50pm
San Diego's defense has allowed only 10.75 points a game, 4 TDs and 5 FGS in 4 games.
Four of the FGs followed turnovers which gave the opposition the ball in Chargers territory. The other was a 50 yarder at the end of a 31 point beatdown of the Cardinals, who otherwise would have been shutout by the defense (their TD was a fumble recovery).
Of the 4 TDs, one was a garbage time TD at the end of the Jags game. Two others were the result of drives which started in Charger territory after the Offense fumbled. The only legitimate long drive they have given up all year was one big run by Jamaal Charles.
Not that it means anything for DVOA purposes, but they are also # 1 in the league in total yardage allowed, # 7 in rushing defense, #3 in passing defense, and have the lowest opponent passer rating. They are within 1 sack and 1 int of being the league leader in both categories. And they also have a safety and a defensive TD.
Its true, all of these thing happened against some not very good teams (except, apparently, the Chiefs). The Charger D has nowhere to go but down, since they will soon play tougher teams and they are dealing with injury and suspension issues. But yes, they have been damn good so far.
#36 by Mr Shush // Oct 05, 2010 - 8:05pm
"Not very good" is, um, a polite way of putting it. KC may be a good team (I still have my doubts) but they're certainly not a good offense, especially not in torrential rain. The Jags offense may be as good as ok. May be. The Seahawks offense is dismal. The Cardinals offense defies description.
I'm not saying the Chargers have a bad defense. They very probably don't. I'm just saying that if I went out there with 9 tackling dummies and an inflatable sheep, the Cardinals might struggle to put a long drive together.
#41 by Sjt (not verified) // Oct 05, 2010 - 8:53pm
And DVOA reflects that. If they had played as well as they have against the Pats or someone like that their defensive DVOA would be even better. That's the whole point of this exercise.
#48 by RickD // Oct 05, 2010 - 9:42pm
What people are saying is that the "D" in DVOA is not playing much of a role yet in the DVOA rankings. San Diego clearly has the best VOA but they've yet to face an offense of a high caliber.
I suspect the Jets, Ravens, and Steelers all have better defenses than the Chargers. And that will eventually be reflected in the DVOA ratings.
#61 by maestro876 // Oct 05, 2010 - 10:39pm
I think you're right. But at the same time, I think there's a decent chance that the Chargers' defense is better than the projections gave it credit for. Perhaps significantly so.
#67 by Big Johnson // Oct 05, 2010 - 11:25pm
im gonna agree with the skeptics on this one. The game against KC was ferocious towards the quarterbacks. Rivers had his worst game of the year and cassel was completely useless that game with about 60 yards total passing. So every game from here on out when they dont play in a hurricane it will make the chargers D look that much better because they hamstrung a decent offense. In the same right i think that game hurts their offensive DVOA and it is very likely they have a better offense than is indicated. SD offense should trend upwards while their defense should trend downwards. And i dont care who you are, a special teams ranking should never be -20+. it should never be -10+. The teams they have played have been all the best teams at special teams, definately contributing to the outrageous ranking. Seattle is first, jacksonville is second, KC is third, and arizona sucks at special teams (26th i think).
#20 by ammek // Oct 05, 2010 - 6:00pm
It is safe to say that the Giants' defense has been inconsistent.
After four weeks, has any team had a worse rushing DVOA than the 2010 Broncos? The huge negative number next to DEN looks completely weird, after all those years of success.
#94 by Mark S. (not verified) // Oct 06, 2010 - 10:57am
Actually, I think the giants' D has looked pretty good so far. Obviously they dominated the Bears. Tennessee did next to nothing until garbage time (lost that game on penalties/field position/ST), and aside from one drive I don't remember Carolina doing much either, despite having great field position all game.
Obviously, they got absolutely smoke in the Colts game, but I put that down to a deeply stupid gameplan (having one LB and 6 DBs on the field basically all first half) that blew up in their faces like an Elmer Fudd cigar. Other than that game, I think they've looked solid, especially when you consider they give up the best starting field position to opponents in the league (Kickoff and punt units are awful).
#22 by ammek // Oct 05, 2010 - 6:11pm
So the Broncos win, and their playoff odds get worse.
Josh McDaniels, probability simulations hate you.
#26 by Treima6 (not verified) // Oct 05, 2010 - 6:50pm
That's likely because the Chargers were making the Cardinals look like a Pop Warner squad on both sides of the ball Sunday, resulting in a MUCH brighter outlook for their playoff odds, and of course Kansas City is still 3-0 and the division leaders. Their win was relatively unimpressive and wasn't against a division foe, so it would not have affected their playoff odds much either way, but the possibility of the AFC West snatching both Wild Cards is extremely thin, so Denver's gotta be successful and hope the wheels fall off of Antonio Gates and, of all people, Mike Tolbert(!)
#37 by Arson55 // Oct 05, 2010 - 8:10pm
You mean the Cardinals aren't a Pop Warner squad? Weird.
#123 by JIPanick // Oct 07, 2010 - 12:54am
They are a qbs.pop(Warner); squad.
#124 by Thomas_beardown // Oct 07, 2010 - 3:14am
I found this way more amusing than I probably should have.
#24 by tunesmith // Oct 05, 2010 - 6:44pm
Surprised to see the Broncos' defense ranked worse than their special teams. The defense has put up some pretty good performances this year, I thought.
#104 by MarkV // Oct 06, 2010 - 12:46pm
I am surprised their special teams has people below them.
But, the Defense ranking that low is a bit surprised. The defense played ok in 3 games i would have thought.
#27 by battlered90 (not verified) // Oct 05, 2010 - 7:03pm
Looking forward to seeing the Seahawks plummet down these rankings. I'm guessing they are being buoyed by the number 1 special teams ranking. I wouldn't count on two kick return tds every four weeks.
#29 by Theo // Oct 05, 2010 - 7:04pm
Pittsburgh is that good? Wow.
I read PFP, but wow.
#30 by South Bay Bengal (not verified) // Oct 05, 2010 - 7:13pm
wish i could argue with the DVOA this week, but the Bengals suck. dammit.
#31 by Lou // Oct 05, 2010 - 7:47pm
I can't disagree with the rankings. It does seem odd that the Bears are ranked below all 3 of the teams they beat.
#35 by Kal // Oct 05, 2010 - 8:00pm
Odd, but fair given the games. The Lions should have beaten them but for a rule that was bad, the Cowboys outplayed them but had some odd luck, and the Packers beat themselves with penalties (which don't show up on DVOA).
This was the whole point of DVOA - that a team that wins is not necessarily showing that they're a better team over the team that loses, just that they were better that day.
That being said, it's incredibly depressing for this Bears fan to think about. I was really hoping that with Martz, Cutler was going to turn things around. Instead I can watch Kyle Orton ripping defenses apart with Brandon Lloyd as his receiver and wonder where the hell things went so wrong.
#38 by Thomas_beardown // Oct 05, 2010 - 8:21pm
You might want to refresh your memory of what DVOA looked like after week 1.
http://www.footballoutsiders.com/dvoa-ratings/2010/week-1-dvoa-ratings
The Bears outplayed the Lions by a significant margin, and were extremely unlucky in where they fumbled.
Also, it's easy to see where things went wrong. They have no talent on the offensive line, and no depth behind what is already a backup quality starting unit.
#34 by The Hypno-Toad // Oct 05, 2010 - 7:59pm
It's interesting... I have been very, very skeptical of the local Bronco fans who have assserted that the Broncos defense has "looked great" but I also find it hard to believe that there are only 4 defenses worse than the Broncos this season. So once again I find myself a man between worlds.
#70 by PerlStalker // Oct 06, 2010 - 12:28am
I wouldn't go as far as "great" but better than last is pretty accurate. The thing is, as bad as the defense was last year, even 20-25 ranking looks great by comparison. I also think their defensive DVOA will look better as opponent adjustments are more heavily weighted.
#40 by thok // Oct 05, 2010 - 8:51pm
I'm disappointed the 49ers rose in the rankings this week. I was hoping they'd still be last and you'd make a special hell category for them as per last week.
Although I guess hell is other people's turnovers, and there's a limit to how much you can punish them for that.
#43 by chemical burn // Oct 05, 2010 - 9:02pm
It's strange I feel like there are really no good teams this year so everyone is surprised that their favorite team ranks so high. I for one cannot believe the Eagles rank 7th. This team blows. Certainly, they look to be the most inept Eagles team in a decade. The defense is really pitiful - but I guess only for chunks of each game and not the entire time. Still, they are average in DVOA? And their offense, on that has looked helpless (not bad, just helpless) for at least 1 complete game ranks 6th? I feel like 2010 is going to produce one of the worst Superbowl winners in history. And that it could be literally any team at this point. If we can't truly rule out teams like the Lions and Cardinals at this point, anything seems possible.
#45 by JIPanick // Oct 05, 2010 - 9:28pm
A good but not great champion from a blurry year wouldn't really be that bad as recent champions go, in my opinion. Worse is when there is a great team (or two or four or whatever) and some weak team sneaks into the playoffs and steals the title from them.
Look at 2007; there were four legitimately great teams in New England, Dallas, Indy, and Green Bay but instead the Super Bowl rings went to an average Giants team.
Other examples include 2001 (loaded Rams, indifferent Pats) and 1987 (Niners and Skins). Arizona almost pulled it off two years ago too.
#47 by chemical burn // Oct 05, 2010 - 9:35pm
I actually mean there's real chance of not just an average team (like the 2001 Pats or 2007 Giants) taking the title, but a real shitbomb of a team sneaking into the playoffs and then making a run at the title because there's only mediocrity standing in their way.
#51 by Mr Shush // Oct 05, 2010 - 9:56pm
Someone presumably has to win the NFC West, right? I mean, I know the league should take that division's playoff berth away, but I don't think they have.
I seem to remember a PFR simulation a while back which suggested that about one time in 10,000 the Superbowl would be won be the actual out-and-out worst team in the league.
#52 by Will Allen // Oct 05, 2010 - 10:01pm
Well, "average" is a hard word to use in regard to something with as many moving parts as an NFL team. I might use "weird" for the 2007 Giants; mediocre in some, maybe most, areas, but joined to a defensive front which, come playoff time. was primed to just stomp the feces out any primates put in their path. Kind of like some World Series winners who fail to get 90 wins in the regular season, but because of two great starters, a relief man, and three or four hitters, are able to prevail over hundred win teams in the playoffs. Maybe "average" is correct in the technical sense, but the word may imply more luck than is actually the case.
#55 by chemical burn // Oct 05, 2010 - 10:13pm
Instead of average, how about just "not terrible?" That's all I meant. And that this year a "terrible" team could win it. Like if the Rams took the NFC West and then beat the Cowboys and (I don't know, Vikings?) to go to the big game. And then all they had to beat was the Jets or Chiefs or Pats something? None of that scenario sounds like "no freakin' way!" Or if the Seahawks or Chiefs or Jaguars did the same thing?
#65 by Will Allen // Oct 05, 2010 - 11:08pm
Well, predictions are hard, especially about the future, but I can't see any unit of the Rams, if they were to win that terrible division, being anywhere near as good as the Giants defensive front was, so if the Giants hoisting the trophy was strange, the Rams doing so would be like hitting the lotto.
#75 by t.d. // Oct 06, 2010 - 1:50am
That Giants team excelled at rushing the passer, and they were adequate throwing the ball. The 2008 Cards excelled throwing the ball, and had a decent pass rush. Those are the two most important skills in the modern NFL, and they can cover a lot of warts.
#97 by Mark S. (not verified) // Oct 06, 2010 - 11:25am
Somebody on these boards made a really interesting point after the Steelers, Colts and Giants won SuperBowls 40-42 although now that I think about it, they may have been talking about the 05-06 Steelers team that won as the sixth seed in the playoffs).
Anyway their point was that while these teams were not great in the years they won those SBs, they did have dominant seasons in preceding or successive years - i.e. the Giants were only "OK" the year they won the SB, and much better the following year (they were 11-1 and first in DVOA before Burress shot himself) and the Colts were only "pretty good" in 06, the year they went on to win it, although they were dominant the previous year; same for the Steelers, etc. The crux of the point being that there was a mini-trend of teams that might be really good or dominant in a given year, but their statistical dominance did not correspond to the years they actually won titles.
#44 by Vincent Verhei // Oct 05, 2010 - 9:23pm
Fodder for the "No great teams" discussion:
In Week 4 2010, there are three teams over 30% and the best team (New England) is at 34.1%
In Week 4 2009, there were seven teams over 30%, three over 40%, and the best team (New Orleans) was at 58.5%.
In Week 4 2008, there were four teams over 30%, two over 40%, and the best team (Baltimore) was at 60.6%
In Week 4 2007, there were five teams over 30%, three over 40%, and the best team (New England) was at 72.4%
In Week 4 2006, there were five teams over 30%, four teams over 40%, and the best team (San Diego) was at an ungodly 83.8%.
In Week 4 2005, there were six teams over 30%, five teams over 40%, and the best team (Cincinnati) was an even more ungodly 92.0%.
That's as far back as the weekly DVOAs go, but it's clear that the cream of this year's crop is decidedly less creamy than usual.
#46 by chemical burn // Oct 05, 2010 - 9:34pm
Also, one of those 3 over 30% this year (KC) feels like it almost certainly going to crash and burn at some point - or maybe just get hammered by opponent adjustments.
#50 by RickD // Oct 05, 2010 - 9:49pm
I suspect it'll be the latter. They've got a schedule filled with teams from the AFC West, NFC West, AFC South, and the other last place AFC teams (Browns and Bills).
While they probably won't leave Indy undefeated, they should get to 8 wins relatively easily. (Jags, Bills, Cardinals and Raiders all visit KC - that alone should get them to 7 wins.)
With the Rams also in first place, the people of Missouri must be horribly confused.
#53 by Mr Shush // Oct 05, 2010 - 10:06pm
I think it might be a schedule artifact, actually, coupled with the fact that the best team in the league has played its first four games with its third and fourth string quarterbacks under centre, but will probably not have to do so again, while the second or third best team in the league has missed a load of time from its best player, who will also presumably be back. There just seem to have been an unusually high proportion of games between two good teams and games between two bad teams. The Jets have played the Ravens, Patriots and Dolphins. The Steelers have played the Ravens and Falcons. The Ravens have played the Jets and Steelers. There's not enough connectivity yet for that sort of thing to smoothe out. I think as time goes by we'll see something like Steelers otherworldly, Ravens and Jets elite, Falcons, Patriots, Packers, possibly Saints and maybe Cowboys if their offensive line stays healthy very, very good.
#57 by chemical burn // Oct 05, 2010 - 10:15pm
You're being way too abstract in the beginning there. I guess the "best" team is the Steelers - but what on earth is the 2nd or 3rd best team you're referencing? And the "best player" in question?
#64 by tomvons (not verified) // Oct 05, 2010 - 11:02pm
I'd guess the 2nd/3rd is the Ravens missing Ed Reed...?
#71 by Kurt // Oct 06, 2010 - 12:32am
Has to be Revis.
#80 by Mr Shush // Oct 06, 2010 - 7:06am
I meant Revis, yes.
Although of course Reed being out for the Ravens is a huge deal too. I'm just not sure he's obviously more important than Ngata or Suggs, in the way that Revis is obviously more important than . . . Harris? Ferguson?
#72 by Kurt // Oct 06, 2010 - 12:35am
(double post)
#59 by Led // Oct 05, 2010 - 10:20pm
You left out the Chargers. For my money, the best teams are, in no particular order, Steelers, Ravens, Jets, Pats, Chargers, Colts. With the Texans, Saints, Packers, Falcons in the next tier. This changes if the Pats actually trade Moss for picks this year, as is being reported in some outlets. I think the Pats take a big hit to their offensive productivity without Moss on the field changing the way teams play defense.
#81 by Mr Shush // Oct 06, 2010 - 7:30am
You're right, the Chargers may well be pretty good. Ok.
Tier 1: Steelers (with Roethlisberger)
Tier 2: Ravens, Jets
Tier 3: Patriots, Chargers, Falcons, Packers, maybe Saints, maybe Cowboys
Tier 4: Texans, Colts, Dolphins, Giants, Bears if Cutler survives, maybe Titans, maybe Chiefs
Tier 5: Jaguars, Broncos, Eagles, Redskins, Vikings
Tier 6: Bengals, Browns, Buccaneers, Rams, Lions, maybe 49ers (I have no idea what the hell is up with that bunch of schizoids - they could be higher or lower)
Tier 7: Bills, Seahawks, Panthers
Tier 8: Raiders
Tier 9: Cardinals
I think that's about how I see it. I'm bound to be wrong about a fair amount of that, of course, but that's my current best guess. Where I disagree substantially with DVOA, I'm pretty sure it's mostly about the "D" being (inevitably) low-strength and inaccurate this early in the season.
#153 by leviramsey (not verified) // Oct 09, 2010 - 11:39pm
Interesting point on the bad-vs.-bad and good-vs.-good matchups early.
When Bert Bell was scheduling the NFL, he would typically have the better teams play each other early and likewise with the worse teams, on the theory that it would leave the most teams in contention (and thus able to sell tickets) midway through the season.
Perhaps with this being an uncapped year, Goodell took a page from the Bell playbook?
#58 by Scott C // Oct 05, 2010 - 10:17pm
No dominating teams _so far_.
By week 8 it might look not too different than half the years above.
#60 by chemical burn // Oct 05, 2010 - 10:29pm
Not in terms of DVOA - teams only end up with 70% or 90% DVOA because of a small sample size and weak opponent adjustments. I can safely say no team is going to look "ungodly" or even dominant this year - because no team has played dominant for a quarter of the season. Some teams have played pretty good, but nothing above and beyond that...
#62 by JIPanick // Oct 05, 2010 - 11:00pm
There's certainly a possibility that a team off to a decent but unspectacular start could put the pedal down and take off to a dominant season. I think the no longer QB-less Steelers are the best candidate.
#79 by ammek // Oct 06, 2010 - 4:34am
Surely you have weekly DVOAs from 2003-04, at least? I mean, this site was alive back then.
#68 by Treima6 (not verified) // Oct 05, 2010 - 11:33pm
The three worst runningbacks in DVOA terms are my three fantasy backs. http://fantasy.nfl.com/league/11171/team/1
/weeps
#69 by Treima6 (not verified) // Oct 05, 2010 - 11:33pm
Er, whoops. I mean in DYAR terms.
#73 by ebk95 (not verified) // Oct 06, 2010 - 12:39am
I'm curious as to what the Bears' offensive DVOA was for this week and how it compares to some of the worst offensive performances all time?
As a Bears fan I can remember some shockingly bad offensive performances (Thanksgiving 2004 comes to mind) but this past game was easily the worst :(
#74 by Independent George // Oct 06, 2010 - 1:10am
Can any Chargers fans explain how they have a worse Special Teams DVOA than both the Giants and the Dolphins? Are they punting the ball backwards into their own end zone?
#77 by Vincent Verhei // Oct 06, 2010 - 2:17am
See last week's DVOA ratings commentary. San Diego just gave up the best single-game special teams DVOA in history to Seattle, and in Week One the Chiefs averaged 32 yards per punt return with a 94-yard touchdown against the Chargers.
#110 by Ben B. // Oct 06, 2010 - 3:19pm
A chronicle of the woes:
Week 1 against the Chiefs: 3 touchbacks on punts from inside the KC 45 (my favorite was the touchback from the 33), a 32 yard punt return ending with the punter making a nice tackle, a 24 yard punt return, and a 94 yard TD punt return.
Week 2 against the Jaguars: A punt blocked (out of two total attempted).
Week 3 against the Seahawks: (covered by Vince) 2 kickoffs returned for TDs, another that might have been except Leon Washington slipped on his own. One 31 yard punt return. A kickoff return was fumbled.
Week 4 against the Cardinals: A missed field goal and a kickoff out of bounds.
#78 by Alternator // Oct 06, 2010 - 3:01am
New England is clearly ranked too high because no way are there nine worse defenses so far this year. Blind homerism is way better than this. Dear God have mercy and let the young guys mature fast.
#82 by PatsFan // Oct 06, 2010 - 7:53am
Looks like "Zero catches from Moss" will be true for NE every week from now on...
#84 by ChaosOnion (not verified) // Oct 06, 2010 - 8:16am
The average height of NE skill players is now hovering at about 5'9".
I had DeSean Jackson and Randy Moss in my fantasy league. Maclin was just sitting around on the waiver doing nothing but catching a TD every week, so I added him for some insurance. The strength of me team is now whatever defense is playing ARZ/CAR/BUF this week.
#83 by andyavatar (not verified) // Oct 06, 2010 - 7:57am
What is wrong with your combined score is that it combines distributions of unequal dimension. Your defensive DVOA percentages have a far different mean and standard deviation than your offensive DVOA. The same for special teams. The scatter pattern is also different. You can't in good conscience compare them straight up. At the very least, you ought to be using the standard deviation to combine these two percentages to get the deviation from the mean for the score. A simpler way is to adjust using a rule of thumb, i.e., defensive DVOA has one half the variance that offensive DVOA has, or something like that, but you're stat guys. You ought to be able to handle a second order metric.
As for combining the offensive and defensive percentages, you're all over the map. You don't report your method of combination, so one has to guess at the system you are using which, frankly, doesn't seem very systematic. Even if I subtract the def DVOA from the offensive DVOA, I don't get the scores or the rankings you report. I would think you would just convert the percentages to a deviation from the mean, then add them, or, you would adjust the defensive percentages to fall on a distribution determined by the mean and standard deviation of the offense's distribution. You could add in special teams (corrected) directly or weight it, depending upon your point of view.
#88 by DaveRichters // Oct 06, 2010 - 9:10am
You don't report your method of combination, so one has to guess at the system you are using which, frankly, doesn't seem very systematic. Even if I subtract the def DVOA from the offensive DVOA, I don't get the scores or the rankings you report.
The combination method is just additive, Offense minus Defense plus Special Teams. This does equal the total reported, perhaps you were not adding special teams when you checked this? Or perhaps you were using DAVE instead of DVOA as the total?
Your point is correct, I've made it here before, but it will fall of deaf ears I'm afraid. The authors of this model do not feel obliged to explain or justify anything about this model beyond an extremely superficial level, so critiquing it is pointless.
#93 by andyavatar (not verified) // Oct 06, 2010 - 10:41am
That has been my experience also. There is a serious flaw, for example, in DVOA in that it overvalues interceptions on behalf of defenses. Thus, last year, Green Bay was rated at or near the top, yet, they had given up 30 touchdowns through the air. Nearly two per game! The problem seemed to be interceptions, but nobody seemed willing to address the issue, or even seriously discuss it. the issue for them seemed to be my dissatisfaction. Their only remedy was to challenge me to prove it. I asked for the relevant stats, and they declined to provide them. So much for scientific neutrality and a willingness to question your own assumptions.
#100 by DGL // Oct 06, 2010 - 11:48am
IANAS (I Am Not Aaron Schatz), but he has discussed and posted about how he evaluates DVOA during the season and the offseason; based on that, I suspect he experimented with lowering the importance of defensive interceptions to the DVOA calculation, and found that on the whole it gave a worse (poorer correlation with wins) result.
#102 by RichC (not verified) // Oct 06, 2010 - 12:00pm
Interceptions are worth a lot in DVOA because they are worth a lot in football.
now, I agree that FO has been terrible with dealing with comments/criticizm of their lack of transparancy, but c'est la vie.
#90 by qed // Oct 06, 2010 - 9:23am
As for combining the offensive and defensive percentages, you're all over the map. You don't report your method of combination, so one has to guess at the system you are using which, frankly, doesn't seem very systematic.
Offensive DVOA, plus (-1 times Defensive DVOA), plus Special Teams DVOA produces the Total DVOA. Not sure if that was your question or not. If you aren't familiar with the way the stats are computed the primer is here:
http://www.footballoutsiders.com/info/methods
The 0% point is set by the long-term average results for each type of play, although I'm not sure what time period those long-term averages are calculated over. I don't see the problem with weekly or yearly offensive and defensive DVOA having different distributions, it's entirely possible that any particular year happens to be a good or bad year for offenses or defenses.
Since the idea of the total DVOA is to try to estimate the overall quality of the teams I don't think you want to normalize the offensive and defensive DVOA before combining them... if a team happens to be an extremely low defensive DVOA it should be because their defense is playing extremely well, and you wouldn't want to suppress that outlier just because their doesn't happen to be any team with a great offense DVOA.
#92 by andyavatar (not verified) // Oct 06, 2010 - 10:32am
"Since the idea of the total DVOA is to try to estimate the overall quality of the teams I don't think you want to normalize the offensive and defensive DVOA before combining them... if a team happens to be an extremely low defensive DVOA it should be because their defense is playing extremely well, and you wouldn't want to suppress that outlier just because their doesn't happen to be any team with a great offense DVOA."
The problem is that a variant that detracts from a total ought to be measured on the same scale as one that adds to it or you have unequal contributions. If offensive DVOA has a mean of 8 and a stddev of 18, while the defensive numbers have a mean of 0 and a stddev of 12, then one point of offense is worth much more than one point of defense because the defense must first offset the differences in the mean before they can offset the offense. Moreover, every percentage point of defense must offset more than one percentage point of offense since offense is freer to vary than defense. Therefore much of the true variance in the defensive number is lost reconciling the different means and stddevs. Is this statistically fair?
For example, with the Patriots, their 23rd rank on defense is less than one third as important as their 1st rank on offense. Does this reflect variance in the real world? Is a bottom ten defense only one third the contributor to failure as a top ten offense is toward success?
#95 by DeltaWhiskey // Oct 06, 2010 - 11:01am
Just push the frickin' "I believe" button.
#96 by qed // Oct 06, 2010 - 11:17am
I would argue that offensive and defensive DVOA are already measured on the same scale since they are both drawn from the same set of plays. You can argue that the fundamental formula doesn't accurately measure what its supposed to. But if you start from the assumption that DVOA is accurate then I think it's best to combine the raw DVOA scores. It should definitely be more accurate than, say, just adding the rankings in each category and saying that the team with the lowest combined ranking is the best. I wouldn't say that the Patriots 23rd ranked defense is "one-third as important" as their 1st ranked offense, I'd say that their offense is substantially better than their defense is bad. But if I want to compare the Patriots to the Chiefs, I'd rather compare the differences in their DVOA scores (Pats offense is about 30 points better, Chiefs defense is about 30 points better, so they cancel out) then their rankings (Chief's offense is 9 ranks below the Pats, Chief's defense is 18 ranks better, so the Chiefs are a substantially better team),
#98 by AudacityOfHoops // Oct 06, 2010 - 11:33am
"If offensive DVOA has a mean of 8 and a stddev of 18, while the defensive numbers have a mean of 0 and a stddev of 12, then one point of offense is worth much more than one point of defense because the defense must first offset the differences in the mean before they can offset the offense."
Maybe I am misunderstanding, but I don't see how this is true at all. How would you solve this "problem?" By subtracting 8 points from all the offensive DVOA's, right? If you do that, does anyone's rank change? No, because that 8 point adjustment is applied to all teams equally.
"Moreover, every percentage point of defense must offset more than one percentage point of offense since offense is freer to vary than defense. ... For example, with the Patriots, their 23rd rank on defense is less than one third as important as their 1st rank on offense. Does this reflect variance in the real world? Is a bottom ten defense only one third the contributor to failure as a top ten offense is toward success?"
I think that is an open question, but not a clear black mark against the method. Given the rule changes in recent years that seem to tilt the field toward the offense, it seems possible that great offenses can do some things that are nearly impossible to stop, making defensive quality less important. For an extreme example, consider free throws in basketball. Every team has a FT% (offense) and an oppFT% (defense). FT% varies more across teams than oppFT%, which makes sense, because oppFT% is mostly luck (getting to face bad FT-shooting teams) combined with tending to foul the right/wrong guys. You wouldn't want to normalize them when making a combined metric, because FT% really is more important.
That said, it's something that needs to be tested, and not just assumed to be true. And as you said, FO hasn't been very forthcoming in either making data available for testing, or writing articles about tests they've done.
#101 by Aaron Schatz // Oct 06, 2010 - 11:50am
If you feel like you have a way to improve the accuracy and quality of DVOA, or any of our other stats, we're very happy to listen to your ideas. Let me see if I can provide some guidelines that might help you get your point across. (This is a general comment, not addressed at any of the people who made criticisms in this specific thread.)
1) Send your comments in an e-mail rather than putting them in an article discussion thread.
2) Make your statement as a constructive suggestion rather than a snarky criticism.
3) Specifically direct your comment towards one FO writer rather than "the FO mass." If you want to talk about DVOA, ask for me. If you want to talk about injury data, ask for Barnwell. And so forth.
(For example, to "andyavatar," if you are complaining that "they wouldn't listen to me," I don't understand what you are talking about. Who is they? Me? Barnwell? Even from the open mailbag e-mail account, we tend to all separately sign our names to responses. Again: Not a hive mind.)
4) We've always been very open about the fact that nobody on the FO staff is a trained statistician. We are writers first, statisticians second. (Or, in the case of Mike Tanier and Doug Farrar, writers first and scouts second.) If you want to talk about statistical theory and how it relates to the numbers on this site, your comments should be very clear and basic.
5) Accept the fact that your suggestion may have already been made by a dozen people in the seven years this website has been online. Sometimes, there is a good reason why things are as they are, and it isn't because we do not like you personally.
6) Have patience. It is often hard to get to things in-season.
#103 by AudacityOfHoops // Oct 06, 2010 - 12:26pm
Thanks, I probably should have added "I'm not sure HOW to test it, exactly." to my previous post.
FYI, I just clicked the more info link from this article (above the charts), and I got an error page.
#105 by Aaron Schatz // Oct 06, 2010 - 1:22pm
Whoops. Typo. Fixed now.
#107 by nat // Oct 06, 2010 - 1:52pm
Send your comments in an e-mail rather than putting them in an article discussion thread.
Personally, I hope people do both. I would hate to miss out on some of the ideas people come up with. There's a lot of claptrap, true, but there are occassional gems.
#108 by DeltaWhiskey // Oct 06, 2010 - 2:21pm
Agreed, and I don't think Aaron means that quite of a rigid interpretation, what I think he's referring to, having been a perpetrator of this myself, is not beating the dead horse and expecting anyone in the hive to respond and listen. This sort of discussion is vital to FO's metrics, but eventually, if they're going to incorporate these ideas, they have to be communicated directly to a member of the hive.
#109 by DaveRichters // Oct 06, 2010 - 3:12pm
2) Make your statement as a constructive suggestion rather than a snarky criticism.
Don't be such a baby.
#111 by Dean // Oct 06, 2010 - 3:20pm
Well, do you actually want to affect positive change? Or do you just want to try and impress random strangers on the internet with how witty you believe you are?
#112 by DeltaWhiskey // Oct 06, 2010 - 3:52pm
Can't I do both?
#113 by billsfan // Oct 06, 2010 - 5:17pm
I've been trying to do the latter for the better part of 20 years.... why stop now?
(I also like the Eagles)
#114 by DaveRichters // Oct 06, 2010 - 5:27pm
Well, do you actually want to affect positive change? Or do you just want to try and impress random strangers on the internet with how witty you believe you are?
I guess I just want them to not be babies. I am a scientist, my work is under peer review, and I don't have the luxury of ignoring criticism when it doesn't come from someone trying to be especially nice to me. I would think that THEY want to make the model better, I don't really care about how good it is.
#115 by andyavatar (not verified) // Oct 06, 2010 - 6:37pm
Aaron,
You personally blew me off. I was trying to make a point to Barnwell, and I asked for data so I could demonstrate the issue and you interrupted the exchange and basically said the discussion is over. You never dealt with the problem I addressed and you declared the discussion at an end. I never got a response from Barnwell after that. This happened at the end of last season before the playoffs when I complained about the Packers being rated too highly on defense. What data I had available showed a significant bias. The weeks where the Packers had a monumentally high (or low) DVOA on defense were weeks where they had multiple interceptions. I asked for data from other teams rated highly on defense, but it never came.
As it turned out, the Packers #2 defense didn't exactly play like one in the playoffs. I'm sure you'll say that was anecdotal evidence.
I know its hard to respond to everything that is said when you are traveling between the bunker and Berchtesgarden, but you might want to occasionally acknowledge that some of your readers might know something about variance and central tendencies, and aren't interfering idiots. Even your tone in an email is imperious.
I wouldn't waste any time running after your reputation. It precedes you.
Andrew
#116 by AudacityOfHoops // Oct 06, 2010 - 7:08pm
It would be cool to see a couple alternate versions of DVOA that deal with turnovers differently:
A) defensive DVOA where all interceptions/fumbles are merely treated as incompletes/tackles - this would measure how good a team is at preventing the sticks from moving. A team like last year's Packers (apparently) wouldn't do as well here
B) I imagine one could model the probability of an interception, given the location of a pass on the field, down/distance, time, opposing quarterback, seconds elapsed prior to the throw, etc. Then you could find the expected number of interceptions for a defense, and compare that to the actual number. OK, I guess this isn't at all DVOA. But maybe this metric would have shown last year that GB wasn't super awesome at intercepting the ball, they just had more dangerous throws against them by bad QBs? It'd be interesting to look at.
C) along the same lines, maybe interceptions could be regressed to the mean somehow. Admittedly, I have no idea how this one would work.
#117 by Big Johnson // Oct 06, 2010 - 7:17pm
andy quit pms'ing. if you think u can put together better stats than the stuff they put on footballoutsiders then do it. They arent forced to put the research in their work. I had a thought when it came to new research with ALY and average yards gained by a runningback. I talked about it for some time and they decided to devote an article to it over the offseason. If they think you are onto something they will look into it, otherwise go make your own stats.
#119 by andyavatar (not verified) // Oct 06, 2010 - 9:01pm
I was just responding to Schatz, reprinting for the five hundredth time, his pat response to criticism. Thank God for cut and paste. If you don't want criticism, don't have a comments section and don't include a format for criticism in the body of your article. I, too, am a scientist and his haughty unwillingness to entertain authentic criticism is galling. We all like the site, and many of us would like to point out problems and inconsistencies to make it better. But, he has no interest in any but his own ideas, apparently.
#122 by Big Johnson // Oct 07, 2010 - 12:17am
Its unfortunate you feel that way. Ive seen him comment back to people before and ive seen him snub people before. Like i said earlier with the ALY to runningback yards coorelation the staff personally didnt respond to me. But either by reading it and thinking i was on to something or by stumbling on it themselves without reading my comments, they somehow came up with an article devoted to the same exact stat. If you are right with the interceptions overrating defensive dvoa then give more examples of who its overrating before it happens. More cases like the green bay one last year will make your theory stronger. Personally you have made this so well known that there is no way it wont be looked at. Take a deep breath, u have won.
#126 by Aaron Schatz // Oct 07, 2010 - 10:45am
I think I remember this conversation with Andy now. If I remember correctly -- and I may not -- my response was that I couldn't take the complaint seriously until he could come up with some examples from other franchises in other seasons. That's generally our response to most criticisms related to one specific team. We want to see evidence that the stats are systematically ignoring something over a period of multiple years with multiple franchises. "Green Bay played like crap in one playoff game in 2009" is simply not a reason to change the system so it is less predictive for all teams in the years 1993-2008.
Or maybe I just straight blew you off. If that's the case, I apologize. I try not to do that much. Sometimes, there's just too much of a time crunch.
(Note: I went into the mailbag and searched for Andy's e-mails, and this is exactly what happened. Honestly, I don't want to be that guy who blows people off for no reason. The exact quote to Andy was: "We're done with the back-and-forth here unless you want to talk about a team other than the 2009 Green Bay Packers. I'm sorry you feel we're being dismissive, but you have to understand how many of these complaints we get about single teams." I'm happy to consider the suggestions if he can think of two or three other teams with a similar issue.)
#127 by Revenge of the NURBS (not verified) // Oct 07, 2010 - 11:37am
FWIW, the 2005 Bengals had 31 interceptions, and finished 23rd in defensive DVOA. I'm not sure who's position that supports, but it's an example of another team that lived on turnovers.
#129 by andyavatar (not verified) // Oct 07, 2010 - 12:51pm
Aaron,
Thanks for the welcome candor.
I wanted to provide you with other examples but I didn't have the history of your ratings week by week except for the Packers, since that's all that Barnwell sent me. After I explained this to him, you popped in and cut off the conversation. I never got any data to analyze so, needless to say, I couldn't support the contention I was making. I just figured you had no real interest in seeing if your ratings were actually biased.
If you're interested in seeing if there is a problem, I'd be glad to help, but again, I don't have the data. If you want my help, I need to know what the weekly DVOA (all three categories) on defense was for each team for a season. I'm sure it would be easy to pick out candidates. For 2010, already Tennessee seems overrated on defense, but that's just a hunch.
As for the problem of unmatched means and stddevs that contribute to your rankings consistent with this article, that would seem like Statistics 101. You have unequal variances and unequal means and you're directly adding and subtracting points. It makes no sense. You might want to ask yourself how NE, the #1 offense and the #23 defense can possibly be the best team in the NFL. That #1 offense, by the way, only really scored thirteen points in that game against Miami and the week before it only put up 14 points against the Jets. Is that a #1 performance over the two most recent consecutive weeks? And yet, their rating went UP. In statistics class we used to call that obvious "by inspection."
Andrew
#130 by ammek // Oct 07, 2010 - 1:54pm
You might want to ask yourself how NE, the #1 offense and the #23 defense can possibly be the best team in the NFL. That #1 offense, by the way, only really scored thirteen points in that game against Miami and the week before it only put up 14 points against the Jets. Is that a #1 performance over the two most recent consecutive weeks?
Scientists operate from facts:
— The Pats' offense scored 20 points last week, not 13. No matter how short a drive may be, it still has to get from the starting point to the end zone. You can't completely dismiss a TD drive just because it's short.
— The Miami and Jets games were not consecutive. New England scored 38 points against Buffalo in the intervening week. It also put up 38 against the Bengals, which you omit to mention.
— Ranking isn't everything. As the commentary points out, the top-ranked teams of 2010 would not be top-ranked in some previous years. Why is it so hard to accept New England as the #1 offense? Even without the return TDs, they're in the top three in points per game. And as has been pointed out above, they're only averaging nine offensive drives a game — one per game fewer than Indianapolis and three per game fewer than San Diego (the other top scorers). DVOA measures efficiency, after all.
#132 by andyavatar (not verified) // Oct 07, 2010 - 2:38pm
I'm just cherry picking facts to underline the problem. I don't doubt that New England's offense is good. I've watched three of their games. Still, they're eleventh in yards overall. sixteenth in passing and thirteenth rushing by that standard. They have no running game to speak of, and their offensive line is average. There's no balance as the Jets game demonstrated. They also only put up six points against a mediocre Dolphins defense in the first half with the game in doubt, and before the Fins imploded. Add to that a clearly porous defense that, going into last week was giving up around 27 points a game and you have a statistic that has no basis in actual team performance.
If you want to define efficiency as getting the most out of your players, then the Patriots may indeed be #1. They're clearly not that talented so their 3-1 record must have some other explanation. Maybe that's what DVOA measures, after all. But, FO didn't proclaim them the least inefficient team in the NFL. They said "the Football Outsiders mainframe has proclaimed New England the best team in the NFL." Well, where's the objective verification? In drive stats they're fifth on offense, thirtieth on defense. That sounds about right. But, not by DVOA. Statistics have to reflect patterns in real life. Efficient teams won't beat themselves and, if other teams blunder, they'll win. However, without playmakers, there's no one to generate points. That, to me, is why they looked over matched against the Ravens in the playoffs last year, and against the Jets this year.
I started this by saying that the DVOA totals, inappropriately combine three unmatched distributions. I'll stand by that. Let the chips fall where they may, but at least level the playing field so they don't make erroneous projections.
#134 by Jonadan // Oct 07, 2010 - 3:42pm
Still, they're eleventh in yards overall. sixteenth in passing and thirteenth rushing by that standard. They have no running game to speak of, and their offensive line is average.
If the Pats are 13th rushing - that is, better than average - and they have "no running game to speak of", then the NFL as a whole is having a terrible rushing year, and you can't blame them for that. The Pats are first on offense because they score. Even if you take out all the ST/DEF touchdowns for NE while letting everybody else have those, the Patriots would still be 8th in scoring. And they're 3rd in net points, so except for the Jets game the Patriots D has been as good as it needs to be.
They also only put up six points against a mediocre Dolphins defense in the first half with the game in doubt, and before the Fins imploded.
But the 'fins did implode - don't they get any credit for that?
...and you have a statistic that has no basis in actual team performance.
Well, that is the problem with statistics - but it's a problem with any statistics. Saying, "DVOA sometimes uses statistics that don't seem to reflect reality," is just long-hand for "DVOA is a statistical approach."
I started this by saying that the DVOA totals, inappropriately combine three unmatched distributions. I'll stand by that.
If I understand what Aaron Schatz and others have said, the offense and defense are drawn from the same data, so they ought to be matched. Just eyeballing the data we've got here, it looks like there's more variation in how good DVOA says the offenses are, but you can't just "adjust" the variation away. Otherwise you're essentially ranking offense and defense and then averaging those numbers, which is imprecise at best (even if it would, for instance, drop the Patriots, and Colts for that matter).
Special teams is the problem, then. Currently, though, the percentages are small enough - apart from San Diego (terrible) - that they shouldn't affect that much. And then you've got teams like Seattle which are literally being carried by their special teams. I have no idea how the percentages are derived to begin with, but at some point you just have to leave well enough alone or you end up skewing everything.
#135 by ammek // Oct 07, 2010 - 4:05pm
I'm not sure what you're doing on this site if you think that total yardage, without any context or qualifiers, is "objective verification" of performance. The Patriots are #1 in third-down conversions, fourth in turnover differential, near the top in red-zone efficiency on offense.
And you continue to misread and misinterpret FO's stats. Yes, New England's offense is #5 in yards per drive. But it's #2 in DSR — and if this site and DVOA have preached anything at all, it's the importance of success rates. (Even Brian Burke is onto it now!)
In addition, I'd say that what the DVOA rankings (which are headed, in the drop-down menu, 'Team Efficiency') claim to measure is a lot closer to "least inefficient team in the NFL" than to your misquote about the mainframe.
Your empirical assertions aren't even very convincing. How does a team with "no running game to speak of" end up ranked #13 in yards? How exactly is it "clear" that the Patriots "aren't that talented"?
And I wonder which teams are "clearly" better than New England? If it's balance you're after, there are only five teams with DVOA in the top dozen in both offense and defense. Two of those are ruled out through abysmal special teams (which empirical models tend to underestimate, regarding them as fluky or unimportant — as a Packer fan, I have witnessed for too long what happens when you get consistently crappy field position). That leaves the Jets and Chiefs (both within a whisker of the Patriots in the rankings) and Atlanta. New York's unadjusted VOA is superior to New England's, but is adjusted downwards for playing the Patriots' defense.
#136 by AudacityOfHoops // Oct 07, 2010 - 5:01pm
Not to beat a dead horse, but ...
It seems like DVOA is driven mostly by:
A) converting 1st downs (or TD's on goal-to-go)
B) on plays where you are NOT converting, at least picking up enough yards to set yourself up for (A), plus not shooting yourself in the foot with a big loss
C) avoiding turnovers
I tried to measure these by the following metrics. Everything is divided by plays because DVOA is a per play metric:
A) non-penalty conversions per play (Conv): (1st downs + TD - [1st downs via penalty])/plays
B1) wasted play rate (Wast): (Stuffs + Incomplete passes)/plays
B2) sack rate (Sack): Sacks/plays
C) mistake rate (TO): (Interceptions + Fumbles)/plays
Here are all the offenses that rank in the top 10 in all of these metrics, along with their values and ranks for each:
TEAM / Conv% (rnk) / Wast% (rnk) / Sack% (rnk) / TO% (rnk)
IND / 40% (1) / 24% (2) / 1.5% (3) / 2.6% (7)
NE / 38% (4) / 24% (2) / 2.1% (6) / 2.1% (5)
NYJ / 33% (7) / 24% (2) / 2.1% (6) / 1.7% (2)
So, I don't see a problem with NE's offense being rated very highly at this point.
#131 by Mr Shush // Oct 07, 2010 - 2:08pm
Why on earth is it prima facie unlikely that there really is more variation in the quality of offenses than defenses? Or is that not the point you're making? I'm not saying I think the Patriots actually are the best team in the league (they're not), but it doesn't strike me as implausible that a team that was #1 in offense and #23 in defense should be #1 overall. A little surprising, but not outlandish.
#133 by DeltaWhiskey // Oct 07, 2010 - 2:45pm
I believe the issue is that by using a strictly additive model to come up w/ total DVOA, FO is assuming equality of variation between offenses and defenses. It makes me wonder if this relatively simple observation may account for the annual DVOA loves Philly debates and explain why the Ravens were the top rated team last year, and that if properly calculated DVOA, would not more accurately reflect reality.
#137 by nat // Oct 07, 2010 - 5:22pm
To be more precise, FO (Aaron in particular) is assuming that defensive and offensive DVOAs are measured on the same scale: per-play success compared to the same average. There is no need for the variations to be the same for adding to be reasonable. For example, if I want to know the weight of everyone in my car, I don't care that women have less weight variation than men, or that they average a lower weight. All I need to know is that everyone is weighed in the same pounds.
#138 by andyavatar (not verified) // Oct 07, 2010 - 5:49pm
But if I said that men are fatter on average than women, you'd scream bloody murder because the average for men was different from the average for women. You'd say, that's not a fair comparison, men are already heavier on average so how can we compare? And what if my scale introduced a lot of variability when the person being weighed was over 160 pounds? Men would vary from 140 to 400 pounds while women would only vary from 100 to 220 pounds. Could I then say that men were fatter than women? Could I directly compare an average male with an average female? The FO response is, we used the same scale for both, so it doesn't matter. But, didn't we do that also?
#139 by AudacityOfHoops // Oct 07, 2010 - 6:31pm
"But if I said that men are fatter on average than women, you'd scream bloody murder because the average for men was different from the average for women. You'd say, that's not a fair comparison, men are already heavier on average so how can we compare?"
The baseline 0% is set so that in the years on which DVOA was built, the average offense and defense both equal 0%. If offenses average higher than 0% this year, it means that offenses are doing better than in past years, and defenses are doing worse. In your scenario, if we took the average weight of men for the last five years, and the average weight of women for the past five, then compared them to current averages, and found that men are heavier now than in the past, and women are lighter now ... then YES, we would say that men are fatter than they used to be. That is what DVOA is saying - offenses are having more success than they used to.
And what if my scale introduced a lot of variability when the person being weighed was over 160 pounds? Men would vary from 140 to 400 pounds while women would only vary from 100 to 220 pounds.
This I can see being a problem, but I'm not sure it's analogous. If you know nothing about the scale, and you weigh everybody, and find that men's weights vary more than females, do you assume it's a problem with your scale, and try to correct the men's weights?
I still don't understand why you think offensive DVOA must have the same variance as defensive DVOA. Did you see my free throw percentage example from earlier? There is more variance in offensive FT% than defensive FT%, and that's a good thing, because offenses really are more in control of their FT% than defenses are. I don't know if that's true for NFL offenses, but it's at least plausible.
#140 by AudacityOfHoops // Oct 07, 2010 - 6:48pm
P.S. I thought I put this earlier, but must have edited it out:
I'm honestly curious to hear what your proposed solution is to the unequal variance issue. I am sure that sometimes this really is a problem (maybe it is here with DVOA, I don't know), and I'd like to know how to deal with it when I run into it. I can think of obvious things like shifting and scaling the numbers so that the means and variances are equal, but that forces unreal constraints on the system.
[For example, let's say we start with an NFL where the means and variances of offensive and defensive DVOA really are equal. Then we add clones of Jerry Rice in his prime to the 5 best teams. I'm not exactly sure how this works after defensive adjustments, but I think the average offensive and defensive DVOA will go up, the variation of the offensive DVOA's will increase (because the bests got better, and all others stayed the same), and the variation of the defensive DVOA's will stay the same (assuming a balanced schedule). ... Checking the numbers, that looks a lot like this year: Ave OffDVOA 1.1%, ave DefDVOA 1.2%, stdev OffDVOA 19%, stdev DefDVOA 14%.]
OK, that started as just an example of when I wouldn't want to tweak the average/variance, and ended up being an argument not to tweak it at all, I think. At any rate, I do still want to know your method of correcting for this error, so I can apply it in other fields (i.e. at work).
#141 by Led // Oct 07, 2010 - 6:52pm
But the issue with DVOA is not that extreme, right? I see what you're saying and, although I'm not a statistician, it makes sense to me intuitively. I guess I don't mind if straight addition of offense, defense and special teams is not appropriate from a rigorous mathematical perspective because I don't view DVOA, however it is intended, as a precise measurement. It's interesting and very useful general measure of performance and a good tool for identifying the reasons for the good/bad performance. The rankings are fun, but I take them with a little grain of salt. (My questions/comments at the top of this thread weren't addressing the NE's ranking, per se, but rather trying to understand the reasons DVOA goes up and down.) Simply adding offense, defense and ST makes the numbers a lot easier to use and understand. As a non-specialist, I appreciate the simplicity even if it's not methodologically perfect.
#142 by DeltaWhiskey // Oct 08, 2010 - 8:55am
The problem is that DVOA is measured in % deviance from the average. I suspect that the difference in variance is due to different measuring metrics.
For example, if we're measuring weight of men in kilograms and women in pounds and we find that Mr. A. is 5% above average and Mrs. A. is 5% above average, and we then to describe Couple A's weight as above average as 10%. Next, we measure Mr. B. and find he is 2% above average and Mrs. B is 8% above average, and then we again add them up and say they are 10% above average - our weight above average measure says they're the same mass; however, clearly they're not.
#143 by Eddo // Oct 08, 2010 - 10:33am
That's an interesting point, though the kg-vs-lbs part is a red herring. Those are just units. Once you start measuring in percent over average, the units don't matter. (That is, 10% more than 100 kg is the same mass as 10% more than 220 lbs.)
#145 by DeltaWhiskey // Oct 08, 2010 - 12:29pm
The units matter if you don't convert everything to a common unit before combining them.
Okay, agreed. In my head it seemed to work, but when I actually worked it out, it does come out the same; however, the problem occurs when adding percentages.
However, as I work this out, I don't know that straight up adding the unit DVOA percentages is appropriate. For example, example if the average weight of a man and woman is equal, say 220lb then 10% over average for a couple should be 484 lbs (440*1.1); however, if you calculate it independently for each person the way I calculated Couple A, Mr. A weighs 231 lbs (220lbs * l.05) and Mrs. A weighs 231 lbs for a grand total of 462 lbs, which ends up being 5% over the combined average. Moreover, with different means, I think this gets even messier.
This suggests to me that if FO is strictly adding percentages to come up with overall DVOA their could be a problem. It seems that instead they should be calculating success points for each unit (offense, defense, S.T.) and summing them and dividing by a summed demonimator to ascertain team DVOA.
As far as OFF and DEF DVOA being the same metric, I don't think they are as certain penalties (false start I think) impact one but not the other, and S.T. DVOA is definitely measured differrently.
I may be totally wrong on all of this and I hope I was reasonably clear, but the bottom line is that percentages are not always additive.
#147 by AudacityOfHoops // Oct 08, 2010 - 2:11pm
"It seems that instead they should be calculating success points for each unit (offense, defense, S.T.) and summing them and dividing by a summed demonimator to ascertain team DVOA."
I started out creating a hypothetical situation to argue against this point, but I think I changed my mind in the middle. Check this out:
Let's say you have a team with an offense that gains exactly 10 yards every play, a defense that gives up exactly 10 yards every play, and a special teams unit that is so great that every drive (offensive and defensive) always starts from the opponent's 20 (so the offense only has to go 20 yards for a TD, and the defense has 80 yards to work with). The offense will get about 1 success point per play, and the defense will allow about 1 success point per play. If we assume the average success is 0.5, their DVOA's should be about equal, at roughly +100% (0.5 above average, divided by the average of 0.5). Obviously the ST DVOA will be something crazy, say +100% also. Under the current system, the team's overall score would be 100 - 100 + 100 = +100%. Yet, they will have a net scoring margin of 0. DVOA is severely overrating the overall team strength, while AT THE SAME TIME being very accurate in the ratings of the units.
Now, say you took DW's suggestion, and summed all success points, then divided them by the summed denominator. On a typical sequence (offensive possession, kickoff/punt, defensive possession, kickoff/punt), here's what we'd have:
Offense runs 2 plays and scored: 2 success points, average team would get 1, so +1 better than average
Kickoff is a touchback: some number near 0
Defense allows 8 plays and a TD: 8 success points allowed, average team would get 4, so -4 worse than average
Kickoff Return to the opponent's 20: some large number, which if calibrated correctly should offset everything else that has happened so far (since we know over the long run this team has a 0 point differential). I think +3 better than average is reasonable, given that this is the same amount of yardage as the defense, but still leaves the score in doubt, and is in a slightly less valuable portion of the field
Add together: [+1 offense] + [0 kickoff] + [-4 defense] + [+3 return] = +0 = 0% DVOA.
This is a much better estimate of the overall strength of the team, while the evaluation of the individual units will not change.
I guess that's why it's called a thought experiment and not a thought I-already-know-how-this-turns-out.
#148 by AudacityOfHoops // Oct 08, 2010 - 2:19pm
Hmm, now that I think about it, maybe all I showed is that there are important interactions between the strengths of a team's offense, defense, and special teams (read "field position").
#149 by Thomas_beardown // Oct 08, 2010 - 4:18pm
Special teams is only counted as 1/3.
So the example DVOA would be 33%
#150 by AudacityOfHoops // Oct 08, 2010 - 5:01pm
If you're saying that my equation from the the first half of the post should be:
100 - 100 + (1/3)*100 = 33.3%
Then I respectfully disagree. Look the charts at the top of this page. Total DVOA is the sum of OFFENSE DVOA, DEFENSE DVOA, and S.T. DVOA. Now, I believe special teams DVOA is totally unrelated to "success points" and is just scaled to match DVOA, and that scaling is done in a way that gives special teams about 1/3 the weight of offense or defense. Maybe this is what you're thinking of?
If you're saying that when I pulled a number out of my ass to represent crazy awesome special teams, I should have retrieved 33.3% instead of 100%, then I ... also respectfully disagree. We have normal teams scoring in the 20% range, so an average return of 80 yards is going to be waaaay higher than that.
#151 by Thomas_beardown // Oct 09, 2010 - 2:32am
All I know is that special teams are only worth 1/3 of offense or defense in DVOA, so if your imaginary special teams were 100% better than average, while offense and defense canceled out, total DVOA would be 33%
They would have to be 300% better than average to get 100%, and if your special teams are 4 times as good as average, you might deserve that rating.
#152 by AudacityOfHoops // Oct 09, 2010 - 3:33pm
OK. Either way, it doesn't change the point or outcome of the example. Whether the overall DVOA is 33% or 100%, the team is being overrated by DVOA, since by definition the team has a net scoring margin of 0.
#144 by AudacityOfHoops // Oct 08, 2010 - 11:25am
I still don't get this. Isn't "success points" the metric for both offensive and defensive DVOA? And aren't the points the same for offense and defense (i.e. giving up 10 yards of 1st and 10 is worth 1 point to both the offense and the defense)? Am I missing something?
As indicated above, your analogy isn't quite right, so to let's modify it ... if the average weight of men is 170 pounds, and the average weight of women is 110 pounds, then [man +5% / woman +5%] is not equal to [man +2% / woman +8%]. For DVOA, this would be equivalent to an average offense gaining 0.5 success points per play, but an average defense only giving up 0.4 success points per play, right? (I'm not 100% positive, here.) Is that even possible, given the way DVOA is constructed?
Do the defensive adjustments throw this off somehow, so that it's not possible for VOA, but it is possible for DVOA? Let me walk through a scenario and ask for help or an explanation:
TeamA (+30% offensive DVOA) vs. TeamB (0% defensive DVOA), 1st and 10. TeamA gets a first down and gets credit for 1 success point, while TeamB has 1 success point charged against them. OK, but we have to compare those to what an average teams would do in that situation. Let's say the average outcome is 0.5 success points. Then both teams are at +0.5 success points (to be later used in VOA calculation). Same scale so far.
Now we try to take opponent strength into account. TeamA is facing an average defense, so no change there: 1 sucPt - 0.5 aveSucPt = +0.5 sucPtAboveAve. TeamB is facing a very good offense, though, so they're expected to allow a worse play. Maybe their equation is now: 1 sucPt - 0.6 aveSucPt = +0.4 sucPtAboveAve.
If those were the ONLY plays the team ran, their DVOAs would be:
TeamA = [0.5 above ave]/[0.5 ave] = 1 = +100%
TeamB = [0.4 above ave]/[0.6 ave] = 2/3 = +67%
Did that make sense, or did I just make a bunch of mistakes here?
#146 by DeltaWhiskey // Oct 08, 2010 - 12:41pm
Your guess is as good as mine. I think your tracking somewhat along the way I'm thinking. Could you expand your example a little bit to include the hypothetical opposite, that is, what happens when Team A goes on DEF and Team B is on OFF, b/c at that point we have both sides of the ball for both teams and add them up; however, we're not adding the raw data, but the percentages. I think that is where the problem is occurring.
#128 by Alternator // Oct 07, 2010 - 12:33pm
You might do better with genuine criticism rather than childish whining, you know. If you're really a scientist of some sort, you should understand the concept of "attack the data, not the author" which is not what you're doing.
#87 by ChaosOnion (not verified) // Oct 06, 2010 - 8:39am
As I was watching the Eagles drive down the field 4-6 yards at a time for 6 plus minutes, I knew DVOA was going to love it. DVOA loves the long, methodical drives and they are awesome ahead by 10 but, they are suicide down by 14. They were taking the underneath all second half against WAS's Tampa-2 but, it sure looked ugly since McCoy seemed to be the only one open.