Writers of Pro Football Prospectus 2008

05 Oct 2010

Week 4 DVOA Ratings

by Aaron Schatz

A young, porous defense? No problem. Zero catches for Randy Moss? Easily identifiable fluke. While the New England press (and many fans) obsess over every last Patriots weakness, the Football Outsiders mainframe has proclaimed New England the best team in the NFL, albeit with a sizeable "so far" asterisk. Last night's big win over Miami vaults the Pats over Kansas City, Green Bay, and the New York Jets and into the top spot in our DVOA ratings. In fact, the Patriots zoom to number one even though two of the biggest plays of the night do not count in our special teams ratings. The Patriots get no credit for the blocked field goal, because blocks are rare and scoring on a missed field goal is rarer still. They get no credit for the punt block, because our punt return ratings for the receiving team only judge punt returns. The Dolphins do get penalized for both events, although the penalty for a blocked field goal is no different from the penalty for any other missed field goal.

Week 4 is when we introduce opponent adjustments into our formula for the first time. The hardest schedules so far have belonged to Buffalo, San Francisco, and Baltimore; the easiest have belonged to St. Louis, New Orleans, and Green Bay. The Jets have one of the biggest gaps between DVOA and non-adjusted VOA, but that's not because of schedule -- that's because of fumble recovery. The Jets have recovered four of five fumbles on defense and two of three fumbles on offense.

This year's parity -- with no dominant teams and lots of close games -- has also led to some strange results in the early DVOA rankings. Winless Detroit ranks three spots higher than 3-1 division rival Chicago. 2-2 Arizona is in last place, with two very close wins over below-average opponents far outweighed by two embarrassing blowout losses. The 1-2 Cowboys rank higher than six different teams with winning records. As for surprise teams, it's worth nothing that both Kansas City and Houston have played roughly average schedules so far. Based on current DVOA, however, Houston has the hardest remaining schedule in the league, while the Chiefs have the second-easiest.

There are also some strange goings-on with the player ratings so far, so let's take a look.

QUARTERBACKS

The pass-run balance in the NFL continues to tip towards the pass this year, with a number of quarterbacks off to historically ridiculous starts. Kyle Orton, Peyton Manning, and Philip Rivers are all on pace to break Dan Marino's record of 5,084 passing yards in a season. Obviously, we're used to hearing these kind of "on pace" stats early in the season, in every sport, and those players rarely stay on that pace. However, these three quarterbacks aren't on pace to barely top the record. All three are on pace to blow by it by at least 200 yards. Even with the inevitable slowdown, somebody is going to end up as just the third quarterback in NFL history with at least 4,900 passing yards. The other one, besides Marino, was Drew Brees two years ago, and that Brees season goes a long way towards explaining some of what we're seeing this year.

Drew Brees was very good in 2008, but he wasn't great. He put up great numbers because he threw the ball a ridiculous amount trying to keep up with the points given up by the Saints' terrible defense that year. He ended up with 635 attempts, the fifth highest season total in NFL history. This year, both Kyle Orton and Peyton Manning have been passing non-stop, stuck in close games because of poor defense. The Broncos rank 28th in defensive DVOA, the Colts 27th. Orton has the added albatross of a horrible running game, the worst in the league so far this season. Orton is a very good eighth in DVOA, but he's an even better second when it comes to DYAR, because he's thrown the ball 175 times. Manning is playing better on a per-play basis -- he leads the league in DVOA, not much of a surprise -- but he's also throwing the ball non-stop, with 173 attempts. Those numbers pro-rate to 700 attempts and 688 attempts over an entire season. The record is 691 attempts, by Drew Bledsoe in 1994, and no other season really comes close. Warren Moon (655 in 1991) and Drew Brees (652 in 2007) are the only other quarterbacks with more than 640 pass attempts in a season. Even if they slow down a little, it is hard to imagine Orton and Manning not hitting 640 pass attempts if they stay healthy.

Usually when teams throw this much, you don't expect a high level of accuracy, but that's just another way that the current offensive environment is so unique. So far this year, quarterbacks have completed 62 percent of passes, which would be an NFL record if it stays that high through 16 games.

RUNNING BACKS

Raise your hand if you expected the three most valuable running backs in the NFL to be Arian Foster, BenJarvus Green-Ellis, and Mike Tolbert? Those are the top three guys in rushing DYAR right now. LaDainian Tomlinson is fourth; Neil Paine of the P-F-R blog did a good write up this week about how historically unprecedented it is for a running back to have a career rebound like Tomlinson's after age 30.

Meanwhile, a series of strong veteran running backs have been below-average so far this year. Maurice Jones-Drew, Steven Jackson, Michael Turner, and Ricky Williams all rank 25th or lower in DYAR. Matt Forte's expected statistical rebound has so far only taken place in his receiving numbers -- he's dead last in rushing DYAR by a significant margin. But the strangest name in the strangest place is Chris Johnson. Johnson is currently 35th out of 37 running backs in DYAR. He's averaging just 3.8 yards per carry, a far cry from the 5.6 yards per carry he had a year ago. Even when he had 125 yards against the Giants in Week 3, it took him 32 carries. As we wrote in the book, Johnson was definitely going to face regression towards the mean, but he has to be better than this, right? The Titans line is 26th in Adjusted Line Yards but it isn't like Johnson had far better blocking last year, because the Titans were only 21st in ALY in 2009. You know Johnson's goal of 2,500 yards this season? Yeah, not happening.

WIDE RECEIVERS

If you think seeing Chris Johnson's name near the bottom of the running back rankings is odd, that's nothing compared to the receiving rankings, where Larry Fitzgerald currently ranks 74th in DYAR out of 86 wide receivers. It's not Fitzgerald's fault that Arizona can't find a starting-quality quarterback, but plenty of top receivers have put up good seasons with bad quarterbacks. (Hi, I'm Steve Smith.) Fitzgerald ranked in the DYAR top ten for four seasons, 2005-2008, and then ranked 11th last year. His catch rate, consistently over 60 percent until this year, is currently 42 percent for the season.

Another great receiver with bad numbers this year, partly because of the quarterback situation, is Lee Evans. Evans is 77th in DYAR, but what's even stranger is that the Bills quarterbacks have only thrown to him 22 times (including one Pass Interference). This guy is the best receiver on the team by a mile, and they barely throw him five passes a game? The Bills have thrown the same number of passes to Steve Johnson. Johnson is actually doing more with those passes -- 63 percent catch rate, 36th in DYAR -- but that's because no team is going to make sure they have a safety playing over the top to keep the defense from getting burned by Steve Johnson.

Meanwhile, at the top of the wide receiver rankings, the most valuable receiver this year has been... Brandon Lloyd.  Yes, the guy who was run out of San Francisco, the guy who gave the Washington Redskins 25 catches and no touchdowns for $10 million guaranteed, the guy who barely could get on the field for Denver last year (two games, eight catches). Will Lloyd continue to produce for the Broncos? Or, as in every one of his previous career stops, will teammates and coaches gradually sour on him because he is, apparently, a total jackass?

Anyway, on to the ratings. All stat pages will be updated in the next few minutes. The FO Premium database of DVOA splits will be updated later tonight. Apparently, something went wrong with last week's update, but this week's update should get all the searchable splits fully updated through Week 4.

* * * * *

These are the Football Outsiders team efficiency ratings through four weeks of 2010, measured by our proprietary Defense-adjusted Value Over Average (DVOA) system that breaks down every single play and compares a team's performance to the league average based on situation in order to determine value over average. (Explained further here.)

OFFENSE and DEFENSE DVOA are adjusted to consider all fumbles, kept or lost, as equal value. SPECIAL TEAMS DVOA is adjusted for type of stadium (warm, cold, dome, Denver) and week of season.

Opponent adjustments are currently at 40 percent strength and will steadily grow stronger until Week 10. As always, positive numbers represent more points so DEFENSE is better when it is NEGATIVE.

DAVE is a formula which combines our preseason projection with current DVOA to get a more accurate forecast of how a team will play the rest of the season. Right now, the preseason projection makes up 40 percent of DAVE for teams that have played four games and 55 percent of DAVE for teams that have played three games.

To save people some time, please use the following format for all complaints:

<team> is clearly ranked <too high/too low> because <reason unrelated to DVOA>. <subjective ranking system> is way better than this. <unrelated team-supporting or -denigrating comment, preferably with poor spelling and/or chat-acceptable spelling>

TEAM TOTAL
DVOA
LAST
WEEK
TOTAL
DAVE
RANK W-L OFFENSE
DVOA
OFF.
RANK
DEFENSE
DVOA
DEF.
RANK
S.T.
DVOA
S.T.
RANK
1 NE 34.1% 6 29.0% 1 3-1 42.9% 1 13.0% 23 4.2% 4
2 KC 33.6% 2 14.7% 9 3-0 11.5% 10 -16.3% 5 5.8% 3
3 NYJ 31.2% 4 23.7% 3 3-1 22.8% 5 -5.5% 12 2.9% 8
4 GB 27.6% 1 22.8% 4 3-1 27.0% 4 -7.9% 8 -7.3% 29
5 SD 26.0% 15 13.4% 10 2-2 20.8% 7 -26.0% 1 -20.7% 32
6 PIT 24.5% 3 25.4% 2 3-1 -1.9% 19 -22.9% 2 3.6% 6
7 PHI 20.4% 5 18.0% 5 2-2 20.9% 6 -0.9% 16 -1.4% 19
8 DAL 18.0% 8 9.6% 12 1-2 19.1% 8 2.2% 18 1.1% 12
9 ATL 16.2% 7 17.4% 7 3-1 7.6% 12 -7.8% 9 0.8% 13
10 NYG 14.5% 20 12.2% 11 2-2 1.6% 16 -22.5% 3 -9.7% 30
11 IND 13.9% 10 16.5% 8 2-2 32.6% 2 15.0% 27 -3.7% 25
12 TEN 11.4% 13 4.1% 13 2-2 -9.2% 21 -20.2% 4 0.4% 14
13 SEA 8.8% 9 2.6% 15 2-2 -14.3% 24 -6.2% 11 16.9% 1
14 BAL 8.6% 18 17.8% 6 3-1 -1.6% 18 -10.9% 6 -0.7% 17
15 HOU 4.1% 21 -4.9% 20 3-1 31.2% 3 26.1% 31 -0.9% 18
16 CIN 2.3% 16 0.3% 17 2-2 -0.2% 17 -4.6% 13 -2.1% 22
TEAM TOTAL
DVOA
LAST
WEEK
TOTAL
DAVE
RANK W-L OFFENSE
DVOA
OFF.
RANK
DEFENSE
DVOA
DEF.
RANK
S.T.
DVOA
S.T.
RANK
17 NO 0.2% 12 4.0% 14 3-1 12.6% 9 14.3% 25 1.9% 10
18 TB -1.4% 17 -7.0% 21 2-1 -10.3% 22 -6.9% 10 2.0% 9
19 CLE -5.1% 19 -11.3% 23 1-3 2.9% 15 8.0% 21 0.0% 16
20 WAS -7.3% 25 0.4% 16 2-2 8.5% 11 13.6% 24 -2.1% 21
21 MIA -11.0% 11 -1.4% 19 2-2 3.2% 14 4.1% 20 -10.2% 31
22 STL -11.7% 23 -15.8% 25 2-2 -17.4% 26 -8.1% 7 -2.4% 23
23 DEN -15.6% 22 -12.0% 24 2-2 4.3% 13 16.3% 28 -3.6% 24
24 DET -17.4% 26 -23.8% 27 0-4 -11.6% 23 9.3% 22 3.5% 7
25 MIN -18.9% 24 -1.1% 18 1-2 -21.8% 28 -2.9% 14 0.1% 15
26 JAC -20.3% 30 -16.7% 26 2-2 -2.7% 20 25.1% 30 7.6% 2
27 CHI -21.9% 14 -7.5% 22 3-1 -22.8% 29 0.2% 17 1.2% 11
28 SF -31.7% 32 -26.7% 29 0-4 -23.0% 30 3.4% 19 -5.3% 28
29 CAR -35.5% 31 -24.4% 28 0-4 -35.6% 32 -1.8% 15 -1.7% 20
30 BUF -40.4% 28 -28.7% 30 0-4 -15.1% 25 29.0% 32 3.7% 5
31 OAK -41.4% 29 -37.0% 32 1-3 -19.9% 27 17.1% 29 -4.4% 27
32 ARI -45.3% 27 -29.3% 31 2-2 -27.0% 31 14.5% 26 -3.8% 26
  • ESTIMATED WINS uses a statistic known as "Forest Index" that emphasizes consistency as well as DVOA in the most important specific situations: red zone defense, first quarter offense, and performance in the second half when the score is close. It then projects a number of wins adjusted to a league-average schedule and a league-average rate of recovering fumbles. Teams that have had their bye week are projected as if they had played one game per week.
  • PAST SCHEDULE lists average DVOA of opponents played this season, ranked from hardest schedule (#1, most positive) to easiest schedule (#32, most negative). It is not adjusted for which games are home or road.
  • FUTURE SCHEDULE lists average DVOA of opponents still left to play this season, ranked from hardest schedule (#1, most positive) to easiest schedule (#32, most negative). It is not adjusted for which games are home or road.
  • VARIANCE measures the statistical variance of the team's weekly DVOA performance. Teams are ranked from most consistent (#1, lowest variance) to least consistent (#32, highest variance).



TEAM TOTAL
DVOA
W-L NON-ADJ
TOT VOA
ESTIM.
WINS
RANK PAST
SCHED
RANK FUTURE
SCHED
RANK VAR. RANK
1 NE 34.1% 3-1 34.8% 3.1 2 -4.5% 20 1.4% 12 12.1% 18
2 KC 33.6% 3-0 31.6% 3.1 1 -3.6% 16 -12.9% 31 8.6% 12
3 NYJ 31.2% 3-1 40.8% 3.0 4 -2.2% 14 -3.1% 23 5.7% 10
4 GB 27.6% 3-1 35.0% 3.0 3 -14.8% 30 -1.1% 18 10.6% 14
5 SD 26.0% 2-2 31.8% 2.7 5 -5.8% 23 -4.8% 25 42.1% 29
6 PIT 24.5% 3-1 22.9% 2.7 6 8.7% 7 -5.0% 26 14.5% 19
7 PHI 20.4% 2-2 27.7% 2.6 7 -4.3% 19 2.6% 10 10.8% 16
8 DAL 18.0% 1-2 16.8% 2.4 10 -8.3% 26 1.1% 16 21.5% 26
9 ATL 16.2% 3-1 18.6% 2.3 13 -13.0% 29 -1.9% 20 19.9% 24
10 NYG 14.5% 2-2 6.5% 2.3 14 -8.0% 24 3.8% 6 45.6% 31
11 IND 13.9% 2-2 16.4% 2.5 9 -4.3% 18 7.7% 2 19.0% 22
12 TEN 11.4% 2-2 12.8% 2.5 8 -4.5% 21 6.3% 4 18.6% 21
13 SEA 8.8% 2-2 6.3% 1.8 20 -8.2% 25 -14.1% 32 19.8% 23
14 BAL 8.6% 3-1 0.9% 2.4 11 13.2% 3 -2.3% 22 5.2% 8
15 HOU 4.1% 3-1 7.1% 2.2 15 -4.2% 17 9.6% 1 16.9% 20
16 CIN 2.3% 2-2 3.5% 2.0 17 0.5% 13 7.3% 3 10.4% 13
TEAM TOTAL
DVOA
W-L NON-ADJ
TOT VOA
ESTIM.
WINS
RANK PAST
SCHED
RANK FUTURE
SCHED
RANK VAR. RANK
17 NO 0.2% 3-1 3.2% 2.1 16 -17.5% 31 -1.8% 19 2.4% 3
18 TB -1.4% 2-1 -0.2% 2.4 12 -5.4% 22 -7.4% 28 30.4% 27
19 CLE -5.1% 1-3 -2.4% 2.0 18 10.8% 5 2.9% 9 1.8% 2
20 WAS -7.3% 2-2 -8.6% 1.8 21 7.7% 8 3.4% 7 3.5% 5
21 MIA -11.0% 2-2 -6.6% 1.8 19 1.5% 12 1.1% 14 10.6% 15
22 STL -11.7% 2-2 2.4% 1.7 22 -21.3% 32 -7.8% 29 5.1% 6
23 DEN -15.6% 2-2 -8.0% 1.4 25 3.5% 10 -0.7% 17 3.4% 4
24 DET -17.4% 0-4 -18.7% 1.4 26 1.8% 11 1.1% 15 1.8% 1
25 MIN -18.9% 1-2 -14.0% 1.5 24 -9.4% 27 1.5% 11 5.2% 9
26 JAC -20.3% 2-2 -23.6% 1.6 23 11.2% 4 1.4% 13 39.1% 28
27 CHI -21.9% 3-1 -18.4% 1.1 28 10.7% 6 -2.3% 21 49.1% 32
28 SF -31.7% 0-4 -33.8% 1.3 27 14.7% 2 -10.4% 30 20.1% 25
29 CAR -35.5% 0-4 -40.7% 0.8 31 3.9% 9 -3.5% 24 5.1% 7
30 BUF -40.4% 0-4 -47.4% 0.5 32 20.5% 1 3.3% 8 8.1% 11
31 OAK -41.4% 1-3 -34.3% 0.8 30 -10.4% 28 6.0% 5 11.5% 17
32 ARI -45.3% 2-2 -50.2% 0.9 29 -2.7% 15 -6.4% 27 44.5% 30

Posted by: Aaron Schatz on 05 Oct 2010

153 comments, Last at 09 Oct 2010, 11:39pm by leviramsey

Comments

1
by Led :: Tue, 10/05/2010 - 5:15pm

"Last night's big win over Miami vaults the Pats over Kansas City, Green Bay, and the New York Jets and into the top spot in our DVOA ratings."

That's a complete post hoc ergo propter hoc, no? The Miami game didn't "vault" them into first place. Special teams plays (of which only the TD return is measured by DVOA) aside, NE didn't play a particularly impressive game. Not a bad game, just not a great game. Applying the adjustments to the raw VOA has to be what "vaulted" NE to the top.

3
by Aaron Schatz :: Tue, 10/05/2010 - 5:21pm

Nope. Last night the Pats had the third-best game of this week, higher then the teams below them, so last night's game is in fact what vaulted them ahead. When I ran the ratings before MNF, the Pats were sixth.

8
by Led :: Tue, 10/05/2010 - 5:29pm

I'll take your word for it but it doesn't make much sense based on my understanding of the way DVOA works. I must be misunderstanding. NE had lower YPA and YPC than the Dolphins and a worse 3rd down percentage. Does not seem very efficient on a per play basis. The INTs are important but usually INTs are not enough to make up for the fact that the other team was marching up and down the field. There's obviously a nuance to DVOA that I'm missing.

18
by morrongiello :: Tue, 10/05/2010 - 5:55pm

I'm not defending their formula but I didn't think 3rd down percentage was that important (?). Has anyone been able to show that teams can maintain it consistently on a game-by-game basis? I'm looking at the first four weeks of this season and some of the best offensive teams have flopped around like crazy (at least with respect to their offensive yards per play).

42
by nat :: Tue, 10/05/2010 - 8:59pm

Back in 2003, Aaron wrote a nifty article about the value of turnovers. The short version is that they are worth about the same as giving up or getting about 55 yards of field position.

It's quite common for people to underestimate the impact of turnovers. Remember how Miami was "marching up and down the field" seemingly at will during the first half, all the while accumulating high YPA and YPC? They scored seven points.

So, no, you're not missing a nuance. You are missing one of the most important points for understanding football: ball security matters enormously.

49
by Led :: Tue, 10/05/2010 - 9:47pm

Right, and the Pats had 6 points in the first half with a sputtering offense and not much production on offense in the second half, either, although they didn't have the ball much. Against the 20th ranked defense. So why doesn't their superhigh offensive DVOA take a bigger hit? And on defense, yes, turnovers are very important, but giving up chunks of yardage on most of the other plays matters too. That's why Chad Henne had positive DYAR for the day, despite 3 picks. And that was just passing. Ricky Williams was the third most valuable running back. (In fairness, the Pats did do a great job on Ronnie Brown.) That doesn't scream out as dominating performance to me. Remember, the issue isn't whether the Pats played well overall. It's whether they dominated enough to cause a major increase in their already very high DVOA. Obviously, they did because that's what the numbers say, but I just don't see it.

54
by Scott C :: Tue, 10/05/2010 - 10:10pm

I agree with Led.

The big plays that turned it into a blowout were mostly not counted by DVOA. What is left did not look 'dominating' though it did look 'very solid'.

On the other hand DVOA values third down conversions (or more generally, first downs) VERY highly, and rightfully so. The research shows that although great third down success = great DVOA, it is not particularly predictive DVOA, especially season to season.

76
by Kulko :: Wed, 10/06/2010 - 2:05am

LED wrote: "Right, and the Pats had 6 points in the first half with a sputtering offense and not much production on offense in the second half, either, although they didn't have the ball much. Against the 20th ranked defense. So why doesn't their superhigh offensive DVOA take a bigger hit?"

A) They were marching up and down the field in the first half too, just ending in field goals. Miami appeared to play superior in the first half because they really carved up the NE Def, and because their was the feeling, that the field goals would not be enough, when the Def gives up TDs.
They were marchign up and down the field in the second half whenever they had the ball, evene converting first downs with the running game, when they were cleraly running out the clock.

B) They had fewer offensive snaps than in their first three games, and DVOA is an per play average, not per game. So the miami game values disproportionally low for their offensive DVOA when compared to common perception which is a per half average.

85
by nat :: Wed, 10/06/2010 - 8:18am

Do you legitimately want to learn, or are you just whining? If you're just whining, you should use the complaint format to save us all time.

DVOA is a per-play measure. When a team plays well for a sustained 16 play or 13 play drive, DVOA doesn't retroactively reduce the value of the first 15 or 12 plays because the last play was a chip-shot field goal. So, rather than saying "sputtering offense" in the first half say "offense with two long sustained drives out of four possessions". Not great, certainly, but not bad at all - on a per play basis.

As for the relative value of turnovers and yardage, you should really read the article I linked. It's not a matter of taste or at this point even much argument: An average turnover is worth about 55 yards of field position. Chad Henne achieved a value slightly higher than a replacement QB - a positive DYAR as you say. He did it by having decent YPA paired with 3 interceptions.

If you want to ignore special teams (why would you?) the Patriots' offense beat the Dolphins' offense 20-7, despite having only 7 drives (ignoring the kneel down drive) to the Dolphins' 10 drives.

So how did this excellent game result in an improved DVOA for the season? Primarily, it was the improved defensive play (average instead of awful) and the dominant special teams (even ignoring the blocked kick and punt, as DVOA must, the special teams were tremendous). The offense maintained its high per-play effectiveness, but that was already high so it wasn't the reason for the DVOA jump.

118
by Led :: Wed, 10/06/2010 - 8:43pm

What I'd really want is for you not be a condescending jerk. My comments have been perfectly reasonable and respectful, and not even close to whining. I have no dog in the race. (I even said I'm not surpised NE has the hightest DVOA, just that the Dolphins game -- rather than adjustments kicking in -- is what did it.) If you're not prepared to discuss things without being insulting, why post at all? Maybe you need to ask yourself why you are so irritated about a good natured discussion. And who is the "us" that you're purporting to speak for?

On the merits, and putting aside the ugliness, T-Ball makes some good points below, particularly with respect to the value of the KO return and all the touchbacks. Gotskowski was, indeed, a monster. Five points of ST DVOA goes a long way to explain the increase. And I confess I was probably discounting/ignoring the Thigpen plays in my assessment. Beating up on Tyler Thigpen isn't all that impressive, but the DVOA formula doesn't know that. As for the offense, it's still hard to see a performance of 6.1 YPA, 3.7 YPC, and 45% on 3rd downs against the 20th ranked defense (by DVOA) as anything more than mediocre.

121
by nat :: Wed, 10/06/2010 - 11:50pm

My apologies. The complaint format is an old FO tradition. People even use it when they have reasoned arguments. I thought you knew.

Did you read that article, though? Turnovers are really, really valuable. The article I linked explains why better than I could here. It's a classic, and ought to be required reading.

You keep repeating that the Dolphins had good YPA and YPC. I agree. The Patriots achieved a better than average defensive result by getting turnovers (3 against Henne, so it's not all or even mostly Thigpen), not by stopping the Dolphins from advancing the ball. It's a legitimate way of being good on defense, although I wonder whether it can be sustained. DVOA considers interceptions predictive of future performance, so perhaps it can.

Keep in mind that the Patriots D had been so bad in three games that merely getting a moderately better than average result (about -8% DVOA, I guess) is still a huge improvement, and enough in combination with their huge special teams play to move their total DVOA upwards.

I hear what you're saying about the offense, but you're forgetting turnovers again. Teams average about an interception per game. By avoiding that interception, the Patriots get about the same value as an extra 55 yards or so of passing. It's as if they were getting 8.4 YPA instead of 6.1. Their effectiveness shows up in their drive stats, too. 20 points on 7 drives (kneel down drives are ignored for this calculation) is 2.86 points per drive, which would lead the league if it could be sustained over a season. The drive success rate (% of sets of downs that lead to a first down or a TD) would lead the league, too. That's not mediocre at all, unless you think the entire league is bad on offense.

The take-away lessons are that turnovers matter a lot, YPA or YPC matters less than you might think, and sustaining drives matters more than you might think.

86
by RichC (not verified) :: Wed, 10/06/2010 - 8:29am

The Patriots had 4 drives in the 2nd half.

2 of them were touchdowns, one was a 3 and out, and the last one ran out the final 4.5 minutes of the game. Thats pretty damn efficient football.

106
by TBall (not verified) :: Wed, 10/06/2010 - 1:44pm

Led,

I'd profess to know less about DVOA than you, but I don't think New England's DVOA should be that surprising. The kickoff return for a touchdown still counts and Gostkowski was putting every kickoff in the endzone. The special teams plays that did count helped improve the Pats by about 5 points in ST over week 3. The offense's DVOA fell back 5 points from week 3 (47 to 42) and the defense improved by 7.

It may feel like the Dolphins were able to move up and down the field with abandon (gaining 275 yards on their first 36 plays) but after the Pat's last offensive TD (third TD of the 3rd quarter), the Dolphins managed 125 yards on 31 plays, which included a turnover on downs and a pick 6. It doesn't seem as important because much of it seemed to occur as the game was out of reach. Thigpen's 2/6 for 15 yards and 1 pick brought down the overall level of QB play for the Dolphins.

Would you rather have 265 yards on 59 plays with 0 TO's or 400 yards on 67 plays with 4 TO's?

2
by >implying implications (not verified) :: Tue, 10/05/2010 - 5:17pm

Best 2-win team: Chargers
Best 1-win team: Cowboys
Best no-win team: Lions

Worst 2-win team: Cardinals
Worst 3-win team: Bears

23
by AudacityOfHoops :: Tue, 10/05/2010 - 6:13pm

Sticking with the theme, I converted Estimated Wins to an Estimated Win% (EW/4), then subtracted that from actual win%, to get a list of which teams are most over-/under-performing their Forest Index. Most of it is obvious by glancing at the DVOA ratings chart("I see 3-1 Chicago is listed near the bottom"), but I like to see it quantified anyway:

W%DIF TEAM
0.475 CHI
0.275 ARI
0.225 KC
0.225 NO
0.200 HOU
0.175 ATL
0.150 BAL
0.150 DEN
0.100 JAC
0.075 STL
0.075 PIT
0.067 TB
0.050 SEA
0.050 WAS
0.050 MIA
0.050 OAK
0.000 NYJ
0.000 GB
0.000 CIN
-0.025 NE
-0.042 MIN
-0.075 NYG
-0.125 IND
-0.125 TEN
-0.125 BUF
-0.150 PHI
-0.175 SD
-0.200 CAR
-0.250 CLE
-0.267 DAL
-0.325 SF
-0.350 DET

120
by Jonadan :: Wed, 10/06/2010 - 10:55pm

I know you're just going off DVOA, but I take issue with this, and I'd extend it a little. My own theory says it looks like this:

Best 2-win team: Chargers
Best 1-win team: Cowboys
Best 0-win team: Lions

Worst 3-win team: Texans
Worst 2-win team: Cardinals
Worst 1-win team: Raiders

I'm not convinced the Texans can really get it together if they keep giving up a bazillion points, and unlike the Pats (now) they haven't shut anybody down. "Worst 1-win team" is an unfortunate stat, but it amuses me.

4
by Big Johnson :: Tue, 10/05/2010 - 5:24pm

best defense according to dvoa is the chargers! wow that brings a smile to my face. i am pretty sure they wont end the year with the best defense but it seems pretty safe to say they arent anywhere near as bad as they were the last couple years. Someone on here also stated the chargers put 5 (or was it 6?) new players on special teams coverage this last week. If they can correct that at all then they just might be the best team in the league.

32
by Mr Shush :: Tue, 10/05/2010 - 7:53pm

Hmm. I think it mostly means that opponent adjustments remain highly inaccurate and are very low strength, and the Chargers have played half their schedule to date against the lousy Seahawks offense and the historically abysmal Cardinals offense, complete with undrafted rookie quarterback.

The best team in the league are the Steelers. I don't like it. I don't like them. But them's the breaks.

5
by Neoplatonist Bolthead (not verified) :: Tue, 10/05/2010 - 5:25pm

Wow. San Diego is ranked 5th, with a -20% Special Teams DVOA. That's crazy.

7
by Pass to Set Up ... :: Tue, 10/05/2010 - 5:28pm

Packers were ranked high last year, despite horrible special teams. I think Special Teams only account for 1/7th of a team's overall DVOA.

17
by Scott C :: Tue, 10/05/2010 - 5:50pm

The average value of DVOA for special teams is roughly 1/7 of the total. But that is already accounted for in the percentages. A -20% ST DVOA means it was so bad so far that it wiped out their offensive value so far.

Look at the above. SD has this line:

26.0% 15 13.4% 10 2-2 20.8% 7 -26.0% 1 -20.7%

26% = 20.8 -(-26.0) -20.7%.

So if their ST DVOA drops over time to -5%, the O stays at 20%, and the D moves down to -20%, then their overall DVOA will be 35%.

39
by Chrisss (not verified) :: Tue, 10/05/2010 - 8:38pm

The problem here is that while special teams may only count as 1/7th of the DVOA, special teams have cost the Chargers 2 games already this year. Special teams could derail SD regardless of their DVOA. That said, if they clean it up, they could be the best team in the league. Most yards per game, fewest allowed. They would be downright dangerous. Time will tell if they can clean that mess up.

56
by Scott C :: Tue, 10/05/2010 - 10:14pm

yes, a season long -20% ST DVOA would be devastating.

Not happening though.

6
by n8 (not verified) :: Tue, 10/05/2010 - 5:25pm

Packers - Bonamego back?

9
by ST-less in San Diego (not verified) :: Tue, 10/05/2010 - 5:30pm

So if San Diego could improve their league worst Special Teams DVOA to the level of the 2nd worst Special Teams DVOA, they might have the highest total DVOA. Come on Chargers Special Teams, rise from abhorrence to mere wretchedness!

10
by wonkothesane1 :: Tue, 10/05/2010 - 5:33pm

I like the estimated wins of 3.1 for Kansas City. Nice touch.

66
by Tundrapaddy (not verified) :: Tue, 10/05/2010 - 11:10pm

'ESTIMATED WINS uses a statistic known as "Forest Index" that emphasizes consistency as well as DVOA in the most important specific situations: red zone defense, first quarter offense, and performance in the second half when the score is close. It then projects a number of wins adjusted to a league-average schedule and a league-average rate of recovering fumbles. Teams that have had their bye week are projected as if they had played one game per week.'

11
by Led :: Tue, 10/05/2010 - 5:34pm

It's like someone chopped a piece of the Colts off and the fragment regenerated itself into the Texans.

33
by Mr Shush :: Tue, 10/05/2010 - 7:58pm

Except that the Texans run arguably better than they pass, and stop the run parsecs better than they stop the pass. It's more as if Ian McDiarmid ordered a clone army of 2004 Chiefs, and they've just come ready for delivery.

12
by B :: Tue, 10/05/2010 - 5:34pm

Patriots the best team in the league? All right, I'll take it. Big "so far" asterisk? I'll just ignore that part.

13
by jebmak :: Tue, 10/05/2010 - 5:37pm

I don't get why you sort by DVOA instead of DAVE if DAVE is supposed to be more accurate.

21
by BucNasty :: Tue, 10/05/2010 - 6:08pm

That was my feeling when it was just DAVE and VOA, but now that there's opponent adjustments (however watered down) I prefer to give DAVE the could shoulder and just go by DVOA.

28
by JSA (not verified) :: Tue, 10/05/2010 - 7:04pm

It would be great if these tables were sortable

63
by Karl Cuba :: Tue, 10/05/2010 - 11:02pm

Agreed

91
by billsfan :: Wed, 10/06/2010 - 9:34am

Also agreed. But the FO web interface has always been buggy for me (for example, there's a big red error box that appears on the top of the page after I post a comment), and if I really want stuff sorted, I'll just import it into Excel.

(I also like the Eagles)

14
by Will Allen :: Tue, 10/05/2010 - 5:41pm

Now that the opponent adjustments are kicking in, the Vikings defensive rank is improving, as I expected, and the offensive rank seems about right. The schedule is brutal for the next four games, and unfortunately a split may be the worst they can survive. Then again, nobody else in the division looks all that great (the Packers may end up being the 4th best team in the league, but I don't think they are close to that now), so maybe the Vikings could survive a 2-5 start. However, that seems unlikely, to say the least.

15
by Scott C :: Tue, 10/05/2010 - 5:44pm

After replacing 50% of their coverage teams, I'm pretty sure the Chargers won't be playing at -20% ST the rest of the way. Its only one game, but the coverage units actually stayed in their lanes on Sunday and looked like professionals.

The ~20% DVOA offense is expected, and watching the D things are definitely more solid there than the last few years on the D-line and secondary. Injury issues at LB though. The pre-season projection for the Chargers D is looking to be off.

With all the talk of 'no dominant teams' so far, it would not surprise me if 4 games from now we're looking at the AFC with at least 3 of:
SD, BAL, NE, PIT, IND

looking very strong. IND has a tougher schedule than the others though.

There's going to be a lot of movement up and down these rankings in the next couple weeks, though that is usual. The median variance this year is 1% higher than last year but 2% lower than 2008.

16
by Rivers McCown :: Tue, 10/05/2010 - 5:49pm

Houston is clearly ranked too low because Matt Schaub will never get hurt again just to spite FO. Calling my uncle up and asking him to make power rankings by which teams have the funniest names is way better than this. Lol u guys r dummmmb, Trolololol Houston wins it all.

I always wanted to do that.

19
by Chris84 (not verified) :: Tue, 10/05/2010 - 5:57pm

San Diego has a better defense than the Steelers? Um, what???

25
by Sjt (not verified) :: Tue, 10/05/2010 - 6:50pm

San Diego's defense has allowed only 10.75 points a game, 4 TDs and 5 FGS in 4 games.

Four of the FGs followed turnovers which gave the opposition the ball in Chargers territory. The other was a 50 yarder at the end of a 31 point beatdown of the Cardinals, who otherwise would have been shutout by the defense (their TD was a fumble recovery).

Of the 4 TDs, one was a garbage time TD at the end of the Jags game. Two others were the result of drives which started in Charger territory after the Offense fumbled. The only legitimate long drive they have given up all year was one big run by Jamaal Charles.

Not that it means anything for DVOA purposes, but they are also # 1 in the league in total yardage allowed, # 7 in rushing defense, #3 in passing defense, and have the lowest opponent passer rating. They are within 1 sack and 1 int of being the league leader in both categories. And they also have a safety and a defensive TD.

Its true, all of these thing happened against some not very good teams (except, apparently, the Chiefs). The Charger D has nowhere to go but down, since they will soon play tougher teams and they are dealing with injury and suspension issues. But yes, they have been damn good so far.

36
by Mr Shush :: Tue, 10/05/2010 - 8:05pm

"Not very good" is, um, a polite way of putting it. KC may be a good team (I still have my doubts) but they're certainly not a good offense, especially not in torrential rain. The Jags offense may be as good as ok. May be. The Seahawks offense is dismal. The Cardinals offense defies description.

I'm not saying the Chargers have a bad defense. They very probably don't. I'm just saying that if I went out there with 9 tackling dummies and an inflatable sheep, the Cardinals might struggle to put a long drive together.

41
by Sjt (not verified) :: Tue, 10/05/2010 - 8:53pm

And DVOA reflects that. If they had played as well as they have against the Pats or someone like that their defensive DVOA would be even better. That's the whole point of this exercise.

48
by RickD :: Tue, 10/05/2010 - 9:42pm

What people are saying is that the "D" in DVOA is not playing much of a role yet in the DVOA rankings. San Diego clearly has the best VOA but they've yet to face an offense of a high caliber.

I suspect the Jets, Ravens, and Steelers all have better defenses than the Chargers. And that will eventually be reflected in the DVOA ratings.

61
by maestro876 :: Tue, 10/05/2010 - 10:39pm

I think you're right. But at the same time, I think there's a decent chance that the Chargers' defense is better than the projections gave it credit for. Perhaps significantly so.

67
by Big Johnson :: Tue, 10/05/2010 - 11:25pm

im gonna agree with the skeptics on this one. The game against KC was ferocious towards the quarterbacks. Rivers had his worst game of the year and cassel was completely useless that game with about 60 yards total passing. So every game from here on out when they dont play in a hurricane it will make the chargers D look that much better because they hamstrung a decent offense. In the same right i think that game hurts their offensive DVOA and it is very likely they have a better offense than is indicated. SD offense should trend upwards while their defense should trend downwards. And i dont care who you are, a special teams ranking should never be -20+. it should never be -10+. The teams they have played have been all the best teams at special teams, definately contributing to the outrageous ranking. Seattle is first, jacksonville is second, KC is third, and arizona sucks at special teams (26th i think).

20
by ammek :: Tue, 10/05/2010 - 6:00pm

It is safe to say that the Giants' defense has been inconsistent.

After four weeks, has any team had a worse rushing DVOA than the 2010 Broncos? The huge negative number next to DEN looks completely weird, after all those years of success.

94
by Mark S. (not verified) :: Wed, 10/06/2010 - 10:57am

Actually, I think the giants' D has looked pretty good so far. Obviously they dominated the Bears. Tennessee did next to nothing until garbage time (lost that game on penalties/field position/ST), and aside from one drive I don't remember Carolina doing much either, despite having great field position all game.

Obviously, they got absolutely smoke in the Colts game, but I put that down to a deeply stupid gameplan (having one LB and 6 DBs on the field basically all first half) that blew up in their faces like an Elmer Fudd cigar. Other than that game, I think they've looked solid, especially when you consider they give up the best starting field position to opponents in the league (Kickoff and punt units are awful).

22
by ammek :: Tue, 10/05/2010 - 6:11pm

So the Broncos win, and their playoff odds get worse.

Josh McDaniels, probability simulations hate you.

26
by Treima6 (not verified) :: Tue, 10/05/2010 - 6:50pm

That's likely because the Chargers were making the Cardinals look like a Pop Warner squad on both sides of the ball Sunday, resulting in a MUCH brighter outlook for their playoff odds, and of course Kansas City is still 3-0 and the division leaders. Their win was relatively unimpressive and wasn't against a division foe, so it would not have affected their playoff odds much either way, but the possibility of the AFC West snatching both Wild Cards is extremely thin, so Denver's gotta be successful and hope the wheels fall off of Antonio Gates and, of all people, Mike Tolbert(!)

37
by Arson55 :: Tue, 10/05/2010 - 8:10pm

You mean the Cardinals aren't a Pop Warner squad? Weird.

123
by JIPanick :: Thu, 10/07/2010 - 12:54am

They are a qbs.pop(Warner); squad.

124
by tuluse :: Thu, 10/07/2010 - 3:14am

I found this way more amusing than I probably should have.

24
by tunesmith :: Tue, 10/05/2010 - 6:44pm

Surprised to see the Broncos' defense ranked worse than their special teams. The defense has put up some pretty good performances this year, I thought.

104
by MarkV :: Wed, 10/06/2010 - 12:46pm

I am surprised their special teams has people below them.

But, the Defense ranking that low is a bit surprised. The defense played ok in 3 games i would have thought.

27
by battlered90 (not verified) :: Tue, 10/05/2010 - 7:03pm

Looking forward to seeing the Seahawks plummet down these rankings. I'm guessing they are being buoyed by the number 1 special teams ranking. I wouldn't count on two kick return tds every four weeks.

29
by Theo :: Tue, 10/05/2010 - 7:04pm

Pittsburgh is that good? Wow.
I read PFP, but wow.

30
by South Bay Bengal (not verified) :: Tue, 10/05/2010 - 7:13pm

wish i could argue with the DVOA this week, but the Bengals suck. dammit.

31
by Lou :: Tue, 10/05/2010 - 7:47pm

I can't disagree with the rankings. It does seem odd that the Bears are ranked below all 3 of the teams they beat.

35
by Kal :: Tue, 10/05/2010 - 8:00pm

Odd, but fair given the games. The Lions should have beaten them but for a rule that was bad, the Cowboys outplayed them but had some odd luck, and the Packers beat themselves with penalties (which don't show up on DVOA).

This was the whole point of DVOA - that a team that wins is not necessarily showing that they're a better team over the team that loses, just that they were better that day.

That being said, it's incredibly depressing for this Bears fan to think about. I was really hoping that with Martz, Cutler was going to turn things around. Instead I can watch Kyle Orton ripping defenses apart with Brandon Lloyd as his receiver and wonder where the hell things went so wrong.

38
by tuluse :: Tue, 10/05/2010 - 8:21pm

You might want to refresh your memory of what DVOA looked like after week 1.

http://www.footballoutsiders.com/dvoa-ratings/2010/week-1-dvoa-ratings

The Bears outplayed the Lions by a significant margin, and were extremely unlucky in where they fumbled.

Also, it's easy to see where things went wrong. They have no talent on the offensive line, and no depth behind what is already a backup quality starting unit.

34
by The Hypno-Toad :: Tue, 10/05/2010 - 7:59pm

It's interesting... I have been very, very skeptical of the local Bronco fans who have assserted that the Broncos defense has "looked great" but I also find it hard to believe that there are only 4 defenses worse than the Broncos this season. So once again I find myself a man between worlds.

70
by PerlStalker :: Wed, 10/06/2010 - 12:28am

I wouldn't go as far as "great" but better than last is pretty accurate. The thing is, as bad as the defense was last year, even 20-25 ranking looks great by comparison. I also think their defensive DVOA will look better as opponent adjustments are more heavily weighted.

40
by Thok :: Tue, 10/05/2010 - 8:51pm

I'm disappointed the 49ers rose in the rankings this week. I was hoping they'd still be last and you'd make a special hell category for them as per last week.

Although I guess hell is other people's turnovers, and there's a limit to how much you can punish them for that.

43
by chemical burn :: Tue, 10/05/2010 - 9:02pm

It's strange I feel like there are really no good teams this year so everyone is surprised that their favorite team ranks so high. I for one cannot believe the Eagles rank 7th. This team blows. Certainly, they look to be the most inept Eagles team in a decade. The defense is really pitiful - but I guess only for chunks of each game and not the entire time. Still, they are average in DVOA? And their offense, on that has looked helpless (not bad, just helpless) for at least 1 complete game ranks 6th? I feel like 2010 is going to produce one of the worst Superbowl winners in history. And that it could be literally any team at this point. If we can't truly rule out teams like the Lions and Cardinals at this point, anything seems possible.

45
by JIPanick :: Tue, 10/05/2010 - 9:28pm

A good but not great champion from a blurry year wouldn't really be that bad as recent champions go, in my opinion. Worse is when there is a great team (or two or four or whatever) and some weak team sneaks into the playoffs and steals the title from them.

Look at 2007; there were four legitimately great teams in New England, Dallas, Indy, and Green Bay but instead the Super Bowl rings went to an average Giants team.

Other examples include 2001 (loaded Rams, indifferent Pats) and 1987 (Niners and Skins). Arizona almost pulled it off two years ago too.

47
by chemical burn :: Tue, 10/05/2010 - 9:35pm

I actually mean there's real chance of not just an average team (like the 2001 Pats or 2007 Giants) taking the title, but a real shitbomb of a team sneaking into the playoffs and then making a run at the title because there's only mediocrity standing in their way.

51
by Mr Shush :: Tue, 10/05/2010 - 9:56pm

Someone presumably has to win the NFC West, right? I mean, I know the league should take that division's playoff berth away, but I don't think they have.

I seem to remember a PFR simulation a while back which suggested that about one time in 10,000 the Superbowl would be won be the actual out-and-out worst team in the league.

52
by Will Allen :: Tue, 10/05/2010 - 10:01pm

Well, "average" is a hard word to use in regard to something with as many moving parts as an NFL team. I might use "weird" for the 2007 Giants; mediocre in some, maybe most, areas, but joined to a defensive front which, come playoff time. was primed to just stomp the feces out any primates put in their path. Kind of like some World Series winners who fail to get 90 wins in the regular season, but because of two great starters, a relief man, and three or four hitters, are able to prevail over hundred win teams in the playoffs. Maybe "average" is correct in the technical sense, but the word may imply more luck than is actually the case.

55
by chemical burn :: Tue, 10/05/2010 - 10:13pm

Instead of average, how about just "not terrible?" That's all I meant. And that this year a "terrible" team could win it. Like if the Rams took the NFC West and then beat the Cowboys and (I don't know, Vikings?) to go to the big game. And then all they had to beat was the Jets or Chiefs or Pats something? None of that scenario sounds like "no freakin' way!" Or if the Seahawks or Chiefs or Jaguars did the same thing?

65
by Will Allen :: Tue, 10/05/2010 - 11:08pm

Well, predictions are hard, especially about the future, but I can't see any unit of the Rams, if they were to win that terrible division, being anywhere near as good as the Giants defensive front was, so if the Giants hoisting the trophy was strange, the Rams doing so would be like hitting the lotto.

75
by t.d. :: Wed, 10/06/2010 - 1:50am

That Giants team excelled at rushing the passer, and they were adequate throwing the ball. The 2008 Cards excelled throwing the ball, and had a decent pass rush. Those are the two most important skills in the modern NFL, and they can cover a lot of warts.

97
by Mark S. (not verified) :: Wed, 10/06/2010 - 11:25am

Somebody on these boards made a really interesting point after the Steelers, Colts and Giants won SuperBowls 40-42 although now that I think about it, they may have been talking about the 05-06 Steelers team that won as the sixth seed in the playoffs).

Anyway their point was that while these teams were not great in the years they won those SBs, they did have dominant seasons in preceding or successive years - i.e. the Giants were only "OK" the year they won the SB, and much better the following year (they were 11-1 and first in DVOA before Burress shot himself) and the Colts were only "pretty good" in 06, the year they went on to win it, although they were dominant the previous year; same for the Steelers, etc. The crux of the point being that there was a mini-trend of teams that might be really good or dominant in a given year, but their statistical dominance did not correspond to the years they actually won titles.

44
by Vince Verhei :: Tue, 10/05/2010 - 9:23pm

Fodder for the "No great teams" discussion:

In Week 4 2010, there are three teams over 30% and the best team (New England) is at 34.1%

In Week 4 2009, there were seven teams over 30%, three over 40%, and the best team (New Orleans) was at 58.5%.

In Week 4 2008, there were four teams over 30%, two over 40%, and the best team (Baltimore) was at 60.6%

In Week 4 2007, there were five teams over 30%, three over 40%, and the best team (New England) was at 72.4%

In Week 4 2006, there were five teams over 30%, four teams over 40%, and the best team (San Diego) was at an ungodly 83.8%.

In Week 4 2005, there were six teams over 30%, five teams over 40%, and the best team (Cincinnati) was an even more ungodly 92.0%.

That's as far back as the weekly DVOAs go, but it's clear that the cream of this year's crop is decidedly less creamy than usual.

46
by chemical burn :: Tue, 10/05/2010 - 9:34pm

Also, one of those 3 over 30% this year (KC) feels like it almost certainly going to crash and burn at some point - or maybe just get hammered by opponent adjustments.

50
by RickD :: Tue, 10/05/2010 - 9:49pm

I suspect it'll be the latter. They've got a schedule filled with teams from the AFC West, NFC West, AFC South, and the other last place AFC teams (Browns and Bills).
While they probably won't leave Indy undefeated, they should get to 8 wins relatively easily. (Jags, Bills, Cardinals and Raiders all visit KC - that alone should get them to 7 wins.)

With the Rams also in first place, the people of Missouri must be horribly confused.

53
by Mr Shush :: Tue, 10/05/2010 - 10:06pm

I think it might be a schedule artifact, actually, coupled with the fact that the best team in the league has played its first four games with its third and fourth string quarterbacks under centre, but will probably not have to do so again, while the second or third best team in the league has missed a load of time from its best player, who will also presumably be back. There just seem to have been an unusually high proportion of games between two good teams and games between two bad teams. The Jets have played the Ravens, Patriots and Dolphins. The Steelers have played the Ravens and Falcons. The Ravens have played the Jets and Steelers. There's not enough connectivity yet for that sort of thing to smoothe out. I think as time goes by we'll see something like Steelers otherworldly, Ravens and Jets elite, Falcons, Patriots, Packers, possibly Saints and maybe Cowboys if their offensive line stays healthy very, very good.

57
by chemical burn :: Tue, 10/05/2010 - 10:15pm

You're being way too abstract in the beginning there. I guess the "best" team is the Steelers - but what on earth is the 2nd or 3rd best team you're referencing? And the "best player" in question?

64
by tomvons (not verified) :: Tue, 10/05/2010 - 11:02pm

I'd guess the 2nd/3rd is the Ravens missing Ed Reed...?

71
by Kurt :: Wed, 10/06/2010 - 12:32am

Has to be Revis.

80
by Mr Shush :: Wed, 10/06/2010 - 7:06am

I meant Revis, yes.

Although of course Reed being out for the Ravens is a huge deal too. I'm just not sure he's obviously more important than Ngata or Suggs, in the way that Revis is obviously more important than . . . Harris? Ferguson?

72
by Kurt :: Wed, 10/06/2010 - 12:35am

(double post)

59
by Led :: Tue, 10/05/2010 - 10:20pm

You left out the Chargers. For my money, the best teams are, in no particular order, Steelers, Ravens, Jets, Pats, Chargers, Colts. With the Texans, Saints, Packers, Falcons in the next tier. This changes if the Pats actually trade Moss for picks this year, as is being reported in some outlets. I think the Pats take a big hit to their offensive productivity without Moss on the field changing the way teams play defense.

81
by Mr Shush :: Wed, 10/06/2010 - 7:30am

You're right, the Chargers may well be pretty good. Ok.

Tier 1: Steelers (with Roethlisberger)

Tier 2: Ravens, Jets

Tier 3: Patriots, Chargers, Falcons, Packers, maybe Saints, maybe Cowboys

Tier 4: Texans, Colts, Dolphins, Giants, Bears if Cutler survives, maybe Titans, maybe Chiefs

Tier 5: Jaguars, Broncos, Eagles, Redskins, Vikings

Tier 6: Bengals, Browns, Buccaneers, Rams, Lions, maybe 49ers (I have no idea what the hell is up with that bunch of schizoids - they could be higher or lower)

Tier 7: Bills, Seahawks, Panthers

Tier 8: Raiders

Tier 9: Cardinals

I think that's about how I see it. I'm bound to be wrong about a fair amount of that, of course, but that's my current best guess. Where I disagree substantially with DVOA, I'm pretty sure it's mostly about the "D" being (inevitably) low-strength and inaccurate this early in the season.

153
by leviramsey (not verified) :: Sat, 10/09/2010 - 11:39pm

Interesting point on the bad-vs.-bad and good-vs.-good matchups early.

When Bert Bell was scheduling the NFL, he would typically have the better teams play each other early and likewise with the worse teams, on the theory that it would leave the most teams in contention (and thus able to sell tickets) midway through the season.

Perhaps with this being an uncapped year, Goodell took a page from the Bell playbook?

58
by Scott C :: Tue, 10/05/2010 - 10:17pm

No dominating teams _so far_.

By week 8 it might look not too different than half the years above.

60
by chemical burn :: Tue, 10/05/2010 - 10:29pm

Not in terms of DVOA - teams only end up with 70% or 90% DVOA because of a small sample size and weak opponent adjustments. I can safely say no team is going to look "ungodly" or even dominant this year - because no team has played dominant for a quarter of the season. Some teams have played pretty good, but nothing above and beyond that...

62
by JIPanick :: Tue, 10/05/2010 - 11:00pm

There's certainly a possibility that a team off to a decent but unspectacular start could put the pedal down and take off to a dominant season. I think the no longer QB-less Steelers are the best candidate.

79
by ammek :: Wed, 10/06/2010 - 4:34am

Surely you have weekly DVOAs from 2003-04, at least? I mean, this site was alive back then.

68
by Treima6 (not verified) :: Tue, 10/05/2010 - 11:33pm

The three worst runningbacks in DVOA terms are my three fantasy backs. http://fantasy.nfl.com/league/11171/team/1

/weeps

69
by Treima6 (not verified) :: Tue, 10/05/2010 - 11:33pm

Er, whoops. I mean in DYAR terms.

73
by ebk95 (not verified) :: Wed, 10/06/2010 - 12:39am

I'm curious as to what the Bears' offensive DVOA was for this week and how it compares to some of the worst offensive performances all time?

As a Bears fan I can remember some shockingly bad offensive performances (Thanksgiving 2004 comes to mind) but this past game was easily the worst :(

74
by Independent George :: Wed, 10/06/2010 - 1:10am

Can any Chargers fans explain how they have a worse Special Teams DVOA than both the Giants and the Dolphins? Are they punting the ball backwards into their own end zone?

77
by Vince Verhei :: Wed, 10/06/2010 - 2:17am

See last week's DVOA ratings commentary. San Diego just gave up the best single-game special teams DVOA in history to Seattle, and in Week One the Chiefs averaged 32 yards per punt return with a 94-yard touchdown against the Chargers.

110
by Ben B. :: Wed, 10/06/2010 - 3:19pm

A chronicle of the woes:

Week 1 against the Chiefs: 3 touchbacks on punts from inside the KC 45 (my favorite was the touchback from the 33), a 32 yard punt return ending with the punter making a nice tackle, a 24 yard punt return, and a 94 yard TD punt return.

Week 2 against the Jaguars: A punt blocked (out of two total attempted).

Week 3 against the Seahawks: (covered by Vince) 2 kickoffs returned for TDs, another that might have been except Leon Washington slipped on his own. One 31 yard punt return. A kickoff return was fumbled.

Week 4 against the Cardinals: A missed field goal and a kickoff out of bounds.

78
by Alternator :: Wed, 10/06/2010 - 3:01am

New England is clearly ranked too high because no way are there nine worse defenses so far this year. Blind homerism is way better than this. Dear God have mercy and let the young guys mature fast.

82
by PatsFan :: Wed, 10/06/2010 - 7:53am

Looks like "Zero catches from Moss" will be true for NE every week from now on...

84
by ChaosOnion (not verified) :: Wed, 10/06/2010 - 8:16am

The average height of NE skill players is now hovering at about 5'9".

I had DeSean Jackson and Randy Moss in my fantasy league. Maclin was just sitting around on the waiver doing nothing but catching a TD every week, so I added him for some insurance. The strength of me team is now whatever defense is playing ARZ/CAR/BUF this week.

83
by andyavatar (not verified) :: Wed, 10/06/2010 - 7:57am

What is wrong with your combined score is that it combines distributions of unequal dimension. Your defensive DVOA percentages have a far different mean and standard deviation than your offensive DVOA. The same for special teams. The scatter pattern is also different. You can't in good conscience compare them straight up. At the very least, you ought to be using the standard deviation to combine these two percentages to get the deviation from the mean for the score. A simpler way is to adjust using a rule of thumb, i.e., defensive DVOA has one half the variance that offensive DVOA has, or something like that, but you're stat guys. You ought to be able to handle a second order metric.

As for combining the offensive and defensive percentages, you're all over the map. You don't report your method of combination, so one has to guess at the system you are using which, frankly, doesn't seem very systematic. Even if I subtract the def DVOA from the offensive DVOA, I don't get the scores or the rankings you report. I would think you would just convert the percentages to a deviation from the mean, then add them, or, you would adjust the defensive percentages to fall on a distribution determined by the mean and standard deviation of the offense's distribution. You could add in special teams (corrected) directly or weight it, depending upon your point of view.

88
by DaveRichters :: Wed, 10/06/2010 - 9:10am

You don't report your method of combination, so one has to guess at the system you are using which, frankly, doesn't seem very systematic. Even if I subtract the def DVOA from the offensive DVOA, I don't get the scores or the rankings you report.

The combination method is just additive, Offense minus Defense plus Special Teams. This does equal the total reported, perhaps you were not adding special teams when you checked this? Or perhaps you were using DAVE instead of DVOA as the total?

Your point is correct, I've made it here before, but it will fall of deaf ears I'm afraid. The authors of this model do not feel obliged to explain or justify anything about this model beyond an extremely superficial level, so critiquing it is pointless.

93
by andyavatar (not verified) :: Wed, 10/06/2010 - 10:41am

That has been my experience also. There is a serious flaw, for example, in DVOA in that it overvalues interceptions on behalf of defenses. Thus, last year, Green Bay was rated at or near the top, yet, they had given up 30 touchdowns through the air. Nearly two per game! The problem seemed to be interceptions, but nobody seemed willing to address the issue, or even seriously discuss it. the issue for them seemed to be my dissatisfaction. Their only remedy was to challenge me to prove it. I asked for the relevant stats, and they declined to provide them. So much for scientific neutrality and a willingness to question your own assumptions.

100
by DGL :: Wed, 10/06/2010 - 11:48am

IANAS (I Am Not Aaron Schatz), but he has discussed and posted about how he evaluates DVOA during the season and the offseason; based on that, I suspect he experimented with lowering the importance of defensive interceptions to the DVOA calculation, and found that on the whole it gave a worse (poorer correlation with wins) result.

102
by RichC (not verified) :: Wed, 10/06/2010 - 12:00pm

Interceptions are worth a lot in DVOA because they are worth a lot in football.

now, I agree that FO has been terrible with dealing with comments/criticizm of their lack of transparancy, but c'est la vie.

90
by qed :: Wed, 10/06/2010 - 9:23am

As for combining the offensive and defensive percentages, you're all over the map. You don't report your method of combination, so one has to guess at the system you are using which, frankly, doesn't seem very systematic.

Offensive DVOA, plus (-1 times Defensive DVOA), plus Special Teams DVOA produces the Total DVOA. Not sure if that was your question or not. If you aren't familiar with the way the stats are computed the primer is here:
http://www.footballoutsiders.com/info/methods

The 0% point is set by the long-term average results for each type of play, although I'm not sure what time period those long-term averages are calculated over. I don't see the problem with weekly or yearly offensive and defensive DVOA having different distributions, it's entirely possible that any particular year happens to be a good or bad year for offenses or defenses.

Since the idea of the total DVOA is to try to estimate the overall quality of the teams I don't think you want to normalize the offensive and defensive DVOA before combining them... if a team happens to be an extremely low defensive DVOA it should be because their defense is playing extremely well, and you wouldn't want to suppress that outlier just because their doesn't happen to be any team with a great offense DVOA.

92
by andyavatar (not verified) :: Wed, 10/06/2010 - 10:32am

"Since the idea of the total DVOA is to try to estimate the overall quality of the teams I don't think you want to normalize the offensive and defensive DVOA before combining them... if a team happens to be an extremely low defensive DVOA it should be because their defense is playing extremely well, and you wouldn't want to suppress that outlier just because their doesn't happen to be any team with a great offense DVOA."

The problem is that a variant that detracts from a total ought to be measured on the same scale as one that adds to it or you have unequal contributions. If offensive DVOA has a mean of 8 and a stddev of 18, while the defensive numbers have a mean of 0 and a stddev of 12, then one point of offense is worth much more than one point of defense because the defense must first offset the differences in the mean before they can offset the offense. Moreover, every percentage point of defense must offset more than one percentage point of offense since offense is freer to vary than defense. Therefore much of the true variance in the defensive number is lost reconciling the different means and stddevs. Is this statistically fair?

For example, with the Patriots, their 23rd rank on defense is less than one third as important as their 1st rank on offense. Does this reflect variance in the real world? Is a bottom ten defense only one third the contributor to failure as a top ten offense is toward success?

95
by DeltaWhiskey :: Wed, 10/06/2010 - 11:01am

Just push the frickin' "I believe" button.

96
by qed :: Wed, 10/06/2010 - 11:17am

I would argue that offensive and defensive DVOA are already measured on the same scale since they are both drawn from the same set of plays. You can argue that the fundamental formula doesn't accurately measure what its supposed to. But if you start from the assumption that DVOA is accurate then I think it's best to combine the raw DVOA scores. It should definitely be more accurate than, say, just adding the rankings in each category and saying that the team with the lowest combined ranking is the best. I wouldn't say that the Patriots 23rd ranked defense is "one-third as important" as their 1st ranked offense, I'd say that their offense is substantially better than their defense is bad. But if I want to compare the Patriots to the Chiefs, I'd rather compare the differences in their DVOA scores (Pats offense is about 30 points better, Chiefs defense is about 30 points better, so they cancel out) then their rankings (Chief's offense is 9 ranks below the Pats, Chief's defense is 18 ranks better, so the Chiefs are a substantially better team),

98
by AudacityOfHoops :: Wed, 10/06/2010 - 11:33am

"If offensive DVOA has a mean of 8 and a stddev of 18, while the defensive numbers have a mean of 0 and a stddev of 12, then one point of offense is worth much more than one point of defense because the defense must first offset the differences in the mean before they can offset the offense."

Maybe I am misunderstanding, but I don't see how this is true at all. How would you solve this "problem?" By subtracting 8 points from all the offensive DVOA's, right? If you do that, does anyone's rank change? No, because that 8 point adjustment is applied to all teams equally.

"Moreover, every percentage point of defense must offset more than one percentage point of offense since offense is freer to vary than defense. ... For example, with the Patriots, their 23rd rank on defense is less than one third as important as their 1st rank on offense. Does this reflect variance in the real world? Is a bottom ten defense only one third the contributor to failure as a top ten offense is toward success?"

I think that is an open question, but not a clear black mark against the method. Given the rule changes in recent years that seem to tilt the field toward the offense, it seems possible that great offenses can do some things that are nearly impossible to stop, making defensive quality less important. For an extreme example, consider free throws in basketball. Every team has a FT% (offense) and an oppFT% (defense). FT% varies more across teams than oppFT%, which makes sense, because oppFT% is mostly luck (getting to face bad FT-shooting teams) combined with tending to foul the right/wrong guys. You wouldn't want to normalize them when making a combined metric, because FT% really is more important.

That said, it's something that needs to be tested, and not just assumed to be true. And as you said, FO hasn't been very forthcoming in either making data available for testing, or writing articles about tests they've done.

101
by Aaron Schatz :: Wed, 10/06/2010 - 11:50am

If you feel like you have a way to improve the accuracy and quality of DVOA, or any of our other stats, we're very happy to listen to your ideas. Let me see if I can provide some guidelines that might help you get your point across. (This is a general comment, not addressed at any of the people who made criticisms in this specific thread.)

1) Send your comments in an e-mail rather than putting them in an article discussion thread.

2) Make your statement as a constructive suggestion rather than a snarky criticism.

3) Specifically direct your comment towards one FO writer rather than "the FO mass." If you want to talk about DVOA, ask for me. If you want to talk about injury data, ask for Barnwell. And so forth.

(For example, to "andyavatar," if you are complaining that "they wouldn't listen to me," I don't understand what you are talking about. Who is they? Me? Barnwell? Even from the open mailbag e-mail account, we tend to all separately sign our names to responses. Again: Not a hive mind.)

4) We've always been very open about the fact that nobody on the FO staff is a trained statistician. We are writers first, statisticians second. (Or, in the case of Mike Tanier and Doug Farrar, writers first and scouts second.) If you want to talk about statistical theory and how it relates to the numbers on this site, your comments should be very clear and basic.

5) Accept the fact that your suggestion may have already been made by a dozen people in the seven years this website has been online. Sometimes, there is a good reason why things are as they are, and it isn't because we do not like you personally.

6) Have patience. It is often hard to get to things in-season.

103
by AudacityOfHoops :: Wed, 10/06/2010 - 12:26pm

Thanks, I probably should have added "I'm not sure HOW to test it, exactly." to my previous post.

FYI, I just clicked the more info link from this article (above the charts), and I got an error page.

105
by Aaron Schatz :: Wed, 10/06/2010 - 1:22pm

Whoops. Typo. Fixed now.

107
by nat :: Wed, 10/06/2010 - 1:52pm

Send your comments in an e-mail rather than putting them in an article discussion thread.
Personally, I hope people do both. I would hate to miss out on some of the ideas people come up with. There's a lot of claptrap, true, but there are occassional gems.

108
by DeltaWhiskey :: Wed, 10/06/2010 - 2:21pm

Agreed, and I don't think Aaron means that quite of a rigid interpretation, what I think he's referring to, having been a perpetrator of this myself, is not beating the dead horse and expecting anyone in the hive to respond and listen. This sort of discussion is vital to FO's metrics, but eventually, if they're going to incorporate these ideas, they have to be communicated directly to a member of the hive.

109
by DaveRichters :: Wed, 10/06/2010 - 3:12pm

2) Make your statement as a constructive suggestion rather than a snarky criticism.

Don't be such a baby.

111
by Dean :: Wed, 10/06/2010 - 3:20pm

Well, do you actually want to affect positive change? Or do you just want to try and impress random strangers on the internet with how witty you believe you are?

112
by DeltaWhiskey :: Wed, 10/06/2010 - 3:52pm

Can't I do both?

113
by billsfan :: Wed, 10/06/2010 - 5:17pm

I've been trying to do the latter for the better part of 20 years.... why stop now?

(I also like the Eagles)

114
by DaveRichters :: Wed, 10/06/2010 - 5:27pm

Well, do you actually want to affect positive change? Or do you just want to try and impress random strangers on the internet with how witty you believe you are?

I guess I just want them to not be babies. I am a scientist, my work is under peer review, and I don't have the luxury of ignoring criticism when it doesn't come from someone trying to be especially nice to me. I would think that THEY want to make the model better, I don't really care about how good it is.

115
by andyavatar (not verified) :: Wed, 10/06/2010 - 6:37pm

Aaron,

You personally blew me off. I was trying to make a point to Barnwell, and I asked for data so I could demonstrate the issue and you interrupted the exchange and basically said the discussion is over. You never dealt with the problem I addressed and you declared the discussion at an end. I never got a response from Barnwell after that. This happened at the end of last season before the playoffs when I complained about the Packers being rated too highly on defense. What data I had available showed a significant bias. The weeks where the Packers had a monumentally high (or low) DVOA on defense were weeks where they had multiple interceptions. I asked for data from other teams rated highly on defense, but it never came.

As it turned out, the Packers #2 defense didn't exactly play like one in the playoffs. I'm sure you'll say that was anecdotal evidence.

I know its hard to respond to everything that is said when you are traveling between the bunker and Berchtesgarden, but you might want to occasionally acknowledge that some of your readers might know something about variance and central tendencies, and aren't interfering idiots. Even your tone in an email is imperious.

I wouldn't waste any time running after your reputation. It precedes you.

Andrew

116
by AudacityOfHoops :: Wed, 10/06/2010 - 7:08pm

It would be cool to see a couple alternate versions of DVOA that deal with turnovers differently:

A) defensive DVOA where all interceptions/fumbles are merely treated as incompletes/tackles - this would measure how good a team is at preventing the sticks from moving. A team like last year's Packers (apparently) wouldn't do as well here

B) I imagine one could model the probability of an interception, given the location of a pass on the field, down/distance, time, opposing quarterback, seconds elapsed prior to the throw, etc. Then you could find the expected number of interceptions for a defense, and compare that to the actual number. OK, I guess this isn't at all DVOA. But maybe this metric would have shown last year that GB wasn't super awesome at intercepting the ball, they just had more dangerous throws against them by bad QBs? It'd be interesting to look at.

C) along the same lines, maybe interceptions could be regressed to the mean somehow. Admittedly, I have no idea how this one would work.

117
by Big Johnson :: Wed, 10/06/2010 - 7:17pm

andy quit pms'ing. if you think u can put together better stats than the stuff they put on footballoutsiders then do it. They arent forced to put the research in their work. I had a thought when it came to new research with ALY and average yards gained by a runningback. I talked about it for some time and they decided to devote an article to it over the offseason. If they think you are onto something they will look into it, otherwise go make your own stats.

119
by andyavatar (not verified) :: Wed, 10/06/2010 - 9:01pm

I was just responding to Schatz, reprinting for the five hundredth time, his pat response to criticism. Thank God for cut and paste. If you don't want criticism, don't have a comments section and don't include a format for criticism in the body of your article. I, too, am a scientist and his haughty unwillingness to entertain authentic criticism is galling. We all like the site, and many of us would like to point out problems and inconsistencies to make it better. But, he has no interest in any but his own ideas, apparently.

122
by Big Johnson :: Thu, 10/07/2010 - 12:17am

Its unfortunate you feel that way. Ive seen him comment back to people before and ive seen him snub people before. Like i said earlier with the ALY to runningback yards coorelation the staff personally didnt respond to me. But either by reading it and thinking i was on to something or by stumbling on it themselves without reading my comments, they somehow came up with an article devoted to the same exact stat. If you are right with the interceptions overrating defensive dvoa then give more examples of who its overrating before it happens. More cases like the green bay one last year will make your theory stronger. Personally you have made this so well known that there is no way it wont be looked at. Take a deep breath, u have won.

126
by Aaron Schatz :: Thu, 10/07/2010 - 10:45am

I think I remember this conversation with Andy now. If I remember correctly -- and I may not -- my response was that I couldn't take the complaint seriously until he could come up with some examples from other franchises in other seasons. That's generally our response to most criticisms related to one specific team. We want to see evidence that the stats are systematically ignoring something over a period of multiple years with multiple franchises. "Green Bay played like crap in one playoff game in 2009" is simply not a reason to change the system so it is less predictive for all teams in the years 1993-2008.

Or maybe I just straight blew you off. If that's the case, I apologize. I try not to do that much. Sometimes, there's just too much of a time crunch.

(Note: I went into the mailbag and searched for Andy's e-mails, and this is exactly what happened. Honestly, I don't want to be that guy who blows people off for no reason. The exact quote to Andy was: "We're done with the back-and-forth here unless you want to talk about a team other than the 2009 Green Bay Packers. I'm sorry you feel we're being dismissive, but you have to understand how many of these complaints we get about single teams." I'm happy to consider the suggestions if he can think of two or three other teams with a similar issue.)

127
by Revenge of the NURBS (not verified) :: Thu, 10/07/2010 - 11:37am

FWIW, the 2005 Bengals had 31 interceptions, and finished 23rd in defensive DVOA. I'm not sure who's position that supports, but it's an example of another team that lived on turnovers.

129
by andyavatar (not verified) :: Thu, 10/07/2010 - 12:51pm

Aaron,

Thanks for the welcome candor.

I wanted to provide you with other examples but I didn't have the history of your ratings week by week except for the Packers, since that's all that Barnwell sent me. After I explained this to him, you popped in and cut off the conversation. I never got any data to analyze so, needless to say, I couldn't support the contention I was making. I just figured you had no real interest in seeing if your ratings were actually biased.

If you're interested in seeing if there is a problem, I'd be glad to help, but again, I don't have the data. If you want my help, I need to know what the weekly DVOA (all three categories) on defense was for each team for a season. I'm sure it would be easy to pick out candidates. For 2010, already Tennessee seems overrated on defense, but that's just a hunch.

As for the problem of unmatched means and stddevs that contribute to your rankings consistent with this article, that would seem like Statistics 101. You have unequal variances and unequal means and you're directly adding and subtracting points. It makes no sense. You might want to ask yourself how NE, the #1 offense and the #23 defense can possibly be the best team in the NFL. That #1 offense, by the way, only really scored thirteen points in that game against Miami and the week before it only put up 14 points against the Jets. Is that a #1 performance over the two most recent consecutive weeks? And yet, their rating went UP. In statistics class we used to call that obvious "by inspection."

Andrew

130
by ammek :: Thu, 10/07/2010 - 1:54pm

You might want to ask yourself how NE, the #1 offense and the #23 defense can possibly be the best team in the NFL. That #1 offense, by the way, only really scored thirteen points in that game against Miami and the week before it only put up 14 points against the Jets. Is that a #1 performance over the two most recent consecutive weeks?

Scientists operate from facts:

— The Pats' offense scored 20 points last week, not 13. No matter how short a drive may be, it still has to get from the starting point to the end zone. You can't completely dismiss a TD drive just because it's short.

— The Miami and Jets games were not consecutive. New England scored 38 points against Buffalo in the intervening week. It also put up 38 against the Bengals, which you omit to mention.

— Ranking isn't everything. As the commentary points out, the top-ranked teams of 2010 would not be top-ranked in some previous years. Why is it so hard to accept New England as the #1 offense? Even without the return TDs, they're in the top three in points per game. And as has been pointed out above, they're only averaging nine offensive drives a game — one per game fewer than Indianapolis and three per game fewer than San Diego (the other top scorers). DVOA measures efficiency, after all.

132
by andyavatar (not verified) :: Thu, 10/07/2010 - 2:38pm

I'm just cherry picking facts to underline the problem. I don't doubt that New England's offense is good. I've watched three of their games. Still, they're eleventh in yards overall. sixteenth in passing and thirteenth rushing by that standard. They have no running game to speak of, and their offensive line is average. There's no balance as the Jets game demonstrated. They also only put up six points against a mediocre Dolphins defense in the first half with the game in doubt, and before the Fins imploded. Add to that a clearly porous defense that, going into last week was giving up around 27 points a game and you have a statistic that has no basis in actual team performance.

If you want to define efficiency as getting the most out of your players, then the Patriots may indeed be #1. They're clearly not that talented so their 3-1 record must have some other explanation. Maybe that's what DVOA measures, after all. But, FO didn't proclaim them the least inefficient team in the NFL. They said "the Football Outsiders mainframe has proclaimed New England the best team in the NFL." Well, where's the objective verification? In drive stats they're fifth on offense, thirtieth on defense. That sounds about right. But, not by DVOA. Statistics have to reflect patterns in real life. Efficient teams won't beat themselves and, if other teams blunder, they'll win. However, without playmakers, there's no one to generate points. That, to me, is why they looked over matched against the Ravens in the playoffs last year, and against the Jets this year.

I started this by saying that the DVOA totals, inappropriately combine three unmatched distributions. I'll stand by that. Let the chips fall where they may, but at least level the playing field so they don't make erroneous projections.

134
by Jonadan :: Thu, 10/07/2010 - 3:42pm

Still, they're eleventh in yards overall. sixteenth in passing and thirteenth rushing by that standard. They have no running game to speak of, and their offensive line is average.

If the Pats are 13th rushing - that is, better than average - and they have "no running game to speak of", then the NFL as a whole is having a terrible rushing year, and you can't blame them for that. The Pats are first on offense because they score. Even if you take out all the ST/DEF touchdowns for NE while letting everybody else have those, the Patriots would still be 8th in scoring. And they're 3rd in net points, so except for the Jets game the Patriots D has been as good as it needs to be.

They also only put up six points against a mediocre Dolphins defense in the first half with the game in doubt, and before the Fins imploded.

But the 'fins did implode - don't they get any credit for that?

...and you have a statistic that has no basis in actual team performance.

Well, that is the problem with statistics - but it's a problem with any statistics. Saying, "DVOA sometimes uses statistics that don't seem to reflect reality," is just long-hand for "DVOA is a statistical approach."

I started this by saying that the DVOA totals, inappropriately combine three unmatched distributions. I'll stand by that.

If I understand what Aaron Schatz and others have said, the offense and defense are drawn from the same data, so they ought to be matched. Just eyeballing the data we've got here, it looks like there's more variation in how good DVOA says the offenses are, but you can't just "adjust" the variation away. Otherwise you're essentially ranking offense and defense and then averaging those numbers, which is imprecise at best (even if it would, for instance, drop the Patriots, and Colts for that matter).

Special teams is the problem, then. Currently, though, the percentages are small enough - apart from San Diego (terrible) - that they shouldn't affect that much. And then you've got teams like Seattle which are literally being carried by their special teams. I have no idea how the percentages are derived to begin with, but at some point you just have to leave well enough alone or you end up skewing everything.

135
by ammek :: Thu, 10/07/2010 - 4:05pm

I'm not sure what you're doing on this site if you think that total yardage, without any context or qualifiers, is "objective verification" of performance. The Patriots are #1 in third-down conversions, fourth in turnover differential, near the top in red-zone efficiency on offense.

And you continue to misread and misinterpret FO's stats. Yes, New England's offense is #5 in yards per drive. But it's #2 in DSR — and if this site and DVOA have preached anything at all, it's the importance of success rates. (Even Brian Burke is onto it now!)

In addition, I'd say that what the DVOA rankings (which are headed, in the drop-down menu, 'Team Efficiency') claim to measure is a lot closer to "least inefficient team in the NFL" than to your misquote about the mainframe.

Your empirical assertions aren't even very convincing. How does a team with "no running game to speak of" end up ranked #13 in yards? How exactly is it "clear" that the Patriots "aren't that talented"?

And I wonder which teams are "clearly" better than New England? If it's balance you're after, there are only five teams with DVOA in the top dozen in both offense and defense. Two of those are ruled out through abysmal special teams (which empirical models tend to underestimate, regarding them as fluky or unimportant — as a Packer fan, I have witnessed for too long what happens when you get consistently crappy field position). That leaves the Jets and Chiefs (both within a whisker of the Patriots in the rankings) and Atlanta. New York's unadjusted VOA is superior to New England's, but is adjusted downwards for playing the Patriots' defense.

136
by AudacityOfHoops :: Thu, 10/07/2010 - 5:01pm

Not to beat a dead horse, but ...

It seems like DVOA is driven mostly by:
A) converting 1st downs (or TD's on goal-to-go)
B) on plays where you are NOT converting, at least picking up enough yards to set yourself up for (A), plus not shooting yourself in the foot with a big loss
C) avoiding turnovers

I tried to measure these by the following metrics. Everything is divided by plays because DVOA is a per play metric:
A) non-penalty conversions per play (Conv): (1st downs + TD - [1st downs via penalty])/plays
B1) wasted play rate (Wast): (Stuffs + Incomplete passes)/plays
B2) sack rate (Sack): Sacks/plays
C) mistake rate (TO): (Interceptions + Fumbles)/plays

Here are all the offenses that rank in the top 10 in all of these metrics, along with their values and ranks for each:

TEAM / Conv% (rnk) / Wast% (rnk) / Sack% (rnk) / TO% (rnk)
IND / 40% (1) / 24% (2) / 1.5% (3) / 2.6% (7)
NE / 38% (4) / 24% (2) / 2.1% (6) / 2.1% (5)
NYJ / 33% (7) / 24% (2) / 2.1% (6) / 1.7% (2)

So, I don't see a problem with NE's offense being rated very highly at this point.

131
by Mr Shush :: Thu, 10/07/2010 - 2:08pm

Why on earth is it prima facie unlikely that there really is more variation in the quality of offenses than defenses? Or is that not the point you're making? I'm not saying I think the Patriots actually are the best team in the league (they're not), but it doesn't strike me as implausible that a team that was #1 in offense and #23 in defense should be #1 overall. A little surprising, but not outlandish.

133
by DeltaWhiskey :: Thu, 10/07/2010 - 2:45pm

I believe the issue is that by using a strictly additive model to come up w/ total DVOA, FO is assuming equality of variation between offenses and defenses. It makes me wonder if this relatively simple observation may account for the annual DVOA loves Philly debates and explain why the Ravens were the top rated team last year, and that if properly calculated DVOA, would not more accurately reflect reality.

137
by nat :: Thu, 10/07/2010 - 5:22pm

To be more precise, FO (Aaron in particular) is assuming that defensive and offensive DVOAs are measured on the same scale: per-play success compared to the same average. There is no need for the variations to be the same for adding to be reasonable. For example, if I want to know the weight of everyone in my car, I don't care that women have less weight variation than men, or that they average a lower weight. All I need to know is that everyone is weighed in the same pounds.

138
by andyavatar (not verified) :: Thu, 10/07/2010 - 5:49pm

But if I said that men are fatter on average than women, you'd scream bloody murder because the average for men was different from the average for women. You'd say, that's not a fair comparison, men are already heavier on average so how can we compare? And what if my scale introduced a lot of variability when the person being weighed was over 160 pounds? Men would vary from 140 to 400 pounds while women would only vary from 100 to 220 pounds. Could I then say that men were fatter than women? Could I directly compare an average male with an average female? The FO response is, we used the same scale for both, so it doesn't matter. But, didn't we do that also?

139
by AudacityOfHoops :: Thu, 10/07/2010 - 6:31pm

"But if I said that men are fatter on average than women, you'd scream bloody murder because the average for men was different from the average for women. You'd say, that's not a fair comparison, men are already heavier on average so how can we compare?"

The baseline 0% is set so that in the years on which DVOA was built, the average offense and defense both equal 0%. If offenses average higher than 0% this year, it means that offenses are doing better than in past years, and defenses are doing worse. In your scenario, if we took the average weight of men for the last five years, and the average weight of women for the past five, then compared them to current averages, and found that men are heavier now than in the past, and women are lighter now ... then YES, we would say that men are fatter than they used to be. That is what DVOA is saying - offenses are having more success than they used to.

And what if my scale introduced a lot of variability when the person being weighed was over 160 pounds? Men would vary from 140 to 400 pounds while women would only vary from 100 to 220 pounds.

This I can see being a problem, but I'm not sure it's analogous. If you know nothing about the scale, and you weigh everybody, and find that men's weights vary more than females, do you assume it's a problem with your scale, and try to correct the men's weights?

I still don't understand why you think offensive DVOA must have the same variance as defensive DVOA. Did you see my free throw percentage example from earlier? There is more variance in offensive FT% than defensive FT%, and that's a good thing, because offenses really are more in control of their FT% than defenses are. I don't know if that's true for NFL offenses, but it's at least plausible.

140
by AudacityOfHoops :: Thu, 10/07/2010 - 6:48pm

P.S. I thought I put this earlier, but must have edited it out:

I'm honestly curious to hear what your proposed solution is to the unequal variance issue. I am sure that sometimes this really is a problem (maybe it is here with DVOA, I don't know), and I'd like to know how to deal with it when I run into it. I can think of obvious things like shifting and scaling the numbers so that the means and variances are equal, but that forces unreal constraints on the system.

[For example, let's say we start with an NFL where the means and variances of offensive and defensive DVOA really are equal. Then we add clones of Jerry Rice in his prime to the 5 best teams. I'm not exactly sure how this works after defensive adjustments, but I think the average offensive and defensive DVOA will go up, the variation of the offensive DVOA's will increase (because the bests got better, and all others stayed the same), and the variation of the defensive DVOA's will stay the same (assuming a balanced schedule). ... Checking the numbers, that looks a lot like this year: Ave OffDVOA 1.1%, ave DefDVOA 1.2%, stdev OffDVOA 19%, stdev DefDVOA 14%.]

OK, that started as just an example of when I wouldn't want to tweak the average/variance, and ended up being an argument not to tweak it at all, I think. At any rate, I do still want to know your method of correcting for this error, so I can apply it in other fields (i.e. at work).

141
by Led :: Thu, 10/07/2010 - 6:52pm

But the issue with DVOA is not that extreme, right? I see what you're saying and, although I'm not a statistician, it makes sense to me intuitively. I guess I don't mind if straight addition of offense, defense and special teams is not appropriate from a rigorous mathematical perspective because I don't view DVOA, however it is intended, as a precise measurement. It's interesting and very useful general measure of performance and a good tool for identifying the reasons for the good/bad performance. The rankings are fun, but I take them with a little grain of salt. (My questions/comments at the top of this thread weren't addressing the NE's ranking, per se, but rather trying to understand the reasons DVOA goes up and down.) Simply adding offense, defense and ST makes the numbers a lot easier to use and understand. As a non-specialist, I appreciate the simplicity even if it's not methodologically perfect.

142
by DeltaWhiskey :: Fri, 10/08/2010 - 8:55am

The problem is that DVOA is measured in % deviance from the average. I suspect that the difference in variance is due to different measuring metrics.

For example, if we're measuring weight of men in kilograms and women in pounds and we find that Mr. A. is 5% above average and Mrs. A. is 5% above average, and we then to describe Couple A's weight as above average as 10%. Next, we measure Mr. B. and find he is 2% above average and Mrs. B is 8% above average, and then we again add them up and say they are 10% above average - our weight above average measure says they're the same mass; however, clearly they're not.

143
by Eddo :: Fri, 10/08/2010 - 10:33am

That's an interesting point, though the kg-vs-lbs part is a red herring. Those are just units. Once you start measuring in percent over average, the units don't matter. (That is, 10% more than 100 kg is the same mass as 10% more than 220 lbs.)

145
by DeltaWhiskey :: Fri, 10/08/2010 - 12:29pm

The units matter if you don't convert everything to a common unit before combining them.

Okay, agreed. In my head it seemed to work, but when I actually worked it out, it does come out the same; however, the problem occurs when adding percentages.

However, as I work this out, I don't know that straight up adding the unit DVOA percentages is appropriate. For example, example if the average weight of a man and woman is equal, say 220lb then 10% over average for a couple should be 484 lbs (440*1.1); however, if you calculate it independently for each person the way I calculated Couple A, Mr. A weighs 231 lbs (220lbs * l.05) and Mrs. A weighs 231 lbs for a grand total of 462 lbs, which ends up being 5% over the combined average. Moreover, with different means, I think this gets even messier.

This suggests to me that if FO is strictly adding percentages to come up with overall DVOA their could be a problem. It seems that instead they should be calculating success points for each unit (offense, defense, S.T.) and summing them and dividing by a summed demonimator to ascertain team DVOA.

As far as OFF and DEF DVOA being the same metric, I don't think they are as certain penalties (false start I think) impact one but not the other, and S.T. DVOA is definitely measured differrently.

I may be totally wrong on all of this and I hope I was reasonably clear, but the bottom line is that percentages are not always additive.

147
by AudacityOfHoops :: Fri, 10/08/2010 - 2:11pm

"It seems that instead they should be calculating success points for each unit (offense, defense, S.T.) and summing them and dividing by a summed demonimator to ascertain team DVOA."

I started out creating a hypothetical situation to argue against this point, but I think I changed my mind in the middle. Check this out:

Let's say you have a team with an offense that gains exactly 10 yards every play, a defense that gives up exactly 10 yards every play, and a special teams unit that is so great that every drive (offensive and defensive) always starts from the opponent's 20 (so the offense only has to go 20 yards for a TD, and the defense has 80 yards to work with). The offense will get about 1 success point per play, and the defense will allow about 1 success point per play. If we assume the average success is 0.5, their DVOA's should be about equal, at roughly +100% (0.5 above average, divided by the average of 0.5). Obviously the ST DVOA will be something crazy, say +100% also. Under the current system, the team's overall score would be 100 - 100 + 100 = +100%. Yet, they will have a net scoring margin of 0. DVOA is severely overrating the overall team strength, while AT THE SAME TIME being very accurate in the ratings of the units.

Now, say you took DW's suggestion, and summed all success points, then divided them by the summed denominator. On a typical sequence (offensive possession, kickoff/punt, defensive possession, kickoff/punt), here's what we'd have:

Offense runs 2 plays and scored: 2 success points, average team would get 1, so +1 better than average
Kickoff is a touchback: some number near 0
Defense allows 8 plays and a TD: 8 success points allowed, average team would get 4, so -4 worse than average
Kickoff Return to the opponent's 20: some large number, which if calibrated correctly should offset everything else that has happened so far (since we know over the long run this team has a 0 point differential). I think +3 better than average is reasonable, given that this is the same amount of yardage as the defense, but still leaves the score in doubt, and is in a slightly less valuable portion of the field

Add together: [+1 offense] + [0 kickoff] + [-4 defense] + [+3 return] = +0 = 0% DVOA.

This is a much better estimate of the overall strength of the team, while the evaluation of the individual units will not change.

I guess that's why it's called a thought experiment and not a thought I-already-know-how-this-turns-out.

148
by AudacityOfHoops :: Fri, 10/08/2010 - 2:19pm

Hmm, now that I think about it, maybe all I showed is that there are important interactions between the strengths of a team's offense, defense, and special teams (read "field position").

149
by tuluse :: Fri, 10/08/2010 - 4:18pm

Special teams is only counted as 1/3.

So the example DVOA would be 33%

150
by AudacityOfHoops :: Fri, 10/08/2010 - 5:01pm

If you're saying that my equation from the the first half of the post should be:
100 - 100 + (1/3)*100 = 33.3%
Then I respectfully disagree. Look the charts at the top of this page. Total DVOA is the sum of OFFENSE DVOA, DEFENSE DVOA, and S.T. DVOA. Now, I believe special teams DVOA is totally unrelated to "success points" and is just scaled to match DVOA, and that scaling is done in a way that gives special teams about 1/3 the weight of offense or defense. Maybe this is what you're thinking of?

If you're saying that when I pulled a number out of my ass to represent crazy awesome special teams, I should have retrieved 33.3% instead of 100%, then I ... also respectfully disagree. We have normal teams scoring in the 20% range, so an average return of 80 yards is going to be waaaay higher than that.

151
by tuluse :: Sat, 10/09/2010 - 2:32am

All I know is that special teams are only worth 1/3 of offense or defense in DVOA, so if your imaginary special teams were 100% better than average, while offense and defense canceled out, total DVOA would be 33%

They would have to be 300% better than average to get 100%, and if your special teams are 4 times as good as average, you might deserve that rating.

152
by AudacityOfHoops :: Sat, 10/09/2010 - 3:33pm

OK. Either way, it doesn't change the point or outcome of the example. Whether the overall DVOA is 33% or 100%, the team is being overrated by DVOA, since by definition the team has a net scoring margin of 0.

144
by AudacityOfHoops :: Fri, 10/08/2010 - 11:25am

I still don't get this. Isn't "success points" the metric for both offensive and defensive DVOA? And aren't the points the same for offense and defense (i.e. giving up 10 yards of 1st and 10 is worth 1 point to both the offense and the defense)? Am I missing something?

As indicated above, your analogy isn't quite right, so to let's modify it ... if the average weight of men is 170 pounds, and the average weight of women is 110 pounds, then [man +5% / woman +5%] is not equal to [man +2% / woman +8%]. For DVOA, this would be equivalent to an average offense gaining 0.5 success points per play, but an average defense only giving up 0.4 success points per play, right? (I'm not 100% positive, here.) Is that even possible, given the way DVOA is constructed?

Do the defensive adjustments throw this off somehow, so that it's not possible for VOA, but it is possible for DVOA? Let me walk through a scenario and ask for help or an explanation:

TeamA (+30% offensive DVOA) vs. TeamB (0% defensive DVOA), 1st and 10. TeamA gets a first down and gets credit for 1 success point, while TeamB has 1 success point charged against them. OK, but we have to compare those to what an average teams would do in that situation. Let's say the average outcome is 0.5 success points. Then both teams are at +0.5 success points (to be later used in VOA calculation). Same scale so far.

Now we try to take opponent strength into account. TeamA is facing an average defense, so no change there: 1 sucPt - 0.5 aveSucPt = +0.5 sucPtAboveAve. TeamB is facing a very good offense, though, so they're expected to allow a worse play. Maybe their equation is now: 1 sucPt - 0.6 aveSucPt = +0.4 sucPtAboveAve.

If those were the ONLY plays the team ran, their DVOAs would be:
TeamA = [0.5 above ave]/[0.5 ave] = 1 = +100%
TeamB = [0.4 above ave]/[0.6 ave] = 2/3 = +67%

Did that make sense, or did I just make a bunch of mistakes here?

146
by DeltaWhiskey :: Fri, 10/08/2010 - 12:41pm

Your guess is as good as mine. I think your tracking somewhat along the way I'm thinking. Could you expand your example a little bit to include the hypothetical opposite, that is, what happens when Team A goes on DEF and Team B is on OFF, b/c at that point we have both sides of the ball for both teams and add them up; however, we're not adding the raw data, but the percentages. I think that is where the problem is occurring.

128
by Alternator :: Thu, 10/07/2010 - 12:33pm

You might do better with genuine criticism rather than childish whining, you know. If you're really a scientist of some sort, you should understand the concept of "attack the data, not the author" which is not what you're doing.

87
by ChaosOnion (not verified) :: Wed, 10/06/2010 - 8:39am

As I was watching the Eagles drive down the field 4-6 yards at a time for 6 plus minutes, I knew DVOA was going to love it. DVOA loves the long, methodical drives and they are awesome ahead by 10 but, they are suicide down by 14. They were taking the underneath all second half against WAS's Tampa-2 but, it sure looked ugly since McCoy seemed to be the only one open.

89
by Alternator :: Wed, 10/06/2010 - 9:13am

This is more of a case of DVOA not attempting to measure idiot coaches. Just because it's better than average performance for the situation doesn't mean the average coach isn't making a bone-headed call because "that's the way it's always been done."

99
by Revenge of the NURBS (not verified) :: Wed, 10/06/2010 - 11:43am

Said it before, and I'll say it again -- Chris Johnson should have held out. He caved for peanuts (relatively speaking), and now the leverage he had has gone out the window. Highest paid offensive player in the league? Not going to happen if you're averaging < 4 ypc. You gotta strike while the iron is hot.

125
by Treima6 (not verified) :: Thu, 10/07/2010 - 6:55am

(The Patriots) are clearly ranked (too high) because (they no longer have Randy Moss.) (Sorting teams based on Peter King's lofty commentary) is way better than this. 3-13 heer we come!