Writers of Pro Football Prospectus 2008

07 Nov 2006

Week 10 DVOA Ratings

by Aaron Schatz

Here's a look at this week's DVOA ratings. This week's commentary is now available at FOXSports.com. You'll find there a fairly detailed argument against the Colts as the best team in football, despite their 8-0 record. Last year, we got in a lot of trouble when we played around with Indy's rating, but in that case we thought that the DVOA system needed fixing. This time, nothing needs any fixing. They really aren't as good as people think.

(By the way, wait until you see how different the DVOA ratings are from Peter Schrager's subjective FOX power rankings. It's almost impossible to believe that the two lists run on the same website. I understand that most people think that wins and losses are the best measure of a team's quality, and our ranking of Philadelphia is thus too high, but I honestly have no idea how anybody could put Philadelphia below four other 4-4 teams.)

In each of the DVOA comments you'll see the projected final win-loss record according to our infamous DVOA midseason projection system. Those win-loss projections go along with DVOA projections and midseason reviews that we've run every year since 2003. The rest of that material will go up on Football Outsiders this Friday.

There is one thing that I meant to mention a week ago, and I'm not sure where to fit it in at this point, so I'll just say it here. One of the common criticisms of Football Outsiders is that you can't analyze football stats in the same way that you analyze baseball stats because the season is only 16 games long. There's no doubt that an element of randomness exists in the NFL. We can't be sure that our ratings truly represent a listing of the best teams in their proper order, and unpredictable upsets often occur. The best team in DVOA doesn't always win the Super Bowl or even make it that far. But it turns out that even after 162 games and three rounds of playoffs, baseball is no different.

Our partners at Baseball Prospectus have a stat they call "third-order winning percentage." This is sort of their version of DVOA. People are used to advanced baseball analysis that looks at the expected winning percentage based on runs scored and allowed (i.e. the Pythagorean projection). Second-order winning percentage looks at expected wins based on the expected runs scored and allowed based on the team's specific totals of hits, walks, extra-base hits, and so on. Third-order winning percentage is then adjusted for opponent. Does this all sound familiar?

According to third-order winning percentage, the best team in baseball this year was the New York Yankees. It's not even close -- they have 97.4 estimated third-order wins and no other team is above 92. The Yankees, of course, did not even make it past the first round of the playoffs. Two of the top five teams in estimated third-order wins didn't even make the postseason (Blue Jays and Angels) while the Twins were swept in the first round by a team with fewer third-order wins, Oakland.

And what about the team that actually won the World Series, the St. Louis Cardinals? The Cardinals won the NL Central despite 75.8 estimated third-order wins, the 22nd highest total in baseball. Think about that for a second. Imagine if a team finished the season 22nd in DVOA, won a terrible division with an 8-8 or 9-7 record, and then went on to win the Super Bowl. Right now, the lowest team in DVOA to win the Super Bowl was New England in 2001; they were 12th overall and ninth in weighted DVOA.

The point here is that even after 162 games and three rounds of best-of-five or -seven playoffs, the best team doesn't always walk away with the championship. So of course weird stuff is going to happen in football. We can't tell you what is going to happen in the NFL. We can only try to predict the most probable outcome.

For example, the Indianapolis Colts are probably not going to win the Super Bowl this year. But we could be wrong.

* * * * *

Remember that you can always use the keyword "DVOA" to access the latest DVOA commentary at FOXSports.com. Also, someone asked me to post last week's rank in both regular and weighted DVOA; don't forget that "last week" on the FOXSports.com table is last week's weighted DVOA rank.

* * * * *

These are the Football Outsiders team efficiency ratings through Week 9 of 2006, measured by our proprietary Defense-adjusted Value Over Average (DVOA) system that breaks down every single play and compares a team's performance to the league average based on situation in order to determine value over average. (Explained further here.)

OFFENSE and DEFENSE DVOA are adjusted based on strength of opponent as well as to consider all fumbles, kept or lost, as equal value. Opponent adjustments are currently set at 90% and will increase each week until they are full strength after Week 10. SPECIAL TEAMS DVOA is adjusted for type of stadium (warm, cold, dome, Denver/Mexico City) and week of season.

To save people some time, please use the zlionsfan template for all complaints:

<team> is clearly ranked <too high/too low> because <reason unrelated to DVOA>. <subjective ranking system> is way better than this. <unrelated team-supporting or -denigrating comment, preferably with poor spelling and/or chat-acceptable spelling>


TEAM TOTAL
DVOA
LAST
WEEK
W-L WEIGHTED
DVOA
RANK OFFENSE
DVOA
OFF.
RANK
DEFENSE
DVOA
DEF.
RANK
SPECIAL
DVOA
S.T.
RANK
1 CHI 36.0% 1 7-1 34.3% 1 -2.3% 21 -26.6% 2 11.7% 1
2 PHI 34.2% 2 4-4 33.0% 4 22.8% 2 -12.3% 7 -0.9% 20
3 SD 34.1% 4 6-2 34.0% 2 22.7% 3 -8.0% 10 3.3% 7
4 NYG 33.4% 3 6-2 33.1% 3 17.1% 4 -15.7% 6 0.6% 15
5 BAL 26.8% 5 6-2 26.3% 5 -7.3% 23 -29.0% 1 5.1% 3
6 JAC 23.9% 7 5-3 23.3% 6 -1.7% 19 -24.4% 3 1.2% 13
7 IND 18.9% 9 8-0 20.0% 7 32.7% 1 11.2% 26 -2.5% 28
8 NE 18.4% 6 6-2 19.8% 8 11.9% 7 -4.6% 14 1.9% 11
9 DEN 17.9% 12 6-2 19.7% 9 9.1% 8 -7.6% 11 1.2% 14
10 DAL 16.5% 8 4-4 17.4% 10 5.9% 11 -11.8% 8 -1.2% 22
11 KC 12.8% 10 5-3 14.5% 11 2.1% 14 -7.2% 12 3.5% 6
12 PIT 9.7% 11 2-6 10.1% 12 -0.2% 16 -16.1% 5 -6.1% 31
13 NO 9.5% 15 6-2 9.5% 13 7.8% 10 2.4% 20 4.1% 4
14 STL 6.3% 14 4-4 4.1% 14 14.2% 5 7.4% 24 -0.5% 19
15 CIN 3.0% 13 4-4 1.1% 15 8.2% 9 7.9% 25 2.7% 9
16 CAR -2.1% 17 4-4 -0.6% 16 3.3% 12 1.7% 19 -3.8% 30
TEAM TOTAL
DVOA
LAST
WEEK
W-L WEIGHTED
DVOA
RANK OFFENSE
DVOA
OFF.
RANK
DEFENSE
DVOA
DEF.
RANK
SPECIAL
DVOA
S.T.
RANK
17 MIN -2.5% 18 4-4 -2.5% 17 -18.8% 27 -16.2% 4 0.1% 16
18 ATL -4.6% 16 5-3 -6.9% 20 -1.3% 17 0.0% 17 -3.3% 29
19 WAS -4.7% 20 3-5 -5.0% 19 12.5% 6 19.5% 30 2.3% 10
20 GB -5.6% 19 3-5 -3.7% 18 1.0% 15 4.3% 22 -2.2% 26
21 MIA -10.0% 26 2-6 -9.7% 21 -18.3% 26 -10.2% 9 -1.9% 25
22 BUF -10.7% 23 3-5 -11.8% 22 -11.1% 25 3.7% 21 4.1% 5
23 SEA -12.5% 22 5-3 -12.9% 23 -6.2% 22 6.4% 23 0.1% 17
24 CLE -14.6% 21 2-6 -14.0% 24 -20.4% 28 1.0% 18 6.8% 2
25 DET -17.9% 27 2-6 -17.2% 25 -1.4% 18 15.3% 27 -1.3% 23
26 NYJ -18.0% 25 4-4 -18.5% 26 -1.8% 20 19.1% 29 2.9% 8
27 TB -23.5% 24 2-6 -22.8% 27 -21.4% 29 -0.2% 16 -2.3% 27
28 HOU -24.2% 28 2-6 -23.8% 28 3.2% 13 26.2% 32 -1.2% 21
29 SF -31.6% 30 3-5 -32.6% 29 -10.5% 24 21.0% 31 -0.1% 18
30 OAK -34.0% 29 2-6 -33.4% 30 -38.1% 32 -5.4% 13 -1.3% 24
31 ARI -35.1% 31 1-7 -34.9% 31 -27.3% 31 -1.4% 15 -9.2% 32
32 TEN -36.9% 32 2-6 -35.3% 32 -23.4% 30 15.3% 28 1.8% 12

  • NON-ADJ VOA shows what the rating looks like without adjustments for strength of schedule, luck recovering fumbles, or weather and altitude on special teams.
  • ESTIMATED WINS uses a statistic known as "Forest Index" that emphasizes consistency as well as DVOA in the most important specific situations: red zone defense, first quarter offense, and performance in the second half when the score is close.  It then projects a number of wins adjusted to a league-average schedule and a league-average rate of recovering fumbles.  Teams that have had their bye week are projected as if they had played one game per week.
  • PAST SCHEDULE lists average DVOA of opponents played this season, ranked from hardest schedule (#1, most positive) to easiest schedule (#32, most negative). It is not adjusted for which games are home or road.
  • FUTURE SCHEDULE lists average DVOA of opponents still left to play this season, ranked from hardest schedule (#1, most positive) to easiest schedule (#32, most negative). It is not adjusted for which games are home or road.
  • VARIANCE measures the statistical variance of the team's weekly DVOA performance.  Teams are ranked from least consistent (#1, highest variance) to most consistent (#32, smallest variance).



TEAM TOTAL
DVOA
W-L NON-ADJ
TOT VOA
ESTIM.
WINS
RANK PAST
SCHED
RANK FUTURE
SCHED
RANK VAR. RANK
1 CHI 36.0% 7-1 52.0% 5.4 7 -15.7% 32 -1.2% 20 35.1% 2
2 PHI 34.2% 4-4 27.9% 6.7 1 -0.2% 18 2.0% 12 4.8% 31
3 SD 34.1% 6-2 42.9% 6.2 3 -6.8% 31 -5.8% 26 5.5% 30
4 NYG 33.4% 6-2 30.6% 6.4 2 0.3% 16 9.8% 3 9.0% 24
5 BAL 26.8% 6-2 37.1% 5.7 4 -1.2% 22 -4.0% 24 9.9% 20
6 JAC 23.9% 5-3 27.5% 5.0 9 3.0% 9 0.2% 18 35.4% 1
7 IND 18.9% 8-0 21.9% 5.7 5 1.2% 15 -0.3% 19 8.5% 25
8 NE 18.4% 6-2 24.5% 5.5 6 -1.5% 24 -6.6% 29 9.2% 22
9 DEN 17.9% 6-2 11.5% 5.2 8 5.5% 5 -3.7% 22 14.2% 11
10 DAL 16.5% 4-4 17.1% 4.7 11 2.6% 10 2.1% 11 11.3% 18
11 KC 12.8% 5-3 10.0% 4.8 10 -1.0% 20 1.3% 16 34.4% 3
12 PIT 9.7% 2-6 2.6% 4.0 15 5.4% 6 1.4% 14 16.5% 9
13 NO 9.5% 6-2 12.5% 4.7 12 -1.2% 21 2.8% 10 11.6% 15
14 STL 6.3% 4-4 14.9% 4.5 13 -4.7% 30 -10.8% 32 10.1% 19
15 CIN 3.0% 4-4 0.7% 3.9 16 2.5% 11 9.8% 4 6.9% 28
16 CAR -2.1% 4-4 -5.0% 3.7 19 1.3% 14 7.8% 5 7.7% 26
TEAM TOTAL
DVOA
W-L NON-ADJ
TOT VOA
ESTIM.
WINS
RANK PAST
SCHED
RANK FUTURE
SCHED
RANK VAR. RANK
17 MIN -2.5% 4-4 0.1% 4.4 14 -3.2% 26 -6.2% 27 11.5% 17
18 ATL -4.6% 5-3 3.7% 3.8 17 -2.9% 25 5.5% 7 27.1% 5
19 WAS -4.7% 3-5 -8.4% 3.8 18 5.7% 4 11.4% 2 12.9% 13
20 GB -5.6% 3-5 -8.6% 3.6 20 1.8% 13 -3.8% 23 12.4% 14
21 MIA -10.0% 2-6 -1.9% 3.0 23 -3.9% 28 3.1% 8 9.0% 23
22 BUF -10.7% 3-5 -12.0% 3.6 21 2.4% 12 1.8% 13 17.8% 8
23 SEA -12.5% 5-3 -11.0% 3.1 22 -0.1% 17 -8.6% 31 9.4% 21
24 CLE -14.6% 2-6 -19.4% 2.5 29 4.6% 7 1.2% 17 3.6% 32
25 DET -17.9% 2-6 -13.0% 2.7 26 -1.5% 23 -1.7% 21 5.9% 29
26 NYJ -18.0% 4-4 -13.1% 2.9 24 -3.6% 27 -4.1% 25 15.7% 10
27 TB -23.5% 2-6 -34.4% 2.4 30 10.1% 1 2.9% 9 7.1% 27
28 HOU -24.2% 2-6 -31.5% 2.7 25 9.7% 2 -6.6% 30 13.1% 12
29 SF -31.6% 3-5 -38.2% 2.5 28 6.7% 3 -6.2% 28 27.2% 4
30 OAK -34.0% 2-6 -37.8% 1.9 31 -0.7% 19 5.6% 6 11.6% 16
31 ARI -35.1% 1-7 -27.1% 2.5 27 -4.2% 29 1.3% 15 25.3% 6
32 TEN -36.9% 2-6 -33.5% 1.9 32 4.1% 8 17.5% 1 18.5% 7

Posted by: Aaron Schatz on 07 Nov 2006

1
by Greg (not verified) :: Tue, 11/07/2006 - 6:42pm

Looks like the gap between 20 and 21 is closing.

2
by dbt (not verified) :: Tue, 11/07/2006 - 6:44pm

Click my name for the sortable chart.

3
by Travis (not verified) :: Tue, 11/07/2006 - 6:45pm

Three of the top five wins in estimated third-order wins didn’t even make the postseason (Twins, Blue Jays, Angels).

The general point still holds, but Twins made the playoffs (and got swept in the first round).

4
by Ilanin (not verified) :: Tue, 11/07/2006 - 6:45pm

And Arizona are still beating out Pittsburgh for #32 in ST DVOA. Santonio Holmes is not pleased with this disrespect.

More seriously, Aaron, have you tried an escalating VOA penalty for turnovers after the first in a game? It occurrs to me that once you start getting several turnovers in one game you really are throwing it away, whereas an otherwise superior team can probably overcome a single turnover. Actually, thinking about it more this is probably something for the Forest Index rather than VOA, which doesn't really care about W-L particularly.

Yes, this is partly a suggested fix to the "Pittsburgh problem", but I think it has overall applicability.

5
by Moridin (not verified) :: Tue, 11/07/2006 - 6:46pm

Interesting to see the Vikings still at 17th in the middle (though their rating dropped some). I figured their offense would take a larger hit.

6
by donald (not verified) :: Tue, 11/07/2006 - 6:49pm

Ah, this crap again. I hope you don't really believe this shit means anything. For the love of God, there comes a time when you have to admit you're wrong. That time has come.

7
by donald (not verified) :: Tue, 11/07/2006 - 6:53pm

Look at Chicago and San Diego. Why are they ranked high? Because they run up the score on pathetic opponents. Wow, that's so meaningful. Clearly all the points they score in garbage time against inferior foe ought to be taken into account here. And teams who's coaches know when enough is enough and settle for a "mere" victory ought to be penalized because they don't see the point in rubbing it their opponenent's face.

8
by Becephalus (not verified) :: Tue, 11/07/2006 - 6:53pm

Every week someone has a better chart. By the end of the year I fully expect it to be 3D and interactive, with sprinkles as well.

9
by B (not verified) :: Tue, 11/07/2006 - 6:55pm

I'm surprised at the difference in Estimated Wins and DVOA for Chicago. I don't remember seeing a gap that large before. I wonder if this tells us something about how they can seem so dominating most of the time, but put up real stinkers against teams with good defenses and lousy offenses. I do feel that EW does a better job of indicating which of these teams are really good, however. It also moves Jacksonville below Indy and Denver, which I think is correct.

10
by Becephalus (not verified) :: Tue, 11/07/2006 - 6:56pm

way to make some rather worthless posts donald...

if you don't care about statiscal veracity why come here?

and if you do Aaron has stated oh idk 100,000,000 times that removing such "garbage time" points does not increase the predictive ability of the system. There are modifiers for situation, but gnoring chi or sd crush bad times makes the system worse not better. surely you can understand that? and if you don't just leave and never come back, because this type of anaylsis is over your 6th grade mind.

11
by jebmak (not verified) :: Tue, 11/07/2006 - 6:56pm

donald, I think that you are going to have to be more specific.

12
by Fnor (not verified) :: Tue, 11/07/2006 - 6:56pm

PIT... only... drop... one... wha?

13
by D (not verified) :: Tue, 11/07/2006 - 6:56pm

I have a feeling that once the ratings go up on FOXsports this message board is going to get blasted. Not as bad as the Atlanta incident last year, but people are not going to be happy with a 4-4 team at #2 and an 8-0 team outside the top 5.

14
by Tally (not verified) :: Tue, 11/07/2006 - 6:57pm

Teams would run up the score if they could. Remember guts and stomps.

15
by jebmak (not verified) :: Tue, 11/07/2006 - 6:57pm

Sorry, saw post #6 but not #7 when I typed that.

16
by Becephalus (not verified) :: Tue, 11/07/2006 - 6:57pm

9 A high variance is very hard on good teams. It also helps poor teams.

17
by admin :: Tue, 11/07/2006 - 6:58pm

Super duh. Fixed Twins reference.

18
by John (not verified) :: Tue, 11/07/2006 - 6:58pm

With the Raiders Offensive DVOA being so bad, what would the Seahawks have had to do to improve their Defensive DVOA on Monday night?

Could this be a flaw in DVOA that puts too much emphasis on strength-of-schedule, essentially making it impossible for a team to improve their DVOA when playing a terrible team?

19
by Richard (not verified) :: Tue, 11/07/2006 - 6:59pm

8: No kidding. #2 has outdone us all.

In other news, is donald getting dumber or what?

20
by ABW (not verified) :: Tue, 11/07/2006 - 7:02pm

Re: 7

Hilarious. Someone is accusing the Marty Schottenheimer-coached Chargers of running up the score? If there's a God out there, please let a Chargers fan come on here and see this.

21
by Devin (not verified) :: Tue, 11/07/2006 - 7:04pm

RE: #6, you'll have to forgive Donald, he just has sand stuck in his vagina.

22
by DaveO (not verified) :: Tue, 11/07/2006 - 7:04pm

For example, the Indianapolis Colts are probably not going to win the Super Bowl this year. But we could be wrong.

Geez, Aaron, can't you give us a flippin' full week of irrational exuberance after slaying (or at least badly wounding) the damn monkey on our backs? Is that too damn much to ask? Man, you Pats fans are all alike...

PS just kidding. Keep up the tasty goodness - it's Lord's work you're doing here...

23
by donald (not verified) :: Tue, 11/07/2006 - 7:04pm

Classy response Becaphalus. Top notch.

My sixth grade mind knows this much-the rankings are seriously flawed, and yes, taking into account gargabe time is one of those flaws, no matter how good a job Aaron has done of convincing you that it's not.

I'd like to know if Aaron stands by his rankings so firmly that he would be able to pick Philly, or Chicago, or any of the "big four", to beat Indy more times than not on a neutral field. My guess is if he said he would he'd be full of shit.

Just for the record, I hate Indy, but they are the NFL's best team right now and it's not close.

24
by D (not verified) :: Tue, 11/07/2006 - 7:04pm

Alter my last comment to say a 4-4 team at #4 (I forgot that the FOX ratings were based on weighted DVOA).

25
by B (not verified) :: Tue, 11/07/2006 - 7:05pm

mmmmmmm, sprinkles.
(drools)

26
by Richard (not verified) :: Tue, 11/07/2006 - 7:06pm

Philadelphia leads the NFL in Estimated Wins. That's incredible. Have they really been that unlucky? It seems like there has to be something wrong there, but I can't imagine what.

27
by Yaguar (not verified) :: Tue, 11/07/2006 - 7:06pm

I'm a Colts fan, and I know their weaknesses as well as anyone. I understand the logic behind DVOA. I don't think the Colts are the best team in football either. But I think Jacksonville over Indianapolis is just goofy. It seems to me like DVOA is probably crediting the Jaguars a little too much for pounding the Jets. Even to a FO-style fan, I think that just looks odd.

Jacksonville is one of those teams whose wild inconsistency screws up the ratings. Sure, they've pounded multiple teams, but they lost to the Texans, too.

I don't think anyone would take the Jags over the Colts at this point. I probably wouldn't take them over New England or Denver, either.

28
by Wanker79 (not verified) :: Tue, 11/07/2006 - 7:07pm

A few interesting goupings in there. I find it interesting that the "Big 4" are all within 1.3% (essentially equal). I also find it interesting that Indy, NE, and Denver are all within 0.3% of each other (essentially equal). And I also find it interesting that the "Abismal 4" are all with 2.7% (essentially equal).

And even though I'm an Eagles' fan, I'm pleased to see them dropped down to #4.

29
by DrewTS (not verified) :: Tue, 11/07/2006 - 7:10pm

For what it's worth, I don't think you should have to apologize for not having the Colts #1. 7th seems a bit low, but I'll freely admit they have flaws. I'm surprised to see NE as low as they are. My brain instinctively rejects the notion that the Colts and Patriots aren't both better than Jacksonville. But I don't want to draw the wrath of the FOMBC, so I'll drop it.

As for the Colts, the numbers basically say what everyone knows -- offense is excellent, defense... not so much. The low ST ranking surprises me a little though. To me, they seem to be better this year than in past years. What part of that is dragging them down?

So in summation...
Indianapolis is clearly ranked too low because I frickin' say so. Reading goat entrails, and then simply listing teams in descending order by record is way better than this. Go Horse!! Get Crunked!!1!1!!1

30
by Tom Kelso (not verified) :: Tue, 11/07/2006 - 7:12pm

Just to show DaveO and I can agree -- I get more decent football information out of this site than any other running. Kepp up the Good Work, Aaron!

Never thought I would see an Indy partisan defending current DVOA, and a self-professed Indy "hater" saying blow it up. Methinks donald doth protest too much.

31
by johnt (not verified) :: Tue, 11/07/2006 - 7:16pm

Normally I agree with DVOA, but I just don't think it's right here. Contrary to popular belief, there is no inherent reason DVOA is infallible. While it is common to jump on anyone who questions DVOA, just because it has shown a general predictive ability for teams on average does not mean certain teams are not poorly described by it.

Chicago pretty clearly seems like one of them. Chicago is not the best team in the league and I don't care what formula says otherwise. They have feasted on a creampuff schedule (well.. sometimes feasted and sometimes vomitted uncontrollably). This is one case where I favor Cold Hard Football Facts (pro-Pats homers that they are) idea of Quality Wins. I have noticed this as a consistent issue with DVOA, that it overvalues wins over pathetic opponents and weak schedules that seem to have very little predictive value when it comes to good opponents.

Anyway. Feel free to paste the standard accusation for people who question DVOA, but I would like to see a logically supported argument as to how Chicago is possibly the best team in the NFL considering their schedule and performances against Cards/Dolphins.

32
by admin :: Tue, 11/07/2006 - 7:17pm

If you are reading the site regularly, you know the following:

1) I've noted, nearly every week, that blowouts tend to skew the ratings, and therefore teams like Jacksonville and Pittsburgh may be a little too high.

2) I've noted, nearly every week, that I've spent hours trying to figure out a way to account for this in the numbers without subjectively deciding which plays should not count just because I don't feel like it. I still can't come up with something that doesn't make the system LESS accurate. Until I can make the system better by changing how it treats blowouts, we don't change how we treat blowouts.

At a certain point, I hope people understand I can't remember to say this stuff every single week.

33
by Fnor (not verified) :: Tue, 11/07/2006 - 7:18pm

#23: It wasn't much convincing. To make a ridiculously obscure reference:

Aaron: "I'm making DVOA!"
Us: "YAY!"
--DEAD AIR--
Aaron: "DVOA doesn't discount late points entirely because that makes the system have a lower correlation with the end result!"
Us: "YAY!"

To be fair, DVOA is different from delicious cake, but the idea is the same. We don't need to know all the steps involved, only that the cake is the best it can be and is, in fact, delicious.

34
by John (not verified) :: Tue, 11/07/2006 - 7:21pm

johnt, I would read the guts and stomps article. Despite common football wisdom to the contrary, feasting on weak opponents--as you put it--is more indicative of future success than barely beating a good opponent.

35
by BB (not verified) :: Tue, 11/07/2006 - 7:21pm

I think looking at the combo of expected wins and DVOA is pretty enlightening. The Bears, when they are on, are clearly the best team, and it might not be close. But they're not always on (mainly due to the boom or bust offense) and so they rank 7th in estimated wins. Makes perfect sense to me. Thus, they have the #1 DVOA but one of the highest variances -- the great DVOA of their stomps is pulling up their two putrid performances, but estimated wins is not fooled.

I guess that means teams better cross their fingers in the playoffs and hope they can get pressure on Grossman, or hope that he doesn't start figuring out what he's doing wrong and fixes it.

36
by Adam (not verified) :: Tue, 11/07/2006 - 7:22pm

I took the estimated wins and made a linear adjustment based on the strength of each team's future schedule (1 win for 15.5% in DVOA) to predict The future win/loss of each team. Here are the results:

CHI 12.5
PHI 10.6
SD 12.6
NYG 11.8
BAL 12.0
JAC 10.0
IND 13.7
NE 11.9
DEN 11.4
DAL 8.6
KC 9.7
PIT 5.9
NO 10.5
STL 9.2
CIN 7.3
CAR 7.2
MIN 8.8
ATL 8.4
WAS 6.1
GB 6.8
MIA 4.8
BUF 6.5
SEA 8.7
CLE 4.4
DET 4.8
NYJ 7.2
TB 4.2
HOU 5.1
SF 5.9
OAK 3.5
ARI 3.4
TEN 2.8

37
by admin :: Tue, 11/07/2006 - 7:22pm

The Bears are bad in estimated wins because they suck in the second half of close games (on both offense and defense) and have a variance that goes through the roof.

My last comment for the week before I go back to work: Please, everybody complaining about blowouts and quality wins needs to read Guts and Stomps. There is no evidence whatsoever that "quality wins" are actually an indicator of which team is likely to win the Super Bowl.

38
by MJK (not verified) :: Tue, 11/07/2006 - 7:22pm

I would pick Philly, or Chicago, or San Diego to beat Indy more times on a neutral field. Well, not San Diego if it counted...gotta take Marty Schottenheimer into account. Indy's offense is fantastic, but any of those three teams have the defense to slow them down a little, if not stop them. Meanwhile, all three have pretty good offenses in their own right, which against Indy's Defense would take on, well, Indy-like abilities. Yes, I think it is fair.

That being said, remember, DVOA isn't supposed to predict who would win more. It's supposed to tell us something about how well teams have been playing in each phase of the game, and hence how they're likely to match up. Chances are, games between Indy and some teams rated above them would probably be closer than DVOA implies, largely due to special teams. Indy has horrible special teams, which is probably weighing their DVOA rating down. However, to a team that has a potent offense and a lousy defense, special teams aren't as important, because their offense can overcome bad field position, and their defense is so bad that opposing offenses do just as well with good or bad field position. So in actual Indy games, special teams probably won't be as significant a factor as they would for say...Jacksonville versus Baltimore. But that doesn't change the fact that Indy hasn't played well on special teams.

39
by jebmak (not verified) :: Tue, 11/07/2006 - 7:23pm

Re #32

I don't see how "blowouts skew the rankings" and "changing it makes it less accurate" can both be true.

Are you saying that no formula makes it more accurate, but you could make it more accurate were you to subjectively lessen the effects of certain parts of games?

40
by Richard (not verified) :: Tue, 11/07/2006 - 7:24pm

If one were to do something stupid like say find the z-score for each team's Weighted DVOA and their Estimated Wins and then add them, they'd get this list:

Philadelphia
New York Giants
San Diego
Chicago
Baltimore
Indianapolis
New England
Jacksonville
Denver
Dallas
Kansas City
New Orleans
St. Louis
Pittsburgh
Minnesota
Cincinnati
Carolina
Washington
Atlanta
Green Bay
Buffalo
Miami
Seattle
New York Jets
Detroit
Cleveland
Houston
Tampa Bay
San Francisco
Arizona
Oakland
Tennessee

41
by C (not verified) :: Tue, 11/07/2006 - 7:24pm

#26: Yes, Philly has gotten very unlucky. NYG only won because of a "fumble forward" for a touchdown and TB won on a 62 yard FG. They laid an egg against JAX and lost a narrow one on the road against NO.

They have played sloppy at times and lethargic at others, but when the click, they are incredible. There has been some kind of strange Mojo on the team this year, however, and I need to see how they respond to the bye week. I still think 10 wins is probable for them.

42
by Will Allen (not verified) :: Tue, 11/07/2006 - 7:26pm

Having watched the past two Vikings games, I find it hard to believe on first impression that there are five offenses that are worse. Certainly the Raiders, but four others is difficult to accept. Then again, they actually get decent line play and performance from their running backs, so maybe it isn't so off base, which indicates just how awful they are at the other ball handling positions, wide receiver especially.

43
by Richard (not verified) :: Tue, 11/07/2006 - 7:27pm

41: Y'know what? I know all that and it still troubles me. The fact that it makes sense doesn't seem to help me. Oh well.

44
by Wanker79 (not verified) :: Tue, 11/07/2006 - 7:28pm

Re: 26

Philadelphia leads the NFL in Estimated Wins. That’s incredible. Have they really been that unlucky? It seems like there has to be something wrong there, but I can’t imagine what.

Here's a quick recap of Philly's season:

They've only recovered 10 of 29 fumbles. That's VERY unlucky. They've also had at least 2 INTs returned for TDs, also unlucky.

They've convincingly defeated Houston, San Fran, Green Bay, and Dallas.

They lost to the Giants in OT in a game where Plaxico Burress lost a fumble that bounced 20 yards into the endzone where it was recovered for a TD by a fellow Giant and had the game tying FG setup by a mindnumbingly stupid personal foul on Trent Cole for kicking Kareem McKenzee in the nuts with just a few seconds left in the game. They had a 17 point lead at the start of the 4th and got sloppy. You can't just discount 3 quarters of superb play and only emphasize the last 17 minutes of the game (including OT). They should have won this game.

They lost to NO on a last second FG that was setup by a Too Many Men on the Field penalty on a 3rd down sack that would have resulted in a punt. They could have won this game.

They lost to TB on a the 3rd longest FG in the history of the NFL on the last play of the game. During the game there were 2 INTs returned for TD which are essentially random events. They should have won this game.

They got man-handled by Jacksonville. This is the only game this year that they clearly didn't play well enough to win.

45
by DrewTS (not verified) :: Tue, 11/07/2006 - 7:31pm

Looks like me and #27 had the same thought about Jacksonville. That's the only one that's a little too bitter to swallow.

Re 30

While you were writing that, two more Indy people showed up to more-or-less agree with the rankings. My hope is that maybe, finally, some of us have come to realize that rankings and respect and even W-L records aren't that important, so hopefully there won't be quite so much ire (but I know better than to wish for that).

After all, for teams like the Colts, Pats and Broncos, the regular season is mostly just jockying for position. It's like qualifying for a F1 race -- it's not really a question of if they'll be in, it's just a question of where. The Colts are currently P1, and that's good enough for me.

46
by Darrel Michaud (not verified) :: Tue, 11/07/2006 - 7:33pm

Donald -

And teams who’s coaches know when enough is enough and settle for a “mere� victory ought to be penalized because they don’t see the point in rubbing it their opponenent’s face.

It would seem as though you're talking about Indianapolis here, and indeed, you confirm this in a later post. If DVOA penalizes coaches that don't run up the score, shouldn't this appear as a year-in, year-out bias? Indy was first in last year's rankings, third in 2004's, and 5th in 2003's.

Furtheremore, if you examine the individual games, your premise that the Colts intentionally keep the score close when they're ahead falls apart. Look at the Houston game, where they won by 19. The game was 3-20 in the 4th quarter, yet Peyton was still throwing. He had passes of 37 yards (on 1st and 10), 18 yards (on 2nd and 6) and 27 yards (on 2nd and 11). Indy scored 13 points in a quarter they entered up 27. What is that, if not "rubbing it in their opponent's face?"

Indianapolis won by a combined 4 points to Tennessee (the 32nd ranked team by DVOA) and the Jets (26th) because Indianapolis played horribly for 3 quarters, not out of the goodness of their hearts.

47
by Yaguar (not verified) :: Tue, 11/07/2006 - 7:33pm

Aaron: I noted in my "Jacksonville is too high" whine that I thought the Jets win was the cause of it. I think we're all acquainted with the phenomenon, and understand why DVOA hasn't been modified, so you probably don't need to go through the trouble of posting that every time.

Really, I think if we're going to rank what teams we think are "best," I think incorporating variance in some way would probably help. Good teams with high variance, I think, come out too high in the rankings.

The two teams most of us think are too high are the Bears and the Jaguars. They are also first and second in variance. I don't think it's a coincidence.

Rex Grossman scares the hell out of Bears fans because he's so erratic. I think most of us would prefer the consistency of the Chargers.

48
by MJK (not verified) :: Tue, 11/07/2006 - 7:34pm

I wonder if some of the strength of schedule tiltings would smooth themselves out more with more "D" adjusting iterations. I remember last year Aaron fixed the problem caused by Indy's pathetic schedule (just what IS Polian paying the league... j/k) by iterating the strength of schedule adjustment one more time. This made DVOA seem to make a lot more sense, and I assumed it improved its predictive capabilities because Aaron adopted it as standard practice. But theoretically, we still don't know if one extra iteration is enough to get near an equilibrium state--I think I remember Pat bringing that up last year? I wonder how much the ratings change if you keep iterating, and do they become more predictive?

I also wonder if that might do something about the blowout problem as well?

A question regarding the blowout situation. DVOA compares a team's performance on every play against the league average in a similar situation, which I assume includes the current score differential and the time remaining in the game. E.g. Being down by 10 with 2 minutes to go is not the same as being down by 2 with 2 minutes to go, or down by 40 with two minutes to go, or down by 10 with 25 minutes to go. Someone please correct me if this is wrong. But, given that blowouts do not happen very often, I would imagine the sample size for plays "when down by 40" or "when up by 40" is very small. So how does DVOA compare performance in these situations to the average, since there's probably not enough data to say what the "average" is? I would imagine this is a large part of the reason why blowouts tend to skew the ratings oddly. Teams play fundamentally differently once it is clear that a blowout has commenced, and the average or typical behavior and performance in these situations is probably unknown.

49
by James C (not verified) :: Tue, 11/07/2006 - 7:34pm

While people are producing super clever charts and tables of this stuff (and more power to them) can anyone with the technical know how produce a clever graph or other visual representation that shows DVOA as a team (or teams) progresses through the season (please).

As I have previously stated on this site I once got 8 out of 70 on a statistics exam and clearly shouldn't be trusted with as much as a histogram or I would try it myself. However as I am clearly inept I am asking others with clever programs and expert abilities to facilitate the visual rendering of all this excellent data. I for one would be tremendously grateful.

50
by Josh (not verified) :: Tue, 11/07/2006 - 7:41pm

Regarding the Bears and blowouts, I don't think their blowout wins are a result of running up the score. As the clearest example, their most recent blowout win over SF, they went up 41-0 by halftime, ended up winning 41-10. In my mind, there is no such thing as running up the score in the first half.
I'd be interested to see what the Bear 1st half and 2d half DVOA is in their blowout wins. I suspect that the 2d half is considerably lower, that the Bears are blowing many teams out in the first half and doing a little more than keeping it even in the 2d half of those games.

51
by jonnyblazin (not verified) :: Tue, 11/07/2006 - 7:41pm

Its interesting how big the difference in the AFC is between the haves and the havenots. Between PIT at 10% and MIA at -10%, there is only one 'average' team, the Bengals. Everyone besides them is either considerably above average and considerably below average.
Also its good to see the Ravens offensive DVOA% has substantially increased the past couple games (despite not shooting up the rankings), coinciding with Fassel's departure and McNair's increased familiarity with the offensive system. A couple weeks ago people were calling him washed up; now, not so much.

52
by chris clark (not verified) :: Tue, 11/07/2006 - 7:41pm

re 2: I hate to be critical of such excellent work, but could you increase the vertical scale of the chart? I'm trying to eyeball the chart to see if there are gaps or other "interesting" shapes in the chart, and that's most easily done if the 1st to last line is roughly 45% (rather than the 15% your chart now has). Well, actually you want the trend (regression) line to be 45%, not the 1st to last line, as they could be outliers. Acutally drawing in the linear regression line would be really cool--did someone say sprinkles!

For example, from your chart (using DVOA) I can make out that chi/phi/sd/nyg are tightly grouped and so are ind/ne/den--heavens to RPS, but I'm not sure I can discern much more, and I really want to.

BTW, by "playing" with the chart (i.e. showing different stats), I managed to a weird state where the boxes were kind of jumbled and didn't "connect" to the origin line (i.e. chi & ind had 0 high, with phi & ne having lower 0's). Hopefully, if I just start over it will work better (or maybe it's IE's fault).

53
by DrewTS (not verified) :: Tue, 11/07/2006 - 7:42pm

Re 48

I'd bet that the "up by 40" situation is more likely characterised as something like "up by 28 or more." You get more data points that way, and they're essentially the same situation.

54
by Peremptor (not verified) :: Tue, 11/07/2006 - 7:44pm

Re: #20

Exactly :). If not for overt martyballing in the @OAK game and @BAL game the Chargers would be #1 in DVOA right now ;).

55
by Brian G (not verified) :: Tue, 11/07/2006 - 7:45pm

Dear pals,

This ranking is absurd and doesn't mean jack squat. I'm sorry but this isn't baseball, people... games cannot predicted by computer... and this is by far the worst one their is. I would like to point out that this "vaunted" computer 7 and 7 this week. Keep wasting your time believing this garbage.

PS. I can't wait to Indianapolis mop the floor with "#2" Philly. What a joke.

56
by The Ninjalectual (not verified) :: Tue, 11/07/2006 - 7:46pm

The 2005 season was weird because, as I recall, there were few upsets. DVOA was magnificent at predictiong future results. In 2006, not so much. I used to think that it was the subjective DAVE screwing up the system, or an issue of small sample size. Now we're done with DAVE and we ought to have meaningful data on the season. I still don't know what's going to happen in any game.

How does Atlanta have a negative DVOA and a winning record? Craziness!

57
by johnt (not verified) :: Tue, 11/07/2006 - 7:47pm

Aaron: I don't necessarily think Quality Wins are predictive of the Super Bowl and I thought the article was interesting. But it's not testing the relevant issue: what you looked at was "whether it's better to have more close wins over quality opponents than blowout wins over easy opponents". My biggest problem with this is that it ignores the issue of schedule differentials. The Bears will wind up playing MAYBE 3 "quality" teams this year, while the Colts will play around 7. An absolute comparison of "stomps" versus "guts" for each team seems futile - it would be shocking if the Bears didn't have more stomps than guts with less than half as many chances for them. If anything it says "teams that play really easy schedules tend to do better".

58
by Yaguar (not verified) :: Tue, 11/07/2006 - 7:47pm

56 - DAVE wasn't subjective.

59
by Sid (not verified) :: Tue, 11/07/2006 - 7:48pm

I agree. The Colts are not the best team in football right now and they will most likely not win the Super Bowl.

The Bears are easily the best at this point.

60
by johnt (not verified) :: Tue, 11/07/2006 - 7:50pm

That probably wasn't phrased so great. I meant "teams that play really easy schedules on average do better, regardless of the actual quality of the team". IOW, if you have the same team, getting the benefits of an easy schedule helps a lot in terms of setting up the playoffs to be a lot easier for you.

61
by Wanker79 (not verified) :: Tue, 11/07/2006 - 7:50pm

Re: 55

Probably the more amazing thing I've learned from coming the the site is that there is an astonishing ability for illiterate people to still be able to type. Please, before you make yourself look even more idiotic than you already do, go read Aarons commentary at the top.

62
by CA (not verified) :: Tue, 11/07/2006 - 7:51pm

Re: 55

Brian G, serious question: Is that your actual opinion or a parody? I honestly can't tell. Although it differs in form, it captures the spirit of the zlionsfan template perfectly.

63
by Yaguar (not verified) :: Tue, 11/07/2006 - 7:53pm

59 - Really? "Easily" the best?

64
by Richard (not verified) :: Tue, 11/07/2006 - 7:54pm

62: I was wondering the same thing.

65
by Will Allen (not verified) :: Tue, 11/07/2006 - 7:55pm

How likely is is that Grossman will go three consecutive playoff games without being awful in at least one. If he is awful in one, how likely are the Bears to beat a playoff-caliber team, to say nothing of a Super Bowl calibe team? If the Bears meet the Colts in the Super Bowl, what ends up being the more likely deciding factor, Grossman being terrible, the Colts' defense being terrible, or the Colts' o-line having "protection issues"?

66
by John (not verified) :: Tue, 11/07/2006 - 7:56pm

Damn, I don't want to harp on this, but it seems like unless you say something inflammatory no one pays attention...

So: DVOA is lamer than a two legged horse.
-and-
DVOA is dumber than Helen Keller's corpse.

Now, does anyone have any idea if outliers like Oakland's offense preclude their most recent opponents, for instance Seattle, who’s defense completely dominated Oakland on Monday, from improving their corresponding DVOA?

Seattle's Defensive DVOA declined after Monday night's performance, largely I assume because of SOS, but Seattle got 9 sacks and allowed only 185 net yards. I want to know what the Seahawk's D would have had to do to overcome such a large SOS adjustment, and if such a performance is really reasonable given one NFL team playing another NFL team?

Also, what was Seattle's defensive VOA?

67
by Peremptor (not verified) :: Tue, 11/07/2006 - 7:57pm

Re #32- If there is one sorry I even mentioned it, but I havent seen the link. Wouldnt a FAQs page for all the first timers on the major points of contention one would have with DVOA at first sight be ideal?

68
by MJK (not verified) :: Tue, 11/07/2006 - 8:01pm

I would like to point out that this “vaunted� computer 7 and 7 this week.

7 and 7 according to what? Just picking the higher ranked team over the lower ranked team every time? Or maybe the higher total DVOA? Weighted or non-weighted? Are taking home field advantage into account? How about allowing for the possibility that when two closely ranked teams play it's too close to predict? Are you predicting the game by comparing O-versus-D matchups according to DVOA? How about Running-O versus Running-D and Passing-O versus Passing-D? How much are you weighting each category?

DVOA gives us a number (actually a bunch of numbers) that indicates how well a team has played relative to some baseline. It's probably better than most numbers. It probably even predicts who is more likely to win on a neutral field between two stylistically similar teams (not who will win--the best team in the world probably only has about a 95% chance of beating the Raiders this year). But that doesn't mean you should pick whichever team has a higher DVOA, any more than you would pick a winner by picking whichever team has averaged more yards per game, or more points per game.

If you really want to see how good DVOA is at picking games, you should give it as a tool to intelligent analysts that understand football and see how they do. For example, if the Outsiders, using their knowledge of football and DVOA, picked teams every week, and did no better than 7-7 (i.e. random), then maybe you could argue that DVOA is useless (even that would not be conclusive--there is still the possibility that the Outsiders might just be dumb when it comes to using DVOA to predict games...;-) j/k). But all saying "DVOA was 7-7 this week" tells me is that you are too dumb to know how to use a statistical tool to predict games. It's like saying a rifle is a lousy rifle because if I just point it randomly and squeeze the trigger, it never hits the target. The best rifle in the world won't help you if you don't know how to aim.

69
by James C (not verified) :: Tue, 11/07/2006 - 8:05pm

#61

Blame it on spellcheckers. I once worked with an idiotic restaurant/function room manager who would rattle off letters on his computer, run his spelling and grammar checker and send off the letter assuming that it was fine (this was for functions for up to $5000 plus bar takings and quite often). He never seemed to grasp the difference between putting a booking 'in the diary' and 'in the dairy' and despite not having any vocation with the catholic church was constantly arranging for people's 'confirmation'. Some people are just so dumb that they fail to observe that other people are much more clever than them.

70
by milo (not verified) :: Tue, 11/07/2006 - 8:08pm

So, which gamma quadrant do you live in that computes the Redskins passing offense as better than ..... oh, say the Saints?

71
by Chicago, Dallas, Kansas City, New England, New York Giants, (not verified) :: Tue, 11/07/2006 - 8:09pm

66:
I'm going to guess the main factor is that they forced ZERO turnovers from a team that has averaged 2.6 per game (31st in the NFL).

72
by Richard (not verified) :: Tue, 11/07/2006 - 8:09pm

I don't know about the other readers, but I'd gladly pay some sort of fee to have access to sortable stats, splits, etc.

Is there any chance of such a thing happening in the future?

73
by DavidH (not verified) :: Tue, 11/07/2006 - 8:10pm

oops. I am not those 6 teams.

74
by We\'re the only teams better than 0% in offense, defense, an (not verified) :: Tue, 11/07/2006 - 8:13pm

San Diego
New York Giants
New England
Denver
Kansas City

75
by tunesmith (not verified) :: Tue, 11/07/2006 - 8:15pm

I do kind of wonder how DVOA would count things differently if it only counted plays that happened while the score was within a touchdown either way. But I seem to recall that Aaron looked into something like that and it made DVOA less accurate...?

76
by Sid (not verified) :: Tue, 11/07/2006 - 8:19pm

Wow, Chicago is like 7th in estimated wins. That's insane.

77
by admin :: Tue, 11/07/2006 - 8:21pm

72, yes, we're just finishing up the technical parts of that. By December, I hope. It will go back to 1997, and include the current season as well.

78
by Kal (not verified) :: Tue, 11/07/2006 - 8:21pm

Okay, Aaron, this came up when talking about Guts/Stomps at another site I go through. You say that Guts are not a good indicator of who will win the superbowl, and that is fine - but how are Guts and Stomps as far as good indicators of regular-season performance? In general, is it better for a team in the regular season to have more Guts or more Stomps?

I guess you'd have to normalize it somehow, but I'd be curious to see how well that holds up.

The Bears haven't run the score up on anyone. They tend to score early and then sit on their lead. That's one of the dumbest arguments I've ever seen on this site.

I think a lot of the problems are skewed by how great Jacksonville is appearing here. For instance, Jacksonville beat Philly, so they look good. Teams that beat jacksonville look abnormally better - Washington's passing DVOA is going to look great after they put up 36 through the air, mostly. Their doing great against the Jets and Tennessee also doesn't hurt them. But DVOA doesn't do a good job of tuning down the variance - it never really has - and that's likely an issue.

79
by We\\\'re the only teams better than 0% in offense, defense, (not verified) :: Tue, 11/07/2006 - 8:25pm

70:
Again, I'm guessing it has mostly to do with turnovers.

NO - 0.028 INT/Att
Was - 0.013 INT/Att

Granted, NO is better in most other things, but then again who knows what happens if you go through and look at it play-by-play. Maybe NO has a lot of worthless 3rd and long yardage? Maybe Was threw more (and well) against Jax?

80
by underthebus (not verified) :: Tue, 11/07/2006 - 8:26pm

So does St Louis beat SEA on the road?

81
by Mnotr (not verified) :: Tue, 11/07/2006 - 8:29pm

Does anyone know how much regression to the mean DVOA uses? I know that an extreme performance in one area, like say 3rd down defense, doesn't boost a team's defensive DVOA much, but is that because extreme performances are regressed some or because 3rd down defense isn't considered very important to DVOA?