Writers of Pro Football Prospectus 2008

Most Recent FO Features

WareDeM05-1.jpg

» Week 7 DVOA Ratings

Denver: great team, or the greatest team? Would you be satisfied with "one of the ten greatest teams?" Plus: hard times in the NFC South, where defense goes to die.

22 Nov 2005

Week 12 DVOA Ratings

by Aaron Schatz

FOXSports.com commentary is here.

We didn't stick it in Audibles, but I sent this e-mail to the other Outsiders on Sunday night:

I know that a lot of people will want to know if today's Colts win will move them back to number one in DVOA. I have not run the numbers but I'm guessing it will. Partly that is the Colts beating the Bengals. Partly that is the 49ers opponent adjustment becoming slightly smaller, and the Browns (another team in Indy's past) had a big win as well. But also the Jags narrowly beat a bad team, the Seahawks narrowly beat a bad team, and the Steelers lost. It's possible the Broncos could be number one. My nightmare has the Chargers at number one.

When I ran the numbers, then, I was surprised to see this:

1) Cincinnati
2) San Diego
3) Denver
4) Jacksonville
5) Indianapolis

The weighted ratings switched the Bengals and Broncos and moved the Colts ahead of the Jaguars, but the Colts were still fourth behind a team with four losses and a team they had just beaten. Except they only beat them by a few points, and therefore DVOA saw both the Bengals and Colts playing well against a top team and moved them both up.

After I was done banging my head against the wall and cursing the existence of the Houston Texans, it was time to figure out what to do. It's one thing to defend controversial numbers. It's another thing to defend controversial numbers that I don't agree with. But to defend controversial numbers that clearly seem to show a flaw in the system? A man can only do that for so long.

The answer was clear: it was time to put aside other things (exercise, lawn work, sleep) and start looking at some of the possible improvements that were supposed to wait until the off-season. And lest anyone think that I'm just a wuss who can't take criticism on the Internet, this issue was not just about the Colts. 2005 definitely has the most skewed schedules in recent NFL history, with whole divisions above or below average. Every AFC West team, for example, has a non-adjusted rating of 14th or better. Every AFC East team has a non-adjusted rating of 19th or worse.

From 1998-2004, only three teams had a gap of 15% or greater between DVOA and VOA: the 2000 Ravens, 2000 Chargers, and 1998 Bears.

Through 11 weeks of the 2005 season, there were eight teams with a gap of at least 15% between DVOA and VOA: the Colts, Bucs, Panthers, and Seahawks were all getting moved way down, while the Redskins, Ravens, Texans, and Vikings were all getting moved way up.

I sat down and tried one of the more popular suggestions to solve the schedule problem, second-order opponent adjustments. Previously, opponent adjustments were based on the unadjusted ratings of opponents. So if the Colts play the Bengals, we adjust the Colts offense based on the numbers for the Bengals defense.

With second-order adjustments, we're now running things twice. So if the Colts play the Bengals, we adjust the Colts offense based on the numbers for the Bengals defense, after the Bengals defense has itself been adjusted for the offenses it has faced all season.

It wasn't enough to just try this change, however. I could not make the change unless it improved the year-to-year correlation of DVOA as well as the ability of DVOA to predict future performance. That meant putting these numbers together for a few years, not just 2005. It helped that the NFL decided to schedule a Monday night game that didn't hold much interest for me.

The result was a resounding success. Based on correlation coefficient (explained here), DVOA with second-order adjustments was more accurate from year to year and from the first half of the season to the second half:

Correlation from year to year, 2002-2005:
Wins: .253
Pts Dif: .309
VOA: .311
DVOA with first-order adjustments: .358
DVOA with second-order adjustments: .374

Correlation from Weeks 1-9 to Weeks 10-17, 2002-2004:
VOA: .437
DVOA with first-order adjustments: .373
DVOA with second-order adjustments: .447

(Note that these correlations are somewhat low because 2003 is an outlier year, where the rating was nowhere near as stable as other seasons. Any time I try to measure the accuracy of DVOA, it looks more accurate the more non-2003 seasons I include.)

Correlation from Weeks 1-9 to Weeks 10-17, 2004 only:
VOA: .547
DVOA with first-order adjustments: .571
DVOA with second-order adjustments: .620

Using second-order adjustments not only improves the accuracy of DVOA overall, it also gives Indianapolis a rating that makes a lot more sense. But that rating is still not number one. It's number three for the full year, and number two in weighted DVOA (a.k.a. FOXSports.com power ranking) behind Denver.

You'll find a lot about Indianapolis and Denver, along with comments on all 32 teams, over in the commentary for the math-o-phobic on FOXSports.com.

It's going to be interesting to see what happens this week. Denver fans, who few weeks ago believed I was the devil incarnate, now get to point to us and say, "See, we're really the best team in the NFL right now." And Indianapolis fans, who loved the fact that a Boston-based sportswriter was constantly defending Peyton Manning from his critics, now have to think of new curse words to describe my existence.

In the meantime, bear with us as we gradually switch all the DVOA ratings on the site over to the second-order opponent adjustments. For now, only the 2005 TEAM numbers will have the second-order adjustments. Individual numbers and years before 2005 are still old school for now. There's a lot of new code to be written into the tools I use to create my numbers and tables, and doing the second-order opponent adjustments manually each week until the coding is done will take a lot of time. We'll include the first-order numbers each week for comparison purposes.

We also may fiddle with the tables a bit, moving different stats around so we can give lots of good info but also fit it all on the page. This week we're also adding VARIANCE for offense only and defense only to those pages, and weighted special teams DVOA to that page.

What about trying third-order or infinite-order opponent adjustments? That can probably wait for the off-season. The next thing on the to do list, if I can get time, is to test which is better for forecasting future performance: DVOA, weighted DVOA, estimated wins, or "weighted" estimated wins.

As I noted in the mailbag last week, this is not the first time we've dramatically changed the formula in the middle of the season. We didn't even have special teams until the middle of 2003. But this stuff gets a lot more attention when you're writing for one of the biggest sports websites on the Internet instead of just for a little site you started with your college buddies.

* * * * *

These are the Football Outsiders team efficiency ratings through 11 weeks of 2005, measured by our proprietary Defense-adjusted Value Over Average (DVOA) system that breaks down every single play and compares a team's performance to the league average based on situation in order to determine value over average. (Explained further here.)

OFFENSE and DEFENSE DVOA are adjusted based on strength of opponent as well as to consider all fumbles, kept or lost, as equal value. SPECIAL TEAMS DVOA is adjusted for type of stadium (warm, cold, dome, Denver/Mexico City) and week of season.

WEIGHTED DVOA represents an attempt to figure out how a team is playing right now, as opposed to over the season as a whole, by making recent games more important than earlier games. This is the statistic used for the FOXSports.com Power Rankings. WEIGHTED DVOA for offense, defense, and special teams is available on those separate pages.

IMPORTANT: Beginning with Week 12 of 2005, DVOA is based on second-order opponent adjustments. Until other years are updated with this new system, only first-order DVOA (second table) can be compared to previous seasons.

As always, positive numbers represent more points so DEFENSE is better when it is NEGATIVE.


TEAM TOTAL
DVOA
LAST
WEEK
W-L WEIGHTED
DVOA
RANK OFFENSE
DVOA
OFF.
RANK
DEFENSE
DVOA
DEF.
RANK
SPECIAL
DVOA
S.T.
RANK
1 CIN 39.0% 2 7-3 34.4% 3 29.2% 3 -9.0% 9 0.7% 15
2 DEN 37.6% 3 8-2 41.2% 1 28.2% 5 -9.7% 6 -0.2% 21
3 IND 35.4% 7 10-0 35.3% 2 32.9% 2 -9.0% 8 -6.6% 31
4 SD 34.4% 4 6-4 34.0% 4 38.5% 1 4.4% 22 0.3% 17
5 JAC 27.2% 1 7-3 25.6% 6 3.7% 14 -22.3% 2 1.3% 14
6 NYG 26.9% 8 7-3 26.7% 5 7.6% 12 -9.1% 7 10.1% 2
7 PIT 26.5% 6 7-3 24.9% 7 9.0% 10 -19.2% 3 -1.7% 25
8 SEA 25.5% 5 8-2 24.9% 8 29.0% 4 4.1% 21 0.6% 16
9 CHI 21.2% 14 7-3 20.1% 9 -17.9% 26 -38.9% 1 0.2% 18
10 DAL 18.2% 10 7-3 18.4% 10 3.4% 15 -12.4% 5 2.4% 9
11 KC 13.7% 12 6-4 14.0% 12 16.6% 7 -0.1% 17 -2.9% 26
12 WAS 13.7% 9 5-5 14.6% 11 6.5% 13 -7.4% 10 -0.2% 20
13 CAR 8.8% 11 7-3 10.2% 13 -8.7% 19 -15.1% 4 2.4% 10
14 PHI 7.6% 15 4-6 5.3% 15 8.9% 11 -5.3% 14 -6.6% 32
15 OAK 7.1% 17 4-6 9.5% 14 9.0% 9 1.0% 19 -1.0% 23
16 NE -2.7% 13 6-4 -4.0% 16 19.5% 6 24.3% 30 2.1% 11
TEAM TOTAL
DVOA
LAST
WEEK
W-L WEIGHTED
DVOA
RANK OFFENSE
DVOA
OFF.
RANK
DEFENSE
DVOA
DEF.
RANK
SPECIAL
DVOA
S.T.
RANK
17 TB -3.3% 18 7-3 -6.8% 18 -9.5% 21 -6.3% 11 -0.1% 19
18 ATL -5.8% 19 6-4 -6.8% 17 9.8% 8 14.2% 26 -1.4% 24
19 MIA -9.0% 16 3-7 -14.1% 25 -19.2% 27 -5.6% 13 4.6% 6
20 CLE -9.3% 24 4-6 -10.5% 22 -4.1% 17 7.0% 24 1.8% 13
21 BAL -12.1% 22 3-7 -8.6% 19 -22.1% 30 -6.0% 12 4.0% 8
22 GB -12.8% 20 2-8 -9.7% 20 -8.8% 20 0.2% 18 -3.8% 28
23 DET -12.9% 26 4-6 -10.1% 21 -12.7% 24 -4.5% 15 -4.7% 29
24 TEN -14.1% 27 2-8 -13.0% 24 -4.6% 18 14.3% 27 4.8% 5
25 MIN -14.2% 21 5-5 -12.4% 23 -14.9% 25 1.3% 20 2.0% 12
26 BUF -18.9% 23 4-6 -19.4% 26 -21.9% 29 5.5% 23 8.6% 3
27 ARI -23.0% 29 3-7 -20.7% 27 -12.1% 23 15.4% 28 4.5% 7
28 NO -24.0% 28 2-8 -23.7% 28 -10.0% 22 8.8% 25 -5.2% 30
29 STL -27.8% 25 4-6 -27.4% 29 -3.6% 16 23.9% 29 -0.3% 22
30 NYJ -31.6% 30 2-8 -31.8% 30 -30.3% 31 -1.8% 16 -3.1% 27
31 HOU -47.1% 31 1-9 -43.8% 31 -21.3% 28 36.2% 32 10.5% 1
32 SF -67.9% 32 2-8 -69.2% 32 -49.0% 32 24.8% 31 5.9% 4

  • 1st ORDER DVOA is the "older style" DVOA from before Week 12 2005 which only runs opponent adjustments once. This stat should be used to compare with previous seasons.
  • NON-ADJ TOTAL VOA is VOA without any adjustment for opponent strength, luck in recovering fumbles, or the effects of weather and altitude on special teams.
  • ESTIMATED WINS uses a statistic known as "Forest Index" that emphasizes consistency as well as DVOA in the most important specific situations: red zone defense, first quarter offense, and performance in the second half when the score is close. It then projects a number of wins adjusted to a league-average schedule and a league-average rate of recovering fumbles. Teams that have had their bye week are projected as if they had played one game per week.
  • PAST SCHEDULE lists average DVOA of opponents played this season, ranked from hardest schedule (#1, most positive) to easiest schedule (#32, most negative). It is not adjusted for which games are home or road.
  • FUTURE SCHEDULE lists average DVOA of opponents still left to play this season, ranked from hardest schedule (#1, most positive) to easiest schedule (#32, most negative). It is not adjusted for which games are home or road.
  • VARIANCE measures the statistical variance of the team's weekly DVOA performance. Teams are ranked from least consistent (#1, highest variance) to most consistent (#32, smallest variance).


TEAM TOTAL
DVOA
W-L 1st ORD
DVOA
RANK NON-ADJ
TOTAL VOA
ESTIM.
WINS
RANK PAST
SCHED
RANK FUTURE
SCHED
RANK VARIANCE RANK
1 CIN 39.0% 7-3 37.4% 1 43.8% 7.7 4 0.1% 17 -2.2% 17 19.7% 17
2 DEN 37.6% 8-2 34.0% 3 34.1% 7.8 3 8.7% 7 7.1% 10 23.8% 7
3 IND 35.4% 10-0 26.1% 5 46.9% 8.0 2 -16.2% 31 12.7% 4 5.2% 32
4 SD 34.4% 6-4 34.3% 2 28.7% 8.1 1 8.4% 9 16.4% 2 10.8% 27
5 JAC 27.2% 7-3 28.2% 4 26.1% 6.7 8 3.1% 11 -21.0% 32 23.7% 8
6 NYG 26.9% 7-3 22.7% 6 35.4% 7.0 7 -4.6% 24 14.3% 3 22.0% 11
7 PIT 26.5% 7-3 19.4% 8 30.3% 7.1 6 -1.0% 20 9.8% 7 17.1% 21
8 SEA 25.5% 8-2 20.4% 7 36.7% 7.4 5 -16.3% 32 -4.2% 20 8.4% 28
9 CHI 21.2% 7-3 16.0% 11 17.8% 6.5 10 -9.2% 28 -3.7% 19 35.1% 3
10 DAL 18.2% 7-3 16.8% 10 19.1% 6.6 9 1.9% 13 12.1% 6 21.7% 14
11 KC 13.7% 6-4 7.6% 13 10.7% 6.3 11 0.1% 16 25.6% 1 7.4% 29
12 WAS 13.7% 5-5 17.0% 9 -5.0% 6.0 12 8.6% 8 6.0% 12 27.9% 6
13 CAR 8.8% 7-3 8.3% 12 25.2% 5.5 14 -11.2% 29 -6.6% 25 18.0% 19
14 PHI 7.6% 4-6 4.9% 15 -1.1% 5.3 16 9.6% 4 0.4% 14 21.0% 16
15 OAK 7.1% 4-6 4.9% 14 9.5% 5.6 13 10.3% 3 8.2% 9 6.4% 31
16 NE -2.7% 6-4 2.4% 16 -11.6% 5.3 15 9.2% 6 -13.4% 29 6.9% 30
TEAM TOTAL
DVOA
W-L 1st ORD
DVOA
RANK NON-ADJ
TOTAL VOA
ESTIM.
WINS
RANK PAST
SCHED
RANK FUTURE
SCHED
RANK VARIANCE RANK
17 TB -3.3% 7-3 -6.2% 19 11.0% 4.8 18 -15.1% 30 -4.4% 22 17.7% 20
18 ATL -5.8% 6-4 -4.6% 17 8.5% 4.5 19 -8.3% 27 -0.2% 15 14.7% 23
19 MIA -9.0% 3-7 -6.0% 18 -9.1% 4.3 20 -3.5% 23 -4.3% 21 22.3% 10
20 CLE -9.3% 4-6 -8.7% 22 -14.8% 4.9 17 1.4% 15 12.2% 5 15.4% 22
21 BAL -12.1% 3-7 -6.6% 20 -22.4% 4.1 24 10.8% 2 -1.1% 16 18.5% 18
22 GB -12.8% 2-8 -11.5% 23 -6.5% 3.6 26 -0.9% 19 8.4% 8 21.4% 15
23 DET -12.9% 4-6 -11.7% 24 -14.3% 4.2 22 -0.5% 18 1.5% 13 35.5% 2
24 TEN -14.1% 2-8 -16.1% 25 -12.4% 4.1 23 1.6% 14 -6.0% 24 21.9% 13
25 MIN -14.2% 5-5 -7.8% 21 -23.9% 4.2 21 2.4% 12 -2.4% 18 22.7% 9
26 BUF -18.9% 4-6 -20.8% 28 -10.5% 3.7 25 -6.8% 26 7.0% 11 29.6% 4
27 ARI -23.0% 3-7 -18.9% 26 -14.8% 2.9 29 -4.6% 25 -5.2% 23 12.5% 24
28 NO -24.0% 2-8 -23.9% 29 -20.6% 3.4 27 -3.4% 22 -8.0% 26 28.7% 5
29 STL -27.8% 4-6 -20.6% 27 -22.5% 3.0 28 -1.2% 21 -15.0% 30 12.1% 26
30 NYJ -31.6% 2-8 -29.4% 30 -39.6% 2.3 30 7.3% 10 -8.4% 27 12.2% 25
31 HOU -47.1% 1-9 -38.1% 31 -58.9% 1.4 31 16.0% 1 -19.6% 31 21.9% 12
32 SF -67.9% 2-8 -61.2% 32 -69.8% 0.1 32 9.4% 5 -9.9% 28 48.3% 1

Posted by: Aaron Schatz on 22 Nov 2005

117 comments, Last at 02 Dec 2005, 7:43pm by RBELL

Comments

1
by pawnking (not verified) :: Tue, 11/22/2005 - 8:04pm

The fox website was kind of screwy when I checked it.

2
by James Gibson (not verified) :: Tue, 11/22/2005 - 8:11pm

My guess is that it is best to keep adding new order adjustments to the ratings, but at some point there will be a convergence or at least diminishing returns where the time it takes to make the new adjustments won't be worth it any more. Those 2nd order adjustments cause a pretty big jump in the correlations you reported. Looking at the oscillations from VOA to 1st order DVOA to 2nd order DVOA, I'm guessing you may have to go pretty far out in order before it stabilizes.

3
by Yuri (not verified) :: Tue, 11/22/2005 - 8:18pm

"Power rankings are swell, but if you are holding the Lombardi Trophy at the end of the Super Bowl, you're the best team in football."

This seems a bit against ths spirit of the site. I thought Aaron also said that the "2001 Patriots were a colossal fluke." So winning matters, or does not matter?

4
by admin :: Tue, 11/22/2005 - 8:26pm

Sure, I'll answer that. The entire point of the season is to win the Super Bowl. The entire point of the research we do on this site is to figure out how to build teams with a better chance to win the Super Bowl or which teams are more likely to win the Super Bowl. So when all is set and done, no matter how much luck they needed to get there, the team that wins the Super Bowl achieved the goal that all 32 teams had from the beginning of the year.

Of course, then we start thinking about the next season, and a team that wins a Super Bowl on a fluke -- like the 2001 Patriots -- doesn't start that next season as "a proven winner" in our eyes.

5
by Brian (not verified) :: Tue, 11/22/2005 - 8:27pm

Awesome, that's great that it actually did improve the correlation. As a Colts fan, it's obvious that there are some issues even though they are undefeated. If the Bengals aren't offsides on that first drive, there's a good chance they win that game.

How is the Colts' variance so low given their seemingly wildly different games this season? I suppose since the defense was racking up more numbers early and the offense is racking up more numbers lately the total DVOA for each game was still pretty even?

6
by Sean (not verified) :: Tue, 11/22/2005 - 8:31pm

All I know is that something has to be terribly wrong if the Jets aren't the worst team in the league after the weighted DVOA. I guess they didn't truly fold up the tent until three weeks ago and they're still riding those close games against Jacksonville and Cincinnati, but for the last two weeks in particular, they have been far less competitive than Houston or San Francisco. They've been downright hopeless.

(And yes, I'm being facetious...sort of.)

7
by Brad (not verified) :: Tue, 11/22/2005 - 8:32pm

As a Colts fan and FO devotee, I completely understand why they aren't #1 right now. But what I find interesting is that while people carp about their easy schedule, I hear very little about Seattle's schedule -- one which has them with the best record in the NFC.

8
by Richie (not verified) :: Tue, 11/22/2005 - 8:37pm

And when Brett Favre gets traded to Miami, it will work out well because his old flame, Mary, lives there.

9
by Terry (not verified) :: Tue, 11/22/2005 - 8:41pm

Top 5 Estimated Wins

THROUGH WEEK 11

1. SD (8.1)
2. IND (8.0)
3. DEN (7.8)
4. CIN (7.7)
5. SEA (7.4)

Wow.

10
by jim's apple pie (not verified) :: Tue, 11/22/2005 - 8:44pm

At least one part of the nightmare scenario came true: the Chargers are now the leader in estimated wins. There is somebody here that always comments that estimated wins is usually a better predictor of who is going to go to (and win) the Super Bowl. But if the Chargers have even one more close loss, and 5 dominating wins, they could be the leader in estimated wins at the end of the season and not be in the playoffs.

Is that going to happen? Probably not. It looked a lot more likely before Pitt and Cincy loss this last weekend, plus Jax survived a close scare against woeful Tenn. I think we're still looking at one 10 win team not making the playoffs, and we could even see two if KC manages to play well enough the rest of the year. It's even possible that we might see an 11 win team not make the playoffs! I doubt that we'll see Jax, Cincy, Pitt, and SD all play well enough for that to happen, but it could. And that seems to be a pretty damning indication of the schedule disparity (as well as team ability disparity) in the NFL this year.

11
by bobman (not verified) :: Tue, 11/22/2005 - 8:49pm

Nice one, Richie, but I think Mary has hooked up with her old high school flame--and I don't mean Woogie.
She was always a Niners fan, anyway.

Regarding the idiotic exercise of the Colts and their chances of running the table, where are we now folks? (he asks, continuing the idiocy)... Despite a pretty decent win, I still come in at about 12 percent chance, which is pretty good, if you think about it.

12
by Aaron Boden (not verified) :: Tue, 11/22/2005 - 8:49pm

I would love to see Favre in Chicago, even though we all know that would not happen. He would go to a team with many peices in place, and he would still be able to keep the late season cold weather advantage.

Hey, I can dream can't I?

13
by Alec (not verified) :: Tue, 11/22/2005 - 8:52pm

So SF, as well as they have intermittently played, weights to .1 wins? Wow, projected winless against their opponents. How often has that come up that they're that bad?

14
by Chris (not verified) :: Tue, 11/22/2005 - 8:53pm

It finally happened.

The Giants have lost their stranglehold on the number 1 ST rank.

Houston better run with it. It might be the only thing they'll win for the rest of the year.

15
by Comrade Jason (not verified) :: Tue, 11/22/2005 - 8:54pm

#7: I think that's because everyone is still assuming Seattle will go into their late season collapse, although I guess we'll find out this week when they play NYG. Plus, as you said, the easy schedule makes it hard to take them seriously. The way things are going, NYG this week might be their only tough game: the Eagles certainly aren't what they were, and by the time they play Indy, the Colts might have already clinched HFA.

15
by bobman (not verified) :: Tue, 11/22/2005 - 8:54pm

New curse words to describe Aaron? This Colts fan has a curse for ya, buddy: May you have to mow your lawn all damn winter! For eternity. bwa-ha-ha-ha!

Seriously, what lawn work do you have in mid-November? The leaves should all be gone in Boston--unlike Seattle where I'm still raking. But even in Seattle, where grass grows faster in the winter than summer, it's pretty minimal work.

And if I give up sleep to read FO, you can damn well give up sleep to slice and dice, write code, analyze, and otherwise, do my bidding. Hmm, maybe that came out wrong. Thanks for all the work! Now go cut that grass.

17
by Brian (not verified) :: Tue, 11/22/2005 - 9:09pm

This Colts fan has their chance to run the table at 11%.

Which brings up some silly questions I had: What happens if they get to 14-0? 17-0? Is this equivalent to the '72 Dolphins even if they lose later? Or is it strictly defined as a perfect season?

18
by Playit (not verified) :: Tue, 11/22/2005 - 9:15pm

"Of course, given the large home-field advantage Dallas normally has on Thanksgiving, Denver could be dropped out of the top spot as early as next week"

What is the actual numbers for this? I see where an average home field advantage is around 17% VOA, any idea what the calculated Turkey Day DVOA bonus is?

Also I'd guess, like the other poster above, that infinite loops on weighted averages are going to be necessary to accurately reflect these relative stats. I know others (Sagrin (sp)?) have been using the infinite loops for some time now because of that issue.

19
by Purds (not verified) :: Tue, 11/22/2005 - 9:26pm

I am a Colts fan with no way to judge these things from Sunday, as I had to suffer though Jets-Broncos until bitter last second (stupid tv contracts!), so please help me out:

1) Is the Bengals offense that good?
2) Did the Colts blitz at all?

I ask these in trying to guess Indy's chances for Monday night against Pitt. I know Ben will be coming back, so that will change things, but I am trying to get a sense of whether the Colts D has reverted all the way back to 2004, or if they just had a bad day?

On a side note, I am very glad that Cincy went no-huddle, so that the Colts could see it now and try to deal with it, as opposed to seeing it for the first time in the playoffs.

Finally, I think the DVOA standings are about as good as they can be. As opposed to past years, when all the big boys played one another by this point in the season, this year the big teams all seem to be avoiding one another in the schedule, or they have a significant injury (QB or the like) to skew the outcome.

Okay, one last thing: Are the Chargers really all that good if they are 6-4 with the highest estimated wins, or are they simply a team that can have a great day against anyone, but is not consistent?

Whew.

20
by Jeremiah (not verified) :: Tue, 11/22/2005 - 9:27pm

It looks like the only thing keeping the Colts out of the top spot is their special teams performance. That brings me to the main topic of this post.

I've wondered for a long time if there shouldn't be a different formula for incorporating the special teams DVOA into the final DVOA.

For a team in the middle of the pack, special teams can mean all the difference between winning and losing. But for a team with a stellar offense (Indianapolis) or a terrible defense (Houston), special teams don't seem to contribute as much to the final outcome. In addition, the more consistent a team, the less important special teams becomes.

Perhaps ST DVOA could be split into offensive and defensive parts. FG kicking, and punt/kick returns could go into the offensive category, while kicking and punting can go into the defensive category. Each can have their own opponent adjustment, and their own strength of contribution to the total score based on the corresponding offense/defense of the team overall. Then, a final (small) adjustment based on the overall consistency of the team.

It would require some work to identify proper contribution values, but if it improves the system I think it'd be worthwhile. Does this seem reasonable to anyone, or am I just crazy? :)

(For some reason, the formatting of this post is not being saved.)

21
by mm (not verified) :: Tue, 11/22/2005 - 9:28pm

is there something to the fact that W/L is a discrete variable, whereas the variables listed here are all continuous? i mean, the Pats look like they have the easiest time of it going forward in the AFC, bar JAC, but if you consider how awful the Jets are, plus the fact that that awfulosity can only ever possibly add +2 in the W column, no matter were the Jets to fall off the edge of the known world. In other words, they could plummet to -infinity, yet the Patriots are limited to a total of 2 wins from this fact.

22
by Jeremiah (not verified) :: Tue, 11/22/2005 - 9:30pm

Well I guess the formatting was saved. Just that the preview function strips out all the newlines I put in.

23
by Purds (not verified) :: Tue, 11/22/2005 - 9:30pm

Let me restate with a stab at analysis:

I think this year may be a particularly hard year for DVOA as a predicting tool because who knows what teams do (strategically, or psychologically) when playing a bad team. How could athletes/coaches prepare to play Houston the same way they would prepare to play Denver? Or, should coaches of good teams even prepare with as much creativity against those very bad teams? Last year, for example, we all watched NE go pretty basic against bad teams, just doing enough to win, and we all applauded Belichick's not "tipping his hand" with anything unusual, keeping those strategy quirks for the playoffs.

24
by Daniel (not verified) :: Tue, 11/22/2005 - 9:35pm

With SD with the most estimated wins, how good is San Diego really? Considering they lost their 4 games by a total of 12 points, DVOA seems to love them. Shouldn't there be more weight given to the actual win or loss?

Is San Diego better than Indy? Denver?

25
by Fnor (not verified) :: Tue, 11/22/2005 - 9:44pm

There's actually no weight given to the actual win/loss. It's calculated by play and then adjusted by other stuff.

Also: hooray, higher correlation! That's great! Though sorry if we barraged you with suggestions about SoS. By the end of the thread I wasn't even sure what I had said in the beginning...

And if anyone's wondering (no one is), yes, I'm avoiding Ned's column. I've got work to do and a game to chart and no time to get in a fight over Kyle Orton :/.

26
by Blake (not verified) :: Tue, 11/22/2005 - 9:48pm

Are opponent adjustments dynamic?

Say if a team like the 99 Seahawks goes up 8-2 at the start of the season then bombs the rest, wouldn't that favour a team that played them later in the year and hurt a team that played them earlier?

27
by zip (not verified) :: Tue, 11/22/2005 - 9:50pm

I think San Diego against Indy or Denver(again) would be a helluva game, and DVOA agrees with me.

28
by mm (not verified) :: Tue, 11/22/2005 - 9:56pm

@fnor: the weight given to W/L exists in the real world, however (i.e., the seedings in the playoffs).

my question is: does the continuous nature of these variables fail to reflect the discrete nature of the variables that matter in the end?

29
by Pat (not verified) :: Tue, 11/22/2005 - 10:12pm

My guess is that it is best to keep adding new order adjustments to the ratings, but at some point there will be a convergence or at least diminishing returns where the time it takes to make the new adjustments won’t be worth it any more.

It might, but it might not. It's entirely possible the system is unstable, and as you take more and more orders, the corrections start becoming wilder and wilder.

One quick thing I would really recommend Aaron check is whether or not the size of the correction for the second-order (in terms of DVOA percentage) is smaller than the first-order. That's a safe way to estimate whether or not the convergence might be a little unstable.

If it is unstable, though, then it might be worth looking into matrix deconvolution algorithms. Fundamentally the two methods are the same, but convergence will only work when you start near the "correct rating". In other words, you're probably only screwed if the San Francisco 49ers would have played the Colts nine times.

30
by TomC (not verified) :: Tue, 11/22/2005 - 10:15pm

Fnor -

I can't believe it's come to this, but the Kyle Orton fight now has a thread all its own. Move over Peytom Branning, make room for Kylex Grortman. He's got two injured legs, one scraggly porn 'stache, and a combined minus 50-something DVOA in one package.

31
by David Brude (not verified) :: Tue, 11/22/2005 - 10:17pm

Glad to see that something that I had mentioned to Aaron in an e-mail once and I'm sure others mentioned finally got incorporated into DVOA. That being second order adjustments. From my experience the gains you get from 3rd and 4th order adjustments tend to be pretty minimal in value especially once you get more games under your belt. It could be that with these huge schedule disparities that it takes more adjustments for convergence.

Regarding San Diego. Let's look at their 4 losses. They lose to Dallas week 1 without Gates. Lose to Denver in Denver. Lose to Philly on a few fluke plays. Lose to the Steelers by 2. They've had almost as hard a shcedule as Indy's has been easy and their 4 losses are all by less than 4 points.

32
by David Brude (not verified) :: Tue, 11/22/2005 - 10:21pm

Pat does makes a very valid point when he says "It might, but it might not. It’s entirely possible the system is unstable, and as you take more and more orders, the corrections start becoming wilder and wilder.

One quick thing I would really recommend Aaron check is whether or not the size of the correction for the second-order (in terms of DVOA percentage) is smaller than the first-order. That’s a safe way to estimate whether or not the convergence might be a little unstable"

33
by tunesmith (not verified) :: Tue, 11/22/2005 - 10:22pm

I'd love to see SD destroy the colts and then collapse against Denver. The end of the season will be fun.

34
by Brian (not verified) :: Tue, 11/22/2005 - 11:10pm

Re: #19
Yes, the Colts blitzed more than I have seen them all year. Thorton even lined up on the line on quite a few plays. And they still didn't get to the QB. Not good...

35
by Craig B (not verified) :: Tue, 11/22/2005 - 11:12pm

Re #6
Um...the Jets never played Cincinnati. You might have meant San Diego.

36
by Phil (not verified) :: Wed, 11/23/2005 - 12:19am

As much as I like to view statistics, I am very confused by all this DVOA, twice-filtered, artificial number-crunching. I see Indy without a loss, but they did have a tough time with Cincy. Denver is now playing well, despite the choke against the Giants and the mid-game scare from Philly. The Bears?? No offense to speak of, yet they are 7 and 3. The Panthers were okay until they laid the egg in the Second City, and who can figure out the Jags?? I see them as capable of flopping when expected to win big.

So, is Indy the true great team, or will they get exposed in December?? I'd like to see SOME team go unbeaten so we don't have to hear the self-serving '72 (rotting) Fish any more!

37
by King (not verified) :: Wed, 11/23/2005 - 12:27am

The correlation you ran is comparing those variables to what? Comparing the previous week's result to the next week's result for each variable and the same for the year results? Could I get clarification?

DVOA is obviously not a perfect stat. The best team in the league could always just do enough to beat each opponent even if they are playing the Texans or the Broncos. They play down to competition but always get the job done by just being that much better when it matters. That would obviously destroy play by play success ratings. Estimated Wins could be the better stat, specifically in the circumstance I laid out above.

With special teams, I agree with Jeremiah. Special teams have a smaller effect on good offensive teams or horrendous defensive teams. Field position matters little for teams on opposite sides of the continuum. They either can't stop anyone anyways or will score anyways regardless of field position (atleast it has a much smaller effect than middle of the pack teams). Some kind of incorporation of the idea of win probability ala The Hidden Game of Football where events after a game is decided should have no effect. The colts have had a punt team this year (kickoff team is another matter) that hasn't been that bad, but it has been destroyed by fluke irrelevant plays. Against the Texans they had two muffed punts where the ball went off guys running down to recover (including off someone's head). It destroyed their DVOA, but it was really irrelevant.

38
by Pat (not verified) :: Wed, 11/23/2005 - 12:35am

Look, the statistics aren't that tough to understand. Really.

Here's the problem with corrections in a nutshell: suppose you look at Philadelphia last year in the NFC, and say hey, they aren't that great, they played in a crappy NFC.

But the problem is this: does the rest of the NFC look crappy because they played Philly? It's called a "convolution". You can either solve the problem the way that Aaron's saying (start by using the numbers to correct themselves, then the corrected numbers to correct themselves, etc.), or by matrix deconvolution, which you learned if you ever took a linear algebra class in college.

39
by thad (not verified) :: Wed, 11/23/2005 - 12:35am

Purds,
the Bengals offense looked very good.
The Colts did not seem to blitz a whole lot, hardly at all.
But the Colts were ahead the whole game, even though the Bengals offense looked good, the Colts looked unstoppable.
At one point in the third quarter the Bengals had scored 17 straight points.
The Colts were still winning and I never really got the sense that the Colts would lose.
The Colts pass defense looks to have slipped a notch though recently.

40
by Pat (not verified) :: Wed, 11/23/2005 - 12:37am

King: I don't agree regarding special teams. If a strong team plays a strong team, special teams will play a big role. When there's a big disparity, I agree, but it's not pointless for any team.

41
by Fnor (not verified) :: Wed, 11/23/2005 - 12:57am

King: Tell me special teams didn't mean anything in the 1st quarter of StL/IND. It left the Colts reeling. They overcame it, sure (with a little help from Bulger's shoulder), but it could have easily decided the game. Special teams is always important.

And thanks for reminding me of my linear algebra class, Pat. I had it with the same HUGE ex-soviet mathematician/programmer that I took Numerical Methods with. We would all wince after getting a problem wrong, because we always imagined a threat at the end.

"Your answar is not wisin se margin of errar. Sat is unacceptable. [I vill break you.]"

Nightmares.

42
by Joon (not verified) :: Wed, 11/23/2005 - 1:27am

special teams matters, but i do wonder about the lumping. like, does it matter if the colts suck at punt coverage? they punt way less than most teams. for great offensive teams, punt coverage should matter less and kick coverage more; likewise for terrible defenses, punt returns should matter less and kick returns more. as for FG kicking, it's a little less clear; it's probably just fine to weight it equally for all teams.

43
by Brian (not verified) :: Wed, 11/23/2005 - 1:52am

#39
The Colts didn't blitz a whole lot, but in the 14 previous games (including preseason) I can only think of a couple times total where they blitzed. And I can't remember them using the 5-man line at all this year.

44
by Nuk (not verified) :: Wed, 11/23/2005 - 2:18am

After the Colts went 8-0, I made up a formula based on DVOA to estimate the odds of the Colts going 16-0. It was 1.8% then. After they beat Houston, and their DVOA dropped, their odds of winning the last 7 dropped to 0.96%. Now, with improved rating and only 6 games left, their odds are up to 5.5%.

45
by Kibbles (not verified) :: Wed, 11/23/2005 - 2:22am

Re #42: From what I understand, not all situations are given the most weight. If a team punts once for a DVOA of -20%, and kicks off 10 times with a DVOA of 2% each time, from what I understand, their special teams DVOA would be 0.

In other words, teams that don't punt don't have special teams DVOA killed by poor punt coverage. However, if the Colts really suck on kickoffs, then they're in for a world of hurt, because with that offense they'll be kicking off a ton.

Special teams are often ignored, but they're every bit as important to a team's success as offense or defense. Ask New York.

46
by Peremptor (not verified) :: Wed, 11/23/2005 - 2:37am

I got two observations. First, is there anyway to factor in Denver's unusually high homefield advantage into DVOA? Maybe the Colts should be fist in DVOA afterall...

Also I don't know if anyone has noticed, but the Chargers have pretty much the same DVOA offensive rating that the fabled 2004 colts had last year (2k4 Colts better pasing offense, 2k5 Chargers better rushing offense). Of course those Colts had a easier schedule (a 0.0% average) than San Diego has had this year so its not obvious. Of course, the Colts did it over 16 games, not 10, but it should be fun to watch the Chargers for the next 6 weeks.

47
by Joon (not verified) :: Wed, 11/23/2005 - 3:36am

Re: #46

it's been fun to watch the chargers all year, actually. i've only seen three of their games (and one of them was the one where tomlinson had like 7 yards), but they're still damn impressive.

Re: #45

i'm not sure that's right. special teams are a little weird, in that they take the point value of the field position and then afterwards try to convert it to a percentage to add to the offense and defense DVOA with the appropriate relative weight (which is something like 3:3:1). so i do think they probably take the correct relative weights for each play, but then i'm less sure that something funny doesn't happen when that gets converted to DVOA %. maybe i'll go re-read the special teams manifesto.

48
by Andrew (not verified) :: Wed, 11/23/2005 - 4:17am

Purds #19:

San Diego has no defense. +4.4% DVOA on defense, 22nd in the league. That is why they have 4 losses. Late game collapses on defense and special teams destroy great offensive outings.

Ditto for Seattle. Seattle has no defense. They are in for a rude awakening soon, both during the season with 3 potential-to-likely losses against the Giants, Eagles, and Colts, and in the playoffs. I've said it before so I'll repeat it here. The Seahawks will lose in Philadelphia, barring last minute miracles they received as when they played against Dallas. The Seahawks work by feeding the ball to Alexander. When he is stopped, as happened in the Jacksonville, Washington, and Dallas games, Seattle loses, barring last minute interceptions of Drew Bledsoe.

Jacksonville Loss (Alexander 14 for 73, 0 TD)
Atlanta Win
Arizona Win
Washington Loss (Alexander 20 for 98, 1 TD)
St. Louis Win
Houston Win
Dallas fluke Win (Alexander 21 for 61, 0 TD)
Arizona Win
St. Louis Win
San Francisco Win

When playing the four Little Sisters of the Poor teams in the NFL (St. Louis, Arizona, San Francisco, and Houston), Alexander has 15 TD's. He has 2 TD's against the Jacksonville, Atlanta, Washington, and Dallas. Matt Hasselback has just 12 TD's in 10 games (6 against non-Little Sisters of the Poor), and Josh Brown has only 3 FG in 5 attempts against the non-Little Sisters of the Poor portion of the league. This is not enough offense to get it done without Alexander.

Mr. Alexander, meet Mr. Trotter.

49
by Andrew (not verified) :: Wed, 11/23/2005 - 4:39am

King:

Special Teams certainly matter. Look at Philadelphia and its Special Teams collapse from this past year to this one due to injuries to Akers (kicker) and Johnson (punter), and poor coverage play.

By DVOA and watching them play, Philadelphia has both a good offense and defense. Unfortunately, the offense has generally had terrible field position to start drives - deep in opposition territory, while the defense is all too often faced with the opposition near, at, or beyond the 50 yard line. It is really tough to stop a team from scoring that can begin several drives per game near midfield, since just 20 yards - a very subpar drive, puts you in field goal range.

The collapse of Special Teams is one of the primary reasons that Philadelphia has become a shell of itself compared to the past 5 years.

50
by Peremptor (not verified) :: Wed, 11/23/2005 - 4:43am

Re: #49

Guess Alexander hasn't done anything to shed the label of being too "soft" after all.

51
by King (not verified) :: Wed, 11/23/2005 - 4:51am

I was unclear on my initial post. In no way did I mean to say special teams are unimportant. I said they are much MORE important for teams in the middle of the continuum of offense and defense. A good offense is going to go ahead and score anyways, while a terrible defense is going to give up the points. Now, not at the same rate as if they had better field position, but no where close to the improvement that would occur for a bad offense.

The colts had a fumble on special teams against the Rams. Ok, obviously a turnover is going to have a big effect, that is an outlier to the situation at hand. That does not negate the fact that the colts easily overcame it with their offense, nor the irrelevant plays that took place against the Texans which are weighted into DVOA but by no way reflect true performance from teams or outcomes from games.

I have a feeling the special teams is weighted correctly, but I too need to look over the manifesto.

52
by Jason (not verified) :: Wed, 11/23/2005 - 5:16am

#48. Another 'Dallas victory is a fluke' argument. C'mon.

You complain about Seattle's defense, and then call the win against Dallas a fluke, which was the game where Seattle made Dallas' offense look positively inept in the red zone. If you want to call that a fluke win, than you have to call the Redskins game a fluke loss, and your point goes out the window. (Especially since Bledsoe's last minute INT happened in a TIE game, with nothing to suggest that Dallas was going to win in the next few moments, since it was a defensive contest throughout).

Seattle has a defense, just a suspect secondary, and even those numbers would be better if they could hold onto a few more surefire INTs. They lead the league in sacks, lead the NFC in sack yards, haven't given up a single 100 yard rusher this year, and in the redzone, they are the second best team in the league at stopping their opponents from crossing the goal line (only Chicago is better).

Your insight into what games Seattle will or won't win in the future is also highly suspect, since you also predicted a loss against Arizona after the bye-week.

And I don't get your point about 'Mr. Alexander meet Mr. Trotter'. The Eagles defense gives up an average of 112 yards per game rushing... but all of a sudden they are going to hold the number 1 rusher in football down? They are going to go from very middle of the pack to holding down Shaun Alexander?

53
by Comrade Jason (not verified) :: Wed, 11/23/2005 - 6:02am

#48: I second #52--if the Dallas win was a fluke, so was the Washington loss. Also, Seattle was missing their top 2 receivers (Jackson & Engram) against Dallas, which let Dallas put 8 men in the box, single-cover the receivers, and get away with it. No way that happens with Jackson & Engram in the game. (Jackson isn't your traditional number 1, but I think he was 3rd in the league in receptions before getting hurt.)

But that being said, they're defense definitely does have problems once you get beyond the line, which seems to be better than expected. I can see them losing to the Giants and Indy (if Indy has anything to play for), but I'd think they'd beat the Eagles, with all their injury problems. Guess we'll see.

54
by putnamp (not verified) :: Wed, 11/23/2005 - 6:33am

I don't see them losing to the Giants at home. The Giants aren't exactly a team without flaws either, and their DVOA looks somewhat inflated by their special teams rank, which I believe the general OF population predicted would return to the mean.

55
by FastEddy (not verified) :: Wed, 11/23/2005 - 9:16am

Andrew #48: More specifically, San Diego has no passing defense. They're pretty fine against the run, but oh momma, that pass D! Bollinger almost pulled off a win at the last second a little while back, they couldn't stop anything until the attempted 2-point convert as I recall.

Actually, San Diego is fun to watch against teams that run a lot. But it might not be as much fun against Indy or Cincy, with their precision passing games. Of course, Brees/Gates/LT will rack up some points of their own, but ultimately they probably will lose to one of those teams. Which is too bad, because I like LT and want to see him get way into the playoffs one year.

56
by Tom W (not verified) :: Wed, 11/23/2005 - 9:50am

The new weighted DVOA rankings look like an improvement. I may be beating a dead horse, but I still feel that to really get an idea of how accurate DVOA is, in terms of measuring teams' relative strengths and weaknesses, you need to convert it to points. I realize that this process, in and of itself, would be subject to disagreement, but people have an intuitive understanding of what it means when you say this team is 5 points better than that team. But I still really don't know what it means, in practical terms, to know that Denver is 4.9% better than Indy. Perhaps this is just a personal mental block. If so, if anyone can help me get around it, I'd welcome your insight.

57
by Tom W (not verified) :: Wed, 11/23/2005 - 10:15am

For example, when you compare how well DVOA correlates to wins with other measurements, are the percentage differentials somehow factored in, or are you just using the rankings, or have I missed something (entirely likely)?

58
by budman (not verified) :: Wed, 11/23/2005 - 10:28am

I personally don't see what the problem is with all this hoopla regarding Indy and how DVOA has them measured. Even though Indy is unbeaten at 10-0, this mental fog record is clouding what DVOA is trying to tell us and what was exposed this past sunday in Cincy.

Indy has only played a team ranked in the top 15 (retrodictive) ONCE so far before sunday and it was way back in week two where they almost lost to a tough Jags team. Their first real test of the season came Sunday and even though they did win and looked to never be in doubt of winning, they were definitely exposed. I have no doubt in my mind that a loss is coming looking at their remaining schedule. I predict the Jags game Dec 11th or at Seattle on Xmass. Hopefully it comes this Monday night so we can move on from this perfect record talk of a team who has probably played the easiest schedule in NFL history for nine weeks.

59
by James Gibson (not verified) :: Wed, 11/23/2005 - 10:28am

Pat - I seriously think it will converge after multiple orders. And it doesn't surprise me at all that 2nd order is better than 1st order. However, if you look down the chart, the difference between 1st order and 2nd order is a lot smaller in percentage points than the difference between 1st order and 0th order (or just VOA).

Here's a list of teams who's adjustment from 1st order to 2nd order is greater than from 0th order to 1st order:

Denver
Kansas City
St. Louis
Chicago

That's it. The other ones that are still swining wildly (like Indianapolis) still have smaller second order corrections than first order corrections. Most of the rest hardly budged on running it rhough a second time.

Looking at those 4 more carefully, Denver's 1st order DVOA and VOA were virtually identical, and the move to 2nd order was only 3 percentage points. Kansas City is a little bit more troubling - down 3 percentage points with the first correction, up 6 perecentage points (for a total of 3) in the second order. Still not a huge gap, and I'm not surprised to see teams with similar schedules (KC and Den) move in the same direction on the second correction. St. Louis is up 2.5 and then down 7.2 for a total of down 4.7. This one would concern me more. Chicago is down 1.2 and then up 5.2. That's not a huge jump in the 2nd, but possibly a little disconcerting.

Still, the number of teams that have a 2nd order DVOA in between the 1st and 0th orders is 24 (out of 32), which are already on there way to convergence, and some of those outside that group really don't move all that much (such as San Diego which went from 28.7 to 34.3 to 34.4). I don't think you'd have to run that many more order corrections before you have a pretty rock solid system.

Now, on the weighted DVOA, I'm a little less convinced.

60
by budman (not verified) :: Wed, 11/23/2005 - 10:31am

OK, major exhaggeration at the end there, but you get my point.

61
by Yakuza Rich (not verified) :: Wed, 11/23/2005 - 11:17am

***Especially since Bledsoe’s last minute INT happened in a TIE game, with nothing to suggest that Dallas was going to win in the next few moments, since it was a defensive contest throughout***

Um, in order to win that game Seattle needed a 50 and 55 yard field goals by Josh Brown in the cold rain AND needed for Jose Cortez to miss a 28 yarder. If that doesn't scream "fluke", I don't know what does.

62
by COINFLIP (not verified) :: Wed, 11/23/2005 - 12:01pm

OFF TOPIC post:
Just wanted to insert a quick Thanks to Aaron and all the FO team for another outstanding year of analysis, insight and great writing.
Happy T-Day and all that, too.

63
by Criag (not verified) :: Wed, 11/23/2005 - 12:05pm

Hey, Aaron, since you are already moving things around, is there any way to tweak the ratings so the Dolphins look like a good team?

64
by admin :: Wed, 11/23/2005 - 12:28pm

The Week 1 ratings are somewhere in the archives...

65
by Xian (not verified) :: Wed, 11/23/2005 - 1:08pm

re: #46 (Denver's HFA)

I can't remember if it's buried in the website (possibly under the ramblings or an old weekly DVOA column), or was simply explained in more detail in Pro Football Prospectus 2005, but home field advantage is calculated for a few different settings. Dome, warm, cold, and Denver.

So, IIRC, it's already factored in. I don't think I'm misremembering that.

66
by admin :: Wed, 11/23/2005 - 1:22pm

That's for special teams only. There is no HFA factored into offense/defense DVOA -- yet. (Remember, the standard HFA washes out over the course of 16 games because until this year, every team played 8 home games and 8 away games.)

67
by Peremptor (not verified) :: Wed, 11/23/2005 - 1:52pm

re 66#

I would think HFA would even out for most teams in the course of 16 games except for Denver though, since they have the best homefield advantage in the league no? I mean the loud, racous fans plus the altitude advantage to which their better acostumed to than the opposition. Its kinda like in soccer for Mexico in their Azteca Stadium where they win/tie like 99.9% of their games.

68
by Scott de B. (not verified) :: Wed, 11/23/2005 - 2:26pm

They play down to competition but always get the job done by just being that much better when it matters. That would obviously destroy play by play success ratings. Estimated Wins could be the better stat, specifically in the circumstance I laid out above.

I don't think estimated wins catches 'playing down to your opposition' since I believe it assumes an average strength of schedule.

69
by hrudey (not verified) :: Wed, 11/23/2005 - 2:26pm

54: "Indy has only played a team ranked in the top 15 (retrodictive) ONCE so far before sunday and it was way back in week two where they almost lost to a tough Jags team"

True. And I'll point out that during the first four games, Jacksonville's offensive line was, in TMQ-speak, cover-your-eyes awful thanks to the stylings of Ephraim Salaam. Since the insertion of Khalif Barnes, the offensive line has been markedly improved, and the offensive DVOA has gone from -12.1% after four games to 3.7%.

On a related note, the game with Tennessee this weekend was nothing like the score indicated. Tennessee came out early with a lot of Cover 0, and once the Jags adjusted and started getting the ball to single-covered WR's late in the second quarter, the game was mostly one-sided. Tennessee got a great KO return to set up their second score, a sack/fumble recovery returned for the third, and the last one came as Jacksonville played prevent.

70
by Brian (not verified) :: Wed, 11/23/2005 - 2:30pm

#51
The 2 muffed punts against the Texans basically gave them 10 points. This is VERY significant and would have made a huge difference in the outcome against a better team.

71
by Dan (not verified) :: Wed, 11/23/2005 - 2:39pm

Excellent changes, Aaron. Second-order (or really, infinite-order) DVOA is what you should have been doing all along, since you want to adjust for how good your opponents have played, not for how they've measured up against their particular competition. So you really are making the stats better (as the correlations prove), not just trying to play around with a team that seems mis-ranked. When you have time in the off-season, you should see how many orders you have to do to get the numbers to become stable (so that additional orders would change the numbers by less than epsilon), and then figure out how to write the programming to do nth-order DVOA (which should probably just be called "DVOA").

And of course I like that you've added the offense & defense variance statistics, which I'd asked about earlier in the year.

72
by Joe (not verified) :: Wed, 11/23/2005 - 3:14pm

Why don't the '86 Mets and Giants get credit for a New York baseball/football double?

Is this a calendar year issue? I demand a strict definition.

73
by Jeremiah (not verified) :: Wed, 11/23/2005 - 3:33pm

Of course special teams is still important. That's also why I suggested splitting it up into offensive and defensive categories, so that certain parts can be treated differently depending on the overall offensive/defensive capability of the team.

For example, the offensive ST (kick/punt returns mostly) numbers of the Colts don't matter anywhere near as much as they might for lesser offensive teams, who need better field position in order to score. The Colts (or Chargers, Bengals, etc.) are likely to score regardless of their own starting offensive field position.

Similarly, defensive ST numbers can be relative to the defensive strength of the team overall. So that kicking/punting for the Colts (SD, Seattle, etc.) is just as (or more) important as it is now, because their defense isn't so great. They don't want to be giving opponents' offenses good field position - they won't be able to stop them.

For a great defense, like the Bears, it may not matter as much what the opposing offense's field position is - they will be stopped anyway. Just as a very poor offense doesn't care what its own starting field position is - they won't score anyway.

74
by Jeremiah (not verified) :: Wed, 11/23/2005 - 3:43pm

And of course, ST plays that give away the ball to the other team with great field position should always be weighted heavily. However, they're likely to be so random and infrequent that they don't have any real predictive power.

75
by fluke2001 (not verified) :: Wed, 11/23/2005 - 3:48pm

haughty pats fan : stats are stupid , winning means everything , individual plays ? and what they mean to an invitation to the super bowl , i dont know , probably not much ,... how do actual wins mean nothing ? i can grasp the scientific breaking down of everything but it should inevitably bring us to some form of reality like the pats are going to win 4 of 5 heheheh...
thats really all i wanted to say

76
by B (not verified) :: Wed, 11/23/2005 - 4:15pm

Re #72: It's a calendar year thing, which, incidently, gives Boston the much less esteemed award for losing both the Superbowl and the WS in the same year.

77
by Andrew (not verified) :: Wed, 11/23/2005 - 4:16pm

Jason #52:

The Seattle-Dallas game. Dallas choked in the end, and coughed up 10 points in 2 minutes. There was no way Seattle had any business winning that game. And Alexander had a piss poor performance.

The Seattle-Redskins game was competitive, and Seattle merely came up short because Alexander was stopped all day long. Completely different outcomes, but both being close games predicated on Alexander being stuffed.

Seattle's defense. On sacks. A high sack total doesn't make you the champ. Just ask Buffalo, Tampa Bay, and Kansas City from last year. High sacks AND a porous secondary implies that Seattle is facing some really bad teams, like Frisco and Arizona and Houston. Seattle has 8 sacks against Jacksonville, Atlanta, Washington and Dallas, and 26 sacks against the Little Sisters of the Poor in 6 games.

Ditto for scoring defense. When most of your games are blowouts against the Little Sisters of the Poor, it would be a shame not to have a low scoring total. That said, 12 other teams have allowed fewer points than Seattle. Since 12 teams go to the playoffs, being 13th in scoring defense is not a playoff caliber showing.

"Your insight into what games Seattle will or won’t win in the future is also highly suspect, since you also predicted a loss against Arizona after the bye-week."

I believe I guessed that Seattle might lose two of the three games following their bye, and that this would be a good test for them. They nearly lost to San Francisco last Sunday, and St. Louis more shot itself in the foot than was defeated, so I consider this almost vindicated. You can be proud of those performances. If I were a Seahawks fan, I wouldn't be.

If you don't understand Mr. Trotter and his style of play, please look at these stat lines for non-system Running Backs (i.e. Denver).

Jordan - 16 for 19 yards, 0 TD
Tomlinson - 17 for 7 yards, 0 TD
Holmes - 18 for 84 yards, 1 TD
Johnson - 7 for 34 yards, 0 TD, 1 FUM
Thompson - 20 for 75 yards, 0 TD
Jones - 16 for 72 yards, 0 TD
Portis - 21 for 67, 1 TD
M. Barber - 13 for 46 yards, 1 TD, 1 FUM
Jones - 8 for 16 yards, 0 TD
T. Barber - 21 for 112 yards, 0 TD

If you think Seattle can win convincingly against teams supposedly not as good as them (such as Atlanta, Dallas, Redskins, Jacksonville) while Alexander rushes for under 3.5 yards per carry and 0-1 TD's, go right ahead and provide the evidence of the games where they have done this.

"but all of a sudden they are going to hold the number 1 rusher in football down? They are going to go from very middle of the pack to holding down Shaun Alexander?"

Again, look at the game lines for Tomlinson, Jordan, Barber/Jones in game 2 with Dallas, etc. The Eagles are fully capable of doing exactly that, and it is what every opponent of Seattle who is not in the NFC West or Houston HAS ALREADY done to Alexander. It won't surprise if this happens again.

The Eagles rushing stat lines are skewed by the Atlanta game, where Trotter was ejected pre-game, and Denver, where they were run over by the same Alex Gibbs line system Atlanta uses after their failed second half comeback.

78
by Josh (not verified) :: Wed, 11/23/2005 - 4:54pm

Interesting concept here (first time I've seen it), but any ranking system that puts the Vikings three spots behind the Packers when they are three games ahead of them in the standings and have beaten them twice needs work.

I'm not outraged or anything....I'm a math guy and I'm intrigued by the whole thing, actually. But I think it points out a bit of a flaw in trying to analyze "power rankings" in a purely statistical fashion. Stats are nice and all, but wins are what really matters in the end.

79
by Kibbles (not verified) :: Wed, 11/23/2005 - 4:55pm

Re #67: I would think HFA would even out for most teams in the course of 16 games except for Denver though, since they have the best homefield advantage in the league no?

You'd think so, but Aaron said he ran the numbers for all the seasons he had DVOA, and it came out that Denver didn't have a greater HFA than any other city in the entire NFL. He said that he felt like that was wrong, but that's what the numbers said.

Re #77: It's not the system that makes the runningbacks, anymore than that it's the system that makes the LBs in Pitt. Quite the contrary. It's the RBs that make the system (and in Pitt, it's the LBs that make the system).

Just check out Portis and Droughns. 9th and 14th in DPAR behind poor offensive lines.

80
by Tony (not verified) :: Wed, 11/23/2005 - 5:02pm

I would call the DVOA rating system the "Kordell Stewarts" of rating systems. You come up with all sorts of excuses every week for why certain teams aren't where they should be. You weight things differently to try to overcome the obvious errors but they still don't correct them. Lastly you use the fluke rule or as Kordell would say "the best team doesn't always win" to explain away embarrassing failures. Have you thought for a second that maybe because your dealing with a team of humans playing a game and much depends on the heart and effort they put into a specific game that this more determines the outcome? You can manipulate all the data you want to try to get the Colts in first place, but I can promise you the Colts won't win the superbowl this year or any other with Manning at QB. You can use whatever stats and formulas you would like but you can't measure the things that win playoff football games and that is why you look so ridiculous each week trying to predict this year's SB winner. Answer this, how many correct superbowl champions would your system have picked in the last 10 years?

81
by putnamp (not verified) :: Wed, 11/23/2005 - 5:04pm

Andrew, you're ignoring again the fact that Seattle was without its #1 and #2 WRs for both the Dallas and Washington game, and in both games, they were frequently facing 8-9 men in the box. Alexander doesn't run well against a 9-man front? What does that prove, exactly?

82
by B (not verified) :: Wed, 11/23/2005 - 5:28pm

Weighted DVOA winners sinc 1998:
2004 - Indy
2003 - NE
2002 - TB
2001 - StL
2000 - Ten
1999 - Stl
1998 - Atl

Looks like the power ranking would be 3-4 picking the superbowl champion.
On the other hand, if you just look at the two teams in the Superbowl, the one with the better DVOA won each year except 1998 and 2001.

83
by Chance (not verified) :: Wed, 11/23/2005 - 5:41pm

Re:#78 But I think it points out a bit of a flaw in trying to analyze “power rankings� in a purely statistical fashion. Stats are nice and all, but wins are what really matters in the end.

If wins are all that matters, why pay attention to power rankings at all? Just go to ESPN, check the standings, order the teams by most wins to least wins, and you're done. Well, except for calling up Vegas and putting your money down on every matchup, picking the team that has the better record. I think you understand that not only would you not get rich doing that, you'd be flat broke in no time. So, perhaps wins aren't everything after all when it comes to predicting future performance.

Power rankings of any kind attempt to quantify all the variables that go into team performances. Just because most other power rankings don't use a computer and consider far fewer variables, doesn't mean the ranker isn't doing the same sorts of things in their head. "Let's see, Team A is 7-3, but they've had a soft schedule, so I'm putting Team B ahead of them even though they're only 6-4."

DVOA isn't perfect, and every year a few more variables are adjusted or taken into account. It's an evolving process. Personally, I'll take the kind of analysis DVOA provides over some ink-stained wretch's regurgitations of Conventional Wisdom any day.

84
by Chris (not verified) :: Wed, 11/23/2005 - 5:58pm

While DVOA is 3-4 in picking SB champions from the start of the playoffs, its also worth noting -

Every single year since 1998, one of the teams ranked in the top 2 of DVOA has made the Superbowl.

It's 5-2 in picking the winner when those two teams are playing ('98 Broncos and 2001 Pats are the 2).

In 2002 and 2004, Both AFC and NFC teams with the highest DVOA ranking in their conference made the Superbowl.

85
by Chris (not verified) :: Wed, 11/23/2005 - 6:06pm

Fixing something.

They wouldn't be 3-4. Indy wasn't ranked as the number 1 team in 2004 - New England was. It would be 4-3 if you went by Total DVOA.

86
by B (not verified) :: Wed, 11/23/2005 - 6:12pm

Chris: the "power rankings" are based on weighted DVOA, Indy was actually #1 in 2004, mostly because of the Miami/NE game.
Anyways, I compared DVOA to a few other ranking methods. Notice that total wins is a lousy predictor of superbowl winners: (If the graph looks crappy, I apologize).

Year SB Record Points DVOA W-DVOA EW
2004 NE Pitt NE NE Indy NE
2003 NE NE KC KC NE NE
2002 TB Phi Phi TB TB TB
2001 NE Stl StL Stl Stl Stl
2000 Bal Ten Oak Ten Ten Ten
1999 Stl Jax Stl Stl Stl Stl
1998 Den Min Min NYJ Atl Atl

87
by Ray (not verified) :: Wed, 11/23/2005 - 6:20pm

Hey, the trolls have finally made it to the thread! They're so cute when they get all mad and attack their straw men. *sigh* What a wonderful place the internet is. ;^)

88
by Andrew (not verified) :: Wed, 11/23/2005 - 6:34pm

Kibbles #79:

I was not saying the system made the running backs. I was saying the style of running in Denver (and Atlanta), is different than most other teams given the way the offensive line plays. To defeat Atlanta and Denver's running game you have to defeat the offensive line system to even get to the runner. With other teams, especially with runners who like the edge or running to a particular part of the line, you just need a good run blitz and gap control.

89
by B (not verified) :: Wed, 11/23/2005 - 6:45pm

I did a little more looking at the numbers about predicting the superbowl.
The Best Record 1-6
Point diff. 2-5
DVOA 3-4
Weighted DVOA 3-4
EW 4-3

See, this is why I think FO's power rankings should be based on Estimated Wins, or maybe Weighted Estimated Wins, to give teams that finish strong a boost.
Incidently, the next time somebody says "Wins are all that matters" ask them how having the best regular season record worked out for Pittsburgh in 04 or Minn in 98. Both teams went 15-1 and didn't even make it to the superbowl.

90
by admin :: Wed, 11/23/2005 - 6:48pm

Re: 72. Whoops, sorry.
Re: 75. The funniest hate mails and troll comments BY FAR are the ones that accuse me of having a bias against the Patriots.
Re: 80/82/86, etc.

Yes, in 2004 Weighted DVOA did not have the Pats number one, solely because of the Miami game, but if you included the playoffs they would be number one going into the Super Bowl.

In 2003, KC was higher in total DVOA because of the first half of the year but it was completely obvious by January that they were falling apart.

In 2000, the Titans and Ravens were virtually tied and the Super Bowl was really played in the second round of the playoffs when those teams met. I have a feeling, also, that doing second-order DVOA for 2000 will move the Ravens into first place.

Finally, one of the things people are missing about DVOA is that ranking total DVOA for each team is really not what gives the system value. What gives the system value is the ability to give a team's performance in various splits like run vs. pass, by down, by area of field, and so forth, to figure out not just which teams are playing better but why, and where teams need to improve and fix weaknesses. I hope that when I give those little split stats in the FOX power rankings, people are finding them interesting and useful.

Re: 89. As you may note in the commens before the ratings, the next thing to be tested will be "Which is more predictive, weighted DVOA or estimated wins." After the turkey.

91
by Andrew (not verified) :: Wed, 11/23/2005 - 6:50pm

putnamp #81:

You are totally overstating the importance of wide receivers.

The leading receivers in the Redskins game for Seattle were the #1 and 2 wideouts Jackson (9 for 106) and Engram (7 for 55, 1 TD). Those guys were only out for the Cowboys game.

In the Cowboys game, Hasselback was 23 of 42 for 224 and 1 TD, with 1 rush and 1 sack. In the Redskins game, he was 26 of 38 for 242 and 1 TD, with 2 rushes and 1 sack.

I fail to see a major significant difference between 41 and 44 called pass plays, and 24 and 28 positive plays on passing downs. Both games had 21 called running plays.

Against Jacksonville with Engram and Jackson, Hasselback was 21 of 37 for 246 and 2 TD, with 2 rushes and 2 sacks. Again, 41 called passing plays and 17 called running plays.

Where's the beef?

Its perfectly clear that Alexander was unable to be utilized in these games and Seattle lost or nearly lost, while he has only been successful against the extraordinaily weak run defenses of the NFC West, Atlanta, and Houston.

It will be quite interesting to revisit Alexander's success once he plays the Giants, Eagles, and Titans, all of whom have good run defenses. He's close to a pace for 2000 yards on the season, but his games against those 3 teams won't help that quest.

92
by admin :: Wed, 11/23/2005 - 7:50pm

OK, I was curious, so I put together 2000 second-order DVOA ratings.

Baltimore doesn't move ahead of Tennessee, but they move closer.

FIRST ORDER
TEN: 36.1% DVOA, 38.7% WEIGHTED
BAL: 32.6% DVOA, 24.3% WEIGHTED

SECOND ORDER
TEN: 37.3% DVOA, 39.8% WEIGHTED
BAL: 35.6% DVOA, 28.4% WEIGHTED

It also moves Baltimore past Tennessee to #1 in weighted defense. The Ravens have the same problem as the 2004 Patriots, one late game where they didn't play well. (The Ravens beat the #31 team in DVOA, Arizona, 13-7 in Week 16.)

When I'm done with these second-order ratings (I still need to do 1998, 1999, 2001) I'll put together a little article talking about how they change the past years.

One more thing.
2000 BAL DEF second order DVOA: -36.7%
2005 CHI DEF second order DVOA through Week 11: -38.9%

Whoa.

93
by Jason (not verified) :: Wed, 11/23/2005 - 8:17pm

Re: #77
Give me a break with that choke away a win business. The Dallas/Seattle game was no less competitive than Washington/Seattle. Only difference was one game went the way you wanted, the other didn't. Dallas was up by 1 touchdown. That's it. They had NEVER put the game outside of winning for the best 2 minute offense in football.
This 'Seattle had no business' winning business is predicated on the idea that Seattle shouldn't be able to win defensive games when they are an offensive team. It was assumed that a low scoring game meant a Dallas win. But it didn't. Time to stop making excuses and give Seattle credit for winning a defensive game. (To #61... Josh Brown has been nearly lights out from 50+ yards, so the distance on his two kicks means little in how 'flukey' the win was.)

And the Washington game was lost on a field goal that went 2 inches to the left well within Josh Brown's range. If the Dallas win is a fluke, than the Washington loss is a fluke. That's all there is to it.

And you seemed to misread what I said about Seattle defense. Number 1, sacks don't mean wins. Never said they did, but high sack numbers mean they are good at getting to the QB.
And I didn't say they had the lowest scoring defense (so your argument about why that's inconsequential goes out the window), but that they are the second best team in the league at stopping TDs IN THE REDZONE. Being a strong team in the red zone means a whole lot for winning, so being the second best redzone defense and third best redzone offense means good things for Seattle.

As for feeling validated for your prediction (you did indeed predict a loss at Arizona), if you did guess that they would lose 2 out of 3 following the bye, going undefeated in that stretch, no matter how close ONE game was (saying St. Louis shot itself in the foot indicates you didn't actually watch that game), I'd feel very little 'validation'.

In regards to #91, the Jaguars loss had less to do with Alexander, and more to do with Matt Hasselbeck's play at the end of the game, because it was quite close until Hass started coughing it up. One of those reasons why it's hard to good analysis a game without watching it.

Seattle stayed in that game (despite uninspired play from Alexander), until Hasselbeck started making a lot of mistakes trying to 'rally'. That's your example of Seattle being good without great play from Shaun.

94
by Anthony (not verified) :: Wed, 11/23/2005 - 8:36pm

Instead of all the machinations to adjust and explain the rankings, shouldn't there just be a disclaimer at the top of the page, "The better team does not always win the game"? And can't we further have a subtitle to that disclaimer, "The team with the better statistics is not always the better team"? It seems to me that this site attempts to present what it believes are the most meaningful statistics when determining how to rank teams by quality. Given that each football season leaves us with inevitably small sample sizes, we are never going to approach perfect information. Even with perfect information, the future is indeterminate and probabilistic. Can't we just acknowledge this, and accept your stats for how great they really are?

I'm more than pleased to see you try to improve the system, but I would propose that we mostly ignore those in the thread who want to attack (or justify) your rankings based on the results of 7 Super Bowls. First of all, that's obviously an incredibly small sample size. Second, even Belichick admitted that he couldn't believe his Pats beat a better Rams squad in the 2001 game. See above. "The better team does not always win the game." I'm not positing an opinion on whether it's more important to win or "to be the better team;" I'm just saying we should acknowledge that those two states of being are not always one and the same.

Further, I'm not sure why it would be such a nightmare to see SD atop your rankings. They lost their first game without their most important red zone pass catcher and then two tough road games (Denver and Philly) in which the difference was a D/ST touchdown. You were saved from this "nightmare" because their defense is not holding up well under the DVOA analysis. Of course, I'm guessing that this has a lot to do with the Jets and Chiefs games in which Godfrey, Foley and Florence were all out. All statistics have to be analyzed in context, and the Chargers' poor defensive statistics certainly have a lot to do with the injuries to the above three as well as the gradual incorporation of the three 1st round draft picks of the past two years, Olshansky, Castillo and Merriman (and the injuries that each of those three has suffered). A mostly healthy defense held up pretty well (even with a huge lead) against the Bills on Sunday, and I would expect the defense to continue to do so as long as it remains healthy.

As to the Colts, I do happen to believe they are a very good team, but they still do not have the statistics to back that up because of a schedule with extremely poor teams. Again, this site contains statistics about past events that have to be taken in context. The Colts have little reason to play their best against the inferior opponents that have littered their schedule. Just because they have won all their games and just because they beat the Bengals does not mean their statistics (even weighted statistics) are going to be better over 10 games than the Bengals or any other team. Let's just acknowledge this and move on. To bring up a basketball analogy, I doubt that an alien who only saw the Pistons beat the Spurs by a collective 46 points in two consecutive NBA Finals games would have given the Spurs a 0.001% chance of winning four out of the other five games played, but that is, of course, exactly what happened.

95
by Kibbles (not verified) :: Thu, 11/24/2005 - 1:17am

I would call the DVOA rating system the “Kordell Stewarts� of rating systems. You come up with all sorts of excuses every week for why certain teams aren’t where they should be. You weight things differently to try to overcome the obvious errors but they still don’t correct them. Lastly you use the fluke rule or as Kordell would say “the best team doesn’t always win� to explain away embarrassing failures. Have you thought for a second that maybe because your dealing with a team of humans playing a game and much depends on the heart and effort they put into a specific game that this more determines the outcome? You can manipulate all the data you want to try to get the Colts in first place, but I can promise you the Colts won’t win the superbowl this year or any other with Manning at QB. You can use whatever stats and formulas you would like but you can’t measure the things that win playoff football games and that is why you look so ridiculous each week trying to predict this year’s SB winner. Answer this, how many correct superbowl champions would your system have picked in the last 10 years?

More than any other system out there, which is a pretty strong statement. DVOA is also better than any objective observer at spotting trends that casual observation misses. For instance, DVOA projected a huge year for San Diego last offseason- at the same time that one of the hometown sportswriters was famously writing that he looked at their schedule and they were the worst team on it. DVOA also projected the "surprise" turnarounds by Carolina and Green Bay last midseason, when all the bobbleheads were collectively writing them off as finished.

As for the bit about Indy not being able to win the SB with Manning... unless you mean that Indy can't win the superbowl unless Manning takes a paycut and the team signs some more quality defenders, I strongly disagree. Do you mean to tell me that you don't think Chicago could win the superbowl with Peyton Manning at the helm? Or that the 2000 Ravens could win the superbowl with Trent Dilfer, but not Peyton Manning? Sure, what the Ravens needed was a caretaker. Someone like Trent Dilfer who wouldn't cost the team the game with any stupid mistakes. Someone who was averaging an INT every 20.5 attempts. Not someone like Manning, who over the past 3 years has averaged an INT every 51 passes. How could a superb defense ever win them a game with a scrub like Manning coughing up the football at such an astronomical rate?

And for the record, Peyton Manning absolutely positively cannot beat New England *IN* New England. Bill Bellichick is inside his head. Tom Brady is clutch. No way can Manning win in New England. Or the playoffs. He lost his first 3 playoff games, which is obviously an indictment of the caliber of QB and not the quality of the surrounding team. No sirree, he'll never be able to win in the playoffs, or New England. Because the team has nothing to do with who wins- no sirree, it's just Manning and Brady out there, man to man.

And even if Manning DID win a superbowl... what would it matter? Everyone knows he couldn't repeat.

And even if he did repeat, he could never do it without Marvin Harrison.

And even if he did it without Marvin Harrison, he still couldn't do it without Tony Dungy.

And even if he did it without Tony Dungy, there's absolutely, positively no way he could pull it off without any arms. So as you can see, the proof is in the pudding, my friend. The proof is in the pudding.

96
by Kibbles (not verified) :: Thu, 11/24/2005 - 1:20am

P.S. Do you even bother reading the articles? Aaron changed the numbers, yes, but he changed them to drastically improve the predictive powers of the numbers. In other words, he changed the numbers, and then checked the numbers against actual results to see if they were better than the original numbers. And they were.

How dare you, Aaron. How dare you improve your numbers. Don't you know that standard operating procedure for sports bobbleheads is to develop your schtick and then mindlessly, blindly stick with it, even when there's irrefutable evidence that your schtick could be easily and efficiently be improved upon. Don't you know how uncomfortable change is for the average sports fan? Do you have any idea how disconcerting it is for a writer to actually admit that his work is... *gasp*... FALLIBLE?

What a hack.

97
by Walt Pohl (not verified) :: Thu, 11/24/2005 - 3:19am

You all are just upset that Tony knows something you don't: the Cigarette Smoking Man has decreed that Peyton Manning will never win a Super Bowl. He lacks heart, effort, _and_ in a key moment this season he's going to be injected with a sedative by secret government agents.

98
by pcs (not verified) :: Thu, 11/24/2005 - 6:07am

Have you thought for a second that maybe because your dealing with a team of humans playing a game and much depends on the heart and effort they put into a specific game that this more determines the outcome?

So "heart" is something a team can just switch on and off, like a coffee machine, depending on the opponent? You, sir, just blew my mind.

Someone help me understand this business about "heart." I hear this same damn tune played every week, and I can never follow it. DVOA is a measure of performance. If a team's "heart" isn't reflected in that team's performance, then "heart" doesn't mean anything. If a guy regularly performs worse than the average player, that great big "heart" of his isn't doing him much good. Unless ... hey, if a team gets only 2 yards on 3rd and 4, can the officials credit them 3 extra yards and a first down for "heart" but leave it off the stat sheet? Is that why DVOA keeps missing "heart"?

Aaron: Next time Victimized Fan of the Week howls at you because his 6-5 team (or, as he calls it, "we") is ranked two spots below a 4-7 team, just explain that the 4-7 guys have more heart. Now yer speakin his language.

99
by Andrew (not verified) :: Thu, 11/24/2005 - 11:22am

Jason #93:

"Only difference was one game went the way you wanted, the other didn’t."

Actually Jason, I rather like Seattle, while I utterly despise the Cowboys. And I don't really care for the Redskins either, being an Eagles fan. Nor do I really like Atlanta. I was almost ecstatic as I watched Dallas choke again for the second time this season at the end of a game. That was beautiful.

When the playoffs come around, I'll be pulling for Seattle and Chicago in the NFC this year.

"Dallas was up by 1 touchdown. That’s it. They had NEVER put the game outside of winning for the best 2 minute offense in football."

The probability of winning when trailing by 7 with 2 minutes left is quite low - under 20%. Most 4th quarter comebacks fail. Also, don't forget to factor in the blind luck for Seattle of Jose Cortez missing a 29 yard field goal off of a botched special teams play by Seattle. That's nothing more than a chip shot that should be made 99% of the time. Seattle's defense didn't make him kick it wide left.

"In regards to #91, the Jaguars loss had less to do with Alexander, and more to do with Matt Hasselbeck’s play at the end of the game, because it was quite close until Hass started coughing it up."

Why was the game close late with such an explosive offense? Because Alexander, who is 2/3 of their offensive scoring, was held in check by Jacksonville. If Alexander and Seattle's running game could actually play well against some besides the bottom of the league, Hasselback wouldn't have been coughing up the ball late, because it would have been in Alexander's hands, unless Matt was taking a knee.

As for the Arizona game, as terrible as they are, Arizona wins around half their games at home. They played both St. Louis and Carolina close in home losses this year. They also had a good opportunity for the same reason to beat Seattle, but couldn't do it. One FO truism is that most of the time, NFC West teams play 150% at home and 50% on the road. Look no further than San Francisco this year, Seattle in years past, or St. Louis most of the time. At no time has San Francisco been competitve on the road this year, but they've played great most games at home, including nearly knocking Seattle off last Sunday.

The really big deal about the last three games for Seattle is that they were finally able to beat their divisional mates on the road, and beat St. Louis twice in a season.

100
by Brian (not verified) :: Thu, 11/24/2005 - 12:13pm

#97
Is that what happened to McNabb?

101
by Andrew (A.B.) (not verified) :: Thu, 11/24/2005 - 2:47pm

Re: 98

Not to defend the trollers around here, but keep in mind that DVOA essentially weights all plays equally. In any football games, some plays are more important than others.

And it's entirely possible for a team to perform better in the important plays than in the less important plays.

(If you don't agree, think about the significance of any fourth quarter play by your backup quarterback when you're leading by 21 points)

102
by Andrew (A.B.) (not verified) :: Thu, 11/24/2005 - 2:48pm

Re: 100

No, in that case, it was the Madden curse.

103
by Another Craig B (not verified) :: Thu, 11/24/2005 - 4:50pm

From my experience the gains you get from 3rd and 4th order adjustments tend to be pretty minimal in value especially once you get more games under your belt. It could be that with these huge schedule disparities that it takes more adjustments for convergence.

Sample size works to one's advantage here. A schedule adjustment after 10 games is made from 10 opponents. A second-order schedule adjustment is made from 100 opponents - a sample that will certainly be made of multiple copies of nearly every team. A third-order adjustment is made from 1000 opponents, who overall are probably going to average as close to .500 as makes no difference.

104
by Another Craig B (not verified) :: Thu, 11/24/2005 - 4:57pm

Aaron, one thing that I am wondering about is the relative precision of the DVOA system. It seems to me that, in particular, the tenths of a percent are perhaps overestimating the precision of the system? Would it be better, and more reflective of the system's predictive power, to present the numbers in 5% chunks?

105
by Androo (not verified) :: Fri, 11/25/2005 - 12:57am

Re: #92
So does the adjustment to the 2000 Ravens' defense jump them ahead of Tampa 2002 now as "Best in FO history?" Tampa's non-adjusted defense wasn't that much different so I'm guessing they don't get as much boost from the new system. (I don't know what Miami 1998 had before adjustment for schedule but they had the seventh hardest.)

As a Ravens fan, hearing that the formula would be tweaked has been the best news all year. I don't mind that they trail Tennesee overall (that was a fun rivalry) but it always bothered me that Tampa Bay had the better defense stat. And it's sad that people (like Bill Simmons) remember that team as a fluke, or forgettable, or, most-often, as the win-in-spite-of-the-QB gold standard; when the team should be remembered for the dominating defense.

Interesting about this year's Bears, unlike those Bucs or Ravens they are leading the league in both pass and rush D.

Now if only we had those 1985 Bears' DVOA....

106
by Androo (not verified) :: Fri, 11/25/2005 - 1:08am

One more interesting thing, while the comparison isn't completely valid due to the different adjustments:

2004 IND OFF DVOA: 38.9%
2005 SD OFF DVOA: 38.5%

I for one will be watching San Diego and Chicago a little closer for the rest of the season.

107
by Jason (not verified) :: Fri, 11/25/2005 - 7:02am

#99. "Actually Jason, I rather like Seattle..."

Than your 'glass is half empty' attitude about the team is all the more mystifying... unless you are one of those 'burned too many times' Seahawk fans that are never willing to accept when Seattle is playing well because you are waiting for the other shoe to drop. I can understand that, I suppose.

"The probability of winning when trailing by 7 with 2 minutes left is quite low - under 20%."
This is granted, but that's taking a look at a the whole league. Seattle is not the average 2 minute offense (still the only team to score points in the final 2 minutes of a half in every single game this year), so the fact that Seattle was able to put up 7 points in under 2 minutes is not such a rare occurence for them as it would be for any other team.
And even if Seattle was a league average 2 minute offense (which we've established they aren't), 20% isn't such a ridiculous percentage as to say 'they have no business winning'. That means, 1 out of 5 times a 7 point lead won't hold up. Seattle was never put out of the game. Dallas didn't do enough to do that, and Seattle's defense stepped it up to make it so.

"Also, don’t forget to factor in the blind luck for Seattle of Jose Cortez missing a 29 yard field goal off of a botched special teams play by Seattle. That’s nothing more than a chip shot that should be made 99% of the time. Seattle’s defense didn’t make him kick it wide left."
This is quite true, but that miss wasn't significantly more lucky than the Seahawks giving the Cowboys the ball within the red zone on that botched special teams play. In all reality, what we have here is two screw ups on special teams play that cancelled eachother out. The only difference is that the Seattle forced Dallas into that special teams mistake with STRONG defense. The very fact that Cortez was even on the field given Dallas' starting field position is an automatic testament to Seattle's defense (particularly in the redzone, where they have been, as mentioned, great this year).
Dallas had to get lucky first before Seattle got lucky, so the slate is wiped clean.

"Why was the game close late with such an explosive offense? Because Alexander, who is 2/3 of their offensive scoring, was held in check by Jacksonville."
Once again, true, but you are only looking at one side of the picture.

Jacksonville is a good team, but even when they contained Alexander, they were still scratching and clawing for that victory until Hasselbeck gave the game to them on a platter. Seattle lost two thirds of their offensive scoring (as you noted) and was still right there with a team that is good. That should be an obvious sign that it's not a 'guaranteed' victory if you can stop Shaun. If Hasselbeck gets back a couple of INTs, that game could just as easily have been Seattle's, despite being on the road against a good team who had stopped Shaun.

"As for the Arizona game, as terrible as they are, Arizona wins around half their games at home."
I understood your reasoning, but Seattle has been playing so far above Arizona this year, that it was still a questionable prediction. (As the game eventually proved.)

I do not think Seattle has a great defense, but I don't think saying they have no defense is accurate at all. Coach Holmgren has called them a 'bend but don't break' defense before, and looking at the numbers, it's true.

Yeah, their secondary is prone to coughing up a big play between the 20s and that needs work, but when they got the second best redzone D in the league, it shows that they can step it up when it counts the most, and that's going to help you win games no matter what team you are.

108
by Greg Lord (not verified) :: Fri, 11/25/2005 - 12:07pm

The Minnesota Vikings have won three games in a row & have won four of their last five games. However in the power rankings the Vikings dropped from 21 to 23.

The Colts have not lost a game, but dropped to #2 behind a two-loss team.

These facts alone are indicative of how bad the FoxSports power ranking formula is flawed.

Greg Lord, Bismarck, ND

109
by Walt Pohl (not verified) :: Fri, 11/25/2005 - 7:10pm

#102 - who do you think put McNabb on the cover of Madden? There are wheels within wheels...

110
by Andrew (not verified) :: Sat, 11/26/2005 - 4:37am

Jason #107:

unless you are one of those ‘burned too many times’ Seahawk fans that are never willing to accept when Seattle is playing well because you are waiting for the other shoe to drop

I'm not a Seahawks fan. I merely admire them from 3000 miles away, as opposed to my loathing of Dallas.

This is granted, but that’s taking a look at a the whole league. Seattle is not the average 2 minute offense (still the only team to score points in the final 2 minutes of a half in every single game this year), so the fact that Seattle was able to put up 7 points in under 2 minutes is not such a rare occurence for them as it would be for any other team.

Just to set the record straight:

Jacksonville - final 1st half TD is on a 4 minute drill, 2nd half 2 minute drill ends in Hasselback INT
Atlanta - 2nd half TD is on 2 minute drill, 2nd half has unsucessful attempt to run out the clock
Arizona - 1st half 2 minute drill ends on downs, 2nd half TD is on a long grind out the clock drive
Washington - 1st half 2 minute drill ends on downs, 2nd half TD is on a long grind out the clock drive
St. Louis - 1st half 2 minute drill ends in FG, 2nd half ends with Seattle running out the clock
Houston - 1st half TD is on a 4 minute drill, 2nd half ends with Seattle running out the clock
Dallas - 1st half 2 minute drill ends in FG, 2nd half TD is on 2 minute drill
Arizona - 1st half TD is on 2 minute drill, 2nd half has unsucessful attempt to run out the clock
St. Louis - 1st half 4 minute drill ends in FG, 2nd half TD is on a long grind out the clock drive
San Franciso - 1st half 2 minute drill ends on downs, 2nd half Seattle does not have the ball to make a drive

That is nine 2 minute drills. Results: INT, TD, Punt, Punt, FG, FG, TD, TD, Punt.

So, down 7 points with 2 minutes or less left against Dallas, the expectation should have been a failure by Seattle to score a TD in at least 2 out of 3 tries, let alone to gain a win in regulation with a TD and a FG off an INT. Hence the term, fluke win. To tie might have been expected occasionally, but not to win.

If Hasselbeck gets back a couple of INTs

Hasselback can't get them back, especially when he is being relied upon to win the game with his arm. In fact, that is when all QB's are most prone to interception.

I understood your reasoning, but Seattle has been playing so far above Arizona this year, that it was still a questionable prediction.

So how to explain Miami beating Denver, or San Francisco beating Tampa Bay, Baltimore beating Pittsburgh, or Cleveland beating Chicago except by the old "home team wins" prediction once again coming true? Would these have been questionable predictions?

How was Arizona so much better last year when essentially the same Seattle team as we have in 2005 lost to them in Phoenix in the 6th game of the season by 8 points?

Yeah, their secondary is prone to coughing up a big play between the 20s and that needs work, but when they got the second best redzone D in the league

Said reputation coming from playing the Little Sisters of the Poor. Jacksonville had 1 redzone TD and 2 FG, Atlanta 2 TD and 1 FG, Washington 2 TD, Dallas 1 TD and 1 FG and 1 missed FG. So in 11 opportunities by real teams, they came up with 6 TD's, 4 FG's, and 1 missed FG. Note 0 redzone INT's by Seattle and 0 turnovers on downs forced by Seattle. The typical outing against Seattle by a real team thus produces 13-17 redzone points, plus other points produced on longer FG's and TD passes, plus any return scores. Seattle has only broken 17 points against such teams offensively 1 of 4 times, while their opponents have done so 3 of 4 times. Normally, such a scoring record will produce a 1-3 record. Seattle is lucky due to the fluke win over Dallas to have gone 2-2 in these games.

111
by Andrew (not verified) :: Sat, 11/26/2005 - 4:50am

Androo #105:

What I'd really like to see is the 1991 Eagles Defensive DVOA. They not only led the league in both conventional NFL rushing and passing yardage defense, but they also are the only NFL unit ever to hold opponents to under 4000 total yards in 16 games, to under 3600 net yards including sacks but not penalties, and one of the few to hold opponents to 850 plays or less in 16 games, and under 200 total first downs.

It would be very interesting to see how they compare to the 2002 Bucs, 2000 Ravens, 1985-6 Bears, and 2005 Bears and other great defensive teams.

112
by Jason (not verified) :: Sat, 11/26/2005 - 7:11am

#110.

"So, down 7 points with 2 minutes or less left against Dallas, the expectation should have been a failure by Seattle to score a TD in at least 2 out of 3 tries, let alone to gain a win in regulation with a TD and a FG off an INT."

I do apologize, I forgot that Seattle didn't score points in the final two minutes in either half against the Niners.

"Hence the term, fluke win."

And the term is wrong. A 'fluke' win suggests that Seattle was facing a team that outclassed them and they could not beat, than got amazingly lucky. Dallas didn't outclass Seattle in this game. It's not a fluke win. Relying on that point to justify why Seattle 'isn't that good' is ridiculous.

You only established that in somewhat (but not entirely) similar situations, Seattle scores a touchdown in 1 out of 3 2 minute situations. That is not a 'fluke' situation. 1 out of 5 (your 'leage average') is not even enough to be considered a fluke, 1 out of 3 is not even close. It's ridiculous to call it a fluke.

Yeah, winning during regulation, ok, that was a surprise, but it was a coin toss at best that Dallas would win in OT (especially with a kicker who can't be counted on).

You keep calling it a fluke, but there is nothing to support that.

"Hasselback can’t get them back..."

Almost entirely outside the point. You were questioning Seattle's ability to stay in games with good teams when Shaun has been shut down. They did it in Jacksonville (until the INTs), they are the ones who gave it away in Washington, they beat Dallas. Your point that Seattle can't be good without Alexander getting a 'good' game is out the window.

"So how to explain Miami beating Denver, or San Francisco beating Tampa Bay, Baltimore beating Pittsburgh, or Cleveland beating Chicago except by the old “home team wins� prediction once again coming true? Would these have been questionable predictions?"

I don't know... how do you explain Denver beating Oakland or Tamba Bay beating Minnesota or Chicago beating New Orleans? Or Seattle beating St. Louis, Arizona and San Francisco? Indy beating the Texans?

Yes, those would have been questionable predictions. When the difference in performance is that large, it's always questionable. Yes, in a game of any two random teams, the home team is always the better choice. But if evaluating the two teams and knowing who they are you still rely on simply the 'homefield advantage', it's questionable.

"How was Arizona so much better last year when essentially the same Seattle team as we have in 2005 lost to them in Phoenix in the 6th game of the season by 8 points?"

It's not essentially the same Seattle team (and it's amazing that you've continued to say that in all these threads). That loss shouldn't have happened, but Seattle was on a downward slide at the time anyway and the establsihed gap in performance between the two wasn't as wide.

"Said reputation coming from playing the Little Sisters of the Poor."

It gets tiring that Seattle's stats vs. these teams don't count. Indiannapolis has played Houston twice, the 49ers, the Titans, Cleveland, the Ravens, and St. Louis. THEIR redzone defense allows a TD nearly 20% more often playing an even easier schedule than Seattle.

Jacksonvile (#2 Defense rank by DVOA) has played Houston, the Jets, St. Louis, Houston, Tennesee, and the Ravens. They also allow redzone TDs nearly 20% more often than Seattle.

The only team with better redzone stopping numbers, the Chicago Bears, #1 in DVOA defense (and nobody questions their defense these days), has played Detroit twice, Cleveland, Baltimore, New Orleans, and San Francisco. Not exactly the world's scariest group there, is it?
So why is that Chicago's stats still matter despite being given the same opportunities to outshine just as many bottom feeders as Seattle, but Seattle's stats agains the 'Little Sisters of the Poor' don't?

The reason why bad teams are bad is because they get beat by good teams. There is no team with a really good record (except Denver) who hasn't gotten that way by stepping on just as many horrible teams as Seattle.

Here is the bottom line on this redzone defense stat: Of all the teams that Seattle played, they average a 50% touchdown percentage in the redzone (including the games against Seattle, which brings this average down, obviously). That includes the Little Sisters of the Poor.
Seattle is allowing touchdowns in the redzone 36% of the time. That's a significant improvement over the rest of the league vs. the same competition.

"Seattle has only broken 17 points against such teams offensively 1 of 4 times, while their opponents have done so 3 of 4 times."

Ah, lovingly misleading statistic. Of course, you pick a random number (17?) to choose, don't count the 17 points against Washington (guess they needed 1 more?).

"Normally, such a scoring record will produce a 1-3 record."

If 1 of the 3 times Seattle doesn't bust 17 is against another team that scored even less and 1 of the 3 times that their opponent breaks 17 is against Seattle during the 1 game that they did by an even larger amount, than a 2-2 record is actually PREDICTABLE.
This stat does not at all indicate that Seattle should be 1-3 against those teams, you just state that it should, but without establishing the situations of the actual games, there is no reason to expect a 2-2 record less than a 1-3 record given this 'scoring situation'.

"Seattle is lucky due to the fluke win over Dallas to have gone 2-2 in these games."

Or they are unlucky due to the fluke loss in Washington to not have gone 3-1 in those games.

113
by Pat Curley (not verified) :: Sun, 11/27/2005 - 3:09pm

Hey, using correlations to test the accuracy of your ratings? I wonder where you got that idea? :)

Seriously, this is why I love your site and your system, because you are constantly working to improve things.

I agree that Indy seems like it should be the highest rated team, but as Bill James pointed out, ratings that never surprise aren't very interesting.

114
by Pat (not verified) :: Mon, 11/28/2005 - 7:10pm

I seriously think it will converge after multiple orders. And it doesn’t surprise me at all that 2nd order is better than 1st order. However, if you look down the chart, the difference between 1st order and 2nd order is a lot smaller in percentage points than the difference between 1st order and 0th order (or just VOA).

James Gibson:

Just a bit of math here: what Aaron is doing when he goes to second order is doing the exact same thing that minimization packages do - iterative convergence. This will usually converge if the initial starting point is close to the final point. Otherwise you could start wildly oscillating and fly off to boundary conditions. In other words - yah, you're probably fine, because Indianapolis has played a few good teams and beat them, too. But if all they had played was Houston, San Francisco, the Jets, the Rams, and New Orleans, you might worry.

Honestly, though, matrix deconvolution (which *always* works) isn't really that hard. It's a really impressive sounding name for "write out all of the equations, look at it, and say oh, yah, I can just solve all of these simultaneously."

115
by James Gibson (not verified) :: Tue, 11/29/2005 - 9:12pm

Pat - yes I know what Aaron's doing is the same thing as a minimization package. And I can also see from his output that he'll likely achieve convergence. It's highly unlikely that he would hit boundary conditions here - there is enough overlap and there are enough plays that the starting point should be "close." Maybe if Indy only played SF, Hou, and Ari would I be worried, but in this case, I think the iterative process will work. It's also easier to understand than matrix math.

116
by Greg Lord (not verified) :: Thu, 12/01/2005 - 5:42pm

Week #13. One power poll has the 9-2 Seahawks as #2, another as #3. The Fox Sports DVOA ratings has the Seahawks as number 11 one behind the 5-6 Redskins.

You're cluching on to your DVOA ratings like an old lady cluching her purse. The DVOA ratings are ineffective and are making Fox Sports look like a joke.

117
by RBELL (not verified) :: Fri, 12/02/2005 - 7:43pm

I agree with #116 Greg Lord response. These ratings are a joke. How can you move the Redskins up a notch from last week with a meltdown against the chargers. How can you move Seattle down to #11 with a 9-2 record and a wins over pretty solid teams like the Cowboys,Falcons, and Giants. 3 missed field goals or not, it was a close competitive game and if you watch the reveiew of Shockey's first half TD, his left foot never touched the ground before he got LEVELED and the ball popped out. It was third down and NY would have been forced to 3 pts. A lot of the so-called sports experts are saying Seattle got lucky, but that game never should have gone to OT if the refs would have had a couple calls correct.
I don't know how things will end up for Seattle, but it sure feels like they have some destiny on their side, which is sometimes what it takes to go all the way
There are about 4 or 5 good(not great) teams in the NFC that have a shot to go all the way and are all equally good, but I'll have to admit, the AFC seems to be the power of the NFL.