Writers of Pro Football Prospectus 2008

Most Recent FO Features

FitzgeraldLar04.jpg

» Scramble for the Ball: The DVOA Schism

Mike and Tom try to figure out what kind of secret sauce Arizona is feeding the media to sit at the top of the power rankings and in the middle of our DVOA rankings.

24 Oct 2006

Week 8 DVOA Ratings

by Aaron Schatz

Here's a look at this week's DVOA and DAVE ratings, with commentary available now on FOXSports.com.

At this point the ratings are starting to look really weird to the average fan. Undefeated Indianapolis and all the 5-1 teams are all beneath teams that are 4-2, 4-3, and even 2-4 (Pittsburgh). With this in mind, this week's commentary really tries to explain DVOA and why it is different than conventional wisdom for each specific team. My hope is that this will help those readers who are a) new to DVOA and b) open-minded.

Some mathy stuff here for the FO readers: In the last couple of weeks, I've noted the gap between the top 20 teams and the bottom 12. A similar gap has opened up between the top 4 teams and the teams ranked 5-20. In that middle section, the teams are really packed close together, which makes a sentence like "New Orleans is only ranked 12th" sound a lot more severe than what the reality is. For a comparison, check out last year's DVOA ratings from this same week. Last year, there were only seven teams between 0% and 15% at this point in the season. This year, there are 15, or nearly half the league.

On the other hand, eight teams from last year's Week 8 would fit into the space between the Giants (fourth right now) and the Ravens (fifth).

One more tidbit: Last week, I said I would go looking to see if the Chicago-Arizona game had the biggest difference in single-game DVOA between two teams where the team with the lower rating actually won. I never got to this on the blog over at FOX, so I wanted to mention it here. It turns out the answer is no. In fact, there are two games in the last two years where the difference was even greater and the team that played better overall lost: Pittsburgh's 33-30 victory over the Giants in Week 15 of 2004, and Philadelphia's 17-16 victory over the Rams in Week 15 of 2005.

Remember that you can always use the keyword "DVOA" to access the latest DVOA commentary at FOXSports.com.

* * * * *

These are the Football Outsiders team efficiency ratings through Week 7 of 2006, measured by our proprietary Defense-adjusted Value Over Average (DVOA) system that breaks down every single play and compares a team's performance to the league average based on situation in order to determine value over average. (Explained further here.)

OFFENSE and DEFENSE DVOA are adjusted based on strength of opponent as well as to consider all fumbles, kept or lost, as equal value. Opponent adjustments are currently set at 70% and will increase each week until they are full strength after Week 10. SPECIAL TEAMS DVOA is adjusted for type of stadium (warm, cold, dome, Denver/Mexico City) and week of season.

DAVE is the new early-season formula that combines early-season performance with our preseason projection to get a more accurate picture of how well teams will play over the course of the entire season. (DAVE stands for "DVOA Adjusted for Variation Early.") This is the rating used to rank teams at FOXSports.com. At this point, the preseason projection is just 15% of the rating, and is not used at all for the four teams on bye in Week 8. Next week, we get rid of DAVE and start using WEIGHTED DVOA, which is based on 2006 only, but with early-season games given less value.

To save people some time, please use the zlionsfan template for all complaints:

<team> is clearly ranked <too high/too low> because <reason unrelated to DVOA>. <subjective ranking system> is way better than this. <unrelated team-supporting or -denigrating comment, preferably with poor spelling and/or chat-acceptable spelling>


TEAM
TOTAL
DVOA
LAST
WEEK
DAVE RANK NON-ADJ
TOT VOA
W-L OFFENSE
DVOA
OFF.
RANK
DEFENSE
DVOA
DEF.
RANK
SPECIAL
DVOA
S.T.
RANK
1 CHI 45.0% 2 39.4% 1 55.2% 6-0 5.4% 10 -26.8% 2 12.7% 1
2 PHI 39.2% 3 35.2% 2 33.6% 4-3 25.0% 2 -15.1% 8 -0.8% 18
3 SD 34.0% 1 31.9% 3 44.1% 4-2 15.5% 4 -13.6% 9 4.9% 4
4 NYG 33.3% 5 29.2% 4 27.8% 4-2 21.3% 3 -10.8% 11 1.1% 12
5 BAL 18.2% 7 14.9% 7 30.8% 4-2 -17.2% 27 -32.0% 1 3.4% 7
6 PIT 17.3% 8 17.7% 5 8.8% 2-4 8.5% 8 -15.4% 6 -6.5% 31
7 NE 14.8% 11 14.2% 8 27.6% 5-1 11.7% 6 1.3% 22 4.3% 5
8 JAC 14.1% 4 12.4% 9 17.9% 3-3 -3.7% 19 -19.1% 4 -1.2% 19
9 IND 13.8% 13 15.7% 6 18.7% 6-0 27.3% 1 9.4% 26 -4.2% 30
10 STL 11.8% 9 6.5% 14 26.3% 4-2 13.0% 5 1.0% 21 -0.2% 15
11 DAL 11.8% 6 9.9% 12 11.6% 3-3 -1.9% 16 -15.5% 5 -1.8% 23
12 NO 11.3% 10 5.7% 15 16.2% 5-1 7.0% 9 0.9% 19 5.2% 3
13 KC 10.1% 17 10.0% 11 3.6% 3-3 -2.8% 18 -9.4% 13 3.5% 6
14 MIN 9.6% 19 5.5% 16 11.7% 4-2 -10.8% 24 -20.6% 3 -0.1% 14
15 DEN 9.3% 12 11.1% 10 12.0% 5-1 -4.6% 20 -15.3% 7 -1.3% 20
16 CIN 7.7% 15 9.1% 13 8.9% 4-2 0.4% 13 -5.1% 14 2.3% 9
TEAM
TOTAL
DVOA
LAST
WEEK
DAVE RANK NON-ADJ
TOT VOA
W-L OFFENSE
DVOA
OFF.
RANK
DEFENSE
DVOA
DEF.
RANK
SPECIAL
DVOA
S.T.
RANK
17 CAR 4.0% 14 5.4% 17 2.3% 4-3 3.3% 11 -2.5% 17 -1.8% 24
18 ATL 0.1% 20 1.7% 18 4.3% 4-2 -6.7% 22 -9.6% 12 -2.9% 27
19 WAS -3.6% 18 -3.8% 20 -5.4% 2-5 11.1% 7 16.8% 28 2.1% 10
20 SEA -5.1% 16 0.2% 19 -14.5% 4-2 -7.5% 23 -2.8% 16 -0.4% 17
21 TB -15.3% 23 -13.6% 21 -26.3% 2-4 -12.8% 25 1.0% 20 -1.6% 22
22 GB -15.7% 26 -15.2% 22 -20.7% 2-4 -6.3% 21 5.6% 24 -3.8% 29
23 BUF -16.8% 21 -17.3% 23 -20.5% 2-5 -14.9% 26 4.3% 23 2.4% 8
24 NYJ -17.4% 27 -19.1% 25 -12.0% 4-3 0.4% 12 18.2% 29 0.3% 13
25 DET -19.3% 22 -19.3% 26 -19.3% 1-6 -1.3% 14 15.4% 27 -2.6% 25
26 MIA -20.1% 24 -20.1% 27 -9.3% 1-6 -20.5% 28 -3.0% 15 -2.7% 26
27 CLE -20.2% 28 -18.2% 24 -25.3% 1-5 -21.4% 30 7.2% 25 8.5% 2
28 ARI -24.3% 25 -23.6% 28 -16.9% 1-6 -26.6% 31 -12.7% 10 -10.4% 32
29 SF -25.9% 29 -26.4% 29 -31.3% 2-4 -2.7% 17 22.9% 31 -0.3% 16
30 HOU -29.2% 31 -27.6% 30 -30.0% 2-4 -1.3% 15 29.7% 32 1.8% 11
31 OAK -39.1% 32 -34.4% 31 -43.1% 1-5 -35.2% 32 0.8% 18 -3.2% 28
32 TEN -41.0% 30 -37.1% 32 -35.8% 1-5 -21.0% 29 18.5% 30 -1.5% 21

  • ESTIMATED WINS uses a statistic known as "Forest Index" that emphasizes consistency as well as DVOA in the most important specific situations: red zone defense, first quarter offense, and performance in the second half when the score is close.  It then projects a number of wins adjusted to a league-average schedule and a league-average rate of recovering fumbles.  Teams that have had their bye week are projected as if they had played one game per week.
  • PAST SCHEDULE lists average DVOA of opponents played this season, ranked from hardest schedule (#1, most positive) to easiest schedule (#32, most negative). It is not adjusted for which games are home or road.
  • FUTURE SCHEDULE lists average DVOA of opponents still left to play this season, ranked from hardest schedule (#1, most positive) to easiest schedule (#32, most negative). It is not adjusted for which games are home or road.
  • VARIANCE measures the statistical variance of the team's weekly DVOA performance.  Teams are ranked from least consistent (#1, highest variance) to most consistent (#32, smallest variance).


TEAM TOTAL
DVOA
W-L ESTIM.
WINS
RANK PAST
SCHED
RANK FUTURE
SCHED
RANK VARIANCE RANK
1 CHI 45.0% 6-0 5.7 3 -11.9% 31 -4.4% 25 27.7% 5
2 PHI 39.2% 4-3 6.1 1 -5.0% 25 2.9% 13 2.4% 32
3 SD 34.0% 4-2 4.9 4 -8.6% 28 -6.4% 28 9.8% 17
4 NYG 33.3% 4-2 5.7 2 9.4% 2 3.6% 12 8.5% 19
5 BAL 18.2% 4-2 4.4 8 -4.5% 24 -0.6% 19 12.7% 15
6 PIT 17.3% 2-4 4.1 12 7.6% 6 -2.6% 21 16.8% 10
7 NE 14.8% 5-1 4.6 6 -9.0% 29 -6.0% 27 6.3% 27
8 JAC 14.1% 3-3 3.7 15 2.4% 12 -3.7% 24 44.8% 1
9 IND 13.8% 6-0 4.7 5 -6.3% 26 -1.1% 20 6.8% 24
10 STL 11.8% 4-2 4.3 9 -11.6% 30 0.5% 17 10.2% 16
11 DAL 11.8% 3-3 3.6 17 1.8% 17 4.4% 9 15.9% 11
12 NO 11.3% 5-1 4.1 11 -1.1% 22 5.3% 7 8.0% 21
13 KC 10.1% 3-3 3.9 13 3.0% 10 -3.6% 23 40.8% 2
14 MIN 9.6% 4-2 4.5 7 0.7% 19 -6.7% 29 8.1% 20
15 DEN 9.3% 5-1 4.2 10 -0.6% 21 2.5% 14 9.1% 18
16 CIN 7.7% 4-2 3.8 14 1.5% 18 7.0% 5 7.1% 23
TEAM TOTAL
DVOA
W-L ESTIM.
WINS
RANK PAST
SCHED
RANK FUTURE
SCHED
RANK VARIANCE RANK
17 CAR 4.0% 4-3 3.6 16 1.9% 16 10.6% 2 6.6% 25
18 ATL 0.1% 4-2 3.3 19 3.8% 9 3.8% 11 36.1% 3
19 WAS -3.6% 2-5 3.4 18 2.0% 13 13.5% 1 14.9% 13
20 SEA -5.1% 4-2 3.1 20 8.0% 5 -9.0% 31 7.7% 22
21 TB -15.3% 2-4 2.7 22 9.4% 3 9.4% 3 6.5% 26
22 GB -15.7% 2-4 2.8 21 11.3% 1 -3.0% 22 4.8% 28
23 BUF -16.8% 2-5 2.6 25 3.9% 8 -4.8% 26 19.2% 9
24 NYJ -17.4% 4-3 2.5 26 -7.8% 27 -8.0% 30 21.9% 8
25 DET -19.3% 1-6 2.2 27 1.9% 15 -0.5% 18 3.8% 30
26 MIA -20.1% 1-6 2.0 29 -14.7% 32 5.4% 6 4.8% 29
27 CLE -20.2% 1-5 1.9 30 1.9% 14 4.3% 10 3.1% 31
28 ARI -24.3% 1-6 2.7 23 -0.5% 20 1.1% 16 23.1% 6
29 SF -25.9% 2-4 2.7 24 5.3% 7 1.8% 15 31.6% 4
30 HOU -29.2% 2-4 2.1 28 9.2% 4 -10.0% 32 15.4% 12
31 OAK -39.1% 1-5 1.2 32 -1.5% 23 4.9% 8 14.0% 14
32 TEN -41.0% 1-5 1.5 31 2.6% 11 8.0% 4 22.0% 7

Posted by: Aaron Schatz on 24 Oct 2006

268 comments, Last at 05 Nov 2006, 3:01pm by Jean Crawford

Comments

1
by Jason (not verified) :: Tue, 10/24/2006 - 4:37pm

Aarrgghhh! Why Eagles, why do you torture me so with your 3 boneheaded losses when you should easily have your 6.1 victories?!!

2
by Bobman (not verified) :: Tue, 10/24/2006 - 4:37pm

I see Indy's ST performance is returning to its customary dungeon after looking competent in the first two weeks.

Oh for the days of Brad Pyatt returning kicks and, um, invisible trip wires in the turf tackling the opposing returners. So much for the theory that all their young blood on D makes for good tackling on ST.

3
by Pat (not verified) :: Tue, 10/24/2006 - 4:46pm

And Philly's still the most consistent team in the league. Un-freaking-believable. I cannot, for the life of me, understand how that team's not blowing teams out of the water.

4
by Xian (not verified) :: Tue, 10/24/2006 - 4:53pm

I see that Chicago has also joined SD in the ranks of top-10 in everything. Interesting.

I, for one, welcome our new bear-shaped overlords.

5
by princeton73 (not verified) :: Tue, 10/24/2006 - 4:53pm

Why Eagles, why do you torture me so with your 3 boneheaded losses when you should easily have your 6.1 victories?!!

that missing 0.1 is gonna come back and bite them at the end of the season--guaranteed

6
by mediator12 (not verified) :: Tue, 10/24/2006 - 4:53pm

Indy's ST's have been severely hurt by Defensive injuries to the DB's and LB's. Plus, Vinatieri has not played that much either.

Still interesting to see how low Denver is to the supposedly stronger defenses in the league. Seventh in DVOA and twenty two of 44 points surrendered from TO's in scoring position.

Great to see how much Jake Plummer makes this defense look worse than they are. Hey let's start him against the Colts Mike Shanahan. He should help boost Peyton's stats some more this week.

7
by The Leon Express (not verified) :: Tue, 10/24/2006 - 4:54pm

Mark my words:

The Jets will make the playoffs with a DVOA in the 20s!!!!!!!!!! (Oh, how I love the NFL scheduling gods!!))))

8
by Becephalus (not verified) :: Tue, 10/24/2006 - 4:54pm

-3

Its simple, teams are playing well/getting lucky against them. Philly is consistent playing at level say 70 out of 100. Many other teams fluctuate +/- 20 with a mean of say 57. Thus when those other teams have an upward variation they beat PHI 77 to 70 in my mythical "play index".

9
by Independent George (not verified) :: Tue, 10/24/2006 - 4:55pm

Pittsburgh's special teams is only 31st? Arizona's must be really some kind of awful to be worse.

10
by Independent George (not verified) :: Tue, 10/24/2006 - 4:56pm

Pittsburgh's special teams is only 31st? Arizona's must be really some kind of awful to be worse.

11
by VarlosZ (not verified) :: Tue, 10/24/2006 - 4:56pm

I can feel it. This is the week when the Angry Troll Hatred comes to visit us. . .

12
by tunesmith (not verified) :: Tue, 10/24/2006 - 5:02pm

Yeah, I don't get the Broncos, either. At this point it looks like it's just a matter of time before Plummer costs them what should otherwise be a victory. Afterwards they'll be justified to switch. It seems like Indy would be a great opportunity for Cutler - Indy's defense isn't fabulous, and their run defense sucks, so the running game will let him get his feet wet.

13
by Jerry (not verified) :: Tue, 10/24/2006 - 5:02pm

Freaky...last year CAR was 17 and ATL was 18 on week 8. This year; CAR 17, ATL 18.

"At this point the ratings are starting to look really weird to the average fan. "
You think! I understand the ratings but even so, I would hate to have to explain them to a couple hundred thousand Fox readers. Good luck Aaron!

14
by Ilanin (not verified) :: Tue, 10/24/2006 - 5:04pm

I'm beginning to wonder if DVOA should start penalising turnovers differently depending on field position (if it already does, penalising turnovers *more* differently...). Pittsburgh's anomalously high placing is a result of outplaying their opponents in two games they lost (Bengals, Falcons) because they turned over the ball in positions where the opposition really couldn't help but score. Ryan called them "potentially the best 6-10 team ever" which currently wouldn't suprise me if this keeps up. Admittedly, the Steelers have also had pretty bad luck in terms of fumble recovery (note to Steelers: this would matter less if you would QUIT FUMBLING), and the playcalling hasn't been the greatest on occasions (Jacksonville, hello). Ultimately, I think there has to be something wrong with the system here. As a Steelers fan, I know we're better than this, and I'm glad that VOA agrees with me, but they didn't actually deserve to win those games, and I think VOA should reflect that.

15
by Tom S (not verified) :: Tue, 10/24/2006 - 5:12pm

Wow, Pitt at 2-4? That's phenomenal. how much are the Squealer fans paying you Aaron? Are you listening Atlanta Falcon fans? That right there is your problem! :-D I think that the Squealers are playing sooo well off the charts and are so far out to the right that they've gone LEFT!

LMAO! Well, I guess if they make the playoffs again, we will sure know why!

16
by Sophandros (not verified) :: Tue, 10/24/2006 - 5:13pm

11: I can feel it. This is the week when the Angry Troll Hatred comes to visit us. . .

Yep. Pittsburgh is at 6 and Atlanta is at 18...

I'm curious to see how far Seattle's going to plummet in the coming weeks.

17
by Tom S (not verified) :: Tue, 10/24/2006 - 5:15pm

Oh,BTW, Aaron, just once I'd like to see you just post the DVOA on FOXSports WITHOUT explanation and then post the email retorts that you get on here!

Now THAT would be entertaining!

Great Work! Just don't check your email for the rest of the week! LOL!

18
by Kal (not verified) :: Tue, 10/24/2006 - 5:18pm

#16: Seattle may not plummet; they've gone through the toughest part of their schedule by a longshot, and go into a very easy schedule coming up - KC, Oakland, and SF in 3 of their next 4 games. STL will likely be a loss, but KC and Oakland are very winnable, even with Seneca Wallace.

Pitt at 2-4 ranked above, well, everyone is baffling, but the one that really baffles is NO. What does NO have to do to improve in the eyes of DVOA? I know, the defense needs to improve.

I do like the schedule strength. It'll be interesting to see GB do great and hear all the Favre-ists all go crazy for a bit. It'll also be interesting to see Carolina kind of tank it. And Washington...I'm wondering if they'll win another game.

19
by cd6 (not verified) :: Tue, 10/24/2006 - 5:18pm

Steelers special teams are clearly ranked too high because they are pathetically horrible at fielding punts. Paying somebody to stand there and fair catch every single time is way better than this. I will destroy you, Santonio Holmes. OMFG WTF

20
by BillWallace (not verified) :: Tue, 10/24/2006 - 5:19pm

Wow great rankings this week, I wish we could get footage of the exploding heads of the Foxsports fans from Denver, Cincy, Atl, Carolina, etc....

1) I know there have been crazy 'explanation needed' phenomena in the rankings in the past, but this is the craziest I can remember.

2) The Broncos and Falcons again... I still say there's something inherent in the running games of those two teams that lead to this.

21
by MJK (not verified) :: Tue, 10/24/2006 - 5:19pm

Hi,

I live under a bridge in Denver and turn to stone by the light of day. I just wanted to let you know that me and a bunch of my friends and relatives will be visiting you all real soon. See you then!

22
by Rick "32_Footsteps" Healey (not verified) :: Tue, 10/24/2006 - 5:21pm

Well, for da Iggles, I imagine that some of it just comes down to absurd bad luck. I mean, how many opponents have missed field goals against them? (No wonder Gruden tried the 63-yarder - it's practically guaranteed against Philly this season.) And the forward fumble haunts me to this day.

I'm beginning to wonder if there should be a "luck count" to show how many times a team has just been plain fortunate or unfortunate. Like something that accounts for a team that seems to never/always recover fumbles, and things like that.

A theoretical example would take a team where they've recovered 18 of 29 fumbles, have made 1 of 4 45 yard+ (to pick an arbitrary distance - you'd probably want to pick the yardage beyond which FGs are 50% or less likely to make) FG, and have had 3 for 5 such FGs made against them.

Assuming that fumbles should be a 50/50 proposition, our team is 3.5 fumbles ahead of the game, one field goal behind projection, and have given up .5 field goals more than projection, resulting in a luck count of +2.

I'd be interested in seeing how teams would rank on such a metric - and more importantly, whether this would explain at least some of the disparity between projected and actual records.

23
by MJK (not verified) :: Tue, 10/24/2006 - 5:22pm

My friends from Atlanta will probably drop by, too...

24
by Luz (not verified) :: Tue, 10/24/2006 - 5:22pm

i'll third independent george's post (9/10). i can't even imagine what arizona must be doing to actually be worse than the steelers ST. seriously, if someone knows please tell me. i haven't heard about the cards allowing multiple return TDs a game...

25
by MJK (not verified) :: Tue, 10/24/2006 - 5:24pm

Isn't comparing actual wins to estimated wins a pretty good quantitative indicator of luck? Or actual wins to wins predicted by DVOA?

26
by Luz (not verified) :: Tue, 10/24/2006 - 5:25pm

MJK (21) wins the the funniest post of the day. by a mile.

27
by zach (not verified) :: Tue, 10/24/2006 - 5:28pm

Teams are getting lucky vs. the Eagles, plain and simple. Two of those losses are a fumble recovery in the endzone and a friggin 62 yard field goal.

This is why I'm not worrying about those three losses too much yet... but playing Jacksonville next week will be no fun.

28
by cd6 (not verified) :: Tue, 10/24/2006 - 5:29pm

"Pittsburgh’s 33-30 victory over the Giants in Week 15 of 2004"

Oh man I remember that game... that was Willie Parker's coming out party. James Harrison had a TD! Brian St. Pierre threw passes!!

The Bills wanted that game so bad to make the playoffs and the Steelers scrubs totally embarrassed them.

29
by Kal (not verified) :: Tue, 10/24/2006 - 5:29pm

If the Eagles and Pats meet in the superbowl, I think Belichick will just go for the free kick every time the Eagles kick off to him. Why not? 70, 80 yarders? Bah. They are no match for the power of the Eagles Curse.

30
by Rich Conley (not verified) :: Tue, 10/24/2006 - 5:29pm

I'm really starting to think that DVOA loving teams that play dink and dunk ball is causing problems.

Also, how are the Giants so high? How would their DVOA change if you took out the 27 junk points they scored in the Seattle game?

31
by Bobman (not verified) :: Tue, 10/24/2006 - 5:30pm

Mediator12,

I forgot all about their strategy of having Corey Simon as a gunner on kicks. After all, he'd cover half the field.

Hopefully everyone bust a gut laughing at the image--mainly the returners would stumble in the craters his footsteps left in the ground. I suppose you are right about injuries, plus they either lost or moved a couple ST stalwarts to the starting lineup (Mathis), and lost their top ST players from last year. And by "top" players, I mean, well, guys with a pulse.

You know how the Gruden hire a few years back signalled the era of paying sky high prices for coaching talent because it was uncapped? Supposedly. Well, is there an ace ST coach out there that Indy could hire? I know Irsay isn't made of money, but maybe freakin' Manning could contribute some cash toward his salary? Is $1M a year too much to expect for the absoute top, walk-on-water, ST coach? Sounds cheap to me. Between NFL and endorsements last year, Manning made about $21M. Hell, I'll kick in 5% of my income as gravy.

And if this ROBO-ST coach moves them up to about 12 in ST, it will have been worth it.

32
by Anthony (not verified) :: Tue, 10/24/2006 - 5:32pm

This week the most consistant team(PHI) plays the most inconsistant(JAX). Thats pretty scary for me as an eagles fan, you can pretty much be sure the good Jacksonville team will show up with the Eagles' luck.

33
by Disco Stu (not verified) :: Tue, 10/24/2006 - 5:32pm

Doesn't seem to be anything wrong with the Denver ranking. They played a bunch of crappy teams (and New England) and didn't win by that much. So objectively they're only a little bit better the average (crappy) team they played. Subjectively we can say that Denver is "only doing enough to win", but there's no quantifiable way to represent that.

9-6 over KC? 17-7 over Clev? 13-3 over Oak? 13-3 over Bal? Lose 10-18 vs. StL? 17-7 over NE? A middle of the pack ranking objectively seems about right.

34
by D (not verified) :: Tue, 10/24/2006 - 5:34pm

Jeremy Green I believe it was over at ESPN said something interesting about the way the Eagles are playing. He said they say seem to think they are great team who can win just by showing. The thing is, as he pointed out, the Eagles aren't a great team. They are good, but not as good as they seem to think they are. It sort of reminded me of what PFP '06 said about the Eagles playing at the beginning of last season; they were like the lazy smart kid who didn't study for the test but assummed he would pass anyways. The fact that it has happened two years in row tells me that Andy Reid should try and change the attitude in his locker room.

35
by Pat (not verified) :: Tue, 10/24/2006 - 5:36pm

I’m beginning to wonder if DVOA should start penalising turnovers differently depending on field position (if it already does, penalising turnovers *more* differently…).

It does. And I really challenge you to find a better way of doing it than is already done here.

but they didn’t actually deserve to win those games, and I think VOA should reflect that.

That's not DVOA's job. That's estimated win's job. Where Pittsburgh is 12th.

36
by JasonK (not verified) :: Tue, 10/24/2006 - 5:37pm

#30:

The Giants are high mostly because of their running game, their run defense, and the opponent adjustment (note the #2 ranking in "past schedule").

37
by ToxikFetus (not verified) :: Tue, 10/24/2006 - 5:41pm

34:

I remember those days well. That cocky smart kid would show up to the test and all of a sudden he'd get sports hernias, torn hamstrings, broken feet, and the class clown would walk up and stab him in the back. If only he'd studied...

38
by Israel (not verified) :: Tue, 10/24/2006 - 5:42pm

28 - Aren't you mixing the Bills and the Giants?

39
by D (not verified) :: Tue, 10/24/2006 - 5:42pm

30-
I'm not sure I see your point. Of the top 10 offenses according to DVOA the Eagles are the only team that uses a Walsh-style West Coast Offense. Maybe I'm misinterpretting what your saying, or missing something in the rankings, but I'm just not seeing the problem you're refering to.
(Sorry about the double post).

40
by Pat (not verified) :: Tue, 10/24/2006 - 5:44pm

Well, is there an ace ST coach out there that Indy could hire?

Yup. Bobby April, Buffalo Bills. Or John Harbaugh, Eagles, but you want April. Really, you want April.

41
by Kal (not verified) :: Tue, 10/24/2006 - 5:47pm

#35: I'd be curious to see a similar study done from 2005's turnovers instead of 2002. Not that it should be different, but I wonder if it is.

42
by Chris M (not verified) :: Tue, 10/24/2006 - 5:48pm

30 - explain that comment about dink and dunk.

I'll be forwarding this week's rankings to my NFC hating friends.

43
by mediator12 (not verified) :: Tue, 10/24/2006 - 5:53pm

Hey, even I do not question DEN's overall rankings, just the defenses.

Den is right where they should be overall, if that. I could see them statistically being even worse than they are, due to a horrendous passing game and a QB that is incapable of getting the offense out of bad plays pre-snap. Plummer has been even worse than his numbers, because the numbers can never relate opportunity cost and he has blown way more plays than he has made.

DEN's defense has also had a bunch of missed TO opportunities, more than seven Dropped INT's at last count, but has been extraordinary in surrendering points. Last time I checked, points determined the outcome of games not first downs and yards.

DEN's offense has been way worse than the Stats show since they have given back at least 21 points of the 70 they have scored too.

They have played a poor schedule for sure and that has easily boosted their rankings as well as their wins. We will find out if the defense is for real this weekend. If they can play as consistently as TEN's did in INDY two weeks ago they can and should win the game versus the Colts.

44
by Rich Conley (not verified) :: Tue, 10/24/2006 - 6:34pm

Is the fox thing up yet? Link doesnt seem to be there.

And 2, your webserver sucks. C'mon here, this is awful. Everytime theres a new article, the site shuts down. Word Press just can't handle it.

45
by Ilanin (not verified) :: Tue, 10/24/2006 - 6:35pm

That’s not DVOA’s job. That’s estimated win’s job. Where Pittsburgh is 12th.

Which is all very well, but even estimated wins has the Steelers at 4-2 instead of 2-4. I suppose "average" fumble recovery luck might well have got them to 3-3 , but 4.1 seems like at least one too many wins.

46
by admin :: Tue, 10/24/2006 - 6:37pm

As I wrote in the FOX blog two weeks ago, Denver's current level of red zone defense is simply not sustainable. While it is no longer past -100%, it is still better than the 2000 Ravens at this point, and the gap between that and their total defense is still the largest ever. (Click link on name.)

We're working on the server. We really thought we had things fixed and we're not sure what's going on. We're on a dedicated server now, so there shouldn't be a problem.

47
by tunesmith (not verified) :: Tue, 10/24/2006 - 6:43pm

Beatpaths power rankings are up. I have Minnesota ranked #2! DEN #6... NO #8... NE #10... ATL #12.

Last week's "picks" (loose term; done by comparing placement in the power rankings) were 7-6, which were surprisingly good, tied for the best weekly record in King Kaufman's panel o' experts.

48
by tunesmith (not verified) :: Tue, 10/24/2006 - 6:43pm

Denver hasn't played a particularly weak schedule - it's more that everyone wants to pretend the win over New England didn't happen. "Oh, Denver just owns New England, it doesn't, like, mean they are BETTER than them or anything."

There's no reason to think that win was an asterisk.

"Oh, but I mean BESIDES the win over New England... yeah, yeah, BESIDES that..."

49
by Rich Conley (not verified) :: Tue, 10/24/2006 - 6:57pm

48. The thing is, its tough to not call it an asterisk.

There are some spots where teams just dont match up well. Denver ALWAYS beats NE. Just like NE always seems to beat Indy, and Indy always seems to beat Denver.

yes, Denver beating NE may say that Denver is a very good team, but it also may just be another occurance of Denver beating NE.

50
by emcee fleshy (atl) (not verified) :: Tue, 10/24/2006 - 6:58pm

Don't expect too many ATL trolls. We are all-too-aware that this team is mediocre.

After the Giants debacle two weeks ago, most of us were just happy the Falcons showed up to the stadium this week. Sure was nice of Pittsburgh to fumble fourteen times. (Lost 12)

51
by Will Allen (not verified) :: Tue, 10/24/2006 - 6:58pm

Pat, I think luck does play the largest role in Philly not blowing people out, but I still think there is a possibility that something is being missed in regards to interceptions that occur in the middle of the field, like how Barber schooled Mcnabb on Sunday. Maybe td returns for touchdowns really are random, but it might be interesting to see an analysis of ints between the 35 yard lines, broken down between inside and outside the tackle box, within five yards of the sideline, and more or less than 20 yards from the line of scrimmage.

If I remember correctly, the Vikings' special teams have made dramatic leaps in the past few weeks, along with a gradual climb by the defense. I think the defense is definitely for real, and if the special teams maintain their performance, they will be able to tolerate their current offensive performance pretty well, and given their remaining schedule, 10 wins, or if they get a little luck, 11 wins are a possibility.

52
by admin :: Tue, 10/24/2006 - 7:03pm

I would have to say that at this point, when somebody says "DVOA would be better if it could include X" or "DVOA is wrong about this team because of X," 75% of the time X is something that I've already tried to put in the formula that didn't actually improve it and in many cases made it less accurate, and 20% of the time X is something that we simply can't measure because it's not in the standard play-by-play and the charting project doesn't turn data around fast enough. It's been four years now; I've tried a LOT of different improvements.

On the other hand, Will, you're the person who's been harping on the Vikings, right? I'm coming around. They've played much better in the last two weeks. I just can't figure out why, for the second straight year, the defense does all the work and Brad Johnson gets all the praise.

53
by tunesmith (not verified) :: Tue, 10/24/2006 - 7:07pm

#49 - the problem is, it's impossible to come up with a set of standards to reliably state when it's an asterisk or not. That judgment is totally subjective. Based off of the game outcomes this season only, there is absolutely no reason to believe that New England is better than Denver, as they haven't beaten any teams that have beaten Denver. And there's every reason to believe Denver is better than New England, since Denver beat them head to head.

Besides, Indianapolis beat New England last time they met.

54
by Rich Conley (not verified) :: Tue, 10/24/2006 - 7:18pm

53

Right its somewhat subjective, but so is DVOA.

The thing is, I'm a NE fan, and I had money on Denver that night, even though VOA/DAVE had them as a road-dog.

55
by Earl (not verified) :: Tue, 10/24/2006 - 7:20pm

#33:

Denver beat Baltimore (#5), New England (#7) and Kansas City (#13). St. Louis, the only team they actually lost to, is #10. Yes, wins against Oakland and Cleveland were unimpressive, although interestingly Oakland's defensive rank is actually up at 18, which is middle of the pack.

At what point on that list is a team no longer considered crappy?

Not to say that I'm happy with Denver's offensive performance, and I actually believe they're rated accordingly; but I don't buy it when people say "they beat a bunch of crappy teams + New England".

56
by Kal (not verified) :: Tue, 10/24/2006 - 7:25pm

I think it's more the perception of how Denver won than whether they won + the perception that STL isn't that great. I don't quite understand why; their defense has been strong against all sorts of teams. Their offense has been strong enough. I guess they've not looked like a ridiculously dominant team in any of their games, so they don't stand out.

57
by Jerry (not verified) :: Tue, 10/24/2006 - 7:26pm

It will be interesting to see what Houston does throughout the rest of the year with their upcoming schedule (#32 ranked) and considering a 2 -4 record with their past schedule (#4 ranked).

58
by Just Another Falcon Fan (not verified) :: Tue, 10/24/2006 - 7:33pm

Yes, I know this Atlanta fan is waiting for the Falcons to actually stomp someone other than the Cardinals.

59
by Rick (not verified) :: Tue, 10/24/2006 - 7:37pm

Well, last week I was critical of these DVOA rankings because you had 2-3 Pittsburgh ranked 8th based apparently on an overestimate of their defense's quality. So what happens? On Sunday they give up a ton of points to the Falcons and lose.

And the Steelers move up from 8 to 6.

At some point you guys have to acknowledge that, whatever DVOA is supposed to be measuring, the placement of the Steelers is exposing a serious flaw. I realize you have a lot invested in this system, but it's becoming embarrassing.

I've followed the link to your explanation, but it doesn't really explain what your system is. I will say that there does seem to be some kind of weird inertia to your system. Last week, Philly was rated third. They lost to an inferior Tampa team and climbed to 2nd.
Pittsburgh was 8th, lost to an inferior Falcons team, and climbed to 6th. Jacksonville was 4th, lost in an embarrassing fashion to a very weak Texans team, but they only dropped to 8th, where they are still ranked one spot higher than the undefeated Colts.

This thing ain't working.

60
by John (not verified) :: Tue, 10/24/2006 - 7:37pm

"At this point the ratings are starting to look really weird to the average fan.... My hope is that this will help those readers who are a) new to DVOA and b) open-minded."

I am open minded, and I’m not new to your DVOA ratings (following it for 3 years now). But I think it’s time for you to go back to the “drawing board� and rethink how to calculate your ranking.

61
by Dan (not verified) :: Tue, 10/24/2006 - 7:44pm

45
Pittsburgh is estimated at 4-3, not 4-2. Estimated Wins are assuming 7 games for every team, which goes a long way towards explaining why they seem to have one win too many.

62
by Kal (not verified) :: Tue, 10/24/2006 - 7:46pm

#61: even then, it still doesn't explain quite how Pittsburgh's defense is rated so high despite losing. Or for that matter how their offense is rated so high either. That one really confuses me. Hopefully the FoxSports commentary will talk about that, because it's odd seeing them ranked above Jacksonville, who is ranked above Indy, when Indy beat Jacksonville and Jax beat Pitt.

63
by Vince (not verified) :: Tue, 10/24/2006 - 7:47pm

#58: They did stomp the Smith-less Panthers and the Bucs. Maybe not the FO definition of stomp, but they clearly outplayed those two teams.

64
by paytonrules (not verified) :: Tue, 10/24/2006 - 7:47pm

#60

I'll pick on John, cause he's there. John - why? "Cause I don't think this is right" isn't a good answer. Aaaron has put a lot of work into the formulas for this, and it's got the best correlation coefficient of any method out there.

Footballoutsiders.com has spent four years working hard and proving their formulas to be accurate, and the burden of proof no longer rests on them. It's up to, at this point, the challengers to come up with better solutions.

65
by Jake (not verified) :: Tue, 10/24/2006 - 7:49pm

The Giants are way up there, didn't realize this...

66
by Vince (not verified) :: Tue, 10/24/2006 - 7:50pm

Pittsburgh lost to Atlanta because they LOST three fumbles, not just that they fumbled three times. They also lost another possession when the Falcons recovered the onside kick. Those are fluke plays, not clear indicators that Pitt is better than ATL. I don't know, but I'd expect that DVOA had Pitt ahead of ATL in that game. That's why they moved up.

67
by admin :: Tue, 10/24/2006 - 7:52pm

Hmmm.

PHI last week: 39.9%
PHI this week: 39.2%

Maybe I failed introductory math, but that looks like down to me. Not a lot of down, but definitely not up.

68
by Kaveman (not verified) :: Tue, 10/24/2006 - 7:54pm

#49: Just like NE always seems to beat Indy, and Indy always seems to beat Denver.

Two wild card blowouts is "always"?

Denver and Indianapolis have met 16 times in the regular season and twice in the post season. Denver leads the series 11-6.

In the regular season only, Denver is 11-4 against the Colts.

In Denver, the Broncos are 6-1 against the Colts. Which means that in the regular season, Denver is 5-3 against the Colts in Indianapolis.

The last two regular season games were won by the Broncos, 31-17 and 33-14 (you can asterisk one, because the Colts were resting their starters).

Those playoff blowouts can color your perspective, but take a closer look at the data. Indy certainly does NOT "always" beat Denver.

Remember too, the Roc Alexander factor--trying to cover Reggie Wayne with an undrafted free agent CB who has had little to no practice time because of injuries to two starting DBs... well, that might have had an impact on one of those games, eh? And the RCA dome... what is NE's record against Indy in that stadium? Anyone know?

But yeah, let's please lay this misconception to rest. Much as it may delight people to imagine a rock-paper-scissors relationship between the Broncos, Colts and Patriots, it is no such thing.

69
by Will Allen (not verified) :: Tue, 10/24/2006 - 7:55pm

Hey, Aaron, I hope you didn't interpret my remarks about interceptions as criticism; a football game is comprised of a huge amount of information, and until you get an army of charters providng you data to the most minute detail, everything can't be analyzed. If I were really bothered, I should be charting games for the betterment of the analysis.

As to the Vikings, I hope my comments don't seem like harping; I think DVOA and PFP are easily the best objective
analysis available. I just thought it interesting that my projected wins for the Vikings, even accounting for my status as a fan, my was so different than PFP 2006, since I like to think I am a fairly objective fan.

When I looked at the Vikings before the season started, I saw a team with above average offensive and defensive lines, that would be much better coached than in the recent past, and that was playing one of the weakest schedules in the league. I just thought it extremely unlikely that such a team would win fewer than six or seven games. What was the last team with above average lineman on both sides of the ball, that played a schedule as weak as the Vikings', and did not win seven games? This isn't a knock on FO methods, it just again reflects the huge amount of information contained in a football game, perhaps the fact that line play is more difficult to quantify, and perhaps that new personnel on the line really messes with win projections.

Finally, Brad Johnson gets the praise because all he does is win, baby. (sarcasm flag) Seriously, though, one of the best things that can be said about Johnson is that they don't pay him much, which frees up cash to lock up players like Mckinnnie, Hutchinson, probably Kevin Williams this off-season, and two cornerbacks that have played decently this year. What kills a team's prospects is devoting too much cap space to guys who don't provide enought in terms of improving the chances of victory, which is why I think a cap space-adjusted DPAR rating for qbs would be so interesting.

70
by Andrew (not verified) :: Tue, 10/24/2006 - 7:56pm

The Eagles have lost all three of their games on the last play of the game. They just have to start winning that last play!

71
by NewsToTom (not verified) :: Tue, 10/24/2006 - 8:04pm

Tennessee is clearly ranked too low because I refuse to believe we're not better than Houston and Oakland. The CareerBuilder monkeys throwing darts are way better than this. K3RR3 C0771N$ iS 43H SuXX0R!

72
by Kaveman (not verified) :: Tue, 10/24/2006 - 8:07pm

Aaron or someone here who knows: while Denver's red zone defense (2 for 13 TD rate, 15.4%) is unlikely to be sustained, what about 3rd downs? Are defenses that seem to be particularly good on 3rd downs able to sustain it? Or is it random, like fumble recoveries?

Little factoids from the Broncos' news release this week: opponents are 0 for 9 on 3rd downs inside Denver's 20. Further, opponents have converted only 3 of 34 3rd downs of 9 yards or longer (8.8%). I'd love to know what other teams' numbers are like.

73
by Kaveman (not verified) :: Tue, 10/24/2006 - 8:11pm

Sorry for the multiple posts, but I have another question--are penalties a factor in DVOA? Denver is the second least penalized team in the league thus far, with 22 penalties. Does DVOA give them credit for that discipline?

74
by Subrata Sircar (not verified) :: Tue, 10/24/2006 - 8:11pm

The Fox commentary mentions three situations, but the preceeding paragraphs in the commentary only mention two. (I'm sure that there are at least three such situations and probably more, and it's probably an editing gaffe, but ... writing correctly and well is the primary tool of your trade, so it's important.)

75
by Staubach12 (not verified) :: Tue, 10/24/2006 - 8:17pm

What were the DPAR numbers for Bledsoe and Romo? Romo had a significantly higher QB rating than Bledsoe for the evening (though neither one was very good). Does DPAR offer a similar picture?

76
by BlueStarDude (not verified) :: Tue, 10/24/2006 - 8:18pm

Aaron - I'm really surprised by Dallas's continued high defensive ranking (happy, but surprised). So often they seem to give up four or more yards on first down. Are they propped up by an overachieving third-down defense? An ability to force turnovers? Or is my anecdotal memory that far off?

77
by Luz (not verified) :: Tue, 10/24/2006 - 8:34pm

#74 is right, you only list two scenarios but say there are three. i'm assuming the third is the bengals vs tampa bay two weeks ago?

78
by DavidH (not verified) :: Tue, 10/24/2006 - 8:44pm

22:
I like your Luck Index idea.

25:
Isn’t comparing actual wins to estimated wins a pretty good quantitative indicator of luck? Or actual wins to wins predicted by DVOA?

Critic of DVOA: Why are Eagles only 4-3 despite 6.1 estimated wins?
Aaron: Because they're unlucky.
Critic: How do you know they're unlucky?
Aaron: Because they're only 4-3 despite 6.1 estimated wins.
Critic: ... [head explodes]

14, 35, 51:
I just skimmed that link that Pat provided about turnovers, but it doesn't look like the type of turnover is accounted for. I think Will Allen is right - a breakdown of where INT's happen and how often each type gets returned for a score would be pretty interesting. Intuitively, it seems like some QB's might be more prone to getting jumped on short sideline throws, and hence getting more picks run back (*cough* Drew Bledsoe *cough*).

52:
75% of the time X is something that I’ve already tried to put in the formula that didn’t actually improve it and in many cases made it less accurate, and 20% of the time X is something that we simply can’t measure because it’s not in the standard play-by-play and the charting project doesn’t turn data around fast enough.

I'm guessing that where an INT was caught is one of the things that are not included in the play-by-play. And that the only semi-solution I can think of, which is to treat INT's that are run back as worse than INT's that are not, is one of the things that has been tried and has not helped.

73:
I'm fairly sure (80% sure?) DVOA does not take penalties into account.

79
by Jim (not verified) :: Tue, 10/24/2006 - 8:55pm

51. Aaron is correct that the Vikings are winning because of defence, but I didn't think Johnson was getting much credit. He has not played well since week 1, but the receivers have really killed him with some key drops including one that likely lost the Buffalo game.

80
by Nate (not verified) :: Tue, 10/24/2006 - 8:57pm

The Rackers field goal at the end of the Arizona game did "graze" the finger of a Bears player, so the Bears had something to do with it. See link.

81
by Richard (not verified) :: Tue, 10/24/2006 - 9:04pm

Correlation between DAVE and Power Rankings...

CBS : .781
ESPN: .783
FOX : .832

82
by Jim (not verified) :: Tue, 10/24/2006 - 9:05pm

I suppose there is some real complicated answer to this, but I don't understand how a team averages -.2 yards per play (NE) ranks way higher than a team that averages +.7 (Minn) yards per play.

That just seems way too big a gap to me to explain away. I thought the same thing leading up to the Minn vs Sea game last week.

83
by Will Allen (not verified) :: Tue, 10/24/2006 - 9:05pm

Yep, Johnson hasn't been good, or to be more accurate, he has been about as good as his now-limited physical abilities allow, but the Vikings' receivers have been hideous, likely costing them the Bills game, and nearly costing them the Redskins game, which probably should have been a ten point margin of victory. They drop a lot of passes, and make practically zero above-average plays on the ball. This is the worst Vikings receiving corps I can ever remember; you may have to go back to the Norm Van Brocklin era (before my time) to match it. Oh well, when the receivers are this bad, at least one doesn't worry as much when they get injured.

84
by Moridin (not verified) :: Tue, 10/24/2006 - 9:06pm

Aaron, in the Jets commentary, it says the '3 wins' instead of '3 loses' in middle of the commentary.

85
by Vince (not verified) :: Tue, 10/24/2006 - 9:06pm

#75: The freakish thing is, Romo basically doubled Bledsoe's numbers for the game.

Bledsoe was 7 for 12 for 111 yards.

Romo was 14 for 25 for 227 yards.

The difference? 7 for 13 for 116 yards.

Bledsoe threw 0 TDs (he did run one in, but the passer rating doesn't give him credit for that) and 1 INT. Romo threw 2 TDs and 3 INTs.

So the QB rating would have these guys neck-and-neck in completion percentage and yards per attempt. Bledsoe would actually have a better interception percentage than Romo. But Romo gets credit for 2 TDs, while Bledsoe gets a big fat zero in that department.

So that, really, is why Romo's rating was higher: He threw 2 TDs, Bledsoe had none.

86
by Pat (not verified) :: Tue, 10/24/2006 - 9:10pm

Critic of DVOA: Why are Eagles only 4-3 despite 6.1 estimated wins?

There are better answers to that, though.

1) When a good team plays a good team, one of the teams gets all of the win, even if it's a close game. Close only counts in horse shoes, hand grenades and nuclear weapons, but to treat close losses and close wins just like blowout losses and blowout wins is crazy.

2) Opposing field goal kickers have been ridiculously more accurate versus the Eagles than they have been the rest of the year.

3) Fumble recovery.

I suppose “average� fumble recovery luck might well have got them to 3-3 , but 4.1 seems like at least one too many wins.

See above. You're quantizing wins too much, and it's too early in the season for that. (And they could've been 3-3 with a different flip of a coin. That's absolutely luck. It's the way the game goes, and I personally don't have a problem with it, but that's luck.)

It's not just fumble recovery luck. It's also "single-play luck" - that is, any outcome that's dependent completely upon one single play might as well be luck. Why? Because it is just luck. There probably was a hold on that play that could've been called, or interference, or any number of occurances which probably could've happened, but didn't.

What people really don't understand is that two teams can both be lucky to win in a game. In fact, in almost every game that's separated by only one score, teams were at least somewhat lucky to win. That's what makes it a game.

The Cincinnati game was single-play luck. The Jaguars game was single-play luck. And the Falcons game was absolutely single-play luck.

This happens all the time at about this time of the year. People think that 6 or 7 games is enough for a record to describe how a team played. No way. 16 games isn't enough. (Heck, 162 games isn't enough) 7 certainly isn't.

2-4 is certainly not far from 4-2.

87
by Pat (not verified) :: Tue, 10/24/2006 - 9:13pm

but I don’t understand how a team averages -.2 yards per play (NE) ranks way higher than a team that averages +.7 (Minn) yards per play.

NE Special Teams: 4.3%
MIN Special Teams: -0.1%
Difference: 4.4%

NE-MIN DVOA: 5.2%

There's your answer. It's amazing how many people don't consider special teams. If you mean the FOX rating (DAVE) instead, that's just because of Minnesota's low preseason projection. It'll go away next week.

88
by Kal (not verified) :: Tue, 10/24/2006 - 9:16pm

Thanks for the explanations in the fox article, Aaron. That does help a lot.

89
by NF (not verified) :: Tue, 10/24/2006 - 9:17pm

22, 78: The Luck Index couald actually be calculated from the VOA charts in the first few weeks of the rating. The early charts had total VOA and non-adjusted total VOA. Non-adjusted total VOA does not treat fumble recoveries as random, and uses special team VOA that is not adjusted for opponent kickoff and punt length and field goal percentage. By subtracting total VOA from non-adjusted total VOA, you had a measure of the luck of teams. The Eagles were one of the unluckiest team a few weeks into this season as well.

78 (again): According to the PFP 2006, DVOA 5.0 does account for the location of interceptions on the field, and adjusts for variation in interception returns by assuming that the interception was returned the average distance given the location of the line of scrimmage and where the interception was caught. Aaron has stated that the main reason for not having how good an offense is at stopping returns of interceptions is that it is a questionable, and difficult to weight, measure of the overall quality of an offense.

Also, DVOA 5.0 counts defensive pass interference.

90
by Nathan (not verified) :: Tue, 10/24/2006 - 9:17pm

Perfect Cards riff. Thought of that when I was watching the game.

Good times.

91
by admin :: Tue, 10/24/2006 - 9:18pm

I think that comment 74 wins an award for most anal, nitpicking troll of all time. Originally, I talked about the Chargers-Chiefs game in the intro also, and when I decided to cut that, I apparently missed one word. Alas, once again I am felled by my human weaknesses. Truly, all is for naught.

92
by DavidH (not verified) :: Tue, 10/24/2006 - 9:19pm

87:
Whoa, hold on, accounting for special teams gets them even. So then the question changes to

"I don’t understand how a team averages -.2 yards per play (NE) ranks the same as a team that averages +.7 (Minn) yards per play."

I'm not saying the rating is wrong, just that ST isn't the whole explanation.

89:
Cool, happy to be corrected if it means DVOA accounts for more than I thought.

93
by Kaveman (not verified) :: Tue, 10/24/2006 - 9:21pm

#78: I’m fairly sure (80% sure?) DVOA does not take penalties into account.

It seems as though it must though. If you compare a play against the average play in that situation, and the result of your play is a penalty, doesn't that count as a loss of yards on the play? And that should always have negative DVOA, if I understand it correctly...?

94
by Kevin (not verified) :: Tue, 10/24/2006 - 9:22pm

The Eagles fan crying is getting a bit tedious. As a Giants fan I can't give the Eagles a "We should've won all our games!" pass like Aaron wants to. After everything that happened in the Giants-Eagles game, the Eagles still got the ball at their 44 yard line in overtime and needed only 20 yards to make a reasonable GW FG attempt. Instead, they went backwards 8 yards. Allowing Eli Manning to complete his last twelve pass attempts also doesn't strike me as something a team does that "should've won". In all three of their losses, a reasonable person could point to something a winning team rarely does such as allowing 8 minute game ending drives, allowing 14 points on INT returns, or an inability to convert 4th and 1 carries. The best team Philadelphia has beaten involved a closely contested game against a team that was flattened by the Giants last night. That says more to me about the Eagles level of play than the fact they lost three close games.

95
by Staubach12 (not verified) :: Tue, 10/24/2006 - 9:30pm

#85: Thanks. That makes more sense.

Aaron: Does DPAR suggest that there is a difference between Bledsoe and Romo?

96
by Malene, cph (not verified) :: Tue, 10/24/2006 - 9:31pm

"I honestly have no clue how good this team really is, and neither do you. For that matter, neither does Herman Edwards."

No, but he knows that you play to win the game.

Or so I've heard.

97
by MRH (not verified) :: Tue, 10/24/2006 - 9:31pm

The 49ers actually rank fourth in the league in first-quarter offense, which is usually one of the leading indicators of a good team.

By "leading indicator" do you mean it's one of the key statistics that signifies a good team now (this season), or, as it's used in economic analysis, one of the statistics that predicts future performance, i.e. the 49ers are looking good for 2007? Thanks.

98
by Yaguar (not verified) :: Tue, 10/24/2006 - 9:33pm

Pittsburgh is clearly ranked too high because Roethlisberger clearly did not score on that scramble in the Superbowl. Beating a dead horse is way better than this. Ph4nt0m holds r teh suxx0r!!11

99
by MRH (not verified) :: Tue, 10/24/2006 - 9:41pm

Re #18 Seattle may not plummet; they’ve gone through the toughest part of their schedule by a longshot, and go into a very easy schedule coming up - KC, Oakland, and SF in 3 of their next 4 games. STL will likely be a loss, but KC and Oakland are very winnable, even with Seneca Wallace.

I'm not seeing the KC part of your comment. STL and KC are essentially equal in DVOA. SEA plays STL at home and you see that as a loss. SEA plays KC on the road and you say that is "very winnable". My memory of young/inexperienced QBs coming into Arrowhead (Alex Smith, Phillip Rivers this year alone, although Rivers was better in the 2nd half) is that they play poorly. At the beginning of the year, I had this game as about even but more on the loss side of the column. After Green went down, it was a definite loss. With Hass and Alexander out, and Huard playing respectably, I see it as a win. Not a definite win, but the line today on a couple of books was KC by 6 (most books had it OFF). That seems about right.

100
by Bill (not verified) :: Tue, 10/24/2006 - 9:43pm

89 - Is that solely the difference between the LOS and the interception point? Realistically, you'd like to have it based on the intercepting point relative to where the pass is thrown, in two dimensions. Obviously, we don't have that information without full game charting, though.

101
by JasonK (not verified) :: Tue, 10/24/2006 - 9:49pm

94:

You can always cherry-pick individual plays to make a team look bad or good. The statistical formula takes every play into account, including, for example, the three quarters of total ineptitude by the Giants ("something a winning team rarely does").

(Also a Giants fan, btw)

102
by Kal (not verified) :: Tue, 10/24/2006 - 9:50pm

#99: KC is winnable. It really depends on what version of KC shows up that day. Is it the one where LJ goes crazy on a great run defense, or the one where LJ sucks against a mediocre one? Where Huard looks stellar or Huard looks weak? I don't know. I do think Wallace will look better in his first start than he did getting thrown into the mix, but I can't say that Seattle will instantly lose it.

St. Louis, OTOH, I say because STL's a division rival. STL needs to win if they want to stay in the hunt for the NFC west. I think they will, even with the advantage of Qwest.

Still, you're probably right. I doubt they'll win more than 2 of their next 4 games at best. But that still puts them at 6-4, and they'll likely have Alexander and Hass back by then. That's not the worst situation ever.

103
by Pat (not verified) :: Tue, 10/24/2006 - 10:07pm

that was flattened by the Giants last night.

The Dallas/Philadelphia game was just as close as the Dallas/New York game. Just switch the order of a touchdown and an interception returned for a touchdown. (Feel free to say "but... but... it was a prevent defense!" but prevent defenses aren't intended to allow a 74-yard touchdown in under 30 seconds).

I’m not saying the rating is wrong, just that ST isn’t the whole explanation.

No, you're right - it's probably about half of it. It's the only half I could see right away, though. :)

I thought it was because Minnesota doesn't force many turnovers, but that's not it - Minnesota's defensive DVOA is better than their rank in yards/drive.

I'm pretty sure it's because Minnesota's offense is worse than it looks in terms of yards/drive (hence yards/play). They're second last in the league in terms of touchdowns per drive. So they're driving a fair distance (and unlike Denver, they aren't starting with the worst field position in the league), but stalling out before the end zone. So they don't score as many points on offense as you'd expect given their yards per play.

104
by Will Allen (not verified) :: Tue, 10/24/2006 - 10:13pm

I'll go farther than that, Pat. I think there have been more than a few teams that went 10-6, as opposed to 6-10, and vice versa, due to luck, than is generally acknowledged, which is why I really like the PFP win projection system. Toss in schedule strength and injuries, and going with a team's win/loss record in any single season as an indicator of team quality is pretty problematic.

105
by DavidH (not verified) :: Tue, 10/24/2006 - 10:20pm

103:
I wonder how much Chester Taylor's 95 yard run skews the stats. Well, hell, why don't I find out. Let's say he only gets 15 yards on that play.

Current offensive yds/play - 1958/374 = 5.2
New offensive yds/play = 5.0
So there's 20% of the difference. (numbers are probably not entirely accurate, but should be close enough to get the magnitude of the change right)

106
by DavidH (not verified) :: Tue, 10/24/2006 - 10:21pm

The mention of three tiers of teams reminded me of the REAL Standings I've seen on the Phog Blog (linked). For the Big 12 college basketball season, the teams are split into 3 tiers, and wins and losses are projected based on those tiers and the site of the game. Essentially, you're supposed to

a) a teams should win all home games against same- or lower-tiered teams
b) a team should win 1/2 of home games against teams in the tier above you
c) Tier 1 teams should win ALL games (home and road) against Tier 3 teams

Using these peredictions, you can look at the standings midseason and account for how the rest of the year will go. I thought it'd be interesting to do the same thing in the NFL with your 3 tiers. ONly problem is that I don't know if a, b, and c will really apply given the tiers we have here.

So, I took a look at the season to date. For this chart, "-X" means a team is two tiers below. So Chicago hosting Detroit would be "vs -2" from CHI's perspective, and "@ +2" from Detroit's perspective:

MATCHUP ..... RECORD ..... WIN%
vs -2 ..... 4-0 ..... 1.000
vs -1 ..... 22-2 ..... 0.917
@ -2 ..... 6-1 ..... 0.857
@ -1 ..... 19-7 ..... 0.731
vs same ..... 23-16 ..... 0.590
@ same ..... 16-23 ..... 0.410
vs +1 ..... 7-19 ..... 0.269
vs +2 ..... 1-6 ..... 0.143
@ +1 ..... 2-22 ..... 0.083
@ +2 ..... 0-4 ..... 0.000

Yes, I know half those lines are redundant, but I like the nice smooth progression from 1 to 0, along with the very logical ordering of the matchups. Anyway, you can see that home games against the same tier are not gimme's by any means.

So, I used those values as predicted win values for teams' future games. And, it's really boring. The final standings look exactly like the current standings, except that SD passes Denver. And the current tie between Baltimore and Cincinnati ends with Baltimore ahead. Predicted playoff seeding is

AFC - SD, NE/Ind, Bal, Den, Jax/Cin
NFC - Chi, NYG, Sea, NO, Phi/Min
1st draft pick - Oak/Ari

107
by Kevin (not verified) :: Tue, 10/24/2006 - 10:25pm

#103... Pat, I'm really not sure how you could say those two games were similar or that the DAL/NYG game was close. The last time Dallas had the ball with the opportunity to tie the game or take the lead was Tony Romo's first play/interception/first play of the third quarter. In the DAL/PHI game, the Cowboys' final 6 possessions were all with the team having a chance to either tie or take the lead. The two possessions prior to those for DAL were when they had a 4 point lead. Unless you consider needing a FG, TD, and a two point conversion "close", I don't understand your point.

108
by Will Allen (not verified) :: Tue, 10/24/2006 - 10:30pm

Pat, your analysis of the Vikings offense is pretty much on the mark. Their offensive linemen to the right of Birk have also taken a lot of penalties in the red zone, which has hurt their touchdown percentage as well. I have reason to hope that the offensive line play will continue on it's upward curve, so maybe the offense will start earning it's pay. The offensive line played pretty darn well is a very noisy Seattle stadium Sunday, so there is reason to have hope against the Pats at home. Given the Pats' weakness on defense, however, the Vikings' receivers better have a conversion experience. I have a feeling Belichik will do something interesting to reflect that Vikings particular strengths, and widespread weaknesses on offense.

109
by Will Allen (not verified) :: Tue, 10/24/2006 - 10:33pm

I think I read somewhere about DVOA cutting a run off statistically at 40 yards, but perhaps that is an older version.

110
by Becephalus (not verified) :: Tue, 10/24/2006 - 10:35pm

If there are large gaps between the teams why not sort them that way instead of arbitrary groupings of 4? Might make things more clear. You could even go really unorthodox and provide each team with a graphical representation of there DVOA and have a chart.

CHI IIIIIIIIIIIIIIIIIIIIII
PHI IIIIIIIIIIIIIII
SD IIIIIIIIIIIIII
NYG IIIIIIIIIIIII
PIT IIIII
...
OAK ------------
TEN -----------------

I am as math savy as anyone and I appreciate the tables and such, but such a graph would even help me notice thigns I might otherwise miss, and it would probably be much more appealing to the average fan. Just an idea of how to make th fox site more informative.

111
by CaffeineMan (not verified) :: Tue, 10/24/2006 - 10:37pm

On Denver's defense (defense only):

Looking from this simply from a high level point of view (not attempting to scout them), Denver's D has faced two really good offenses, 1 mid-pack offense, and 3 bottom of the barrel offenses. Listed below are the opponents offensive DVOA's (OO-DVOA) and the average of all their OO-DVOA's. I'm looking at the real ones, not DAVE, since I'm interested in the performance just this year so far:

STL L10-18 OO-DVOA 13.0% (5th)
NE W17-7 OO-DVOA 11.7% (6th)
KC W9-6 OO-DVOA -2.8% (18th)
BAL W13-3 OO-DVOA -17.2% (27th)
CLE W17-7 OO-DVOA -21.4% (30th)
OAK W13-3 OO-DVOA -35.2% (32nd)

AVG OO-DVOA -8.7% (would rank 24th)

Denver's defensive DVOA is -15.3% (7th), which seems right, given the the strengths of the opposing offenses. Maybe they're capable of shutting down more good teams in the future, but so far, it seems like DVOA is doing what it's supposed to do. Even if Denver's O is leaving their D with bad field position, the Denver D is facing some bad offenses that can't take full advantage of the bad field position. Denver shuts down those bad offenses even more than they have been shut down in the past and thus they end 7th. I can't see a problem with DVOA here, given the data it has to work with.

112
by blahblahfalcons (not verified) :: Tue, 10/24/2006 - 10:41pm

"Green Bay is hoping that a terrible day for Chris Chambers — only two catches out of nine passes in his direction — is a signal that shutdown corner Al Harris is coming out of his early-season funk."

I didn't see the GB-MIA game - was Al Harris manned up on Chambers more than Woodson?

113
by Pat (not verified) :: Tue, 10/24/2006 - 10:46pm

If there are large gaps between the teams why not sort them that way instead of arbitrary groupings of 4?

I agree with this - there's some ranking (NFL.com?) that splits up the NFL into groups, and then just lists the teams alphabetically. I don't really agree with that part - might as well rank 'em, there's some difference - but grouping the rankings might be a good compromise.

Although the graphical indicator would be cool. You'd have to give it a neat name or something.

114
by Pat (not verified) :: Tue, 10/24/2006 - 10:49pm

Even if Denver’s O is leaving their D with bad field position

Woah, did someone say this? If so, they're wackball nuts. It's the other way around. Denver's defense is leaving their offense with bad field position. Worst in the league. Getting an interception in the end zone is better than giving up a touchdown, but it's way worse than stopping a team 3-and-out on their own 20.

Denver's offense is leaving their defense with good field position. Their defense has been starting with the 11th best LOS/drive in the league.

115
by Vince (not verified) :: Tue, 10/24/2006 - 10:55pm

I think the graphical indicator of a team's DVOA is a great, great idea.

116
by Will Allen (not verified) :: Tue, 10/24/2006 - 10:59pm

I'll fourth that.

Don't you love how we suggest more work for you Aaron?

117
by Pat (not verified) :: Tue, 10/24/2006 - 11:05pm

Oh, and I forgot:

Kudos to Aaron for a fantastic comparison between the Eagles and the Bears. I didn't even realize that their situations were so ridiculously identical until you pointed it out.

And it even included the quarterback who's been fantastic all year throwing multiple interceptions in both games.

118
by Becephalus (not verified) :: Tue, 10/24/2006 - 11:11pm

-52 Thats easy, it is very easy to put one mans face in a TV spot, to talk about one man, it is much harder to do so for a whole unit of roughly equally contributing men. I really think the TV culture with the requirements for stories etc. reinforces the QB RB bias greatly. I (and I am sure many other) football fans have been stressing the importance of line play for so many years. Yes the QB is the most important player, but not by much. Lazy reporters who pander to ignorant audiances are why Orton, Johnson etc. are seen as sucess stories.

The QB is what 10% maybe a MAX of 15% of the total contribution to the quality of a team?

Anyway as an educated football fan I am pretty sure you know all this, just answering your question.

119
by CaffeineMan (not verified) :: Tue, 10/24/2006 - 11:14pm

Pat, I was addressing this comment by mediator12, back in #6:

Still interesting to see how low Denver is to the supposedly stronger defenses in the league. Seventh in DVOA and twenty two of 44 points surrendered from TO’s in scoring position.

Great to see how much Jake Plummer makes this defense look worse than they are. Hey let’s start him against the Colts Mike Shanahan. He should help boost Peyton’s stats some more this week.

I took that as a claim that one factor in why DVOA didn't correctly characterize Denver's defense was because Denver's offense turned the ball over deep in it's own territory, so the opposing offense had much less work to do to score and thus Denver's D was that much better for having prevented them from doing so. I was pointing out that even if Denver's D WAS constantly working with it's back to the goal line, it was still only doing so against bad offenses, so keeping them out of the end zone is only worth so much. Thus, they're 7th and no higher, given what data there is available.

120
by NewsToTom (not verified) :: Tue, 10/24/2006 - 11:20pm

Re #112
Ask me in 4 hours when I've done my half charting bit. From my last game-watching recolletion, I don't think so, nor did Harris stand out when the ball was thrown in his direction. But, with 62 passes and two teams I don't really follow, I don't trust my recollection.

121
by NewsToTom (not verified) :: Tue, 10/24/2006 - 11:23pm

Re #110, 113, 115-16
I think Aaron should describe that as a Dvoa RAnge Graph.

122
by Pat (not verified) :: Tue, 10/24/2006 - 11:23pm

Yeah. The other thing he neglects to realize is that the only reason Denver's offense has turned the ball over so deep in its own territory is because they've had an abnormally large amount of chances to... due to the defense constantly putting them in a bad spot. (Note that when I say "constantly" what I really mean is "much more than the top, top defenses in football", like Chicago or Baltimore).

123
by Jim (not verified) :: Tue, 10/24/2006 - 11:31pm

87 Pat. Yes I understand that the Patriots apparently have better special teams play, but special teams account for 1/7 of the game (according to this site)...I'm very skeptical that special teams differences could make up such a huge difference (.9 yards per play) for the part of the game that takes up 6/7 of the game.

There has to be more to it than special teams.

124
by admin :: Tue, 10/24/2006 - 11:40pm

Hey. I answer BlueStarDude's question about the Dallas defense (comment 76) in a new post at the FO FOX blog. Click on my name for the link.

125
by Richard (not verified) :: Tue, 10/24/2006 - 11:44pm

Something like... [click my name]?

126
by Pat (not verified) :: Tue, 10/24/2006 - 11:45pm

I’m very skeptical that special teams differences could make up such a huge difference (.9 yards per play) for the part of the game that takes up 6/7 of the game.

I'd think in terms of yards/drive more than yards/play. That way adding field position via special teams is easy. Minnesota nets 4.96 yards/drive. New England nets 3.13 yards/drive.

But considering the LOS/dr, that amounts to about 5 extra net yards per drive for New England compared to Minnesota. So yes, it does boost New England that much.

(Incidentally, on the drive stats page, the reason that the team that's top in net LOS/dr - Atlanta - is not anywhere near the top in special teams DVOA is because net LOS/dr ignores field goal kicking, which gets folded into points/dr, and Atlanta's field goal kicking has been horrendous).

But you're right, that isn't all of it. The rest is what was mentioned earlier. So, to summarize: The reason Minnesota's ranked so much lower (and it's really not that much) is because 1) special teams (NE good, MIN bad), 2) red zone performance (NE good, MIN bad).

127
by Richard (not verified) :: Tue, 10/24/2006 - 11:51pm

Should have been this link, not the other one.

128
by Kaveman (not verified) :: Tue, 10/24/2006 - 11:51pm

Just to be clear--I, at least, am not asking why Denver isn't ranked higher. In #72 I'm asking about 3rd downs and whether there are any numbers on the ability of a defense to be better on 3rd downs than on other downs. In #73 I'm asking about penalties and DVOA.

I tend to agree that Denver's D cannot be classified among the league's best as long as it continues to give up field position. But... hypothetically, if the Colts have 400 yards of offense but score under 20 points in a Broncos win, DVOA will still not like it. Broncos fans, however, will. :-)

129
by Moridin (not verified) :: Tue, 10/24/2006 - 11:54pm

re: 125, Personally, I think a column graph would be better (up/down comparison comes across easier than left/right normally), but that either way, I think something that that would be a great addition to the fox posts.

re: Dallas D
I just don't see how its possible for a D to be so bad on 1st down and great on every other one. Do they just utterly suck at guessing what teams are going to do when offenses are looking at getting 4-ish yards

130
by Pat (not verified) :: Tue, 10/24/2006 - 11:57pm

#125: Yes, but you lose points for using Excel.

But… hypothetically, if the Colts have 400 yards of offense but score under 20 points in a Broncos win, DVOA will still not like it. Broncos fans, however, will.

What if Denver only scores 7?

Of course, Denver fans will then blame Jake Plummer. Honestly, I think Shanahan knows that Plummer is playing better than his points per game indicates. At least, I hope so. Cutler can do better, of course, but he could play a lot, lot worse. Plummer's basically replacement level right now. If Cutler comes in and starts playing at McNair's, or Andrew Walter's, or Charlie Frye's level, things would be very bad.

131
by Pat (not verified) :: Tue, 10/24/2006 - 11:59pm

I just don’t see how its possible for a D to be so bad on 1st down and great on every other one.

Problems in the deep secondary. First down is when a lot of teams occasionally take a shot downfield to try to get the safeties off the line of scrimmage. If those shots (which aren't intended to work!) tend to work, that'll kill you.

132
by admin :: Wed, 10/25/2006 - 12:02am

The answer about penalties and DVOA is that right now the only penalty in DVOA is defensive pass interference. The other penalties I tried to add this off-season didn't improve things, and in some cases made the system less accurate. I'll do more testing next off-season. It's addressed in the essay on DVOA 5.0 at the start of PFP 2006.

By the way, Kaveman, I definitely took your questions as reasonable and curious, not trolling.

133
by Marko (not verified) :: Wed, 10/25/2006 - 12:03am

I thought the following excerpt from the Fox commentary on the Vikings was interesting: "By the way, check out the Vikings' schedule in November: San Francisco and Miami on the road, Green Bay and Arizona at home. Holy playoff spot, Batman."

Will Allen has been saying this repeatedly for months. He has said on multiple occasions that if the Vikings could win 3 of their first 7 games, they have a good shot at the playoffs because their schedule after that point is so easy. (A few weeks ago, I posted a comment agreeing with Will, stating that I think the Vikings will make the playoffs.) Well, the Vikings already have 4 wins in the bank through 6 games.

If the Vikings make the playoffs, there likely will be 2 playoff teams from the supposedly weak NFC North. I also think the Giants will win the NFC East, as they have gotten through the toughest part of their schedule in first place with a 3-0 division record, having beaten their two closest division challengers on the road.

Assuming the Bears (what, you thought I was referring to the Packers as the other likely NFC North playoff team?), Vikings and Giants make the playoffs, it's interesting to consider who will be left out. There are so many supposed contenders in the NFC that already have 2 or 3 losses. Seattle might not win the West, with Hasselbeck out for at least 3 weeks and Alexander still recovering from the Madden curse. If the Rams win the West, then Seattle or Philadelphia (with 3 losses already) definitely will be out, because there will be only 1 wild card spot left. And Carolina is in a tough position, in third place now with 3 losses. Plus, teams in the NFC South and East will beat each other up. I still think the Panthers will win the South, but if they don't, it's going to be tough to get that 1 remaining wild card spot.

134
by Pat (not verified) :: Wed, 10/25/2006 - 12:03am

Oh, and regarding that chart? Try adding 50% to every team, and using it as a single graph from 0 to 100%.

The cool thing about that would be that teams that exceed 50% would be "off the charts good" and teams that exceed -50% would be "off the charts bad." :)

135
by Moridin (not verified) :: Wed, 10/25/2006 - 12:06am

re: 131
what, you mean not every team goes deep on 3rd and short like the Vikings? Though, I suppose for them, that's just as pointless as their normal 2 yards short of the 1st down dump passes & draws they run the rest of the time.(vikings fan btw)

Interesting. I would think if it was that obvious a problem, they would get picked on alot more often on the other downs as well. Either way, still an amazing bipolar defense.

136
by Catfish (not verified) :: Wed, 10/25/2006 - 12:06am

re 125,

That's very nifty looking. It really helps you see the big gaps. Even if Aaron doesn't have the time or desire to make one of those, it would still be cool if you or someone else could make one up every week. The only problem is that the legend got cut off.

137
by Travis (not verified) :: Wed, 10/25/2006 - 12:09am

Big fan of the chart idea.

I realize that this would be more time-consuming, but using team colors on the charts (like the charts at NFL Forecast) would make them easier to read.

138
by Pat (not verified) :: Wed, 10/25/2006 - 12:33am

#137: Wow, hadn't seen that site before. What's scary is that it took me five seconds of glancing at the "How Predictions are Made" page to figure out what they were doing, and how. It's a maximum-likelihood method (like Sagarin's ratings, etc.). I'll never figure out why ratings like that don't actually take into account the error in the ratings (which is big, and clearly not the same from team to team) when doing predictions. Just sloppy statistics.

139
by CaffeineMan (not verified) :: Wed, 10/25/2006 - 12:33am

Kaveman (#128):

Yeah, understood, good comments and valid questions, they weren't what prompted my comments. Mine were more in response to the "Defensive DVOA is heavily affected by mistakes by the offense" comments combined with schedule strength comments. While I totally disagree with the hand-waving dismissal of the victory over the Pats, I do think Denver has had some games against bad teams, especially some bad offenses and so I think their defensive and overall DVOA's are pretty reasonable.

140
by navin (not verified) :: Wed, 10/25/2006 - 12:34am

I agree with the idea of a graphical representation of each teams rating. I was going to comment that a lot of the FOX readers don't understand the difference between a ranking and a rating. Aaron tried to explain it but I'm sure a lot of people didn't grasp the distinction.

A good way to do the number on the graph would be to renormalize the difference between the best and worst team to 100 and 0. Of course this would mean that the "value" of 1 is a little different every week, but if the purpose is to show how closely teams are grouped together, this shouldn't be a big deal.

141
by Richard (not verified) :: Wed, 10/25/2006 - 12:35am

Vertical, +50%, Still Excel

142
by Scott de B. (not verified) :: Wed, 10/25/2006 - 12:41am

Among all the hullabaloo, the Patriots continue slowly making their way up the rankings. That's something I like to see.

143
by Pat (not verified) :: Wed, 10/25/2006 - 12:44am

I'm going to stick with the "horizontal is better" there. But otherwise that looks perfect.

A good way to do the number on the graph would be to renormalize the difference between the best and worst team to 100 and 0.

Eh. The only team that would break the "+50%" idea over the past 8 years is San Francisco, last year. And I think it's kinda cool to have the possibility of teams being off the charts, because man, the 49ers last year were off the charts bad.

144
by Travis (not verified) :: Wed, 10/25/2006 - 12:45am

Re: 138

Yeah, I consider that site a toy, not something whose numbers should be taken seriously. The graphics are nice and clean, however.

145
by Will Allen (not verified) :: Wed, 10/25/2006 - 12:45am

Marko, thanks for the plug, but to be completely accurate, I thnk I've said that a 3-4 start would just about ensure nine wins, which doesn't always get you to the playoffs. Now I'm thinking ten wins is the most likely outcome for the Vikings, with a shot at eleven if they get some luck. Of course, if they shoot pure snake eyes from here on out, they could get seven. The improvement of the special teams in recent games should give Vikings fans a lot of encouragement.

If Koren Robinson hadn't fell off the wagon (which is kinda' like saying "If Robert Gallery had a quick first step"), I think the Vikings would have had an outside shot at twelve wins.

146
by Richard (not verified) :: Wed, 10/25/2006 - 12:50am

143: And I suppose I could use team colors, but that'll make my "Excel-to-Paint" adventure a little longer.

147
by Moridin (not verified) :: Wed, 10/25/2006 - 12:53am

re 146
I like that graph the best so far, but of course, that's just my one vote.

148
by navin (not verified) :: Wed, 10/25/2006 - 12:55am

Disclaimer: I am a SF fan.

San Fran was off the charts bad in their road games last year, but they were pretty competitive at home. They went 3-5 at home, and were within seven points in another three games. I think the early game really messes with the west coast teams. I remember a lot of early games where the west coast team goes down early, then rallies in the fourth quarter when they finally wake up. (Think Seattle-Washington last year.)

SF 28 STL 25
SF 31 DAL 34
SF 3 IND 28
SF 15 TAM 12
SF 6 NYG 24
SF 25 SEA 27
SF 10 ARI 17
SF 17 HOU 10

Uh oh, another DVOA suggestion... One thing I wish DVOA did differently is to moderate blowouts if it wants to create predictive ratings. Is there really that much of a difference between a -100 loss and a -75 loss? Maybe it could use diminishing returns to create a third new rating, how about fitting that into the name DANE?

Of course if the purpose is to measure exactly how you played on every play, then DVOA is very good, but it doesn't work as well as a predictor that way.

149
by PatsFan (not verified) :: Wed, 10/25/2006 - 12:56am

Hard to believe the Pats are that high, given that they've played 2.5 real teams (Jersey/B gets a 0.5). On the flip side, it's hard to argue with where DVOA puts the Pats defense, ST, and run offense. I have to admit I'm having a tough time wrapping my brain around the Pats pass offense being rated 5th, with Brady only completing 56% of his passes. I guess he manages to complete them when they count :).

150
by Travis (not verified) :: Wed, 10/25/2006 - 1:09am

Re: 146

Here's a page listing the official colors for all NFL teams, if you want to waste the time.

151
by Becephalus (not verified) :: Wed, 10/25/2006 - 1:51am

Nice article Travis. And I am glad everyone likes my graph idea. :) Presentation of information is very important. Thanks for making those graphs Richard.

152
by Scott P. (not verified) :: Wed, 10/25/2006 - 1:57am

These darn teams winnin' when they oughta lose & losin' when they shoulda won are messin' up your purdy stats.

Let's throw out the actual won-loss records & just use the DVOA because "Wins & losses don't tell you which team is essentially better". The games on Sunday are exciting but you don't really know who is better until the DVOA comes out on Tuesday!
(Yes, that is sarcasm.)

This wacky season has left your DVOA system in shambles, Aaron. Is it back to the drawing board?

153
by Becephalus (not verified) :: Wed, 10/25/2006 - 2:10am

Scott how does it feel to be completely ignorant? Seriously, you obviously don't understand this at all if you think something like a bad week is going to make anyone associated with DVOA start over.

Moreover how did you do against the spread last week buddy? Or even straight up picking winners? I saw the average experts stright up picking winners lat week were 4-8. I personally was 5-7 against the spread. That DVOA had a "bad week" isn't an indictment of DVOA.

IT IS a sign hat this week teams play was much less related to their past performance, which is all anyone has to work off of.

So not much to be done. And stop completely misconstruing the comments about the games. No one has is or will ever suggest the winner of the game isn't whats important vis a vis the standings etc.

BUT EVEN a complete f%#^$^@g dimwit like you has to understand the idea that in many games, not close to the majority, but a sizeable minority, the BETTER team loses. Can you honestly say that you believe the team that wins is always the team that played best?

What posesses you to come to a website you clearly don't like to criticse something you clearly don't understand? Too much free time, compensating for a poor self worth and need to bring others down, what is it with you people?

154
by mediator12 (not verified) :: Wed, 10/25/2006 - 2:20am

OMG,

1. Hey Pat, when a defense has surrendered 50% of its total points allowed due to TO's in scoring position, has allowed 2 TD's against 6 NFL opponents, and has yielded double digit scoring in one game where 12 of those points were the direct result of TO's I tend to believe the external validity of the theory is being tested.

I find it humorous that you state that DEN's offense is actually helping the defense with FP as Denver's defense has started 10/52 drives in scoring position and accounts for 50% of points allowed. Five of those drives were within five plays after a defensive TO or loss of downs which means the defense has just exited the field and has to come right back on.

Plus, they are 11th in yards per drive offense, 27th in points per drive, and 31st in TD's per drive. Meanwhile, DEN's defense is sixth in yards per drive, 1st in TD's per drive, and 1st in pts per drive. Yet the only relevant part is LOS/Dr? For whose purpose?

Also, you have again underestimated the lack of any return game in those FP battles as well. Denver has the least amount of Kickoff return yardage and the worst average return at 15.1 yards. They are in the bottom 1/4 in punt return yardage too. Remember that kicks in Denver average more distance as well? Well, Denver's coverage unit has outperformed the return unit and they have only been 18th in the league in defensive starting LOS/Dr. Funny, how you point out ST's are relevant in other arguments but not Denver's.

2. I wanted to wait and see how DEN did against CRAPPY offenses and they allowed 10 Points on two TO's the offense surrendered in scoring positions. They played Horrendous offenses and almost shut them out if not for TO's by their offense.

3. To say that Jake Plummer is playing at replacement level is to admit you have not watched many DEN games to this point. Replacement level is not what he has done even by DVOA -16.9, DPAR -2.7, or any other measure. In fact, he almost single handedly let CLE back in the game hitting two CLE Defenders in the hands on consecutive drives after he already threw an INT against CLE last weekend that allowed them to score their sole TD. The Browns had NOT been inside Denver's 42 all game until that INT.

4. The link to Aaron's blog is two weeks Old and his argument is dying a slow death. The Denver defense has not only allowed a whopping 2-14 Red zone TD's, they have failed to yield a TD from OUTSIDE the redzone. A fact that is omitted from the argument all together.

5. I agree they will not be the best DVOA defense this year because their DL is very average on their best day. Every other great defense has a dominant DL and total front seven. Injuries have already started to destroy some of those excellent units as well. Denver, unfortunately, has a great back seven that can be exposed at times due to an inconsistent pass rush. We will see how they fare against the Colts offense.

155
by NF (not verified) :: Wed, 10/25/2006 - 2:24am

Becephalus, ease up a little, at least he wasn't supporting a particular team, and was wrote reasonably well. Also, amusingly cheeky. Some of the Atlanta fan comments from last year on the other hand...

156
by Bobman (not verified) :: Wed, 10/25/2006 - 2:46am

Tunesmith,
Why is Indy so low on the beatpath chart when nobody has beaten them? Yes, they're at the top of their "stack" but lower than three teams that have all been beaten at least twice....? Is there a calculation/reason, or is it that the teams Indy beat are lower and, like water's bouyant properties in a canal lock, only lift them so far? Why not as high as Chi? Thanks.

157
by deflated (not verified) :: Wed, 10/25/2006 - 2:54am

Can't stop trying to make sense of Denver...

Anyone done any analysis on correlation between variance and scoring/drive outcomes for defence? Wondering if above average consistency combined with an above average defence is inflating the Broncos points/TDs against stats.

On a similar note I'd be interested to see an average plays or first downs per drive stat for defence. From an quick look at the drive charts for Denver's games it seems they're doing a good job of preventing big plays and playing a numbers game; if the other guy has to keep making first downs he's eventually going to take a penalty/screw up/etc.

158
by Bobman (not verified) :: Wed, 10/25/2006 - 3:03am

In the Fox commentary Aaron notes that Indy has 6 road games left. But he leaves out (not that this has much to do with DVOA) that Indy has the best road record in the NFL over the past 4 (+/-) seasons. Of course anything they did in 2004 has zero impact on the current FO calculations, but if I were a betting man, I'd look at their historic away record of 22-5 (or whatever it is) and say they'll probably go 4-2 in the road games. Maybe lose one at home and finish about 13-3. That's without looking at opponents.

Then looking at future schedule difficulty, Indy's 20 is not much worse than NE and SD at 27/28, about the same as Pit (21--and Pit already has a heap of losses), and better than Denver (14).

Despite the huge DVOA/DAVE gaps between them, it looks from this perspective that SD, NE, and IND will all be within a game of each other for HFA through the playoffs. Which seems to be supported in the estimated wins table (the range is a tight 4.6-4.9 wins from NE to SD with IND in the middle). Did I just barf the obvious all over the discussion board? If so, sorry. It's 11 pm in Seattle and I have a couple hours work to get to.

159
by Chris (not verified) :: Wed, 10/25/2006 - 3:07am

Re: 138, 144
As the author of the site, I’m open to criticism, but I feel compelled to play a little defense first. 1) The purpose of my software first and foremost is to predict playoff seedings. Not outcomes of individual games or absolute rankings of teams. 2) The playoff tiebreaker rules are coded; I don’t know of any other up to date web-stie that considers these and they are critical to determining a particular team’s playoff chances 3) The software is interactive, so you can consider your own scenarios by making assumptions about the outcomes of upcoming games, with the tiebreaker scenarios considered automatically; 4) The track record is pretty good—teams qualify for the playoffs at about the expected probability.

Now as to the power rankings that determine the game outcome probabilities, this is definitely an area that I would like to improve. For my purposes, the power rankings have to give me the probability of game outcomes, so that is why I chose a maximum likelihood method. I’ve been playing around with a regression of win likelihood vs. DVOA difference, but it is not clear that this approach is significantly better. Still need to do some more research.

I’m interested in your comment, Pat. How do I go about determining the error in each team’s rankings? I’m not a trained statistician so a little elaboration (or link) on your comment would be appreciated.

Philosophically, I recognize that there is uncertainty about a given team’s actual rating in addition to the variability of performance in a given game. But I haven’t seen any evidence that you can reliably separate the two in such a small sample size (say based on 6 games played by each team). Take for instance the DVOA variance of the Eagles (very low) versus the Chiefs (large without apparent reason). Are the Chiefs intrinsically more variable or is the apparently larger variance due to the small sample size? When one samples the performance of 32 teams each 6 times you are by chance going to sample some teams more heavily from the center of the distribution, while other teams by chance happen to be sampled more heavily in the tails. How can you discern the true variance of a distribution versus sampling variability with only six samples? At the end of the season when you have 16 samples it might be possible, but by then all the fun is over.

160
by Becephalus (not verified) :: Wed, 10/25/2006 - 3:10am

Yeah but I have seen several comments like this, and perusing the fox blog doesn't help. IDK just drives me nuts to see someone come in a be a complete idiot for no reason. Just strikes me as super crass, like going into a AA meeting after someone fell off the wagon and being like "See AA suxorz you bitches why do you even bother? Bacardi Limon is the pnwzorz!" Its insensitive to the amount of time and effort put in, and completely misses the point. That it was well written should be a minus not a plus as it is a sign the person should know better. Some of the trolls I can sympathize with as they strike me as people who probably didn't finish highschool.

161
by Bobman (not verified) :: Wed, 10/25/2006 - 3:19am

deflated,

What I think you are looking for would be dsr (drive success rate) on the drive stats page. I think that corresponds to # of plays per drive (i.e. more plays means it's sustained and therefore more successful) and to some extent 3rd down success. Probably pts too. Not sure what the correlation would be but assume it is high. In this regard, Denver's DSR is ranked #5 (and #8 in punts per drive), while their TD rate against is #1. (Part of the problem--if there is one--is that TOs are lower than I'd expect for such a standout unit.) The implication of points and DSR is that those numbers should converge to some extent over the season, no? (Note how Chi is ranked 2/1/1 in those 3 areas for D and Indy 1/1/1 and SD is 2/2/2 for offense, so they should have high correlation.) Denver's rankings are pretty close already, but if you're letting a few more drives succeed they're gonna score eventually.

Maybe this week, since they're facing the offense with the #1 ranking for DSR, TDs and Pts.

162
by Bobman (not verified) :: Wed, 10/25/2006 - 3:30am

Absolute last note regarding est wins and off-the-cuff projections for SD/NE/IND: What is not completely factored in is NE's emerging chemistry on offense (it's been getting better recently, which is reflected in the past couple games but presumably it will continue to improve, which is not reflected), IND's emerging starting RB, who had a couple games w/single-digit carries but looks to become more of a staple (same thing) plus the addition of Booger last week (only reflected in one "avg" defensive performance), and SD losing Merriman for a likley 4 games in Nov. (no way to reflect that at all, yet).

Despite SD having a much higher DVOA, I now think the AFC looks like your basic Indy/NE death match and on paper, the teams look a lot like they did a few years ago. Ah crap, did I just write that? As they say in horror movies, I don't have a good feeling about this.

163
by Becephalus (not verified) :: Wed, 10/25/2006 - 4:05am

Another IND NE deathmatch sounds just fine to me. They are highly entertaining. It will exspecially be entertaining to see the hype surrounding the comming clash of titans in the AFC championship game only to see 1 or both lose beforehand. :)

Personally I can't wai for the bears to beat the vikings in the playoffs 51-7 (just going out on a limb here for fun as a vikings fan). It would be just like them to upset some team and give us all hope before getting completely embarassed.

164
by Bobman (not verified) :: Wed, 10/25/2006 - 4:09am

Don't you need Randall Cunningham or Jeff George for that?

And you're right about the media hype machine if it looked like IND/NE were 1/2 in the AFC. I'd get to relive every sack and interception of Manning in excruciating detail all Dec/Jan. The only thing louder would be CHI's media parade if they approached 16-0 (which looks less and less likely recently).

165
by donald (not verified) :: Wed, 10/25/2006 - 5:13am

It's deplorable how Aaron continues to back his system despite its numerous failings. His plethora of supporters here make me sick. Why don't you all stop polishing his nuts? I'm sure he's a very wealthy man by now and the last thing he deserves is a bunch of lameass internet browsers sticking up for him.

Football is probably the hardest sport to predict, and you're no better at it than the random fan. Get off your high horse and admit that. I love it how every now and then a team will be ranked too high, or too low, and couple weeks later, that team,s ranking finally settles about where it should, and you say, "See, DVOA is finally showing us what we all already knew!" Well, fan-fucking-tastic! What good is it when it's two weeks behind the curve? I can't begin to explain how wrong it is that you get paid to publish this nonsense every single week, and on top of that you have hundreds of lameass internet losers slobbing all over your knob, all of whom if given the chance would probably gladly take a bullet for you. All hail the amazing Aaron Schatz!

166
by tunesmith (not verified) :: Wed, 10/25/2006 - 5:28am

#156 bobman - yeah, bouyancy is a good way to look at it, I suppose.

The power rankings are done by looking at all teams without a beatloss (currently only three) and then picking one using a tiebreaker. Then the other two teams are considered, along with any team who only had a beatloss to the team that was just picked.

The tiebreaker is the strength of beatwins.

Right now the biggest dynamic in the graph is St. Louis' win over Denver. Well, I say that subjectively, because that win just looks weird to me. If that win gets beatlooped away, then Seattle's beatwins won't look so strong anymore, and as a result, neither will Chicago's.

And this week there are actually something like three or four different ways that that beatpath segment could be beatlooped away. It'll be an interesting week for beatpaths.

167
by queequeg (not verified) :: Wed, 10/25/2006 - 8:09am

Donald is wrong. I could show him why, but that would make him think that I must answer his criticisms and inquiries. Don, I'm sorry your nuts aren't getting as much sweet rubbin' as Aaron's, but I think if you read your post you'll figure out why no one cares about whatever point you're trying to make.

168
by zip (not verified) :: Wed, 10/25/2006 - 9:46am

“See AA suxorz you bitches why do you even bother? Bacardi Limon is the pnwzorz!�

rofl.

169
by Thad (not verified) :: Wed, 10/25/2006 - 9:53am

re 165
I think baseball is harder, but thats just me.

170
by Duff Soviet Union (not verified) :: Wed, 10/25/2006 - 9:57am

Donald. Fuck off.

171
by Pat (not verified) :: Wed, 10/25/2006 - 10:04am

To say that Jake Plummer is playing at replacement level is to admit you have not watched many DEN games to this point. Replacement level is not what he has done even by DVOA -16.9

DPAR of 0, or DVOA of -13.3%, is replacement level. That's close enough for me.

I don't think you realize exactly how bad replacement level is. Most teams with a replacement level quarterback want a new one. However, there are things much, much worse than a replacement level quarterback.

I’m interested in your comment, Pat. How do I go about determining the error in each team’s rankings? I’m not a trained statistician so a little elaboration (or link) on your comment would be appreciated.

Depends how you're doing it. It certainly looks like a maximum likelihood method, right? The easiest thing to do there is do a fit of a Gaussian to the likelihood as a function of rating. Then you need to take the win probability function you've got and figure out how the error in the rating propagates to the error in the win prediction, and that'll give you the error in the win probability as well. You can then use the error in that win probability to simulate the season, rather than any artificial variation (like the normal distribution used in Baseball Prospectus's).

There are other ways to figure out the error in the rating as well - you might want to look up something called a bootstrap method, which could also be used (it'd probably be much better, as well, though it wouldn't be usable until late in the season).

Basically, depending on the restricting power of each win, certain teams will have a lot of "flex" in their ratings (specifically ones that have played crap schedules, most likely), and certain teams will be pinned down more than others. Treating the uncertainty of all of them as the same means that on average, you'll be right, but there'll be an entire class of teams which get underpredicted and overpredicted.

172
by Pat (not verified) :: Wed, 10/25/2006 - 10:17am

How can you discern the true variance of a distribution versus sampling variability with only six samples?

The simple answer is you don't really need to. What you're really interested in is "how well do we know what we know now"? Why that variance is there is unimportant, except in an esoteric way: the "are the Chiefs inherently more variable?" question. Whether or not that variance goes down with more games isn't really important, because right now, it's still there. You aren't trying to guess how your predictions are going to turn out. You're trying to guess how the season turns out. And in, say, the first week, you're just going to have a lot of error. That's just unavoidable.

The more complicated answer is that you measure it. You wait until you've got a lot of data, then determine the average variation in any given game (using, say, a bootstrap method), and then subtract that out on average.

I’ve been playing around with a regression of win likelihood vs. DVOA difference,

I doubt it would look better in any simple way. You'd almost definitely need to break it up into matchups. While the DVOA average treats them all as equal, in any given game, they don't have to be. If a team never punts, for instance, the quality of their punter and the opposing punt return team don't matter.

173
by azibuck (not verified) :: Wed, 10/25/2006 - 10:23am

There were jokes in here before?

174
by Riceloft (not verified) :: Wed, 10/25/2006 - 10:37am

A bit off topic:

Where is there an article that talks more about "teams run because they're winning, they dont win because they run". I know I've read such a beast either here or in PFP 2006. I dont have my copy of PFP 2006 handy, so if someone could point me in the right direction..

175
by Rich Conley (not verified) :: Wed, 10/25/2006 - 10:55am

re:114

Then why isnt DVOA indicating that? DVOA says their D is great, and their offense is crap. So if its really that the D is leaving the O with bad field position, then theres a problem here.

176
by Pat (not verified) :: Wed, 10/25/2006 - 11:06am

So if its really that the D is leaving the O with bad field position, then theres a problem here.

Nono, you're misunderstanding. If you look at their offense in terms of points per game, or even points per drive, they're essentially last in the league. But DVOA has their offense at 20th in the league. The reason for the difference is because of the poor field position the offense has been left in, frequently by the defense.

It's just a minor tweak. The defense isn't quite as good as it looks, and the offense isn't quite as bad as it looks. But we're not talking about a major difference here.

177
by jebmak (not verified) :: Wed, 10/25/2006 - 11:18am

That Vertical +50% chart looks really good. It shows the breaks very well.

178
by Rich Conley (not verified) :: Wed, 10/25/2006 - 11:21am

142:

An interesting thing:

While the Pats have been climbing the charts, their Defence has gone from +10% to +1.3%.

So thier defence has been playing better the last couple of games then DVOA says, theyre just still getting a penalty for all the wacky stuff that happened in the first two games.

Is there anywhere you can get the DVOA ratings for each game so far?

179
by Wanker79 (not verified) :: Wed, 10/25/2006 - 11:24am

Re: 129

I'm with you. And It'd probably be a good idea to color code the tiers (just to idiot-proof...um...sorry...I mean Fox blog-proof it a little more. (click my name)

180
by Kevin (not verified) :: Wed, 10/25/2006 - 11:29am

As a way to compare relative division strength, I've taken to summing team DVOA's to come up with a "net" division DVOA. This is what it looks like this week:

1. NFC East: 80.7%
2. AFC North: 23.0%
3. NFC North: 19.6%
4. AFC West: 14.3%
5. NFC South: 0.1%
6. AFC East: -39.5%
7. AFC South: -42.3%
8. NFC West: -43.5%

From the information, there is clear separation between the NFC East and everyone else, and between the 5th and 6th divisions on the list. The NFC North doesn't appear nearly as bad as it's made out to be, but I'm not sure how much of that has to do with the Bears being that good on their own.

181
by Will Allen (not verified) :: Wed, 10/25/2006 - 11:32am

I really shouldn't laugh at the mentally ill when they go off their meds, but gosh, Donald is hilarious!

182
by Chris (not verified) :: Wed, 10/25/2006 - 11:32am

Pat, my method is not exactly a maximum likelihood method. I calculate the game outcome probabilities as a function of power rating much like is done in the ML methods based on a normal distribution. Then my fitting procedure is to guess a set of power rankings, determine the home-team win probabilities for each completed game based on those power rankings, bin the games according to win probability, then compare the games in bins against the actual outcomes of the games using least squares. This seems to give more consistent power ratings than the maximum likelihood methods that I tried.

In the past, I’ve considered fitting Gaussians to each team’s power rating like you described. I've always been scared away by the computational demands, but I think I just thought of a more efficient way of doing it. Thanks for the feedback.

183
by Pat (not verified) :: Wed, 10/25/2006 - 11:35am

#178:

The other thing to note is that their unadjusted VOA is -7%. That's a pretty big adjustment - New England's been playing bad offenses (2x Buffalo, Miami, and Denver will do that to you). So it's not really surprising that the public perception of their defense is that it's better than it is.

The biggest adjustment is to their pass defense, though: from 3.9% to 17.8%.

184
by Rich Conley (not verified) :: Wed, 10/25/2006 - 11:45am

178

Right, the question is, how much of that is from all the flukey crap in the first 2 games, and how predictive is it?

The whole Jerico Cothery matrix thing certainly isnt an indicator (unless Jericho is the One), but that Coles run later in the game may be telling us something.

Pat, just to make things clear, I dont think the Pats defence has been playing great, I just really dont think that the current rating is predictive. I think theyre much better than theyre showing up as, and that trending up backs me up a little.

I really want to see the individual game DVOA/VOA scores.

185
by Pat (not verified) :: Wed, 10/25/2006 - 11:50am

Pat, my method is not exactly a maximum likelihood method.

For the most part, that is a maximum likelihood method. You're just using a slightly different maximization method, that's all. If you want to see a radically different approach, look up Laplace's method.

By "based on a normal distribution" you mean "based on a logistic function", right? That's kindof what I was stating earlier regarding DVOA. The power ratings given by a maximum likelihood method are, in fact, defined by the logistic function. That's their basic property: a rating of "78.4" doesn't mean anything, other than the meaning that the logistic function gives it.

DVOA, on the other hand, is a measurement. There's no reason to believe a logistic function determines the win probabilities. You could always *measure* the win probabilities based off of DVOA, though, using past performances. It'd probably be like a 3 or 4 dimensional problem, though. Not pretty.

I’ve always been scared away by the computational demands

The ROOT analysis package (root.cern.ch) would make short work of something like that. Then again, the demands required to actually learn ROOT might be higher. But the graphs would be pretty. :)

186
by Chris (not verified) :: Wed, 10/25/2006 - 12:16pm

Pat, no I don’t use the logistic function. The win probabilities are given by:

normCum((home strength – away strength + home field advantage)/variance)

where normCum is the normal cumulative distribution function. It has the same shape as a logistic function. My procedure optimizes the power ratings to match the win probabilities of the games already played. The assumption is that these power ratings will reasonably predict the win probabilities of future games. And yes, there is some wiggle in the parameters which could be accounted for by considering each rating to be a distribution, instead of a single value.

187
by Pat (not verified) :: Wed, 10/25/2006 - 12:18pm

#184: A lot of the Pats' bad defensive DVOA comes not from the Jets game, but from the Denver game (not surprising, considering they gave up 36 yards/drive to a team that normally has 29 yards/drive). Their defensive VOA went from 4.5% to 14% the week of the Denver game, and that was before opponent adjustments got ahold of that game.

188
by billvv (not verified) :: Wed, 10/25/2006 - 12:19pm

Donald, calm down man. This is a site for math/football nerds who only argue about the numbers which purport to prove that their devotion to their team is warranted. Note that only four teams are involved in the discussion and all other teams are not. This should tell you all you need to know. Nobody here is full of admiration for team play in general only whether the numbers confirm their view. Relax, it's actually interesting, although the consistently improving Jets are never mentioned, except by me and #7, but we're ignored.

189
by Wanker79 (not verified) :: Wed, 10/25/2006 - 12:26pm

Wow, billvv, you mean the teams with the most discrepancey between public opinion and the FO stats are the teams that everyone is talking about?!? Why color me shocked and amazed.

190
by Pat (not verified) :: Wed, 10/25/2006 - 12:34pm

where normCum is the normal cumulative distribution function. It has the same shape as a logistic function.

Not quite the same shape. But the difference between the logistic distribution and the normal distribution is pretty small. If memory serves, though, I think there's a paper which explains why the logistic function is typically used, though I doubt it'd apply for football. I actually meant a sigmoid function in general, though.

My procedure optimizes the power ratings to match the win probabilities of the games already played.

Wait, I'm confused. So basically you're making sure that all teams that are ranked, say, 0.6-0.7, when playing teams that are ranked 0.4-0.5, win with the probability given by the given function? (etc., using least squares) It seems like that would allow for a lot of play early in the season given sample size issues. Are you doing the least-squares fit with counting error?

191
by Wanker79 (not verified) :: Wed, 10/25/2006 - 12:34pm

Re: 180

I’m not sure how much of that has to do with the Bears being that good on their own.

The way I like to look at the relative strength of the divisions is by taking out the outlier (the team with the biggest distance, high or low, from the 4-team average) and then averaging the remaining 3 teams. The way I see it, a division with one ridiculously good team and 3 mediocre-to-poor teams is a weaker division overall than a division with 3 good-to-mediocre teams and 1 ridiculously bad team. So doing that gives:

NFCE (-Was): 28.10%
AFCW (-Oak): 17.80%
AFCN (-Cle): 14.40%
NFCS (-TB): 5.13%
AFCS (-Ten): -0.43%
NFCN (-Chi): -8.47%
AFCE (-NE): -18.10%
NFCW (-StL): -18.43%

192
by Moridin (not verified) :: Wed, 10/25/2006 - 12:36pm

re: 179
Wanker79, that's basically the graph I was thinking of when it was first mentioned. Thanks for taking the time (however much) to put it together.

193
by Pat (not verified) :: Wed, 10/25/2006 - 12:46pm

#191: You could also try the central mean, where you weight the center teams more than the outliers. So add the top and bottom teams, then add the two middle teams and multiply them by 2, then divide by 6.

194
by Disco Stu (not verified) :: Wed, 10/25/2006 - 1:03pm

I like the chart centered around zero, whether vertical or horizontal- it really hammers the A in DVOA. Cuz after all, when a DVOA is -34%, that's really saying they're far below average, so the chart should reflect that.

195
by Chris (not verified) :: Wed, 10/25/2006 - 1:08pm

Pat, game probabilities (calculated from power ratings) go into the bins, not team power ratings themselves. To counteract the problem with too little data at the beginning of the season, I wait until week 5 to do the power ratings and weight the ratings with preseason ratings up until week 9. Counting error weighting might be another thought, but I think including the error in the ratings estimates is really going to help.

196
by Crushinator (not verified) :: Wed, 10/25/2006 - 1:10pm

Didn't read all the comments, so I don't know if this was suggested, but -

There might be a better way to show the teams so your average reader can see how closely packed together some teams are. I'm pretty sure your average fan reading it just looks at it and goes "wtf! Eagles rated over Giants, Giants are winning the division!"

The problem with a 1-32 list is just that it forces absolutes, even if numbers 15-20 for example are all ridiculously close.

197
by B (not verified) :: Wed, 10/25/2006 - 1:15pm

I like the idea of doing it graphically, athough it would take up the space used for comments now. I picture a series of horizantal bars, sorted from longest to shortest in team colors with the helmet logos.

198
by chris clark (not verified) :: Wed, 10/25/2006 - 1:32pm

re 188: This is a site for math/football nerds who only argue about the numbers which purport to prove that their devotion to their team is warranted.

So, as a long term Broncos fan, I'm really concerned about what DVOA says about that team (and as a nerd, I think DVOA might just be right and the pundits wrong). Reading Aaron's comments, his opinion is that the Broncos are actually quite mediocre since they haven't had STOMPS against some teams they should have if they weren't mediocre. Moroever, DVOA as opposed to DAVE points that direction, ranking them 15th (i.e. near the median) rather than 10th.

So, if one assumes that rock-paper-scissors holds (as one poster pointed out it may not statistically) and IND beats DEN next Sunday, then I presume that DEN will fall substantially in the rankings (i.e. out of the top 10 and near to the 15th slot DVOA currently gives it), unless they play better in the game than their current DVOA predicts.

Yes?

199
by Bobman (not verified) :: Wed, 10/25/2006 - 1:53pm

chris clark,
That sounds about right. Of course Indy has left a few STOMPS on the table as well (NYJ, Tenn) and so they're in a very similar boat regarding ranking. Of course if Den loses, they will be losing to a decent team, not, say, AZ. (sorry, LMAO for a second there). It still depends on the individual plays, however. If Indy and Den both fumble 4 times and Indy recovers all 8 and that leads to the win, it would not necessarily affect things too much for ranking purposes because the fumble recoveries are largely random and luck-based. They'd both get dinged for fumbles, but Indy would not get credit for the recovery (nor would Den get hit for non-recovery). The win is not necessarly the thing to affect DVOA/ranking... the individual plays are.

Which is why some teams outperform others and still lose. Which would you rather have that day? The win, of course. But which would you rather have long-term? The better team which is usually the better performer.

That help at all?

Oh, and it's my turn to polish Aaron's nuts this afternoon, but I misplaced my 4.5-inch angle grinder with that chamois buffer pad he likes so much... anybody got a spare?

200
by Wanker79 (not verified) :: Wed, 10/25/2006 - 1:58pm

Re: 198

Well Chris, considering that the difference between Denver and Indy isn't all that big, I'd assume that as long at each team plays respectably that the DVOA won't really change much. Even if Denver loses (as long as they play reasonably well) they could even wind up gaining some ground since Indy is the higher ranked team. The other by-product of them being relatively even is that a blowout in either direction would probably really hurt the loser and help the winner (as opposed to a far superior team blowing out a weaker team).

201
by Wanker79 (not verified) :: Wed, 10/25/2006 - 2:00pm

Curse you, Bobman. *shakes fist of fury*

202
by Matthew Furtek (not verified) :: Wed, 10/25/2006 - 2:35pm

Don't people get that these rankings ignore record and focus on "How well the team is playing..."?

203
by Crushinator (not verified) :: Wed, 10/25/2006 - 2:48pm

202

No, they don't.

People only see the ranking on a list and get pissed off when it varies from what they think.

I doubt many even read the comments or the giant intro paragraph.

It's true of all lists, from power rankings to VH1's "top 100 of all time"

204
by Kal (not verified) :: Wed, 10/25/2006 - 2:55pm

I don't know how well Indy will do against Denver, even with their historical domination in the playoffs (which is, after all, all of two years of history). Denver's running game is still as strong as ever, and Indy's running defense looked better but...that was only one game. I'm more suspicious of Indy than I am of Denver at this point.

205
by MJK (not verified) :: Wed, 10/25/2006 - 3:06pm

Suggested name for graphical DVOA:

Graphical Representation of the Effectiveness of All Teams

So we get to actually see how GREAT the teams are.

206
by Luz (not verified) :: Wed, 10/25/2006 - 3:10pm

i'm way late to mention this but... big fan of the graph idea.

207
by Tim (not verified) :: Wed, 10/25/2006 - 3:13pm

RE #14: I can't believe a Steelers fan and a Bengals fan actual agree on something! Yes, the Steelers are still a good team. But I can't understand why the DVOA formula doesn't punish the Steelers and Eagles (et al.) more for giving up turnovers...especially for consistently giving up turnovers that make/break games.

The rankings for the Carolina Steve Smiths seem a bit suspect, and just how do the Titans drop two spots during a bye week after their first win (and a strong showing against the Colts)? I love the DVOA, but something seems wacky this week.

Cincinnai is clearly ranked too low because they beat the Steelers and the Chiefs. Tuesday Morning Quarterback is way better than this. wHO dEy think gonna beeet dem bENgaLS?!?!!!!!1111one

208
by DWL (not verified) :: Wed, 10/25/2006 - 3:24pm

Tim,

DVOA doesn't have to penalize the Steelers for turnovers, won/loss does that quite well thanks.

209
by Pat (not verified) :: Wed, 10/25/2006 - 3:27pm

But I can’t understand why the DVOA formula doesn’t punish the Steelers and Eagles (et al.) more for giving up turnovers…especially for consistently giving up turnovers that make/break games.

Because it's not predictive. Teams don't just choose to give up turnovers only when it's bad. They just happen, with very poor timing (or very good timing by the opposing team).

Any game that can be entirely decided by one play is essentially a crapshoot. If they win, we say they gutted it out. If they lose, we say they didn't have that killer instinct. But that's just crazy. The difference between a team that loses on one play and a team that wins on one play is basically nothing.

210
by jebmak (not verified) :: Wed, 10/25/2006 - 3:37pm

Re#189
You reminded me of my economics shirt from my college days.
"Economists do it graphically"

The pharmacy students got the best shirt though.
"Drugs are my life"

211
by jebmak (not verified) :: Wed, 10/25/2006 - 3:39pm

Rather Re: #197

212
by Ray (not verified) :: Wed, 10/25/2006 - 3:46pm

RE: 205 MJK
"So we get to actually see how GREAT the teams are."

Awesome. ;^)

213
by Wanker79 (not verified) :: Wed, 10/25/2006 - 4:08pm

Re: 207

DVOA does punish teams for interceptions and fumbles, but it treats fumble recovery and the return yardage and TDs resultant from fumbles and interceptions as random events. This is because those things are largely a product of right-place-right-time kind of luck. If a fumble happens to bounce just right so that a defender has a clean recovery on the run, that is not a reflection of the quality of the team that fumbled.

So while the Pittsburgh gets credit for causing 13 INTs/Fumbles and penalized for committing 18 INTs/Fumbles, they don't get penalized for only recovering 6 of 15 fumbles. Likewise, Philly gets credit for causing 23 INTs/Fumbles and penalized for committing 17 INTs/Fumbles, but they aren't penalized for only recovering 9 of 27 fumbles. Also, Pittsburgh doesn't get credit for the 1 turnover they returned for a TD. Likewise, Philly doesn't get credit for the 2 turnovers they've returned for TDs but they also don't get penalized for the 4 turnovers their opponents have returned for TDs.

214
by deflated (not verified) :: Wed, 10/25/2006 - 5:52pm

Re. 161: Bobman, I don't think DSR is doing what I'm after.

I guess the question is whether an 80 yd drive in 12 plays is 'more successful' than an 80 yd drive taking 6 plays. From a defensive point of view I'd consider the 12 play drive more successful; every play is another shot at a TO and the offence isn't scoring (the main objective). Neither stacks up against 3 and out but intuitively if you force them to throw twice the passes you increase the chances of an int. which should be better.

215
by B (not verified) :: Wed, 10/25/2006 - 6:08pm

214: From a defensive point of view, the 12 play drive ss worse be because it tires out the opposing defense and takes more time off the clock, leaving your offense less oppurtunity to score.

216
by chris clark (not verified) :: Wed, 10/25/2006 - 6:09pm

re 202:

yes, but how well a team is playing is usually (maybe not in PHI's case, but still usually) correlated with record. Thus, unless your team plays very well, but loses because of a fluke (or plays poorly and wins by a fluke), the record and the DVOA will eventually converge. If DVOA didn't converge to the record over time, it wouldn't be a good measure.

217
by Pat (not verified) :: Wed, 10/25/2006 - 6:26pm

Right. It's the converse of Atlanta's situation last year, and we're very near that time of the year again. Atlanta was 6-2 last year, and everyone was crowing that they couldn't possibly be ranked near teams at 3-5 or 4-4. Time goes on, things even out, and Atlanta ends the season at 8-8.

Philly is more likely to go 8-1 or 7-2 for the rest of the season than 5-4. If I had to guess, they'll go to Indy and beat them, and suddenly everyone will be talking about how the team is "putting it all together".

218
by Jack Neefus (not verified) :: Wed, 10/25/2006 - 6:31pm

I suspect that one reason the Giants ranked #4 is that their rushing game this year has resulted in a lot of runs consistently gaining 7 to 10 yards with few breakaways for long yardage. Under the way that DVOA scores success, those intermediate-length plays score extremely high and lead to a higher offensive rank. Similarly, the system rewards West Coast dink-and-dunk passing offenses over the deep passing game. As a comparison, two weeks ago Atlanta had a 90-yard run which impacts the DVOA much less than ten 9-yard runs. Not to agree or disagree with the system, but it seems to be the result of a play-by-play analysis rather than drive- or game-based statistics.

219
by Fnor (not verified) :: Wed, 10/25/2006 - 6:34pm

The long drive is statistically better for the defence because there is a higher chance that the defence will stop the offence on one series that takes place in that drive. More series = more chances to stop the offence. I believe there was an article about this in PFP 2005.

Count another vote for GREAT. You can subtitle it "Wonderful Statistical Learnings For The Betterment Of The People Of The FOX Websites."

220
by B (not verified) :: Wed, 10/25/2006 - 6:35pm

216: Doesn't that assume that W/L records are the true measure of a team? I could come up with many examples of two teams where the better team didn't have the better win/loss record. The most obvious example of this would be the 2001 Bears.

221
by Peter (not verified) :: Wed, 10/25/2006 - 7:42pm

220: Ultimately wins and losses are the only things that matter. The point of DVOA is to describe why and how teams win and lose. Your examples are unimportant unless they are really extremely large numbers; the point is that IN GENERAL, bad teams lose, good teams win. If your statistical measure is NOT based on winning or losing, what use does it have, if winning is the only goal?

222
by B (not verified) :: Wed, 10/25/2006 - 8:36pm

The point of DVOA, or any ranking tool, is to figure out which team is better, and therefore which team is more likely to win the next game.

223
by cnelsonw (not verified) :: Wed, 10/25/2006 - 9:11pm

RE: 165

Donald - all that complaining and yet here you are reading it. I'll bet it's not your first time either. : )

224
by Pat (not verified) :: Wed, 10/25/2006 - 9:25pm

Similarly, the system rewards West Coast dink-and-dunk passing offenses over the deep passing game.

Why does everyone say this? It's not true. Heck, McNabb rated extremely high in DPAR in the Cowboys game which was nothing but deep passes. The difference is that if the deep passing isn't succeeding, yah, it'll be penalized. But so would short passes as well.

Ditto with the running game bit. The whole "short runs are better" thing isn't true. Longer runs are great. If they're accompanied by a crapload of runs straight into the line that lead to tons of 3-and-outs, yeah, it sucks.

225
by Pat (not verified) :: Wed, 10/25/2006 - 9:26pm

the point is that IN GENERAL, bad teams lose, good teams win.

What happens when two good teams play each other?

226
by jetsgrumbler (not verified) :: Wed, 10/25/2006 - 10:26pm

forgive me for not knowing the details of the forumlas, but how much do you discount one game blowouts as anomalies?

for example, if you discount the jacksonville game, i think the JETS skyrocket, statistically speaking. although they haven't beaten anyone good, they played well against NE and Indy, and the losses showed them to be a better team than they appeared in the jacksonville slaughter.

also, i hope the DVOA and DAVE stats are really accurate. JETS drafted almost exclusively for offense and special teams this year and look at their ranks in those departments. i can only hope that a defensively focused draft next year will yield the same results.

227
by calig23 (not verified) :: Wed, 10/25/2006 - 10:32pm

Re:#225

The gooder team beats the badder team?

228
by Pat (not verified) :: Thu, 10/26/2006 - 12:04am

for example, if you discount the jacksonville game, i think the JETS skyrocket, statistically speaking.

Uh, it says, right in the commentary, exactly how much the Jets skyrocket. Quoth I:

Without the Jacksonville blowout, the Jets would be at the bottom of the pack of good teams, instead of down here with the losers.

So up to about 21st. Still, you can't ignore the Jaguars game. It did happen. They've clearly got the potential to have the crap beaten out of them.

229
by Pat (not verified) :: Thu, 10/26/2006 - 12:06am

The gooder team beats the badder team?

Or is it the gooder team beats the good team? I'm always confused about that one. What happens when two bad teams play each other, too? What about when the team that wins the bad-badder game plays the team that loses the gooder-good game? Is that still okay? :)

230
by Jim (not verified) :: Thu, 10/26/2006 - 8:18am

Where on the site is there information about red zone performance?

Thanks.

231
by chris clark (not verified) :: Thu, 10/26/2006 - 12:47pm

re 220:

Winning the SB is what matters. WL records, DVOA, power rankings, whatever, are just tools to know what chances any given team has of being in the position to win the SB (or to get the top draft pick). And part of the reason I like this site is not only the DVOA, but the analysis Aaron puts on what those stats mean, like at the end of last year when he predicted how certain teams (e.g. the Steelers iirc) were actually better than their DVOA suggested.

That, of course, concerns me when I see the Broncos not only 10th in DAVE but 15th in DVOA and Aaron writing that maybe they aren't as good as those numbers suggest. Moreover, that fear is accentuated when I read that the stats are actually middle-heavy, with little difference in the rankings of the middle teams. If you read that carefully, you can easily learn that it is quite potential that the Broncos are a mediocre team in a parity driven NFL where, luck is more predominant a factor in who wins a game than team skill.

Especially, if one lives in hostile territory (I live in NE), where your favorite team is considered anathema, and merely an inconvenient road-block to crowning the local team with the assumed god-status. The last part has little to do with DVOA other than explaning why it is important for me to understand its implications....

232
by Rick (not verified) :: Thu, 10/26/2006 - 2:05pm

Let's see...I said (#59)

"Last week, Philly was rated third. They lost to an inferior Tampa team and climbed to 2nd."

Aaron's response was#67

Hmmm.

PHI last week: 39.9%
PHI this week: 39.2%

Maybe I failed introductory math, but that looks like down to me. Not a lot of down, but definitely not up.

Well, Aaron, apparently your "introductory math" never taught you the difference between ordinal numbers and real numbers. I don't give a crap what the actual value of the DVOA is, what was bizarre to me was that its placement among the ratings of all the teams had climbed to second highest. And I think that's pretty clear from the comment I made. I didn't say "their DVOA increased". I said "climbed to second". I'm pretty sure your smart ass comment was aimed at me since nobody else at that point in the thread was talking about Philly's move up in the DVOA rankings.

Your whole attitude here seems to be "It's my system, it works, and if you don't like it, you need to show me why", but your pointers do not lead to a complete explanation of what the system actually is and your response to criticism is snarky and juvenile.

If you want to make snarky comments about math, you might want to first presume that the person you are talking to has a Ph.D. in said subject (as I do) and that perhaps the problem is your own reading comprehension.

So, tell me again. How does Philly lose a game and climb in the ordinal rankings? How did Pittsburgh do the same thing? The only explanations I have seen thus far seem to be based on blaming turnovers for the problem, as if turnovers ought to be randomly distributed and any team that kept coughing up the ball and losing the possession was somehow just "unlucky". Offhand it seems to me that turnovers are Very Bad things and should not be lightly shrugged off. Turnovers correlate highly with losing and teams that repeatedly give up turnovers are going to lose, even if their play for the other 95% of the plays is as good (or sometimes even better) than that of their opponent.

Something important to keep in mind with any mathematical model is that it is just a model, and it makes little sense to become too dogmatic about it. Any model that starts to fail ought to be revised: that is the first lesson of mathematical modeling. Snarky responses based on misreading complaints is amateurish.

233
by Riceloft (not verified) :: Thu, 10/26/2006 - 2:18pm

Clearly they went up because the #2 team's DVOA went down as well, only further than the Eagles'. Also, no team lower than the Eagles played well enough to push them ahead. San Diego also lost, and apparently, according to DVOA, they played worse than the Eagles.

The "ordinal" ranking is based 100% on DVOA/DAVE, so answering the way he did, though fairly ass-like, was enough that should've answered your question.

Chi - 45.0
Phi - 39.2
SD - 34.0
NYG - 33.3
Bal - 18.2

234
by B (not verified) :: Thu, 10/26/2006 - 2:36pm

232: The answer to your question is because SD played worse than Philly did.

235
by stan (not verified) :: Thu, 10/26/2006 - 2:53pm

132,

Aaron -- I understand treating PI as basically the same as a completion. And it has the beauty of giving you the same yardage. Was the problem with including illegal chuck or def holding that the yardage penalty (plus first down) didn't seem to match up as well?

236
by Pat (not verified) :: Thu, 10/26/2006 - 3:08pm

I don’t give a crap what the actual value of the DVOA is

Hence, the problem.

237
by Pat (not verified) :: Thu, 10/26/2006 - 3:17pm

Something important to keep in mind with any mathematical model is that it is just a model,

Are yards a model?

I hate it when people say DVOA is a model. It's not a model. It's a measurement. Sagarin's ratings (on USA Today) are a model - the numbers themselves don't mean anything without the model for winning percentage.

DVOA, though, does. A team with a 30% offensive DVOA scores 30% more points, on average, than an average team, against an average opponent. That's what it means. It's more useful than points because it's subject to less variation.

The only "model" involved is that it makes sense to rank teams based on their production throughout the whole game, rather than just the results of the game. If you ranked teams by (yards gained/game)-(yards allowed/game), Philly wouldn't've changed much after that game, either.

238
by Stillio (not verified) :: Thu, 10/26/2006 - 3:17pm

I have analyzed all the data extensively and come to the following conclusion about this week's Jax/Phi game: Phi will score 21 points and Jax will score either 35 or 7. Hope that helps in handicapping the game...it's really not much fun being a fan of the most inconsistent team in the NFL. /sigh

239
by Wanker79 (not verified) :: Thu, 10/26/2006 - 3:45pm

Rick,

If you're so damned good with numbers, why in the name of God is it so hard for you to understand that the ordinal rankings means very little. I find it hard to believe that anyone with a high school diploma would have trouble understanding that, let alone a PhD in mathematics.

Fumbles may not be randomly occurring, but they sure as hell are randomly recovered. IIRC, teams are in fact punished for fumbling and rewarded for causing fumbles. But the actual act of recovering a fumble has more to do with the football just happening to bounce in your direction than anything else. Go back and read post #213. Philadelphia has only recovered 9 of 27 total fumbles. Do you actually believe that says something about the quality of the team? It sure as hell hurts the team’s chances of winning, but it doesn’t say anything about the ability of the team to win.

Why is it so hard for people to recognize that W/L tells you very little about that actual quality of a team's play?!? Philly and Pittsburgh lost, so they must not be very good. Or maybe you should consider that Philly lost on a 62-yard FG in the waning seconds of the game, and Pittsburgh lost on a FG on the opening drive of overtime.

Philly (in a loss) and Chicago (on a bye) both moved up a spot, and that spot was formally held by San Diego. Do you really think that SD losing to KC in a game that KC, for the most part controlled, says SD is a better team than either of them??? Pittsburgh (in a loss) and Baltimore (on a bye) both moved up 2 spots, and those spots were vacated by Dallas and Jacksonville. Can you really not see why the way Dallas played NY and the way Jacksonville played Houston caused Pittsburgh and Baltimore to move up???
I just don't understand why people can't see the difference between a team's W/L record and their play-by-play value. If that's just too difficult for you, why don't you just go to ESPN's NFL page, click on Standings and then set Breakdown By League. That outta make you feel better. Aaron's response may have seemed snarky, but I can't really blame him since he's constantly being attacked by people who just don't get it.

240
by Wanker79 (not verified) :: Thu, 10/26/2006 - 3:48pm

Re: 238

And you think being a fan of the team playing the most inconsistent team in the league is any better?

241
by Stillio (not verified) :: Thu, 10/26/2006 - 4:37pm

@240: Nope, but Iggles fans only have to deal with the Jags this week, while I have to worry and wonder for at least 10 more weeks. I won't bore people with a long analysis of what's what with the Jags, but they are one of the youngest teams in the league as is and now suffering significant injuries. I have absolutely no idea which team will show up in Philly this week, or against the Titans next week, or the Texans after that or...

242
by chris clark (not verified) :: Thu, 10/26/2006 - 4:39pm

re: 237

I hate to nitpick, especially on someone normally so rational, but....

Yes, DVOA is a measurement, but the choice of what to measure and how to sum the measurements is a model, so DVOA is also a model. Your statement that 30% offensive DVOA teams scores 30% more points on average than an average team is a statement about the predictive power of that model, because offensive DVOA is not simply a measure of points scored. and that statement is based upon the past performance of the model. That is the model was designed so that in past games, the measurements and sums incorporated into DVOA predicted the scores of the actual games played. and, now comes the traditional disclaimer about past performance not guaranteeing future results.

BTW, any system of measurements that predicts future performance is by definition a model. Because, it is a model which incorporates the rules about how those measurements are expected to be predictive. If DVOA actually measured future performances, rather than predicting it, it wouldn't be a model, it would be something much more amazing....

Now, DVOA is a very good model, because it has been tested and improved. For example, the way DVOA treats fumbles as random is done because Aaron (or someone) did analysis and showed that treating fumbles as random improved the predictive power of DVOA as a model. Now, in actuality fumble recoveries are not truly random, they occur because the right person is at the right place, but from a predictive point of view, it is best to treat them as random.

The same thing might apply to red zone defense. There may be factors involved that are predictive, or it may be that red zone defensive strength is not a maintainable attribute. I think that's why someone asked about 3rd down performance, as I recall, that is predictive.

243
by Martial (not verified) :: Thu, 10/26/2006 - 5:10pm

Speaking of graphical representation of DVOA, there are alwasy sparklines.

Sparkline of Week #7 DVOA. Check out the groupings of teams.

I used the Sparkline Generator Web App at Bitworking .

244
by Wanker79 (not verified) :: Thu, 10/26/2006 - 5:12pm

In actuality fumble recoveries are not truly random, they occur because the right person is at the right place

Did you just say that fumble recoveries are not truly random because they are random? ;-)

245
by Pat (not verified) :: Thu, 10/26/2006 - 5:42pm

Yes, DVOA is a measurement, but the choice of what to measure and how to sum the measurements is a model

Philly's DVOA went from 39.9% to 39.2%. There's no model involved there. It's a measurement. The belief that Philly didn't decline significantly against Tampa Bay because it was still producing at about as high a level as it previously had is a model.

It's the same thing as pointing out "yeah, Philly lost. They also outgained the Buccaneers by 300 yards."

BTW, any system of measurements that predicts future performance is by definition a model.

DVOA doesn't predict future performance. It measures past performance. It correlates to future performance better than other measurements. The DVOA preseason predictions are a model.

Models, by definition, are simplified versions of the processes they're trying to explain. I'll agree with you in some sense that it's a model - but only in the same sense that "yards" are a model, or a play-by-play itself is a model, in that it doesn't contain all the information about the game.

246
by Pat (not verified) :: Thu, 10/26/2006 - 6:17pm

Incidentally, DVOA isn't the only place where Philly moved up after losing: in the Aikman Efficiency Ratings, they moved from 6 to 5, and in the NFL's rankings (combined offense/defense) they moved from 12th to 4th. Fluke losses happen. Sometimes they even happen two, three times in a season.

247
by chris clark (not verified) :: Thu, 10/26/2006 - 6:52pm

re 244:

If I did, that's not what I meant to say. It is possible that there are ways to improve fumble recovery percentages. Some teams may even exploit them--I remember at one time reading about some teams having fumble recovery drills. However, because they are not universally exploited (if they are exploited at all) and teams with better fumble recovery precentages tend to regress to the mean. Thus, it is best to model them as random. Stock market motions aren't random either, but for many purposes it is best to treat them as random also (for example, that is an argument for a diversified portfolio).

248
by chris clark (not verified) :: Thu, 10/26/2006 - 7:26pm

re 245:

I don't want to belabor this and am just willing to admit that we simply disagree on terms. However, any system of measurements is a model. If there was only one thing being measured by DVOA, it would simply be a measurement. However, things are summed and weighted and so forth. Those weightings and choices of what to sum are "the model".
They are what simplify the series of games down to one number (or one set of numbers DVOA, offensive, defensive, ST, DAVE, ...).

However, I agree with your fundamental point, the decline in PHI's number is simply a measurement. It's not until someone interprets what that measurement means that we have something significant.

Which side of the interpretation we put the term "model" on is not important. I just put it differently than you do, and I believe I put it on the same place where Rick does (not that I really want to be associated with him). The only difference is that I'm not as critical of the model as he is. I believe the model is working (as well as it has ever worked).

However, it is just a model. Aaron has played around and tuned the model as well as he can, but it is still a model, a simplification. Reducing it to ordinals (as Rick does and I suspect many of us do, can't wait for GREAT) is an even further simplification, and can cause some foolish unnecessary emotional traumas. So, if PHI being 2nd causes one emotional stress, perhaps one shouldn't be reading this site.

I get the same stress from DEN being 10th (or 15th), but it doesn't make me reject the model. It makes me worry that I'm in for some rough Sundays, because perhaps the model is right (and maybe if I'm lucky the model is wrong).

249
by chris clark (not verified) :: Thu, 10/26/2006 - 7:28pm

re: 244 just caught the smiley and saw your previous post where you said right-person in right-place = random. *sigh*

250
by B (not verified) :: Thu, 10/26/2006 - 7:42pm

248: If a person's favorite team's position on this or any ranking is causing them personal trauma, they have problems that can't be solved by adjusting a formula.

251
by Pat (not verified) :: Thu, 10/26/2006 - 10:25pm

Those weightings and choices of what to sum are “the model�.

But there isn't a choice. The weightings weren't chosen. They were determined. There was no simplification of the dataset there, or assumption about the behavior of the game. Just an optimized measurement. It's the same as someone saying "net yards per play correlate with wins, therefore, I'm going to rank teams by net yards per play."

If you're wondering what I'm comparing it to, I'm comparing it to something like William Krasker's dynamic programming model, which is a model of how the game behaves. The model's based on reality in some sense, but it's still a simplification of the game designed to reproduce results. That's a model.

Or, likewise, Sagarin's ratings are a model. They model football by saying that football is equivalent to a game effectively determined by the flip of a weighted coin, where the weight is determined by the true strengths of two teams. The rankings are then determined to maximize the outcome probabilities.

They are what simplify the series of games down to one number

That's my entire point. The idea that the average DVOA of every play in the season is a measure by which the true strengths of teams can be measured is the model. Not the values itself. DVOA is no more a model than using yards themselves.

But by virtually every fair metric in football, Philly is far better than their record indicates. At some point, you have to just say they are a good team, and the record is what's a fluke.

Of course, the fact that games can be decided by fluke (and hell, they're decided purely by chance in many overtime games) is what makes the game interesting.

However, any system of measurements is a model.

I don't see how you can say that. A model is a simplification of a system. If someone used DVOA to predict wins and losses, by saying "team with higher DVOA always wins", that's a model, because they're trying to replace the game with a simplified version of it.

But part of the reason why I'm getting so defensive is that there's no claim as to what a game with a high total DVOA versus a low total DVOA implies - DVOA's real strength is being able to break things down into matchups. No one's claiming it's a model for football. It's just a measurement.

252
by CaffeineMan (not verified) :: Thu, 10/26/2006 - 11:05pm

"It's only a model!"
"Shh!"

253
by thad (not verified) :: Fri, 10/27/2006 - 12:06am

Pat, excluding dvoa, what would you consider a fair metric, and how would you define a fair metric?

254
by Pat (not verified) :: Fri, 10/27/2006 - 11:25am

The first question there is what a metric is. A metric is a gauge - a standard of measurement. To be honest, when I say "DVOA is a measurement" what I should say is DVOA is a metric, not a model. It's a way of measuring the quality of teams.

By fair, what I mean is unbiased. Lack of bias is really fairly crucial in a measurement. Uncertainty is fine - your brain is usually pretty good at picking out fuzz in a measurement - but bias is a killer.

Points - probably points/drive, to normalize against pace - are a pretty fair metric, although not really for offense and defense separately (can't score less than zero points, and folds special teams into either side randomly). Other than that... there aren't really any at all. Most of them blatantly ignore several large portions of football: the Aikman ratings specifically ignore special teams, for instance.

The problem with points is the utterly gigantic quantization in them. If you take a look at the preseason article I wrote, you can see what I mean by the ridiculous spread present. Winning by 3 isn't significantly different than winning by 7 - in fact, teams will usually abandon trying to win by 7 to win by 3, because it's safer. See the New Orleans-Philly game. But that doesn't make points an unfair metric. It just makes it a crappy one.

Things that aren't a fair metric include things like yards, and yards/play, both of which are totally situationally biased, and neither of which includes special teams in any way, shape, or form.

Things that aren't a metric at all are things like Sagarin ratings. You know this automatically because you can't compare Sagarin ratings from year-to-year in the tiniest bit. But that's fine - they're not intended to be metrics.

Note that I don't actually believe DVOA is a completely unbiased metric, but from everything Aaron has ever said, that's his goal.

255
by Wanker79 (not verified) :: Fri, 10/27/2006 - 12:53pm

Re: 193

You could also try the central mean, where you weight the center teams more than the outliers. So add the top and bottom teams, then add the two middle teams and multiply them by 2, then divide by 6.

Pat, I worked out your central mean idea. Here are the results:

NFCE: 20.97%
AFCN: 8.00%
AFCW: 5.62%
NFCN: 2.25%
NFCS: 0.70%
AFCS: -9.62%
NFCW: -12.15%
AFCE: -12.28%

Taking your idea in a slightly different direction, if you feel that the strength of a division is it's best teams, you could weight the division averages so that they're top heavy (2*(1st+2nd)+3rd+4th)/6

That give you:
NFCE: 25.53%
NFCN: 12.37%
AFCN: 9.75%
AFCW: 9.73%
NFCS: 2.57%
AFCS: -2.40%
NFCW: -6.13%
AFCE: -6.92%

So to sum-up, the only things consistent through the three rankings (minus the outliers, central mean, top heavy) are that the NFCE is the best division. The AFCN and AFCW are 2 of the top 4 with very little difference between them. The NFCS is better than the AFCS. The NFCW and AFCE are the 2 worst divisions with almost no difference between them (less than a single percentage in each case). And the NFCN is completely dominated by how much weight you give Chicago (surprise!).

I don't really know if any of this actually means anything, but it was a fun little tangent (for me anyway).

256
by chris clark (not verified) :: Fri, 10/27/2006 - 1:21pm

re 254:
Ok, I'll call it a metric. The metric was determined to attempt to best correlate with team "quality". That's a subjective term. More precisely, the metric was determined to best correlate with "something". What is that something? I'm not sure how to Google to find the one line description of what that something is, although I'm sure Aaron has written down exactly what DVOA correlates to. Presumably it is something else which has been measured.
(Eeeks, I'm having these nightmares that I read the DVOA was specifically designed to correlate with DVOA, i.e. past with present. Tell me it isn't so.)

If the DVOA metric doesn't correspond to your image of quality, then stop using it, Rick. If you want to change the definition of DVOA, then do the statistical work to come up with a different metric that correlates better, and then you have a chance of arguing your point.

Now, I will try to shut up. My apologies to everyone.

257
by Wanker79 (not verified) :: Fri, 10/27/2006 - 1:42pm

Re: 256

I'm almost positive that DVOA correlates to past points scored. I'm not positive about that (it could be wins I guess).

258
by Pat (not verified) :: Fri, 10/27/2006 - 2:06pm

What is that something? I’m not sure how to Google to find the one line description of what that something is, although I’m sure Aaron has written down exactly what DVOA correlates to. Presumably it is something else which has been measured.

It's in the FAQ. VOA correlates to team wins. DVOA (which includes stuff like penalizing fumbles equally, adjusting for opponent field goal kicking and kickoff length) correlates year-to-year, so yeah, DVOA is tuned to correlate with DVOA (but only if it doesn't lower VOA correlating to wins significantly).

So if you ranked teams by VOA, that's really the same as ranking them by points/game+points allowed/game, but with less fluctuation (though Philly still looks wacky-screwy there, but that's life this year with Philly). Ranking them by DVOA is correcting for things outside of a team's control (schedule, fumble recovery, opponent kickoff length, and opponent field goal accuracy) that are not likely to continue in the future. And that's really the key point there.

259
by AlexDL (not verified) :: Fri, 10/27/2006 - 7:15pm

How about this for a way to integrate the graph into the DVOA. I think it's GREAT.

260
by AlexDL (not verified) :: Fri, 10/27/2006 - 7:19pm

I'll try again....click on name for new graph.

261
by AlexDL (not verified) :: Fri, 10/27/2006 - 7:20pm

One more time. Hope this works.

262
by DavidH (not verified) :: Sat, 10/28/2006 - 4:23pm

I like the way it looks, but if it keeps me from cutting and pasting the data into Excel, then it is EVIL.

263
by chris clark (not verified) :: Sat, 10/28/2006 - 5:19pm

re 261:

Thanks for the great graphic. It looks different than I expected it to given the previous descriptions. Yes, the two large gaps are there, but to me it looks more like a cubic (or tangent) curve. The extremes are exaggerated and the middle is flattened, with the exception of the gap CIN and TB, where it is steeper again. Curiously, it looks if you draw a line between CHI and TEN, it crosses over right at the middle near DEN/CIN. It will be interesting to compare the graphs as we have them for different weeks.

264
by Newbie Greg (not verified) :: Sun, 10/29/2006 - 4:14am

About fumble recoveries - are they truly random? What is the average fumble recovery rate of an offensive team? Are there some teams that do better or worse at recovering fumbles year after year?

Aaron - have you done any analysis on the types of plays during which fumbles occur, or the place on the field in which they occur (either absolute or relative to the line of scrimmage)? I'm sure there are other factors I haven't thought about (maybe within 5 yds of the sidelines are safer for the offense?) Perhaps some of these statistics aren't available, but if they are, it's worth considering. I'd hypothesize that sack fumbles are recovered more by the offense than receiver fumbles downfield as the former is likely to have more offensive players around the ball than the latter. But I have no data to back this up - just wondering.

265
by Trevor (not verified) :: Mon, 10/30/2006 - 5:20pm

If Rick has a PhD in mathematics, I must be Stephen Hawking...

My seventh grade sister can understand how someone can move up in ordered rankings while still having their data by which the rankings are ordered decrease.

Maybe you should read (and comprehend, while that might be a little difficult for you) a full explaination of DVOA before you go ragging it...

266
by Rick (not verified) :: Tue, 10/31/2006 - 9:21am

Hi Trevor,

Hope that disease is coming well for you.

I can understand what happened with the difference between ordinal rankings and underlying DVOA scores. What my question was about was why it happened. Aaron misread my math question (deliberately in my opinion) and tried to take a cheap shot. I didn't appreciate it.

Why does ordinal ranking matter? It doesn't really, but it was what my question was about. Deliberately misreading a question is childish. Also, the NFL is a competitive league, and it really begs the question how a team can repeately lose and be ranked highly in any ordinal system. Whether this ordinal system is implied from numerical ratings or not, it is a simpler statistic to watch.

In any case, these ratings have been flawed for weeks now, as we've been repeatedly told that the Steelers are much better in reality than their performance on the scoreboard would indicate. The proof is in the pudding, and the Steelers just lost to the mighty Raiders.

As for a "full explanation" of DVOA - it isn't there to be found. There's a lot of chatty explanation of what the intent of DVOA is supposed to be, but there isn't any formula online. It's either being treated as too complicated for lay people or as propietary information. In either case, I think it's entirely appropriate to tweak people putting forth numbers in an authoritative manner when those numbers repeatedly misrepresent the reality.

At the end of the day, the DVOA rankings have been diverging radically from rankings given by every major football watcher out there. And it's really reaching a near-dogmatic level here. What happens when 2-4 Pittsburgh is rated higher than 6-0 Indy? Rather than stating the obvious, that the system isn't doing exactly what one would hope for it to do, Aaron gets his back up and stands by the system.

So now Indy is 7-0 and Pittsburgh is 2-5. Maybe (I hope), FINALLY this week DVOA will notice that Indy has been playing better than Pittsburgh all season.

267
by Wanker79 (not verified) :: Tue, 10/31/2006 - 4:35pm

Rick,

Your original question wasn't misread, it was treated as idiotic. There a big difference there. Why in the world would anyone take a question like that seriously when the answer would be painfully obvious even to a high schooler?? And if you really think that the NFL being a competitive league means that the better team always wins, than this conversation should just end right here because your view and that of every single person who supports DVOA are so divergent that reasoning is just not humanly possible.

Half of this discussion thread has been about how much it would help the Fox readers to understand what the numbers are actually saying by including a graphical representation. There are (were) 4 teams way above everyone else, a few teams way below everyone else, and then a whole bunch of teams in the middle (Pittsburgh included) packed pretty close together. Pittsburgh has had the 6th most difficult schedule so far. They've convincingly won one game, completely dominate another, lost 2 games by a single single score (one of which they should have won, Cincy; and another on the first possesion of overtime), lose one game where they gained 3 times as many yards as their opponent, played another game to a virtual stalemate, and lose one game to arguably the best team in the league. How in the hell can you look at that and complain about a system that is based on per play performance rating them as a middle-of-the-pack kinda team?!?

The whole purpose of the FO stats is to ignore conventional wisdom. I see the fact that they've "been diverging radically from rankings given by every major football watcher out there" as a positive. If I just wanted to read knee-jerk reactions to the w/l records, I'd go read one of the countless idiot "experts" like Greg Easterbrook or Peter King. And maybe that's what you should do.

268
by Jean Crawford (not verified) :: Sun, 11/05/2006 - 3:01pm

How dare you bash the Broncos for every thing that you don't like about Colorado. I thought that this was supposed to be a professional NFL rankings site. You are definately not a professional sportscaster, or you would stick to the facts of the games only. Who cares how you feel about Colorado and Denver, rate the teams on their playing, not on the state's political views, or the colleges or anything else that you don't like about the state. Aaron Schatz....you are a moron.