Writers of Pro Football Prospectus 2008

Most Recent FO Features

RiceSid07.jpg

» Sidney Rice: What Could Have Been?

Sidney Rice has retired. Is he the most random single-season DYAR leader ever? One-year wonder? Injury prone? We offer a career retrospective for the second-best wide receiver named Rice in NFL history.

20 Oct 2009

Week 6 DVOA Ratings

by Aaron Schatz

New Orleans is now the clear number one atop this week's Football Outsiders DVOA ratings. That's no surprise after the Saints dismantled the previously unbeaten New York Giants on Sunday, 48-27. The Saints look ready to cruise to a first-round bye -- they are also the most consistent team in the league by VARIANCE, and their future schedule ranks 30th in the NFL.

The team that ranks second behind New Orleans is a bit more of a surprise -- or not, if you're a long-time Football Outsiders reader. Yes, the Philadelphia Eagles are still number two despite their embarrassing loss to Oakland this past weekend. The explanation for this is pretty simple, to be honest: Their three wins have been huge. They came against very bad teams, but DVOA won't fully penalize them until opponent adjustments are at full strength in Week 10. (We don't do full opponent adjustments now to make up for the small sample size that comes with fewer games. Who knows -- to give an example, maybe Carolina is better than everyone thinks.) Meanwhile, the Eagles two losses have been close by DVOA standards. They only lost to Oakland by four points, so their DVOA for that game is currently -12.0%. They lost to New Orleans by a big score, 48-22, but of course that's a loss to the best team in the league, so DVOA moves the rating up a bit. There were also three fumbles in that game that were all recovered by the Saints, so overall Philly's DVOA for that game is currently listed as a positive 2.3%.

Anyway, enough of that longtime Football Outsiders hobby, apologizing for our Eagles stats. Let's move on to another of our longtime hobbies, trying to figure out where our preseason projections went wrong. We could look at early performance and wonder about flukes, but at this point it is pretty clear that Denver is far better than we projected and San Diego is far worse. Obviously, San Diego has had some injury troubles, but they don't have enough injuries to represent the difference between a historically strong projection and ranking 24th in DVOA through five games. That leads to the question: What could the projection system have missed? We're talking about objective numbers here; our projections weren't based on a desire to criticize Josh McDaniels or an irrational love of Antonio Cromartie. The trends all still make sense looking back -- the Broncos brought in a quarterback with lower career numbers, the Broncos had a historically bad defense a year ago, old secondaries often struggle, San Diego had a lot of defensive injuries last year, and so forth. So what did we miss? At this point, I'd actually like to hear suggestions from the audience. We're looking for objective facts that we knew before the season that should have been indicators that Denver's defense would make a colossal turnaround, or that San Diego wouldn't be the best team in the league. They need to be suggestions that would be useful when looking at teams in general, trends that we can see with other teams from the past that significantly improved or declined from one year to the next. For example, "the Broncos hired Mike Nolan" wouldn't work, because as I discussed last week, there's no consistent history of well-regarded veteran coordinators significantly improving bad defenses. I also don't think "Norv Turner sucks" works, because Norv Turner's poor coaching is already reflected in San Diego's ratings from the last two seasons.

Most readers know that I often don't have the time to read through all the comments in the discussion threads, but I will look through this week and see if people have offered some serious suggestions that could help make the projection system more accurate in future years.

Finally, you may have been wondering where the Patriots' 59-0 shellacking of the Titans fits in on the list of the best DVOA games of all-time. The surprising answer: It doesn't. New England's DVOA rating for this week's win was "only" 117.5%. That's definitely the best single-game rating so far this year, but it doesn't make the list of the top ten DVOA games. That could change by the end of the year, but the Titans would have to get their act together enough to significantly change the opponent adjustments. (The list of the ten best DVOA games is found here.) The main reason why the Patriots don't make the list is that they didn't just take their foot off the gas in the fourth quarter -- they pulled the emergency brake and let the second-string offense skid all over in the snow. In their final two drives, the Patriots fell short on two third downs and a fourth down, and added a false start just for fun. Obviously, it doesn't matter when you are winning by 59, but DVOA does count every play, although at reduced strength because of the score. Here is New England's offensive DVOA by quarter on Sunday:

  • Q1: 47.4%
  • Q2: 116.5%
  • Q3: 44.2%
  • Q4: -67.5%

Fumbles were also a reason why the Patriots did not set a DVOA record -- not fumble recovery, but the fumbles themselves. The Titans fumbled six times, but three of those were aborted snaps; in the new DVOA introduced last July, the defense no longer gets any credit for "causing" a bad snap.

Of course, the Titans still get penalized for blowing those snaps, and so while the Patriots don't land on the list of the ten best single-game DVOA ratings, the Titans do land on the list of the ten worst single-game DVOA ratings:


Worst Single-Game DVOA Ratings, 1994-2009
Year Team DVOA Week vs. Score Opp DVOA
Rank for Year
2002 ARI -169.2% 13 KC 49-0 4
2005 SF -166.0% 7 WAS 52-17 7
2005 SF -146.8% 2 PHI 42-3 18
2000 CLE -165.2% 14 JAC 48-0 16
2007 KC -157.1% 14 DEN 41-7 17
1999 CLE -153.6% 1 PIT 43-0 19
2009 TEN -153.0% 6 NE 59-0 4
2003 ARI -144.5% 11 CLE 44-6 22
2000 ARI -142.7% 8 DAL 48-7 24
2008 STL -137.5% 10 NYJ 47-3 17


* * * * *

Housekeeping notes: We've finally had a chance to update the "DVOA as of Week X" numbers in the Premium database to reflect the new version of DVOA we introduced in July. All of the week-to-week numbers have been redone, so you can see what DVOA would have looked like in Week 8 of 1994 or Week 12 of 2002 or whenever you would like. In addition, the game-by-game DVOA page now features not only single-game offensive and defensive DVOA but also both offense and defense split into rushing and passing. If you want to know the rushing DVOA of the Bengals when Corey Dillon broke the all-time record with 278 rushing yards, you can (Week 8 of 2000, if you want to look it up). All Premium DVOA stats are now updated through Week 6 of 2009, as are all individual stats and team stats pages, plus the playoff odds page (with a few new "special Super Bowl" listings).

The other housekeeping note: I got myself on the Twitter. There's an "official" FO address, fb_outsiders, but that's actually Bill Barnwell talking. I'm now on at FO_ASchatz. I'm planning to mostly use it to follow NFL reporters rather than telling you what I'm eating for lunch or whatever, but feel free to follow my feed as well as Bill's, and use it to ask questions which I may or may not have time to answer. At some point we'll get up a list of all the FO writers with public twitter pages so you can follow whoever you would like.

* * * * *

These are the Football Outsiders team efficiency ratings through six weeks of 2009, measured by our proprietary Defense-adjusted Value Over Average (DVOA) system that breaks down every single play and compares a team's performance to the league average based on situation in order to determine value over average. (Explained further here.)

OFFENSE and DEFENSE DVOA are adjusted to consider all fumbles, kept or lost, as equal value. SPECIAL TEAMS DVOA is adjusted for type of stadium (warm, cold, dome, Denver) and week of season. Because it is early in the season, opponent adjustments are currently at 60 percent strength.

As always, positive numbers represent more points so DEFENSE is better when it is NEGATIVE.

DAVE is a formula which combines our preseason projection with current DVOA to get a more accurate forecast of how a team will play the rest of the season. Right now, the preseason projection makes up 19 percent of DAVE for teams with six games and 27 percent of DAVE for teams with five games.

To save people some time, please use the following format for all complaints: <team> is clearly ranked <too high/too low> because <reason unrelated to DVOA>. <subjective ranking system> is way better than this. <unrelated team-supporting or -denigrating comment, preferably with poor spelling and/or chat-acceptable spelling>


TEAM TOTAL
DVOA
LAST
WEEK
TOTAL
DAVE
RANK W-L OFFENSE
DVOA
OFF.
RANK
DEFENSE
DVOA
DEF.
RANK
S.T.
DVOA
S.T.
RANK
1 NO 54.4% 1 37.6% 2 5-0 41.0% 1 -19.4% 2 -6.0% 29
2 PHI 44.2% 2 33.6% 4 3-2 13.2% 13 -26.7% 1 4.3% 5
3 IND 41.2% 4 39.6% 1 5-0 35.3% 2 -7.0% 9 -1.1% 19
4 NE 34.8% 9 33.7% 3 4-2 34.1% 3 -1.5% 15 -0.8% 18
5 NYG 33.5% 3 30.1% 5 5-1 21.4% 7 -11.0% 6 1.2% 11
6 DEN 33.1% 7 22.5% 8 6-0 17.1% 10 -18.8% 3 -2.8% 24
7 MIN 29.7% 5 27.6% 6 6-0 19.5% 8 -1.7% 14 8.5% 2
8 GB 28.9% 8 19.6% 9 3-2 13.4% 12 -15.8% 4 -0.3% 15
9 BAL 27.6% 6 25.3% 7 3-3 24.6% 6 -6.3% 11 -3.3% 25
10 ATL 19.9% 10 10.3% 11 4-1 18.3% 9 2.2% 17 3.8% 6
11 PIT 14.1% 12 15.8% 10 4-2 27.1% 5 7.1% 21 -6.0% 28
12 DAL 12.5% 11 9.0% 12 3-2 27.2% 4 13.9% 25 -0.7% 17
13 ARI 8.3% 22 0.7% 16 3-2 2.5% 18 -7.3% 8 -1.6% 21
14 MIA 3.3% 17 -2.7% 18 2-3 13.5% 11 8.9% 22 -1.3% 20
15 JAC 2.7% 15 5.8% 13 3-3 13.1% 14 11.8% 23 1.4% 10
16 NYJ 0.3% 13 -3.6% 19 3-3 -13.6% 20 -11.2% 5 2.7% 8
TEAM TOTAL
DVOA
LAST
WEEK
TOTAL
DAVE
RANK W-L OFFENSE
DVOA
OFF.
RANK
DEFENSE
DVOA
DEF.
RANK
S.T.
DVOA
S.T.
RANK
17 SEA -3.1% 14 -0.4% 17 2-4 -3.1% 19 0.8% 16 0.7% 12
18 HOU -4.4% 20 -5.0% 20 3-3 8.7% 15 16.1% 26 3.0% 7
19 CHI -5.2% 16 2.3% 15 3-2 -16.4% 25 -3.1% 13 8.1% 3
20 CIN -6.3% 18 -6.3% 21 4-2 5.8% 17 5.8% 19 -6.3% 31
21 SF -7.3% 21 -11.4% 23 3-2 -15.0% 24 -10.3% 7 -2.5% 23
22 WAS -8.8% 19 -8.6% 22 2-4 -14.8% 23 -6.1% 12 -0.1% 14
23 BUF -10.4% 24 -14.0% 24 2-4 -17.5% 26 -6.9% 10 0.2% 13
24 SD -10.5% 23 2.8% 14 2-3 7.0% 16 19.2% 28 1.7% 9
25 KC -22.3% 26 -18.4% 25 1-5 -14.5% 22 13.6% 24 5.8% 4
26 CLE -30.7% 25 -26.7% 27 1-5 -26.1% 31 18.3% 27 13.8% 1
27 CAR -34.9% 29 -25.1% 26 2-3 -22.6% 29 6.1% 20 -6.2% 30
28 STL -41.3% 30 -34.3% 28 0-6 -18.4% 27 21.1% 29 -1.7% 22
29 TB -41.3% 28 -35.8% 30 0-6 -13.9% 21 27.1% 32 -0.4% 16
30 TEN -46.4% 27 -34.5% 29 0-6 -18.5% 28 21.3% 30 -6.6% 32
31 OAK -47.3% 32 -40.9% 31 2-4 -38.9% 32 5.1% 18 -3.3% 26
32 DET -54.3% 31 -45.6% 32 1-5 -25.9% 30 22.7% 31 -5.7% 27

  • ESTIMATED WINS uses a statistic known as "Forest Index" that emphasizes consistency as well as DVOA in the most important specific situations: red zone defense, first quarter offense, and performance in the second half when the score is close. It then projects a number of wins adjusted to a league-average schedule and a league-average rate of recovering fumbles. Teams that have had their bye week are projected as if they had played one game per week.
  • PAST SCHEDULE lists average DVOA of opponents played this season, ranked from hardest schedule (#1, most positive) to easiest schedule (#32, most negative). It is not adjusted for which games are home or road.
  • FUTURE SCHEDULE lists average DVOA of opponents still left to play this season, ranked from hardest schedule (#1, most positive) to easiest schedule (#32, most negative). It is not adjusted for which games are home or road.
  • VARIANCE measures the statistical variance of the team's weekly DVOA performance. Teams are ranked from most consistent (#1, lowest variance) to least consistent (#32, highest variance).


TEAM TOTAL
DVOA
W-L NON-ADJ
TOT VOA
ESTIM.
WINS
RANK PAST
SCHED
RANK FUTURE
SCHED
RANK VAR. RANK
1 NO 54.4% 5-0 57.9% 6.0 1 2.2% 17 -11.2% 30 2.2% 1
2 PHI 44.2% 3-2 55.2% 4.3 5 -15.2% 30 10.5% 6 23.1% 29
3 IND 41.2% 5-0 47.7% 4.7 2 -5.9% 23 -1.5% 19 16.2% 20
4 NE 34.8% 4-2 32.4% 4.2 7 4.0% 14 1.4% 12 21.0% 26
5 NYG 33.5% 5-1 33.9% 4.3 6 -8.8% 26 13.8% 5 11.9% 16
6 DEN 33.1% 6-0 39.1% 4.6 4 -7.9% 25 5.0% 8 5.0% 9
7 MIN 29.7% 6-0 37.7% 4.6 3 -12.8% 27 -2.4% 21 3.1% 3
8 GB 28.9% 3-2 35.7% 4.1 9 -12.9% 28 -5.0% 26 18.4% 21
9 BAL 27.6% 3-3 31.8% 4.2 8 -0.9% 20 -1.2% 18 12.4% 18
10 ATL 19.9% 4-1 28.8% 4.0 10 -1.5% 21 6.3% 7 4.6% 6
11 PIT 14.1% 4-2 30.5% 3.8 12 -25.6% 31 4.4% 9 4.2% 5
12 DAL 12.5% 3-2 15.3% 3.9 11 -5.3% 22 14.7% 4 3.9% 4
13 ARI 8.3% 3-2 9.3% 3.2 14 4.9% 12 -14.2% 32 35.9% 32
14 MIA 3.3% 2-3 2.8% 2.8 18 6.8% 8 0.4% 14 18.9% 22
15 JAC 2.7% 3-3 1.8% 3.4 13 -7.6% 24 -4.2% 25 22.3% 28
16 NYJ 0.3% 3-3 0.8% 3.0 15 5.2% 10 -3.8% 24 14.1% 19
TEAM TOTAL
DVOA
W-L NON-ADJ
TOT VOA
ESTIM.
WINS
RANK PAST
SCHED
RANK FUTURE
SCHED
RANK VAR. RANK
17 SEA -3.1% 2-4 -6.0% 2.4 24 -0.2% 19 -11.6% 31 24.7% 30
18 HOU -4.4% 3-3 8.1% 3.0 16 -14.8% 29 1.5% 11 8.0% 12
19 CHI -5.2% 3-2 1.0% 2.8 19 0.9% 18 2.9% 10 8.1% 13
20 CIN -6.3% 4-2 -4.6% 3.0 17 11.4% 5 -9.8% 29 9.3% 14
21 SF -7.3% 3-2 -15.3% 2.7 21 2.3% 16 -2.9% 22 12.4% 17
22 WAS -8.8% 2-4 6.3% 2.6 23 -26.8% 32 19.7% 1 4.9% 8
23 BUF -10.4% 2-4 -15.2% 2.7 22 3.5% 15 -0.6% 16 19.3% 23
24 SD -10.5% 2-3 -14.8% 2.7 20 5.1% 11 -6.1% 27 2.6% 2
25 KC -22.3% 1-5 -24.4% 1.8 25 10.3% 6 -3.3% 23 4.6% 7
26 CLE -30.7% 1-5 -39.7% 1.6 26 14.6% 3 -7.2% 28 20.6% 25
27 CAR -34.9% 2-3 -43.1% 1.3 28 4.4% 13 18.7% 2 19.7% 24
28 STL -41.3% 0-6 -46.2% 1.4 27 7.0% 7 -0.8% 17 5.6% 10
29 TB -41.3% 0-6 -37.6% 0.9 32 6.0% 9 17.8% 3 7.9% 11
30 TEN -46.4% 0-6 -56.8% 1.1 29 14.8% 2 -2.1% 20 35.0% 31
31 OAK -47.3% 2-4 -48.9% 0.9 30 12.3% 4 0.9% 13 21.9% 27
32 DET -54.3% 1-5 -64.9% 0.9 31 18.9% 1 0.1% 15 9.9% 15

Posted by: Aaron Schatz on 20 Oct 2009

306 comments, Last at 15 Nov 2009, 6:22pm by Slinkster

Comments

1
by Will Allen (not verified) :: Tue, 10/20/2009 - 6:12pm

The Vikings have made a huge error if they were not working the phones, trying to swap real draft value, for a servicable cornerback, before today's trade deadline.

It really is weird how the Eagles, year after year, have a win/loss record that is worse than what one would expect from their DVOA.

I think the Denver defensive turnaround is nearly a completely unforseeable event, and if there is some pundit somewhere who did predict it, I would say that picking the winning Powerball number also doesn't mean much.

11
by Fan in Exile :: Tue, 10/20/2009 - 6:24pm

take a look through the Mile High Report archives they've got some really in depth analysis about why the team was going to be better.

43
by bravehoptoad :: Tue, 10/20/2009 - 7:21pm

I feel like they more have reasons about why the Broncos won't be the worst team in the league, and might even qualify for decent. Nothing I've seen from preseason was predicting anything like their actual performance.

Man, that was a pretty second half last night.

98
by Piglet (not verified) :: Tue, 10/20/2009 - 10:34pm

My guess would be that tweaking DVOA to try to capture what is, based on the extreme off-season turbulence and changes, pretty much just an unpredictable outlier, is unlikely to be productive -- it would probably just make things worse on average. Unless you were a Broncos fan for the last couple of years, it would be hard to have understood just how turbulent the offseason was and how wholesale the changes have been, nor would it be easy to project that the catastrophic train wreck that was the Denver D last year could be fixed so quickly. Not the kind of thing that comes up very often -- this was about the most messed up offseason imaginable from this fan's perspective.

There was a good article on Mile High Report right before the season that predicted the Broncos at 11-5 with some apparently-sound reasoning as to why. I was persuaded by this analysis, except I thought the brutal schedule would do them in and they'd finish closer to 9-7 or 8-8. It's worth a read to those interested to see the kind of thinking that went into one (blessedly accurate ;) prediction.

http://www.milehighreport.com/2009/9/8/1017306/shallow-thoughts-nearsigh...

Ultimately, I'd chalk this whole issue up to the limitations of objective statistical analysis for the NFL, and an illustration that subjective, well-informed scouting is another important tool for predicting football performance. What DVOA does do is, to me, pretty amazing though.

105
by Pat (filler) (not verified) :: Tue, 10/20/2009 - 11:05pm

It's also important to realize that attempts to fix a massive train wreck by wholesale replacing huge amounts of the team often completely fail. How many times have the Lions cleaned house now? Or the 49ers?

203
by Paulo Sanchotene, RS, Brazil (not verified) :: Wed, 10/21/2009 - 2:53pm

That was completely my point. It's possible to do everything by the book and sink, or do everything wrong and still manage to succeed.

How can one predict that a defense with EIGHT new players will work?! That the players will understand each other, complete each other, and play together as a team? Normally, they won't.

On the other hand, club presidents, GMs and HCs, doing their job correctly, can only manage to make the odds being on their favor, but it's impossible to eliminate the chance of failing.

Maybe the problem wasn't on FO predictions, but it's a matter of luck, chance or providence.

106
by Bobman :: Tue, 10/20/2009 - 11:07pm

While I agree with the sentiment of "don't tweak DVOA to capture extreme outliers" because it's counter productive, I DO think they should keep trying to make it better. It seems like every year or so they add a refinement or two.

The Eagles, though, I just can't explain it aside from those polaroids Tanier has of Aaron at a barnyard social with some very alluring lambies of the evening....

49
by Anonymous Jones :: Tue, 10/20/2009 - 7:38pm

I too find the Eagles situation really weird. It may just be coincidence. DVOA is still mildly opaque to me, but I do see one possible explanation -- Reid's passing offense is consistently good at generating successful plays but is also consistently poor when generating the plays that really matter (i.e., the conversions when the game is *really* on the line). I don't totally believe this explanation, but if there is a non-coincidence reason, this one seems as good as any.

As for San Diego and Denver, here are a list of cliches that might or might not be helpful (probably not...).

Cliche #1 "Football is a team game." What I mean in this context is that it is difficult to break down all the components of an offense and a defense, especially considering all the necessary information loss inherent in statistics, and then build the pieces back up. Over half of Denver's defensive starters are new. I consider it highly unlikely that anyone could accurately predict how well they would play together as a unit, especially with a new coordinator and new scheme. Comparing them to other teams that faced similar circumstances (new starters, new coordinator) just seems not very valuable to me. The circumstances are just too unique to be tackled in this manner. Yes, maybe they were more likely to fail, but large outliers can never be precluded. Also, the multivariable system of any offensive or defensive unit is probably highly chaotic and just like with the weather, small changes in certain interacting variables might cause huge deviations in ultimate result.

Cliche #2 "You are only as good as your weakest link." Regarding SD, you take away Jamal (one of the best run stoppers in the league) and you replace him with Nobody X (one of the worst), and you are not just a little worse, you are a lot worse. The fact that there are "injuries" (or even the quantity of the injuries) is not necessarily as important as the magnitude of the impact for any given injury. In watching the games, it seems as if SD had no feasible backup plan if Jamal went down. Just because other teams have lost Pro Bowl D-lineman and not collapsed does not mean that SD could lose Jamal and not collapse.

Cliche #3 "Norv Turner sucks." I don't totally believe this one either, but I'm also not convinced that it is totally inaccurate.

57
by Pat (filler) (not verified) :: Tue, 10/20/2009 - 8:13pm

2008 PHI Estimated Wins: 11.7 actual wins: 9.5 sched 1.8%
2007 PHI Estimated Wins: 10.2 actual wins: 8 sched 3.8%
2006 PHI Estimated Wins: 11.9 actual wins: 10 sched 0.8%
2005 PHI Estimated Wins: 7.7 actual wins: 6 sched 5.2%
2004 PHI Estimated Wins: 11.4 actual wins: 13 (*) sched -5.2%
2003 PHI Estimated Wins: 10.8 actual wins: 12 sched -0.7%
2002 PHI Estimated Wins: 10.5 actual wins: 12 sched -5.8%
2001 PHI Estimated Wins: 11.4 actual wins: 11 sched 0.1%
2000 PHI Estimated Wins: 10.9 actual wins: 11 sched -7.0%

I don't think that the Eagles are consistently ranked worse or better than they're supposed to be. When they win fewer games than we would expect, they've had a worse schedule than average. When they win more games than we would expect, they've had an easier schedule than average. Note that estimated wins is *not* adjusted for strength of schedule - it's average wins vs. an average schedule.

They might seem high right now, but 1) they're not that high in estimated wins, where they would have been expected to win 3.6 games and they actually won 3 ((4.3/6)*5) and 2) it's early in the season (cue obligatory "how could the 14-2 2004 Patriots lose to the 4-12 2004 Dolphins" for those who want to say 'but they lost to the Raiders').

If they keep playing this bipolar throughout the year, they'll end up somewhere around 8-8, and have around the same number of estimated wins. If they play more like they did vs. the Raiders, their DVOA will drop. If they play more like they did vs. the Panthers/Chiefs/Bucs, their variance will drop, DVOA will stay the same, and the fact that they lost to the Raiders will be a curiosity of the season.

(n.b.: the fact that I'm an Eagles fan is unimportant to this analysis - trust me, right now, if there was a way I could show that the Eagles are crap, I'd love to do it.)

188
by tuluse :: Wed, 10/21/2009 - 12:29pm

Clearly losing TO has affected Philly's ability to win in ways DVOA doesn't measure :)

207
by Might as well be Micheal Lombardi (not verified) :: Wed, 10/21/2009 - 3:47pm

The problem with the "weakest link" theory is that we've seen it fail time and time again. Look at the Pats last year. 11-5? Without Brady? Or the year they won the Super Bowl with Troy Brown playing nickel?

I think football is MUCH harder to quantify than a game like baseball (I know, you know this; just sayin'). The Pats are not 59 points better than the Titans, especially since the Titans probably would have beaten the Pats 59-0 if they had played them in week 2 of 2008.

And maybe that's the variable. Football teams change throughout the season. There are 16 mini-seasons, and each team is quite different from week to week. What does a preseason projection have to say about who is injured, who is playing hurt, who has something to prove? The Pats will let down next week against the Bucs and the Giants are going to absolutely murder the Cards this week, right? Don't we all know that, despite the stats?

Simmons had his third-year-of-a-bad-coach theory, and that might be sound. Look at how the Chargers and Cowboys are underachieving.

2
by Joseph :: Tue, 10/20/2009 - 6:12pm

If the Saints play close to this level, aren't they favorites for the SB? Sure, special teams could blow them a playoff game, but they are SOOOO dominant that they have averaged 1.2 est. wins for each game. :) No, it's because they whooped up on "bye week." :p
My hope (as a Saints' fan) is that they don't pull a 2007 Pats--dominant 1st half of the season, keep winning in the second half (I really don't care about 16-0) and then choke in the SB after looking so dominant in their first 18 games. I'd much rather 18-1/17-2 as long as it ends in the longest party NO has ever seen.

65
by JasonK :: Tue, 10/20/2009 - 8:27pm

You know, the Ravens did sign David Tyree recently...

3
by young curmudgeon (not verified) :: Tue, 10/20/2009 - 6:12pm

Why don't you just throw in the towel and include an "Eagles adjustment" to avoid the seemingly ever-present embarrassment? Were you able to keep a straight face while typing "they only lost to Oakland by four points"? That's kind of like Mark Twain's account of his fight with a stronger man, "Thrusting my nose firmly between his teeth, I threw him to the ground on top of me." (paraphrased from memory) In fact, that would be a good thread: "saying 'they only lost to Oakland by four points' is like saying..." Any takers?

DVOA is a great tool, but any metric that identifies Philadelphia as the second most successful team so far this year still needs a little tinkering.

32
by billsfan :: Tue, 10/20/2009 - 6:46pm

It was easy enough to keep a straight face last season when the DVOA-Overrated Eagles made the playoffs on a confluence of improbabilities and still managed to make it back to the NFC championship game. Those howling about the loss to the Ravens and the tie with the Bengals quickly and conveneintly forgot the stomping of the eventual champion Steelers. The problem is that the Eagles just tend to do well all the things necessary to have a high DVOA.

There's a reason for that old saying about "any given Sunday." Football games are not won and lost by average performances, and with a 16-game season, three freak occurrences can be enough to turn a 10-6 playoff team into a 7-9 also-ran. Unless I'm way off-base here, DVOA, or any other "efficiency" metric, doesn't take into account "big-play" tendency, which can often win or lose a football game. A key play in that game was a huge catch-and-run by Zach Miller with *two* huge down-field blocks by Louis Murphy. No statistical model can predict that. Akers also missed two field goals, which would have made the difference in a four-point loss. Fortunately, Jim Mora isn't in charge in Philly. IIRC, previous FO research has shown that FG% fluctuates somewhat wildly from season to season, even for one kicker, largely due to small sample size. If the Eagles played the Raiders 10,000 times this season, I'm fairly certain that they'd win more than half of them. Furthermore, despite having a DVOA that frequently exceeds their record, the Eagles miraculously end up in the playoff hunt almost every year. Funny how that works out. I'm sure when the NFL inevitably moves to a 10,000-game season, DVOA will be a hell of a lot more accurate.

There was an interesting, and I think related, series of posts at PFR and Smart Football this summer about evaluating rushing performance. IIRC, it turns out that good and bad running backs all have about the same median YPC. What separates the good RBs is the frequency of longer runs. I don't know if, and to what extent, "big plays" are considered in DVOA, but it would be interesting to see how their inclusion would affect ratings (or just break the whole damn model).

As for the Eagles adjustment, KUBIAK used to feature a "Peyton Manning" adjustment, since their fantasy projections consistently put McNabb ahead of Manning. Enough rambling from me.

(I also like the Eagles)

52
by Scott C :: Tue, 10/20/2009 - 7:46pm

What part of

"the opponent adjustments aren't factored in yet" did you not understand?

Also, for several years in a row the Eagles were rated what appeared to be too low, consistently. The trend the last few years has been the opposite.

Small sample size?

63
by Pat (filler) (not verified) :: Tue, 10/20/2009 - 8:26pm

"Small sample size?"

Nah. Crappy division -> great division. Your eyes don't opponent adjust very well.

60
by Pat (filler) (not verified) :: Tue, 10/20/2009 - 8:25pm

"Were you able to keep a straight face while typing "they only lost to Oakland by four points"?"

The 14-2 2004 Super Bowl champions only lost to a 4-12 team by 1 point.
The 13-3 2003 Super Bowl champions lost to a 6-10 team by 31 points and a 5-11 team by 3 points.
The 12-4 2006 Super Bowl champions only lost to a 6-10 team by 3 points.

It happens.

87
by chemical burn :: Tue, 10/20/2009 - 9:26pm

Do you really think this Raiders team will get to 6-10 or even 5-11? Or does the difference between 3-13 and 6-10 not really matter?

I just don't know what to think anymore. I feel like I understand DVOA well enough to understand exactly why the Eagles didn't get burned in the ranking by this loss (and really didn't get burned by the Saints loss), but at the same time I watch the Eagles and feel like they display the same problems year after year (McNabb having cold streaks that cost them games, abandoning the running game, defense giving up a lot of first downs out of 3rd and long, moving the ball great between the 20s and not in the redzone, an inability to come from behind late in the game, bad clock management and complete incompetence in the hurry-up offense) and I just don't know if I'm overstating these problems or if those issues (specific to these Reid/McNabb teams) are overlooked somewhat by DVOA.

Pat, this is one case where I welcome you stridently telling me why I'm wrong and what I'm missing...

89
by Pat (filler) (not verified) :: Tue, 10/20/2009 - 9:53pm

They're 2-4, and one of the games I mentioned was a 4-12 team beating the 14-2 and eventual Super Bowl champion Patriots. I think it's fairly likely that the Raiders get to 4-12, and if the Eagles end up, say, 9-7 or 10-6, a 10-6 team losing to a 4-12 team is less surprising than a 14-2 team losing to a 4-12 team.

In terms of DVOA this is probably one of the biggest upsets ever right now. But I really, really doubt that the Eagles will end up a 30% DVOA team. 10-15% or so, I can believe. And I doubt the Raiders will finish a -40% DVOA team, so in the end I don't think this will look any more bizarre than, say, the Colts losing to Houston or the Patriots losing to the Dolphins.

but at the same time I watch the Eagles and feel like they display the same problems year after year

Yeah, so? That's because they're the same team. McNabb's accuracy isn't that great. We know that. Aggressive playcalling hurts on 3rd and long. We know that. The redzone thing I think is way overblown by fans - it was a problem last year. It certainly wasn't a problem on Sunday, when they weren't able to move the ball period. It hasn't really been a problem this year except in the New Orleans game, but that had a lot more to do with being down a lot rather than being the reason *why* they were down a lot.

95
by chemical burn :: Tue, 10/20/2009 - 10:18pm

I think you missed my point a little (although I agree with everything you have to say): the Eagles have the same problems over and over, most fans are able to see these problems, but the problems are somewhat invisible to (or at least downplayed by) DVOA. Does that mean DVOA is missing something (something that would help more accurately describe the Eagles performance in terms of a quantifiable number) or that these problems on the whole aren't as big as they seem to an average fan like me?

Also, I take issue with the idea that the Eagles weren't able to move the ball "period" - Westbrook was in top 5 DYAR (a counting stat, no less!) and I guarantee Celek had on ok DVOA. Their problem was they tried to keep moving the ball in a way that wasn't working. Those are two entirely different things. Throwing 20 deep passes when McNabb had no time in the pocket is a different failure than simply sputtering in all phases (which DVOA clearly thinks was not the case).

My question is: how does generalized success weight against specific succes in DVOA? Does the fact that the Eagles have an excellent running game that they underutilize matter so much to DVOA? I know they are rewarded in that they are more successful in general at low-percentage plays (deep passes) than most teams, but how does DVOA deal with the fact that they are attempting these low percentage plays more than the average team? (assuming, of course, that they really are - I didn't check, so... so it's a nice question? right?) And at the expense of higher percentage plays (like the running game). DYAR is a counting stat, but DVOA is not - how does that matter in these overall DVOA rankings?

104
by Pat (filler) (not verified) :: Tue, 10/20/2009 - 11:03pm

Does the fact that the Eagles have an excellent running game that they underutilize matter so much to DVOA?

The NFL is littered with examples of plays, ideas, players, and schemes that have success with limited use, and decrease dramatically in effectiveness as their use increases. We don't know they're underutilizing it. The coaches are really the only ones who know whether or not other teams are adapting to the success they're having such that they can't use it too much or else it won't be effective at all.

But anyway, it's important to realize that DVOA is a per-play stat. A highly effective underutilized component won't raise a team's DVOA. If they're unsuccessful more than they're successful, they're going to have a lower DVOA.

Also, I take issue with the idea that the Eagles weren't able to move the ball "period"

They weren't. They had successful plays, but they were few and far between. It's funny that you blame that on scheme - I tend to blame that on *players*. I think a good scheme and players that aren't playing well tends to produce a boom/bust output. I think a bad scheme and good players tends to produce drives that move, but stall.

and I guarantee Celek had on ok DVOA.

That's because DVOA doesn't measure blocking. If it did, oh, God, that'd be painful. This was easily Celek's worst game this year. Easily.

know they are rewarded in that they are more successful in general at low-percentage plays (deep passes) than most teams,

Having seen this argument for years, it's honestly hilarious. This is exactly the opposite argument that people made regarding Philly previously. The previous argument was that a WCO gets a boost because short, effective passes move the chains a lot, and that helps. Now you're saying they might be getting a boost because they're better at deep passes than average.

how does that matter in these overall DVOA rankings?

I think it would hurt them, actually. An 85 yard TD from the 15 gets you 7 points, but it's not really any different than a 40 yard play that ends up in a drive that stalls and punts. That cap means that it's entirely likely that a single 3-and-out (of which the Eagles had *a lot*) will offset a long TD.

150
by C (not verified) :: Wed, 10/21/2009 - 8:59am

"Pat, this is one case where I welcome you stridently telling me why I'm wrong and what I'm missing..."

Hahahah, but seriously, I don't buy any of it. Nobody has offered a legit reason as to why the Eagles are ranked #2... Yes, a 14 point underdog winning is about as big of an upset as you will see. Seeing dogs more than 14 points are rare, and seeing them win is even more rare...

We can all agree that DVOA is not a 100% perfect system right? Well the Eagles at #2 right now is your imperfection staring you right in the face. Defending that ranking is showing a blind allegiance to DVOA, when in fact the authors will tell you it isn't "perfect"... at least not yet. There is no reason to be angry.

172
by chemical burn :: Wed, 10/21/2009 - 11:36am

one quick thing, Pat: I'm not trying to make an argument one way or the other. I'm trying to figure why the Eagles consistently underperform (in relationship their DVOA) against teams like the Raiders this year and the Bengals last year.

Also, thanks for putting words in my mouth on the "WCO boost" when I was arguing on your side last year and put up the exact same argument I did just now. I know you can't remember what some anonymous internet poster said a year ago, but honestly, it frequently feels like you're not trying to understand me or help further any ideas here, you're just trying to take down whatever I write, even if I DIDN'T EVEN WRITE IT.

Let me quote you: "I think a good scheme and players that aren't playing well tends to produce a boom/bust output. I think a bad scheme and good players tends to produce drives that move, but stall." But DVOA is saying Westbrook played well and taking into account the boom/bust. We all agree McNabb played badly and Westbrook played well - how is calling 43 plays (including sacks, minus the passes to Westbrook) to McNabb and only 15 to Westbrook not a failure of what's loosely being called "scheme?" Are those coaching decisions, the meat of the play-calling, not part of what you're calling "scheme?"

"I think it would hurt them, actually. An 85 yard TD from the 15 gets you 7 points, but it's not really any different than a 40 yard play that ends up in a drive that stalls and punts. That cap means that it's entirely likely that a single 3-and-out (of which the Eagles had *a lot*) will offset a long TD." Obviously, I understand that. What I meant more was this: if a play works 15% of the time for the Eagles, but only 1% for other teams, DVOA will reward the Eagles for their success - one of its big strengths is that it compares apples to apples in that way. If a play works for the Eagles 80% of the time and 70% for other teams, DVOA will reward the Eagles less than it does for the 15%-to-1% play, correct?

My question has to do with, does DVOA overpraise the Eagles because they are calling the 15%-to-1% plays more frequently than the 80%-to-70% plays? For instance, the Eagles were calling a high amount of deep passes on Sunday vs. the Raiders and having just ok success with it, but they weren't calling running plays They were having more consistent, but also more likely success with their running play.

So, the Eagles drives were going basically like this: 40-yard gain on a deep pass, sack, incomplete, incomplete. But they were hitting big pass plays with some regularity - McNabb did, after all, have 269 yards on only 22 completions. And then, on the other side of things, Westbrook had (proportionally) big successes on 3 of his 6 runs. So, on a per-play basis, the Eagles don't look that bad to DVOA (they succeeding on deep passes - a low percentage play - with some regularity and were above average on running plays), but lost spectacularly: their offense was a complete and total failure in reality.

The Eagles failure seems to be something DVOA can't see accurately: coaching mistakes, idiotic scheme, QB making bad decisions (not looking for quick passes to Celek and Westbrook), bad clock management. Is there any way for DVOA to account for these things or are they not really descriptive of past failures or helpful for predicting the future?

196
by Pat (filler) (not verified) :: Wed, 10/21/2009 - 2:07pm

I'm not criticizing you at all. I'm saying it's hilarious that three years ago people were saying DVOA overrates the Eagles because the WCO relies on short passes that extend drives. Now people are saying that DVOA overrates the Eagles because they rely on deep passes too much.

I know they're not the same people, but it is the same *team*, and so to hear literally opposite arguments about the team is hilarious.

how is calling 43 plays (including sacks, minus the passes to Westbrook) to McNabb and only 15 to Westbrook not a failure of what's loosely being called "scheme?"

Because we don't know that calling more plays to Westbrook would've made a difference. It might've just reduced Westbrook's effectiveness for the day.

It should also be noted that you have no idea if the sacks and lots of the passes weren't intended for Westbrook. DYAR measures Westbrook's success when the play moderately succeeds - that is, when he carries the ball or when McNabb actually is able to target him. You don't know how often the play was a run, but McNabb audibled out, or how often the play was a short pass to Westbrook that was taken away or McNabb got sacked.

The Eagles failure seems to be something DVOA can't see accurately:

This is the part I just don't understand. Why do you say this? The Eagles at this point have been a team that can score a ton of points on offense and can prevent an offense from moving at all. That's an awesome team, and DVOA says "yeah, they've been pretty good." They've *also* been a team that can let another team run up and down the field on them, and a team that stalls on 2/3 of its drives. That's a terrible team.

But they've had 3 great games, an okay game (vs. New Orleans) and a bad game (vs. Oakland), which averages out to "a pretty great team" but with a ton of variance. Which is exactly what DVOA says.

Heck, if you want one *other* explanation, note that the Eagles have had early-season struggles for a long time now. Heck, even in 2004, they nearly lost to the Browns early in the season. We know that football performance changes as the weather gets worse and injuries pile up as the season goes on. It's perfectly possible that the Eagles are built more for the latter portion of the season than the early portion of the season, so early on they look overrated but as the weather progresses public perception lines up with what the rating is.

Personally I'm more inclined to point out that the Eagles have had a lot of early-season injuries for the past 5 years, plus the fact that the division turned around entirely means that, like the NFC West where people think the teams might be underrated, all the teams in the NFC East tend to appear overrated.

205
by Dales :: Wed, 10/21/2009 - 3:33pm

"I'm saying it's hilarious that three years ago people were saying DVOA overrates the Eagles because the WCO relies on short passes that extend drives. Now people are saying that DVOA overrates the Eagles because they rely on deep passes too much."

There is a commonality to these arguments or postulations beyond them being merely anti-Eagles.

How about this as a hypothesis? DVOA overrates the Eagles because they rely on passes too much.

Below I made a post where I try to quantify this. One way of interpreting the numbers I posted is that the Eagles' heavy reliance on the pass causes them to underperform their expected wins by about 1 game a year. Looking at the preseason DVOA ratings, it looks to me as if the difference in ~1 win corresponds with a difference in DVOA of about 10-20% (for example, see the difference between Indy and either Pittsburgh and Jacksonville, or between Tennessee and Carolina). Knock 10% off of the Eagles DVOA and their ranking this year would 'look' about right to me.

209
by Pat (filler) (not verified) :: Wed, 10/21/2009 - 3:56pm

How about this as a hypothesis? DVOA overrates the Eagles because they rely on passes too much.

Yeah, it's possible, but even there I wouldn't call it "overrating." DVOA measures a team's ability to score, because that's what's most consistent, week to week, year to year.

It doesn't measure the ability of a coach to completely botch a game plan against another coach, and this is not a unique feature of the Eagles (see the examples posted elsewhere). It only measures success at what they have done, not success in what they will do.

And for people who think I'm 'defending DVOA', I'm not. I'm saying that measuring the ability of a coach to gameplan effectively shouldn't be in DVOA, because I don't think it's a first order parameter. I think it's a second order parameter - that is, I think a coach's ability to gameplan is dependent on the team he's facing. You couldn't put that in DVOA because you can't quantify that in an ordinal ranking.

As I mentioned elsewhere, you can show that that sort of effect is negligible in the NFL on a single-season timescale. Might be able to pull something out on a longer timescale.

(It's also worth noting that I'm not sure your hypothesis is even right - did the Eagles start passing significantly more after 2003? Not sure, but I don't think so. But I do think the possibility that a higher-order parameter is at fault here.)

217
by Dales :: Wed, 10/21/2009 - 4:26pm

Oh, I am not sure it is right, either. I don't think I have enough information to prove or disprove it. It's just a hypothesis.

"did the Eagles start passing significantly more after 2003?"

From 2000-2003, the Eagles averaged 1.24 passes attempted to every rush attempted.
Since then, they have averaged 1.45 passes attempted to every rush attempted. I'd say that's a fairly substantial increase in the passing rate.

ETA-- just had a passing thought. In 2004, the Eagles spiked up to 1.45 PA/RA. That was also the last year they outperformed their expected wins (I understand your point regarding the strength of schedule, but in nearly all of the years their SoS was close enough to average that I don't think it matters much). Every year since they have lagged it. So here is another hypothesis to build on my other one-- it took a full year for other teams to fully appreciate how pass-happy Andy Reid had become and adjust their planning accordingly. I am sure an in-prime TO leaving also made a difference. The first year (2004) he had success with the approach, but teams adjusted and he did not, and since then the Eagles have consistently underperformed their estimated wins.

Again, not picking a fight. I consider the Eagles to be a very good team, and a serious contender for the NFC crown.

219
by Pat (filler) (not verified) :: Wed, 10/21/2009 - 4:29pm

Huh, that is a noticeable increase. My mistake, though, as it should be 2004 - the Eagles weren't "overrated" in 2004 - if anything they were underrated due to the tanking of the last few games.

To be fair, my main explanation (it's not DVOA, it's the division) also has a much more substantial change. 2001-2004 NFC East, non Eagles, avg. wins (excl. Cardinals in 2001): 6.58 wins. 2005-2008 NFC East, non Eagles, avg. wins: 9.41.

221
by Dales :: Wed, 10/21/2009 - 4:51pm

The division has gotten harder, without doubt. But even with that, the Eagles' strength of schedule by DVOA has been over 4% just once, and I would argue that it has been close enough to league average each year to not account for them underperforming the estimated wins each year by 1-2 wins. Heck, this year they are already 1.3 wins below the estimated wins despite a past schedule below -15%.

While I have not compiled the numbers, I will bet that the other NFC East teams have not had the same negative gap between their expected wins and their actual wins over years 2005-2009 that the Eagles have had. If this suspicion is correct, then it would be an indication that the strength of the NFCE is not the primary culprit, and instead it is either random chance or some other factor.

231
by Pat (filler) (not verified) :: Wed, 10/21/2009 - 5:35pm

The reason they outperformed their estimated wins in 2004 was because they tanked the last few games. Look at the late-season DVOA rankings articles for that year and you'll see they're pretty dead on, taking into account the strength of schedule.

(I understand your point regarding the strength of schedule, but in nearly all of the years their SoS was close enough to average that I don't think it matters much).

Really, I use the average strength of schedule there as a proxy, because you can't boil down the schedule to one number (playing a 50% team, then 5 -10% teams will give you an average SoS but if you're a good team, you'll have a 5-1 record). The Eagles' record vs. NYG/WAS/DAL from 2001-2004 was 21-3. The Eagles' record vs. NYG/WAS/DAL from 2005-2008 was 9-15.

The change in the division from 2001-2004 is just massive, and considering that's the exact point when people started calling the Eagles overrated (early in 2005 they were highly rated early on), I can't just call that coincidence.

Although it's worth noting their subpar NFC East record since that time, which might imply that both of us are right, and the biggest problem is that the teams in the NFC East figured out the Eagles quickly, too.

199
by tuluse :: Wed, 10/21/2009 - 2:28pm

I'm pretty sure DVOA just see incomplete passes, it doesn't care how far you are throwing them.

204
by Pat (filler) (not verified) :: Wed, 10/21/2009 - 3:28pm

"For instance, the Eagles were calling a high amount of deep passes on Sunday vs. the Raiders"

Keep in mind that DVOA doesn't think the Eagles had a good game on Sunday. It thinks they played like a team in the bottom third of the league.

If you want to ask "is DVOA getting the Eagles right" look at the two lowest ratings they have - versus New Orleans, and versus the Raiders. Aaron listed their DVOA for those games - ~1% (NO) and ~-15% (OAK).

Let's look at those objectively. New Orleans has a 40% DVOA on offense - that means against an average defense in a ~11-ish drive game, they'll score 30 points. They have a -20% DVOA on defense - that means they'll allow about 17 points in a similar game. The Eagles game was about 30% longer than an average game, so you'd expect 40-22 if the Eagles were an average team. Less the interception return, that's basically exactly right. So the 1% DVOA looks maybe a bit high (obviously the interception return accounts for something, although how much is debatable), but not much.

For the Oakland game, Oakland has a -40% DVOA on offense. That's 13 points on offense. They have a 5% DVOA on defense. That's ~22 points allowed on defense. This was again a bit longer, so you'd kindof expect a 16-28 loss by the Raiders to an average team. The Eagles allowed 20% fewer points, but scored 70% fewer, so you'd kindof expect a -50% DVOA there.

So I might say that they should have a worse DVOA for the Oakland game, based purely on points. Two things are worth noting, though:

1) Even if the Eagles had a -50% DVOA rating for that game, they'd only drop by about 7% overall. That would drop them... one spot.

2) Low scoring games tend to deviate more from DVOA's expectation a lot more than high scoring games. That's just fluctuation - the Raiders game turned entirely on one single play. The Saints game most definitely did not. If you treat the long TD as a ~FG-ish play, something like a -30% DVOA would make more sense for the Eagles (for the pedantic, the same for the Saints game - neglecting the Jackson TD - is offset by the fact that the Eagles were forced to go for TDs late in the game due to the score).

If you really want to ask "right now, what is DVOA missing about the Eagles" the answer is just "opponent adjustments aren't strong enough." Every single game the Eagles have had will be strongly affected by opponent adjustments, and in only *one* case will it make their rating for that week better.

Double the strength of the opponent adjustments and the Eagles are a mid-teen to low-20s team with probably 20% fluctuation. Statistically, that looks right. And I can't stress enough how big a 20% fluctuation would be if it were full season.

212
by Dales :: Wed, 10/21/2009 - 4:12pm

(comment removed- realized I was repeating myself to someone I already replied to)

86
by Anonymouse (not verified) :: Tue, 10/20/2009 - 9:22pm

The 2003 SB Champs were 14-2, same as the year after that.

It only strengthens your point, I'm just saying.

4
by tunesmith (not verified) :: Tue, 10/20/2009 - 6:15pm

I wish I had a better suggestion than I do on what FO can miss, but it's the same one I always have had: scheme. It's what led me to (tongue-in-cheekingly) create beatpaths.com five years ago, I was annoyed that Denver was ranked too low, and believed that Shanahan's scheme strength would regularly give the team a boost above what the individual performance stats would suggest. I didn't have a way to measure that either, so my approach was just to measure everything that FO doesn't (uh... only wins and losses), and not measure anything FO does (uh... everything but wins and losses). And make some pretty graphs that amuse me. But anyway, players that perform badly in some games will perform much better in other games as the scheme changes. Last year, Denver's defensive scheme just *sucked*. The year before, it sucked too - that's when they had Bates, who last I heard, is stinking up the joint in Tampa Bay. Before that, he was at Green Bay, who did okay, but whose defense improved after they dumped Bates' religion. And Slowik has just sucked everywhere. So, as far as Denver switching everything over, that wasn't reason to believe they would be better, but it was reason to believe it was less likely they'd be as lousy as before.

And as for San Diego and Philadelphia, I rail on them pretty regularly over in our power rankings. They seem like sister teams to me, and the theme is more talent than team. They're going to have gaudy stats and high expectations, but they're going to underperform. That's something that is reflected in their wins and losses.

83
by Pat (filler) (not verified) :: Tue, 10/20/2009 - 9:03pm

I wish I had a better suggestion than I do on what FO can miss, but it's the same one I always have had: scheme.

There are lots of ways you could interpret that statement.

1) Scheme, meaning when a team switches schemes, their performance can dramatically change. I think this is a "yes, but it can change dramatically good or dramatically bad, and there's no way to know."

2) Scheme, meaning that some teams match up better with other teams: i.e. the (untrue) "Manning struggles via a 3-4". This can actually be shown to be unimportant in the NFL. I wish I had the link - it's in the "does the preseason matter" article. Basically, the NFL is best modeled by a single ordered hierarchy (i.e. A greater than B greater than C greater than D... etc.) with the average win expectancy gap between teams being about 10-15% (i.e. the best teams will win ~80% of the time versus the worst teams).

I was annoyed that Denver was ranked too low, and believed that Shanahan's scheme strength would regularly give the team a boost above

You could've been looking at DVOA rather than estimated wins. DVOA isn't tuned to the ability of a team to win. That's what estimated wins does.

Looking at Denver, for instance:
DEN 2008: Est. 7.0, act. 8, sched -1.4%
DEN 2007: Est. 7.5, act. 7, sched 0.0%
DEN 2006: Est. 7.5, act. 9, sched 1.1%
DEN 2005: Est. 11.9, act. 13, sched 7.0%
DEN 2004: Est. 10.9, act. 10, sched 1.5%
DEN 2003: Est. 9.8, act. 10, sched 1.9%

The only big deviation was in 2005, when Denver played a pretty easy schedule. The others are all within a win.

(The response might be "wtf do we care about scoring rather than winning?" and the answer is "because a team's ability to score is more consistent year-to-year - and during the year - than their ability to win")

5
by MileHigh (not verified) :: Tue, 10/20/2009 - 6:15pm

I don't think there is a good objective rating for what you see with Denver and SD. Denver switched defensive schemes, but also turned over 7 out of 11 positions. Last years ratings are worthless given that amount change.

In SD, it really does come down to coaching. There should be some sort of "Coach DVOA" that could show that Norv Turner is just not a good head coach. Besides that, injuries have taken thier toll on a defense that finsihed last year ranked #20 in DVOA. And an offensive line that ranked #18 in adjusted line yards and #17 in sack rank, seems to be headed back toward the bottom half of the league.

Come to think of it, what onjective standard did you use to come up with your preseason rankings. A team that would get a number one ranking in the beginning of the year should have displayed a storng upward trend the prvious year and be bringing back a large majority of returning starters in the same scheme. Outside of that it's a subjective guessing game.

16
by tunesmith (not verified) :: Tue, 10/20/2009 - 6:31pm

I'm curious about that too... I am not really sure what went into the preseason rankings. I hope it's not preseason performance - some teams are infamous for playing badly in preseason and then turning it on for the season.

In San Diego, I've really wondered if they have some sort of cancerous locker room presence. The only coach in recent years that has succeeded in getting them close to their talent level was Schottenheimer.

59
by Scott C :: Tue, 10/20/2009 - 8:20pm

What went into the ranking (read FO Almanac maybe?) --

Scheme stability.
Very good offense last year with key young players entering their prime (Rivers, Jackson, etc).
Defense was expected to improve as Merriman returned (there is a "returning pro bowl sackmaster" variable from my understanding). Overall improvement of health on D.

Basically the system said that if the offense was as good as before, and the defense became above average, the team would be very good. (See Saints, New Orleans).

That didn't happen, what did?

1. Norv's style has shown now for 3 years to have the team play poor to start the season. The projection system may not have caught on to that and might even see the 'rise' as the season goes on as positive.
2. On offense, two linemen were out in the first game (one returned, he's a rookie).
3. on defense, they ended up LESS healthy than last year, not more. two starting D-lineman out, others injured. New coaches on D (D-line coach esp, linebackers perhaps) struggling, new defensive scheme not as effective as expected. The projection system thought it would be the same, but it isn't.

Projection system improvements:

Can the "teams that play better in the second half" part of the projection adjust for teams (coaches, in particular) that always have a upward or downward profile?

When a big player comes back from injury, maybe we should not expect a big impact unless the scheme (D-coordinator) is the same from the year before the injury?

Before the season began, most Charger fans knew where depth was lacking:
At the O and D line.
Injuries there would be bad. Most fans knew that losing Jamal Williams would be the worst thing for the D -- all the backups are projects at best for NT. On the O-line there are also serious depth concerns that were validated when two interior linemen went down. The skill positions are stacked depth wise. Some depth concerns in the secondary exist, but its not bad.

138
by Mr Shush :: Wed, 10/21/2009 - 6:45am

I don't think the issue for Merriman is change of scheme - it's that he's plainly not fully healthy. The projection system thought the team had effectively added one of the five best pass-rushers in the league. It hadn't. That's a huge difference. Whether there's any statistical work to be done on injury recovery that would allow this to be usefully predictive in future, I don't know.

66
by Pat (filler) (not verified) :: Tue, 10/20/2009 - 8:29pm

"I hope it's not preseason performance - some teams are infamous for playing badly in preseason and then turning it on for the season."

It's not, but I still think it should be. There's no serious evidence for your claim. The only teams that it's even marginal for are the Colts and the Patriots, and it's not even that serious there.

Most of the time "poor preseason performance" is due to a difficult preseason schedule. When you look at how teams do in the regular season versus how they do in the preseason versus the same team, there's a very strong correlation.

6
by Tundrapaddy (not verified) :: Tue, 10/20/2009 - 6:16pm

Okay, FO has determined that bringing in a new defensive coordinator doesn't historically show a difference in DVOA. Having said that - what about the offensive coordinator?

More specifically - could Orton's 'career numbers' have been bad because Chicago's offensive playcaller (Ron Turner?) had a horrible, horrible system? And now that Neckbeard is playing elsewhere, his numbers are improving?

Let's face it - other than Manning, QBs are out there to run the plays they're given from the sideline.

As further evidence, look at how Cutler is doing. He was 5th in DYAR last year. This year, he's currently 21st.

17
by tunesmith (not verified) :: Tue, 10/20/2009 - 6:33pm

This ties into the scheme theory. And also, while adding a new defensive coordinator might not correlate to much, getting rid of an old defensive coordinator might.

37
by Matt Groves (not verified) :: Tue, 10/20/2009 - 7:07pm

The main thing that is measurable is turnovers; but not that predictable. In 2008 Cutler had 18 interceptions and the team had 18 fumbles loosing 12, which wasn't too bad.
But they had a -17 on the turn over ratio.
This year, four fumbles and one interception after 6 games. They are +7 on turn over ratio.
However, Orton was full of interceptions during the preseason, so I guess it goes back to the Offensive Coordinator. Maybe he said, "In the preseason throw whatever you want, test your limits, because during the season you have three options: throw it away, run or take the sack." I don't know what their preseason turnover ratio was but maybe it all comes down to, you can't use the preseason to predict or turnovers, which are a huge part of the game are unpredictable.

223
by Rich Conley (not verified) :: Wed, 10/21/2009 - 5:06pm

Don't forget luck.

I've seen Orton throw about 8 passes that have been deflected up into the air, and come down into his receiver's hands. One won the Bengals game. There were two in the Pats game, one of which bounced off 3 patriots and was then caught by Marshall.

208
by TomC :: Wed, 10/21/2009 - 3:52pm

I would guess that going from one of the worst O-lines in the game to one of the best has more to do with Orton's success than scheme. He had as much time to throw in that Chargers' game as any QB I've seen since '07 Brady.

7
by Scott P. (not verified) :: Tue, 10/20/2009 - 6:17pm

I'd much rather 18-1/17-2 as long as it ends in the longest party NO has ever seen.

I don't know -- even if NO wins the Super Bowl, the 2001 Pats' celebration will give it a run for its money.

P.S. My captcha is 'Klingon rioting'

9
by Tundrapaddy (not verified) :: Tue, 10/20/2009 - 6:20pm

I want your captcha!

58
by coboney :: Tue, 10/20/2009 - 8:19pm

Having seen the Saints return to the superdome I don't doubt it.

8
by Fan in Exile :: Tue, 10/20/2009 - 6:20pm

If you want to take a serious look at why the Denver ranking was so off I would suggest that you start to look at the statistical differences between the various offensive and defensive systems.

It's something that Carrol talks about in his book about how a turn over in one system is not as harmful as a turn over in a different system. I think that would allow you to adjust for Orton being in a system he fits and not Ron Turner's. It might also allow you to adjust for someone like Dumerville being put in a 3-4 and not a 4-3.

On a side note I think this might also help alleviate the problem that you have with the Eagles. What they do may work for a generic DVOA but clearly it isn't working for the type of system that they are running.

On a less comprehensive overhaul note one of the things that you guys pointed out was that an all over 30 secondary struggles with injuries. I can't help but look at that and think that's a performance from small sample size theater. I mean how many teams have really started secondaries that are all over 30?

I also think that someone who follows the BB school of 53 person team building should get a smaller penalty for injuries since there is so much depth there. I mean even if someone goes down we'll be starting Smith, JMFW, Bell, Barret, McBath, Bruton. I love Shanahan but we never had this kind of depth.

12
by Tundrapaddy (not verified) :: Tue, 10/20/2009 - 6:24pm

'we'll be starting Smith, JMFW, Bell, Barret, McBath, Bruton.'

For a fleeting moment I replaced 'Bell, Barret, McBath' with 'Bel, Biv, DeVoe'. And then tittered upon picturing Denver's backup secondary.

50
by James-London :: Tue, 10/20/2009 - 7:39pm

That would be Poison.
Sorry.

Phil Simms is a Cretin.

79
by Rocco :: Tue, 10/20/2009 - 8:48pm

Never trust Kyle Orton and a smile?

191
by Trevor (not verified) :: Wed, 10/21/2009 - 1:29pm

i really laughed hard at this! +1

139
by Mr Shush :: Wed, 10/21/2009 - 6:58am

I'm afraid to say that my (pre-Wiki-ing) reaction to these two comments was, "DeVoe? Do you mean De Ville? And who the hell are Bel and Biv?". Although on balance, absent Jamal Williams it's more the Chargers defense that's inclined to Open Up and Say . . . Ahh!

10
by goforit (not verified) :: Tue, 10/20/2009 - 6:21pm

Isn't San Diego just getting screwed by allowing the most 3rd and 4th down conversions of any team. FO has taught me that these stats make a huge difference in teams winning and losing, especially with a team like SD, where a 3rd down stop would get their high powered offense back on the field. Usually these revert to the mean right?

Also do you have stats on 4th downs this year. I think coaches are going for it a lot more than normal... is there any data to support this feeling?

14
by Joseph :: Tue, 10/20/2009 - 6:25pm

Can't comment about Denver--except winning, even with luck, breeds confidence, better attitude,chip-on-the-shoulder-nobody-respects-us-even-though-we're-W-L, etc. Obviously, the projection system doesn't get that.
Re: SD, I will posit that A) teams overly respected SD run game, which allowed Rivers to post great passing numbers--which this year they don't; B) Their defense hasn't played good at all--specifically, nobody would have expected Merriman to play poorly, nor what would happen if Williams got injured; C) Not playing with the lead has had a domino effect on both sides of the ball; but the biggest factor is:

Wait for it...

That lots of NFLers read the Almanac, saw the SD win projection, and have circled that game on the calendar as if they were the defending SB champs. Thus, NOBODY is turning in an Eagles-type stinker against SD. ;)

13
by chappy (not verified) :: Tue, 10/20/2009 - 6:24pm

Well here's my question about the projection system. How do you know you aren't asking a question that won't leave to overfitting? Do you go back an test prior year to make sure it works? Anyway, it seems to me that every year your projection system is wrong to some degree, but you need to just have confidence in the model structure. Of course you're going to be wrong--you're dealing with a sport that is all about small sample sizes! Sometimes an anomoly is just that.

38
by David W. (not verified) :: Tue, 10/20/2009 - 7:07pm

+1. If your model said Matt Forte rarely fumbles, would you throw it out because of the Falcons game?

62
by Scott C :: Tue, 10/20/2009 - 8:26pm

My understanding is that they only incorporate changes that yield better predictions when run on years past.

15
by t.d. :: Tue, 10/20/2009 - 6:26pm

I guess it shouldn't be a surprise that seven of the 10 worst dvoa performances came from the nfc west.

31
by Tundrapaddy (not verified) :: Tue, 10/20/2009 - 6:45pm

That's exactly what I thought when I saw that chart.

And you wonder why the Seahawks have often been 'flat' when they hit the playoffs. Stuffing themselves on all that divisional fat can't be good for the system.

18
by BaconAndWaffles :: Tue, 10/20/2009 - 6:34pm

Denver's Defensive System for Success:

Step 1 New Coordinator
Step 2 New Personnel
Step 3 . . .
Step 4 Profit!

I think that Denver is very legitimate this year, but I still can't help thinking that they are a Stokley tip drill away from being a different team. I usually think of luck as something that alters the outcome of a given game, but in Denver's case I think it gave them some very much needed momentum that has changed them from a potentially average/good team to a great one.

19
by BigDerf :: Tue, 10/20/2009 - 6:37pm

I'm admittedly taking a Bill Simmons idea here but "Norv Turner Sucks" isn't factored in enough because "Norv Turner Sucks" is a bigger growing problem every year.

The first year after Marty left a 14-2 team Norv Turner turned it into an 11-5 team. Then last year they went 8-8.. another 3 game loss off the year before.

Bad Coaching shows up immediately in poor decision making but it's effects aren't fully realized on the record until a few years later.

On Denver's end - I feel like they just improved at a lot of positions. They replaced shoddy veterans with guys who are playing better (Dawkins and Goodman in the secondary) and their young guys like Dumervil and DJ Williams are simply progressing into stars.

41
by Mac (not verified) :: Tue, 10/20/2009 - 7:19pm

I think BigDerf is right -- it's not just that Norv sucks, but his suckiness is progressive. The longer he stays on a job, the worse he gets.

163
by wr (not verified) :: Wed, 10/21/2009 - 10:42am

I don't think that's quite right. I would say that the (lack of) quality in Norv's coaching doesn't vary much, what is progressive is its effect on the team. See Switzer, Barry and the Cowboys. The problem is trying to capture this objectively - How easy is it in general to distinguish a bad coach from an average/good coach in a bad situation?

176
by zlionsfan :: Wed, 10/21/2009 - 11:41am

Maybe this is part of it. It seems clear that the Chargers are being steadily Norved into the ground, and yet we have no way to demonstrate this. We need to borrow a time machine and let Norv demonstrate that he can take his'n and lose to your'n and then take your'n and lose to his'n.

238
by Jerry :: Wed, 10/21/2009 - 6:35pm

LOL. Too bad Wade's not nearly as quotable as his father.

20
by t.d. :: Tue, 10/20/2009 - 6:38pm

The Patriots in 2008 didn't decline as much as you might have expected losing the 'premiere' quarterback in the league. The Broncos still had a great deal of continuity on one of the better offenses in the league from last season (the line in particular). The Broncos didn't just change coordinators, they also changed schemes. What's the track record of teams in the first year after switching schemes, particullarly if they're changing to one their opponents will be less familiar with? The Packers also have improved a good amount on defense after switching to a 3-4, though it's early.

234
by dave b (not verified) :: Wed, 10/21/2009 - 5:51pm

The GB defensive rank jumped from 10 to 4 this week after shutting out Detroit. This was a Stafford and Megatron-less Detroit team, so adjust you scorecards accordingly.

21
by JuridianSantaal... :: Tue, 10/20/2009 - 6:39pm

My guess for the Denver question would be just the ridiculous turnover on defense - a huge number of changed players, new DC, new head coach. The other thing that might also be under represented is safety play (though I don't know how one would quantify that). Better safeties = less big plays = longer drives = more chances for an offense to screw up. New Orleans turned around D is also benefiting from stronger safety play as well, meanwhile, Safety play has been a major concern for SD last year and remains this year.

Maybe its just an opinion, but I think defense is all about minimizing big plays and having as many pass rush threats that force an offense to scheme very creatively to neutralize. It's easy for teams to shut down one good pass rusher (Ware and Peppers this year) and if the team has no other pass rush options, their D struggles. I count strong CBs as a pass rush threat because it allows teams to blitz more often. I think Denver's D is playing better because of Dumervil's development and stronger secondary play that's letting them get away with blitzing more often. Against SD, they blitzed A gap nearly the entire game with huge success.

30
by ammek :: Tue, 10/20/2009 - 6:45pm

Yeah, speaking of player turnover. I don't know if you do this, but let's go back ten years and produce a pretend projection for the 1999 Rams. It is based on previous years' data, but uses all the trends that you have picked up from the imminent twenty-first century. What does it look like?

141
by Mr Shush :: Wed, 10/21/2009 - 7:04am

"My guess for the Denver question would be just the ridiculous turnover on defense - a huge number of changed players, new DC, new head coach."

Right, but plenty of teams with terrible defenses have brought in a bunch of new players, a new DC, a new scheme and a new head coach. Most of them still sucked - see the Texans in 2005 vs. 2006 for one example. The question is, was there any reason to expect the Broncos to be different to those teams? "Champ Bailey healthy rather than playing hurt" and "Elvis Dumervil a much better fit for the 3-4" seem like pretty tricky things to quantify.

213
by Might as well be Micheal Lombardi (not verified) :: Wed, 10/21/2009 - 4:12pm

Well, that's a key point, that "playing hurt" is impossible to quantify. Several posters have pointed out that Orton's career numbers wouldn't predict his great start this season. But his only career numbers were his rookie season and last year, and anyone who followed Orton closely last year (I traded Warner for 2 RBs becuause I thought he'd be a good enough starting QB for my fantasy team) knows that he simply was not the same guy after he got hurt. Same with Favre. If the post-bicep tear Favre is thye QB of the Vikings, we'd probably be talking about Rosenfels right now.

And everyone gets hurt in the NFL. How can you possibly quantify it all when the injury reports only tell a small part of the story?

225
by Rich Conley (not verified) :: Wed, 10/21/2009 - 5:12pm

"Same with Favre. If the post-bicep tear Favre is thye QB of the Vikings, we'd probably be talking about Rosenfels right now."

Favre has that same decline in the second half every year though.

259
by Faux Lombardi (not verified) :: Thu, 10/22/2009 - 3:21am

Evidence? I just spent 10 minutes poring over Favre stats. I don't see it. Last year he fell off a cliff after week 11, he went from averaging triple digit QB rating to not breaking 61 the rest of the season.

It seems in other seasons that he mostly stays the same, and he even usually finishes strong in the last game or two.

22
by MJK :: Tue, 10/20/2009 - 6:40pm

I bet when Aaron ran the numbers and saw Philly at #2, he said:

"NOOOOOOOOOOOOOOOOOO!!!!!!!!"

One thing I am still curious about is how many "iterations" the D in DVOA represents. I recall a number of years ago, the Colts broke DVOA because their schedule was so pathetically easy, and they were good enough to destroy all their pathetic opponents, that opponent adjustments got hopelessly skewed. Aaron fixed it by "iterating" the D a second time...calculate VOA, determine how good opponents were based on their VOA, adjust all VOA's by opponent to get DVOA iteration 1, and then instead of stopping, do it again...re-determine how good opponents were using their DVOA (iter1) and re-adjust DVOA(iter1) by opponent DVOA(iter1) to get DVOA(iter2). My question is--does Aaron now stop at one cycle, or does he iterate to "convergence", or at least several more times. That could "fix" some of the problem with the Eagles...if they creamed three easy teams, maybe DVOA doesn't yet realize how easy those creamings were.

Regarding Denver and San Diego:

I don't know if there's any objective system that could have predicted Denver. From what I've seen and read, the actual things that have caused the turnaround are (1) better offensive playcalling, (2) greater focus by the players on their job and better situational preparedness, especially on defense, and (3) a complete and utter housecleaning of defensive players.

(1) and (2) are a result of good coaching, by McDaniels and Nolan (and possibly by Orton as opposed to Cutler...I'm not sure how involved in play selection Orton is and Cutler was). The projections have a "coaching change" component, if I recall correctly, but they're all negative...right? I.e. if you change coaches, you're expected to decline. I actually think this is wrong in some cases...replacing Lane Kiffin or Pete Carrol for example is bound to make you improve or at least stay the same...but Shannahan was a pretty good coach, so there's no way a "getting rid of bad coach" modified would have helped you peg the Broncos.

The third factor could possibly be a predictive measure...if you had a way of objectively evaluating the quality of a defensive player in a vacuum and then estimating how likely a bunch of disparate players brought together were likely to play. I think that is simply not possible with the data available today, even with game charters.

67
by Scott C :: Tue, 10/20/2009 - 8:31pm

I don't think the D in dvoa goes past '2-deep' in the iterations. It could go to full recursion in one pass if it used the right matrix math.

Plus, so far there is no D in it -- these are VOA numbers.

143
by crack (not verified) :: Wed, 10/21/2009 - 7:27am

There is some D in them, the unadjusted VOA is in the bottom chart.

112
by Boston Dan :: Tue, 10/20/2009 - 11:52pm

Reading that post just ripped a hole wide open in my brain.

142
by Mr Shush :: Wed, 10/21/2009 - 7:09am

"replacing Lane Kiffin or Pete Carrol for example is bound to make you improve or at least stay the same"

I hate to break it to you, but replacing Pete Carroll with Bill Belichick "resulted" in a drop from 8-8 to 5-11.

Also, while the Broncos offense has been good this year, it was better last, so I'm not sure about the "better offensive playcalling" thing either.

154
by Todd S. :: Wed, 10/21/2009 - 9:50am

W/R/T Shannahan, could it be that long-tenured coached have players start to tune them out? Thus even players who were sub-par the year before react positively to a change in messenger? I have no idea how you could work this in mathematically, however. Sample size would be a big issue. (And, could we be seeing the same thing with Fisher and the Titans this year?)

240
by Jerry :: Wed, 10/21/2009 - 6:45pm

The Steelers were 6-10 in 2003, 15-1 in 2004, 11-5 plus a championship in 2005, and 8-8 in 2006. Maybe that's a result of when they were and weren't listening to Cowher, but it doesn't look at all predictable.

23
by wardh2o (not verified) :: Tue, 10/20/2009 - 6:40pm

Orton has more swagger than Cutler ever did and you just can't account for swagger in DVOA ratings.

123
by Achaean (not verified) :: Wed, 10/21/2009 - 12:35am

What about "He just like a kid out there." They have an adjustment for that, right?

I think i remember an article on FO a few years back which argued there was a correlation between justKIDness and GunSlingerosity, but said that there was no way to show causation. Also there were problems with a small sample size.

214
by Might as well be Micheal Lombardi (not verified) :: Wed, 10/21/2009 - 4:17pm

For Cutler it's "he's just a sullen teen out there." I swear if he thows another interception he's going to start wearing a black trenchcoat in the huddle.

24
by ammek :: Tue, 10/20/2009 - 6:40pm

I re-read the FOA Chargers chapter not long ago. The obvious misprognostication is the defense: DVOA expected it to return to 2007 levels after a sub-par 2008, but that hasn't happened, and you need to look more deeply into why. It appears that the 2007 interception totals were flukily high: did DVOA expect that kind of performance again? And I suppose there is a Merriman factor.

The run defense is another matter; as I assume you examined on ESPN, the loss of Williams and Oshansky is a severe blow. NTs in the 3-4 are really so valuable — isn't it like losing your Pro Bowl quarterback?

Finally, someone has to explain the run offense. ALY isn't great, but the onus falls on the backs more than the linemen, and if the #1 ranking off left tackle is meaningful at this stage of the season, we might even imagine that Marcus McNeil is back. Tomlinson is obviously cooked, but DVOA already knew that and didn't care.

Sorry, I've nothing to offer but musings.

76
by Still Alive (not verified) :: Tue, 10/20/2009 - 8:42pm

My completely subjective memory is that players who get caught with steroids etc. (particularly in other sports) rarely ever return to that form again (presumably because they can no longer risk using). I for one was not surprised that Merriman hasn't been quite the same player recently. I know a few guys even now who use and it definitely makes a difference in their bodies. Also these things are often done in groups, and perhaps the Chargers are comparatively cleaner than they were two or three years ago.

Substance abuse suspension in year X >> lower quality of play in years X+1, X+2 etc. This type of adjustment is more for an individual sport like baseball, but, just some random speculation.

122
by THE Sean C (not verified) :: Wed, 10/21/2009 - 12:30am

I don't necessarily disagree with this, but to be fair to Merriman, he did have a major knee injury, and that has been known to leave players permanently affected for the worse...less so than in decades past, but it still happens.

215
by Might as well be Micheal Lombardi (not verified) :: Wed, 10/21/2009 - 4:20pm

Steroids only count in baseball.

Can you imagine the outcry if the Minnesota defensive line's success were translated to baseball? Tom Verducci would have stabbed them both to death by mow for the integrity of the game.

25
by ebinary (not verified) :: Tue, 10/20/2009 - 6:40pm

Me feeling is that wins or losses that are more than 2.5 TDs should be normalized to some maximum impact, because most teams are not going to play the same if they are up 2.5 TDs at the end of the game. And most teams down by 2.5 TDs or more are not playing normally either.

Obviously, a 59 point blowout by NE didn't help them a lick the week before in Denver, nor is NE in any better position to beat Denver today than they were a week ago (they were, in fact, shut out in the 2nd half, so if anything, they'd be worse off).

Regarding predicting Broncos success, how about a "team change" factor that lowers the influence of the previous season based on the degree of change in personnel and coaching. I.E. if you'd started with a clean slate in Denver's case, you would have more quickly got to the proper ranking.

There is also a problem with assuming every team is always attempting to score on every down. Teams with tough defenses and a marginal lead will often run a much more conservative offense, because the chance of winning (the ultimate goal) is improved even though the chance of scoring is reduced. Similarly, every team that kneels down for 3 downs at the end of a game should get credit for the wasted series.

185
by Teddy (not verified) :: Wed, 10/21/2009 - 12:07pm

Actually, I think the current DEN-NE rankings look about right. They have almost identical ratings, with NE just slightly better. That lines up well with the results of their actual game, where DEN won in OT with HFA (and won the OT coin toss).

194
by ebinary (not verified) :: Wed, 10/21/2009 - 1:54pm

Well... except for the little problem of Denver beating NE (NE was shut out in the 2nd half, so likely the coin flip loss would have simply given the Broncos better field position after another stop).

And Denver has won 33% more games.

Eric

226
by Rich Conley (not verified) :: Wed, 10/21/2009 - 5:15pm

The game was in Denver. Denver is supposed to win that game if they're pretty even teams.

218
by Might as well be Micheal Lombardi (not verified) :: Wed, 10/21/2009 - 4:27pm

"Teams with tough defenses and a marginal lead will often run a much more conservative offense, because the chance of winning (the ultimate goal) is improved even though the chance of scoring is reduced"

Are you Gregg Easterbrook? Seriously, the Pats prove this theory wrong over and over, and made it an art form in 2007. For everyone who complained about the Pats running up the score in the snow, I'll remind you that the Chargers almost came all the way back against the Steelers when they went conservative a couple weeks ago.

If your team just scored 35 points in 10 minutes, why can't the other team do the same?? Especially in a snowstorm when the attacker has the advantage fo knowing where they are going!

26
by Kal :: Tue, 10/20/2009 - 6:41pm

I think one thing you could do is try to isolate the relative value of a position and (at a simple level) see how the improvement at that position could improve a team's offense/defense. Just do a plus, minus type thing, and say that on average, teams who get an improvement at position x improve their DVOA by y in the following year.

You can then try and figure out whether individual players are more or less likely to be an improvement. Sometimes that works great, some times it doesn't, but it should at least give you a statistical range to work with; you can assume (for example) that with replacing 7 defensive starters, assume that each is a 'better' replacement, and going from there that their upside is X.

I think also that figuring out why Philly is consistently rated so highly should be a top priority. I vaguely understand why; they consistently get first downs, do a good job of extra yardage, tend to be okay on special teams and defense...yet those intangibles always affect Philly. If it's this consistent week in and out, year in and out, there should be something measurable. Either something that hasn't shown a correlation with other teams but does in Philly or something that isn't being measured currently, such as clock management.

Hmm. That's another thought; can you quantitatively assign value to good or bad clock management? Might be interesting.

48
by bravehoptoad :: Tue, 10/20/2009 - 7:34pm

I believe this over-rating of Philly has only been the past 3 or 4 years; for the 3 or 4 years before that, Philly was under-rated by DVOA (if I'm recalling correctly) even though it was still the McNabb-Reid show.

Curses! If I had the premium database I could look it up in an instant!

92
by BigDerf :: Tue, 10/20/2009 - 10:02pm

Did any adjustments in DVOA come into play at the same time DVOA started overrating Philly? Cause those could be key.

107
by Pat (filler) (not verified) :: Tue, 10/20/2009 - 11:08pm

The before/after point for Philly is 2004/2005. Before 2005, the NFC East was a mess. After, they've been dominant. That's really the major change.

You may note that the other team that people constantly say DVOA overrates is Washington, the least-successful team in that division over the same period.

119
by Still Alive (not verified) :: Wed, 10/21/2009 - 12:15am

Personally I think people underrate Washington, not DVOA.

People see a team that many predict to be average or slightly above, then they finish slightly below average, and the conclusion always seems to be: "OMG Washington is so terrible! They are just awful."

Actually no. A slightly below average team isn't terrible, DET is terrible, STL is terrible, WAS is just bad, and plays below expectations consistently. But people love their hyperbole.

I know this sounds like splitting hairs but there have been several different occasions when I saw people lumping them in with the bottom tier of teams the last couple years. The difference between being 31 and say 23 may not seem like a whole lot when you are trying to make a flashy argument, but its 1/4 of the league.

159
by jebmak :: Wed, 10/21/2009 - 10:12am

But people love their hyperbole.

Our paper today refered to the H1N1 "Pandemic" sweeping the city.

Of course that could just be fear mongering to sell papers. Or that they have trouble with definitions.

201
by Pat (filler) (not verified) :: Wed, 10/21/2009 - 2:48pm

The virus's official name from the WHO is the pandemic H1N1/09 virus. They declared it a pandemic back in July.

267
by jebmak :: Thu, 10/22/2009 - 11:17am

Well then maybe *I* am the one who has trouble with definitions.

220
by Might as well be Micheal Lombardi (not verified) :: Wed, 10/21/2009 - 4:34pm

Nice trolling there, OP, but of course you realize that Detroit beat Washington and St. Louis only lost by 2 points. STL's losing margin to its other opponents is a whopping 16.8 points.

Washington is a pandemic of awfulness.

280
by Still Alive (not verified) :: Thu, 10/22/2009 - 3:40pm

We are mainly talking about PAST seasons. Also picking out 2 of their poorest results is an easy way to prove 3/4s of the teams int he league are "awful" You are the one trolling. Read the posts you are responding to.

27
by jimm (not verified) :: Tue, 10/20/2009 - 6:43pm

I suspect the most likely reason is coaching. Norv Turner has a career .446 winning percentage. He's two double digit wins in 11 seasons as a head coach. What do you think the odds were he would produce a 13-14 win team that it would take to be the best in the league.

As for Denver - six games in it still a very small sample but I suspect it's coaching and personnel decision making. But we'll find out if McDaniel is one of those rah rah guys that inspire the troops before falling back to reality - or is one of the next really good coaches. I think QBs are the most overrated assets in sports - anyone who dumps one and picks up additional assets to some other team - strikes me as understanding value. I wasn't too impressed with how he handled it in the media but he kept Marshall and he's the real superstar talent wise - Cutler is just another QB that could be replaced by 20 other guys.

28
by joenamath :: Tue, 10/20/2009 - 6:44pm

denver was hard to predict but as for san diego norv turner has proven he can't win as a head coach in the nfl
they were 14-2 the year before he came then 11-5 then 8-8 and this year?

29
by Jason Duby (not verified) :: Tue, 10/20/2009 - 6:45pm

With regards to Denver, specifically Kyle Orton, could it be that Josh McDaniels really is that good at coaching quarterbacks?

125
by RickD :: Wed, 10/21/2009 - 1:26am

Orton was always a decent QB, but not great. I think the big difference in Denver is how much better his receivers are. Chicago never had WRs like Marshall and Royal.

145
by xLittleP :: Wed, 10/21/2009 - 8:25am

I'm just throwing this out there, because I'm a just simple fan with a love for math, and I don't "see" football games the way most of the people here do.

But, as an Atlanta fan, last year we had an extremely surprising season. Sure, we had a new coach and a new GM, and they nailed the draft, and got us a new QB. Didn't some of those things happen in Denver? However, what most people don't realize is that our season last year was so successful to Atlanta fans because of the utter disaster that was the season before (Vick arrested, Coach quitting midseason). Perhaps some level of prior-season\off-season controversy can be factored in, possibly with the combination of a new coaching staff and QB?

Like I said, I'm just throwing it out there, but Miami got a new coach, too. Since I'm just an Atlanta fan, I don't know how much controversy there was in Denver and Miami previous to their surprising seasons. Miami might not look that hot this year, but then again, they don't have the "added the best Tight End of all time" factored in to their play ;-)

179
by Anonimoose (not verified) :: Wed, 10/21/2009 - 11:52am

I don't know how you can predict these sorts of turnarounds (considering how many times the Lions have failed to blow up and rebuild, for instance) but they seem to happen with decent regularity.

People have been mentioning the sudden appearance of competency from the Saints D, but if you go back just a bit farther there's an even bigger and surprising resurgence in during the 2006 season where they went from the 3-13 Katrina season to the NFC Championship the next year. Even if you throw out the 2005 season as a special exception and say no team could perform well in those circumstances, it is still pretty clear that they were at best an average team leading up to then.

The uniting factor in all of these turnarounds, I think is wholesale change. All new coaching staff, nearly complete turnover in personnel. Now, there are plenty of instances where this doesn't really work (I'm looking at KC this year), but maybe the thing to do is to lessen the strength of the previous season on predicting the upcoming season, perhaps even on a percentage basis depending on how much turnover there is.

It might just have to amount to throwing your hands in the air and saying it's impossible to predict if a complete rebuild will work, but here are our best guesses.

162
by Eddo :: Wed, 10/21/2009 - 10:35am

Having watched them both quite a bit this year, I'd say the bigger issue is the offensive lines. Orton's in Denver is excellent, Cutler's in Chicago is awful.

Orton does have better receivers, but that's all on Marshall. Royal is a quality receiver, but at this point, he hasn't impressed me more than Hester, Bennett, and Knox. Either Cutler's really making his receivers look better than they are, or the WR difference is being overblown.

But to repeat my first point: the Bears' offensive line is bottom-ten (maybe bottom-five), while the Broncos' is top-five. Never underestimate the crappiness of Frank Omiyale.

174
by bravehoptoad :: Wed, 10/21/2009 - 11:40am

You can look at Cutler as an ideal opportunity to test how important an excellent quarterback is. His switch to a new team is unprecedented -- who trades pro-bowl quarterbacks in their prime? How many times have we wanted to see what Payton Manning would play like if he were on the Raiders? With Cutler we have a (less extreme) example of that.

33
by deflated (not verified) :: Tue, 10/20/2009 - 6:59pm

As a possible partial fix for the problems with the Denver forecast, how about a bigger adjustment for regression with that defence. I suspect that with outlier units like last years Bronco D greater turnover in the players/coaches will cause a bigger regression to the mean; the more you change a very good/very bad unit the more likely it will swing the other way. It wouldn't predict the extreme swing they've had from historically bad to top-5 but it would help get a better W-L prediction for the team, given that last year's offence with even an average defence would be a pretty damn good team.

On the offence side you could give weight to an very good O-line. As Denver has shown over the last 10 years with RBs a very good O-line can make average talent look very good (and good look all-world). They returned intact one of the top 3 lines (plus coaching) from last season, dropping the offence all the way down to 25th seemed too far at the time.

241
by cjfarls :: Wed, 10/21/2009 - 6:47pm

These actually sound like very good adjustments to me.

Many of the "positive" things folks on MHR saw about the DEF were scouting things that I think would be almost impossible to quantify into a projection. I don't know how you factor in a remove "Bob Slowik Suck Factor" or a "Nate Webster & the safetys of suckitude factor" into last years defensive performance. Same with the "Norv Turner Suck Factor" for SD.... I just don't know that those will ever be quantifiable.

What I think many could say is it would have been almost completely impossible for the Defense to actually get worse... regardless if variables like "new D coordinator" and "changed personnel" typically are negative adjustments. Basically, for the really good/bad teams, perhaps a "complete change" variable should give a bigger regression to the mean. Except for the total orange & blue kool-aid crowd, no one expected the DEF to be more than mediocre this year however... no way anyone could have predicted that.

However, on offense, the projection really seemed screwy. Predicting a team to regress from ~5th to 25th, when they only changed 1 player seems absurd to me. Cutler is a good player, but he was no Peyton Manning... If anything, it should have been further mitigated by the fact that they wouldn't be running their 7th string RB, which while maybe not enough to replace Cutler, should have at least mitigated the drop.

This projection was especially goofy after we watched how QBs can come into a good team and succeed when we saw Cassel come in after Brady's injury.

Basically, I think many folks thought Denver would likely be mediocre on both offense and defense... which equates to a mediocre team. Projections that they'd be the "worst in the NFL" therefore seemed off, but I can't necessarily point to too many things besides the offense projection that should've been seen as likely wrong.

34
by joenamath :: Tue, 10/20/2009 - 7:00pm

should philly lose points for andy reid's game management

35
by Tundrapaddy (not verified) :: Tue, 10/20/2009 - 7:00pm

Dayumn! The '% change' in 'Playoff Odds' for both Seattle and Arizona is massive! One game can apparently change quite a bit.

Granted, much of that is probable the fact that 'DAVE' is slowly losing preseason projection strength, which might have been valid if DAVE could accurately project a 2nd consecutive season of O-line injury woes.

36
by Tim (not verified) :: Tue, 10/20/2009 - 7:03pm

I am still amazed that the Vikes variance is still so low. The defense was so dominant in the first half until the 2 minute warning and then were debacled late in the 4th quarter. Interesting.

In regards to SD, I believe the problem may be that a lot of their players, particularly on defense, are not as good as they seem. With a lot of solid players,, the weaker players are not exposed; now with the loss of Williams, I think we're seeing that the linebackers aren't as good as they seem. Although the chargers games I have seen this year have had terrible tackling, which certainly doesn't help.

I think a lot of Denver's success on D is obviously due to better coaching, particularly designing the defense for the players present. They had so many D coordinators in the last few years trying to force players into the scheme, drafting for that scheme and then the next year that player doesn't fit. The 3-4 is inherently more flexible and allows Nolan to put his players into a position to succeed.

I don't think Denver's success could have been predicted, although i think it was clear that Orton could succeed with a good X and O's coach, an actual NFL O line and some viable receivers, things he did not have in Chicago.

39
by MarkV :: Tue, 10/20/2009 - 7:10pm

RE Denver - I don't have the data to see how common it is to start 8 new defenders, or hiring 2/3rds of a new defense. FO has looked at new schemes, but I think changing scheme might additionally help to increase this fact. I would suggest that THAT large of a turnover would have the effect of mostly randomizing a teams performance, or that if it provides any consistency that it is towards average (a mostly new defense in a new scheme might not correlate to the previous regime very effectively). The numbers could prove this one wrong, but it seems possible to me.

Or maybe Jamal Williams is REALLY REALLY good, and Marlon McCree is really really bad to ways that statistics just can't quite compensate. Ok, that ones not quantifiable.

40
by mrh :: Tue, 10/20/2009 - 7:16pm

If a unit (offense or defense) loses a number of starters, you COULD assume that the new starters will play at replacement level. If one 16-game starter leaves, that would mean 1/11 of the unit is now assumed to be replacement level. So the predicted DVOA would be adjusted by a factor of 1/11 * (last year's DVOA - replacement level DVOA). I don't know what "replacement level" is for a defense or offense since DVOA is expressed in terms of average vs. replacement level but presumably Aaron could calculate that. Maybe 1/11 is not the right coefficient but that's a start point. Essentially, this is an estimator of the regression to the mean that would be expected in the absence of other factors already included. If the unit well below replacement level (DEN D), this would give an idea of the magnitude of the expected regression (improvement).

Using the known player losses on the DEN D at the time of the Almanac (players with * by their name), I count 87 known starts "lost" out of 176 total starts on defense (using pfr start data). The DEN defence could be predicted to improve by an estimated 87/176 * (DEN 2009 DVOA - Replacement level DVOA). Obviously it won't work out that all lost players are below replacement level or that all the new players will be at replacement level, , but it might yield a useful adjustment factor.

42
by Paul A (not verified) :: Tue, 10/20/2009 - 7:20pm

After forcing myself to watch the entire Redskins game, I must say that I miss the good old days when Norv was calling the plays.

44
by Adam (not verified) :: Tue, 10/20/2009 - 7:27pm

Re: Denver Defense.

Bob Slowik is the worst defensive coordinator in the history of football. It didn't help that most of the defense players were also garbage, but take the core group of good players, add some role players, throw in a real defensive coordinator and guess what, you have a MUCH better defense and anyone who did not see an IMPROVED defense coming has no idea about football but has their head buried in stats.

Now, to predict their turnaround THIS drastic no one could have predicted but FBO predicted Denver to be the worst team in the league based on...nothing. They looked at stats, but failed to look at the most important part of the equation, the removal of a completely inept moron and the removal of completely inept players.

45
by Raiderjoe :: Tue, 10/20/2009 - 7:28pm

something still wrong with compter. now ay can Raiders beat egales and only go from 32 to 31. Raiders have 2 wins and getting to 3 next week when beat crpapy jets team,. Raiders kmore wins than rams, Titans, Chuefs, Bucs, lions, Rams, and Browns and have same wins as panthers, redskinbs, Bills, Seagawks, and dolphins. Cimputer clauclations really off if it deceide Raiders are next to worse team in league.

46
by ChiTown11111 (not verified) :: Tue, 10/20/2009 - 7:32pm

The way to get philly in line is to double/triple/quadruple the value of offensive goal line plays. Thus when they fail to get it in the endzone because no one is falling for the Westbrook screen they drop like a rock.

People wonder why Philly didn't run more against Oakland, its because they SUCK at running.

47
by Brendan Scolari :: Tue, 10/20/2009 - 7:34pm

"We're looking for objective facts that we knew before the season that should have been indicators that Denver's defense would make a colossal turnaround, or that San Diego wouldn't be the best team in the league"

Perhaps there's a hint at the problem in that statement. I commented about this a couple weeks ago, but maybe DVOA isn't weighing last season's performance enough. While Denver's defense was truly terrible, Denver was 8-8 as a team. Why were they expected by DVOA to be the worst team in football? They switched Cutler for Orton but Cutler isn't Peyton Manning or anything close to it and Orton actually played fine last year before he got hurt.

I bought into the Denver will be terrible this year theory but looking back now it seems foolish. The downgrade from Cutler to Orton isn't that big and the rest of the offense is still elite. And the defense should be expected to make some improvement just by replacing players, at the very least they couldn't do any worse.

As for San Diego, what evidence was there that they'd be the best team in the league? They were 8-8 last year too! I know they had a higher DVOA than Denver but still it's not like they looked like a great team at any point last year. Merriman and other guys coming back on defense should have helped, but I don't think that was all the difference.

There's obviously some bad luck involved, who knew before the season Merriman would be terrible or Dumervil would be the defensive MVP? But still, just based on last year there was no reason to think there was such a big difference between Denver and San Diego.

175
by bravehoptoad :: Wed, 10/21/2009 - 11:41am

...but Cutler isn't Peyton Manning or anything close to it....

There's a small wave of fashion to say that Cutler isn't anything special in light of his performance on the Chicago Bears. No one was saying he was crappy this summer. He may not be in Peyton Manning's class, but he was firmly considered to be in the class right below him, with Rivers & Palmer &c.

Strangely, no one's saying, wow, look what Cutler's doing with an offense that could make the great Kyle Orton look terrible.

222
by Might as well be Micheal Lombardi (not verified) :: Wed, 10/21/2009 - 4:53pm

I was saying he was crappy! And Orton looked really good running that offense before he got hurt last year, despite his horrible receivers.

Sorry, what has Cutler done that has been more impressive than Orton's first half of 2008? Beating the Lions? Beating the Seahawks? Throwing horrendous picks to cost games against the packers and Falcons??

I will cut Cutler some slack in that Orton didn't have the taxedermied corpse of Orlando Pace playing left tackle for him. Yikes.

Protip: Nobody releases good left tackles. The Rams cut him becasue he was D-O-N-E.

Seriously, I'll wager Cutler has pouted his way out of the league by 2011 unless he has some kind of Brees-esque transformation. But the more he's rewarded for sucking, the more he will suck.

230
by Rich Conley (not verified) :: Wed, 10/21/2009 - 5:26pm

"will cut Cutler some slack in that Orton didn't have the taxedermied corpse of Orlando Pace playing left tackle for him. Yikes."

He doesn't just have the Corpse of Pace in front of him, he also has the Corpse of Kreutz in front of him. Kreutz snaps every 5th snap or so directly into the ground. At one point he was a terrific blocker and made up for it, but at this point, the only player who I'd have any interest in is Williams at RT, who looks good. The rest of the line is below replacement level.

Cutler is doing a terrific job at this point.

51
by Avenger00782 (not verified) :: Tue, 10/20/2009 - 7:41pm

Is there a way to account for team unity? How many times have events occured where something has unified a team and they go on a run, or divided a team and that team has fallen appart. Denver looked divided last year, but have somehow managed to unite under the new coach. Jay Cutler had divided the Broncos lockerroom, and we have a situation where there was addition by subtraction of the divisiveness of Cutler, the rotating D cordinaters and the head coach who was enableing both of those problems. This might also explain San D and Philly. San D has a bunch of players that are not sufficiently motivated to play well together until they have no other choice. On Monday, LT threw a fit because he wasn't being used on the goal line. Meanwhile, Marshall was giving up his body blocking linebackers on the run plays. Every game, a different Bronco is Orton's primary reciever. It seems like everyone understands that Orton is going to exploit the biggest mis-match. Now it may be possible that McNabb is one of the divisive influences in the Eagles lockerroom. It does seem like the Eagles do better when Donovin is injured.

165
by bravehoptoad :: Wed, 10/21/2009 - 10:46am

Jay Cutler had divided the Broncos lockerroom....

What makes you think so?

53
by gravning :: Tue, 10/20/2009 - 7:53pm

If you're really looking for suggestions...

...is it possible that you just need to weigh the coaching staff's impact a lot more heavily? Can the answer be that simple?

Because I think it would be wrong to dismiss "Norv Turner sucks" as a theory. I'd look at the math and juice the numbers.

54
by sfckoski :: Tue, 10/20/2009 - 7:53pm

Is health the answer?

The Chargers haven't held a team under 100 yards rushing this season. Denver had 101 and likely would've had a lot more if not for the two kick return TDs. In 2008, the Chargers held teams to under 100 yards rushing in eight of eighteen games (including playoffs)and won six of them. If you extend the RYA up to 111, the Chargers were 9-3 in those games.

I know Jamal Williams was injured in game one of 2009 and I'm not positive how much of the game he played, but the Raiders accumulated 148 rushing yards that game. Shawne Merriman has been very, uh, Lights On(?)this year and who knows how much should be attributed to his knee, but he has been invisible. Cromartie was supposedly nicked up most of last season and may not be fully healthy this year.
I haven't looked up the specifics, but is Igor Olshansky a lot > Jacques Cesaire? It sure seems like it.

The Jets minus Kris Jenkins should provide a good case study on the impact the loss of an immovable and irreplaceable NT is to a defense/team.

My two cents, is that injuries to key positions is what is keeping the Chargers away from projections.

74
by Scott C :: Tue, 10/20/2009 - 8:41pm

Ryon Bingham was supposed to be Olshansky's replacement. He went on IR in the pre-season. Cesaire was the rotating backup who played downs at end and nose but wasn't a starter. The new young guys are projects at best.

So, compared to last year, THREE d-lineman were lost that played significant time:

Olshansky, Williams, Bingham.

Castillo is quite good, but he is only one man.

202
by bravehoptoad :: Wed, 10/21/2009 - 2:51pm

Interesting idea. We know the FO projections are testing for injuries. Are they weighting injuries to key positions? Such as NT for a 3-4, or quarterback, or maybe middle linebacker in a Tampa-2? Maybe rush linebacker in the 3-4, rush end in a 4-3, etc?

EDIT: Whoops, they use something called HGL for that. It looks very cool...I'd love to see more about it on the site.

55
by LetsGoJets (not verified) :: Tue, 10/20/2009 - 7:57pm

I don't see much of a problem with the Eagles as the second best team. I'm a NY Jets fan, but the Eagles were outstanding last year. If that pass interference no-call is called Rogers-Cromartie on 4th and long in the NFC Championship game, the Eagles would have gotten to the Superbowl and won it all. I think the Eagles might just be off to a shaky start with their injuries, but they always seem like the Patriots to me. No matter how many players are hurt, their system makes them one of the best.

The team that shocks me the most are the Jaguars at 15. They got trambled on by the Cardinals and Seahawks, beat the Texans because of a Chris Brown fumble at the one, and almost lost to the Rams. I would have expected JAX in the bottom three, at least on the Oakland/Cleveland level of bad teams.

56
by Karl Cuba :: Tue, 10/20/2009 - 8:08pm

I still think that Denver are being flattered by record. They should have lost to the Bengals, that game was a fluke. Even though the Pats just destroyed the Titans, I still think that they're a team with some issues and the Broncos only managed to beat them in overtime at home. McDaniels has a rather extensive level of insight into the Pats inner workings while BB would not have any idea what he has kept or changed from the Pats scheme. The familiarity would have been to Denver's advantage. The win at home over Dallas . . . meh, even if Dallas are all that good, (a big maybe) Romo is capable of throwing the game away against anyone. Wins over Cleveland and Oakland, who could care (Philly?). The Chargers are not the same team, losing two of their starting DLs, plus two return touchdowns is a little unsustainable. One thing that could have been predicted is the slow return of Merriman, which coupled with injury leaves the bolts with 40% of the front five from two years ago.

However, Denver are a better team that last year. For years they'd have been great with a defense and now they have a competent unit with the elderly secondary benefiting from a healthy Champ Bailey. Orton is interesting, he is probably benefitting from better coaching, my opinion of Ron Turner is not great, and he also has better personnel. I reckon Denver will finish 10 - 6.

71
by tunesmith (not verified) :: Tue, 10/20/2009 - 8:33pm

I think you could talk down any team if you went about it that aggressively. They've got two KC games, home games against OAK and KC, and WAS left on the schedule. There is no way you would have picked Denver beating San Diego before yesterday.

127
by RickD :: Wed, 10/21/2009 - 1:32am

I picked Denver to beat San Diego. The coaching matchup was a rout!

Seriously, San Diego has historically beaten Denver by running on them. Denver's no longer super-vulnerable to the run, and Tomlinson is basically dead as a big-time RB. That leaves us with Rivers vs. the excellent Denver secondary.

I figured it'd be close, but I've seen Norv lose too many close games to have any confidence in him. In any case, this result is part of this casual observer's theory of the three-year plan by Norv to turn a Super-Bowl caliber team into an also-ran.

When a loose coach like Norv takes over from a disciplinarian like Schottenheimer, you typically have one year of excellent results, and then the lack of discipline begins to kick in. I don't see the Chargers making the playoffs this year, and I expect this will be Norv's last season.

61
by C (not verified) :: Tue, 10/20/2009 - 8:25pm

Eagles: Not only is their team rated too high, their defense at #1????? The Giants defense gets torched by the Eagles and drops down, the Eagles lose to the Raiders and move up? You are kidding right?

What does DVOA like about a team playing down to their competition, losing to a good team and a terrible team, and throwing what, 27 or so incompletions? How does that move the chains on a per play basis?

Denver: Brendan Scolari makes a fantastic point... Denver was 8-8 last year and they really should have been 9-7 if not for a bad Ed Hochule call! They had a great offense ( that was predicted to get worse), and they had a terrible defense that was to regress better...

So why were they supposed to go from mediocre to terrible? Mike Shannihan & Cutler

Offense: They have a fantastic offensive line, and Shannihan did a good job of moving the QB in waggles, roll outs, running TE screens, RB screens etc. to slow down a D-Line and change that point of target for rushers. Shanny has always have a very QB friendly system in that ( I see the same with Kubiak in Houston) and I consider this a strength.

Denver has good Receivers in Marshall ( a game breaker) and Royal ( solid), and Stokely solid at his role.... Scheff is a good tight end. The Broncos have good offensive talent if you look at the team minus the QB...

The big question mark was Orton, who has done alright... he's surrounded by good players, a good coach, and the offense probably isn't as explosive as they were last year but they aren't committing turnovers ( yet).

The defense: So far nobody mentioned Champ Bailey has been healthy... Getting back one of the top defenders in the NFL helps... I couldn't break down all of the X's and O's yet, but Nolan has new players, a new scheme and it's working so far...

Denver was predicted to fail because
1) People know Shanny is a top notch head coach and they lost him.
2) They lost Cutler (Orton hasn't been as explosive, but he hasn't had the turnovers either).

I'm still not a big believer in Denver. I know you are admitting you were wrong on the team, but I did see in one of the football preview mags one editor that was high on Denver... so high he might have predicted them to win the west, and upset their way to the AFC championship...

So my analysis says that right now Denver is overrated... before the season they were underrated due to the (admittedly ) important Coaching & QB change to an average team... The jokes were mean too!

SD: To run a successful 3-4 you need good D-Lineman and in particular a good Nose Tackle in the middle. Jamal Williams was one of the best, and you took away a key cog in that defense and replace him with a terrible player that gets pushed around and probably should still be working at the rent-a-car service. They also lost Igor Olshansky. Igor, Fire Heidrant and Castillo were a big part of that defense in allowing Merriman and company to do their THANG. Now they can't... It puts stress on the whole unit.

Also, why all the Norv hate? It seems like such a pop opinion and not like something you'd see from FO. Some Denver fans were critical of the work done on Denver saying it was "lazy" and "nothing you wouldn't see from any other football website", and I think you could say a lot of the same stuff about the Chargers analysis. "Norv sucks".

Why do people hate Norv so much? Because he's a "nice guy"? Because he doesn't yell at his players, because he doesn't LOOK like a football coach? At the very least he's a good offensive play caller. I don't think he's a great coach myself, but people hate him like he's the worst thing ever... I've met Norv and he's a nice guy...

75
by Bowl Game Anomaly :: Tue, 10/20/2009 - 8:42pm

Nobody thinks Norv is the worst coach ever. The thing about Norv is that he's proven over a long period of time that he's clearly not a good head coach and yet he's still got the job somehow. The longer he remains a HC, the more bewildering it is to people who know he's below average at best.

If he were REALLY bad, he'd never have made it this far.

128
by RickD :: Wed, 10/21/2009 - 1:35am

"Nobody thinks Norv is the worst coach ever".

I live in the DC area, and I know people who would have disagreed with you - at least until the Zorn era started.

Norv is, admittedly, well above the Richie Kotite level.

77
by Scott C :: Tue, 10/20/2009 - 8:45pm

In the 2007 PFP, they did a piece on Norv's history as a coach.

Statistically, he has one of the worst W-L to pythagorean wins ratios, one of the worst "hold the 4th quarter lead" ratios, and a few other things that statistically jump out as among the worst coaches in history.

So, its not just hate, its some facts too.

160
by goodhit (not verified) :: Wed, 10/21/2009 - 10:22am

Actually, Denver benefited from that blown Hochuli call, if you please.

255
by cjfarls :: Wed, 10/21/2009 - 11:01pm

I'll agree that Denver could be a bit overrated at this point. They will not be undefeated the rest of the year. But given the commanding lead in the division, loser teams ahead (4 games vs. KC, OAK, WAS), etc. they're pretty much a playoff shoe-in at this point.

Denver may not be dominant like some other teams, but they look like a very solid team that can play anyone tough. Just like AZ last year, that could easily translate into a hot play-off run and potential SB.

Norv may be a nice guy, but his results speak for themselves... a decent OC, that as a HC is decidely less than mediocre.

64
by Ven (not verified) :: Tue, 10/20/2009 - 8:27pm

Wow, talk about worthless "rankings".

Philly loses to a pathetic Oakland team, doesn't even score a TD.
DVOA remains virtually the same? Huh?

And I'm not saying NE isn't good, but beating up a winless team 59-0 that had 3/4 of their secondary as practice squad guys moves them up so much?

You guys should find something more productive to do with your time than come up with worthless stats like these. The Philly thing is laughable.

81
by Raiderjoe :: Tue, 10/20/2009 - 8:53pm

doubler

80
by Raiderjoe :: Tue, 10/20/2009 - 8:52pm

Eagles dsidnt move much? Raiders didnt change either. Last week Eagles like 3 or 4 in dova and Dave and then Raiders beat them but Raiders only go up from 32 to 31. Raiders clearly better than 7 or 8 teams so for to have them at 31 is worng.

91
by Anonymouse (not verified) :: Tue, 10/20/2009 - 9:57pm

Yes, sir, you know you've got a winning comment when Raiderjoe agrees with you.

68
by Still Alive (not verified) :: Tue, 10/20/2009 - 8:31pm

1) I wouldn't sweat this year's results at all. Denver was unforeseeable preseason, and the Eagles as completely schizo if I understand anything about the NFL. While I completely disagree with the Norv bashing (I think he is probably pretty average, and think that is what the evidence points to as well), I did find the SD prediction odd. So I guess the SD prediction is where I personally would start looking for errors. What was being overvalued there? I don't have your data so I have no idea, but I would isolate the things most positive about SD and re-evaluate them.

2) Along the lines of what many people have hinted at or mentioned above, but perhaps not outright said, I think team psychology is playing some part.

I think for a team like Denver that has had a lot of changes (but really for any team) winning a game early is very important to future success above and beyond what the win itself means. People buy in more try harder etc. Perhaps long strings of wins might lead to complacency, but long strings of losses definitely decrease a team's performance.

In a rather mundane example in our adult hockey leagues teams that lose their first few games tend to have a lot of trouble even turning out all their players for future games. Obviously NFL teams don't have that problem except with fans, but in one way or another the same effect occurred at every level of competitive sport I have played. Early wins lead to effort and seriousness, losses lead to discouragement, and indifference.

I have a hard time imagining that some acknowledgment of recent game results (not play-by-play) would make DVOA more predictive, but it seems so simple I am sure you have probably tried it.

To just throw some made up number out there as an example:
DVOA/DAVE *1.05 if last game was a win
DVOA/DAVE *.80 if last three games were a loss

I would think there would be some way to work that data in and get a more predictive system, but I also know enough about statistics and the lack rigor I put into this post to realize that I could just be 100% off base.

my $.02

69
by Danny Tuccitto :: Tue, 10/20/2009 - 8:33pm

Hey Aaron, it's Danny from Niners Nation.

From previous analyses I've done based on 2002-2008 DVOA, I've found one thing that might help explain DEN's defensive turnaround. Basically, I've found a distinct regression to the mean for DVOA from year to year. What's more interesting is that the amount of regression to the mean is predicted by the actual value of DVOA and its OFF and DEF subcomponents. The two subcomponents that are affected most are pass defense DVOA and pass offense DVOA. As "deflated" (comment #33) hints at, the lower a pass defense DVOA in year N-1, the larger the regression to the mean in year N. DEN's pass defense DVOA was 42.4% last year, so it stands to reason from what I've found that they were going to have a huge leap. Just going from 42.4% to 0.0% in pass defense DVOA fully accounts for their 44.1% turnaround in total DVOA. My analysis didn't predict THIS big of a turnaround overall, but I think that was because their pass offense DVOA regression to the mean offset the pass defense's. So I guess my point here is to perhaps look at pass defense DVOA regression to the mean in year N being a function of acual pass defense DVOA in year N-1.

One other possibility that I've noticed anecdotally watching the 49ers over the past 5 seasons, though I haven't done any analyses on it, is the indirect effect switching to a 3-4 has on pass defense DVOA via an improvement in ASR. Basically, DEN's ASR has gone from 19th at 5.8% last year to 2nd at 9.8% this year. Looking at last season's sack stats for players who had 4 or more sacks, the vast majority of sacks (i.e., 75%) came from 3-4 OLBs and 4-3 DEs. However, the 16 3-4 OLBs w/ 4 or more sacks averaged 8.6 sacks, whereas the 44 4-3 DEs w/ 4 or more sacks averaged 6.4 sacks. So it stands to reason that switching from last season's 4-3 to this season's 3-4 would have had a positive effect on DEN's player sack totals. This has obviously happened in re Dumervil, but that's retrospective. The part that I haven't researched is the next link in the prospective indirect effect; namely, the effect of ASR on pass defense DVOA. Of course, it's pretty obvious that ASR is going to positively predict defense DVOA in a given season because sacks are accounted for in DVOA on a per-play basis. However, what I'm not sure about is the effect that CHANGES in ASR from season N-1 to N have on changes in pass defense DVOA from season N-1 to N. So, to make a long story short, perhaps an additional objective reason for DEN's defensive turnaround is a causal chain like "switch to 3-4 from 4-3 --> increased ASR --> improved pass defense DVOA." Add this to my first point about massive regression to the pass defense DVOA mean, and i think it's a descent start and figuring things out.

131
by ammek :: Wed, 10/21/2009 - 3:13am

So it stands to reason that switching from last season's 4-3 to this season's 3-4 would have had a positive effect on DEN's player sack totals.

That depends on the personnel. Aaron Kampman, for example, might take issue with you.

132
by Jerry :: Wed, 10/21/2009 - 4:05am

Nice post, Danny.

I'll defer to your solid statistical analysis, but if the 3-4 is really more effective than the 4-3, than nobody would be running a 4-3 anymore. Like most things, it involves having the right players for whatever scheme you run.

157
by Temo :: Wed, 10/21/2009 - 10:02am

The variable isn't "Teams that run the 3-4", it's "Teams that switch to the 3-4". It may point to certain players been used in far different roles than they are usually asked to perform, and the positive effects that this may entail.

171
by Karl Cuba :: Wed, 10/21/2009 - 11:28am

Danny, love the Niners Nation blog. Don't agree about with what you say about Nate Clements though, I just think he's asked to do too much with awful safeties behind him. That 90 yard catch by Roddy White sould only have been a 20 yard gain with a competent safety playing single high (ie not Mark Roman).

70
by Adrian (not verified) :: Tue, 10/20/2009 - 8:33pm

There HAS to be a coaching metric or play calling metric added into DVOA eventually.

Example 1: The Oakland Raiders until Sunday had given up the second most rush yardage in the NFL this year. The play calling from the Eagles? 54 passes and 12 rushes (the passes include drop back situations; source: Tuesday Morning Quarterback). I know the Eagles are a passing team, but c'mon! McNabb ended up being 22 of 46, so even those short passes didn't work. You have to penalize a team somehow when their coaching staff doesn't put them in a position to win the game.

Example 2: In the first half of last night's game, the Broncos called all out blitzes numerous times leaving the short, middle passing lane wide open. Twice Rivers made them pay with short dump offs that turned into long gains. In the second half, the Broncos realized this full blitz wasn't working, ended up blitzing the 'A' gap from a zone blitz structure and left spies out in the short, middle passing lane for incoming backfield receivers. The result? 0 third down conversions and 3 points for the Chargers in the second half.

If FO can add a metric of first half to second half improvement, play calling versus strengths and weaknesses of opposing teams, clock management (blowing time outs when unnecessary, not using timeouts when essential), replay challenge success rate, and perhaps a qb success rate under a certain coach. Maybe all of these components can add a "Coaching Efficiency" to the overall DVOA.

Maybe a muppet could coach some teams: '85 Bears, early 90's Cowboys, 70's Steelers. However, there are teams nowadays where the coach is just as essential to the team success rate as the starting quarterback or the defense's third down efficiency rate.

72
by The Guy You Don't Want to Hear (not verified) :: Tue, 10/20/2009 - 8:36pm

I think perhaps everything I would say has already been said, but since I'm a Denver fan who picked the Chargers to win the Super Bowl, I'll throw in what I'm thinking about where the problems could be.

I'll start with Denver:
I know you've done studies on the effect of hiring a new head coach, but I think it's unusual for a new head coach to take over a team that went 8-8 with a great offensive line, a true number one receiver, and a young QB in place. There has probably been at least one in the past, but the aggregate is going to look very different, since a new head coach is usually taking over a truly bad team.
I also would echo many others about the defense: There was just so much turnover that it was all but impossible to guess what was going to happen.

San Diego:
First, there was no way to know that Jamal Williams would go down and it at least seems like the nose tackle in a 3-4 is disproportionately important to a defense.
Second, Merriman didn't look like the old Merriman last night (that's the only time I've seen him), so if that's how he's been all season, they didn't really get Merriman back. They got back 60% of Merriman instead. It at least sounded like the projection system was expecting Merriman back to being at least near his old level.

73
by Zheng :: Tue, 10/20/2009 - 8:41pm

I'd be interested in seeing if FBO can come up with a metric or adjustment for team morale. It might better account for letdowns after big wins, or improved play in rivalry situations. This season it seems as though morale might partly explain the Titans, Broncos, and 49ers, at minimum.

78
by Bowl Game Anomaly :: Tue, 10/20/2009 - 8:47pm

And while they're at it, it would be great if they could create a metric for intangibles. I know intangibles are critical to success and failure, but I've never seen anyone quantify them.

100
by dbostedo :: Tue, 10/20/2009 - 10:41pm

Yeah...and I see a lot of talk about "swagger" but I can't seem to find it in the team stats tables. Can someone help me out?

108
by anotherpatsfan :: Tue, 10/20/2009 - 11:12pm

I see what you did. Intangibles. Metric. Quantify. Nice.

82
by jmp (not verified) :: Tue, 10/20/2009 - 8:58pm

Week 2 DVOA Ratings by Aaron Schatz

So, how about those New Orleans Saints? Two big wins give Drew Brees and his boys first place in the Football Outsiders DVOA ratings after two weeks. The Saints are first in offense and third in defense -- isn't playing Detroit before opponent adjustments kick in great? --

It is amazing how much better they are now ............Who dat?

96
by Raiderjoe :: Tue, 10/20/2009 - 10:21pm

shows to show you it doesn't matter what team Saints played in first 6 games. scored vs crap team detroit, scored vs giants, had torubles vs jets but still won game

180
by mm (not verified) :: Wed, 10/21/2009 - 11:53am

isn't playing Detroit before opponent adjustments kick in great? --

The current DVOA should include an opponent adjustments for the Detroit numbers, though not yet at the full value yet...every week it's recaculated. Anyway, the opponent adjustments for the Giants and Eagles should help the Saints as much as adjustments for the Lions and Bills hurt.

84
by VarlosZ :: Tue, 10/20/2009 - 9:07pm

I suggested this last week, but I imagine it went unseen.

For Denver, perhaps you could look at a very specific subset of new coordinators: those who had previously been hired as head coaches (mid-season interim replacements excluded). Probably some sample size issues, but if they're not too severe it could be worth taking a peek.

85
by Steve (not verified) :: Tue, 10/20/2009 - 9:09pm

A few thoughts (probably repetitive) regarding the Chargers under-performance.

Between FA and Injuries, they lost 3/5ths of last year's D line rotation. Cesaire is still there, but he was limited both in training camp and in the beginning of the season. They traded for a Texans cast-off to try and short things up, and he promptly got hurt too. Their formerly reliable, veteran group has been replaced by an UDFA, a 4th round project from Canada, and a Chiefs castoff. Luis Castillo remains the only steady presence on the D-line, and while he's actually played well he faces constant double teams and has never been a great pass rusher anyway.

Injuries haven't spared the rest of the D either. Merriman was re-injured and still hasn't returned to form, and has missed significant portions of games. Steven Cooper, their leading tackler the last two years, is also hurt. Finally, Clinton Hart played so poorly that he was cut this last week. You know things are bad when you cut a starter in week 5 and his potential replacements are a career backup, an UDFA, and a 6th round rookie.

On the other side of the ball they lost their starting center and guard (who was a rookie), plus LT, who has really only played 2 and a half games and never really had a shot to get going in the Pittsburgh game. As explosive as Sproles is in the open field, he's not a real running threat, and most of the league has caught on to the fact that you better keep an eye on him as a receiver.

But injuries are only part of the story. On defense, the tackling has been suspect much of the time, and the lack of a pass rush is a real mystery. The Chargers rarely blitz anymore, instead relying on 3 and 4 man rushes much of the time even in obvious passing situations. The result is no pressure, no sacks, no turnovers and a terrible 3rd down percentage. I have to hang both of those problems on D-coordinator Ron Rivera. The thing is, though, these were the exact same problems they had last year (and parts of the year before). So maybe it shouldn't be surprising at all: take a mediocre to bad defense, subtract several key parts, and watch it fall apart.

As for the offense, its all about the Red Zone, which is where Charger drives go to die. Their Red Zone percentage is awful (not helped, I'm sure, by the absence of LT and playing the teams they have played). This is great if Nate Kaeding is on your fantasy team but not good for winning football games. They also don't do a great job up front of either run blocking or pass protecting, except in max protection. They have three rushing TDs all year: one on a one yard dive from LT, and two on draw plays in obvious passing situations. Essentially their offense has consisted of Rivers throwing it deep with no hint of a running threat, and that's just not sustainable, especially against some of the pass rushes they have faced.

Of course, after all is said and done, they have been in every game. The Ravens game came down to the wire. The Steelers game was awful but they did have a shot at the end if they could have gotten one stop or a turnover.

In last nights game they gave up two special teams TDs, something you just can't count on, especially given that the Chargers are typically strong on kick coverage. They again struggled in the red zone, converting just once. And they had two legitimate drives in the 4th quarter which could have put them ahead. In the end its a failure of execution on a few key plays which has been the ultimate difference, and I think at least some of this blame has to go to Norv and his playcalling/scheme.

88
by Rick A. (not verified) :: Tue, 10/20/2009 - 9:51pm

What's "weird" about Philly rating higher in DVOA than their results on the field? McNabb has been a career long inconsistent QB. And what do people not get about the Broncos? They got rid of a coach that was killing them, now they have a great defensive mind in Mike Nolan and a former college QB at head coach. I like him and, obviously, so do the players. Furthermore, Dawkins adds to an already talented unit that had been torpedoed by the aforementioned previous over-rated coach. Those guys had been on the field way too long during the Cutler era with all of his all or nothing style, you know, turnovers or really quick strikes.

As for the Vikings secondary getting torched Sunday--yes, it's obvious that is their weakness--but isn't it everyone's in today's football ? The rules favor offense in a huge imbalance. If you can stop any team's pass rush you're going to get them in today's game. The Ravens did what they had to do with No. 69 and it started becoming a different game. Not a whole lot of teams are going to be able to do that with him, and then, on top of it, be able to get those long downfield strikes that Flacco got. The fact is the Vikes are still undefeated, so on which team would you guys like to be ?

Those rule changes, combined with free agency, are going to make for lots more of what we are seeing this year--i.e. several really hot teams, and several pitiful excuses for teams. And there are going to be more and more ridiculously non-competitive games like the Pats/Titans yesterday, or the Giants/Raid the week before. It's becoming a parallel with baseball, naturally, now that we are a few years into the free-agency era. When the season begins there are really only a few teams that have a chance. This is completely the diametric opposite of what Pete Rozelle wanted and got when I was growing up. Parity is now the foggy, misty history. And DVOA need only measure a few teams, the rest is superfluous nonsense...

118
by Jovins :: Wed, 10/21/2009 - 12:03am

About the Vikings/Ravens game.
I'd hazard that the apparent secondary vulnerability was less a matter of containing Jared Allen, and more a function of having Antoine Winfield, one of the better corners in the NFL, get injured and be out for the game.

144
by crack (not verified) :: Wed, 10/21/2009 - 7:34am

And Winfield's normal replacement, Sapp, went out with a concussion. A 3rd string corner was on in non-nickel situations.

152
by peterplaysbass (not verified) :: Wed, 10/21/2009 - 9:37am

Something called "Karl Paymah". Why they didn't throw Asher Allen out there, I'll never know. I hope he gets the start this week at Pittsburgh if the Vikings wisely let Winfield rest before going to Green Bay in week 8.

173
by Pacifist Viking (not verified) :: Wed, 10/21/2009 - 11:40am

Allen was inactive for this game; I'm guessing Paymah was active because of his special teams role, and the coaches didn't expect him to have to play CB. I would certainly hope that Allen is active for future games, because he might suck, but I'd be surprised if he sucks as bad as Paymah.

182
by bravehoptoad :: Wed, 10/21/2009 - 11:55am

The rules favor offense in a huge imbalance.

Completely disagree.

The rules favor defense in a huge imbalance. If you really wanted to level the playing field, you'd get rid of the ridiculously one-sided rule that says defensive lineman can shift before the snap and offensive linemen can't.

You'd also get rid of the single most imbalancing rule in football, the one about "illegal" men downfield. This guarantees that the offense is playing virtually 5 men down on every pass play. Let those centers release up the field like they would be able to if the rules weren't so biased in favor of the defense. Then you'd see some real, fair action.

210
by Pat (filler) (not verified) :: Wed, 10/21/2009 - 4:07pm

I think it's worse than that. The game itself favors defense, because on the majority of plays, the defense has a numbers advantage.

That's one of the reasons the good Xs and Os guys have stressed that the Wildcat is likely here to stay, because it works to offset that by getting rid or mitigating the useless player on a run. Problem is that running plays just aren't that great, because you can't utilize the full field as well, which tilts in favor of the defense again.

That advantage for the defense just gets magnified as the players get more athletic, which is why they've had to continually adjust the game to protect offenses to keep scoring at a constant level.

As an aside, you also left off another of the big advantages the defense has: when shifting, they can be moving towards the line at the snap. The offense can't.

246
by bravehoptoad :: Wed, 10/21/2009 - 7:48pm

That advantage for the defense just gets magnified as the players get more athletic....

A corollary might be that another way to protect offensive scoring would be to enlarge the field.

253
by Pat (filler) (not verified) :: Wed, 10/21/2009 - 9:50pm

Yup. And that's why when football was forced to find a way to increase the safety (i.e. reduce the numbers of collisions = improve the offense) there were two main suggestions: enlarge the field, or allow the forward pass.

90
by Rick A. (not verified) :: Tue, 10/20/2009 - 9:54pm

Oh yeh, and Norv Turner could screw up a round steel ball. He's been a career long really bad coach. DVOA might as well not bother with attempting to rate the Bolts. They were finished as a force the moment they hired him. This third year will finish them off completely. There will be no comeback for them this year...

93
by Raiderjoe :: Tue, 10/20/2009 - 10:05pm

N Truner and L Kiffin worst two Raiders head coached ever. j Bugel had rough year in 1997 but tema was better than record, played more like 7-9 or 8-8 team. Marty feldman crap record of 2-15 in 1961-2 but that with not very good personal so tough to list Feldman as worst one. Kiffin and Tuener had very good talent to coach but both of them were crap. horrible coaches especially Kiffin but Turner pathetic too becuause look what he did wirh Washinton and San diego crap and more crap. Chargers going nowhere this year. Raiders passing them liike thief in the night when get revenger on them on Novrember 1.

129
by RickD :: Wed, 10/21/2009 - 1:40am

Marty Feldman coached the Raiders? Was that before or after Young Frankenstein?

94
by BigDerf :: Tue, 10/20/2009 - 10:13pm

Idea for Coach factoring - Take the Coach's win percentage or Expected Win Percentage (Generated from data about expected win percentages for rookie coaches and historical data for coaches under 3? years experience) and expect the team to head in that direction somewhat? It could also be done with DVOA's under the coach? But I don't know which is better as i don't have all the numbers on hand.

I.E. - Cowher hired to coach Redskins next year. Previous coach has bad Win/Loss percentage. Team should be expected to improve from coaching. Fits a logical progression of things.

Other example - Norv Turner take over 14-2 Bolts. Team should be expected to head towards his career win percentage as head coach. Goes 11-5... Should be expected to head that way again.... Goes 8-8.... I dunno.

Maybe you just need a Norv adjustment.

97
by Otis Taylor (not verified) :: Tue, 10/20/2009 - 10:27pm

Another item re: Denver - they haven't put anyone on IR or made any other roster moves (sounds like MIA from '08?). When you don't have any major injuries, or in this case ANY injuries, your level of success increases.

99
by Paulo Sanchotene, RS, Brazil (not verified) :: Tue, 10/20/2009 - 10:40pm

I just couldn't understand how can New Orleans have 6 estimated wins in 5 games.

101
by dbostedo :: Tue, 10/20/2009 - 10:43pm

It's because of the way bye weeks are factored in...the Saints are getting an expected win for a week when they had a bye.

102
by nat :: Tue, 10/20/2009 - 10:43pm

Read the explanation for Estimated Wins. Teams that have had their bye week are estimated as if they played every week. So New Orleans is playing well enough that - had they played six games - they would be expected to have won all six of them.

189
by Paulo Sanchotene, RS, Brazil (not verified) :: Wed, 10/21/2009 - 12:47pm

Thank you, guys. Now, I understand.

103
by Staubach12 :: Tue, 10/20/2009 - 10:55pm

My thoughts on Philly are that their offense consists of a style of play that is generally successful but risky--a very high reliance on the passing game. Overall, a high level of passing will be effective because passing is better than running. However, the passing game is like an extreme version of a boom-or-bust running back. An offense like that is going to have lots of variance because the sample reflected in a given game is too small. So the result is a very good team on average that appears to self-destruct when things don't go right.

Chris Brown over at SmartFootball.com has some posts that suggest that when a favorite employs risky strategies it is more likely to lose to inferior opposition than an equally good team that employs less risky strategies (the opposite is true for bad teams--they benefit from risky strategies).

Links:
http://smartfootball.blogspot.com/2009/05/david-strategies-and-goliath-s...
http://smartfootball.blogspot.com/2006/07/runpass-balance-response-to-co...

Overall, I think your statistics have the Eagles nailed. They are a good team that employs a risky scheme. As a result they should have both a high DVOA and a high variance. Overall, I would expect good teams with high variance to underperform in terms of wins and losses and to have both blowout wins and head-scratcher losses.

DVOA gets the Eagles exactly right, but perhaps the expected wins formula needs a tweak to incorporate variance (bad teams with high variance should get a boost, and good teams should be docked some in the expected wins formula).

211
by Pat (filler) (not verified) :: Wed, 10/21/2009 - 4:10pm

Estimated wins already has that. Adjusting for the bye week, the Eagles have around 3.5 estimated wins. That seems about right to me.

109
by KillerB (not verified) :: Tue, 10/20/2009 - 11:29pm

News flash - the Eagles consistently do well in DVOA because they consistently do well. Yes, you see awful games like this, but the Eagles have done the following this decade:

2000 - 11-5, lost in divisional round of the playoffs
2001 - 11-5, lost in NFC Championship
2002 - 12-4, lost in NFC Championship
2003 - 12-4, lost in NFC Championship
2004 - 13-3, lost in Super Bowl
2005 - 6-10, McNabb out for much of the year, depending on Mike frickin' McMahon
2006 - 10-6, lost in divisional round of the playoffs
2007 - 8-8, another McNabb injury cost them a few games
2008 - 9-6-1, lost in NFC Championship

That means the Eagles had winning seasons 7 out of 9 years, made the playoffs 7 out of 9 years, and made the conference championship [b]more than 50% of the time[/b] at 5 out of 9 seasons! That puts them in elite company, with only the Patriots and Colts having similarly consistent success.

It doesn't make their inconsistency - especially at inopportune times - any less frustrating. But I really can't think of a better way to run a team than to consistently get it into the playoffs, where a few lucky bounces can mean the difference between a first-round loss and a Super Bowl win. The Eagles could just as easily have won more Super Bowls this decade than the Steelers did in the 70s. The fact that they didn't doesn't warrant all the kvetching about Reid and DVOA.

Remember, folks, we could have Rich Kotite. Or Jim Zorn.

120
by Still Alive (not verified) :: Wed, 10/21/2009 - 12:23am

exactly. The Philly sports scene just boggles my mind. The eagles are probably one of the best run franchises in ANY of the major leagues. Definitely in the top 10%. And in response all the get is whining. I never sense the same amount of bitching from NE or IND fans, even before IND won the SB.

228
by Might as well be Micheal Lombardi (not verified) :: Wed, 10/21/2009 - 5:23pm

Dude, are you kidding? I'm a Pats/Sox fan and all we ever do is whine. We complained ALL year about our 95-win baseball team and we have been whining all season about our "underperforming" football team. All our teams have to do is be absolutely perfect, like go 18-0 or win 59-0 and we're happy. For now.

But it's hard to whine about Belicheck, I'll give you that.

110
by anotherpatsfan :: Tue, 10/20/2009 - 11:36pm

Harkening back about 40 days, the Steelers' fans had their torches blazing and pitchforks sharpened at the heresy of the FO DVOA projection that Steelers' defensive DVOA would rise to almost average, to -2.0%. The vitriol/calls of BS from them was legion, reminiscent of the Philly over-DVOAage rhetoric. To date, of course, that Steeler' D DVOA projection is wrong. It underestimated how much worse the Steelers' D would be this year, as it currently sits at PLUS 7.1% (which seems about right from a visual perspective). Missing Troy's flowing locks may explain some of that slippage, but I don't see any Steelheads still banging on that projection -- the silence is pretty deafening. Of course, if they play better D and weird things happen in the NFC, they could probably squeak out another SB...

161
by DMC :: Wed, 10/21/2009 - 10:25am

I think I saw a few comments about it last week. FO was completely correct about the defense. On the other hand, the offense has a line that learned to pass block, two running backs that can actually catch the ball, and four scary recieving threats in Holmes, Ward, Miller, and Wallace.

263
by bengt (not verified) :: Thu, 10/22/2009 - 8:55am

I disagree that a plus 7% defensive DVOA for the Steelers seems about right and that FO correctly predicted the D's 'decline'. I pose that they are actually a very good defense (quick & dirty: 40 points against in quarters 1-3) that cannot and should not play prevent in the fourth quarter, but has done it nonetheless and has suffered for it. IMHO the split is so big (I seem to remember a fourth quarter DVOA of 100% in one game) that making global statements like the two above is unsubstantiated.

Apart from that, the DVOA formula made the prediction, the FO writers did not believe it themselves and expressly said so.

269
by DMC :: Thu, 10/22/2009 - 11:50am

The overall number doesn't take into account the split in quarters.

Being a Steelers fan I do believe that there is a "correction" coming in the defense, although my confidenece is shaken a bit with A Smith gone for the year. Let's see how they fare against the unbeaten teams that are coming up.

I think the point that I was trying to make was not that the defense is horrible but that FO was right about the relative strength of the the team. Even if the defensive DVOA goes up, I think the offensive is absolutely 30% better than the defense. I also expect that to be the case for the whole year.

296
by bengt (not verified) :: Fri, 10/23/2009 - 7:45am

I, too, see some evidence for regression to the mean for the Steelers' D. The really big jump is, however, obviously caused by the abysmally bad play of the prevent D in some fourth quarters lately. But I doubt that this is what DVOA predicted, since it came to this because of the individual game situations. And I'm convinced that DVOA does not 'think' along the lines of 'The Steelers' O will improve, therefore they will more often be nursing leads late in the game, so we subtract 20% defensive DVOA because of their expected horrendous prevent D'.

297
by bengt (not verified) :: Fri, 10/23/2009 - 7:53am

Is someone filing all the 'CAPTCHA is reading my mind' moments?
While posting I was in parallel reading an article about a man who was beaten to death by youths lately. He had already called the police, and now it turned out that apparently his mobile phone had inadvertently redialled the number while he was beaten, so the police had a tape recording of the whole thing.
My CAPTCHA: 'resubmit 112', which is the emergency number for fire brigade and rescue services in Germany...

111
by MaineRaider (not verified) :: Tue, 10/20/2009 - 11:41pm

Mr. Schatz,
I wouldn't get too worked up over the Denver thing. Two games were lucky flukish wins (I mean the Cincy tip catch and the OT win after a coin toss, which is a random event). Two wins came against very bad teams (CLE, OAK), one came against a mediocre team (DAL by 7), and that leaves the impressive win at SD. Two quality wins. That's it. Also, their aged DBs have ten games left to pull up lame and vindicate FO.

114
by Fan in Exile :: Tue, 10/20/2009 - 11:58pm

This is why you should never listen to a raider fan.

158
by Vandal :: Wed, 10/21/2009 - 10:04am

I love the Anti-Denver apologists that base this on strength of schedule. Their opponents are 16-12 against teams not named Denver.

Remember the "we'll see after week 6" that everyone was preaching in Week 3?

113
by Fan in Exile :: Tue, 10/20/2009 - 11:57pm

I was just thinking for figuring depth that it might be good to look at how many players a team should bring in and use that to adjust the injury expectations.

By that I mean if teams need to replace x players every year just to keep up with the ones who retire then any team bringing in

115
by Fan in Exile :: Tue, 10/20/2009 - 11:58pm

I don't know why but it cut off part of my previous post. so I'm going to double post to see if that gets it in.

I was just thinking for figuring depth that it might be good to look at how many players a team should bring in and use that to adjust the injury expectations.

By that I mean if teams need to replace x players every year just to keep up with the ones who retire then any team bringing in

167
by jebmak :: Wed, 10/21/2009 - 10:55am

You can't use less than or greater than signs. It cuts it off.

181
by zlionsfan :: Wed, 10/21/2009 - 11:54am

Right. If you want to display < or >, then you have to use the HTML entity for them (&lt; and &gt;).

200
by Fan in Exile :: Wed, 10/21/2009 - 2:32pm

thanks it's good to know where I was screwing up.

So as I was saying figure out how many players a team should be bringing in to replace and then see if there is a penalty for bringing in fewer players. I believe that this is what is really sinking San Diego right now, they simply haven't got the depth that they need and it's not like Williams was a spring chicken.

116
by BigCheese :: Tue, 10/20/2009 - 11:59pm

Since Denver now has a pretty significant chance to go to the SB, and several Chicago SB match-ups are calculated, shouldn't the odds of a Cutler Bowl (DEN-CHI) be added as well? Or should that be an Orton bowl?

- Alvaro

117
by Random Joe (not verified) :: Wed, 10/21/2009 - 12:00am

> Obviously, it doesn't matter when you are winning by 59, but DVOA does count every play, although at reduced strength because of the score.

If the data 'obviously' useless, why did you proceed to add (even at reduced strength) what amounts to noise to the DVOA of both NE and TE?

Football play calling is highly situational and ignoring the situation of a blowout is a pretty glaring flaw in the stats.

I propose that plays only count toward DVOA while the starting quarterback is still in the game (exception: backup QBs subbing in for an injured QB). I'd really like to suggest that you somehow ignore plays after the point the game is 'over' but I can't state a clean way to measure that :-)

That said, if Tom Brady is joking it up on the sidelines 5 minutes into the 3rd quarter, it's a pretty sensible thing to turn off the meter.

121
by Still Alive (not verified) :: Wed, 10/21/2009 - 12:27am

It would be an interesting experiment to try to mark instances when this occurs and strike them from the record. But perhaps not so interesting that it is worth more than an hour or so of work :) I wonder how long it would take.

134
by Andrew Potter :: Wed, 10/21/2009 - 4:08am

Hasn't Aaron talked before about trying to do that, and finding that it actually made the system as a whole less accurate (or at the very least, not more accurate)? I may have that in the wrong context, but I'm sure I remember reading from the guys that removing "garbage time" plays from the equation didn't improve the metrics.

187
by zlionsfan :: Wed, 10/21/2009 - 12:20pm

Yes, he mentioned that when talking about DVOA 6.0.

I should point out that there were also a number of adjustments that I tried but didn't include because they didn't improve the correlation of DVOA to wins in the current year or DVOA the following year. That includes adjustments based on climate rather than just indoors/outdoors, "blowout" adjustments for plays before the fourth quarter, and adjustments for home-field advantage. That last one really surprised me when it didn't actually improve correlation from year to year or from the first half of the season to the second half, but it didn't, so it doesn't go into the new DVOA.

232
by Rich Conley (not verified) :: Wed, 10/21/2009 - 5:39pm

Yes he has, but we've asked time and time again, and he refuses to define "blowout".

So we have no way to tell what plays in the 3rd quarter up 50 points are being compared to. Is it all plays over 20 points in the 3rd quarter? 30? Is it still consdiered a relevant game because its not the 4th quarter?

He refuses to define the buckets.

130
by RickD :: Wed, 10/21/2009 - 1:44am

Statistics ¬= story telling.

One very important principle to stick with in statistics is to avoid ad hoc adjustments made after the fact like "turning off the meter" because Tom Brady is on the bench.

Statistics are not supposed to be perfect. They're supposed to require interpretation.

124
by Rick A. (not verified) :: Wed, 10/21/2009 - 1:18am

Jovins--It doesn't matter who is in or out in the Viking defensive backfield if Jared Allen is creating havoc. When he is doing his thing it's the easiest time in the life of any defensive back. When he is not, only then can teams go to work on any weaker corners or safeties. The Ravens became the first team this year to control the guy as the game went along. They did it by really well executed screens to his side and very efficient double teaming. Most teams can't even double team him. Right now he is one of the two or three most dominant players in the game...

133
by Jovins :: Wed, 10/21/2009 - 4:06am

Yes, on the whole a pass-rushing defensive end is certainly a defensive secondary's greatest friend, as we saw when the giants beat the patriots, just to bring up a well-known example.
But you're discounting the complete inability of Karl Paymah to cover anything. He's primarily a special teams guy; having him as a starting corner is downright negligible. It's a lot easier to contain Jared Allen when you know you have a receiver who can get open at will because of a ridiculous mismatch.

That's why football is such a hard game to get numbers for - every phase of the game is interrelated. Antoine Winfield can cover his man a second or two longer than Karl Paymah, or is able to play tighter coverage, denying a quick bailout for a quarterback. That half second turns into a sack instead of a completed pass.

To say that the Ravens contained Jared Allen better than the past couple teams is a fair statement, but Jared Allen has had a couple of games in a row that had some pundits talking about him as a possible MVP candidate (albeit pundits I don't much respect). To expect that level of play every game is not reasonable. It was very likely less a matter of what the Ravens were doing schematically or matchup-wise and more a case of Allen's play regressing to the mean.

126
by B-Rick (not verified) :: Wed, 10/21/2009 - 1:27am

First off in the NFL there are always a few teams every season that vary considerably from their projections.

As for Denver's turn-around, I will give some variables to attribute to their turn-around, that could have been used to at least see they would be better than projected.

1. When comparing Kyle Orton and Jay Cutler's career turnover percentage in the red-zone Orton is far superior to Cutler, especially considering Cutler played with better talent. I don't have the exact stats, but last night they said Orton only has two INTs in the red-zone in his entire career, while Cutler seems to have one every other week. In addition how many 'pick 6s' have each of them thrown, guaranteed Cutler has more.

2. McDaniels inherited a truly great offensive line that was returning all starters. Denver's lack of ability to keep a RB healthy in '08 led to an unreliable running game with Denver/Cutler passing much more than they would have liked in '08. According to budgoodesports.com, yards per pass differential is the most telling individual statistic in the NFL. Throwing more passes is going to negatively affect this 'killer stat'. It is very reasonable to expect that Denver would throw less this year given the freak injury situation they had at RB in '08 as well as the stability and strength of their OL (which is the best element of this team).

3. As for the defensive turn-around, the only argument that could have been made hinges on the fact that Denver's defense was awful in historic proportions last year and that some deviation toward the mean had to be expected. Their defensive improvement (should it continue) would have to be considered a rare unforeseen variable approaching scale to Kurt Warner emerging Roy Hobbs-like 10 years ago.

135
by bubqr :: Wed, 10/21/2009 - 4:52am

DVOA struggles : Maybe, because what is visible to us (fans / journalists) doesn't represent as much as what we think it does. A football game is 3 hours in a week of practice, coaching, gameplanning, players/coaches meeting, workout sessions, locker room life, all those stuff we don't see, but affect a game tremendously.

Maybe football is a counter argument for the 20% - 80% rule.

Maybe the 20 % we see (player/coach signing + games) doesn't represent 80 % of what a team is about.

Same problem with coaching : Game management is 16 x 3 hours, and is probably less than 1% of the time of coach spends working. Of course these 1 % are worth a lot more than the 1% some coaches spend molesting children, or the other 1% spent at punching your assistants, but still, maybe it's time to say that we don't have enough data to really evaluate coaches and their effect on a team based on game evualuations.

Just my 0.02 $

147
by DeltaWhiskey :: Wed, 10/21/2009 - 8:29am

- Excellent comment

Other thoughts:

- After "National Jump to Conclusions Day," my second favorite FO Holiday is "DVOA is broken because it doesn't explain everything day" - typically celebrated with the "WTF is going on w/ PHI and DVOA parade"

- Just because intangibles like "swagger" are difficult to measure does not mean they don't exist.

164
by DrewTS (not verified) :: Wed, 10/21/2009 - 10:45am

Nice random joke there at the end. I LOLed at it.

136
by Henrik Madsen (not verified) :: Wed, 10/21/2009 - 5:19am

"The trends all still make sense looking back -- the Broncos brought in a quarterback with lower career numbers..."

Most of the Denver turnaround was probably undpredictable, but I really think this factor was overrated. As was discussed last year, when Cassell came in for the Patriots, qb performance is very much a question of scheme and system. Cassell wasn't Brady but when he learned the routine they could move the ball. Denver's offense was strong last year with a much less skilled quarterback than Brady, so why not expect a reasonably qualified qb to be able to almost match this success?

Further, Orton's career numbers are not really indicative of his qualities. He had horrible statistics as a rookie, but it should be remembered, that he wasn't supposed to start, and that even his lackluster numbers were enough to not get in the defense's way. As the Titans found out with Vince Young and the Bears with Rex Grossmann the ability to not stand in the defense's way is a quarterback skill of quite some importance. When he was groomed to be a starter last year, he played effectively until injured. Obviously you can't base projections solely on someones peak, but the average numbers seem too low in this case.

So how can this be quantified? Perhabs the qb factor should just be reduced. When the team can run, block and catch - like Denver could last year - the differences in productivity between qb's 5-25 in the NFL are probably minimal.

233
by Rich Conley (not verified) :: Wed, 10/21/2009 - 5:41pm

"[Cassell wasn't Brady but when he learned the routine they could move the ball. Denver's offense was strong last year with a much less skilled quarterback than Brady, so why not expect a reasonably qualified qb to be able to almost match this success?"

The thing is, from week 10 on, Cassell pretty much WAS brady.

137
by Ven (not verified) :: Wed, 10/21/2009 - 5:46am

O.K. I had a critical comment earlier but let me offer some constructive criticism here.

There are two problems I have with your DVOA stats:

1. Let's say Cortland Finnegan and Nick Harper come back healthy in the last half of the season for the Titans. They aren't going to the playoffs or anything, but let's say their defense (particularly their secondary) improves. Now, NE's 59-0 win this past week will count as if they played against Finnegan and Harper and the real Titan's secondary, where in reality they played against 3 out of 4 guys who are practice squad players and couldn't cover anything. That's a major problem with DVOA.

2. Let's say your favorite team plays against Team X, some team that is tough and competing for the playoffs. Your favorite team gets a good game from them, you beat them and eliminate them from playoff contention. After, the last few weeks of the season they sit half their starters and mail-in the last few games. Another problem here, because your favorite team played in effect a "different" team than the one on the field the remainder of the season. This doesn't have to happen at the end of the year either, it can happen whenever a team realizes they aren't going to the playoffs that year.

---

Until you can factor these things into DVOA, your system is not going to work.

183
by mm (not verified) :: Wed, 10/21/2009 - 12:06pm

1. This is why they use "weighted DVOA" later in the season. This factors in the recent games more heavily than previous games.

2. They said they've tried reducing or not counting games in which teams rest starters, but it actually makes the formula less accurate.

184
by MJK :: Wed, 10/21/2009 - 12:06pm

The problem is that there simply aren't enough data points to try to do opponent adjustment on a game-by-game basis.

What the end of year VOA tells us is how well a team did over the course of the year, assuming all their opponents were average. However, we know strength of opponent matters...a lot. So you have to adjust for opponent strength.

However, if you're adjusting Team X that played Team Y for Team Y's strength...how do you know how good Team Y was on that day? You can't base it on their performance on that day...in your example above, there's no way of the statistics knowing whether the Titans happened to be going through a bad spell that day or the Patriots happened to be going through a good spell. Remember, the statistics don't know about the CB injuries...and even if they did, no one has successfully derived a model that quantitatively measures the performance of an individual player to overall performance. I.e. if you tell me that a team, when healthy, normally has skill N, but is missing their two starting CB's, I have no quantitative method of saying "well, in that case, their skill will be M". Maybe Barnwell's AGL research, cross referenced with DVOA, could eventually go there, but it will be a while, I suspect.

So the only thing you can judge how good Team Y on is their performance against opponents OTHER than Team X. In other words, you're seeing how well Team X did against Y relative to how every other team did against Y.

I suppose you COULD look just at other games in reasonably close temporal proximity...i.e. when determining opponent adjustments, rather than weighting every game equally to determine a team's overall DVOA, you could use a system of decreasing weights for games separated from the game in question. This would produce a temporally varying DVOA for every team, kind of similar to Aaron's "Weighted DVOA", but more sophisticated and looking into the past and the future. I'm not sure if Aaron tried this at one point and concluded that it didn't improve DVOA's correlation with wins, or if it has never been tried. However, I suspect you would quickly run into sample size issues.

186
by zlionsfan :: Wed, 10/21/2009 - 12:16pm

1. The article is old, but Aaron has commented on your concern here.

2. Aaron has commented on your concern here.

140
by Anymouse (not verified) :: Wed, 10/21/2009 - 7:02am

Anecdotally I've noticed that it seems to take about 3 years to build a great offense, but a solid defense can come out of nowhere in 1 year. I think the Colts several years ago were a good example of that.

How about this adjustment to preseason predictions of defensive DVOA: If a team has high personnel turnover (including coaching) on defense, give them a certain probability of having a drastically improved defense the coming year. The improvement may not happen at all, but if it does, it can be very large.

Another idea for prediction is to use the "wisdom of crowds." As a Saints fan, I know the team and the fans were much more optimistic than usual this off-season about the defense. One statistic that has proven to be a predictor of economic trends is the number of times the word "recession" is seen in newspapers. The more often the word is used, the more likely a recession is to occur (I don't know the source, but I have read this in The Economist). Maybe something similar would work in football. For example, the more times the words "defense improved" appear in a team's hometown paper, fan pages, etc, the better we expect their defense to be in the coming year.

146
by Erithtotl (not verified) :: Wed, 10/21/2009 - 8:29am

The Patriots - Titans game and some comments about it in the opening paragraphs remind me of one of the things that really bugs me about opponent adjustments with DVOA.

No one can argue that right now the Titans are playing truly awful football. The fielded a number of unready rookies against the Patriots who might not even make a lot of other teams. Now lets say they get a bunch of starters back from injury, fire their D-coordinator and finish the season 7-9.

For DVOA, this makes the Patriots (and the Colts, and whoever else whipped the Titans in the first half) look like they beat badly a 7-9 team, while the reality was that the team they beat in no way resembled the team that finished the season.

I proposed this last year and I would like to throw it out again. I'd like to see some kind of 'sliding window' adjustment made on strength of schedule calculations, where the 4 games around any given game weight a team's schedule adjustment. This helps take into account widespread injuries and other issues that could effect performance and give us better insight onto what was actually happening at the time the game was played.

166
by Ven (not verified) :: Wed, 10/21/2009 - 10:53am

I agree with you, posted the same thing 2 spots above you. Spot on.
I'm not sure how they would fix it, but I'm sure they can think of something and need to for these ratings to mean anything. It also goes the other way. I'm a Colts fan, but I remember 2 years ago the Ravens gave NE all they had in that Monday night game. After that game, Baltimore mailed in their season and didn't play half their starters and the Colts walked all over them the next week. And yet the Ravens counted the same for both IND and NE despite the fact that they were two "different" teams really.

Exact same thing (only vice-versa) this past week with the NE / TEN game. TEN counts the same for Pit and NE despite the fact that the Steelers faced a VERY different team in the opener.

193
by Still Alive (not verified) :: Wed, 10/21/2009 - 1:45pm

Sounds like a great idea to me.

148
by jimm (not verified) :: Wed, 10/21/2009 - 8:30am

The NFC East still rates as the strongest division in total DVOA by a wide margin:

NE - 81.4
AE - 28
AN - 4.7
NN - -0.9
NS - -1.9
AS - -3.8
NW - -43.4
AW - -47

But Sagarin Ratings - a points scored based system has the NFC East as the worst division in football.

NN - 23.57
AE - 22.63
AN - 22.32
NW - 20.07
NS - 18.88
AW - 18.63
AS - 18.33
NE - 17.61

I find that extremely odd.

The NFC East has largely played very weak teams or each other. All of their wins come against each other or the following teams:

TB, KC, Oak, Car, StL,

In other games outside their division they are they are 0-4 and negative 59 pts in losses to NO (2), Den and Detroit.

I really doubt that these 4 teams are anywhere near as good as DVOA has them rated.

216
by Pat (filler) (not verified) :: Wed, 10/21/2009 - 4:20pm

Sagarin ratings are historically biased against strong defenses, because it's game-output function is one-dimensional and paced (i.e. it doesn't take into account length of game). 3/4 of the NFCE teams have above average defenses, and New Orleans tends toward long games.

The other thing to note is that when teams don't have great connectivity to the rest of the league (as you pointed out) an Elo-type ranking will always pull to league-average. DVOA won't, as it's based on league performance.

I really doubt that these 4 teams are anywhere near as good as DVOA has them rated.

Eh, it's just as likely that the reverse is true. As is usual, the truth is probably somewhere in the middle.

149
by DrewTS (not verified) :: Wed, 10/21/2009 - 8:37am

Bam! I called -150% for the Titans in the Audibles thread this week. I feel like Simmons when he pretends to know how to guess the lines for the coming games.

151
by huston720 :: Wed, 10/21/2009 - 9:06am

Is it possible that you looked at the coordinator issue the wrong way? What if it isn't that Nolan is a great coordinator, but that the corrdinator he replaced was truly awful. You would have to find a way to differentiate between average coordinators that are fired, and terrible coordinators that are fired. This seems to make sense to me since it would seem more likely that a coordinator could squander his sides talent, than improve it. Meaning a good coordinator can coach up his side to its full potential but probably can't go past that mark, however a bad coordinator through poor play calls, and scheme can cause his unit to play significantly worse than the talent would be expected to play.

I'm not really sure if this applies or not, but it seems worth examining to me. just my two cents.

153
by peterplaysbass (not verified) :: Wed, 10/21/2009 - 9:48am

/i I often don't have the time to read through all the comments in the discussion threads, but I will look through this week i/

155
by peterplaysbass (not verified) :: Wed, 10/21/2009 - 9:52am

I meant to hit 'preview'. How do people put itallics in their comments?

I haven't any clue why DVOA missed on Denver and San Diego this year, but I do find myself wondering where Denver would be if McDaniels had successfully obtained Cassel from the Patriots. Any thoughts?

The Vikings look like they're going to sign DeAndre White to the practice squad as CB depth (he was a 6th round pick by the Giants), with other possibilities including Trae Williams and Mark Parson. Not the answer for an injured Winfield. It also looks like Asher Allen will get time at CB since Paymah doesn't know how to play. (sigh)

156
by nat :: Wed, 10/21/2009 - 9:59am

Follow the link "More information about formatting options" below for an explanation of the various tags you can use to format a post. a, em, strong, and cite are probably the most useful.

168
by Dales :: Wed, 10/21/2009 - 11:15am

I think by now it is relatively clear that the recent vintage Eagles hit the sweet spots in the DVOA formulas to an extent that has them consistently measured a bit too high.

I am a Giants fan, but last week I was commenting in the DVOA thread that I was not sure just how good the Giants actually are. With this in mind, please don't jump to the conclusion that I am being a fanboy with the following comments.

I don't believe there is a good explanation for the Eagles DVOA being 10 percentage points higher than the Giants, based on how well each team did or did not play this year.

The two teams have played many common opponents so far.

The Eagles had a dominating win, at home, against the Bucs. The Giants had an even more dominating win, on the road, against the same Bucs (see comment 61 here for more details).

The Eagles had a dominating win, at home, against the Chiefs. The Giants had a dominating win, on the road, against the Chiefs in which the Giants had poor fumble luck (lost 1 of 1 while recovering only 1 of 5).

The Eagles got beaten by a solid 26 points at home by the Saints. The Giants got thumped by 21 points on the road. Fumble luck was nearly a wash (Giants lost 1 of 1, recovered 0 of 1-- on average they should have lost 0.5 and recovered 0.5-- while the Eagles lost 1-1 and recovered 0 of 2-- on average they should have lost 0.5 and recovered 1, making for a 'luck' difference between the teams of half of a fumble recovery). Philly did have more 1st downs and was closer in yards to the Saints, and I am comfortable in assuming that the Eagles performed better in this game (at home) than the Giants did (on the road), by a comfortable margin.

The Giants, at home, completely obliterated the Raiders. The Eagles, on the road, lost. Aaron's comments aside about how the Eagles' performance wasn't quite so bad, the Eagles definitely performed below average that game. On the other hand, the Giants win was total domination, across the board. The difference between the two games should have been hugely in the Giants favor-- and should have been more than the difference in the single game DVOA ratings for the respective Saints games.

For the four game pairs above, the Giants should be 'ahead' of the Eagles. That leaves the difference being the Giants two wins against Dallas and Washington versus the Eagles dominating win against the Panthers. I'll stipulate that the Eagles DVOA in that game should be higher than the Giants DVOA in the other two games. What would be impossible for me to believe would be that this game averaged in with the four mentioned above, and the Giants two wins there averaged in with the four mentioned above, moves the needle so much that Philly ends up with a 10 percentage point lead over the course of the season over the Giants.

Again, I am not saying the Giants deserve to be higher in the DVOA ratings. I have a lot of concerns about the team. What I am saying is that based on the performance of the teams to-date, the Eagles have not performed better than the Giants, and a calculation that suggests they have is demonstrating its own flaws.

Of course, bitching about it isn't helpful-- but who is bitching? It does not bother me at all that DVOA has the Eagles higher. One can critique something, even something one appreciates, without being overly bothered by its weaknesses. However, it would be more helpful to come up with ideas that might lead to improvement.

I don't think a Philly Fudge Factor (tm) would be helpful. The formulas don't root for the city of brotherly love. However, I do think it is likely that there is something to the approach of the Eagles that has led to them, over recent history, to hit the sweet spots of the formula. What is different about the Eagles, unique to them, that might be causing the DVOA formulas to 'treat' them uniquely? The primary characteristic that I come back to is offensive balance. There are very few successful teams that have such a skewed run/pass balance, at least as far as my memory suggests.

Passing attempts per rush attempt:
2000: 1.45
2001: 1.26
2002: 1.12
2003: 1.16
2004: 1.45
2005: 1.70
2006: 1.30
2007: 1.37
2008: 1.41
2009: 1.44

Pat, above, shared this data:
2000 PHI Estimated Wins: 10.9 actual wins: 11 sched -7.0%
2001 PHI Estimated Wins: 11.4 actual wins: 11 sched 0.1%
2002 PHI Estimated Wins: 10.5 actual wins: 12 sched -5.8%
2003 PHI Estimated Wins: 10.8 actual wins: 12 sched -0.7%
2004 PHI Estimated Wins: 11.4 actual wins: 13 (*) sched -5.2%
2005 PHI Estimated Wins: 7.7 actual wins: 6 sched 5.2%
2006 PHI Estimated Wins: 11.9 actual wins: 10 sched 0.8%
2007 PHI Estimated Wins: 10.2 actual wins: 8 sched 3.8%
2008 PHI Estimated Wins: 11.7 actual wins: 9.5 sched 1.8%

Pat posted this to say that the seasons where the estimated wins exceeded the actual correlates with the schedule strength pretty well. This is true, but the fact is that most of these schedule strengths are not so far from average (0.0) to strike me as being overly meaningful (unless it is meaningful in the sense that the Eagles are especially affected by schedule strength, possibly with the same cause that has DVOA crushing on them). But let's compare the Actual Wins-Estimated Wins to the pass/run ratio:

2000: 0.1 1.45
2001: -0.4 1.26
2002: 1.5 1.12
2003: 1.2 1.16
2004: 1.6 1.45
2005: -1.7 1.70
2006: -1.9 1.30
2007: -2.2 1.37
2008: -2.2 1.41
2009: -1.3 1.44 (so far).

Let's bucket them.
Pass/Run <= 1.20 : 2 seasons, average 1.35 wins above estimated, -3.25% SoS
Pass/Run > 1.20 : 7 1/3 seasons, average 0.97 wins below estimated, -0.87% SoS

Yeah, I cherry-picked that 1.2 threshold. But there are very few highly successful teams of recent vintage that were above that threshold (2007 Packers, last years' Colts). I suspect that the lack of balance causes, somehow, the Eagles stomps to impact their DVOA more than a balanced team has their stomps impact their DVOA.

197
by jimm (not verified) :: Wed, 10/21/2009 - 2:08pm

excellent post - I think you are on to something here.

206
by nat :: Wed, 10/21/2009 - 3:38pm

You put a lot of work into this. I'm impressed.

You really should not have used -1.3 for the Actual-Estimated win total for this year. Their estimated wins are based on a six game season, but they have only played five games.

You should use 3 -(4.3 x 5 / 6 = 3.6) = -0.6. It slightly weakens your case, but not so much that you should give it up.

224
by Dales :: Wed, 10/21/2009 - 5:09pm

Eh. I hate that estimated wins doesn't just scale for how many games the team has played this year. Prorating would give 3.6 estimated wins rather than 4.3, meaning this year they are underperforming their estimated wins by -0.6 rather than -1.3.

However, it doesn't really change things much since this year 'counted' in my averages as just under 1/3rd of a year due to the Eagles having played only 5 games. It would not move the needle on the averages all that much. I think that it would change the end of my post to be:

"2009: -0.6 1.44 (so far).
Let's bucket them.
Pass/Run <= 1.20 : 2 seasons, average 1.35 wins above estimated, -3.25% SoS
Pass/Run > 1.20 : 7 1/3 seasons, average 0.94 wins below estimated, -0.87% SoS"

Thanks for the correction.

235
by Pat (filler) (not verified) :: Wed, 10/21/2009 - 5:55pm

There's an automatic bias in the analysis here: losing generates more passing attempts, and winning generates more running attempts. No one tries to run the ball down 7 with 80 yards to go and 4 minutes left, and everyone runs almost every down when up in the 4th quarter.

This bias isn't small, it's huge. This is the old "run to win" problem.

If you really want to measure propensity of a team to pass vs. run, you have to do it in pass/run choice neutral situations. First and ten in the first half is probably an easy cut. But you can't do "pass attempts vs. run attempts" because all you'll do is generate a win/loss proxy.

I hate to say this, but I think all you've rediscovered is that a team that loses games will pass more than they throw, and note the order that I put those - cause before effect.

This is true, but the fact is that most of these schedule strengths are not so far from average (0.0)

Let me say quick: you don't see the same pattern for the other teams in the NFC East as clear as with the Eagles. I'd say almost all of that is because the Eagles have been below-average against their own division since 2005.

But the point there is simple - being below average against an above average division isn't that bad. If you're then above average against the rest of the league, you're a good team, even though you look average.

The question is then "is the fact that the Eagles being below-average against their own division indicative of something?" I have no idea. But in my mind the reason the Eagles have been perceived as being overrated in the past 4 years is because the NFC East is strong, and the Eagles have a poor record vs the NFC East in the past few years.

239
by MJK :: Wed, 10/21/2009 - 6:35pm

This actually hits close to the problem. People run almost every down in the fourth quarter when up because it is smart tactically, in ways DVOA doesn't recognize because DVOA doesn't consider time consumed when it judges a play a success or not. Running is worth more when up in the fourth quarter than other times, even if it gains fewer yards. But DVOA doesn't acknowledge this. This is one reason why a team with an out-of-wack run to pass ratio like Philly might be favored by DVOA and yet be less successful than DVOA would expect.

For example, DVOA thinks a 1 yard rush is only slightly better than an incomplete pass, and way way worse than a completed 6 yard pass. However, late in the game, if a team is up, a 1 yard rush is way more valuable than an incomplete pass, because it runs ~0:45 off the clock, where as an incomplete pass runs only about 0:05 off. Conversely, to a team trailing, an incomplete pass is way way better than a 1 or 2 yard rush, even though DVOA would prefer the former.

245
by Pat (filler) (not verified) :: Wed, 10/21/2009 - 7:31pm

"This is one reason why a team with an out-of-wack run to pass ratio like Philly might be favored by DVOA and yet be less successful than DVOA would expect."

1) The way this would work, though, is that Philly would lose games because they couldn't close them out in the 4th quarter when ahead. This isn't why Philly's been losing games in the past four years.

2) It would also only work that way if Philly's head coach was too stupid to run the clock out when ahead. He's not. When the Eagles have the lead they run out the clock by running the ball like any other team.

242
by Dales :: Wed, 10/21/2009 - 6:49pm

Pat, first let me say I am enjoying this debate. So thanks.

"losing generates more passing attempts, and winning generates more running attempts"

I think this is one of the reasons that the Eagles' excessive PA/RA numbers are even understating the matter. The Eagles have been good. They have been in positions where teams normally run more. Yet even with those two factors, the Eagles have been passing at a disproportional rate.

Maybe this is helping me crystalize my thoughts on the matter. The Eagles are not behaving like teams normally do, and this might be one reason why a statistical model based off of success rates for the entire league where teams generally behave a certain way might not fit them as well as it does for other teams.

"This bias isn't small, it's huge. This is the old "run to win" problem.
If you really want to measure propensity of a team to pass vs. run, you have to do it in pass/run choice neutral situations. First and ten in the first half is probably an easy cut. But you can't do "pass attempts vs. run attempts" because all you'll do is generate a win/loss proxy."

The Eagles have been a strong team over the past several years, during which Reid has developed his pass-happy approach, yet despite being in the position to bring up their rushing attempts, they *haven't*. That's my entire point. I think it is likely that their abnormally high passing rates is what is distorting DVOA. Naturally, if you do not agree with the postulate that DVOA has been overstating the quality of the Eagles, then all of this is besides the point.

"I hate to say this, but I think all you've rediscovered is that a team that loses games will pass more than they throw, and note the order that I put those - cause before effect."

Yet the Eagles, who win, pass more than they run. You seem to think that we are missing cause and effect when in fact we are pointing out that the Eagles by choice do not behave as a typical team does and as such it may be distorting the model.

"Let me say quick: you don't see the same pattern for the other teams in the NFC East as clear as with the Eagles. I'd say almost all of that is because the Eagles have been below-average against their own division since 2005."

But the Eagles have not been below-average overall. Think of it this way, let's say you have a statistical measure of team strength. And you have this team that does not perform nearly as well against the strong teams it plays most often as the measure suggests it should. Could this not be because the measure is overstating the team's strength? Isn't this exactly what some (including me) are arguing has been happening against the Eagles?

"but in my mind the reason the Eagles have been perceived as being overrated in the past 4 years is because the NFC East is strong, and the Eagles have a poor record vs the NFC East in the past few years."

Which leads to what I am saying. The Eagles have not beaten the relatively strong NFC East teams at the frequency you would expect if their DVOA ratings over that time period were indicative of their true strength. This could be random chance, or it could be indicative of the fact that perhaps the DVOA ratings have them pegged too high, perhaps because the pass-happy nature of Reid or perhaps because of something else. Knock 10 percentage points off of the Eagles DVOA each year since 2004, and wouldn't their record against the NFC East be a bit more expected?

ETA- (this next comment is definitely not aimed at you, Pat) Again, let me stress that I have no interest in cutting down the Eagles. I think they are a strong team. I really could not care less if a rating system rates them better than my favorite team, the Giants, or not. I'm not arguing that the Giants are teh awesome or that the Eagles stink. I am not arguing that DVOA stinks. I value it but don't take it's numbers as gospel. And I am interested, because I do find it valuable, in throwing out suggestions for identifying and/or correcting any weaknesses in its methodology.

244
by Pat (filler) (not verified) :: Wed, 10/21/2009 - 7:26pm

"The Eagles have been a strong team over the past several years, during which Reid has developed his pass-happy approach"

I'm confused. From 2005-2008, the Eagles have won far fewer games on average (~8/yr) than from 2000-2003 (~11/yr). You would expect the number of pass attempts/rush attempts to increase. They did.

But the Eagles have not been below-average overall.

If you *just* look at wins and losses, the Eagles have been almost exactly average since 2005 (33-30-1). Their DVOA has been above average. Thus the perception that they're overrated. However, their record against the NFC East, an above average division, is 9-15. Their record against the rest of the league (an average distribution of teams) is 24-15-1.

Combine those two - a below-average performance against an above-average set of opponents and an above-average performance against an average set of opponents and you get an above average performance.

Thus my argument. Fans don't correct for the above-average division, and they see an average performance.

The Eagles have not beaten the relatively strong NFC East teams at the frequency you would expect if their DVOA ratings over that time period were indicative of their true strength.

Eh, it's consistent with random chance. I have a hard time looking to justify why a team won 9 games when they should've won 12 or so. It's just not anywhere near a large enough discrepancy.
It's even more telling when you add in the postseason games, which brings it up to 11 wins when they should've won 13. That's regression to the mean, so it's probably random.

The Redskins are in a similar situation. The Cowboys and Giants are the ones who performed above average, and unsurprisingly we've heard them described as underrated quite often.

Knock 10 percentage points off of the Eagles DVOA each year since 2004, and wouldn't their record against the NFC East be a bit more expected?

Yeah, but then their record against the rest of the league looks inconsistent.

248
by Dales :: Wed, 10/21/2009 - 8:43pm

"I'm confused. From 2005-2008, the Eagles have won far fewer games on average (~8/yr) than from 2000-2003 (~11/yr). You would expect the number of pass attempts/rush attempts to increase. They did."

Yes, I would expect them to increase. I would expect them to increase to normal levels-- not to go from levels normally associated with bad teams to levels normally associated with really really bad teams, though.

The Eagles have not been a bad team from 2005-2008. They have been a decent-good team on average over that timeframe. Yet, they have passed the ball at a rate that would be normal if they were a team that was trailing all of the time.

Going back to the buckets I used above, over the completed seasons where they passed more than 1.2 times per rush, they averaged 9.8 wins. You would not expect a team that was winning 61% of their games to pass that much. A team that wins 3-4 times a year passes that much. Note I am not saying that passing that much causes a team to be bad; teams that are bad generally are in a position where they have to pass a lot.

The Eagles do it by choice. And that may be causing the statistical model to be less accurate for them than it is for other teams.

"Eh, it's consistent with random chance."

It is also consistent with what I postulated. Both could explain it. However, each year it continues makes it harder to dismiss as random chance.

"I have a hard time looking to justify why a team won 9 games when they should've won 12 or so. It's just not anywhere near a large enough discrepancy."

In any one year, I would agree with you. Over time though?

We are now in the middle of the fifth straight year where the Eagles have underperformed or are underperforming their estimated wins. Over that timeframe (counting the current year as just under 1/3 of a year), they haven't just barely underperformed their estimated wins, it has been by right around 2: 9.9 est. wins on average versus 8.0 average. Is their division hard? Sure is. But their overall schedule during that time has not been-- on average +1.6% which isn't very far at all from dead average. Is it possible for a team to underperform their estimated wins by nearly 2 a year for not only a 4+ year period but consistently each year by nearly 2 a year and have it be random chance? Sure. There is a 3% chance one could flip heads five times in a row, and a 0.4% chance one could roll a 1 or 2 five times in a row. I am not exactly sure what the odds are of a team that has the 'real' strength of a 10 win team actually winning just 8 in any given year against a close to average schedule, but I am guessing it is less than 50% and possibly even less than 33%.

One of a few things is going to happen. Either the team as constituted (many of the same players and same coaches) will suddenly stop underperforming their estimated wins, in which case it will suggest it was random, or the team as constituted will continue to do so, which will suggest that it wasn't random and the statistical underpinning for the estimated wins (DVOA) was measuring the Eagles incorrectly, or the team will change (the players will change, Reid will change or be replaced, or similar).

"Yeah, but then their record against the rest of the league looks inconsistent."

How so? I'd say that if you lopped 10-15 percentage points off of their DVOA each of the last 4 1/3 years, their record against the rest of the league would be pretty close to what it has actually been. Just a guess, though.

252
by Pat (filler) (not verified) :: Wed, 10/21/2009 - 9:48pm

Yes, I would expect them to increase. I would expect them to increase to normal levels-- not to go from levels normally associated with bad teams to levels normally associated with really really bad teams, though.

Uh, why not? I don't understand the logic here. They had 1.2 passes/run attempt from 2000-2003. They averaged ~11 wins. They averaged ~8 wins from 2005-2008. Therefore they should be passing more, so you'd expect something greater than 1.2. They had 1.4 passes/run attempt from 2005-2008.

You would not expect a team that was winning 61% of their games to pass that much.

... Why does this matter? You're postulating that the increase from 1.2 - 1.4 is what caused them to start underperforming. Is this right? Otherwise I'm missing something.

What I'm saying is that the 0.2 increase is exactly what you would expect from a team that goes from an ~11 win team to an 8 win team. It doesn't matter that they started out anomalously high. They're going to go up no matter what.

but I am guessing it is less than 50% and possibly even less than 33%.

Probability of seeing 33.5/64 when true success rate is 0.61875 is around 5%. In a league of 32 teams, you'd expect a team like that. This isn't an outlier.

Again, the situation gets likelier when you consider the fact that the Eagles have won postseason games as well in that time.

254
by Dales :: Wed, 10/21/2009 - 10:31pm

"Uh, why not? I don't understand the logic here. They had 1.2 passes/run attempt from 2000-2003."

True. A rate that is not historically unusual but is not what you would expect from a team that...

"averaged ~11 wins."

In fact, it what you would expect to find in a team that is closer to one that...

"averaged ~8 wins from 2005-2008."

And, specifically, a team that averaged 8 wins while having a poor defense.

But that wasn't the Eagles of the last several years. They have been a 500 team with an above average defense that entire time.

While I would expect a team who is trailing more than they had in previous years to pick up the passing rate, I would not expect a team that is already passing 1.2 times as much as they run, and maintaining an above average defense the entire time, to be finding the need to increase to 1.4 passes per rush attempt when going 8-8. It would be hard to have a good defensive DVOA (lower than -5% during that period, I believe) while consistently putting themselves in position to need to throw.

Heck, let's look at last year in detail. They were throwing 1.4 passes per rush last year. How often were they in situations where their rate of passes to runs was being jacked up by playing from far behind for a substantial portion of the game?
Not week one when they jacked up the Rams.
Not week two when they lost to the Cowboys. They were leading until deep into the 4th quarter.
Not week three, when they were leading the Steelers the entire second half.
Not week four, when it was within a touchdown the entire second half, within 4 points at the start of the 4th, and within 1 point at another point in the 4th.
Not week five, where it was a 2 point game at half, and where the Eagles were within a TD for the last 7 1/2 minutes of the game.
Not week six, where they shellacked the 49ers.
Not week eight, after the bye, where they lead the Falcons the whole game.
Not week nine, where they slaughtered the Seahawks.
Maybe week 10. They were within a TD of the Giants until 9 minutes were left, and then behind by more. So maybe for 1/5th of the game were they forced to be pass-heavy.
Not week 11, when they played the Bengals to a 13-13 tie.
Week 12, we have one. They got skunked by the Ravens and had to throw nearly the entire 4th quarter. The game was not out of hand before then, though.
Not week 13, when they handled the Cards with relative ease.
Not week 14. They lead the Giants the entire way.
Not week 15. They killed the Browns.
Not week 16. It was a 10-3 game, and they were within a single score the entire 4th quarter.
Not week 17 when they slaughtered the Cowboys.

So in a full season where they were passing 1.4 times to every rush, they were in situations where they were being forced to pass to try to score quickly and often to get back into it for at most 1 quarter, total. While it may be true that teams that are losing are forced to pass more, and while it may be true that the Eagles were not winning as frequently as in the early 00s, there is not real way to argue that the Eagles were passing as frequently as they were as a consequence of them being forced to because they were trailing badly. Situations did not dictate that pass/run ratio. Andy Reid did.

They did not pass so much because they were struggling. They passed so much because that's what they did, and what they do. What isn't clear is if this leads them to lose more often than a team of their actual strength should lose-- something I don't believe-- or if it is that their passing proclivity causes DVOA to measure their strength as being higher than it actually is-- which is what I suspect.

256
by Pat (filler) (not verified) :: Wed, 10/21/2009 - 11:03pm

"How often were they in situations where their rate of passes to runs was being jacked up by playing from far behind for a substantial portion of the game?"

Um, no - wrong way. The Eagles are a pass-heavy team. We know this. In a neutral situation, they're likely to pass more than average, so they're likely to pass more than average when they're down because that's how they want to move the ball.

The point is that before 2005, the reason why they ran so much is because they were ahead and closing out the game.

In the past few years they've either lost games or played closer, which means they're less often in a situation where "run" is preferential to "pass." Games where they won are the ones that have a lower pass/run ratio.

In almost every game they won where they were leading in the 4th quarter in 2008, you can go through and find a distinct increase in rushing around the 4th quarter. The Arizona game, the second Giants game (11/13 plays were rushes on the last drive, 8/10 on the previous...), the Cleveland game (6 straight runs on the last drive)... I can go on and on.

You can see the exact same thing in previous years, and that's where the bias comes from. They won more games in previous years, so they had more runs in the fourth quarter.

260
by Dales :: Thu, 10/22/2009 - 7:26am

"Um, no - wrong way. The Eagles are a pass-heavy team. We know this. In a neutral situation, they're likely to pass more than average, so they're likely to pass more than average when they're down because that's how they want to move the ball."

I am not sure why we are arguing past each other.

I've been saying that the Eagles were a pass-happy team, where they were passing as much as a bad team while they were winning 11 games a year, and then they became a really really pass-happy team, where they were passing as much as a really bad team while being near 500.

How is that different than what you just said above?

Let me try recapping for simplicity:
1) The Eagles have underperformed their DVOA for going on five straight years by about 2 estimated wins a year. You think it is a statistical fluke. I concede this might be random chance, but think the odds of a team doing so are less than 1% and suggest that if it continues to happen it would indicate it isn't random.

2) You suggested it is nothing more than their in-division games getting harder. No doubt, they have. I countered by suggesting a team with a 10-15% lower DVOA would have been expected to have about the same record the Eagles have had in-division against Skins, Cowboys and Giants over that time. We could test this out by calculating their expected wins against their schedule each year if we had the algorithm or formula for converting two teams' DVOA plus home field into percentage chance of winning (as is used in the playoff odds calculator). Do you have it? If not, we can't really prove or disprove this.

3) My main hypothesis is, that if DVOA is overstating the Eagles' strength, then it might somehow tie back to them being a pass-happy team, and being even more of a pass-happy team of late. The reasons why they have been more pass-happy of late are irrelevant to the discussion of if this somehow leads to DVOA being less accurate about them. If you want to say they've been passing 20% more the last few years because they've been winning less, I won't argue since I don't really know if it is cause, effect, or a bit of both. Suggesting that DVOA does not accurately handle high success rates for teams far exceeding the norm in terms of passing frequency does not rely on any particular reason for why the team is passing so much in the first place.

264
by nat :: Thu, 10/22/2009 - 9:31am

I've been reading this thread with interest. Although it's pretty clear this is about your Giants getting less respect than the hated Eagles, I applaud you for trying to put it into an interesting form. I also respect that you disclosed your biases early, and have tried to avoid the trap of mouth-foaming homerism.

Personally, I have reasons to respect and dislike both teams. So I think I have a balanced perspective.

I don't accept your premise that the Eagles have (in any meaningful way) underperformed their DVOA this year. DVOA only projects wins via the "Forest Index" - that is, via estimated wins - which takes into account per-play average (DVOA) and variance, and a few other factors. Both the Eagles and the Giants are less than a game off from their estimated wins (adjusted for games played). Their different success comes down precisely to the Eagles' first quarter against the Raiders, and the Giants' two-minute drive against the Cowboys. Shit happens.

That leaves us with an Eagles team that overperforms and underperforms their estimated wins by a game or more in this pattern for the last decade: expected, expected, over, over, over, under, under, under, under, expected.

That is, expected 3, over 3, under 4. That looks pretty normal - maybe with a tendancy for the extremes, although you'd have to study the rest of the league to say that with any certainty.

You focus on the last four years and ask: why did they underperform four straight times? Of course, the more neutral question is "why does a team have four extremes in a row in the same direction?" Or, you could look at all differences and ask "why does a team have five in a row all in the same direction (extreme or not)?".

It's just not that unlikely. If you assume that extremes happen 2/3 of the time (1/3 each way), then the chances of 4 extremes in a row are 2/3 * 1/3 * 1/3 * 1/3 = 2/81, meaning you'd expect it to happen four out of every five seasons.

If you want to include the non-extremes, the chances of 5 going the same direction in a row are 1 * 1/2 * 1/2 * 1/2 * 1/2 = 1/16, meaning you'd expect two teams to do it each season.

So, without your premise being supported by the facts, the rest of your theory falls apart. You need to go back and collect and analyze more data.

I think there might be something interesting in asking whether teams tend to overperform or underperform consistently from year to year. If that turns out to be true, then you might look into whether play-mix is a good predicter for the league as a whole.

266
by Dales :: Thu, 10/22/2009 - 11:14am

"Although it's pretty clear this is about your Giants getting less respect than the hated Eagles"

On this, you are incorrect.

I don't have any need for the Giants to be 'respected'. Fans perception of a team does not impact their chances of winning or losing. Neither does their rating in any statistical system.

I assert, and you don't have to believe me, that I'd be asking the same questions if the team involved was the Texans. I don't gamble much, but I do participate in an office pool vs. the spread for all games, all year. A model that would help in identifying good picks is in my interest. Helping to improve such a model is in my interest.

"I don't accept your premise that the Eagles have (in any meaningful way) underperformed their DVOA this year. DVOA only projects wins via the "Forest Index" - that is, via estimated wins - which takes into account per-play average (DVOA) and variance, and a few other factors. Both the Eagles and the Giants are less than a game off from their estimated wins (adjusted for games played)."

Understood. The only comment I have to add is that this isn't a five game trend, but rather a four year trend that appears to be continuing. Appearances can be deceiving, etc.

"that leaves us with an Eagles team that overperforms and underperforms their estimated wins by a game or more in this pattern for the last decade: expected, expected, over, over, over, under, under, under, under, expected.
That is, expected 3, over 3, under 4. That looks pretty normal - maybe with a tendancy for the extremes, although you'd have to study the rest of the league to say that with any certainty."

Correct- with one exception. If we prorate this year out, it does not end up in the 'expected' bucket but in the 'under' one.

That aside, if overperforming vs. underperforming was a coin-flip, that would look pretty normal. However, we are talking about not just barely underperforming, but underperforming by just about 2 games a year. Are the chances of a team doing that in any one year 40%? 30%? I am pretty comfortable in assuming it is less than 50%. But, for the sake of argument, let's assume it was 50%. The percentage chance of there being a team with a five year streak would be 1/2^5, or 1 in 32-- which means we should see about 1 team in the league having such a streak at any point in time. Over a number of years, that should end up being the average. Right?

Well, with this in mind, I believe it should be possible to calculate the odds that the Eagles would have underperformed their expected wins given their DVOA and their schedule for each of the previous four years. From this, you could calculate the same for each and every team going back through the years for where we have DVOA for the four years going back (what would that be, 1998 on?). Average that together, and you would get the expected number of such streaks. If there are meaningfully more such streaks than expected, then I would take a look at those instances to start mining for commonalities that 1) might explain why, and 2) could be used to improve DVOA.

That's what I am suggesting.

"You focus on the last four years and ask: why did they underperform four straight times? Of course, the more neutral question is "why does a team have four extremes in a row in the same direction?" Or, you could look at all differences and ask "why does a team have five in a row all in the same direction (extreme or not)?"."

Completely agree. That's another reason for throwing a comment out-- so that others might be able to take the idea and run with it, so that others who have already looked at such things might answer to prevent wasted work, and so forth. I am all for removing the label "Eagles" from this and looking at it dispassionately.

"It's just not that unlikely. If you assume that extremes happen 2/3 of the time (1/3 each way), then the chances of 4 extremes in a row are 2/3 * 1/3 * 1/3 * 1/3 = 2/81, meaning you'd expect it to happen four out of every five seasons."

I understand you are just throwing out numbers to make a point, but I think that 2/3 is way too high to use. Is it true that in any year, 2/3 of the teams differ from their expected wins by 2 games (when rounding off to the nearest whole)? I just looked right now at last year-- there were 15 teams that did so. Again, assuming each direction is equally likely and that this year was representative, then you would have roughly a 0.6% chance of it happening to any particular team. (ETA- just looked at the last four years. On average, 13.5 teams have differed from their expected wins by a rounded two or more in any direction. Using that, there should be one team every 8 years or so 'enjoying' a streak of four years where they are over or under their expected wins in the same direction by a rounded 2 or more games).

Of course, there are 32 teams, so in any year that works out to about a 19% chance you would have a team on a four year streak (either over or under). And for a five year streak, that drops to about 4.5% of the years you would expect to see a five year streak going (one reason I suggested that if it continues this year for the Eagles it would certainly start pointing towards it not just being a fluke).

A potential next step would be to identify such streaks and see if it is happening over the years more than would be explained by chance or not, don't you think? If it is, then this would be an indication that there is an opportunity to improve DVOA.

270
by nat :: Thu, 10/22/2009 - 11:52am

I checked.

The chances of a team missing their estimated wins by 1 game or more are approximately 61% - so my guess of 2/3 wasn't too bad for a guess.

Each year, approximately 7 teams miss their estimated wins by at least 2 games. So on average, we would expect the Eagles to do so 1.1 times in a five year period. Instead they have done it 2 times. (Based on a ten year total, in case you care) Doesn't seem like an outlier to me. In fact, with 32 cases in 5 years, it's a virtual certainty that some team was going to do it twice.

It's way too early to prorate the season. You're assuming what you are trying to prove - namely that missing your estimated wins indicates your true strength, and not random fortune.

The data just doesn't support your ideas in this case. Sorry. I know you can fudge by playing games with buckets and rounding (I noticed that), but you're better than that. It was an interesting theory, but it doesn't hold up.

The Eagles have underperformed their estimated wins recently - but in a way that needs nothing more than typical random chance to explain. Each year about 20% of teams are estimated win extreme outliers. It happened to a particular team you dislike twice. Well, it was going to happen to somebody.

275
by Dales :: Thu, 10/22/2009 - 1:19pm

"Each year, approximately 7 teams miss their estimated wins by at least 2 games. So on average, we would expect the Eagles to do so 1.1 times in a five year period."

How many teams per year are you seeing miss it by at least 1.5? I am getting in the ballpark of 13.5 per year (but have looked at just 4 years). The Eagles have done that four years straight, all in the same direction.

Odds are, if you look at all 32 teams, you will find one enjoying such a streak in a five year period. Are there more? That I don't know yet and either am hoping someone else has looked at or will look at when I have more time.

"You're assuming what you are trying to prove"

No I am not. I am saying it strikes me as unlikely that a team would have missed by as much as the Eagles for four straight years. We have one example now with the Eagles. Next it for me or someone to find out how often it has been happening and if that rate is consistent with random chance or not. Or, alternately, wait a few more years and see if the Eagles make it five, six, seven straight years. I'm not that patient.

In other words, we are already in the 'unlikely' designation. How unlikely is debatable. The 'assumption' made in prorating is merely to say that if they continue the way they have done so far this year, they will end up with a fifth straight season. Four is unlikely, five is even less so. But again, how unlikely I haven't defined (and rather than argue about it, I suggest one or both of us try to calculate it).

Since the prorated year is causing you consternation, ignore it. How often are we finding teams that go on four year runs of missing their average wins by 1.5? Are there others? If my hypothesis is correct and the Eagles, for whatever reason, are hitting a sweet spot (for the lack of a better phrase) in the DVOA calculations and therefore are getting a value higher than their ability to win games really warrants, then I would be absolutely shocked if they are the only team to ever do so. As such, if my hypothesis is correct, we (or I) should be able to find more teams going on such runs of consecutive years than would be expected by chance.

This isn't about the Eagles.

276
by nat :: Thu, 10/22/2009 - 1:33pm

You are cherry-picking. And it does seem to be about the Eagles. Why else would you keep tuning your cutoffs to fit the Eagles?

Look, I used 1 for the cut-off and you didn't like that. So I used 2 as the cut-off at your suggestion, and you didn't like that. Now you want to use 1.5? Booorrrrrrrrring!

I could not care less about the Eagles or your Giants. You have given no good reason to suppose that DVOA (and estimated wins) is broken with respect to them, and plenty of reason to dismiss you for blatant homerism.

Get over it. Play-per-play, the Eagles have had (slightly) more success than the Giants. The Giants have won more games. This is due to (a) things that estimated wins models but DVOA does not, and (b) things that neither DVOA nor estimated wins models, probably because they are not predictive of long-term future success.

277
by tuluse :: Thu, 10/22/2009 - 1:55pm

I don't think he's arguing this because of homerism. Do you have any reasons that 1 or 2 are better cutoffs than 1.5?

The Eagles have underperformed 4 years a row. Dales is trying to figure out if 1) They have underperformed by a significant amount, and 2) is this due to a flaw in DVOA that could be corrected. His assertions are that yes it is significant and it could be due to a high number of passes.

I've found the discussion very interesting, and haven't noticed any homerism at all, much less "blatant"

278
by nat :: Thu, 10/22/2009 - 2:08pm

Only that 1 and 2 are whole numbers, and that 1.5 was cherry-picked specifically with the Eagles data in mind. And the 2 games was at his suggestion, and then rejected when he realized it didn't support his already-decided opinion.

Other than that, no.

279
by tuluse :: Thu, 10/22/2009 - 2:19pm

Considering expected wins is rarely a whole number, I don't see how using whole number cutoffs makes sense. Shouldn't you look at standard deviation?

281
by BigDerf :: Thu, 10/22/2009 - 4:49pm

But right now isn't the whole reason we are talking in order to point out that the Eagles data is flawed?

Didn't Aaron ask for suggestions?

The Eagles data just happens to be Exhibit A at the moment. Heck maybe we should look for an Exhibit B but right now we got what we got.

283
by Dales :: Thu, 10/22/2009 - 5:09pm

You are right that 1 and 2 are whole numbers.

But since you have expected wins to 1 decimal place whereas wins are (excepting when ties occur, which is rare) to 0 decimal places, it made sense to me to round to the nearest whole number when subtracting the two (hence, rounded wins). The net effect of this is for it to be 1.5.

It has nothing to do with homerism or a vendetta against the Eagles.

291
by Jovins :: Fri, 10/23/2009 - 3:06am

This is something I've been wondering about - everyone assumes that the Eagle's High DVOA relative to win totals is caused by offensive scheme.
Couldn't it be caused by defensive scheme?
I remember a few years ago the Eagles were credited for their "Bend but don't break defense." I don't know what's changed, but could it be something defensively rather than offensively? This is all pure speculation, and it seems like there is some agreement that offense is the culprit, I just missed the part where the defense was analyzed and deemed accurate.

292
by Jovins :: Fri, 10/23/2009 - 3:06am

double post

284
by Dales :: Thu, 10/22/2009 - 5:50pm

"I used 1 for the cut-off and you didn't like that. So I used 2 as the cut-off at your suggestion, and you didn't like that. Now you want to use 1.5? Booorrrrrrrrring!"

I never suggested using 2 as the cutoff. A team that has an expected wins of 8.5 cannot underperform or overperform that by an exact 1, barring ties which are rare. This is why I used the difference rounded to the nearest whole number. And since I was interested in where the formula was more than just slightly off, that meant my threshold would be, in essence, 1.5, since that rounds up to 2.

Anyhow, I am not interested in trying to please you.

However, for those interested, I just examined all of the data that exists.

We have DVOA information for 373 team/year combinations where the team existed for at least 4 years inclusive.

I don't see any good reason to assume that it is equally likely for teams to underperform their expected wins by 1.5 as overperform. So I looked at them separately.

On average in any given year, 20.26% of the teams underperformed their expected wins by at least 1.5. This means that we would expect to find 0.6 instances (less than 1) where a team had four straight years meeting that criteria. We find, instead, 3 teams.

PHI 2005-2008 (current)
OAK 2003-2006
BAL/CLE 1995-1998

The Bravens came awfully, awfully close to making it five straight, underperforming their expected wins by 1.4 games in 1999. There is just a 0.03% chance that a team would have had a five straight underperform streak of this nature. Given that the Eagles are already at 4 straight, there was about a 20% chance heading into this year that they would accomplish something unique.

That said, when I look at the three above teams, I am not seeing much in common that would explain it.

ETA-- actually, now that I look closer, I do see one thing of interest. The Ravens ended their streak the year they changed coaches from Ted Marchibroda. The Raiders ended their streak when they changed coaches from Art Shell.

For the curious, we should have expected around 15 teams to have had two consecutive years. Instead, there were 11. We would have expected 2 of these teams to make it to 3 straight. 4 did. And 3 of those made it to 4 straight. One of those is still active, one came close to making it five, and one broke the streak (the Raiders). That Raiders team managed to underperform their expected wins by at least 1 rounded game for 6 straight years. Ouch.

Unfortunately, these sample sizes are too small to draw conclusions from. While it is odd to me to have had 3 teams with streaks of four when we would expect at most one (since we can't have a fraction), it still strikes me as possibly a fluke.

The other direction is, in my opinion, even more interesting.

On average, 16.6% of the teams overperformed their expected wins by 1.5 in any given year. From this, we would have anticipated seeing around 9 teams with a 2 year streak. Instead, we have 6 instances of teams with two year streaks. Five of them ended right there. One is still active (Arizona).

Cool-- we have two teams who might, this year, end up doing something that hasn't been done in the DVOA era. And in a fun coinkydink, they were the NFC Championship contestants last year.

Again, the overall sample size is still relatively small. However, if these rates were to continue, it would suggest it is easier to run together years of underperforming the expected wins (doing worse than DVOA expects) than to run together years of doing better than expected.

Further, and again with the caveat that the universe of teams that managed to run together 2 years of underperforming is very small, the teams that did tended to end up having longer streaks than would have been expected.

Now here is something else I found interesting. In any given year, on average 36.86% of the teams are on one of those two extremes. The Eagles have a five year streak of this going (1 over, 4 under) and are pretty darned close to having an active streak of 7 straight-- they overperformed by 1.2 wins in 2005.

In fact, since Andy Reid became the Eagles coach in 1999, their actual wins have differed from their expected wins by at least 1.5 in either direction in 7 of the 10 years, and by 1.2 or more in 8 of the 10 years. (ETA- and, just to annoy those who were giving me a hard time, I'll point out that if you prorate this year out, they are on pace to do it again). It sure looks to me as if Andy Reid makes it difficult for DVOA to get a good read on the Eagles.

Other tidbits:
Three teams have only had their actual wins differ from their expected by 1.5 or more just twice. Two of them are 'new' teams-- Browns v2 and the Texans. Both obviously have had fewer years to work with. The third team is the Redskins.

Teams with three years: Atlanta, Seattle, and Dallas.

At the other end of the spectrum is Philly, as discussed above. But there are five teams that have also given DVOA some problems over the years, with 8 occurrences of differing from the estimated wins in 8 seasons.

The Ravens(Browns v1). All 8 times they underperformed. They had one of the 4 straight underperform-by-1.5 streaks and came close to making it 5.
The Vikings. In particular, they gave the formulas fits from 1994 through 2000. By year, the rounded wins were 3, -2, 3, 1, 4, 1, and 3.
The Colts. In particular, from 1995 through 1999 where they differed in rounded wins by 2,3,-2,-3, and 4.
The Raiders. They had one of the 4 straight underperform-by-1.5 streaks.
And, finally, the Giants. They have had a few years of role-reversal. In 1994, they overperformed by 2 rounded wins and followed that up by underperforming by 3. In 2002, they overperformed by 2 rounded wins and followed that up by underperforming by 2 the next year. Then, in 2006 they underperformed by two rounded wins and followed that up by overperforming by 2 the next year and winning the Super Bowl.

For what it's worth...

287
by The Guy You Don't Want to Hear (not verified) :: Thu, 10/22/2009 - 9:58pm

Kudos to Dales and nat for an interesting, spirited, numbers-filled debate! (And I am serious--in case that sounds like sarcasm.) I for one will do some work with the DVOA/Eagles problem at some point, but I don't have time for it right now (not to mention that my stats calculator has not made it here yet since I moved).

One thing that did come to mind reading this debate is--could Andy Reid be more prone to increase passing than usual, so that his team's passing numbers would spike in situations where another team's would not? I will give a related story as an example of what I'm saying: My freshman year in college we had a dorm tournament using the most recent NCAA Football game. I killed everyone. The one other person who was supposed to be really good at the game lost to me because even though we were close at halftime (I was up by three), he was so unused to being down at all that he got crazy--he was going for it on fourth downs, kicking onside, and throwing deep constantly just out of panic. (I ended up winning by 17 and then they changed the rules to try to keep me from winning anymore. Bastards.)
Maybe Andy Reid is the NFL version of that guy? Where a normal coach wouldn't deviate from his gameplan down 7, he's now throwing like a normal coach does when down 20? I don't have a play-by-play database to be able to run numbers and find out, but it's a thought and it would account for the fact that it does not seem apparent that they have been forced into passing by situation.

And of course there is a bit of an elephant in the room to all of this debate: Maybe the Eagles are actually better than their record the last 4 years and DVOA is capturing it. It's not scientific, but I think we would have to concede it's possible.

294
by Mr Shush :: Fri, 10/23/2009 - 7:18am

I'm far too lazy to do this myself, but given the suspicion that this may be about coaching, have you considered looking at this data for coaches rather than teams? Norv Turner averages 0.96 wins below expectation per season (excluding 2009), and is below par overall in each of his three coaching stops - is that significant? Are there more assclown head coaches than we might expect?

299
by nat :: Fri, 10/23/2009 - 9:19am

It's worth quite a bit, actually. Good job.

I agree that three teams with a streak of four years at 1.5 games below estimated wins is (a) not enough data to be conclusive and (b) interesting nonetheless. Like you, I would expect there to be zero or one other team besides the Eagles. I don't count the Eagles in this case, because we selected the criteria specifically to include them.

It's also interesting that some teams appear difficult to predict, but in both directions. If, as you imply and I am beginning to suspect, this also happens more often than mere chance would suggest, we get to some starting points for investigation.

Are there really teams with low predictability? Or is this mere chance?
If there are, how unpredictable are the teams with underperforming streaks?
If they are unpredictable, how does that change our assessment of their rarity?

My guess - just a guess - is that there are unpredictable teams, that some of the bad-streak teams fit the description, and that having three such teams is not as unlikely as we thought at first. I could easily be wrong.

300
by Dales :: Fri, 10/23/2009 - 10:14am

"It's also interesting that some teams appear difficult to predict, but in both directions. If, as you imply and I am beginning to suspect, this also happens more often than mere chance would suggest, we get to some starting points for investigation."

I implied that because I am starting to suspect it as well.

I also think that it is easier for a team to be 'predicted' too high than too low. The numbers just plain look that way to me, although I want to play with it more. ETA- I put predicted in quotes because estimated wins isn't really a prediction, but more of a calculation of what you would expect based on the DVOA performance.

I'll throw out a suggestion as to what that last part might be. DVOA (which is used to calculate the estimated wins) is based off of success rates in discrete events wherein there has been established a correlation between such successes and winning. We know luck is involved, and luck should over time (and will given enough time) even out. But there is another thing which could be involved, which would be stringing together things in a manner which differs from the norm; by stringing together I mean decision-making or other things that are in the control of those involved. Given that you have professionals who are, by and large, the cream of the crop, you would expect their decision making to have evolved, over time, to where the norm is pretty close to optimal. In this hypothesis, you would find it difficult to do things meaningfully above this norm, and easier to do things below it.

While nothing here proves this hypothesis, it would fit the data to the extent that if this is the case, you would expect there to be more cases where teams go on subpar streaks than the other direction. If all this is true (and I want to stew on it to try and think of ways to prove or disprove it), then it would point to there being some in-game decisions being made where there should be a cost against the team in the DVOA calculations when those decisions are made just like there is a cost when a team fumbles or commits one of the penalties that DVOA includes. What those decisions (again, for lack of a better word) might be, I don't know. Time management? Needless risk taking? Don't know. ETA-- IF this is the case and it is possible to figure out what these might be, a few monumentally big ifs, maybe we'll get lucky and have it be something discernable from the inputs into DVOA (game logs, etc).

Anyways, there is plenty to play with here, but while I am not sure if there will be anything that comes out proving anything, hopefully it'll be interesting.

301
by nat :: Fri, 10/23/2009 - 10:34am

This looks like a promising line of inquiry. Go ahead and mull it over. I have no doubt this will still be an interesting topic in future weeks.

Here's another line of thought. Let's try to design a team that will be wildly unpredictable. And let's try to design a team that will consistently underperform its estimated wins and DVOA.

For mere unpredictability, I could arrange my team to play an unusually high number of competetive games - by putting it into a balanced division, for example. Those games could go either way - and the number of wins would be subject to a high degree of variability that has less to do with overall DVOA-measured skills.

Or I could have the team use a scheme that depends on boom-or-bust results. It could be a defense that gives up yards but forces fumbles and interceptions. Or a running game that relies on breaking long runs rather than consistent yards. It could be a team that goes for it on fourth down a lot, or kicks onsides a lot. Again, the averages might be good, but they would also win or lose games they aren't expected to.

Skewing the wins down without lowering DVOA is harder, but possible. Bad fourth quarter play selection or clock management could do it. Bad fumble recovery skills, or a tendancy to give up long interception returns could do it - but FO analysis says these are essentially random. There may be other ideas.

My sense is that it's easy to change your predictability, making decisions that in the long run make sense. But lowering your wins without lowering your DVOA takes some doing, and there are fewer ways it could be done.

305
by AndyE :: Tue, 10/27/2009 - 10:45am

I think you could have a team vastly underperform DVOA simply by always throwing INTs in the red zone, or failing to convert on fourth down. Rack up a lot of positive DVOA marching downfield, then negate the effect of that by not scoring points.

290
by Pat (filler) (not verified) :: Fri, 10/23/2009 - 1:08am

How is that different than what you just said above?

It isn't. The "wrong way" statement was the fact that you were looking at games when they might've been forced to pass more than you'd expect. Those games aren't important - they'll bias the Eagles pass ratio up a bit, but since it's already high, it won't matter that much.

The games that matter are the games that would bias the pass ratio down due to being in preferential run situations. Those games are the ones where the Eagles have a lead in the fourth quarter and have the ball, and those games happen a lot. Every year. They happened more from 2000-2003 because of the whole "the Eagles won more" thing.

I concede this might be random chance, but think the odds of a team doing so are less than 1% and suggest that if it continues to happen it would indicate it isn't random.

It's not less than 1%. It's 5%. This isn't advanced math here: it's just binomial probability. Like I said, given 32 teams, you'd expect to get one or two outliers like that.

And that doesn't even take into account the fact that estimated wins isn't perfect.

295
by Dales :: Fri, 10/23/2009 - 7:37am

It is binomial math. That part you have right.

Compare what I had written to the actual numbers.

"The Eagles have underperformed their DVOA for going on five straight years by about 2 estimated wins a year. You think it is a statistical fluke. I concede this might be random chance, but think the odds of a team doing so are less than 1% and suggest that if it continues to happen it would indicate it isn't random."

(0.2026^4)*32 = 0.0003 * 32 = 0.0539
(0.2026^5)*32 = 0.0003 * 32 = 0.0109

Since at the time I wrote the above I hadn't compiled the numbers to figure out the probability of it happening in even one year (20.26%), that was a pretty accurate guess on my part.

You wouldn't expect to get one or two every year. You would expect to get one streak of four years (no less five) by one of the 32 teams every 20 seasons or so.

The odds of any team having five straight years like this isn't 5% for any single team (meaning you would get a few each year on average). It's 1.09% that any of the 32 teams would. We should see that once every 100 years. If the Eagles continue as they are so far this year, we're going to be treated to a 100 year flood.

236
by Might as well be Micheal Lombardi (not verified) :: Wed, 10/21/2009 - 6:03pm

Sheesh, more Gregg Easterbrooks. You know why the Eagles DVOA is so high? Because passing works. Running doesn't. Ask the 49ers.

You win in the NFL by throwing the football.

Look, the Eagles didn't show up for the Raider game. I love all the praise heaped on Louis Murphy for his downfield blocking...did anyone SEE the play? The Eagles were completely loafing. That last "block" by Murphy was hilarious. Ellis Hobbs looked like he was trying not to get dirty in his Sunday clothes.

Am I crazy that I totally discount this game? That I still think the Eagles are better than the Giants? No, I am not crazy. See: DVOA.

243
by Dales :: Wed, 10/21/2009 - 6:53pm

I see the point went right over your head.

My point isn't that the Eagles should run more and that this would make them better.

My point is that I suspect that since they don't run the ball more when they could, it leads to DVOA estimating their strength as being higher than it actually is.

As for if the Eagles are better than the Giants, they very well may be. Head to head I would almost bet that they will be due to specific matchup problems they pose for the Giants.

However, for the games that have been played this year, the Eagles have not performed as well as the Giants. And that is what DVOA is supposed to be measuring.

ETA-- the pass happy 49ers averaged just 1.23 PA/RA in 1985, 1.14 in 1968, 0.96 in 1987, 0.95 in 1988...

258
by Faux Lombardi (not verified) :: Thu, 10/22/2009 - 3:12am

How charming. No, I understood your argument, I just disagreed with it. I think you have a false premise, IE that for the games that have been played this year, the Eagles have not performed as well as the Giants.

"I think by now it is relatively clear that the recent vintage Eagles hit the sweet spots in the DVOA formulas to an extent that has them consistently measured a bit too high."

"I don't believe there is a good explanation for the Eagles DVOA being 10 percentage points higher than the Giants, based on how well each team did or did not play this year."

I disagree with both of these statements. I realize that most posters here think there is some statistical anomoly, but what if the stats are actually correct? What if Philly has just had some terrible luck? It wasn't long ago that people felt that there must be something seriously wrong with OPS because there was no way in the world that Adam Dunn was a better hitter than Alfonso Soriano.

The Cardinals were seconds away from winning a Super Bowl last year despite statitics that suggested this was impossible. The year before I think we can all agree that the Pats were statistically far, far superior to the lowly Giants.

Why can't it be that the Eagles are just really good? That while it SEEMS on the surface that the Giants performed better against common opponents, they actually didn't?

And I see my 49ers reference flew over your head. I meant the current team, the one that thinks passing is bad and running is good. That isn't going to work, but the purists will love it. The RBI and 20-game-winner lovers will be all over the 49ers toughness, their power in the trenches. Their swagger.

But the Eagles have a MUCH better chance of winning a championship through the air. Statistically speaking.

PS-over and over in this thread commenters explain why running the ball is so much better when you have the lead at the end of a game. This is incorrect. Ask Marty Schottenheimer. Or Bill Belicheck.

265
by Dales :: Thu, 10/22/2009 - 9:42am

"How charming. No, I understood your argument, I just disagreed with it. "

I am charming. Good looking, too. Not to mention smart and humble.

However, it sure doesn't read to me as if you were understanding my argument and just disagreeing with it. You asserted I was making a TMQ-type claim that the Eagles need to be balanced, which was not and is not my point. My point was merely that it is possible that DVOA would be more accurate about the Eagles if they were more balanced. That states nothing about if the Eagles would actually *be* better if they were more balanced. Would the Eagles be a better team if they were more balanced? I don't know. Maybe. Maybe not. I don't see any reason to assume that balance for the sake of being balanced would be a net positive for a team. However, I can conceptually imagine a statistical model that would be more accurate in measuring a team's strength when a team acts more like teams normally do than when a team acts differently than teams normally do.

Since you called that Easterbrookian, I thought it was pretty clear you missed the point. Does passing 'work'? The question is irrelevant to my point, which is concerned not with how strong the Eagles are, but with how accurate DVOA is about the Eagles.

"I disagree with both of these statements. I realize that most posters here think there is some statistical anomoly, but what if the stats are actually correct? What if Philly has just had some terrible luck? It wasn't long ago that people felt that there must be something seriously wrong with OPS because there was no way in the world that Adam Dunn was a better hitter than Alfonso Soriano."

What if DVOA is correct about the Eagles' strength and they have merely had some terrible luck? Well, for one, we should be able to identify the instances where this terrible luck happened. I don't think you can find enough luck to make up the difference in performance in the respective games vs. the Raiders. Were the Eagles unlucky in the Bucs game? Well, if all fumbles are equally likely to be recovered by the offense as the defense, then they should have had 0.5 more fumble recoveries, which doesn't strike me as much. If looking at the respective games against the Chiefs, the Giants were the ones who were on the wrong side of fumble luck. And so forth.

As for how the two teams have performed so far this year, disagree all you want on that tangent. Doesn't change the fact that the Giants, on the road, beat the Bucs in even more dominating fashion than the Eagles beat them at home. Doesn't change the fact that the Giants, on the road, beat the Chiefs equally as dominating as the Eagles did at home. Doesn't change the fact that the Giants utterly destroyed the Raiders while the Eagles lost. The difference between the two teams' performance in these games, representing half of the Giants played schedule and 60% of the Eagles played schedule, was substantial.

Both teams got skunked by the Saints-- the Giants, on the road, probably a bit worse than the Eagles, at home. That doesn't (or, perhaps stated better, shouldn't) come close to offsetting the gap caused by the three games above.

The rest of the schedule-- Eagles blowing out a weak Panthers team and the Giants beating the Cowboys and Redskins, should be close to a wash. Again, you can argue that the Eagles played better in that one game than in the Giants two, but it would be difficult to argue that they did so well as to overwhelm the difference between the two teams' performance in the Raiders game. At least, not to where it results in a 10% DVOA difference for the season to date.

When you look at your team, I don't blame you for throwing that one out as an outlier or a fluke. However, when you are talking about how well the team has played this year, you can't simply discard it. After all, it actually did happen.

The Eagles have played well. Just not as well as the Giants, and not as well enough to be considered the 2nd best team in football. But, again, that is beside the point. The real point is that for the fifth straight year, the Eagles are winning less frequently than would be expected for a team with their DVOA-- a sign that DVOA might be errantly high for them. Is it random chance? If not, how can it be improved? What is it about this team and the way they perform that causes DVOA to not be as accurate with it as it is in general? Take your Eagles fanhood out of the equation; I assure you I am not asking the question because I am a Giants fan. I'd be asking the same question if the team that was consistently for going on five years appearing near the top of the DVOA heap but performing in terms of wins and losses more like a team that should be meaningfully lower was the Texans and not the Eagles.

"The Cardinals were seconds away from winning a Super Bowl last year despite statitics that suggested this was impossible. The year before I think we can all agree that the Pats were statistically far, far superior to the lowly Giants"

Agreed on all points here. Not seeing how it is really relevant, though. The Cards are not in the midst of underperforming their DVOA by 2 games for the fifth straight year. The Giants were, as I argued often and consistently that year, a team that DVOA was accurately measuring in far as how well they had been performing, but were capable of playing much better than that (which they demonstrated in their playoff run). In no way do I think DVOA had the Giants pegged wrong in 2007, nor do I think DVOA had the Patriots pegged too high.

ETA- "And I see my 49ers reference flew over your head. I meant the current team, the one that thinks passing is bad and running is good. That isn't going to work, but the purists will love it. The RBI and 20-game-winner lovers will be all over the 49ers toughness, their power in the trenches. Their swagger.
But the Eagles have a MUCH better chance of winning a championship through the air. Statistically speaking.
PS-over and over in this thread commenters explain why running the ball is so much better when you have the lead at the end of a game. This is incorrect. Ask Marty Schottenheimer. Or Bill Belicheck."

Yes, I did not understand your 49er reference. If it is going to work for them or not, I don't know and is completely unrelated to my post. I don't have any belief that a team has to be able to run the ball particularly well in order to win at any particular rate. And I am certainly not arguing that the Eagles, as constituted, would have a better chance of winning a championship if they decided to run more often. And if other commenters have been arguing that it is better to run the ball at the end of a game with a lead also is irrelevant, since that isn't what I was arguing. At all.

237
by MJK :: Wed, 10/21/2009 - 6:28pm

Impressive analysis. It actually jives with something I've been thinking.

DVOA is essentially a measure of the probability that you will have "success" on any given play, assuming you're facing an "average" defense. Unless they've changed, "success" is defined as getting at least 40% of necessary yards on 1st down, 60% on 2nd, and 100% on 3rd or 4th.

I don't think anyone here would argue that, short of having Jamarcus Russel throwing the ball, a passing play will, on average, produce more yards than a running play. Passing is a more effective way to move the ball, on average. It's not quite the same thing to say that passing has a higher probability of gaining X number of yards on a given play, but I wouldn't be surprised that if X is at least 4, it is true, although a more careful analysis of rushing trends and passing trends could inform that. But if it's true, a team that heavily favors passing will have a higher DVOA (as well as tend to put up better raw yardage numbers)--because plays aren't successful some of the time, I suspect the X number of yards needed to qualify for DVOA success on any given play is 4 or greater.

So I think a team that favors passing more heavily will tend to put up a higher DVOA.

So the question is, if passing is generally more effective, why doesn't favoring passing correlate with success more? I think it's context related. Obviously, if a team passed every down and never even carried a RB, then their passing effectiveness would go down, and the team probably would lose more. If that wasn't the case, running plays would have vanished. However, DVOA doesn't take context into account...it merely counts the successful plays.

I have some thoughts about how to take context into account, but this is already a long post, so I'll flesh those out later...

169
by Sideshow Bob (not verified) :: Wed, 10/21/2009 - 11:20am

On the Broncos, maybe take a look at the age profile of the roster, if you haven't already done that. Seems like a lot of their 53-man roster was born somewhere between 1981 and 1986, which I would assume means that those guys are either ascending towards peak performance, or already there. I have no idea how their age profile stacks up versus other teams in the league, nor whether there is some sort of 'tipping point' of NFL experience that got crossed this year that didn't happen for them last year. But it seems like something that would be pretty easy to check on, since you have the birthdate data already.

170
by nath :: Wed, 10/21/2009 - 11:27am

I still just think the Eagles suffer from a mix of bad late-game decision making and crapping the bed against teams they shouldn't. This means they lose more close games than should be expected by the rest of their performance / more than can be explained by variance, plus they occasionally do something dumb like go lose to the Raiders.

I suspect San Diego's freefall in relation to its projection is because
A)Losing Jamal Williams is huge;
B)The projection predicted bounce-back years for Tomlinson and Merriman and neither has delivered. Tomlinson will probably never regain his old form; Merriman needs a year to let his knee repair + he doesn't have steroids anymore.

also, C)Norv Turner sucks and lots of parts on this team have been regressing. (Remember when their O-line was an awesome bunch of nobodies, a la the Giants in their Super Bowl run? Who would say that about them now?)

178
by Andy (not verified) :: Wed, 10/21/2009 - 11:44am

I agree with the Tomlinson sentiment completely. KUBIAK and the Almanac were predicting a return to form for LT this year, while a lot of analysts were saying he is probably cooked. Obviously this alone does not account for such lofty projections but had the system pegged LT for a decline we would probably be looking at a lower win projection for SD.

177
by IceBlock (not verified) :: Wed, 10/21/2009 - 11:44am

I don't think the problem is that the projections failed to capture Denver's upside, it is that it horribly exaggerated its downside. On a separate note, as an objective, numbers kind of guy I enjoy DVOA and tend to put more stock in it than other (mostly subjective, group-think) analysis, but interestingly this year comparing the ability of DAVE to predict games vs other popular power-rankings, DAVE has been wrong about 50% more often than anyone else. I measure a 46% error rate based on straight up DAVE comparisons, no hfa.

261
by Mr Shush :: Thu, 10/22/2009 - 7:40am

Not at all. Denver's downside was offensive decline from great to average owing to the change in QB and the possibility of linemen drafted for one scheme being a poorer fit in another, coupled with defensive improvement from historically atrocious to merely utterly bloody awful. That's not what happened, but it was a perfectly plausible scenario. The projections probably also thought some bad teams (notably Oakland and St. Louis) would improve more than they did, and that some formerly above average teams (notably Carolina and Tampa Bay) would not decline as much as they did, thus depressing Denver's projected ordinal ranking (but not their projected DVOA).

What we have is a league in which Denver's a pretty good team and a lot of teams (the most ever?) are terrible. What the formula projected was a league in which Denver was very bad and no teams were truly terrible. That's a bad outcome for the projections, but I don't think it stems from overstating Denver's cognitive downside as of this summer. It's just that Denver have hit their absolute upside and a lot of other teams have hit their absolute downside. This season is freakin' wierd.

190
by Rick A. (not verified) :: Wed, 10/21/2009 - 1:03pm

re 133, Jovins--No agreement that we saw "a case of Allen's play regressing to the mean". The guy exists, on his worst day, at a level far beyond the "mean". The Ravens showed how to attempt to handle the guy, other teams will follow suit, of course. That should provide some comical moments in games to come because few will be able to pull off what Balt did. The guy is peaking right now. The pundits talking him up are not making any ridiculous claims. Granted, the week before the Ravens he was facing quite possibly the worst offense line in the NFL, a game that I saw in person, but there is no minimizing what the guy is at this point in his career. I would venture to say that every coaching staff in the NFL would tell you the same thing. You can rest assured that everyone's first priority in meeting the Vikes is to try and figure what to do about that guy. He is, in fact, one of the most dominant players in the game, someone who entire game plans revolve around...

192
by Jovins :: Wed, 10/21/2009 - 1:40pm

Allen's playing mean, not the mean performance for a defensive end.
I was arguing that his past few games, Allen was playing better than he normally was; he was "hot". Now, he's playing more at a normal level.

And it really helps when you have a top three defensive tackle to play with. It's hard to ignore Kevin Williams; Jared Allen gets sacks every so often, and many hurries, but if he isn't gameplanned for, Kevin Williams consistently pushes offensive lineman back and destroys the pocket.

195
by inkt2002 :: Wed, 10/21/2009 - 1:54pm

" In addition, the game-by-game DVOA page now features not only single-game offensive and defensive DVOA but also both offense and defense split into rushing and passing."

I still dont see where this is available.

198
by tarey (not verified) :: Wed, 10/21/2009 - 2:21pm

The Denver rating is an outlier. But they had so many changes it should be expected that their DVOA (especially defense) would be harder to predict then the average team. I don't think you should change your formula for them.

But if you calculate it based on a bell curve or 10,000 simulations or something, I think it would be good to show where the bottom and top 10% mark is for the bell curve/simulations. So if you say your estimated number of wins is 5 (I'm not bothering to look it up), there consistency will give them a much higher/lower best/worst 10% mark.

To test if my argument is valid, you could put in all the good and bad surprise teams of the previous few years and compare their standard deviation (ignore bad surprise team with an injured star player who is normally not injured)

I think this could have some impact on the eagles since they seem to always (29 currently) have bad consistency. This to means, they can beat and lose to most teams. Which means they have chances of missing the playoffs and also have chances to go deep in the playoffs.

227
by Rick A. (not verified) :: Wed, 10/21/2009 - 5:16pm

True enough Jovins--Minny really has a heck of a DLine...

Then as to some of these posts that are going on about defense, rather than offense, having the advantage : you can't be serious. The Lords of the game have just about turned it into Flag Football regards what a fragile eggshell they have put around the QB. At this point it is truly ridiculous and deleterious to the game. Yes, they have to protect the no. 1 guys, the QB's, and the multi-million dollar investments the teams have in them but it's a really negative situation when they can virtually call their own penalties like Brady did a couple of weeks ago. Each year the number of penalized areas of where and how you can hit a QB increase. Then there was the move by the League over the last several years to penalize DB's in greater and greater ways and numbers. Those are just two of the most blatant moves that were done to INCREASE scoring. Someone in one of these posts said the League wanted to keep scoring consistent. Wrong. They set out to make it more frequent. And now, with the obvious rise of the number of crap teams due to free agency while other teams are getting better due to their owners actually wanting to win--we are starting to see ridiculously one-sided games at a greater frequency with Arena Football sized scores, at least on the part of the talented team--witness 59-0 and the week prior to that the Giant/Raid debacle. There will be more of these as a consequence of the rule changes and free agency era. The owners do not care that a significant percentage of people prefer to see monster defensive teams such as in the L.T. era, the Butkus era, the Singletary era, the Nietzke era, Steel Curtain era, etc. They just think the public goes for lots of scoring...

247
by bravehoptoad :: Wed, 10/21/2009 - 8:03pm

Dead serious. Imagine the offensive performance if suddenly the QB had -- not 1, not 2, but 5 more targets on every pass play. Now imagine how much it would slow the pass rush if offense had the same ability to shift as the defense. No more of those hair-trigger moves off the line. Offensive linemen would be faking out DL all the time. Imagine if there were no "illegal" formation penalties.

When you imagine that, you're imagining a game where the defense doesn't have all the huge advantages they have now that you're taking for granted. Instead you're focusing on some relatively tiny changes in how pass interference is called and how QBs are protected.

229
by Baldnbroke (not verified) :: Wed, 10/21/2009 - 5:25pm

I have a question concerning the playoff odds page. How is the odds of a team winning a game calculated in the simulation? I assume total DVOA is at the base of it, but how does two teams DVOA's add up to winning probabilities?

249
by Rick A. (not verified) :: Wed, 10/21/2009 - 8:56pm

Bravehoptoad, you're talking about the very basics of the structure of the game as it has been for 90+ years. I'm referring to extremely recent era changes designed to get more offense into pro football and break up that inherent to the game defensive advantage. Talk to anyone who has actually played pro football in the last, roughly, period--they will underscore everything I'm referring to. And they generally regard it as something that waters down the record book, makes offense stars out of people that are not really cut from the caliber that stars of generations past were, takes money out of the pocket of defensive players,and bores a significant percentage of the people out there that are actually fans of defensive football...

273
by bravehoptoad :: Thu, 10/22/2009 - 12:42pm

True enough. People who complain that the game is weighted toward the offense forget about the many, huge advantages the defense has. If you want to say that they're changing the rules to help the offense, that's true, and there are reasons for that (bigger, faster players on the same sized field.) But if you say the game favors the offense, you're wrong, and you can prove that by the thought experiment of repealing all the rules that inhibit only the offense (illegal procedure, illegal formation, illegal motion, ineligible man downfield, etc.)

282
by BigDerf :: Thu, 10/22/2009 - 5:02pm

The game of modern football still favors the offense.

Those penalties are in play because otherwise football wouldn't be football my friend. I believe you are talking about a rugby variation featuring downs and a line of scrimmage.

I mean.. yes if we implemented the changes you suggest scoring would probably increase into the triple digits but that doesn't mean the offense still doesn't already have an advantage.

250
by Rick A. (not verified) :: Wed, 10/21/2009 - 8:58pm

Correction : that would be "the last, roughly, 50 year period"

251
by Aussie Bengal (not verified) :: Wed, 10/21/2009 - 9:42pm

The Cincinnati Bengals are clearly ranked too low because they are 4-2 and beat up on a bunch of teams with higher rankings and lost to Denver only because, well, God hates the Bengals. Losing to the Texans is way better than this. WHO DEY!

Also, I'm not really sure what objective measurements would have told you that SD wouldn't be as good. I'm guessing that DVOA already took into account LT's age. It seems to me, though, that SD not being so great was a long time coming. I like them and all, but perhaps they were overperforming during the last couple seasons and it's taken two years for Norv Turner's suckitude to fully damage the talent that Schottenheimer & co. had assembled.

262
by Mr Shush :: Thu, 10/22/2009 - 7:48am

But Schottenheimer didn't assemble the talent - AJ Smith did, and AJ Smith's still there. Maybe Schottenheimer and his staff were vastly superior at the teaching element of coaching, and player development has suffered since he left - that's certainly plausible. But the projections have two years of Norv to work with - it's tough to see how anything Schottenheimer did on the scheme/playcalling front is still factoring into them, and again, neither Norv nor Marty was responsible for talent aquisition. If it's about coaching, it has to be about the way coaching is affecting the play of guys already on the roster (viz. making them progressively worse).

272
by tuluse :: Thu, 10/22/2009 - 12:38pm

Schottenheimer was a big stresser of fundementals. I think it's highly likely that he had a big effect on young talent fulfilling their potential.

Look at how the o-line just collapsed after he left.

257
by DeltaWhiskey :: Thu, 10/22/2009 - 2:50am

With the onset of the “DVOA is Broken Season,” the following comments/observations are relevant/pertinent:

- The overall correlation of DVOA with Wins from 1994 thru 2009 is 0.854.
- On a year to year basis, the correlation of DVOA with Wins ranges from 0.815 (2004) to 0.915 (1997). The average correlation is 0.857 with a SD of 0.028.
- DVOA’s correlation with Wins at this point in the season is 0.836

This data suggests the following:
- DVOA is a robust measure of the factors associated with winning games (R-square = .0.730)
- DVOA is a stable measure of these factors.
- DVOA appears to be performing as well this year as in any other year.

RE: PHI

If there is a problem with PHI, and based on earlier posts, I’m not sure there is, PHI may be an outlier of some sort. The question is, should DVOA be adjusted to try and bring in PHI (and other outliers) in line? My answer is, yes and no. Yes, adjustments should be attempted in an effort to improve DVOA overall. However, given the apparent chronic nature of the PHI problem, at some point the answer has to be “no,” and we have to accept that there is something systemic in how PHI plays that produces these results. That is, the problem (if it exists) is not DVOA, it is PHI.

268
by MJK :: Thu, 10/22/2009 - 11:31am

Very good points, but I disagree with your last statement.

If Philly has just happened to be a random outlier several times in a row, then fine. And this may be the case.

But IF there is something systematic in how Philly plays that consistently causes them to have a higher DVOA than their win total or actual performance on the field would imply (and I'm not sure that there is, but I'll concede that there may be), that means that there is something systematic that DVOA rates more highly than it should. In other words, there is something that DVOA thinks should correlate well to wins that Philly does that actually doesn't work as well as DVOA thinks it should. And that implies that if we can identify it and diminish it's important (or penalize it more heavily), DVOA could be systematically improved across the board.

If I'm running an experiment, and I find an analytical model that consistently predicts almost all my data points in a robust and stable manner, but one particular set of parameters consistently gives a result that lies well above the analytical model's prediction, then that indicates that there is some factor in my model that works well for most parameters, but not for that one particular combination--i.e. my model is missing something.

As to what it could be, I have some ideas, but I'm not even sure that "it" exists...I don't think there's enough data to conclude that Philly is always overrated (although we may be getting there). The right way to look for it is to look for other teams that consistently employ a scheme that are also consistently over or underrated, and see how they are similar, or different, from Philly, or to look for some way the Philly is remarkable. The fact that they are one of the most pass-oriented teams in neutral scenarios is one thing that stands out, so it has been discussed heavily, but there may be other things. Or possibly nothing.

But I think it is wrong to say that if there is something that PHilly does that is systematically different and responsible for the discrepancy means that DVOA should not be altered.

298
by DeltaWhiskey :: Fri, 10/23/2009 - 9:01am

“If I'm running an experiment, and I find an analytical model that consistently predicts almost all my data points in a robust and stable manner, but one particular set of parameters consistently gives a result that lies well above the analytical model's prediction, then that indicates that there is some factor in my model that works well for most parameters, but not for that one particular combination--i.e. my model is missing something”

or the Eagles are missing something that 31 other teams seem to get.

That is, it suggests that one of your subjects is very different from the rest. The question is, are the Eagles different from the rest. For the 10 years that it has been Reid/McNabb, they have underperformed expected wins by -0.6 (SD 1.64). On the surface, this does not suggest grossly underperforming, but I don’t have calculations for what the league average is on this parameter. We may be looking at a case with PHI, where simply, “the coin” for some reason seems to be on a streak of Heads (or Tails if you prefer). The following is the pattern (+/-) over the last 10 years (1999-2008): -, +*, -, +, +, +, -, -, -, -. (*2000 almost =, as Eagles exceeded expected wins by 0.1). This appears more like a random sequence of coin flips and I could easily imagine there being several teams in league with similar profiles.

If there is anything, given the robust nature of DVOA, the PHI organization should be looking internally and asking themselves, why are we not winning at a better rate (i.e what is it that we’re not doing right that the other 31 teams in the league are?). Ultimately, every year the question eventually becomes, what is wrong with DVOA, rather than, what is wrong with PHI (or the other team that just doesn’t look right)? This despite DVOA working pretty well for 97% of the teams.

I would like to see Aaron calculate the Average Deviance from expected wins for the DVOA ERA and calculate the SD of this measure. If PHI falls more than two SD beyond the mean, I’ll begin to believe there is something significant going on - most likely with PHI rather than DOVA. But at present, I think that either there is nothing to see here except a random and coincidental pattern, or maybe (less than 25% chance?) something is wrong with PHI.

274
by nat :: Thu, 10/22/2009 - 1:12pm

The correlation for estimated wins (adjusted for bye weeks) is even better: 0.885.

The Eagles aren't even in the running for outlier of the year. The outliers so far (in terms of beating or missing their estimated wins by one or more games) are the following.

Overachievers: DEN, MIN, OAK (awesome!), IND, CIN
Underachievers: STL, BAL, TEN

PHI is closer to it's estimated wins than 18 teams, even with the egg they layed in the Oakland game. Let's face it: shit happens. But if the Eagles are doing something that DVOA and estimated wins doesn't account for, more than half the league is doing it more. Or less.

More or less.

285
by Dales :: Thu, 10/22/2009 - 6:40pm

Nat,

If I understand it correctly -and I am sure you'll correct me if not :-) - Estimated wins "projects a number of wins adjusted to a league-average schedule"

By the end of the year, the overall schedule a team faces starts getting near enough to zero that I felt confident in ignoring it's effect when doing the data dig I did above.

If you take the Eagles' estimated wins of 4.3 (which is calculated as if they had played 6 games), and take 5/6 of it to get to what it would estimate for them after five games to match the five they've played, it works out to 3.6. (I know you have done this, but others are reading this as well).

By the end of the year, the strength of schedule will likely be close enough to average that there is no big need to account for it (in my opinion) when comparing actual wins to estimated wins.

The Eagles have 0.6 fewer wins than they should have based off of their DVOA rating if they had played a perfectly average schedule. But right now, that isn't the case. They've played a really easy schedule-- 30th most difficult in fact. So while it is true that there are many teams that are further from their estimated wins than the Eagles, most of them do not have it compounded by the fact that they have played a significantly easier schedule than the one used to calculate the estimated wins.

And that's without beating the dead horse that is the fact that the Eagles haven't just been doing it this year, but also the year before, and the year before, and the year before, and, if we are talking about differing meaningfully from what estimated wins would suggest in any direction at all, for most of Reid's tenure.

I know, I know. You can accuse me of homerism. But my Giants are overperforming their estimated wins, there is no history to suggest that will continue, and the easy part of the schedule is long gone. I suspect the G-men's DVOA is going to slide in the coming weeks a bit, as is the W-L record.

286
by mm (not verified) :: Thu, 10/22/2009 - 9:43pm

By the end of the year, the strength of schedule will likely be close enough to average that there is no big need to account for it (in my opinion) when comparing actual wins to estimated wins.

I think you're very wrong here. There are 32 teams, and you only play 10 games against the 28 teams not in your division. Before the last expansions, there was 1 or 2 fewer teams, but you played even less games against non-division teams. Yeah, many teams will play 'normal' schedules...but many will play at each extreme. A team with 3 easy or 3 hard teams in its division might gain or lose some wins even if the rest of the schedule is average.

For an example, look at the AFC East last year. Each team played 6 games against each other, 4 games against the AFC West, and 4 games against the AFC East. Both divisions were thought of as poor. This inflated the AFC East records and their reputation in the public's minds. Last year NE outperformed its estimated wins by 1.4 (despite losing both its games to teams that weren't in those 3 divisions), Miami outperformed its estimated wins by 1.7, Buffalo by 0.1, and the Jets hit their estimated wins perfectly. So the division averaged +.8 wins over average, with nobody underperforming. It certainly seems everyone in the division got a free win from the schedule, 2 teams overperformed by a game (for a net +2), and 2 teams underperformed by a game (for a net +0).

If you want to use an 'estimated wins' number, you'd want to use a figure calculated like the playoff odds are, simulating each of the 16 games against teams on their schedule and then averaging the results. Note that just taking the 'schedule strength' value listed in DVOA isn't very helpful, since 8 games against top teams and 8 games against bottom teams would produce the same average as 16 games against average teams, yet you'd anticipate different results.

For the Eagles this year, they lost 1 game to the team above them in DVOA, they won 3 games against teams below them in DVOA...there's only 1 'fluke' loss. And DVOA ranks them the 3rd least consistent team in the league. There certainly isn't anything dramatically wrong with this picture, especially if you think the Carolina collapse was a bit flukish itself (not the Eagles win, but the sheer amount of self-destruction that Caroling committed). Note that the playoff odds think the Eagles will likely go 7-4 the rest of the way, so the system thinks it will lose 4 more games against teams ranked below it on the DVOA scale.

288
by Dales :: Thu, 10/22/2009 - 10:08pm

"I think you're very wrong here."

Well, I looked at the average schedule DVOA for the past few years. You get the occasional schedule that is more than 10% above or below average but it looks to me like less than 1 in 30 schedules ends up being like that. It would be superior to do it like you suggest, but since I don't have a way of doing that, and since estimated wins each year is on the website...

You don't use the stats you wish you had. You use the stats you have.

302
by mm (not verified) :: Fri, 10/23/2009 - 11:55am

Oh, I acknowledge that those numbers aren't easily available, but my general point is that I believe that we should expect 'expected wins' to deviate from actual wins more often due to schedule. Your numbers fit that picture.

It would certainly be nice if they would incorporate an 'expected wins versus schedule' statistic in the future to examine this more easily.