Writers of Pro Football Prospectus 2008

Most Recent FO Features

LongKyl13.jpg

» Word of Muth: Bears Upset 49ers

The Bears got the win and Kyle Long had the block of his dreams, but Ben Muth found plenty of room for improvement in Chicago's offensive line.

03 Jan 2006

Final 2005 DVOA Ratings

by Aaron Schatz

Indianapolis is the number one team in the NFL for 2005, according to the Football Outsiders DVOA (Defense-adjusted Value Over Average) system. That's no surprise. But you might be surprised that second-place Denver is just 0.1% behind them. More on that in a moment.

Extended FOXSports.com commentary is found here.

This season has been a long and winding road for Football Outsiders and for the DVOA system thanks to our new relationship with FOX. Every year brings a new challenge to this system, and this year it was a tremendously skewed NFL schedule, which in turn led to the week where Indianapolis was both first and sixth at the same time. Doing the FOX Power Rankings gave us the opportunity to reach a larger new audience -- not to mention some financial remuneration -- but it's hard to explain to a wider audience that DVOA is a work in progress. I can't overemphasize how difficult it was to try to improve DVOA while under a public microscope, and the problem was compounded by the longtime FO readers who accused me of "selling out the movement" simply because I did not have time to do hours of research that particular week.

The "second-order opponent adjustments" change definitely improved the accuracy of DVOA but if you check the numbers below you'll see that this ended up being much ado about nothing. The rankings for old-style DVOA (marked 1st ORD DVOA) are basically the same as the rankings for new-style DVOA. The problem was that so many teams had schedules that were front-loaded or back-loaded with easy opponents. The Colts, who at midseason had the easiest schedule in the history of our ratings, didn't even end up with one of the five easiest schedules for the full year.

But wait, just when you thought DVOA was out of the woods, we've got another problem. Week 17 has become a complete joke, and it has destroyed the numbers.

The "sit your starters" thing has reached insane levels, as this year for the first time teams were sitting their starters even though they could improve their seeds by winning. Last year, we noted that teams didn't play that much worse when they had nothing to play for in the final week, with a couple of exceptions. But in the last two years, that couple of exceptions has become a lot of exceptions. We're not talking about backup quarterbacks here, we're talking about entire teams laying down and happily getting steamrolled.

Denver nearly topped Indianapolis in total season DVOA because the Colts were resting starters, and Denver is number one in weighted DVOA. Kansas City moved ahead of Seattle. Cincinnati dropped behind both of those teams, from second to sixth, and all the way down to 12th in weighted DVOA. Chicago dropped from the top spot in weighted defense. New England dropped back behind Miami in total DVOA. Minnesota moved up four spots in total DVOA in one week, and ended up tenth in weighted DVOA. Tenth! It's ridiculous.

This is screwing things up. I'm not just talking about screwing up the total ratings for the end of the season. I'm talking about screwing up my analysis for individual playoff matchups. Should I remove Week 17 in judging how the Patriots do against different types of receivers? Does it count for the Cincinnati run defense? Do two weeks of Jim Sorgi and Aaron Moorehead represent the Indianapolis offense that will be playing in January?

The proper response to this change in the strategy coaches use for the final week of the season would be to study just how much DVOA declines when a team sits its starters, and to create either an objective adjustment for when a team has nothing to play for or a set of rules to be used to remove certain plays from consideration in team ratings. I've got some ideas for how to do this. But -- and I'm sorry to upset some of you -- I just don't have the time right now.

So while the numbers below are the straight DVOA ratings representing every single play of the season, Bam Childress or no Bam Childress, the power rankings on FOXSports.com include a subjective decision to remove large parts of certain Week 17 games after teams had removed most of their starters. We debated this on the FO staff discussion list and there were opinions on both sides, so FOX is getting the subjective and you are getting the objective and if you feel like bitching and moaning about how I'm selling out the movement, fine, you put $500 down on the playoffs based on whether Bam Childress can play defensive end.

* * * * *

Anyway, Indianapolis also ends up as the top team in Pythagorean wins, always a good indicator of which team will win the Super Bowl. The Colts are also unique as the first team to finish number one in DVOA and yet number 32 in variance, meaning they were the best and the most consistent. In fact, the Colts finish number 32 in variance despite sitting their starters in those last two games.

Kansas City wins the 2004 Buffalo Award for team that got hot at the end of the year but didn't make the playoffs.

And here's one more shocking thing about this year's ratings: there is not a single team with a losing record that has a higher DVOA than a team with a winning record. The top 17 teams are 9-7 or better. From 18 down, you've got 8-8 Atlanta and 14 teams that are 6-10 or worse.

The FOXSports.com commentary is now online. Each comment for a team that missed the playoffs presents reasons for optimism and pessimism in 2006. There's also a further exploration of the Week 17 problem and some discussion of the major trends of 2005 such as the disappearance of the 2004 offensive explosion and the number of good teams with poor special teams.

All individual stats page are now up, along with offensive and defensive lines. The usual huge playoff previews are coming on Friday. We also have a very special Too Deep Zone this week, you'll like it.

* * * * *

To save people some time, we remind everyone to put their angry troll hatred into the official zlionsfan angry troll hatred Mad Libs form:

<team> is clearly ranked <too high/too low> because <reason unrelated to DVOA>. <subjective ranking system> is way better than this. <unrelated team-supporting or -denigrating comment, preferably with poor spelling and/or chat-acceptable spelling>

* * * * *

These are the Football Outsiders team efficiency ratings for the entire 2005 regular season, measured by our proprietary Defense-adjusted Value Over Average (DVOA) system that breaks down every single play and compares a team's performance to the league average based on situation in order to determine value over average. (Explained further here.)

OFFENSE and DEFENSE DVOA are adjusted based on strength of opponent as well as to consider all fumbles, kept or lost, as equal value. SPECIAL TEAMS DVOA is adjusted for type of stadium (warm, cold, dome, Denver/Mexico City) and week of season.

WEIGHTED DVOA represents an attempt to figure out how a team is playing right now, as opposed to over the season as a whole, by making recent games more important than earlier games. This is the statistic used for the FOXSports.com Power Rankings. WEIGHTED DVOA for offense, defense, and special teams is available on those separate pages.

IMPORTANT: Beginning with Week 12 of 2005, DVOA is based on second-order opponent adjustments. Until other years are updated with this new system, only first-order DVOA (second table) can be compared to previous seasons.

LAST WEEK represents rating after 16 weeks of 2005. LAST YEAR represents 2004 rating (second-order DVOA version, so these may not be in the same order as the 2004 team efficiency page).

As always, positive numbers represent more points so DEFENSE is better when it is NEGATIVE.


TEAM TOTAL
DVOA
W-L LAST
YEAR

LAST
WEEK
WEIGHTED
DVOA
RANK OFFENSE
DVOA
OFF.
RANK
DEFENSE
DVOA
DEF.
RANK
SPECIAL
DVOA
S.T.
RANK
1 IND 36.6% 14-2 3 1 35.9% 3 30.5% 1 -10.5% 8 -4.4% 30
2 DEN 36.5% 13-3 9 3 43.1% 1 25.4% 4 -11.3% 6 -0.1% 21
3 KC 31.1% 10-6 10 8 40.3% 2 25.7% 3 -6.6% 13 -1.3% 25
4 SEA 28.2% 13-3 20 4 26.3% 4 26.4% 2 -1.0% 18 0.9% 15
5 PIT 26.1% 11-5 2 5 22.1% 6 10.4% 8 -15.4% 3 0.3% 19
6 CIN 25.1% 11-5 11 2 11.4% 12 21.8% 6 -2.1% 17 1.2% 12
7 NYG 24.6% 11-5 23 7 21.0% 7 9.5% 9 -8.6% 11 6.5% 3
8 SD 23.9% 9-7 8 6 17.5% 9 23.3% 5 0.1% 19 0.8% 16
9 WAS 22.6% 10-6 17 10 26.3% 5 6.1% 12 -15.5% 2 1.0% 14
10 JAC 21.7% 12-4 13 9 16.3% 11 7.1% 11 -11.8% 5 2.9% 8
11 CAR 13.0% 11-5 12 12 18.0% 8 -5.7% 18 -14.8% 4 3.8% 7
12 CHI 7.0% 11-5 31 11 3.5% 15 -18.0% 29 -24.6% 1 0.4% 18
13 TB 4.6% 11-5 16 15 3.8% 14 -5.1% 17 -10.0% 10 -0.2% 23
14 MIA 4.5% 9-7 25 16 1.0% 17 -11.6% 21 -10.1% 9 6.0% 4
15 NE 4.4% 10-6 1 14 5.1% 13 15.2% 7 12.9% 27 2.0% 10
16 DAL 1.7% 9-7 24 13 -7.6% 19 -2.7% 14 -4.3% 14 0.1% 20
TEAM TOTAL
DVOA
W-L LAST
YEAR

LAST
WEEK
WEIGHTED
DVOA
RANK OFFENSE
DVOA
OFF.
RANK
DEFENSE
DVOA
DEF.
RANK
SPECIAL
DVOA
S.T.
RANK
17 MIN -0.1% 9-7 15 21 16.7% 10 -4.9% 16 -3.5% 15 1.3% 11
18 PHI -3.0% 6-10 5 17 -8.3% 20 -11.8% 22 -8.3% 12 0.5% 17
19 OAK -4.3% 4-12 28 20 -6.9% 18 3.7% 13 3.5% 23 -4.5% 31
20 BAL -6.1% 6-10 7 19 1.8% 16 -17.9% 28 -10.7% 7 1.1% 13
21 ATL -6.9% 8-8 18 18 -13.8% 22 7.1% 10 13.9% 28 -0.2% 22
22 ARI -14.4% 5-11 27 24 -8.4% 21 -10.5% 20 1.6% 21 -2.4% 27
23 CLE -15.4% 6-10 26 22 -19.9% 24 -13.8% 25 6.1% 24 4.4% 5
24 TEN -18.3% 4-12 29 23 -20.2% 25 -4.7% 15 18.0% 30 4.4% 6
25 DET -19.1% 5-11 19 26 -16.0% 23 -14.3% 26 1.2% 20 -3.5% 28
26 BUF -19.2% 5-11 4 25 -21.7% 27 -18.4% 30 9.5% 25 8.7% 1
27 NYJ -21.6% 4-12 6 27 -22.6% 29 -23.0% 31 -3.5% 16 -2.1% 26
28 GB -22.3% 4-12 21 28 -24.2% 30 -12.4% 23 2.8% 22 -7.2% 32
29 STL -23.0% 6-10 30 29 -21.5% 26 -6.7% 19 15.6% 29 -0.8% 24
30 NO -28.2% 3-13 22 30 -26.1% 31 -13.1% 24 11.1% 26 -4.0% 29
31 HOU -33.3% 2-14 14 31 -21.9% 28 -15.5% 27 26.1% 32 8.3% 2
32 SF -65.9% 4-12 32 32 -60.8% 32 -46.9% 32 21.5% 31 2.4% 9

  • ESTIMATED WINS uses a statistic known as "Forest Index" that emphasizes consistency as well as DVOA in the most important specific situations: red zone defense, first quarter offense, and performance in the second half when the score is close.  It then projects a number of wins adjusted to a league-average schedule and a league-average rate of recovering fumbles.
  • PYTHAGOREAN WINS represent the number of wins projected from the team's points scored and allowed, as described in this article.
  • 1st ORDER DVOA is the "older style" DVOA from before Week 12 2005 which only runs opponent adjustments once. This stat should be used to compare with previous seasons.
  • NON-ADJ VOA is total VOA without any adjustment for opponent strength, luck in recovering fumbles, or the effects of weather and altitude on special teams.
  • 2005 SCHEDULE lists average DVOA of opponents played this season, ranked from hardest schedule (#1, most positive) to easiest schedule (#32, most negative). It is not adjusted for which games are home or road.
  • VARIANCE (VAR.) measures the statistical variance of the team's weekly DVOA performance. Teams are ranked from least consistent (#1, highest variance) to most consistent (#32, smallest variance).


TEAM TOTAL
DVOA
W-L ESTIM.
WINS
RANK PYTH
WINS
RANK 1st ORD
DVOA
RANK NON-ADJ
VOA
2005
SCHED
RANK VAR. RANK
1 IND 36.6% 14-2 13.5 1 12.7 1 36.1% 1 41.6% -4.8% 27 8.9% 32
2 DEN 36.5% 13-3 12.6 2 11.7 3 33.6% 2 32.0% 8.2% 3 17.1% 25
3 KC 31.1% 10-6 11.6 4 10.0 10 27.5% 4 22.7% 7.4% 4 24.3% 11
4 SEA 28.2% 13-3 12.0 3 12.3 2 28.1% 3 38.0% -11.5% 32 16.5% 26
5 PIT 26.1% 11-5 11.5 5 11.6 5 24.6% 6 30.2% 0.4% 17 21.2% 16
6 CIN 25.1% 11-5 11.1 7 9.7 12 24.9% 5 26.5% -0.5% 19 28.1% 5
7 NYG 24.6% 11-5 10.8 9 10.7 6 23.6% 7 27.0% 1.2% 15 21.3% 15
8 SD 23.9% 9-7 11.3 6 10.7 8 22.5% 10 10.1% 12.7% 1 12.3% 30
9 WAS 22.6% 10-6 10.6 10 9.9 11 22.5% 9 12.8% 4.8% 7 25.2% 9
10 JAC 21.7% 12-4 10.8 8 10.7 7 22.6% 8 25.0% -3.8% 24 19.6% 18
11 CAR 13.0% 11-5 9.7 11 11.6 4 13.9% 11 28.0% -8.8% 30 20.8% 17
12 CHI 7.0% 11-5 9.1 13 10.3 9 5.0% 12 6.3% -8.1% 29 32.2% 3
13 TB 4.6% 11-5 8.5 16 8.9 14 4.9% 13 13.1% -9.7% 31 23.0% 14
14 MIA 4.5% 9-7 9.0 14 8.0 17 3.6% 15 5.9% -2.3% 21 17.6% 22
15 NE 4.4% 10-6 9.1 12 9.1 13 4.6% 14 2.6% 3.7% 10 17.3% 23
16 DAL 1.7% 9-7 8.3 17 8.5 15 0.6% 17 -1.3% 5.9% 5 24.8% 10
TEAM TOTAL
DVOA
W-L ESTIM.
WINS
RANK PYTH
WINS
RANK 1st ORD
DVOA
RANK NON-ADJ
VOA
2005
SCHED
RANK VAR. RANK
17 MIN -0.1% 9-7 8.9 15 6.9 18 1.6% 16 -3.0% -2.6% 22 30.2% 4
18 PHI -3.0% 6-10 7.7 18 5.9 23 -5.1% 19 -10.7% 5.0% 6 24.2% 12
19 OAK -4.3% 4-12 7.6 19 5.5 25 -7.6% 21 -4.2% 10.2% 2 10.0% 31
20 BAL -6.1% 6-10 7.0 21 6.9 19 -5.8% 20 -8.2% 3.6% 11 27.9% 6
21 ATL -6.9% 8-8 7.3 20 8.3 16 -4.4% 18 0.2% -4.0% 25 14.3% 29
22 ARI -14.4% 5-11 6.1 23 6.0 22 -13.3% 22 -11.2% -4.7% 26 15.9% 28
23 CLE -15.4% 6-10 6.5 22 5.6 24 -15.2% 23 -22.8% 3.9% 9 19.4% 20
24 TEN -18.3% 4-12 5.3 26 4.9 28 -17.3% 24 -13.7% 0.3% 18 17.3% 24
25 DET -19.1% 5-11 5.8 24 5.2 27 -17.6% 25 -21.8% -2.1% 20 25.4% 8
26 BUF -19.2% 5-11 5.5 25 5.2 26 -19.4% 26 -18.0% 2.2% 14 33.2% 2
27 NYJ -21.6% 4-12 4.8 29 4.5 29 -21.6% 29 -28.9% 4.0% 8 16.1% 27
28 GB -22.3% 4-12 4.5 30 6.7 20 -19.8% 27 -21.9% 0.7% 16 27.0% 7
29 STL -23.0% 6-10 5.0 28 6.4 21 -21.6% 28 -16.8% -5.5% 28 19.5% 19
30 NO -28.2% 3-13 5.3 27 3.6 31 -25.8% 30 -28.1% -2.8% 23 23.2% 13
31 HOU -33.3% 2-14 4.2 31 3.7 30 -31.7% 31 -35.6% 2.9% 13 18.5% 21
32 SF -65.9% 4-12 1.8 32 3.2 32 -63.0% 32 -61.8% 3.3% 12 39.3% 1

Posted by: Aaron Schatz on 03 Jan 2006

104 comments, Last at 11 Jan 2006, 1:04pm by Sid

Comments

1
by David (not verified) :: Tue, 01/03/2006 - 7:30pm

Memo to Washington:

Please lose in the first round. You're making Joe Thiesman look good.

2
by JMM (not verified) :: Tue, 01/03/2006 - 7:40pm

Steelers are clearly ranked too low because of Tommy Gun Maddox. Dart throwing is way better than this. The Steelers have waited too long and have turned their backs on the traditional playoff apporach by going for home field advantage and trying "the road trip" apporachy.
Ewe guise hear shore our tuff, yea, write!

3
by turbohappy (not verified) :: Tue, 01/03/2006 - 7:42pm

Interesting to see San Diego finish the year at 30th in variance given the fact that most people would describe their year as "inconsistent".

4
by DK (not verified) :: Tue, 01/03/2006 - 7:43pm

Week 17 is really just part of a larger problem, and the only way to fix it is with a dramatic overhaul of the system.

In order to fully account for dynamic on-field rosters caused by resting starters in week 17 and in blowouts, and dramatic injury differences such as the Eagles, it would be necessary to record exactly what personnel is on the field for every play, and then determine each player's effect on play-by-play DVOA by looking at every play that player was involved in. Part-timers would probably need to be regressed to a mean.

This player-by-player DVOA could then be used to determine a first-string, second-string, and third-string DVOA for each team, and give more credit to first-string DVOAs on teams that tend to be healthier. Power rankings could remove IR players from the equation altogether.

It would be a difficult undertaking, and it may have too many holes to work properly, but I think it's the only way to truly solve the issue of rotating rosters.

5
by Jon M (not verified) :: Tue, 01/03/2006 - 7:44pm

Re: Kansas City

Is this (#3 in total DVOA) the highest a non-playoff team has ever finished? they must still be gnashing their teeth over the Buffalo and Philly games. I'd say that this bodes well for their future, but then I see the Bills finished at #5 last year, so maybe not so much

6
by admin :: Tue, 01/03/2006 - 7:44pm

And then, I'll appear on a very special episode of Oprah, "My Daddy Had No Time For Me."

7
by EorrFU (not verified) :: Tue, 01/03/2006 - 7:47pm

The AFC West teams all had the hardest schedules in the league with rankings 1-4. Is it normal for a single conference to own the hardest schedule or is it the fact that they had to play the NFC East wich happens to own 5-7 in strength of schedule as well. Those NYG's got lucky playing the Vikes and New Orleans which seemed to have dropped the SOS all the way to 15. Even if they had a fluke loss against hose Vikes.

8
by Bowman (not verified) :: Tue, 01/03/2006 - 7:49pm

Both NYG and JAX narrowly miss ranking in the top 10 Offense, Defense, AND Special Teams. Doing a quick look at prior years, only dominant teams (2004 PIT, 1999 StL + JAX, 1998 ATL) have been good in all three aspects.

9
by dfarrar777 (not verified) :: Tue, 01/03/2006 - 7:52pm

"My Daddy Had No Time For Me..."

And then, Mitch Albom will write a book about you. :-)

10
by jim's apple pie (not verified) :: Tue, 01/03/2006 - 7:58pm

Atlanta sure got a lot of Pro Bowlers for not being anything special. ::ducks::

Did San Diego end up with the hardest schedule in DVOA history? I doubt that DVOA takes into account that they had 5 east coast road trips, plus 5 games against teams coming off of their bye, to boot. Looking at their much easier schedule for next year, I've got to think they'll be fighting Denver for the division crown.

Also, it looks like LT has been playing with a cracked rib since the Oakland cheap shot. Come on, coaches, you've got to have the huevos to pull your stars if they're injured. These guys are competitive, they're not going to pull themselves out. Michael Turner is a pretty good backup, and I actually think that he was the BETTER option in the last two games, despite the LT injury. Why? Poor field conditions. The Chargers OL was atrocious this year. Without blocking, LT has to gain yards through his cutting ability. Muddy field = no cutting. No run blocking plus muddy field plus LT = no yards. Turner, however, is a big bruising type of back that is hard to take down. He would have been a close approximation of Larry Johnson, and probably would have done much better in both games.

It will be interesting to see what happens to Miami next year. This sure seems like a pretty fluky 9-7 though.

I don't see how the Texans can keep the first pick. They need to work some deals so that they can get a great offensive lineman plus some great defenders. Replacing Carr or Davis at huge expense seems like spinning your wheels.

11
by Bowman (not verified) :: Tue, 01/03/2006 - 7:58pm

I'm trying to debate whether I care about the variation of DVOA as it relates to teams losing starters for any length of time - that is what DPAR is for. However, if Vick really does cause his defense to play better when he plays, than that won't show up in DPAR, but will in DVOA - right??

I'm anticipating the playoff preview, where we will see various players, QB or DL, where their team's DVOA shows a massive improvement when they play. Accounting for quality depth is one of many improvements of DVOA over conventional stats - right?? Otherwise, will we throw out all plays / situations where the backups play? (which is the entire problem with the W17 games.)

12
by Bowman (not verified) :: Tue, 01/03/2006 - 8:11pm

I'm surprised to see OAK with the "unluckiest" season - 3.6 wins below their estimated wins. This is the lowest I see since 2001 SD (4.2 below estimated wins!?!).

Forgive my obvious East Coast Bias - did Oakland actually play fairly well, or is this an oddity of the schedule strength?

13
by Bobman (not verified) :: Tue, 01/03/2006 - 8:12pm

This DVOA thing is crap, complete, utter, nonsensical... what's that, the Colts are #1? Whoopee. Never mind.

Seriously though, what to do about Week 17...? In my job, financial analysis of real estate assets, the "textbook" way to do it is as it is done now, with what I'll call the "Jim Sorgi weeks" (actually, check out his rating! the kid's okay) included. I'd say to my client, "here's the whole picture, but note the final weeks are not really reliable," and leave it at that.

But if I were investing my own money, I'd cherry-pick game by game from the final 2-3 weeks to throw out the ones that were obviously atypical because of playoff/injury issues. Miami at NE? gone. Indy at Seattle? Gone. Det and Pitt? That... looks like keeper, unlike Matt Millen. The differences in the two analyses would be small to be sure, but might prove to be crucial (to an investment).

I only make this as the mildest suggestion, because I know that cherry-picking games is instantly subjective, which kind of contaminated the pure objectivity (Ayn Rand be damned) of the DVOA system.

Maybe the answer is to reduce the percentage weighting on the last weeks the way the first weeks of the season are watered down. if Week 16 and Week 16 now count 100%, maybe they should be reduced to 95% and 85% and see if things "look right."

The reason I would tend closer to the cherry-picking is that SOME games in the final week matter, and more in Week 16 matter, so throwing out or reducing the importance of a major win to launch a team in the playoffs (I'm thinking about the Ravens' major late-season surge in their championship year) seems wrong.

Other than that, I love this system. I have some minor quibbles about how 4th downs are handled (already brought up with management), but this system is super.

13
by thad (not verified) :: Tue, 01/03/2006 - 8:12pm

Jeez Aaron, feeling a bit defensive?
Seriously, I know you guys get a lot of negative comments but I think you, and the whole outsiders team, have done a great job this year.

15
by Kal (not verified) :: Tue, 01/03/2006 - 8:16pm

Aaron, is there any way that this could be farmed to the community? I know it's a huge undertaking to do all the work yourself, but could the community do the work under a process that you create?

16
by Kibbles (not verified) :: Tue, 01/03/2006 - 8:24pm

Re #4: I strongly disagree with that. I think that ALL NUMBERS SHOULD COUNT as long as the team is trying its absolute hardest to win games. Did Tommy Maddox submarine Pittsburgh's early season ratings? Yup, but on the other hand, DVOA should take into account that their QB is nicked up and their backup is abysmal. The fact is, they were trying as hard as they possibly could to win those games, so the numbers have to stand, in my opinion.

Likewise, in the Denver-San Diego week 17 contest, San Diego was trying its absolute hardest to win the entire time. Yes, they had to sub Brees out for Rivers due to injury, but they had Tomlinson in and taking carries the entire game, and they had several injured players (listed as questionable) who started and played the entire game. Meanwhile, Denver was trying its hardest to win the game, too. It had done a full gameplan, and was calling fancy blitzes. It kept its starters in for almost the entire time that the game was in question, pulling Plummer at halftime and keeping guys like Champ Bailey in for almost the entire contest. As such, the numbers should count.

On the other hand, you have Cincinnati and New England. I say you should count all of those teams offensive numbers (and their opponent's defensive numbers) for every drive that Carson Palmer and Tom Brady were in. The second that Palmer and Brady got pulled, it was still early and the game was still very much in doubt, so it was clear their offense was no longer trying its hardest to win, so all subsequent numbers for the Cin/NE offense and KC/Miami defense should be considered suspect and removed from consideration. On the other hand, New England played its starting defense for the entire game, if I recall correctly, which means that NE's defensive DVOA from the entire game should remain valid.

I think going through and trying to determine when each unit stopped trying its hardest to win would be a difficult and highly subjective excercise, but personally, I believe that it's the best way to keep the integrity of DVOA as intact as possible.

Also, on a side note, is anyone really surprised that KC finished ranked so high? Everyone was talking about how SD could be the best team in NFL history to miss the playoffs (I dunno, I seem to recall an 11-5 Broncos squad sitting at home once...), but they weren't even the best in their own division to miss the playoffs.

Here's a telling stat: Only three teams in the AFC finished with a winning record against winning clubs. Denver (7-3), Indy (5-2), and KC (6-4).

17
by Larry R. (not verified) :: Tue, 01/03/2006 - 8:25pm

Funny when Sea played Indy it was Indy playing a second and third string players but the DVOA did not drop but when Sea played second and third string players against Green Bay they dropped one whole slot...... And you say this is not biased????? Show how you come up with this...Green Bay played their hearts out and Brett Favre had his best day of the year.

18
by Kibbles (not verified) :: Tue, 01/03/2006 - 8:28pm

Re #12: My take on Oakland was that if you put Oakland in the AFC East and New England in the AFC West this year, Oakland would have a home playoff game and New England would be last in the division. I think they gave up a little bit towards the end of the year, but they struck me as an average team getting blown up by a brutal schedule.

19
by Chris (not verified) :: Tue, 01/03/2006 - 8:30pm

5 -

One of the things about Kansas City is that they're not a young team. Their offense is OLD and Willie Roaf's injury showed how much quality depth they have on their offensive line. They just lost their head coach. I doubt this is a team that will see improvement next year or even stay at their competitive level.

20
by Chris (not verified) :: Tue, 01/03/2006 - 8:30pm

Erm, ignore the link on my name there. Didn't mean to link it again.

21
by Kibbles (not verified) :: Tue, 01/03/2006 - 8:32pm

Re #17: Aaron doesn't rank the teams, DVOA does. Saying that DVOA is biased is the same as saying "F(x) = x^3 + x^2 - x" is biased because as it goes from -1 to 0 it DECREASES, but as it goes from 0 to 1 it INCREASES.

It's a mathematical formula. It can't show bias, it doesn't have a brain.

22
by Paul (not verified) :: Tue, 01/03/2006 - 8:36pm

If we insert chips into helmets, tracking player substitutions could be a piece of cake...real time DVOA could be calculated instantly...but maybe this is overkill...

23
by Bobman (not verified) :: Tue, 01/03/2006 - 8:36pm

I concur with Thad #14. Aaron and crew are quite flexible, willing to listen, and somehow thick-skinned. Great combination. Adam Sandler/Jack Nicholson they are not. I know Tanier is a teacher (math, right?) where those attributes are critical.

Also, Aaron, family first, football second (and Oprah/Montell/Jerry Springer, never!). It's not pleasant, but in 30-40 years, when your daughter (if I recall) is picking your retirement home, you won't end up in the Springfield Retirement Castle.

Now... if you had, say, Ryan Leaf or Ki-Jana Carter or Maurice Clarett picking your retirement digs out for you, you'd be eating cat food in your golden years. That's my logic for making FO my main NFL fix rather than staring at the tube for 16 hours each weekend. When the Colts are on in Seattle I get a 4-hour reprieve... Sad, but true.

24
by Fat Tony (not verified) :: Tue, 01/03/2006 - 8:37pm

Re: 17 Uh, no, the Pats pulled at least 5 defensive starters (McGinest, Vrabel, Wilfork, Seymour, Colvin) in the first half as well. It seemed like Bam Childress played about 90% of ALL snaps (Off, Def, ST) from the beginning of the second quarter on.

25
by Fat Tony (not verified) :: Tue, 01/03/2006 - 8:39pm

24 is in response to Kibbles' post in 16, not 17.

26
by Bobman (not verified) :: Tue, 01/03/2006 - 8:43pm

RE: 17

Larry R, That's because Indy and Sea are both good teams--Indy did not drop whrn they lost to SD either. (And remember, this is not an analysis of score, but on a play by play basis.) When #1 loses to #3, it's not a big deal, but when #3 loses to #20... whoops!

Tanking against Green Bay will make anybody look at least a little bit worse.

No bias.

27
by Harry (not verified) :: Tue, 01/03/2006 - 8:44pm

Re: 17 & 24. I think the announcers may have mistakenly said that the Pats 1st string defense was still playing in the 2nd half, right after one of those sideline shots of Vrabel and Bruschi in warm-up togs. This is the crew that apparently thought Bill Cowher coaches the Pats.

28
by Bobman (not verified) :: Tue, 01/03/2006 - 8:46pm

Kibbles #18, you're a Denver fan, no? Making a statement like that about the Raiders is the kind of thing that gets you nominated for a post at the UN.

Then again, it's an objective site for rational thinkers, so I should not be surprised. My Raiders fan brother would be heartened by that analysis, if he wasn't so depressed, that is.

29
by FO AUTOMATIC PROCESSING AGENT (not verified) :: Tue, 01/03/2006 - 8:46pm

DEAR Larry R.

YOUR SUBMISSION TO FOOTBALL OUTSIDERS COMMENT THREAD "Final 2005 DVOA Ratings" HAS BEEN REJECTED.

REASON

YOU FAILED TO PUT YOUR ANGRY TROLL HATRED INTO PROPER ZLIONSFAN MAD LIBS FORM.

EXPLANATION

You failed to use PROPER ZLIONSFAN MAD LIBS FORM because:

REASON UNRELATED TO DVOA "Green Bay played their hearts out" MUST BE PLACED IN FRONT OF COMMENT

LACK OF POOR SPELLING AND/OR CHAT-ACCEPTABLE SPELLING (ASININE PUNCTUATION DETECTED - NOT SUFFICIENT)

ALTERNATIVE SUBJECTIVE RANKING SYSTEM NOT EXPLICITLY NAMED

USE OF INSULTING OR FOUL LANGUAGE "Brett Favre"

Thank you for using FOOTBALLOUTSIDERS.COM AUTOMATIC PROCESSING AGENT.

**Time 0.13 secs 05:54:13**

30
by Jeff (not verified) :: Tue, 01/03/2006 - 8:52pm

re 4: As one of the game charters, I can assure you that with the footage you get from a network telecast this would be impossible.

re 21: Yes, but Aaron decides whether the formula is x^3 + x^2 - x or 2x^3 + 3x^2 - 5x. Suppose DVOA was based entirely on offensive rushing statistics. DVOA would then be biased towards good offensive rushing teams, even though it's just numbers.

Of course, virtually every change applied to DVOA is designed to increase the predictive value of DVOA, so changing the formula is still generally not subjective.

31
by Falco (not verified) :: Tue, 01/03/2006 - 8:54pm

Re: Week 17 games (or sometimes Week 16), instead of parsing who is starting whom, I think the better solution is to simply run calculations not including those games once a team has clinched its exact playoff seed. Maybe that team plays some of its starters for part of the game and looks really good (Denver) or not as good as it otherwise has (Indy at SEA), or completely lays an egg (Chicago at MIN). In any event, does including those games improve the accuracy of the DVOA number for that team, or potentially blur it?

I would point out that 5 of 7 SB Winners and 4 of 7 SB Losers had lower "weighted DVOA" than "Total DVOA". The weighted DVOA formula works when the next game is played under similar circumstances as the previous, but with so many teams treating end of season games like preseason games, it clouds the accuracy of weighted DVOA.

I also think the league needs to look at the current playoff and division format as a cause of the poor product in week 17. With 4 divisions instead of 3, there is less competition for each division title spot (1 in 4 vs 1 in 5). There is also greater imbalance in scheduling between divisions (formerly, each AFC West team would play 2 from East, 2 from Central for example). Also, the wildcard teams cannot get a home game, even the best one.

I think it would decrease the amount of these exhibition games if the following rules are in place:

--top 2 division winners (applying current tiebreakers) get the #1 and #2 seed
--all division winners qualify for playoffs
--2 wildcard teams are determined same as now
--seeds #3 to #6 are determined using same tiebreaker rules as currently, but for both division winners and wildcards, so wildcards could potentially host playoff games and have something to play for even when they have clinched, and division winners would have something to play for (keep a home game).

This year, it would have prevented the Cincy-NE game of hot potato with the #3 seed. Jacksonville would be #3 seed, Cincy would be #4 seed and host #5 Pittsburgh, and New England would be #6. Of course, NE would not have rested Brady in the 2nd half, and a victory would have meant the #4 seed and hosting #5 Cincy.

I think the rules committee should re-examine whether each division winner in a 4 team division should automatically get a home game, on top of already automatically qualifying for the playoffs by beating out 3 other teams.

32
by David Brude (not verified) :: Tue, 01/03/2006 - 8:56pm

I'm always find it interesting in how a teams true effectiveness gets buried by traditional stats. WAS ranks as the number 2 rushing defense but by looking at their raw rushing numbers in you'd be shaking your head as to how that is the case. I see that the raw rushing numbers for WAS were skewed in large part by a number of long runs which I guess suggests they were very good on most plays but gave up long runs every so often. I remember the SD game seemed to be like that.

33
by putnamp (not verified) :: Tue, 01/03/2006 - 9:00pm

Just wanted to point out in case someone brings up the Seattle/Chicago "easy schedule" thing:

Note that Chicago's only a couple ranks behind Seattle in SOS, and only about 2% points behind in raw ratings.

34
by Vash (not verified) :: Tue, 01/03/2006 - 9:16pm

new england is clearly ranked too high because brady is overrated. expensive psychic hotlines are way better than this. lol, tommy boy was just supported b yhis ohter plyrs and now their gone theirs just no hope for them mister. DOLPHINS ROOL

35
by NF (not verified) :: Tue, 01/03/2006 - 9:48pm

So, can Bam Childress play defensive end?

36
by RCH (not verified) :: Tue, 01/03/2006 - 10:00pm

I think that the exhibition games that Falco refers to (#31) are not only a problem for DVOA but they are also becoming an issue for the NFL. I don't agree that the 4 division set up is to blame, there are now fewer teams chasing each divisional title so they are more likely to be in contention.

I do like the idea of allowing wild cards to jockey for home games, that would definitely create more meaningful games.

I also have a thought regarding draft order...suppose that all teams under a certain threshold (lets say a .500 record) draft in order of success, not futility. So this year a 6 win team would choose first (Philly, StL, Baltimore, Cleveland) and Houston would choose 20th. Granted its far-fetched, but anything that rewards winning and reduces the number of meaningless games will greatly improve the product.

37
by emcee fleshy (not verified) :: Tue, 01/03/2006 - 10:06pm

Atlanta went 6-2 in the first half and 2-6 in the second. Their VAR was 29th.

The strangest thing is - if you watched their games, this isn't surprising.

Poetic Justice - ATL ended up exactly where our DVOA always said they would.

38
by emcee fleshy (not verified) :: Tue, 01/03/2006 - 10:07pm

our? I meant your.

39
by thad (not verified) :: Tue, 01/03/2006 - 10:32pm

re 32
You are correct.
Washington, according to NFL.com, gave up 11 runs of 20+ yards, tied for 21st.
Actually that number is worse than it seems cause the average tean faced about 450 runs and the skins only faced 411.
On the other hand, the Redskins only gave up 74 rushing first downs. As a percentage, their 18% would rank 2nd.
The skins had a stuff % of 14%, good for 12th in the league.
Also I don't know how often you watch them, but williams love to blitz, and since he has been there the defense has really improved. So my own observation would be yes, most of the time they are quite good, every so often they really bring it and get totally smoked.

40
by Jay B. (not verified) :: Tue, 01/03/2006 - 10:35pm

We should institute awards a la Baseball Primer's "Primeys" for great posts. (Call them "Outeys," perhaps? Ew.)

Anyway, I nominate #29, and also zlionsfan's Mad Libs form which it references.

41
by thad (not verified) :: Tue, 01/03/2006 - 10:37pm

re 37
Which makes me wonder how many of the people that came and posted here actually did watch the games.

42
by Loophole (not verified) :: Tue, 01/03/2006 - 10:47pm

Want an easy and objective way to eliminate the "week 17 effect"? Why not just throw out all games that include a playoff team whose playoff position is fixed (with respect to byes and home field). After all, that's what the coaches do.

43
by Chris (not verified) :: Tue, 01/03/2006 - 10:48pm

Atlanta was consistantly average and has been consistantly average.

The funny part also is that a lot of the criticism is going towards Michael Vick and the teams use of him, and their offense has been ranked 10th for most of the year. It's their total swiss-cheese run defense.

44
by B-spectacled (not verified) :: Tue, 01/03/2006 - 10:53pm

Re: Falco 31

You bring up a good idea and point, but I think that system rewards potentially quasi-decent teams in really bad conferences. Admittedly, anything beyond the current format starts getting "complicated" -- at least beyond the scope of your typical foxsports.com visitor (oops -- did I say that out loud?!)

Re 41 Thad

Maybe the graphics on Madden are so good that ATL fans are confusing Madden-enhanced and user-controlled Mike Vick with the real Vick -- thus thinking they are watching a good ATL team?

45
by Chris (not verified) :: Tue, 01/03/2006 - 10:55pm

42

That's really not a bad idea. Though I'd change it a bit

Someone said earlier that if you start doing that, you need to do things like take out Pittsburgh's games with Tommy Maddox at QB - however, games where players rest their starters are VERY obvious. Theres a difference between "Ben Roethlisberger can't play, so we're starting Tommy Maddox next week" and "We're the number 1 seed so half our offense won't be on the field".

When a team is benching their starters and not trying, its very obvious and its an easy line to draw - an easy judgement to make. While Cincy and NE were both playing for the 3 and 4th seed, neither team really cared and it was very obvious as their perfectly healthy Pro Bowl QBs were benched shortly into the game.

I would personally just do what someone else said and discard all stats from the game after the team has pulled their starters.

It's easy to tell the difference between a team that gets drummed though they tried (Carolina vs Atlanta) and benches players when its clearly hopeless, and a team that third series of the game says "Alright, Second team offense, go"

46
by Waverly (not verified) :: Tue, 01/03/2006 - 10:56pm

Re: FOXSports.com and "some financial renumeration".

Is this referring to having to rework the numbers for the Power Rankings?

Or is this referring to Enron-style accounting?

47
by andrew (not verified) :: Tue, 01/03/2006 - 11:20pm

How much difference is there between games in which starters sit at the end, and games in which key injuries take players out earliert in the year, like the Pittsburgh games where Tommy Maddox was the QB?

48
by rick (not verified) :: Tue, 01/03/2006 - 11:28pm

I am wondering if FO has ever done anything about whether a Good QB makes his WRs better or if a Good WR makes his QBs better?

It's a discussion my brother and I have had over the years. His contention is the QB makes the difference. I contend it works both ways. His contention rests on the "fact" that the 3 Amigos did little without Elway. I agreed, but pointed out that several QBs (McNabb/TO, for example) showed vast improvements when they got good WRs, and sharp falloffs without them.

Just curious if anything like this exists?

49
by admin :: Tue, 01/03/2006 - 11:32pm

The individual stats sheets should now all be updated to include the Minnesota-Chicago game. Interesting idea Rick, but no research so far. By the way, just to finish up on Larry R., the issue is not Seattle dropping, it is Kansas City passing them. I don't know if you have heard of this Larry Johnson fellow, but he likes to run around a lot.

50
by doktarr (not verified) :: Tue, 01/03/2006 - 11:35pm

RE: DK #4,

While the idea of using individual players' DVOAs to aggregate toward a team total is interesting, does it really accomplish anything? It may help you come up with numbers like "how Pttsburgh would perform with Duce and Parker healthy, but Roethlisberger and Bettis hurt". But in order to actually use these numbers, you would need to make subjective decisions about which guys to include and which guys to throw away.

You've given yourself more flexibility in addressing the problem, but you haven't created an objective approach. I guess you could throw out the stats of guys who are on IR, but anything else is a value judgement. After all, maybe Peyton breaks his leg on the first snap of the next game, and Sorgi's numbers really do matter.

In past DVOA threads, I've proposed two major refinements. One would be fairly easy to do, but computationally demanding, while the other would take a lot of research. I'll propose a third refinement specific to these week 16 and 17 games.

1) Create weighted ratings for every team for every week of the season, that measure how good they were playing at that point. A week 4 rating would not just consider weeks 1-4, but weeks 5-13 as well. All opponent adjustments would be based on a team's weighted rating from the week you played them. This would be very computationally demanding relative the current system, but it should improve accuracy considerably.

2) Evaluate the probability that a given play will change the outcome of a game using a fairly simple strategic model like the one that the strategic overview guy uses. Using maximum correlation to future performance as your guide, assign weights to plays based on these probabilities. This should slightly reduce the importance of blowouts and meaningless comebacks, but it will take a lot of research to get right.

3) The way I see it, there are four tiers of playoff teams:

a) home field throughout
b) not home field throughout, but bye
c) no bye, but opening home game
d) in playoffs on the road

Any team that is secure in one of these tiers, with no chance to move up OR down, is in "rest for the playoffs" mode. Any game involving one of those teams gets a reduced weight. Again, use maximum correlation with future performance as the guide for weights.

51
by admin :: Tue, 01/03/2006 - 11:57pm

Here you go, a very very very long commentary: http://msn.foxsports.com/nfl/story/5217274

52
by kleph (not verified) :: Wed, 01/04/2006 - 12:23am

with the number of folks popping in from FOX it may be time to add the "message board curse" to the glossary. and "Rusher McFumbles" would probably help as well.

53
by ajn (not verified) :: Wed, 01/04/2006 - 12:52am

"...onterrio smith is coming back, and he's bringing snacks for everybody."

wonderful.

54
by admin :: Wed, 01/04/2006 - 1:13am

An overhaul of the glossary is in the plans for the off-season.

55
by tunesmith (not verified) :: Wed, 01/04/2006 - 1:18am

Woo-hoo! I'm really looking forward to Denver @ Indianapolis. That's the real super bowl this year. Hope it happens. Anyone have DVOA-based thoughts on how the two teams match up?

56
by David (not verified) :: Wed, 01/04/2006 - 2:21am

Hey, I just wanted to take a moment to congratulate the Outsiders on a consistently professional and fun site. Kudos. If you're sincerely concerned about how to fix a problem that's as small as how to distinguish between legitimate information on the depth of a team and times when they are just flat out tanking it, you're doing a heck of a job. There's commentary on pro and college football that's must read because it's funny, must read injury information from the Black and Blue report, and must read stat information throughout the year. I always felt that I'd learn something new here.

Incidentally, I wouldn't let the trolls get you down, Aaron. Seriously, the threads and contact with the writers are one of the things that make this site exciting. You don't just get taught things, you get to hash things out too. I teach intro classes at college and I have to say that I've learned a lot by forcing myself to explain my values to a new and skeptical audience every year. It's got to be rough to go through an in box sprinkled with ignorance, but put me down as someone who really appreciates the open access quality of this site.

Besides, the mispelled troll message of the week feature was hilarious. I referenced it this semester in class as an example of how poor presentation undermines what little authority the writer has.

57
by Bobman (not verified) :: Wed, 01/04/2006 - 3:10am

Aaron,

Now that I have read the full Fox commentary, once again, as a Colts fan I am not too worried about being knocked from the top slot (was it back in week 9 or so...?) I do think you made a mistake with the Indy/Sea game by overweighting the importance of Manning.

(how many times has that thought been uttered, outside of the infamous Brady/Manning thread?)

The defense sat 2.7 pro-bowlers (June, Sanders, and Freeney played 20 snaps out of about 60), their leading sacker (Mathis), and another DL starter (former pro-bowler, leading run-stuffer, however you want to refer to Simon, he's pretty important). Essentially, the D sat 5 of their top players. It was half a D. Plus Harrison was cooling his heels in Indy or Philly, watching on TV and eating Tastykakes.

I know Seattle had iunjuries to their D as well, but those injuries had played out over several games--i.e. they were not one-time voluntary aberrations, but that WAS their D for a few games and they had little choice about it. And after halftime, when it was pretty much the Indy JV against the Seattle semi-V, if you discount one Sorgi fumble that was returned 30+/- yards to inside the 10, it was a 7-7 game. (i.e. Seattle was just as clumsy as Indy offensively.)

I'd throw out the game in its entirety, or to follow your protocol, after Manning left (still, at that point it was 14-3 because the Colts could not punch it in on their first two long-drive series, so it's not like I'm giving the Colts any prize here or anything).

Once again, many thanks for a super, super site and all the work you guys put in. Maybe Bill Ford is looking to hire a smart GM one of these days?

58
by pchase (not verified) :: Wed, 01/04/2006 - 3:27am

Could you post the numbers for the adjusted stats you used for the fox rankings? Or how much the adjustment moved each team up or down? From first glance it looks like the Bengals, Panthers, and Bears were affected most.

59
by Slippery Pete (not verified) :: Wed, 01/04/2006 - 4:08am

Football Outsiders is a great website, and I rarely find things to complain about, but that was some seriously weak commentary on the Rams. FO hasn't written much about the Rams this year (for obvious reasons), but as a Ram fan I was hoping for more than 9 words. Any offensive, defensive, or special team trends? Any thoughts about how the abscence of Martz will effect the team?

60
by VarlosZ (not verified) :: Wed, 01/04/2006 - 4:12am

One relatively easy (I assume) way to ameliorate the problem of teams resting their starters would be to list each team's DVOA performance for each game on the site, as you did in PFP 2005. That way people could look at the numbers and get a clear picture of just how much those games are influencing the rankings.

61
by doktarr (not verified) :: Wed, 01/04/2006 - 4:44am

Aaron, just read the commentary. My favorite two teams are the Redskins and the Colts. Does this mean I hate you double?

62
by Maltodextrin (not verified) :: Wed, 01/04/2006 - 5:40am

re: the Bills comment on Fox- Is this true? Roscoe Parrish is literally AFRAID OF SNOW?

63
by Malene, cph, dk (not verified) :: Wed, 01/04/2006 - 7:08am

re: 62
yup... Florida's Sun-Sentinel reported that Buffalo Bills receiver Roscoe Parrish, who played at the University of Miami, refused to leave his house in Buffalo because he was afraid of snow.

Check out this hilarious quote from the story:

"I'm wondering if Roscoe is coming out, so I go over and see some eyes looking through the blinds. I'm wondering what's wrong and they say he won't come outside ... that he's scared of the snow. Then it dawned on me that he's from Miami and it was the first time he'd ever seen snow."

Ok... so, the fact that you've never seen snow makes it perfectly acceptable to be AFRAID OF GOING OUT?????

Here's Mularkey:
"The weather has been a big subject with him. I think the first time it got down to 50 (degrees) he asked me if this is as cold as it gets.
Unfortunately, I couldn't give him the truth."
Hey coach, I'm from the other side of the planet, and even I know how cold it gets in Buffalo.

64
by Wicked (not verified) :: Wed, 01/04/2006 - 9:27am

About the Packers - they never quit and the defense is not full of holes.

65
by pawnking (not verified) :: Wed, 01/04/2006 - 11:18am

I would not be shocked by a Washington, Pittsburgh SB matchup.

66
by mactbone (not verified) :: Wed, 01/04/2006 - 11:28am

I wonder if GMs have ever thought to question how a player feels about the weather. I heard about the Roscoe Parrish thing but never read the article. After reading the article, I am much more scared for the man. Afraid of snow? When he plays one of the most violent games? I thought it was just one of those "He's never experienced it before so it freaked him out a little the first time" not "He won't leave his house because snow is just that scary."

67
by JMM (not verified) :: Wed, 01/04/2006 - 12:56pm

Why is resting starters considered any different than the impact of injuries? Yes, one is a choice and the other is not. Both measure the importance of depth.

Re: 45 Do you suggest modifications to account for teams changing their late game strategy to run out the clock?Think of the Steelers and their 2nd half vs 1st half run/pass ratios. It skews many of their stats but it is not clear that it changes the usefulness of the stats.

68
by Jake (not verified) :: Wed, 01/04/2006 - 12:57pm

On the other hand, New England played its starting defense for the entire game, if I recall correctly, which means that NE’s defensive DVOA from the entire game should remain valid.
>>>>
Actually with the exception of Ty Warren, the two Safeties and Ellis Hobbs, the Patriots didn't play their defensive starters after the first quarter. Bruschi and Samuel were scratches but the general consensus is they could have gone/will go Saturday.

69
by CrazyBoB (not verified) :: Wed, 01/04/2006 - 1:12pm

Re:67 The problem presented by resting starters is that it puts muultiple backups on the field at the same time, versus an injury that might just put on or two new players on the field. And since its unlikely that the entire Chicago defense will get injured at the same time in the playoffs, the majority of the game against Minnesota has little predictive value going forward.

It seems to me like the biggest problem with resting starters with regards to DVOA is the opponent adjustment. Perhaps the best way to deal with this problem rather than just taking out games with backups, instead the opponent adjustment should be altered. So I would say count the plays from the game but maybe use a replacement level adjustment, since thats essentially what they are going up against with the starters on the bench. This would still be subjective, but at least it would keep the plays in the system.

70
by mawbrew (not verified) :: Wed, 01/04/2006 - 1:18pm

Re: 67

I'm pretty sure the DVOA model already takes into account those late game situations. That is, late game performance with a lead is compared with other performances in a similar situation.

71
by Bill Moore (not verified) :: Wed, 01/04/2006 - 1:19pm

Reasons to be positive - Detroit - Another 1st round WR.

72
by Hank (not verified) :: Wed, 01/04/2006 - 2:07pm

I just wanted to use this space to compliment the staff of Football Outsiders for a great year. A sobering thought: if this site were to suddenly vanish, there just isn't a backup even comparable to a seventh round rookie from Harvard.

73
by B (not verified) :: Wed, 01/04/2006 - 2:25pm

St Louis Reasons to be Optimistic: No Mike Martz

St Louis Reasons to be Pessimistic: No Mike Martz

74
by JMM (not verified) :: Wed, 01/04/2006 - 2:29pm

RE: 69 If you can get Andy Reid to agree, I'll go along. On how it impacts opponent adjustments, I have previously suggested that games against an opponent not count in the team's opponent adjustmetn.

RE:70 Yea, it is set up to do it. I wonder if there enough events to make it meaningful.

75
by Soren N. (not verified) :: Wed, 01/04/2006 - 2:51pm

If the point of DVOA is to assess how good a team really is, then maybe the resting of starters should not be such a concern...a team's total strength should reflect how deep it is. After all, at any moment, Jim Sorgi could have to be the Colts' full-time quarterback. DVOA isn't adjusted to allow a team any grace if it's decimated by injuries in the middle of the season, correct? The important thing is that all NFL teams go out and try to win each and every week with the personnal they have. If they ain't got the bodies on the bench to be competitive....well, that should be reflected because those bodies are sometimes the difference between winning and losing.

76
by Ted Max (not verified) :: Wed, 01/04/2006 - 3:37pm

I also want to praise FO. It is a light in the darkness of NFL discussion. The strain was obvious this year, with testier and testier responses to critics and more and more complaints of not having enough time. You can almost hear the burnout. Hang in there guys, you'll make it.
(Side topic: How do we keep little jewel websites from being killed by the barbarian hordes when they get popular? It's a common problem these days.)
As for the week 17 issue, I see the problem, but I also don't see it. I mean, DVOA is rating the Colts or Patriots, right? Weren't those teams on the field ALSO Colts or Patriots? If we get into a whole thing of whether the DVOA rating is of the "real" team (i.e., its best players), then we have to deal with injuries, team packages on each play, etc., and that's impossible right now.
But yeah, that doesn't fix the whole "predict the playoff winners" problem. How about a simple, empirically determined week 17 (or maybe 16 and 17) deflation factor for EVERYONE. That way you aren't playing around trying to figure out "did the REAL Colts leave when Manning left, or James left, or Freeney left, or what?" Nobody gets a ton of credit for their performance in week 16 or 17 because either they are locked in the playoffs and rolling over, they are locked out of the playoffs and rolling over, or they are fighting to make the final playoff spots and won't get to the SB anyway.

77
by Todd S. (not verified) :: Wed, 01/04/2006 - 4:01pm

#75 said:

"The important thing is that all NFL teams go out and try to win each and every week with the personnal they have."

I disagree. All NFL teams go out and try to win the Super Bowl. Playing healthy starters when your playoff seed is already locked in is counter-productive to that goal. Or, at least, teams seem to think it is counter-productive. This is what Aaron means when he says that sometimes teams aren't even trying to win their game in week 17. I think some type of adjustment is necessary, and I look forward to seeing it in PFP 2006 (or on this site).

78
by Soren N. (not verified) :: Wed, 01/04/2006 - 6:37pm

About the question of whether teams are even trying to win if they do not play their starters: Consider that those bench players come into the game knowing that they only have a limited shot at proving to the coaches that they should be first-stringers or at least survive on the team. Many of them are on the cusp of losing their grip on the NFL entirely, so I seriously doubt if they are giving anything less than 100 percent. Starters likely take a few plays off here and there, but if they choose to take that laissez-faire attitude, then perhaps it's good that the team's DVOA will naturally reflect their overall less-than-savage attitude. Get out there and beat some heads, you lunks!
Look for more meaningless late season games when the NFL, in a blatant cash grab that will diminish the tension of the regular season, adds two more games to the schedule. It will happen, and I'm sure they'll water things down even more by adding more pretender wildcard teams. Ka-ching, ka-ching.

79
by Will (not verified) :: Wed, 01/04/2006 - 8:09pm

What are the odds NE didn't quite want to win that final game? Ohh no, we get the Jags at home instead of Pittsburgh...

Cincinatti is clearly ranked too high because they lack playoff & veteran leadership. Counting on experience is way better than this. Dick Vermeil iz the AntiChrist for spawning Marts.

80
by Bruce (not verified) :: Wed, 01/04/2006 - 8:53pm

Well, why not try a modified 'box and whiskers' approach? It's clearly not so sophisticated as much of the math, but it might yield some insight... simply excluding 'outlier' games (if they're significantly far from the norm and normal deviation patterns) would catch week 17 problems where there are significant deviations in performance, but not otherwise. It would also catch a mid-season game where significant suspensions or injuries are a problem, iff they do represent a true outlier, i.e., the injuries were short-term and bad performance was materially localized to that game.

Just a thought. It's objective, rather than subjective, which is clearly going to be a plus...

81
by Bobman (not verified) :: Wed, 01/04/2006 - 9:33pm

Bruce,

Recalling my last graduate-level statistics class, it seems that tossing out the outlier(s) is often a good move, so maybe you are on to something.

I certainly recall discussions here of the Steelers performance EXCLUDING the Jags game (injured Ben left the doorway to death open for Tommygun). So it was considered as a viable way to view the Steelers back then, maybe it's the way we should evaluate all teams at year end.... (it would certainly clear up Green Bay's outlandish 52-3 win over NO, or the Colts "squeaker" over Jax in Week 2, no?)

Now "box and whiskers"...? Where does that term come from? Oh, that reminds me, gotta go feed the cat. Back soon.

82
by thad (not verified) :: Wed, 01/04/2006 - 10:17pm

I will probably be alone on this one but I vote to count everything.
Seriously if the backups cannot play then they cannot play. This is like the all time cherrypicking example but Tom Brady was a backup! And a football team is so much more than a few starters. The Cardinals scored 13 points, below their season average, against the players the colts put on the field. If you are a Colts fan, that has to be a good sign, right?

83
by Bill Moore (not verified) :: Wed, 01/04/2006 - 10:39pm

Thad, then should we include preseason too?

84
by thad (not verified) :: Wed, 01/04/2006 - 11:09pm

No, the NFL does not include preseason.

85
by Aaron Boden (not verified) :: Thu, 01/05/2006 - 1:00am

I think that including everything is the way to go. Does it screw up the year end stats? You bet, but the teams are clearly making that choice to not play the number 1 players so DVOA should reflect that choice. Everyone knows who the good teams are. Does Indy versus Arizona matter that much to how we veiw Indy? I think not.

86
by Brian (not verified) :: Thu, 01/05/2006 - 3:31am

There was at least one interesting thing in Indy vs. Arizona, Indy's special teams weren't half bad. Polian has been saying for 2 years that the kickoff coverage would be pretty decent if there was just more distance on the kicks and so far Cortez has been holding up his end of that to very improved overall results. We'll see if that holds up, but it looks like so far having a 2nd idiot kicker might be a good thing.

87
by Bruce (not verified) :: Thu, 01/05/2006 - 11:39am

Bobman -

Box and whiskers is a silly high school stat exercise where you plot your data and mark a box around your 'inner' data - say, the quartile of data points nearest the median - then 'whiskers' to represent one or two deviations from the mean. You get an idea for mean, median, and distribution this way. Then, you can exclude anything outside the 'whiskers.' So box & whiskers really is just outlier analysis done graphically to get a point across to kids.

88
by Steve (not verified) :: Thu, 01/05/2006 - 5:57pm

I'm not too concerned with the Colts being number 2 on your chart, but you Shouldn't count the 2nd half of the Colts/Seahawks game in the tabulation...(that's a happy medium)
Colts sat their starting defense in the 2nd half vs Seattles starters. You owe the Colts that if nothing else...

Contrary to Football Outsiders, Freeney played limited plays(about 10), James and Wayne just the 1st half, and Manning just 2 posessions. Not to mention that 4 defensive starters were sitting(who might have played in a meaningful game), that's 2 pro bowlers and the team sack leader. To be fair, while Freeney played only as a situational pass rusher, he did see some action in the second half with the 2nd unit. Shot, Mike Labingo and Darrel Reid, Matt Giordono were playing! I didn't even know Labingo was on the roster.

89
by Rich Conley (not verified) :: Thu, 01/05/2006 - 6:00pm

RE88

Seattle sat a LOT of starters in the second half too, alexander for one.

90
by MaxPower (not verified) :: Thu, 01/05/2006 - 7:30pm

The main problem here is that we are trying to find a perfect statistical representation of the stats. I am talking about determinism. There will never be a perfect way to rank the teams. This is why I hate statistics. Also, statistical interpretation is often biased to prove the writers point. I could easily come up with a ranking system that has Seattle as number one. For all those posts above, where do we draw the line? Alexander had a tummy ache one game and Tiki's athletes foot was flaring up during some runs (have no idea if this is true). Should we factor that in somehow? There is a reason why we can not predict the weather and why data mining our personal information to determine who is a terrorist is so dangerous. The only thing you can do is hope that your system of analyzing data turns out to be correct most of the time. I believe this year the system fell a little short. Colts #1 and Seattle #2 in my book.

91
by B (not verified) :: Thu, 01/05/2006 - 7:44pm

90) Really, how much can we trust a guy who got his name off of a hair-dryer?

92
by MaxPower (not verified) :: Thu, 01/05/2006 - 7:49pm

I love people who watch the Simpsons.

93
by Jake Brake (not verified) :: Thu, 01/05/2006 - 8:04pm

re: 48, I imagine it would be difficult to come up with a universal answer as to whether the QB or the receivers are more important to the success of the QB. To counter your argument, look at Matt Hasselbeck's performance from last year to this year. Last year, when his receivers were leading the NFL in drops, his stats were average; this year, when they started catching the ball he's at the top of the NFC.

Also, taking into account that OL play also has a huge influence on the performance of both QB and receivers - if the QB has no time to throw it doesn't matter how good the receivers are, for starters - you can't really narrow it down to any kind of objective criteria.

94
by Sid (not verified) :: Thu, 01/05/2006 - 10:57pm

Personally, I'd throw out most of the Week 17 nonsense. A lot of the games were meaningless. A number of teams were clearly not playing to win.

95
by Travis (not verified) :: Thu, 01/05/2006 - 11:40pm

Re: 89

Alexander played in every Seahawks drive but one (the last one) against the Colts.

96
by Falco (not verified) :: Fri, 01/06/2006 - 5:47am

First, I want to also congratulate FO on a great year. I discovered the site at the start of pre-season, right before the move to Fox, and it is by far the best out there. You guys are living the dream. Don't let the headaches that come with success bring you down, and sorry if I contributed to any. And as evidenced by the time of this post, it gives me something else to do when I cant sleep, besides watching poker reruns or Blazing Saddles.

Second, a stat column I would like to see next year is median DVOA. I could make an argument that median performance is a better way to rank a team, any given Sunday and all. As I understand it, Total DVOA is the mean of the 16 games. For many teams, the mean and median will be similar, but for some, it will not.

A simplistic example:
Team A has 5 games with DVOA: +30.0, +30.0, +30.0, +30.0, -20.0
Team B has 5 games with DVOA: +60.0, +10.0, +10.0, +10.0, +10.0

Both have a mean of 20.0 and a standard deviation of +/-20.0. However, Team A, the team with the higher median performance, would have had the higher DVOA in 16/25 (64%) of head to head comparisons.

The median number could be a true median number, or it could be something like the mean of the middle 50% of games played, excluding top 25% and bottom 25%. Either way, it would address the outlier discussion in 80 et al.

This year, I suspect the Colts are #1 in median DVOA performance. The Chiefs would not be third (please dont give up draft picks for Herm Edwards, please dont give up draft picks for Herm Edwards . . .), but would drop to somewhere about 6 or 7. San Fransisco's median performance is probably alot closer to the other team's ranked 28-31 than the mean, and better explains why they won 4 games, because if you looked at a mean which is pulled down to shocking levels by some really bad games by Alex Smith in mid-season, you would not expect any victories.

Third, Brad Johnson is right about Chad Johnson being right about Larry Johnson.

97
by Fnor (not verified) :: Fri, 01/06/2006 - 1:37pm

Falco: we sort of have that with variance, which is a nice, simple stat for consistency.

98
by Falco (not verified) :: Fri, 01/06/2006 - 4:56pm

Fnor: yes and no. Variance (standard deviation?) is certainly a useful tool, but I dont think that means median is not. I think the Colts Mean DVOA - variance number is impressive in that this number, representing how well the Colts play in the vast majority of their games, is higher than the mean performance of all but 3 teams (DEN, KC, SEA).

However, variance alone does not tell you whether it is due to uniformly distributed inconsistent performances equally on either side of the mean, one outlier positive performance and one outlier negative performance (See 2004 New England's variance), or due to 2-3 games substantially different from the other 13-14.

Hypothetical: Pittsburgh plays 13 games with QB Roethlisberger as starter. Mean for those 13 games is 32.0% with a low variance. Pittsburgh plays 3 games w/o, has a mean of 0.0% and a low variance. Combine all, the mean is 26.0% with an average variance. But, in this case, the games are not distributed equally on either side of the mean. The median would be higher than the mean, let's say 30.0%. One could look at that and conclude that the median is a better representation of the typical performance by Pittsburgh, when compared with the mean.

99
by Jerry (not verified) :: Fri, 01/06/2006 - 8:06pm

Good thought, Falco. Maybe a median column in the rankings once we get past midseason would be interesting.

100
by Bencoder (not verified) :: Sat, 01/07/2006 - 4:54pm

In regards to median vs. mean application for team performance: It's really pretty elementary. The stat that gives the highest correlation long-term with wins and losses is the most desirable. Unfortunately, sorting that out is not done with the snap of the fingers. Fortunately, FO has the whole off-season to further improve their already juicy product.

101
by Sid (not verified) :: Wed, 01/11/2006 - 12:33pm

Granted its far-fetched, but anything that rewards winning and reduces the number of meaningless games will greatly improve the product.

Totally agree. But the NFL is a professional sports league, so therefore it will not do what makes the most sense.

RE: 29

Nice. :D

RE: 48

It's only logical that it would work both ways, and there is evidence to back this up.

From Power Rankings:
If Washington had won that game, it would be the second seed instead of the sixth seed, emerging from a tangle of four 11-5 teams thanks to an 11-1 conference record.

Nope. They'd be 3rd probably. They'd definitely be behind the Bears, who actually would've tried to beat Minnesota (and succeeded).

102
by Sid (not verified) :: Wed, 01/11/2006 - 12:45pm

RE: 65

I would not be shocked by a Washington, Pittsburgh SB matchup.

I would.

103
by Sid (not verified) :: Wed, 01/11/2006 - 12:54pm

RE: 75

The important thing is that all NFL teams go out and try to win each and every week with the personnal they have.

This is not true of some of the games in Weeks 16 and 17, though. You could argue the Texans were playing to lose for their last 5 or games or so, and the Patriots were not playing to win in Week 17.
Whether a team wants to win is reflected in the play-calling, and in whether the players go all-out or not.
There is a huge difference between injuries and resting the starters. With injuries, the coaches will try to devise a scheme that minimizes the impact of an injury. With the backups playing, they don't really care.
I think you'll notice that in games where a team plays as many backups as it can, the play-calling is moe vanilla than usual (except maybe for the Patriots, because BB sometimes likes to have fun experimenting).

104
by Sid (not verified) :: Wed, 01/11/2006 - 1:04pm

That should say "...more vanilla than usual."

A comment:

San Francisco being easily #1 in Variance makes sense, because they were a totally different team at home than they were on the road. They were close to average at home, and they were historically horrendous on the road.

RE: 83

Thad, then should we include preseason too?

Obviously. After all, were those not the Indianapolis Colts in the preseason games? They were certainly wearing the uniforms. ;)
Like Aaron said in the Power Rankings, many of the Week 17 games were essentially a 5th preseason game (albeit coming postseason instead of pre).

Everyone knows who the good teams are. Does Indy versus Arizona matter that much to how we veiw [sic] Indy? I think not.

That kind of defeats the purpose of DVOA.