Writers of Pro Football Prospectus 2008

Most Recent FO Features

RiceSid07.jpg

» Sidney Rice: What Could Have Been?

Sidney Rice has retired. Is he the most random single-season DYAR leader ever? One-year wonder? Injury prone? We offer a career retrospective for the second-best wide receiver named Rice in NFL history.

28 Dec 2004

Week 16 DVOA Ratings

by Aaron Schatz

Here are the Football Outsiders team efficiency ratings through Week 16 of 2004, measured by our proprietary Value Over Average (VOA) system that breaks down every single play and compares a team's performance to the league averaged based on situation in order to determine value over average.  (Explained further here.)

Moving Up: MIN (19 to 14)

Moving DownJAC (13 to 16), ATL (14 to 19), TEN (23 to 29)

Indianapolis has moved into the top spot in total season DVOA for the first time this season, although the big four teams are basically bunched up in a virtual tie after the Patriots redeemed themselves against the Jets and the Eagles laid down against the Rams.  If you are a fan of any of the top four teams, there's a metric that says your team is the best in the NFL.

  • Indianapolis leads the league in both total DVOA and weighted DVOA, which emphasizes recent performance.
  • Pittsburgh, of course, is now the only 14-1 team, and is a close second to Indianapolis in weighted DVOA.
  • New England leads the league in estimated wins, which is based on DVOA with emphasis on specific situations like the second half of close games and red zone defense.
  • Philadelphia leads the league in pythagorean wins, based on points scored and allowed, and it is easy for Philadelphia fans to write off the drop in DVOA to the fact that the Eagles last night put out a defense consisting partly of radio contest winners.

That last one is a problem, and it is going to be a really big problem when we run ratings next week.  The point of DVOA, after all, is to measure a team's performance on every single play of the season.  But what do you do with the results of a game where the team is basically just mailing it in?

This week, Atlanta and Philadelphia fall in the ratings.  Should the ratings only include Atlanta and Philadelphia through Week 15?  If so, what do we do about New Orleans and St. Louis -- how much of their performance this week was based solely on the fact that their opponents were sitting the starters?  If the Jets have to include games with Quincy Carter at quarterback in their rating, why shouldn't the Eagles have to include games with Koy Detmer at quarterback in their rating?  And yet, if the goal of the DVOA ratings is to determine the true strength of each NFL team, why should I include last night's game when I look at Philadelphia's performance for my playoff previews?

Wait, are you not confused enough yet? OK, the problem gets bigger when you include the issue of opponent adjustments.  A couple of weeks ago, I noted that we needed to take Carolina's defensive rating with a grain of salt because it was partly based on a single game against St. Louis with Chris Chandler at quarterback instead of Marc Bulger:

The opponent adjustments are based on the entire season, even though we know that injuries and trends mean that sometimes a team that was a strong opponent early in the year is no longer a strong opponent. Certainly this is the case with St. Louis. Carolina is getting a bonus for shutting down a top passing offense, but we know that the Rams on the field Sunday were not the Rams that have been on the field most of the year.

Unfortunately, there really is no alternative. Fiddling with the opponent adjustments every week to take into account trends and injuries is not feasible for two reasons. First, it would be massively time-consuming. Second, it would be massively subjective. A one game sample size is not enough to decide that an injury to player X means that team was a much easier opponent for a single week. If I were to change opponent adjustments like this, the numbers would be less about how teams performed on the field and more about what I wanted the ratings to look like.

So, we deal. Numbers are a guide to which team is better, not a rule. We know in the back of our minds that Carolina's defensive rating shouldn't be quite that high. That will have to do.

What happened with Atlanta and Philadelphia this weekend is just another aspect of the same problem.  Every defense that faced Atlanta or Philadelphia this season has slightly dropped in the ratings because the DVOA system now believes that those two teams are not as strong on offense as they were a week ago.  Do I go in and do the opponent adjustments for Atlanta and Philadelphia based solely on the first 14 games?  And since the New Orleans defense now looks stronger because it held down Matt Schaub, every offense that played New Orleans now looks stronger, so do I go in and change the adjustments for the Saints?

I don't want to make it sound like the DVOA ratings are now completely worthless -- two questionable games are small in the scheme of things when there have been 240 NFL games played so far this season.  But after next week, it is going to be a lot worse.  I can't remember a year where anywhere near this many playoff teams had nothing to play for in the final week.  I asked Michael David Smith about this, since he wrote an article for Pro Football Forecast about teams sitting their starters in the last game before the playoffs, and he couldn't remember a year like this either.

Since 2003 was the first season of Football Outsiders, it was the first year that I had to deal with this problem.  When I did my wild card previews, I took Week 17 out of Denver's ratings.  But every other team was still playing for playoff position in the final week of last season.   Looking back, there was one team effected by the "sit down starters" effect in 2002 (49ers), three in 2001 (Bucs, Eagles, Steelers), none in 2000, three in 1999 (Redskins, Dolphins, Rams).  I don't have time this morning to go back to see what removing that game would do to the ratings from those years, but I may try that during the upcoming week.

I better figure out what to do, because there will be more games this season where playoff teams have nothing to play for than in the last five years combined.  Unless I'm reading the tiebreaker rules incorrectly, there are nine: All four AFC division winners and the Packers in Week 17, and both the Eagles and Falcons in Weeks 16 and 17.

For now, the ratings for Atlanta and Philadelphia treat this week's game like any other.  Right now, I'm leaning towards producing two sets of ratings next week.  One will include every single game, just like the ratings always do.  The other set will remove the last one or two games from the playoff teams that aren't playing their top players, and remove the opponent adjustment from the teams that are playing against the backups so that St. Louis' defense doesn't get a bonus for playing well against a Philadelphia offense that isn't really the Philadelphia offense.  Once again, what's important here is to use the numbers as a valuable piece of information, not the only piece of information.

* * * * *

Sitting right below the big four this week is the hottest team in football, the Buffalo Bills.  We make fun of the Bills a lot around here, mostly because a few of us (me, Ian, Al) are definitively anti-Bledsoe.  But with one game remaining on the season, the Bills are currently the top team in both defense and special teams, and Bledsoe can't do anything to ruin that.  They've actually moved up to third place in weighted DVOA, meaning that the numbers believe they are a better team than either Philadelphia or New England.  Subjectively, I don't believe this -- the Bills would fall below the Eagles if you took out last night's game, and their two biggest losses of the season have come against New England.  But if the Bills manage to get into the playoffs, they will be a serious danger to whoever draws them in the first round.  While they probably have a better shot at beating the Chargers, a matchup with the Colts would be the more interesting game, pitting this season's strongest defense against the strongest offense.  The Colts, of course, already played a game like this, since Baltimore was the highest-rated defense when those teams met two weeks ago.  The other thing about a Colts-Bills playoff game is that Dwight Freeney might sack Bledsoe ten times. 

By the way, since the Bills are up to fifth in the league in DVOA, New England's 29-6 win from Week 10 now rates as the best overall single game performance of the season by far (DVOA: 144.6%).  Meanwhile, New England's upset loss to Miami last week is their worst game of the season, also by a big margin (DVOA: -85.3%).  New England is now ranked #4 in VARIANCE almost entirely because of these two games.  No team even comes close to this kind of gap between their best and worst games.  Buffalo is second (their best game being the win over San Francisco this week, their worst being that same loss to New England) and Green Bay is third (their best game was Week 6 over Detroit, their worst game only a week before against Tennessee on Monday Night Football).  Here are the five best and worst performances of the year according to DVOA (notice that the two worst games both came against Buffalo in the last three weeks):


5 Best Games of 2004 by DVOA
(through Week 16)
  5 Worst Games of 2004 by DVOA
(through Week 16)
Team Opp. Week Score DVOA Team Opp. Week Score DVOA
NE BUF 11 29-6 144.6% SF BUF 16 41-7 -147.0%
IND CHI 10 41-10 122.5% CLE BUF 14 37-7 -121.2%
BUF SF 16 41-7 115.7% DET GB 6 38-10 -116.7%
PIT NE 8 34-20 113.3% CHI IND 11 41-10 -108.7%
PHI GB 13 47-17 106.1% ATL KC 7 56-10 -102.6%


* * * * *

Finally, a quick turn to the NFC North.  Yes, despite the fact that Green Bay beat Minnesota, the Vikings move up in the ratings and the Packers move down.  The explanation comes from the number of plays run by each team during the game.  The Packers gained 452 yards to only 416 yards for the Vikings, but they needed 78 plays to do it, while the Vikings had only 49 plays.  Nobody is going to argue against putting six on the board, but the problem with quick strike touchdowns is that the other team gets the ball back.  The Vikings had two one-play drives and a touchdown on an interception return.  It is rare for there to be such a disparity in number of plays that a team loses despite gaining three yards more per play and winning the turnover battle, but it does occasionally happen.  The Vikings had two bad drives at the absolutely wrong time -- they got the ball back in good field position with a tie score and 2:48 remaining, after one first down they stumbled thanks to penalties, and then couldn't stop Green Bay from marching down the field for a field goal.

Some readers will respond that these final plays should be so important to DVOA that they cancel Minnesota's better performance in the rest of the game -- after all, they not only lost, they folded like tissue paper and blew the division.  But usually when a team plays as well as Minnesota did in the first three quarters, the game isn't close enough at the end where two bad drives turn a win into a loss.  That's why we include every play in a team's rating, not just the ones that come in the fourth quarter.  DVOA says that if the Vikings and Packers met again on a neutral field, the Vikings would be favored.  But while stats are nice to help predict the future, wins and losses are what matter in the past, so the Packers are in and the Vikings are still fighting for a spot in the postseason.  To be honest, I'd rather be the Vikings going to Seattle than the Packers having to host (probably) Carolina.

* * * * *

OFFENSE and DEFENSE DVOA are adjusted based on strength of opponent as well as to consider all fumbles, kept or lost, as equal value.  SPECIAL DVOA is adjusted for type of stadium (warm, cold, dome, Denver) and week of season.  NON-ADJ TOTAL VOA does not include these adjustments.  DVOA is a better indicator of team quality.  VOA is a better indicator of wins so far.

You'll notice this week's second table adds "Pythagorean" projected wins based solely on points scored and allowed.

As always, positive numbers represent more points so DEFENSE is better when it is NEGATIVE.


TEAM TOTAL
DVOA
LAST
WEEK
NON-ADJ
TOTAL VOA
W-L OFFENSE
DVOA
OFF.
RANK
DEFENSE
DVOA
DEF.
RANK
SPECIAL
DVOA
S.T.
RANK
1 IND 38.4% 2 43.4% 12-3 40.6% 1 0.0% 16 -2.2% 22
2 NE 38.2% 4 31.1% 13-2 27.3% 3 -10.6% 6 0.3% 16
3 PHI 37.3% 1 33.6% 13-2 24.0% 6 -4.2% 13 9.1% 2
4 PIT 36.8% 3 39.6% 14-1 18.7% 7 -15.8% 4 2.3% 10
5 BUF 26.5% 7 31.7% 9-6 -6.6% 24 -21.9% 1 11.2% 1
6 NYJ 26.3% 5 27.8% 10-5 26.2% 5 1.8% 17 1.9% 12
7 BAL 20.1% 6 10.9% 8-7 -3.3% 18 -17.7% 2 5.6% 4
8 SD 17.5% 9 20.6% 11-4 16.2% 8 -5.2% 11 -3.8% 29
9 KC 14.1% 8 8.0% 7-8 30.7% 2 16.2% 27 -0.4% 18
10 DEN 13.7% 10 14.2% 9-6 5.0% 11 -12.0% 5 -3.2% 25
11 CAR 6.6% 11 12.3% 7-8 1.1% 15 -9.1% 7 -3.7% 27
12 CIN 2.0% 12 -6.8% 7-8 6.7% 10 7.1% 23 2.3% 9
13 HOU 1.7% 15 -4.8% 7-8 -0.3% 16 -4.9% 12 -2.8% 23
14 MIN 1.1% 19 4.5% 8-7 27.2% 4 22.5% 30 -3.6% 26
15 TB -0.1% 16 2.1% 5-10 -3.1% 17 -7.7% 9 -4.7% 31
16 JAC -1.3% 13 -7.5% 8-7 -3.3% 19 -3.7% 14 -1.7% 21
TEAM TOTAL
DVOA
LAST
WEEK
NON-ADJ
TOTAL VOA
W-L OFFENSE
DVOA
OFF.
RANK
DEFENSE
DVOA
DEF.
RANK
SPECIAL
DVOA
S.T.
RANK
17 DET -2.2% 18 0.2% 6-9 -3.7% 20 1.8% 18 3.4% 7
18 WAS -3.6% 17 0.2% 5-10 -16.5% 28 -16.8% 3 -4.0% 30
19 ATL -3.7% 14 2.0% 11-4 -8.3% 25 -0.4% 15 4.3% 6
20 SEA -4.3% 21 6.3% 8-7 2.5% 14 3.6% 20 -3.1% 24
21 GB -7.0% 20 -7.1% 9-6 11.9% 9 20.9% 29 1.9% 11
22 NYG -8.7% 22 -5.2% 5-10 -5.1% 22 5.4% 21 1.8% 13
23 NO -12.3% 25 -4.4% 7-8 -5.7% 23 15.6% 26 8.9% 3
24 MIA -14.9% 26 -20.9% 4-11 -28.5% 31 -8.6% 8 5.1% 5
25 OAK -15.1% 24 -26.1% 5-10 2.9% 13 16.9% 28 -1.1% 20
26 DAL -15.6% 27 -18.3% 6-9 -3.8% 21 12.3% 25 0.4% 15
27 CLE -21.7% 28 -31.7% 3-12 -13.0% 27 9.7% 24 1.0% 14
28 ARI -22.6% 29 -13.7% 5-10 -19.4% 29 2.5% 19 -0.7% 19
29 TEN -22.9% 23 -21.7% 4-11 -12.5% 26 6.6% 22 -3.8% 28
30 CHI -29.3% 30 -29.7% 5-10 -39.0% 32 -6.5% 10 3.2% 8
31 STL -29.6% 31 -25.4% 7-8 3.4% 12 23.9% 31 -9.0% 32
32 SF -48.3% 32 -47.2% 2-13 -20.7% 30 27.9% 32 0.2% 17

  • ESTIMATED WINS uses a statistic known as "Forest Index" that emphasizes consistency as well as DVOA in the most important specific situations: red zone defense, first quarter offense, and performance in the second half when the score is close.  It then projects a number of wins adjusted to a league-average schedule and a league-average rate of recovering fumbles.  It is based on the number of games a team has played so far, so teams which have not had their bye will appear higher.
  • PYTHAGOREAN WINS represent the number of wins projected from the team's points scored and allowed, as described in this article.
  • WEIGHTED DVOA represents an attempt to figure out how a team is playing right now, as opposed to over the season as a whole, by making recent games more important than earlier games.
  • PAST SCHEDULE lists average DVOA of opponents played so far, ranked from hardest schedule (#1, most positive) to easiest schedule (#32, most negative).  You'll notice that FUTURE SCHEDULE has been removed from the table; with only two games left, it is pretty easy to look up the ratings of each opponent.
  • VARIANCE measures the statistical variance of the team's weekly DVOA performance.  Teams are ranked from least consistent (#1, highest variance) to most consistent (#32, smallest variance).


TEAM TOTAL
DVOA
W-L ESTIM.
WINS
RANK PYTH
WINS
RANK WEIGHTED
DVOA
RANK PAST
SCHED
RANK VARIANCE RANK
1 IND 38.4% 12-3 11.9 3 11.3 3 47.8% 1 -0.5% 16 10.9% 28
2 NE 38.2% 13-2 12.7 1 11.5 2 32.1% 5 7.3% 5 29.0% 4
3 PHI 37.3% 13-2 12.5 2 11.7 1 35.9% 4 -5.4% 26 20.3% 13
4 PIT 36.8% 14-1 11.3 4 10.9 4 45.7% 2 2.9% 12 18.9% 16
5 BUF 26.5% 9-6 10.5 6 10.6 5 39.7% 3 -1.5% 18 33.4% 1
6 NYJ 26.3% 10-5 10.7 5 9.9 7 25.2% 6 5.4% 8 16.5% 19
7 BAL 20.1% 8-7 9.5 8 8.9 9 18.1% 8 12.5% 2 24.1% 7
8 SD 17.5% 11-4 9.8 7 10.5 6 20.1% 7 1.5% 15 10.4% 30
9 KC 14.1% 7-8 9.1 9 8.6 10 8.6% 11 5.3% 9 16.3% 21
10 DEN 13.7% 9-6 9.1 10 9.1 8 10.1% 10 -0.8% 17 20.7% 12
11 CAR 6.6% 7-8 8.6 11 8.0 12 14.0% 9 -4.3% 23 15.1% 22
12 CIN 2.0% 7-8 7.7 16 6.8 18 7.1% 13 7.3% 4 13.3% 24
13 HOU 1.7% 7-8 7.8 13 6.9 17 3.7% 15 3.2% 11 13.0% 25
14 MIN 1.1% 8-7 8.5 12 7.8 13 1.5% 16 -4.3% 24 15.0% 23
15 TB -0.1% 5-10 7.8 14 7.6 15 7.8% 12 -6.9% 29 12.9% 26
16 JAC -1.3% 8-7 7.7 15 6.6 19 -6.4% 20 7.0% 6 8.9% 31
TEAM TOTAL
DVOA
W-L ESTIM.
WINS
RANK PYTH
WINS
RANK WEIGHTED
DVOA
RANK PAST
SCHED
RANK VARIANCE RANK
17 DET -2.2% 6-9 7.6 17 6.1 23 -6.0% 19 -3.2% 22 21.1% 11
18 WAS -3.6% 5-10 6.3 22 6.4 20 5.7% 14 -1.6% 19 19.1% 15
19 ATL -3.7% 11-4 7.6 18 7.6 14 -4.7% 17 -6.2% 28 29.6% 3
20 SEA -4.3% 8-7 7.3 19 7.4 16 -13.1% 23 -9.7% 32 10.6% 29
21 GB -7.0% 9-6 7.0 20 8.1 11 -5.3% 18 -1.9% 20 25.8% 5
22 NYG -8.7% 5-10 6.4 21 6.1 22 -19.6% 26 1.7% 14 31.4% 2
23 NO -12.3% 7-8 6.2 23 6.0 24 -9.0% 22 -6.0% 27 7.6% 32
24 MIA -14.9% 4-11 5.9 26 5.3 26 -8.6% 21 5.7% 7 19.2% 14
25 OAK -15.1% 5-10 6.0 25 4.8 28 -17.0% 24 10.8% 3 16.7% 17
26 DAL -15.6% 6-9 6.2 24 4.7 30 -19.4% 25 2.8% 13 11.2% 27
27 CLE -21.7% 3-12 4.9 27 4.2 31 -25.4% 28 14.7% 1 22.4% 8
28 ARI -22.6% 5-10 4.4 30 6.2 21 -26.6% 29 -7.2% 30 21.7% 9
29 TEN -22.9% 4-11 4.8 28 5.2 27 -24.3% 27 4.0% 10 21.6% 10
30 CHI -29.3% 5-10 4.2 31 4.8 29 -38.6% 31 -2.2% 21 16.6% 18
31 STL -29.6% 7-8 4.6 29 5.5 25 -31.6% 30 -5.3% 25 24.5% 6
32 SF -48.3% 2-13 2.1 32 3.3 32 -50.8% 32 -7.8% 31 16.4% 20

Posted by: Aaron Schatz on 28 Dec 2004

comments