Writers of Pro Football Prospectus 2008

Most Recent FO Features

KhanSha1.jpg

» Futures: My Expansion Franchise

You've just been awarded an NFL expansion team and must build your personnel department. How would you do it? Matt Waldman takes on the exercise.

03 Dec 2003

Building a Better Mousetrap

by Aaron Schatz

When I started Football Outsiders, the weekly VOA team efficiency ratings were not supposed to be the main part of the site.  My goal was to do more general statistical analysis like this and this and this, and the VOA system was just one of the tools I was developing.  Over the season, it has become the main tool for a number of reasons.  First, I only have time to do a certain amount of work each week, and once I've updated the play-by-play database and the VOA ratings, they are just the easiest thing to work with.  Second, I felt that since almost every football writer and website does power rankings, that needed to be a central part of Football Outsiders, updated weekly.  Third, I started to use these VOA ratings articles as a forum for my own commentary on each week's games instead of writing separate articles.

Unfortunately, it has been difficult to explain how the VOA ratings differ from other power ratings around the Web.  These ratings are not meant to pick next week's games and they've never been meant to pick next week's games.  First of all, they include the entire season broken down play-by-play.  This means that dominant victories have a big effect, because they include so many successful plays, while close victories are often rated not much differently than close losses.  Individual games are not rated separately, and then added together; instead, every single play is added together no matter which game it took place in, although each play is adjusted for the score and situation in that game at that time.

In an email, one reader accused me of ignoring the forest for the trees, but actually I don't consider that to be an obnoxious comment.  The VOA ratings do ignore the forest for the trees, because their real goal is to be able to break down the trees into smaller groups of trees to see where the trees need to be replanted.  OK, that metaphor went a little too far but you get the point.

It's become clear, however, that people would like a rating that better reflects how teams have performed as it relates to past wins and losses, which means clearing up the trees a bit and looking at the forest.  Never let it be said that I don't take constructive criticism.  I've begun work on a formula which will create a "Forest VOA" rating that does more to split performances in individual games apart, and takes into account the fact that more consistent performance leads to more victories.  Hopefully, while current VOA correlates better with points scored and allowed than it does to wins and losses, the new rating will work the other way around.

You won't find that here, though.  Instead, here you will find a response to the other complaint that has been common, one that was somewhat addressed in this week's VOA ratings commentary: the fact that the ratings take the entire season into account rather than concentrating on recent performance.  If you think about it, that's not a shock.  Most stats take an entire season into account -- not only standard statistics, but "sabermetric" ones.  The NFL rushing leaders are determined by yards for the entire year, and how many yards a back had last week matters no more than how many yards he had in Week 1.  Or, take a look at the Baseball Prospectus Adjusted Standings and notice the Florida Marlins listed third in their division.  The Marlins were terrible for the first few games of the year, and then were one of the best teams in baseball over the last four months, until they won the World Series.  In the NFL, the equivalent would be the Eagles, though whether they end up where the Marlins ended up remains to be seen.

Anyway, I started something called WEIGHTED VOA in order to try to take into account recent performance more strongly than early performance, but it didn't seem to work very well, and people have commented that they wanted earlier games given even less weight.  Well, your wish is my command.  Not only that, but I discovered I had a math error in the previous formula that was giving earlier games more weight than I thought I was giving them.  Whoops.

Therefore, I debut here a new formula for WEIGHTED VOA.  The below table lists teams with their new WEIGHTED VOA, their older formula WEIGHTED VOA, and their actual full-season VOA.  The last two numbers are no different from this week's regular ratings listing.  As you can see, the new WEIGHTED VOA corresponds even better with public perception.  Tampa Bay has moved down some more, although they are still in the top ten.  Tennessee is up to #3 despite losing to the Jets.  (Note: This table was edited December 9 to take into account a mathematical error that had Oakland too low.)

Now, what I need is for people to tell me what order they would prefer I list teams from now on.  Total season VOA, weighted VOA, or the new "forest number" once I develop it?  Oh, and I have to give the requisite link to the article that explains my methods in more detail.  Your responses and comments, as always, are appreciated.

 


TEAM
W-L WEIGHTED
DVOA
RANK OLD FORMULA
WEIGHTED
DVOA
OLD
RANK
FULL
SEASON
DVOA
FULL
SEASON
RANK
KAN 11-1 32.8% 1 38.7% 1 33.5% 1
STL 9-3 24.0% 2 27.0% 3 22.6% 4
TEN 9-3 23.1% 3 23.8% 5 18.6% 6
SEA 8-4 22.4% 4 27.7% 2 25.8% 2
IND 9-3 20.9% 5 26.6% 4 23.7% 3
NWE 10-2 18.0% 6 18.5% 8 14.0% 8
DEN 7-5 15.9% 7 19.0% 7 16.4% 7
TAM 5-7 15.4% 8 20.2% 6 18.6% 5
BAL 7-5 9.9% 9 9.3% 9 6.9% 11
MIA 8-4 7.3% 10 7.3% 12 5.1% 12
DAL 8-4 6.3% 11 8.6% 11 7.0% 10
SFO 5-7 6.2% 12 9.0% 10 9.0% 9
CAR 8-4 4.8% 13 1.8% 15 -1.0% 19
GNB 6-6 3.5% 14 3.9% 13 2.9% 15
PHI 9-3 1.8% 15 1.2% 16 0.7% 16
MIN 7-5 0.3% 16 3.7% 14 4.7% 14
TEAM
W-L WEIGHTED
DVOA
RANK OLD FORMULA
WEIGHTED
DVOA
OLD
RANK
FULL
SEASON
DVOA
FULL
SEASON
RANK
PIT 4-8 -0.1% 17 -0.1% 17 -0.1% 17
CIN 7-5 -0.8% 18 -2.4% 19 -2.7% 21
NYJ 5-7 -1.6% 19 -2.6% 20 -2.0% 20
BUF 5-7 -2.0% 20 -1.5% 18 -0.6% 18
JAC 3-9 -3.8% 21 -5.3% 21 -4.7% 22
OAK 3-9 -6.5% 22 -6.0% 22 4.9% 13
NYG 4-8 -11.4% 23 -11.6% 24 -9.2% 24
SDG 2-10 -12.7% 24 -14.8% 26 -12.9% 26
WAS 4-8 -12.7% 25 -13.6% 25 -10.8% 25
CHI 5-7 -13.0% 26 -18.7% 27 -18.5% 29
CLE 4-8 -13.8% 27 -10.8% 23 -7.9% 23
NOR 6-6 -16.3% 28 -19.7% 28 -17.0% 27
HOU 5-7 -20.5% 29 -21.6% 29 -17.6% 28
DET 4-8 -27.2% 30 -31.8% 30 -27.7% 31
ATL 2-10 -28.3% 31 -32.8% 31 -27.5% 30
ARI 3-9 -43.8% 32 -51.5% 32 -44.6% 32

For those geeks who want to know exactly how much weight I am giving each week, so they can suggest changes, here goes.  This is based on weeks, not games, so if your bye week came with, say 79% weight, then you just have no game given that weight.  Starting in Week 13 and working backwards, the weights are: 100%, 100%, 100%, 100%, 99%, 97%, 94%, 90%, 85%, 79%, 72%, 64%, 55%.  These weights move up each week, so by Week 17 the weights for Weeks 1-4 will be, working backwards once again, 45%, 34%, 22%, 9%.

Note: As of 2004, weights are different from listed here.

Posted by: Aaron Schatz on 03 Dec 2003

comments