Writers of Pro Football Prospectus 2008

05 Aug 2012

Giants: The Postseason Run Matters

Jason Lisk takes a look at our DVOA ratings from the last 21 years and finds that teams that make a playoff run with a low regular-season DVOA, such as the Giants, do tend to be better the next season than similar teams that didn't make a playoff run. It's definitely reasonable research; the question is whether I can figure out a way to incorporate it into the projections before I run the updated numbers on the site right before the regular season begins.

Posted by: Aaron Schatz on 05 Aug 2012

6 comments, Last at 06 Aug 2012, 12:49pm by Paddy Pat


by tuluse :: Sun, 08/05/2012 - 3:15pm

Is there a good reason you can't just consider them having played a 20 game schedule? And the 49ers a 18 games schedule?

It seems a little silly that you're just throwing away extra data you have on teams.

by Jimbo :: Sun, 08/05/2012 - 3:43pm

Totally unrelated note - but where else would one post such a thing - The all 22 Film on NFL Rewind was not retroactive initially, now it seems to be. If anyone has already posted the relative pennies to have access to all of the NFL Rewind games you can see the A22 now. It starts from game 1 of the 2011 season. Of course I may have somehow missed this but still just saying.

At the expensive sounding like an NFL films shill 70 bucks for all this access is nothing. Think of all the frivolous things one blows 70USD on. There is no sound by the way which to me is a good thing though of course you can't hear the line calls (which is probably the point of it being mute).

by Paddy Pat :: Mon, 08/06/2012 - 2:04am

Aaron, I don't see this at all. What's the statistical power of this correlation? I immediately think of the 2003 Carolina Panthers--no significant improvement in 2004; the 2004 Falcons--declined in 2005. I suppose the 1996 Jaguars, the 2008 Cardinals... but this seems like it's probably a wash, similar to the predictive power of a late season winning streak.

by Paddy Pat :: Mon, 08/06/2012 - 3:14am

All right, more thorough reading, I still have many objections!

First, the math seems fuzzy. The author simply averages the difference in DVOA between the big Wild Card Year and the subsequent year, not making any attempt to account for outliers. With his data set of 10 teams, we have 5 that stayed within +/-7 DVOA points, some up, some down, 3 big improvers, and 2 big decliners. That looks an awful lot like a natural distribution--it's just really slanted by the massive improvement of the 2007 Giants, which looks like small sample error. In the writer's second set, the one of 7 teams, he shrinks his sample even further but still includes the Giants. This is shoddy work.

The assessment also overlooks something else--on multiple occasions, DVOA has favored the wild card team (1999 Titans, 2000 Ravens). If these teams are to be compared to the ones that went on "runs" as major underdogs, we might just as well include top billing, first round bye teams in our analysis. I think a more thoughtful analysis would be to look at teams that went on runs of two or more games upsetting DVOA favorites. Those teams are the ones are truly anomalous. That sample would be as follows:

1993 BUF with DVOA 8.7 became 1.4 in 1992
1994 SD 10.6 became 4.9
1995 IND -9.8 became -12.2
1996 JAC -0.2 became 21.4
1998 ATL 18.8 became -19.2
2001 NE 7.9 became 15.7
2003 CAR 0.6 became 1.3
2005 PIT 27.2 became 9.3
2006 IND 16.3 became 28.3
2007 NYG 1.9 became 26.0
2008 ARI -5.0 became 11.2

With this list we have 4 major improvers and 3 major decliners. Again, it really looks like a small sample with a random distribution. I don't think you can make an inference here with any statistical power. The 2012 Giants could easily follow in the footsteps of the 1994 Chargers or 1993 Bills, declining slightly. They could improve like the 2001 Patriots and even miss the postseason. Or they could do something wacky, like what they did in 2008 or what Atlanta did in 1999. Given their recent history (2009-10), a DVOA of 3.0-15.0 seems likely with a 7-10 win season. That's directly in line with FOA.

by ammek :: Mon, 08/06/2012 - 9:41am

Your definition of underdogs is contestable. For instance, the 1998 Falcons were a 7 percentage point DVOA underdog to San Francisco, but you have to give them an extra 17 percentage points for home-field advantage. That makes them a clear favorite. By the same token, the 1994 Chargers become a big favorite over Miami, the 1993 Bills would be favored in both of their playoff games, and the 2001 Patriots would hold a slight edge over Oakland.

by Paddy Pat :: Mon, 08/06/2012 - 12:49pm

I'll grant that my definition omits homefield, but that doesn't change the sample size problem in this data set. If you omit the 2007 NYG from the Lisk's list of 10, you find an average year 2 improvement of .27 DVOA points per team. Maybe that's still better than expected because it should show regression? The list of 7 is even trickier, as Jacksonville from 1995-6 also looks like an outlier at +21.38 improvement. If you run the set on the 5 teams not NYG or JAC, you get a 2nd year improvement of +3.5. It just repeatedly looks like poor sampling. I guess we could try to make the argument that wild card teams that go on postseason runs tend to stay about the same strength the next year as they were in the previous regular season and every once in a long while, they become about as strong as you might have expected them to be to have gone on that run, although they occasionally fall apart and become weak...