A Second Helping of Secret Sauce

by Bill Barnwell

Last season, around this time, I wrote an essay for FO.com analyzing the regular season performance of playoff teams in the DVOA era and whether anything about those regular season numbers correlated strongly with playoff success. You can read the original article here [1].

I found that teams with strong regular season defenses were the best bet for championship winners in the modern NFL, while offensive success bore relatively little relationship with postseason performance.

That, of course, was followed by the Indianapolis Colts, a team with a mediocre-at-best defense in the regular season winning four games on their way to the Super Bowl. Now, granted, they did that based primarily on the strength of their defense while Peyton Manning and co. struggled through two games against Baltimore and Chicago, but the result was the same. I updated my findings in this year's Pro Football Prospectus 2007 [2]. The result was that defenses were still more strongly correlated to playoff success, but that great offenses now had at least a fleeting link to Super Bowls.

Enter 2007, where the undefeated New England Patriots enter the playoffs with the best offense of the DVOA era. If the Patriots win, will we see the research from a year ago totally flipped on its head? It's possible. We'll have to see what happens once we look at two more years of playoffs: this year, and 1995, when another great offense won its third Super Bowl. That year should be ready for analysis in a couple months.

In the meantime, I'm going to encapsulate the research so far into a slick, new little metric: Playoff Index. What does this metric does is take the eight most important splits (which best correlate with our other metric, Playoff Score Points, or PSP) and look at where each team ranked in the league that season. The resulting index is each team's average rank among those eight factors. While I don't ignore the possibility that one or more of these correlations might not be much more than some aberrant data points, these data points are, according to the numbers we have, the likeliest to bear a real correlation to playoff success.

The eight factors that make up the Playoff Index are:

• First Down Total Defensive DVOA
• First Down Defensive Pass DVOA
• Punt Return Value
• Third Quarter Offensive DVOA
• Second Down Total Defensive DVOA
• "Close and Late" Defensive DVOA (score within a touchdown, second half or overtime)
• Defensive DVOA on the Road
• Defensive Red Zone DVOA

The index bears a -.31 correlation to PSP, higher than any single metric or split. The correlation is similar (-.29) if instead of PSP we use a system that simply awards playoff teams one point for winning and none for losing, with a maximum of four points for the Super Bowl winner and none for a team that loses its only game.

If we look at the ten top teams according to this index, we're left with several Super Bowl winners, teams that lost to them, and teams that came a game short.

Can Playoff Index predict playoff results on a game-by-game basis, though? Not to the point where I'd want to place money on it. Over all playoff games from 1996-2006, the team with the lower (better) playoff index won 61 percent of the time. If we focus strictly on games where there was a difference of 10 or more points in the two team's Playoff Indices, the team with the lower Playoff Index won 71 percent of the time. That's a sample of only 28 games, though. One of those 28 games that went against the Playoff Index was a matchup from last year, and one that we could very well see again in the 2007 playoffs: New England and Indianapolis, when the Colts (Playoff Index: 17.9) beat the Patriots (Playoff Index: 6.0).

It will take a shock of equal effect for the upset to be pulled in that matchup again this year, if the AFC Championship Game comes to that. The thing is, though ... the roles will be entirely reversed.

That's right. Last year's team to avoid is this year's team to recommend. Indianapolis is by far the team that has the best batch of "Secret Sauce" according to our metric, and it's not particularly close, as the difference between Indianapolis and second-placed Tennessee is similar to that between Tennessee and tenth-ranked Washington.

The Colts come in way ahead of the Patriots, who show up as the second-least suited team in the postseason according to our Playoff Index. Predictably, the Patriots do well on the third-quarter offensive metric, but our split database is unimpressed with their performance on defense and punt returns, which is decidedly mediocre.

There are a couple of details here which could give Patriots fans hope. First of all, the metric weighs all these ranks equally, but they aren't in real life, and the Patriots are in the top ten in three of the strongest-correlated metrics -- although they are still a ways behind their rivals in Indianapolis. Second, because the standard deviation of defense across the league was so small this year, the difference between being ranked 1 and being ranked 10 doesn't mean as much as it did in other seasons.

It's also worth mentioning what happens if you split "close and late" defense into "close in the third quarter" and "close in the fourth quarter." With the score close in the third quarter, the Patriots had the worst defense in the NFL. With the score close in the fourth quarter, the Patriots had the best defense in the NFL. Put both quarters together, and the DVOA added up to 0.0% exactly. (The Colts had excellent defense in both the third and fourth quarters of close games.)

As we've mentioned more than once in this article, Playoff Index isn't a foolproof methodology by any means. It's the beginning of research into whether there are certain factors that aid teams in winning in the playoffs, and unlike our brethren at Baseball Prospectus, we only have twelve years of data to work with. Using the Bill Simmons "If my life depended on this" betting methodology, I don't know if there's a soul among us who wouldn't choose the Patriots to win in the playoffs. If the Patriots don't win, however, it may have been because they didn't have the mysterious playoff "special sauce."