Writers of Pro Football Prospectus 2008

Most Recent FO Features

JonesJon15.jpg

» SDA: Early Playoff Elimination Round

TCU-West Virginia and Auburn-Ole Miss might as well be early playoff elimination rounds, with the losers likely knocked out of playoff contention.

31 Jan 2009

How Did the Cardinals Get This Far?

by Ned Macey with sidebar by Vince Verhei

The surprising run of the Arizona Cardinals to this year's Super Bowl coming one year after the Giants' surprising championship have raised some concerns that the regular season is becoming meaningless. To me, these concerns are based on two other, slightly different concerns. One is that bad teams are advancing deeper into the playoffs. The other is that major upsets are occurring.

To see how real this "problem" is, I decided to take a quick survey of playoff results since 1995 (i.e., years where we have DVOA). This study is not comprehensive, but it seems informative and worth sharing. The currency is DVOA because, well, that's what I think the best measure of team quality is. Therefore, the Steelers' 2005 run seems much less surprising even though they were a sixth seed.

To answer the first question, I looked at simply the DVOA of teams that have advanced deeply into the playoffs. The Cardinals have the lowest DVOA of any team to make the Super Bowl since 1995. The 2007 Giants are second. So, there is something somewhat unique about the past two seasons.

Still, a closer look makes it seem less unique. The Cardinals and Giants have DVOAs of -0.5% and 0.0% respectively. The 2003 Panthers had a DVOA of 0.7%; the 2001 Patriots had a DVOA of 6.2%. These are hardly radical differences. Also, while one mediocre team has come through, the dominant Patriots eased into the Super Bowl last year, and this year's Steelers, and their 27.0% DVOA, are a worthy team. Here is a list of Super Bowl participants by DVOA since 1995:

Super Bowl teams by regular-season DVOA
(winners in blue)

Year Team DVOA
2007 NE 52.0%
1999 STL 45.8%
1996 GB 40.6%
2001 STL 38.5%
2004 NE 35.6%
2002 TB 34.0%
1995 DAL 30.7%
2002 OAK 30.6%
2000 BAL 30.2%
1998 DEN 29.7%
2005 PIT 28.1%
1997 DEN 27.4%
2008 PIT 27.0%
1998 ATL 26.3%


Year Team DVOA
2005 SEA 26.2%
1997 GB 25.9%
2006 CHI 24.0%
2003 NE 23.7%
2004 PHI 23.1%
2006 IND 19.5%
1995 PIT 18.1%
1999 TEN 15.8%
1996 NE 8.4%
2000 NYG 8.3%
2001 NE 6.2%
2003 CAR 0.7%
2007 NYG 0.0%
2008 ARI -0.5%

There appears to be a trend toward slightly inferior teams making the Super Bowl more often, but it is hardly earth-shattering. The eight worst teams all come from different years. More importantly, most of the weaker teams have one thing in common: fewer road games. Seven of the ten worst teams played one or fewer road games to reach the Super Bowl. They were inferior teams who, through the vagaries of seeding or other upsets, ended up with home field advantage a fair amount.

Furthermore, the 1995 playoffs saw the woeful Colts (-10.9%) make the AFC Championship game, while in 1996 Jacksonville (-0.9%) repeated that feat. In 2000, the Vikings (-2.5%) made the NFC Championship game. In 2004, the Falcons (-2.8%) made the NFC Championship.

If the Packers had won the overtime game against the Giants last year, then their run would have hardly been historic. The Eagles certainly had opportunities to beat the Cardinals this year. Considering the previous runs of 2003 Carolina and 2001 New England, the runs by the Giants and Cardinals just do not seem too unprecedented.

The second question, of course, is whether or not we have seen a rise in large upsets. Since 1995, 40 games have matched teams who were separated in DVOA by 20% or more. The better team is 28-12 in those games. Five of those 12 losses have taken place in the past two seasons. Of course, these upset-filled years are not entirely unique. Four of these games took place in 1995-1996, including three in 1995 alone.

What is notable about the five upsets in the last two seasons is that the Cardinals and Giants account for all of them. They are the two anomalies. Over the past two seasons, those two teams are 7-0, with the closest DVOA split being 8.9%. The other seven games with splits that big or larger have all been won by the "better" team.

Another factor not sufficiently discussed with regards to the Cardinals is home field advantage. Home field advantage is worth about 17% in DVOA. If we adjust for home field, then only the Giants really stand out. The Giants suddenly have four of the ten biggest upsets in this time period. The Cardinals have just one of the top 34, their road win at Carolina. Accounting for home field advantage, the Cardinals' victory over the Eagles was only slightly more surprising than the Eagles' victory over the Giants. Their win over the Falcons was actually expected.

For perspective on how the Cardinals' march has not been too surprising, consider the 2000 Ravens. Few people dispute that they were a "worthy" champion. They ranked second in overall DVOA. Of course, they went on the road to win three games, including wins at Tennessee (the best team in DVOA) and Oakland (fourth best). Like the Cardinals, the Ravens won two games that, once you adjusted for home field advantage, they should not have won.

Two upsets appears to be the magic number. Only two teams have had more than two home-field adjusted upsets in the playoffs: the 2005 Steelers and the 2007 Giants. (The Steelers' first one came after they knocked out the Bengals' best player.) Two upsets happens all the time. The 1995 Colts, 1996 Jaguars, 1997 Broncos, 1999 Titans, 2000 Ravens, 2001 Patriots, 2003 Panthers, 2005 Panthers, and 2006 Colts all had two upsets. It seems what would have been more historically anomalous is if the Eagles had beaten the Cardinals.

Of course, if the Cardinals win Sunday, then we suddenly have three years in the past four where three upsets have occurred. Given how close some of the above teams were to winning a third game -- 1995 Colts, 1999 Titans, and 2003 Panthers –- a close Cardinals win would hardly be unfathomable. If Kordell Stewart had properly been called out of bounds, Mike Jones had missed a tackle, or John Kasay had not kicked out of bounds, we may have seen three upsets more often. Plus, all three of those teams were at least 22% worse in DVOA than the team they almost beat.

The Giants obviously provide a unique story, hard to write off as a minor fluke. They faced each opponent at close to full strength. They had no Music City Miracle, Al Del Greco game, or Tuck Rule. Still, had the Packers beaten them, and they lost in overtime, then we would not be having this conversation. If Asante Samuel had caught the interception, we still might not be having this conversation.

If Arizona loses, it seems the Giants anomaly is merely skewing our perception. Upsets happen all the time in the playoffs (as they do in the regular season). Some blame could go to the new playoff format that let the Cardinals get into the playoffs, but A) the Cardinals probably could have won 10 games if they were trying, and B) the Giants last year made the playoffs with 10 wins as a Wild Card; the 2001 Patriots had a bye; the 2003 Panthers won 11 games.

Furthermore, the league had a similar run of wildness in 1995 and 1996. The Colts and Jaguars in those years respectively made surprising runs to the AFC Championship with bad teams. In 1995, Green Bay (10.5% DVOA) won at San Francisco (41.0% DVOA). In fact, adjusting for home field, three of the five biggest upsets occurred in 1995 and 1996: 1995 Colts over Chiefs, 1995 Packers over 49ers, and 1996 Jaguars over Broncos.

I feel pretty comfortable arguing that nothing radical has changed in how to value the playoffs. The Giants achieved a new level of performance in the playoffs, but one incredible run does not mean the system is "broken." Some odd seeding led to a pretty mediocre Cardinals team getting two home games, which undoubtedly helped it reach the Super Bowl. If we take the 2007 Giants and say they are an anomaly, then we can relax and enjoy playoff football, appreciating the occasional upset but knowing that it takes a very good team to win the Super Bowl.

Now, if Arizona wins by 20, pretend you never read this and wait for my article on how everything you knew about the playoffs has changed.

(Ed. Note: I liked Ned's argument against my recent complaints but wanted to point out two things. First, it wasn't really an upset for the 2004 Falcons to make it to the NFC Championship; 10 of the top 11 teams that year were AFC teams and Philadelphia was the only NFC team with a DVOA rating above 2.3%. Second, we make upsets look a little more common by only going back to 1995; we don't have DVOA yet for 1991-1994, but based on Pythagorean projection, we can say those were four fairly upset-free postseasons. -- Aaron Schatz)

VINCE VERHEI on UNLIKELY SUPER BOWL CHAMPS

About a year ago, I broke down the strengths and schedules of all 42 Super Bowl winners and determined that the 2007 Giants had completed the least likely Super Bowl run of all time. How do this year's Cardinals compare?

At the start of the playoffs, last year's Giants had a 6.9-out-of-1,000 chance of winning the Super Bowl. This year's Cardinals had an 11.3-out-of-1,000 chance. If they beat the Steelers today, they would be the second most unlikely champ of all time. Up next: The 2006 Colts (12.7), 1980 Raiders (35.1), 1976 Raiders (42.1), and 2005 Steelers (55.1).

The difference between the Giants and Cards all comes down to the Super Bowl opponent.


NFC Champs expected win percentage
Year Team Wild Card Div. Round Conf. Champ Super Bowl
2007 NYG 0.411 0.342 0.308 0.158
2008 ARI 0.394 0.368 0.298 0.261

The first three games for the 2007 Giants and 2008 Cards virtually wash out. Arizona had a 4.317 percent chance of REACHING the Super Bowl, compared to the 4.334 percent chance for the Giants. Neither of these is the lowest of all time; that honor goes to the 06 Colts and their 4.137 percent chance.

That being said, judging each Super Bowl by itself, an Arizona victory would be the third biggest upset ever, behind only Giants-Pats and Jets-Colts in Super Bowl III.

Incidentally, at the start of the playoffs Pittsburgh's odds of winning the Super Bowl were 190.5 out of 1,000 (or 19.1 percent), which would rank 22nd out of 43 Super Bowl champs -- almost exactly the median. They would be the 12th least likely Super Bowl Champ to REACH the Super Bowl; they were only slight favorites over San Diego, and were actually underdogs to Baltimore.

Why have the last three years produced such unlikely champions? One reason is the four-team division format, which makes it more likely that a mediocre team will be paired with three subpar opponents and have a free playoff spot handed to them. That's clearly what happened to this year's Cards. In most other divisions, they're not a champion and probably miss the playoffs.

It's less obvious with the 2006 Colts; that team went 12-4 and won the division by four games, but that's somewhat deceiving. The Colts actually finished with a worse point differential than the 8-8 Jaguars (+67 to +97) and were a ridiculous 8-3 in one-score games. If we added a good team to the AFC South, two of those close wins may have turned into close losses, and the Colts would have been 10-6 and fighting for a Wild Card spot.

Speaking of, there are also more Wild Card teams than in prior years. In the 1970s and 1980s, with three divisions and four or five playoff teams per conference, the Cardinals and Giants don't even get in the postseason. In 1988, the sixth seed in the NFC would have been the 10-6 Giants, who had won the Super Bowl two years prior and would win it again two years later. If they get in, maybe they beat a team like San Francisco, which also went 10-6, and went on to win the Super Bowl against Cincinnati.

As I said, that's only part of it -- the six seeds have been around now for almost 20 years, and this trend only goes back three or four seasons -- but I think it's a relevant part.

For more of Vince's writing, be sure to check out his blog at vinnyv.com.

Posted by: Ned Macey on 31 Jan 2009

18 comments, Last at 06 Feb 2009, 5:51pm by Bjorn Nittmo

Comments

1
by Sophandros (not verified) :: Sun, 02/01/2009 - 3:10pm

Excellent work, Ned. Stuff like this is why I love this site. And thank you for not jumping on the most recent media bandwagon.

2
by Key19 :: Sun, 02/01/2009 - 3:15pm

Good stuff, guys. I was actually thinking about the entire "Playoff system is broken" argument last night and shuddering at the idea of converting to a BCS-type system (as some people really do think should happen). I think people are just jumping to a lot of conclusions prematurely. I'd like to see the Cardinals win and have a couple more teams have runs like theirs and the Giants before I go changing the system. Although, I wouldn't mind some experimentation with the Bye Week system. Unless teams who have great records figure out a way to carry over their regular season performance better, I think eliminating the bye week is a decent idea. Overall though, we just need more information before we can start declaring trends.

5
by Eddo :: Sun, 02/01/2009 - 3:50pm

You may very well be onto something with the removing of the bye weeks, but what would your proposal be? Add two more playoff teams to get to eight total in each conference? Stay with six and give one first-round winner a bye week in the second playoff round?

6
by Lord K :: Sun, 02/01/2009 - 4:55pm

Stay with 6, the best loser from the first round goes into the second round.

With 6 teams playing each other there are three winners in wild card round. If you let the best loser advance, then four teams advance to the divisional round. The playoffs then proceed as before.

7
by Eddo :: Sun, 02/01/2009 - 5:14pm

Hmm, not sure I like that idea at all.

To begin, you're rewarding a team even though it lost. That's opposite of the general idea of having playoffs, which is that you need to win or you're eliminated from contention. If you let a loser advance, you'd also be saying that one team's playoff loss is not as important as other playoff teams' losses.

Second, how would you determine the "best loser"? Is it the highest seed? Smallest margin of the first round game? Best DVOA? Again, you're opening a whole can of worms.

9
by Lord K :: Sun, 02/01/2009 - 6:09pm

To begin, you're rewarding a team even though it lost. That's opposite of the general idea of having playoffs, which is that you need to win or you're eliminated from contention.

Who says? Losing teams get to stay in the playoffs in various Australian leagues, has been done so for decades, and it doesn't bother anyone.

If you let a loser advance, you'd also be saying that one team's playoff loss is not as important as other playoff teams' losses.

This is a reward for superior regular season performance. The best team during the regular season is guaranteed another bite at the cherry. The second best time might get another bite if the best team wins. The third best team is a bit more dicey, fourth and below have to win.

Second, how would you determine the "best loser"? Is it the highest seed? Smallest margin of the first round game? Best DVOA? Again, you're opening a whole can of worms.

I don't see any issue. The NFL already makes a judgement on how good the teams are with the seeding system. It gives the best two a first round bye. No one calls that a can of worms.

The bye is no longer working as a reward for superior regular season performance. Extending the playoffs to 8 per conference almost completely eliminates any reward for better regular season performance. I don't see any other way to keep the playoffs at 6 teams and keep an advantage for good regular season teams.

10
by J. Morse (not verified) :: Mon, 02/02/2009 - 12:51am

If the top seed is guaranteed to advance to the second round, they would sit their starters for the first round game. It would be a joke. Better not to have them play then to stage a sham of a playoff game.

11
by Lord K :: Mon, 02/02/2009 - 1:00am

Not really, if the first seed loses in the first round they'll end up being a fourth seed and lose the home field advantage that they won themselves in the regular season.

3
by Kiril (not verified) :: Sun, 02/01/2009 - 3:40pm

I can't help but think some of this comes down the way teams are built. I wonder if there's a metric that will determine "streakiness"... because I feel like there are certain kinds of teams- like Warner-run timing offenses, that are very "min-max"- when they're good, they're nearly unstoppable, but if they're off even a little, they're horrific. Only takes a couple of good weeks where their system clicks, and they run over better teams.

You could say it's a side-effect of Parity, that you only need a little advantage to take down a much better team.

4
by Eddo :: Sun, 02/01/2009 - 3:48pm

Wow, great article(s), especially by Ned. I do have one question, regarding Ned's final line:

"Now, if Arizona wins by 20, pretend you never read this and wait for my article on how everything you knew about the playoffs has changed."

He's basically argued above that there's nothing too crazy about the Cardinals being in the Super Bowl. Additionally, while the Steelers are a very good team (and typical Super Bowl team), they're not a historically great team overall by any means. Therefore, would a 20-point win by an inferior team really be a huge shocker? After all, we saw that a few times this year, when the Browns blowing out the Giants, the Rams blowing out the Cowboys, and the Jets blowing out the Titans. We've even seen it in at least one Super Bowl, when the Raiders beat the Redskins 38-9 in Super Bowl XVIII (I also think the Redskins were considered to be the inferior team when they blew out the Broncos in Super Bowl XXII).

So while a Cardinals win would be a big upset, it wouldn't necessarily change what we think we know about the playoffs. It would just be another upset.

8
by vherub :: Sun, 02/01/2009 - 5:40pm

The Steelers, arguably, have one of those great defenses that are historically in the conversation. And while the offense has struggled, the defense has been consistent. Upsets are caused, in part, by inconsistent performances- one team plays better than they normally do/the other has an off day. I wonder if above average/great offenses are less consistent than above average/great defenses. And if so, that may partly explain upsets where dominant offenses lose to good defenses.
Which would mean a 20 point win by the cards would be a pretty big upset, because it would be an unexpected performance by the steeler d.

12
by t.d. :: Mon, 02/02/2009 - 2:18am

I don't think they're in the conversation anymore. I also think the Steeelers offense was vastly underrated this year, because their struggles were more a brutal schedule than their own shortcoming

13
by starzero :: Mon, 02/02/2009 - 11:19am

in the colts' 06 run, the only team i worried about was the patriots. maybe the stats said the ravens and chiefs were better, but many colts fans believed indy was the team to beat. against the bears, it was obvious rex couldn't lead that team. if peyton got protection and avoided mistakes, basically if he could beat the bears defense, the colts would win. and so it was.

14
by Bjorn Nittmo (not verified) :: Mon, 02/02/2009 - 5:52pm

Assuming the reach-the-SB odds are based on actual opponents (as opposed to ex ante probabilities before the first playoff game), I'm really skeptical that the '06 Colts odds were lower than the '07 Giants. Colts start out with a 19.5% DVOA advantage. They played two of their 3 playoff games at home as favorites (I believe they were favored v. NE, and if not they had beaten them in the regular season); their one upset v. Baltimore, without checking the numbers, seems about on par with the Giants upset over Tampa in the first round, and not nearly as shocking as Giants' subsequent wins v. Dallas and Green Bay, teams against which Giants had gone 0-3 in regular season.

Just looked up DVOAs: Giants (0.0%) beat TB (18.1%), Dallas (24.3%), and GB (21.2%), all on road. Indy (19.5%) beat KC (3.4%) at home, Baltimore (31.8%) on road, NE (23.2%) at home. I.e., DVOA spread in each Giants upset was much greater than any Indy win, even before considering home field.

Either I'm not understanding how the playoff odds are calculated or something doesn't make sense here.

16
by Vincent Verhei :: Mon, 02/02/2009 - 10:11pm

Playoff odds for the sidebar (and the original story, linked in the sidebar) were not calculated using DVOA, because DVOA only goes back to 1995. They were calculated using Pythagorean Theorem and the log5 method, and home-field advantage was not accounted for.

The Giants' odds of winning each game are listed in the sidebar. The Colts' odds were 0.571 (against KC), 0.283 (against Baltimore), and 0.256 (against New England). If homefield advantage was accounted for, I'm sure the Giants' slate would be more impressive than the Colts.

But it's important to note: The playoff odds for that article are NOT the playoff odds listed on the site. They were calculated differently because we do not have DVOA for, for example, the 1972 Dolphins.

17
by Bjorn Nittmo (not verified) :: Tue, 02/03/2009 - 2:09pm

Yes, I re-noticed after I made my post that the playoff odds in this analysis were based on Pythagorean pct, not DVOA. Nevertheless, I repeat my skepticism: concluding that the '06 Colts and '07 Giants playoff runs represent comparable longshot odds just doesn't pass the smell test. Colts had a 25.6% chance of beating New England in the AFCCG? Without forgetting the Colts' weaknesses that year and how poorly they played down the stretch, come on -- there's just no way I can believe that the Colts had a 1-in-4 chance of beating a team they had already beaten a couple of months earlier, never mind home field advantage. Similar skepticism for their odds of beating Baltimore. And throw in the Super Bowls, and concluding that the '06 Colts (1.3% to win SB) represent a comparable longshot to the '07 Giants (0.7%) really doesn't fly. Colts were, I believe, favorites in 3 of those 4 games. I guess what I'm saying is I don't think Pythagorean pct. over 16 games can accurately measure team quality, at this in this example.

18
by Bjorn Nittmo (not verified) :: Fri, 02/06/2009 - 5:51pm

Sorry, I realize the rest of the world has moved on to more important matters, but I meant to post this when this discussion was a little more current, but I'd express the same sort of skepticism about the conclusion that the '76 Raiders had truly a 4.2% chance of winning their 3 post-season games. This translates into an average 35% chance of their winning each game. I know they faced 3 teams worthy of winning the SB that year, but really? A 13-1 team, that incidentally was favored in each game, was in fact almost 2-to-1 underdogs in each game? Admittedly, those are decidedly untechnical measures of team quality, and point spread would consider home field and injuries, significant in the Oak-Pit AFCCG that year. Still, this example and the '06 Colts seem so much at odds with more conventional perception (and in the Colts case, also highly at odds with DVOA) that I question the validity of the whole exercise using Pythagorean measures. Not that I'm saying I'm proving anyone wrong, but just that the conclusions are probably questionable.

15
by scottyb (not verified) :: Mon, 02/02/2009 - 9:45pm

Almost all of our statistics measure means, regressions or, more broadly, Central Tendencies. I'd like to see more about looking at the Variance of teams' performance (FO is the first site I've ever seen to track this in any way, but I'd like to see it taken further). For example, maybe there are teams, like this year's Ravens (just a guess) who have a high DVOA with little variance, and a team like the Cards with a lower mean DVOA but with high variance (variance can also account for changes due to injuries, etc.). Perhaps we should be less suprised by the streakiness of high-variance teams?