NFL's Best Playoff Quarterbacks: DVOA and DYAR

by Scott Kacsmar

With the blood of the Denver Broncos, the Seattle Seahawks wrote the final chapter of the 2013 season. Football Outsiders now has 25 complete years of data for DVOA and DYAR. Some readers have asked about seeing the postseason numbers for these metrics, which is what we're going to do in the final part of our study of quarterbacks in the postseason. Part I focused on the frivolities of postseason win-loss records and defense-adjusted passer ratings. Part II was all about drive stats for the quarterback's offense.

To start the 2014 season there should be eight active Super Bowl-winning quarterbacks, the most ever. Does that make them the eight best quarterbacks in the league? Certainly not, but it's close. The free-agency era has helped the best quarterbacks win a ring, which was definitely not the case in the old days for Dan Marino, Dan Fouts and Fran Tarkenton.

The dig for more DVOA will continue, but going back to 1989 allows us to have full career postseason data for quarterbacks like Brett Favre, Troy Aikman, Kurt Warner and any active great. Oh, and also Mark Sanchez. Do keep in mind this means only partial career data will be presented for the likes of Joe Montana, Dan Marino and John Elway, but we hope to complete their careers at some point.

It will take some time before we know if Dieter Brock had the worst postseason ever in the 1985 season, but a quarter century of data is a good place to start digging.

DVOA: Strength of Schedule

Not to pile on Denver too much -- we have a right to be pissed when the dream Super Bowl became a turd -- but obviously playing San Diego and New England was not the same as playing Seattle, the No. 1 team in DVOA. The opponent matters and that's why there's always a ranking of the hardest and easiest schedules in DVOA each season. I've applied the same concept for opponents of all 42 quarterbacks with at least five playoff starts since 1989.

In the following table the quarterbacks are ranked (Rk) by the average DVOA of their opponents in all playoff starts. The smaller the number, the harder the opponent. The "AVG RK" is the average season rank in DVOA for the opponents. The averages and rankings are also presented for just the opposing defense (AVG DEF DVOA) and then just the pass defenses (AVG PASS DEF). Active players are in bold.

Not surprisingly, Eli Manning and the Giants have played the toughest playoff opponents, including three teams (2007 Patriots, 2008 Eagles and 2011 Packers) that ranked No. 1 in DVOA. The Giants played some offensive-skewed teams, including a few of the all-time scoring juggernauts, which is part of the reason why Manning only played average defenses (ranked 21st out of 42) compared to his peers. We know from part I about how well the Giants stepped up on defense in the playoffs.

Troy Aikman (35th) is the most successful playoff quarterback who appears near the bottom for strength of schedule. Factor in how dominant those Dallas teams were, and the results really come as no surprise.

Warren Moon going 1-5 against the "weakest" opponents (3.7%) does not sound good for him, but no quarterback outside of Peyton Manning may have suffered more heart-breaking playoff losses due to factors out of the quarterback's control. The lone win here for Moon came against the 1991 Jets, who went 8-8 and ranked 20th in DVOA (-4.4%).

Philip Rivers is not known for much playoff success either, but he can point to playing the toughest schedule of playoff defenses (-12.5%), including the toughest pass defenses. Rivers played the 2007 Titans (No. 1 defense by DVOA) and two defenses that were the consensus best in football those seasons (2008 Steelers and 2009 Jets). He even played the 2013 Bengals, who were arguably the AFC's best defense.

Rivers played the No. 1 pass defense three times, but one quarterback since 1989 has had more such games. Peyton Manning just played his fourth top pass defense.

I might as well get the first fawning mention of Joe Montana out of the way. Though this only goes back to 1989, in that time Montana was on the average stretch of postseason play ever by a quarterback and he played the third-toughest set of defenses (-10.9%).

We'll talk more about Mark Sanchez later, but his average opponents having the second-worst defense starts to explain his postseason success. Jake Plummer played the worst defenses, but that's not much consolation when running into buzzsaws like the 1998 Vikings and 2003-04 Colts on the road.

This should help in answering who they played, now what about how the quarterbacks played?

Passing DYAR

Our first table looks at the leaders in playoff DYAR, which includes passing and rushing. There's no minimum play or game restriction here as it's just cumulative value. Here are the top and bottom 30 quarterbacks:

The top two names come as no surprise given how many starts and production those two have had, but on a per-game basis Tom Brady slips a bit to his elite peers. The aforementioned strength of opposing pass defenses plays a factor here, as players such as Manning and Warner played tougher opponents than Brady. Montana leads the way with 155.3 DYAR per game. While he was not consistently dominant in the postseason until the 1988 season, should we eventually complete stats for his first 14 playoff games, there's a very good chance Montana will have the most playoff DYAR to match his reputation as the best playoff performer.

The bottom quarterbacks offer few surprises and many mediocre (at best) names. The best quarterback there might be Phil Simms, but this only represents his final three playoff games. Drew Bledsoe being the worst is no surprise to those who have studied any of his playoff history. Andy Dalton has been as bad as you've imagined and he's been consistent at doing so with games of -54, -64 and -64 DYAR.

Speaking of games, here are the 25 best by DYAR:

Kurt Warner and Peyton Manning lead the way with four games apiece. Out of the 35 playoff games with at least 200 DYAR, Manning has the most with six. Warner and Montana each had four. Amazingly, all three of Montana's games from 1989 rank in the top 13. He topped it off with the best Super Bowl, producing 283 DYAR against Denver's No. 1 scoring defense.

The Denver defense makes a whopping seven appearances, including four of the top seven games. If there's a bright side for the Broncos, it might be that Russell Wilson's Super Bowl performance (151 DYAR) only ranks 91st. I still believe Wilson's best playoff game was the one he lost and now I have some statistical support with his 265 DYAR performance in Atlanta. That game makes up 58.8 percent of his total playoff DYAR and is one of the best games in defeat.

Defeat is almost certain with a pathetic playoff performance from your quarterback. Here are the 25 worst:

Note: Backup Mark Vlasic only got in a pickle after starter Steve DeBerg left with an injury.

Bookended by Super Bowl losses, there are at least five games here from Hall of Fame quarterbacks. Sometimes things just snowball quickly. You knew Jake Delhomme would show up near the top, but even after opponent adjustments, Kerry Collins' inability to move the football against the 2000 Ravens reigns supreme. In addition to the putrid throwing stats, Collins was also sacked four times and fumbled once. The opponent adjustments aren't as strong as you might expect, as the Ravens had the average run defense DVOA in history but ranked only seventh that year in pass defense DVOA.

The worst DYAR in a playoff win belongs to Drew Bledsoe. He had -116 DYAR in the 1996 AFC Championship Game against Jacksonville (20-6 win). On New England's three scoring drives that day, Bledsoe contributed 68 passing yards -- all on one field-goal drive before halftime.

Passing DVOA

Next I looked at the top quarterbacks by playoff DVOA. This is only for passing plays, so all rushing is excluded. Players needed a minimum of 150 pass plays to qualify, which gave us a round number of 40 quarterbacks. The DVOA listed here is a weighted average taken over multiple seasons.

Once again that active trio of Brees, Peyton and Rodgers rises near the top. The guys with multiple rings or a Super Bowl MVP like Brady, Eli, Joe Flacco and Ben Roethlisberger don't stack up as well to their team's success and record. Roethlisberger in particular looks bad here, but he does have a couple of the best passing games by DVOA (minimum 15 attempts):

Nineteen games surpass 100% DVOA and once again Montana's entire 1989 appears. Warner's game against the 2009 Packers wears the DYAR crown, but he's also third in DVOA. Rivers missed the fourth quarter of that Indianapolis game with an ACL injury. Hostetler only played five playoff games so he missed the 150-play cut in the previous table, but his passing DVOA is 58.1%, second only to Montana if we moved the requirement down to 125 plays.

Here are the 22 worst games, all registering a DVOA worse than -75.0%.

Vlasic returns along with two other backup performances: Damon Huard's feeble attempt to follow Dan Marino in the 62-7 Jacksonville debacle and Shane Matthews coming in for Jim Miller against the 2001 Eagles.

It's not a good table for Raiders fans with three appearances in the bottom 10. Those horrific games from Jay Schroeder and Todd Marinovich, which were actually back-to-back playoff failures in 1990-91, both show up as does a Rich Gannon performance against the 2000 Ravens before he left injured.

Mark Brunell picked up the "worst win" for the 2005 Redskins in a game discussed last week. David Garrard has the only other win here, but that one never happens without his 32-yard scramble on fourth-and-2 to set up the game-winning field goal. Of course there was a holding penalty the league later admitted to missing on the play, but so goes the postseason and luck. This is only for passing so such a play is not recognized.

The boost from a rushing quarterback can be seen well with Colin Kaepernick's solid, but unspectacular passing DVOA. Kaepernick has already been one of the most prolific postseason rushers ever, which is why his total DYAR per game (84.5) places him considerably higher than any stat that's just focused on passing. Kaepernick's had prolific drive stats in the playoffs, but those speak more about the offense as a whole. We could use some more quarterback-specific stats to judge the player.

Third-Down Conversions

Breaking down the third-down numbers is always crucial because of their importance to the game. Most third downs directly decide whether or not the offensive drive extends and it's also a very quarterback-dependent down with roughly 80 percent of the attempts being passes.

I wrote a third-down analysis in the build-up for Super Bowl XLVIII and sure enough it proved to be important when the game was actually competitive -- a better word may be "undecided" given how this one played out. Seattle converted a few early, Tony Carter's pass interference in the end zone was a third-and-4 and both Manning interceptions were on third down. The deadly stat of quarterbacks being 1-of-39 at converting on third-and-11 or longer when passing against Seattle provided the game's key moment. Manning was hit as he threw on third-and-13 and the pass was intercepted for a touchdown by game MVP Malcolm Smith.

It should be hard to have a lot of playoff success without a lot of third-down success, but here are the numbers for 19 quarterbacks who made their playoff debut since 1999 (minimum five starts). All plays are included except for kneel downs and spikes. The "Avg. Yds Needed" are the average yards needed to convert on third down.

Surprise, it's Sanchez at the top. While it's only 61 plays, there had to be a reason he had such a good passing DVOA (28.3%), right? We have to call his six-game playoff career for what it was in 2009-10. He had two really good road games (2009 Bengals and 2010 Patriots), a good half in each of his AFC Championship losses and two subpar games where he needed to be bailed out by his teammates, but did get credit for a game-winning drive and fourth-quarter comeback (2009 Chargers and 2010 Colts). Personally, I would like to see him have a career revival at some point -- crazier things have happened -- so he can play in the playoffs again and prove if he can sustain his small sample size of success. If not, then he's going to be an eyesore on these lists, but that shouldn't degrade the merit of the metrics. Sanchez is just proof that the playoffs are too small of a sample to put more weight on than the regular season when judging the true caliber of a player.

Brees only ranking 16th shocked me given his overall statistical success in the playoffs. You can see his average length needed was not even that big, so it's not like he was often in third-and-long. Or was he? I broke down the third-down numbers by ranges of short (1-3 yards), medium (4-7) and long (8-plus). Quarterbacks are sorted by ascending average distance needed.

No one's had shorter third downs than Flacco, yet he ranks 13th or 14th across the board. Matt Hasselbeck ranks dead last on third-and-short and on third-and-long. Kaepernick has the third-shortest average distance, but has converted on third-and-long the best. Keep in mind we are talking about a conversion rate of 10-of-26 plays. Brady is the only player with 100 third-and-long attempts.

Staying out of third-and-long is another offensive goal. Sanchez stands out again in that 36.1 percent of his third downs are third-and-long, which is the second lowest percentage, but he also has the lowest rate of third-and-short attempts. He played on a run-heavy team, so running the ball was more logical in those situations. This creates a split where 50.8 percent of his third downs were third-and-medium, which is the highest rate of all quarterbacks here. Sanchez has the third-best conversion rate on those plays. Roethlisberger had the best and was third best on third-and-long, yet he was just 50 percent on third-and-short.

The only quarterbacks to rank in the top 10 in each range are Matt Ryan and Warner. Peyton just misses out by ranking 11th in third-and-long, trailing his brother Eli thanks to one more conversion. Of course there is some Manning's Law at work here. In this study of 2,128 plays, there were 50 plays involving a fumble. Most were sacks, but there were three plays where a quarterback completed a third-down pass that gained enough yards for a first down, but his receiver fumbled, costing him credit for the conversion. Peyton has two of those plays (Chargers got Marvin Harrison in 2007 and Julius Thomas this year) and Rodgers has the other (2010 Falcons).

This was just success rate for one down and we have already looked at DVOA and DYAR, but there are two more advanced stats we can look at to judge quarterbacks in the playoffs for better context.

Win Probability and Expected Points

Most of us are probably familiar with Brian Burke's work at Advanced NFL Stats for Win Probability Added (WPA) and Expected Points Added (EPA). I love the concepts behind these stats. Manning's lone touchdown pass in the Super Bowl came with a 36-0 deficit on the final play of the third quarter -- the first garbage-time scoring toss of his playoff career. A play like that will do practically nothing to the WPA, which is what should happen, just as an interception should also do nothing with the game already long decided.

So I collected all of the WPA and EPA postseason data from Advanced NFL Stats, which goes back to the 1999 season, leaving us with incomplete career data for some quarterbacks. Ultimately, we have 24 quarterbacks with at least five playoff starts since 1999. The quarterbacks are ranked by Win Probability Added per game.

Again, elder Manning and Brady lead the way in most cumulative WPA and EPA with Manning ahead on a per-game basis. The Super Bowl was a big hit for Manning with -0.19 WPA and -15.3 EPA. I have called it the second-worst playoff game of his career and that was before WPA and EPA, which are not adjusted for opponent, further supported that stance. Only the 2003 AFC Championship loss in New England had a worse WPA (-0.42) and EPA (-20.4) for Manning. However, over the course of 23 games these are still two more stats that show Manning as one of the best postseason quarterbacks ever.

Kaepernick and Wilson have been impressive, but their sample size is right there with Sanchez's. It will be interesting to see where they rank in a few years. Both will soon be looking for a huge pay raise, which in turn makes it difficult to keep such a balanced team around them like they have been able to enjoy so far.

Though his fall from grace was not cap related, Jake Delhomme should make everyone remember the danger of small sample sizes for playoff success. Delhomme once looked like a playoff stud with a 5-1 record, 0.37 WPA per game and 9.65 EPA per game. He would have been at the top of this table, but then the playoff disasters happened against the 2005 Seahawks and 2008 Cardinals. Those two games alone produced -0.64 WPA and -31.2 EPA for Delhomme. I am not saying this will happen to young quarterbacks like Wilson and Kaepernick, but future playoff success does not have many good predictors. Everyone thought Marino would return to the Super Bowl, but that never happened. Many thought the current Packers would go on a dynasty run, but they have not advanced past the Divisional round in the last three seasons. It's hard to keep making the playoffs, let alone doing something productive while there.

Brees does not get to the postseason as often as you'd like, but once he's there the performance is usually pretty good. Yet he's still just a game over .500 thanks to some close losses, including one in San Francisco where he threw two go-ahead touchdown passes and still lost.

Interesting how the three quarterbacks from the class of 2004 are bunched together. They also have been missing the playoffs more frequently these days. Rivers missed three in a row before this year. Roethlisberger's been out of three of the last five and it's four out of five for Eli.

Donovan McNabb looks out of place near the bottom with his 9-7 record, but it makes sense. Most of those Philadelphia playoff wins were thanks to the defense, so McNabb did not have to do that much. When the Eagles lost in the playoffs, it always meant multiple turnovers from McNabb and some came when the game was late and close.

Conclusion

There have now been 504 playoff games in NFL history. Football Outsiders has analyzed the play-by-play and done game charting for 534 games (including playoffs) since the 2012 season. That's just two years. While we may never get a clear picture of how the 1940 Redskins lost 73-0 to the Bears, we should be able to at least track down every playoff game since the merger at some point.

If the playoffs are so important for a quarterback's legacy, then justice must be served to the analysis instead of simplifying things to a win-loss record. Tracking events like dropped passes, pressure and yards after the catch would be very beneficial to understanding how the quarterback played relative to his teammates in the playoffs. We have more information than ever before on what happened in games, but rarely is it put to good use.

When there's a quarterback that's 10-4 in the playoffs and anyone can point to four plays, none of which even involved the quarterback, where if they had gone differently, his record would be 3-6*, then that should matter when comparing him to someone who may have had those crucial plays go against his team every time.

*There's no next week when you lose in the playoffs.

In the regular season, I always hear how "things even out" for quarterbacks in terms of breaks (good or bad) during a game. Over a long career I might believe that, but it's not true for one season and it's especially not true for the postseason. The sample size is not big enough for us not to be able to keep a balance sheet. I have already put in a lot of work to create a balance sheet for the last 30-plus years of the regular season. You want to know how many times a quarterback's kicker missed a clutch field goal or the defense gave up a fourth-quarter lead? I can produce that. Doing so for just the postseason is a piece of cake by comparison.

Now there is nothing simple about quantifying the contributions of a quarterback independent of those around him and adjusted for situations and opponent. But we still try, because we must. When the weight of evidence reveals the same conclusions, then it is pure ignorance to continue rejecting the facts.

Perspective cannot be lost. The numbers validate Mark Sanchez's postseason success, but at the end of the day it's a six-game sample. How could anyone think that means more than the 62-game performance Sanchez has shown in the regular season? Geno Smith wasn't drafted because the Jets felt bad for his draft day wait.

While it's undeniable the stakes change in the playoffs, the marks of individual greatness are the same if it's the first game of the season or the last. A quarterback has to make plays that help his team score points to put them in a position to win games. How those points are scored and how many are needed vary greatly, which is why we analyze data.

I will continue collecting and analyzing playoff data, because unlike Eli Manning, you can count on irrational arguments showing up for every postseason.