How Should We Judge Quarterbacks in the Postseason?
by Scott Kacsmar
Name an NFL quarterback with a lot of playoff losses and I'll show you one of the best quarterbacks to ever play the game.
You can't fall off the top of the mountain if you never made the climb, yet we live in a media-driven world where only winning gets glorified this time of year. For what are supposed to be the most important games of the season, the post-game analysis practically gets simplified into a caveman-like "winner clutch, loser choked" mantra where no one on the losing team could have possibly played well.
I have always refused to buy into that. My earliest game-charting work was for playoff games, scrutinizing every drive to find out what actually won and lost the game. I have collected a large volume of postseason data over the years and will be sharing some in this series of articles that will run through the Super Bowl. Next week we'll run playoff drive stats; the week after the Super Bowl, we'll have DVOA and DYAR playoff quarterback stats back to 1989. Today, we'll start with some general concepts about measuring playoff success.
Over the next two weeks we will hear much about Peyton Manning's legacy in the build-up for Super Bowl XLVIII. If he wins, he'll be 12-11 in the playoffs and will forever be known as the first quarterback to lead two teams to a Super Bowl win. He'll also be the first season leader in passing yards to win a Super Bowl. But if Manning loses, he'll fall back under .500 and own the record for most playoff losses (12) by a quarterback.
Manning is just trying to tie his younger brother Eli in Super Bowl rings, but it's the latter's 8-3 playoff record that many point to in saying little brother is the best big-game Manning. That's rubbish. When I see an 8-3 playoff record from a 10-year veteran, my first thought is "why doesn't he have more playoff losses?" No one's going to win the Super Bowl every year, but a truly great quarterback in this era can keep giving his team chances. The Giants have missed the playoffs in four of the last five seasons and Eli leads the league with 97 interceptions in that span. That playoff record will remain pristine when your team can't even get into the tournament. When the Giants haven't gone on their incredible Super Bowl runs, they are 0-3 in the playoffs under Eli and coach Tom Coughlin.
If a quarterback can only make the playoffs when his team's really good and can only win a playoff game when his team's playing great, then why should anyone think 8-3 reflects well on that one player? Similar to game-winning drive opportunities, a quarterback can make his record look better by playing worse and never getting to that situation.
Critics point to Peyton's interception in overtime against Baltimore last postseason as proof of choking, but at least he had his team in that position (and we know his safety is the one who put Denver in that overtime situation). This year, Eli threw so many game-crushing interceptions in September and October that he ensured there would be no opportunity to do so in January. Super Bowl winners Drew Brees (2007 and 2012 especially) and Ben Roethlisberger (2006 and 2012-13) have similarly had bad runs that doomed their team's chances of making the playoffs. It makes no sense when a player gets vilified for making a big mistake at the end of a close playoff game while the guy who couldn't even get to that point stays out of the crosshairs.
Colin Kaepernick had a miserable fourth quarter in Sunday's NFC Championship Game, but the 49ers are never that close without his brilliant runs and incredible touchdown throw in the third quarter. His Super Bowl counterpart from last year, the highly paid Joe Flacco, had a shot to make the tournament at 8-6. He finished the year with two losses, throwing one touchdown and five interceptions in the process.
But hey, at least Flacco didn't go one-and-done in January, right?
Seattle's Russell Wilson is the other quarterback under the spotlight and could become the fourth sophomore to win the Super Bowl, joining Kurt Warner, Tom Brady and Ben Roethlisberger. Oddly enough, of Wilson's four playoff starts, his most impressive was the only one he lost when he led a 20-point comeback in the fourth quarter in Atlanta before the defense allowed a game-winning field goal. On the road, he had to perform well above his usual output to overcome his team's worst defensive performance. That speaks volumes, but I know that leaves me in the minority because it was ultimately a loss.
Wilson's situation also speaks to another aggravating side of postseason judgment: he can get a very long pass for future failures if he wins a ring right now. If we run Tom Brady's career in reverse, he ends up looking like John Elway. For years he would have been castigated as a playoff fraud who put up huge numbers in the regular season but lost games in the playoffs where his team was heavily favored... until he finally ended his career with three rings on teams that were built more and more around defense and less around his arm.
Past playoff success is no guarantee of future playoff success, but many are quick to believe otherwise. Take the case of a certain Dallas quarterback with a 1-3 playoff record, who threw two touchdowns and eight interceptions in those pressure games. The Cowboys will never win big with Tony Romo, or can they? That quarterback I just described was not even Romo. That's what Troy Aikman did in the playoffs after he won his third Super Bowl. He made three trips to the playoffs in his last five seasons and was dreadful when his team was no longer the most talented in the league.
In this era, a quarterback winning early in his career is common, and it helps when he's on a cheap contract -- Manning makes more per week than Wilson has for all of 2013 -- and can have a more complete team in place around him. Still, that should not automatically absolve future failures, especially when that player gets better individually.
A great quarterback should be consistent enough to give his team a shot to win every week and every season. I will take that any day over someone who might have a one-month hot streak every five years, yet it's an unbelievable grind to convince certain people what's actually better when the postseason is involved.
What's the first sign of a great playoff quarterback? Just getting there often should be one prerequisite. This table shows that 59 quarterbacks have started at least one playoff game -- using the term "starting quarterback" very lightly for the pre-modern guys like Sammy Baugh and Sid Luckman -- in at least four different postseasons:
|Quarterbacks with 4+ Postseason Appearances|
|Peyton Manning||13||22||Dave Krieg||6||9||Sid Luckman*||5||6|
|Brett Favre||12||24||Randall Cunningham||6||9||Jim Plunkett||4||10|
|Tom Brady||11||26||Otto Graham||6||7||Neil O'Donnell||4||7|
|Joe Montana||11||23||Jack Kemp||6||6||Kerry Collins||4||7|
|Dan Marino||10||18||Kurt Warner||5||13||Doug Williams||4||7|
|John Elway||9||21||Joe Flacco||5||13||Brad Johnson||4||7|
|Terry Bradshaw||9||19||Eli Manning||5||11||Dan Fouts||4||7|
|Roger Staubach||8||17||Fran Tarkenton||5||11||Bernie Kosar||4||7|
|Jim Kelly||8||17||Steve McNair||5||10||Ken Anderson||4||6|
|Donovan McNabb||7||16||Danny White||5||10||Jim McMahon||4||6|
|Troy Aikman||7||15||Phil Simms||5||10||Jeff Garcia||4||6|
|Steve Young||7||14||Mark Brunell||5||10||Jake Plummer||4||6|
|Bob Griese||7||11||Aaron Rodgers||5||9||Chad Pennington||4||6|
|Warren Moon||7||10||Philip Rivers||5||9||Matt Ryan||4||5|
|Ben Roethlisberger||6||14||Daryle Lamonica||5||9||Y.A. Tittle||4||4|
|Ken Stabler||6||12||Ron Jaworski||5||8||Norm Van Brocklin||4||4|
|Matt Hasselbeck||6||11||Len Dawson||5||8||Ed Danowski*||4||4|
|Drew Brees||6||11||Johnny Unitas||5||8||Bob Waterfield||4||4|
|Craig Morton||6||10||Billy Kilmer||5||7||Arnie Herber*||4||4|
|Bart Starr||6||10||Sammy Baugh*||5||6||*Leading passer in two-way player era|
A Hall of Fame snub like Ken Anderson has some pretty stats in the playoffs, some collected via garbage time, but it's hard for me to consider anyone on the right column that's retired to be an all-time playoff performer when they just did not get there enough. The playoff system has changed and the field has expanded, but a handful of appearances are still relatively small.
Have you ever looked at Jeff Hostetler's playoff stats? One more big run and he would have qualified for rate stats (minimum is 150 attempts) and be the all-time leader in postseason passer rating. Similar things can be said of Tony Eason and believe it or not, Alex Smith has nine touchdowns and zero interceptions on 114 pass attempts in the playoffs. Before that big blown lead in Indianapolis you could already infer from the NBC announcers how Smith was "raising his game from the regular season."
Jake Delhomme showed us why we can't trust those small sample sizes of playoff brilliance. His first six playoff games could not have been much better; he nearly beat the 2003 Patriots in the Super Bowl, then brought the 2005 Panthers to the NFC Championship game. But Delhomme crashed back to Earth harder than any quarterback has in the postseason with horrific losses in Seattle (2005 NFC Championship) and at home against the 2008 Cardinals (six turnovers). Even though Delhomme was very good for 75 percent of his postseason performances, without the ring, only the last brutal 25 percent becomes his legacy.
Why Wait for the Playoffs to Play Your Best?
The whole concept of a quarterback "raising his game" in the playoffs is a flawed one and the crux of today's feature. Doesn't this imply the quarterback did not play as best as he could have in the regular season? This isn't the NBA or baseball. All 16 games matter and we have seen many teams miss the playoffs after failing to execute in some close games.
Then we have the maddening criticism of "[Quarterback X] does not play as well in the playoffs as he does in the regular season." Over a large enough sample size, no quarterback should be able to outperform their regular-season level of play in the playoffs. The competition is too superior. There aren't any 4-12 Jacksonville squads in the playoffs and usually the worst defenses never make it that far. Numbers should go down naturally and in the cases where you're comparing 180 games to 10, well that's just asking for trouble.
So in an attempt to make it more apples-to-apples, I compared prolific playoff quarterbacks to how they did in regular-season games against teams who made the playoffs that year. Sure, those games do not always carry the same intensity and sometimes you have no idea when that 3-5 team is going to peak later and make the playoffs, but it's better than comparing games against known bottom-dwellers.
For my sample, I used the 21 quarterbacks in NFL history with at least 11 playoff starts. The question is how would I compare them? We are currently a bit limited in advanced stats with ESPN's QBR going back to 2006 and DVOA/DYAR back to 1989. Ken Stabler, Steve Young and Joe Flacco all have a similar playoff passer rating, yet each played in different eras, making any straight-up comparisons irrelevant.
Defense-Adjusted Passer Rating (DAPR)
About four years ago I created a quick and cheap system I call Defense-Adjusted Passer Rating (DAPR). I have sprinkled it into a few of my articles over the years, but never actually explained how I calculate it. Let's go through it now.
First, it's a plus/minus adjustment for passer rating, which I know has flaws and a lot of people hate, but with DAPR we can compare fairly across eras by adjusting directly for the opponent. Just like passer rating, sacks are not included. It's also not using the iterative method that adjusts for the quarterbacks each defense played too, which would make it better, but a simple calculation to adjust for opponent was always my goal. This will never replace DVOA or ESPN's QBR, but it's a quick tool without any black-box equation, so anyone can use it. Also, it doesn't require play-by-play, so we can compute it for seasons prior to 1989. Just be careful to use gross passing yards and not net passing yards (sacks included).
Basically, for the regular season we're going game-by-game and subtracting out the quarterback's passing stats from the opponent's season passing stats and finding what impact his game had on the opponent's defensive passer rating.
Here's a sample calculation for Drew Brees against the Seahawks in Week 13:
Brees was 23-of-38 passing for 147 yards, one touchdown and zero interceptions. For the season, Seattle allowed 309-of-524 passes for 3,050 yards, 16 touchdowns and 28 interceptions for a 63.4 passer rating. Subtract all of Brees' single-game passing stats from those Seattle stats and the new passer rating is 62.3. That means Brees actually raised the Seahawks' passer rating. Subtract the new rating (62.3) from the final (63.4) and Brees gets a +1.10 for the game. So for as bad as Brees seemingly played, given how good Seattle was, he gets a solid adjustment for the night. A 0.0 would mean he played completely average relative to the opponent. A negative would be below average. You then do the same calculation for each game and can add them together for a season or take a per-game average.
Now if Brees had that exact same stat line against a more porous Seattle defense like the 2010 Seahawks (89.7 DPR), then his DAPR would be -0.85, so this definitely acknowledges the opponent's statistical strengths. Throwing a few touchdowns against a defense that allows very few touchdown passes would give a quarterback a higher DAPR than someone who has a similar passer rating, but maybe got it done by having a very high completion percentage and just one touchdown.
The feature I really like about DAPR is it rewards a quarterback for performing well over a larger volume. Consider this example:
- Two quarterbacks played the same opponent.
- Quarterback A: 10/20 for 150 yards, TD, 0 INT, 91.7 PR
- Quarterback B: 20/40 for 300 yards, 2 TD, 0 INT, 91.7 PR
Traditionally, this would be recognized as the same level of passing efficiency given the equality in completion percentage, yards per attempt, touchdown percentage and interception percentage. But we know Quarterback B sustained this play over double the workload. DAPR will give Quarterback B at least twice as much credit for this game.
Defensive stats are prorated to a 16-game season. I found older quarterbacks were getting too much credit in the shorter seasons, especially the nine-game strike season (1982). For the playoffs, I add the stats to the regular-season numbers instead of subtracting, so naturally it would be harder to get a bigger score since your game is now basically 1/17th of the sample instead of 1/16th.
I would not recommend calculating DAPR until the end of each season, which is a minor drawback to this method.
Here are the 10 best and worst playoff games since 1945, according to DAPR:
|Top 10 Best Playoff Games by DAPR (Since 1945)|
|1||Tobin Rote||DET||NFL-C||12/29/1957||CLE||W 59-14||19||12||63.2||280||4||0||146.4||7.99|
|2||Daryle Lamonica||OAK||AFL-D||12/21/1969||HOU||W 56-7||17||13||76.5||276||6||1||133.0||6.50|
|3||Kurt Warner||ARI||NFC-WC||1/10/2010||GB||W 51-45 OT||33||29||87.9||379||5||0||154.1||6.04|
|4||Peyton Manning||IND||AFC-WC||1/4/2004||DEN||W 41-10||26||22||84.6||377||5||0||158.3||6.03|
|5||Joe Montana||SF||SB||1/28/1990||DEN||W 55-10||29||22||75.9||297||5||0||147.6||5.51|
|6||Daryle Lamonica||OAK||AFL-D||12/22/1968||KC||W 41-6||39||19||48.7||347||5||0||119.3||5.21|
|7||Peyton Manning||IND||AFC-WC||1/9/2005||DEN||W 49-24||33||27||81.8||458||4||1||145.7||4.97|
|8||Peyton Manning||IND||AFC-C||1/24/2010||NYJ||W 30-17||39||26||66.7||377||3||0||123.6||4.67|
|9||Terry Bradshaw||PIT||SB||1/21/1979||DAL||W 35-31||30||17||56.7||318||4||1||119.2||4.57|
|10||Johnny Unitas||CLT||NFL-C||12/27/1959||NYG||W 31-16||29||18||62.1||264||2||0||114.7||4.56|
|Top 10 Worst Playoff Games by DAPR (Since 1945)|
|1||Jay Schroeder||RAI||AFC-C||1/20/1991||at BUF||L 51-3||31||13||41.9||150||0||5||17.6||-5.31|
|2||Jake Delhomme||CAR||NFC-D||1/10/2009||ARI||L 33-13||34||17||50.0||205||1||5||39.1||-4.89|
|3||Bobby Layne||DET||NFL-C||12/26/1954||at CLE||L 56-10||42||18||42.9||177||0||6||15.8||-4.82|
|4||Stan Humphries||SD||AFC-D||1/10/1993||at MIA||L 31-0||44||18||40.9||140||0||4||11.6||-4.81|
|5||Richard Todd||NYJ||AFC-C||1/23/1983||at MIA||L 14-0||37||15||40.5||103||0||5||8.8||-4.53|
|6||Craig Morton||DEN||SB||1/15/1978||DAL||L 27-10||15||4||26.7||39||0||4||0.0||-4.25|
|7||Dan Pastorini||HOU||AFC-C||1/7/1979||at PIT||L 34-5||26||12||46.2||96||0||5||16.3||-4.22|
|8||Kerry Collins||NYG||SB||1/28/2001||BAL||L 34-7||39||15||38.5||112||0||4||7.1||-4.07|
|9||Y.A. Tittle||NYG||NFL-C||12/31/1961||at GB||L 37-0||20||6||30.0||65||0||4||1.0||-3.91|
|10||Todd Marinovich||RAI||AFC-WC||12/28/1991||at KC||L 10-6||23||12||52.2||140||0||4||31.3||-3.86|
So using DAPR per game and only using games the quarterback started, I have my data, which you can see in full here for the 21 quarterbacks. The regular-season games are listed above the playoff games. "PR" is passer rating. "PF" and "PA" are the average points scored and allowed per game. Normally I would adjust those for non-offensive scoring, but did not have time to do that for so many games.
|Quarterbacks vs. Playoff Teams (Regular Season vs. Postseason)|
Joe Montana was one bad melon farmer. The other so-called "Joe Cool" had that great playoff run, but going back to last year, he's 1-8 in his last nine regular-season games against playoff teams. Prior to that, he had a 9-0 streak. That's a harsh case of regression to the mean -- regression way past the mean, apparently -- but from this table you can see there's nothing wrong about being around .500. In similar fashion to Flacco, Eli Manning is on a 2-9 stretch against the tough teams.
Before the Wild Card was added, there were obviously fewer playoff teams. Through the 1976 season, Ken Stabler actually had only two fewer playoff starts (six) than he had regular-season starts against playoff teams (eight). There's also no quarterback on the list who could match the Snake when it came to playing better against top competition in the playoffs relative to his regular seasons.
This table shows the differences for each quarterback from the regular season to the playoffs. Anything in red is a decline in the postseason:
|Quarterbacks vs. Playoff Teams: Postseason Decline|
|Quarterbacks vs. Playoff Teams: Postseason Decline|
As you can tell from the averages, this group actually produced better results in every category in the playoffs. That may not be a surprise though, given the theory that with an increased sample of games against playoff competition, statistics would suffer and be fortunate to hover around average.
[ad placeholder 3]
By DAPR, Stabler had the worst passing stats of anyone here in the regular season, yet the best in the postseason. Not even an argument of "the Sea of Hands should have been intercepted and that wasn't roughing the passer on Sugar Bear Hamilton" can make up that huge difference.
Peyton Manning rounds out a group of six quarterbacks above 1.0 DAPR per game in postseason starts. Get used to seeing names like Peyton, Brees and Warner at the top of these postseason lists in the coming weeks. Also get used to some of the more mediocre results from multiple-ring winners like Brady, Elway, Roethlisberger and Eli.
Even if by the slimmest of margins, Jim Kelly is the only quarterback to actually win a higher percentage of games in the regular season than the playoffs. Kelly, Staubach, Brady and Young are among the biggest decliners when it comes to their postseason performances. That doesn't quite fit the narrative.
What if we tried to make it even more apples-to-apples by only including seasons where the quarterback's team made the playoffs? This could help those who started off on really bad teams like Peyton, Aikman and Young. Then again, one could argue this does not penalize those who miss the playoffs too often with solid teams, but here's a look at the regular-season games against playoff teams only in playoff seasons:
|Quarterbacks vs. Playoff Teams in Regular Season (Playoff Years Only)|
Fran Tarkenton and Bob Griese look far more respectable than their abysmal records above, but both did start on bad teams in the 1960s. Elway and Marino are surprisingly still a few games under .500, even in playoff years. While Montana's stats look so impressive again, that overlooked 49ers defense is also a huge factor, allowing a list-low 16.8 points per game (also the lowest at 18.0 points per game in all regular-season games in this study).
We'll conclude with a look at the DAPR per game postseason leaders (minimum 150 attempts) and also the average defensive passer rating of each opponent a quarterback has faced in the playoffs.
|Postseason DAPR Leaders (Min. 150 Pass Attempts)|