Stat Analysis
Advanced analytics on player and team performance

Solving the Peyton Playoff Puzzle: Part I

by Scott Kacsmar

Peyton Manning is going to retire well short of winning the most Super Bowls in NFL history. He can become the 12th quarterback with multiple rings with a win over the Carolina Panthers in Super Bowl 50. If that is his last win, he would also finish with a 14-13 record, making him a winning quarterback in the postseason. Seventeen quarterbacks have started at least 12 playoff games, and Manning and Dan Marino (8-10) are the only players in that group without a winning record. A victory on Sunday may be enough for some, but there will always be those who look at Manning's playoff career as a disappointment. That is a fair viewpoint, but how much of the disappointment should fall on the player versus his teammates?

Somehow, the phrase "greatest regular-season quarterback ever" has become the ultimate backhanded compliment to Manning's career. That title really is not up for debate, either. Manning is the most decorated quarterback in NFL history with five MVP awards, seven first-team All-Pro selections, 14 Pro Bowls and a ton of records. When you call him the greatest regular-season quarterback ever, you only leave a few quarterbacks in the field who can argue they had a better career, and that is even with an overemphasis on the postseason.

So how does one of the best quarterbacks ever spend the majority of his career without a winning playoff record?

This Peyton Manning playoff puzzle has existed for years, but no one ever put in much effort to solve it. They have called him a choker, even though countless other big-name quarterbacks have flopped in big games and crucial moments. They have said his style of offense does not translate to January and that he is too robotic, too studied to succeed in the playoffs. This ignores all the stellar playoff games he has had, or the fact that he is the only quarterback to have a fourth-quarter lead in 13 consecutive playoff games.

When you put the pieces together, you start to see the picture of a great playoff resume, but that win-loss record piece just does not fit. After years of research and an offseason of clarity on the subject, I am ready to solve this puzzle for good.

Note: parts of this article previously appeared on ESPN Insider.

The Playoff Resume You Never Hear About

The standard that Manning set in the regular season was always going to be tough to meet in the playoffs. With each passing year, a quarterback's legacy grows, and stigmas become harder to shake. Manning lost his first three playoff games and took six seasons to get a postseason win, and nine seasons to win a Super Bowl. His stats were not very Manning-like during that Super Bowl run. Since then he had a couple of high-profile losses, including two Super Bowls. Add it all together and you have an easy target for years of playoff criticism. The good parts get brushed to the side, because they do not fit the narrative.

However, the thought that Manning has had a bad playoff career or even merely an average one is one of the silliest things I have come across in 13 years of football research.

There have been 215 quarterbacks (well, one was a true running back) to start the 525 playoff games in NFL history. I do not expect the average analyst to talk about a bell-shaped curve, but you would have to have a pretty repugnant idea of distribution to think Manning's playoff career is anything close to average. He will be just the seventh quarterback to start at least four Super Bowls. Thirty-one quarterbacks have won a Super Bowl, and only 20 were named Super Bowl MVP. That should at least get Manning into the above-average category, especially when you start ignoring some of the ridiculous small sample size issues. Mark Sanchez and Alex Smith have some better playoff marks than Manning and Tom Brady. Enough said.

When you get into the stats, Manning's QBR ranks sixth in the playoffs since 2006, according to ESPN. We have observed in the past that Manning's playoff stats are very similar to his stats against playoff teams in regular-season games, a sample size of 98 games now. Although his last four playoff games have not been too hot, Manning still holds the No. 8 passing DVOA since 1989. (We hope to add 1986-88 very soon.)

Playoffs: Passing DVOA Leaders Since 1989 (Min. 150 Passes)
Rk Quarterback Games DVOA
Rk Quarterback Games DVOA
1 Joe Montana 9 62.3% 24 Randall Cunningham 10 10.7%
2 Kurt Warner 13 42.3% 25 Matt Ryan 5 9.1%
3 Drew Brees 11 32.5% 26 Dan Marino 12 6.6%
4 Troy Aikman 16 31.0% 27 Tony Romo 6 6.6%
5 Mark Sanchez 6 28.3% 28 Ben Roethlisberger 17 6.5%
6 Philip Rivers 9 27.5% 29 Chad Pennington 6 6.1%
7 Steve Young 14 26.2% 30 Rich Gannon 9 5.4%
8 Peyton Manning 26 25.3% 31 Michael Vick 6 4.0%
9 John Elway 14 24.7% 32 Neil O'Donnell 9 3.8%
10 Aaron Rodgers 13 24.0% 33 Brad Johnson 7 3.3%
11 Tom Brady 31 21.7% 34 Jake Delhomme 8 3.2%
12 Mark Rypien 7 21.5% 35 Jeff Garcia 6 2.4%
13 Vinny Testaverde 5 20.0% 36 Steve McNair 10 2.2%
14 Colin Kaepernick 6 19.9% 37 Alex Smith 5 -0.4%
15 Eli Manning 11 17.9% 38 Andrew Luck 6 -1.2%
16 Russell Wilson 10 17.2% 39 Donovan McNabb 16 -1.3%
17 Brett Favre 24 17.1% 40 Jake Plummer 6 -6.5%
18 Joe Flacco 15 16.8% 41 Mark Brunell 11 -6.7%
19 Warren Moon 6 15.8% 42 Stan Humphries 6 -13.0%
20 Matt Hasselbeck 11 14.9% 43 Jim Harbaugh 5 -14.0%
21 Jim Kelly 15 14.8% 44 Kordell Stewart 6 -16.3%
22 Cam Newton 5 13.9% 45 Drew Bledsoe 7 -25.4%
23 Kerry Collins 7 12.1% 46 Andy Dalton 4 -34.9%

Even Manning's criticized Super Bowl run in 2006, when he threw three touchdowns and seven interceptions, holds up here. His DVOA that postseason was 29.8%, because it was the only time a team beat the league's top three defenses in a single postseason. His QBR, which is not adjusted for opponents, was still above average (54.2). How did the Colts win with that stat line from Manning? Five of his interceptions were thrown while leading, and four were thrown on third-and-10 or longer, when turnovers usually do less damage. How rare is the latter? No other quarterback has thrown more than three such interceptions in the playoffs since 1994, let alone four in a single postseason. Manning made up for his most damaging pick (Asante Samuel's pick-six) with 35 points in the rest of the game.

Since 1989, Manning has three of the top six playoff games in DYAR. He has the most playoff games with at least 200 DYAR (six). He is 4-1 in the AFC Championship Game, with his first three wins among the best games of his career. He led the largest comeback (18 points) in any championship game in NFL history. He threw five touchdown passes and had a perfect passer rating against the 2003 Broncos. A year later, he threw for 458 yards against Denver in a wild-card win. He has three 400-yard playoff games, tied with Drew Brees for the most in NFL history.

The only area where Manning's playoff resume looks average is the record, which is dead average at 13-13.

Victim of His Own Success

No one likes to lose a playoff game, but is it not still better to make the playoffs than to miss them entirely? With respect to the butterfly effect, Manning's playoff record could be 13-9 (.591) if he had not lead some of his weaker Indianapolis teams to double-digit wins, including 2000 (when they started 7-6), 2002 (started 4-4), 2008 (started 3-4) and 2010 (started 6-6). Those four teams lost in the wild-card round, including two overtime losses and another game decided on the final snap.

There, puzzle solved. Worse play down the stretch from Manning in those four seasons and he is four games above .500 in the playoffs. Who said this needed a complex solution?

Joking aside, continuously making the playoffs really is a main reason why Manning holds the records for most playoff losses (13) and most one-and-done losses (nine). The only formula for keeping him out of the playoffs (as happened in 1998 and 2001) has been for the Colts to go 1-9 against playoff teams and for the defense to allow the most points per drive. When you make the playoffs year after year, often with a flawed team (eight of Manning's playoff defenses ranked 15th or worse in DVOA), you are going to accumulate many playoff losses unless you become a dynasty.

Worse, if you go one-and-done, you have to win at least two games the next season just to get back to .500. When you earn a first-round bye as often as Manning's teams have (eight times, second only to Brady's 10), two wins means a trip to the Super Bowl. Those opportunities do not grow on trees. Those eight byes Manning has earned should essentially serve as eight wild-card wins, but the quarterback's record does not reflect that advancement.

Name a quarterback with a lot of playoff losses, and I will show you one of the best to ever play the game. I wrote an article about that two years ago. Eli Manning's record still sparkles at 8-3, because the Giants missed the playoffs in six of the last seven years. But hey, that means no one-and-done, or no interceptions in the clutch in January. All of that goes out the window when you lose enough games in the regular season. The Bengals are the latest example of this ass-backwards thinking, because they are deemed a laughingstock for losing in the wild-card round five years in a row. Yet nearly half of the league would trade places with Cincinnati in the last five years. At least they kept winning and gave themselves a shot in the tournament.

Some quarterbacks can only make the playoffs when their team is good, and some can only win in the playoffs when their team is playing great. If you are one of the elite few who can drag just about any roster into the playoffs year after year, you are going to experience some disappointing losses. Just being there should be rewarded, but instead you get branded with a one-and-done loss while the other player who missed the playoffs entirely gets nothing. Oh, if his name is Sam Bradford or Ryan Tannehill, maybe he gets a raise.

That is just one of many reasons why a quarterback's win-loss record in the postseason is such a poor indicator of performance. In the regular season, there is some value to it. Over a large enough sample size, you would be hard-pressed to find a good quarterback with a losing record. But in the playoffs, the record becomes more misleading than ever thanks to the one-and-done system. Come playoff time, you are no longer playing the Browns and Rams. Most of the bad defenses have been eliminated. Beating good teams is harder, and there is no next week when you lose. Many games are ultimately decided by plays where the quarterback is not even on the field.

Big games are not just limited to the playoffs, because the regular season is crucial to NFL success. Manning's season would likely already be over had the 2015 Broncos not earned a No. 1 seed (let's give some big credit to Brock Osweiler there in Manning's absence this time). But as far as navigating the regular season to get to the playoff goes, Manning has done it as well as anyone in his career.

Only 62 quarterbacks have even started in four different postseasons. That number shrinks to five if we look at those who have played 10 postseasons.

Most Playoff Losses Most One-and-Dones
Rk Quarterback Losses Rk Quarterback Losses
1 Peyton Manning 13 1 Peyton Manning 9
2 Brett Favre 11 2 Andy Dalton 4
3 Dan Marino 10 2 Billy Kilmer 4
4 Tom Brady 9 2 Bob Griese 4
5 Jim Kelly 8 2 Dave Krieg 4
6 Joe Montana 7 2 Jack Kemp 4
6 John Elway 7 2 Joe Montana 4
6 Donovan McNabb 7 2 Warren Moon 4
6 Warren Moon 7 2 Y.A. Tittle 4

By simple math, Manning is the quarterback most likely to hold the record for most playoff losses. He holds the record for most appearances with 15, two more than the next closest quarterback. Marino was the first to 10 losses, and he was surpassed by Brett Favre's 11. For Manning to have avoided this record, he would have needed to win a record five Super Bowls -- an unrealistic goal for anyone.

So far we have shown Manning's impressive playoff resume and explained the record number of playoff losses as a result of his consistency at leading teams into the playoffs. But why only 13 playoff wins? People wanted to see more wins from his career, and given the individual success, there should have been more.

The L Word

"See, the luck I've had can make a good man turn bad."

Bringing the element of luck into this might be as dangerous as throwing a hand grenade, but we are going to do it carefully. I have said in the past that I consider Manning the unluckiest quarterback in playoff history based on having the most close losses, among other things like poor starting field position. On a percentage basis, Manning probably is not the unluckiest. That may be Bernie Kosar or Warren Moon. For a cruder single-game example, Teddy Bridgewater has a legit claim to terrible luck in his first playoff game. It was spoiled by brutal weather conditions (one year before Minnesota gets a new roofed stadium), Adrian Peterson's fumble led to Seattle's game-winning field goal, and Blair Walsh shanked the shortest do-or-die field goal in playoff history. Ouch. And that's not even mentioning that Seattle's 9-point comeback started thanks to a botched snap sandlot play from Russell Wilson. Fortunately, Bridgewater is just 23 and has plenty of time to change things, but there is no guarantee luck evens itself out in a playoff career.

[ad placeholder 3]

Let's be clear in how luck is defined in the context of a quarterback's playoff career. From Merriam-Webster: "the events or circumstances that operate for or against an individual." The other important part is that these events are happening out of the quarterback's control. Over the course of a three-hour game, there are a lot of things that could qualify, but we usually just focus on the most important events that have the biggest impact on the game, such as the Peterson fumble and the Walsh field goal in that Bridgewater example. The plays from late in the game tend to be the most significant due to the lack of time left to overcome them. There is also a preference to look at plays that have a general low probability of happening, such as a return touchdown, dropped pass or blocked punt. If a quarterback throws a pass right to a defender for an interception, that is not bad luck for the quarterback. That is a bad play, and it is bad luck for his teammates that they have to absorb the consequences of his actions.

For Manning, we can list a ton of unlucky circumstances that were out of his control. He has 26 playoff games after all, but just look at his three Super Bowls. The Super Bowl is generally known for good weather, but Manning's win in Super Bowl XLI was the most weather-affected Super Bowl of them all, with the Miami rain leading to six fumbles from the Colts and Bears. Manning's receivers dropped six passes, but he still got the MVP anyway. Devin Hester started that one with the then-fastest score in Super Bowl history with his kick return touchdown 14 seconds into the game. The record was broken when Seattle scored 12 seconds into Super Bowl XLVIII against Manning's Broncos, when miscommunication on the first snap from scrimmage saw the bail sail over Manning for a safety. The two quickest deficits in Super Bowl history happened to Manning, and he didn't even touch the ball either time. Then when you mention the most significant surprise onside kick in NFL history, it was Sean Payton's call to start the second half in Super Bowl XLIV. Hank Baskett botched the recovery and the Saints, down 10-6 at the time, went on to score a big touchdown.

Again, due to the sheer volume of playoff games he has, we can go a book's length on this type of bad luck for Manning. But we are looking for the most significant moments for each quarterback, and some of this stuff is just neat context more than anything game-deciding. The fact is, there probably isn't much that has happened to Manning's teams in the playoffs that has not also happened to other teams, but good luck finding someone else that has experienced all of these things in his playoff career like Manning has. That is where I am coming from when I say he has been the unluckiest quarterback in playoff history.

I can actually thank some random Patriots fans for providing me with the proper clarity on this subject last offseason after New England's fourth Super Bowl win. The usual line of "Brady would have six rings without the Giants' two catches" came up, which deserves the usual reply of "he would have two if not for Adam Vinatieri." But the enlightened response of "Brady always has them in position" was my light-bulb moment.

Many of the best quarterbacks always seem to have their team in a position to win a playoff game, but many times the game-deciding play is out of their control. Andy Dalton gets crucified for his postseason history for reasons well beyond his 0-4 record. Not only were the Bengals not close to winning those games, but his stats are horrific. You saw the table above with the DVOA rankings. However, there are 10 active quarterbacks with a DVOA of at least 16.0% in the playoffs. There are eight active quarterbacks with a Super Bowl ring, the most at one time in NFL history. Brady is not the only quarterback capable of having his team in position to win playoff games with respectable individual statistics. We see it from several other quarterbacks, past and present. The more you can make the playoffs and do that, the more respect you'll earn, but the real driving force behind the wins and losses tends to be someone other than the quarterback.

[ad placeholder 4]

Nothing is stopping us from keeping track of this stuff for every game (I already do), but at the very least we can do it for the playoffs, which are supposed to be so much more important for legacies. We can keep track of when the quarterback made the decisive play, good or bad, and when someone else did. It usually is going to be someone else.

Everyone knows about Joe Montana's breakout playoff moment culminating in "The Catch" to Dwight Clark in the 1981 NFC Championship Game, but did you know Montana turned the ball over four times that day, including an interception on the drive before Clark's catch? He may have been a little spooked from a pre-game death threat, but that was a shaky performance. Despite the great drive, it would have gone for naught had the 49ers not finished the game on defense, leading 28-27. Danny White hit a pass to the San Francisco 44 with 38 seconds left. One more good throw and the Cowboys would have been in range for a game-winning field goal, but White was sacked and fumbled, and Jim Stuckey recovered the ball to clinch the win. Montana's moment lives in lore, but we should still make note of the interception he threw on his previous drive and the fumble from White.

When fans talk about Manning's crunch-time turnovers in the playoffs, they speak as if there is a long history of them. Do you know why they only bring up the Tracy Porter pick-six in Super Bowl XLIV and the overtime interception against the 2012 Ravens? Those are the only two times in the playoffs when Manning has turned the ball over in the fourth quarter or overtime, tied or down by one score. Brett Favre has done that in three different NFC Championship Games (1995, 2007, and 2009). Colin Kaepernick had three turnovers in the fourth quarter of the 2013 NFC Championship Game. Mistakes happen.

We can do a better job of charting close playoff games. The following table breaks down close games for 28 quarterbacks with at least 10 playoff starts. Only starts were included, and games where the quarterback did not play in the fourth quarter due to injury were excluded for Tom Brady, John Elway, Jim Kelly, Donovan McNabb, Kurt Warner and Joe Montana.

Playoffs: Breakdown of Starts
Quarterback Games Wins Losses Pct. 4Q/OT Lead Pct. 4QC/GWD W 4QC/GWD L Pct. Win or Close Pct.
Jim Plunkett 10 8 2 0.800 9 90.0% 1 2 0.333 10 100.0%
Ben Roethlisberger 17 11 6 0.647 13 76.5% 4 5 0.444 16 94.1%
Roger Staubach 17 11 6 0.647 14 82.4% 1 5 0.167 16 94.1%
Joe Flacco 15 10 5 0.667 12 80.0% 2 4 0.333 14 93.3%
Aaron Rodgers 13 7 6 0.538 10 76.9% 1 5 0.167 12 92.3%
Kurt Warner 12 9 3 0.750 10 83.3% 3 2 0.600 11 91.7%
Ken Stabler 12 7 5 0.583 8 66.7% 3 4 0.429 11 91.7%
Drew Brees 11 6 5 0.545 7 63.6% 3 4 0.429 10 90.9%
Steve McNair 10 5 5 0.500 5 50.0% 3 4 0.429 9 90.0%
Mark Brunell 10 5 5 0.500 5 50.0% 1 4 0.200 9 90.0%
Russell Wilson 10 7 3 0.700 9 90.0% 4 2 0.667 9 90.0%
Terry Bradshaw 19 14 5 0.737 15 78.9% 4 3 0.571 17 89.5%
Peyton Manning 26 13 13 0.500 19 73.1% 2 10 0.167 23 88.5%
Tom Brady 30 21 9 0.700 24 80.0% 9 5 0.643 26 86.7%
Eli Manning 11 8 3 0.727 8 72.7% 5 1 0.833 9 81.8%
Joe Montana 22 16 6 0.727 17 77.3% 5 2 0.714 18 81.8%
Bob Griese 11 6 5 0.545 7 63.6% 1 3 0.250 9 81.8%
Matt Hasselbeck 11 5 6 0.455 8 72.7% 2 4 0.333 9 81.8%
Jim Kelly 16 9 7 0.563 11 68.8% 1 4 0.200 13 81.3%
Troy Aikman 15 11 4 0.733 11 73.3% 1 1 0.500 12 80.0%
Donovan McNabb 15 9 6 0.600 10 66.7% 1 3 0.250 12 80.0%
Brett Favre 24 13 11 0.542 16 66.7% 3 6 0.333 19 79.2%
John Elway 20 14 6 0.700 14 70.0% 6 1 0.857 15 75.0%
Phil Simms 10 6 4 0.600 7 70.0% 0 1 0.000 7 70.0%
Warren Moon 10 3 7 0.300 7 70.0% 2 4 0.333 7 70.0%
Steve Young 14 8 6 0.571 8 57.1% 1 1 0.500 9 64.3%
Fran Tarkenton 11 6 5 0.545 7 63.6% 1 1 0.500 7 63.6%
Dan Marino 18 8 10 0.444 10 55.6% 4 2 0.667 10 55.6%
4QC = fourth-quarter comeback; GWD = game-winning drive

The "Win or Close" column is basically how often the quarterback "put his team in position to win," whether by winning the game outright or having possession in the fourth quarter or overtime with the score tied or a one-score deficit. Outside of poor Marino, who had eight blowout losses with Miami, most quarterbacks were above 80 percent.

Only three of Manning's playoff losses were not close games late, but a few other things really stand out from this table. For starters, Manning's abysmal 2-10 record at 4QC/GWD opportunities is nothing like the great 56-46 (.549) record he has in the regular season. While you might think that fuels the "choker" narrative, note that Manning had 19 fourth-quarter leads, yet only 13 playoff wins. Now we are onto something.

No quarterback in NFL history has lost more playoff games (six) after leading in the fourth quarter than Manning. Moon is the next closest at four games. Manning saw his team lose after leading in the final 40 seconds in four games. Again, only 28 quarterbacks have even been to the postseason six times.

This is actually worse than it sounds. Earlier, I glossed over Manning's record of nine one-and-done seasons, five more than any other quarterback. Yes, the 15 playoff appearances do a lot to explain why he has this record, but why such a high number like nine? Why not just five or six? Why didn't Manning's teams do better in their first playoff games?

The blown leads fill in so much of the playoff puzzle. Here is a list of every quarterback with at least two playoff losses after leading in the fourth quarter. The last column shows how many of those losses resulted in a one-and-done postseason.

Most Playoff Losses with Fourth-Quarter Lead
Rk Quarterback 4Q Leads Lost One-and-dones
1 Peyton Manning 6 5
2 Warren Moon 4 3
3 Matt Hasselbeck 3 2
3 Roger Staubach 3 2
3 Tony Romo 3 2
3 Aaron Rodgers 3 1
3 Brett Favre 3 1
3 Tom Brady 3 0
9 Bobby Hebert 2 2
9 Philip Rivers 2 2
9 Steve Bartkowski 2 2
9 Alex Smith 2 1
9 Ben Roethlisberger 2 1
9 Bernie Kosar 2 1
9 Dan Marino 2 1
9 Ken Stabler 2 1
9 Jim Kelly 2 1
9 Neil O'Donnell 2 1
9 Charlie Conerly 2 1
9 Joe Flacco 2 0
9 Russell Wilson 2 0

Five of Manning's six blown leads resulted in a one-and-done postseason, two more than any other quarterback in NFL history. While you can argue it hurts more to a fan to lose a lead in the later rounds like in the case of Brady, that does not hurt the playoff record as much as going one-and-done does. Just look at all the times Manning was denied a shot at more AFC Championship Games or Super Bowls because of an early exit. His teams are an impressive 13-4 in the playoffs when they were able to get past the first game, but more often than not they stumbled out of the gate, hence a record number of one-and-done years.

If Manning's blown leads -- his bad luck -- were distributed more evenly amongst all playoff rounds, he would have a better playoff record with more wins. But you do not get to go to the AFC Championship Game when your defense lets backup Billy Volek drive 78 yards for the go-ahead touchdown, and then Dallas Clark does this on fourth-and-ballgame.

It is not as if Manning was only able to lead two go-ahead drives in 12 games with opportunities for his teams. A "lost comeback" is a game where the quarterback brought his team from behind in the fourth quarter to a lead, but still went on to lose the game. There have been 38 of these in postseason history. Manning is the only quarterback to have two of them (against the 2007 Chargers and 2010 Jets), and naturally they were both one-and-done seasons.

Manning is also the only quarterback to lose two playoff games after his kicker missed a clutch field goal (tied or down by 1 to 3 points in fourth quarter or overtime). Mike Vanderjagt was the culprit both times, with kicks that were closer to the parking lot than the uprights. And yes, both resulted in a one-and-done ending for the Colts (in 2000 and 2005).

Manning is one of six quarterbacks in playoff history to lead a go-ahead touchdown drive in a tied game (for a 7-point lead), and still go on to lose the game. That was with the 2012 Broncos, his highest DVOA team that still went, you guessed it, one-and-done.

Manning is one of seven quarterbacks to lose a playoff game in overtime with his offense never getting a possession. That was the one-and-done for the 2008 Colts in San Diego after Mike Scifres had perhaps the greatest punting night ever, pinning the Colts inside the 10-yard line five times.

While Manning is the master of experiencing the improbable losses, his playoff career lacks the improbable wins other quarterbacks have enjoyed. Terry Bradshaw has the Immaculate Reception, Roger Staubach has the Hail Mary, Steve McNair has the Music City Miracle. Manning has no wins like that. Hell, he basically lost a win on a Joe Flacco Hail Mary and had to play those Music City Miracle Titans, a 13-3 wild-card team. No kicker did a Billy Cundiff (Tom Brady) or Scott Norwood (Jeff Hostetler) for him. Manning's "idiot kicker" is in that same club. Tracy Porter did not pull a Lewis Billups and drop that fateful interception in the fourth quarter of a 7-point Super Bowl as Joe Montana's luck would have it. Manning's defense never snatched victory from the jaws of defeat such as Jim Plunkett's Raiders did on Red-Right 88, or when The Butler Did It at the 1-yard line (Brady, again) or Sterling Moore's play on Lee Evans (Brady, a third time). Lest we forget, little brother Eli was able to lead two game-winning drives in overtime of NFC Championship Games without completing a single pass on either one. Thanks, Brett Favre (2007) and Kyle Williams (2011).

This is the part where you expect me to talk about how Manning did get lucky in the 2006 AFC Championship Game, or else he may not have a ring to this day. But that will have to wait until next time. For now, just soak in the fact that circumstances out of Manning's control -- luck, if you will -- are what have kept him at .500 in the playoffs. No one is saying his record should be excellent, but he definitely played winning football in the postseason more often than not.

In Part II tomorrow, we will look at how even Manning's good playoff luck had a poor distribution that did not help his two best teams advance. The meat of the piece will focus on how changing one play not involving the quarterback can drastically alter his playoff fortune. Not only will we look at Manning, but we'll also go through Tom Brady, Aaron Rodgers, Ben Roethlisberger and Russell Wilson, as well as issue last words on Manning's playoff career as he heads into what is likely his final game.

Comments

92 comments, Last at 31 Jan 2017, 8:47am

1 Re: Solving the Peyton Playoff Puzzle: Part I

Will refa thsi later. Before raeding will say Manning better in playoffs than average bloke think. Has sometimes had teammates stink or opponents just play erally good. It is not like olsen times when Sammy Baugh played two ways and a passer oudcl really take blame for team losing. If manning does his job but Cotls defense or Broncos defense or drink kicker or dumb safety or deer in headlights coach lays an egg than Manning loses and people can ssyd, "Oh. Manning lost a play-off game again. He is a choker." in the end, it can sometimes be unfair

2 Re: Solving the Peyton Playoff Puzzle: Part I

2/12 on game winning drive opportunities.

I agree small sample size, but Manning isn't devoid of blame for the Colts losses in the post-season.

Manning is one of the best quarterbacks of all time (Him, Brady and Marino are the three best, in no particular order), and he isn't a choker, but lets stop pretending that Manning is blameless for his playoff losses.

Also 13-13 against playoff calibre teams is not a bad record. It's harder to win games in the playoffs because you're facing opponents who had to be good enough to make the playoffs.

3 Re: Solving the Peyton Playoff Puzzle: Part I

QBR is godawful. Please stop using it. The best that can be said about it is that it doesn't always screw things up. But sometimes it does, and it does very badly when it does. And that leaves us needing to verify that QBR isn't screwing things up by referring to some other statistical measure.

And since QBR isn't a statistic, per se, so much as the end result of some analysis that includes a subjective component, there's no way to say exactly what it's supposed to mean. That's what happens when you throw a dozen different statistics together and use a(n intentionally obscure) transformation to produce a number. That's the opposite of what professional statisticians do. I know DVOA does the same thing to an extent, but DVOA at least hasn't had eye-bleeding results like telling everybody that Brandon Weeden had a better game than Tom Brady when they went head-to-head in Dallas.

13 Re: Solving the Peyton Playoff Puzzle: Part I

Weeden 3 sacks and an interception is arguably worse then 5 sacks if you're looking at it from that angle (especially given Dallas has a far, far better offensive line.).

And Weeden averaged 4.8 yards per attempt, while Brady averaged 10.2. Not sure what Weeden's air yards were per attempt, but for him to have beaten Brady, he'd have need 75% of his yards to be before the catch, which sounds unlikely.

Twice the net yards per attempt (even after sacks) and fewer interceptions. I don't care about the context, if you do that, you played better then your opponent, no ifs ands or buts.

Now if the stat difference is smaller, then we can start looking at the tape and adjusting for talent, but there's no amount of backpedelling that can explain that gap.

I also don't like how some people treat YAC as if the quarterback has nothing to do with it. Any coach and quarterback will tell you that hitting a receiver in stride can be the difference as to whether a receiver makes yards after the catch.

17 Re: Solving the Peyton Playoff Puzzle: Part I

YAC is like many a very tricky stat, and can work for or against the player. I like what Cian Fahey is doing right now with what he calls Simple YAC which is YAC where the ball travels less than 2 yards in the air past the line of scrimmage. I think we can agree that if the ball does not even go 2 yards that a majority of the work goes to the receiver. I also like % of yards in air. Again not foolproof but it does show a lot and often is reflected in completion %, see Winston vs. Mariota.

30 Re: Solving the Peyton Playoff Puzzle: Part I

The only study I've seen on assigning YAC was a terribly flawed one. It ran a linear regression between YAC and compl% and concluded because the relationship was not statistically significant, accuracy of the passer had no relation to YAC.

Not taking into account scheme, variance, opponent adjustments, air yards or anything else is a bad way of conducting a study in a sport as complicated as football.

A better study would convince me that I'm wrong.
Like what's more predictive of future YPA? Air yards per attempt or career YPA?

The main factor in why DIPS became popular in sabermetrics was not because people proved the fielding effected pitching stats, everyone already knew that. The kicker was the revelation that ignoring plays effected by fielders was better at predicting EPA. And even then, sabermetrics has since done a u-turn and once again included fielding influenced plays, though weighting them less.

43 Re: Solving the Peyton Playoff Puzzle: Part I

The one I read was by brian burke.

I agree, los can be biased and inconsistent via omitted variables. That said, it does provide a general picture about things. When you see almost no correlation between the qb and yac, it should raise a red flag. I won't call it definitive, but its enough to make me question my priors that qb play is the driving factor.

Then again, we can see this first hand sometimes too. The pats yac numbers fell off a cliff when their receivers got hurt. We saw Brady fall hugely when gronk was hurt in 2013. It may sound great to say its the qb that accounts for the yac - and even the yac statistics don't entirely dismiss this - they just found that most of the yac is accounted for by receivers. That seems to be consistent when we look at qbs across various schemes or receivers across multiple teams. How else do we explain people Like cassell, Mccown, Flynn, and others?

49 Re: Solving the Peyton Playoff Puzzle: Part I

I can't find YAC per game data for quarterbacks. Can you show a citation showing that Brady's YAC dropped more then his YPA and completion% (big surprise, a QB is statistically worse when he's playing with worse players. I thought we knew that after Brady got Welker and Moss in 2007). I know Brady's yards after catch % was near identical in 2014 and 2015, meaning missing Edelman for half the season didn't do much.

Meanwhile on the receivers side, Dez Bryant had a career YAC of 5.2 when Tony Romo went down earlier this season. Since then his YAC has been 4.1, 0.8 less then any season he had with Romo. That's significant drop that seems to indicate that YAC is controlled by the QB.

But both of these are anecdotes and inconclusive.

But it certainly does look like tons of variables go into YAC, scheme, receivers, offensive line, quarterback, defense, variance all play a factor in a teams YAC. Just because two of those variables don't correlate at a stastically significant level doesn't mean QBs don't factor into it.

As for Cassel. he was statistically better with 2010 Chiefs then 2008 Patriots, so he was able to replicate his success. And yes, if you're arguing the 2008 Patriots and 2010 Chiefs had far better non-QB offensive players then the average team, I agree. Though the sheer gap in the Patriots passing stats in 2007/2009/2010 (1st, 6th and 7th all time by DYAR) with their 2008 (18th in DYAR/20th in DVOA for the season) campaign should show how large the gap is between Cassel and Brady. Though we're comparing a hall of fame quarterback (who I think is one of the three best of all time) to a near replacement level one, so the massive gap shouldn't be a surprise.

Meanwhile Flynn had one big game, that's extremely low sample size and high variance.

Not sure what either have to do with the assigning YAC debate.

Now again. If the numbers show that including YAC is less predictive then air yards per attempt (which I haven't seen a single study showing), then you have an argument that YAC is mostly outside a quarterbacks control.

But trying to argue that certain quarterback passing stats don't correlate with YAC is irrelevant to the debate and why the Brian Burke study fell flat in my view.

50 Re: Solving the Peyton Playoff Puzzle: Part I

There was a study by one of chase stuart's guest writers. Also Burke's looked at across year studies rather than strictly small sample size qb changes.

The argument I'm making isn't one vs the other, its like you said - a lot of variables going into it. But naturally, we want to ask what the driving force is. It doesn't seem to be qb play.

I would submit another piece of evidence. The ypa and anya numbers have all crept up over time. The answer to why is entirely in the short passing game. Were this a qb specific thing, I don't think we'd see it across all the teams; but we are. Every team is going more pass centric. I looked over the data and this is this year set the lowest yards per depth of any year in the last 9 years - a trend continuing from last year and on. Meanwhile, the anya hasn't dropped an inch.

This tells me defenses still have not figured out how to stop the short passing game.

52 Re: Solving the Peyton Playoff Puzzle: Part I

You know what's less predictive and less statistically significant then a QBs YAC? A QBs interception rate. Let's completely ignore that as well.

Given that compl% and sack rate are the only QB stats that are statistically significant. Lets focus 100% of our attention on those.

Can you find me that study done by chase stuart's guest writer? I can't seem to find it.

And actually YPA and NYPA haven't changed all that much over time. The average was 6.0 for the 60s, 5.9 for the 80s, 5.9 for the 90s, 6.0 for the 2000s and 6.3 since 2010. The 70s and their 5.6 was the deadball era but outside of that YPA has hovered around 6.

The driving factor behin changed is passer rating has been the increase in compl%, TD% and massive decrease in int%.

And there's a massive variance in how much each team does short passes/long passes and league tendencies as a whole.

And how the hell does a league increasing it's reliance on short passing prove that QBs have little control over YAC. In fact given how much this era has been dominated by a small handful of QBs, despite the leagues pushes towards parity and the league being salary capped it shows that massive differences a QB has in the short passing game.

33 Re: Solving the Peyton Playoff Puzzle: Part I

Pre snap adjustments are almost impossible to account for. What can actually be measured is the execution after the snap, and if a QB throws a screen then the YAC is on the blocking and receiver not the QB. Executing a screen pass is something that is simply expected of any NFL QB and is therefore not a rewardable play. Even if the QB does change the play the execution has very little to do with the QB making what is a very simple pass.

35 Re: Solving the Peyton Playoff Puzzle: Part I

Agree that pre snap adjustments are hard to quantify or account for but who is to say that it isn't a rewardable play? Screen plays should be made by any NFL qb and from a physical standpoint unremarkable but knowing when to execute it and audibling into it requires skill and if we are measuring qb skill and NOT just physical skill (simple pass) it should be part of the equation. Screen passes and short passes require accuracy and very good execution in tight spaces and obviously skill is involved. Some even argue that completing long passes are more on the receiver adjusting than an NFL caliber qb making it. I'm not arguing that short passes are more difficult than bombs, they are just different and requires different skill sets and if we are measuring performance they should all accounted for.

38 Re: Solving the Peyton Playoff Puzzle: Part I

Which is exactly why we can't ignore the QBs contribution to YAC.

A QB whose contribution is more about reading the defense then throwing downfield will improve his teams offensive stats. We should therefore give the quarterback some credit for when he improves the players around him.

On the other side you have deep threat receivers who contribute far more to the deep passing game then their teams quarterback. Brady has never been able to throw deep unless Randy Moss is on the receiving end and with out Julio Jones the Falcons would have no deep passing whatsoever.

Air Yards is as much about the receiving corps being able to get open downfield and the offensive line being able to hold off pressure for the play to develop as it is about the quarterback.

Yards after the catch is as much about the quarterback being able to read the defense and hit the receiver in stride as it is about the receiver avoiding tackles.

Assigning most or all of air yards to the quarterback and most or all of YAC to the receiver is asinine.

48 Re: Solving the Peyton Playoff Puzzle: Part I

I think I am over complicating this and will go back to my original statement and that is YAC is tricky. I agree with you that placing all of the praise on one or the other is wrong. In a game as complex as football there is no simple answer and it comes down to the individual play as to who did what.

An example for what I mean when I talk about credit is the much talked about PFF grade for Rodgers in his 5 TD game against the Chiefs. Should Rodgers get credit on passes where Cobb was hit at the line of scrimmage or had to work to make defenders miss? Obviously it's not a negative because it was a good pass and complete, but it is also a pass that every single NFL QB should be expected to make. Personally I agree with PFF and think giving it no grade is the best option. There was nothing special about the pass and that Cobb did the work either by avoiding or going through contact.

The reason I brought up % of air yards is that it can be used in some cases to explain abnormal completion % or passer ratings. A perfect example is 2014 RG3 who completed 69% of his passes and had a passer rating of 87. That's not an amazing passer rating but it's not indicative of his play either. Then you see that he 67% of his yards were from YAC. That's an insane amount and the highest by far I have ever seen from someone with any reasonable amount of snaps.

A side note that may only interest me is that from my admittedly limited research it seems that the top QB's rarely have more than 50% of their yards come via YAC. Not really a predictor of anything just something interesting I noticed.

37 Re: Solving the Peyton Playoff Puzzle: Part I

I know I'm not directly addressing your points, but have you seen some of the data points that go into the QBR calculation? I'm pretty sure QBR is highly correlated with EPA/P and you can see the actual EPA breakdowns here (http://espn.go.com/nfl/qbr/_/type/player-week/week/5) and here (http://espn.go.com/nfl/qbr/_/type/player-week/stats/expanded/week/5).

Weeden had .002 EPA/P (0.1 EPA over 51 plays):
-0.2 Pass EPA
0.9 Run EPA
-1.1 Sack EPA
0.5 Penalty EPA

Brady had -.007 EPA/P (-0.3 EPA over 41 plays):
3.8 Pass EPA
-0.1 Run EPA
-3.8 Sack/Fumble EPA
-0.1 Penalty EPA

What do you think?

39 Re: Solving the Peyton Playoff Puzzle: Part I

Multiple problems with EPA:
-EPA on fumbles is assigned based solely on who recovered the football and where he recovered. A strip sack recovered by the offense at the line of scrimmage receives the same EPA as an incomplete pass. The exact same play recovered by the defense is penalized more then an interception, even though the interception was (baring a tipped pass) the quarterbacks fault while the fumble recovery was variance and luck.

Turnovers, which are high variance plays are unpredictive and less relevant when evaluating play compared to the performance as a whole. But because they strongly determine the outcome of the game, a model like EPA or WPA which looks at how the result of the play effected the outcome of the game places way too much weight on them.

QBR suffers from the same problem.

Penalties:
what penalties did the quarterback have? Was it punishing something like false start or delay of game? They're not very predictive and largely outsdie of the Quarterbacks control.

On the other side, QBR rewards QBs for drawing offsides, but there's no statistical evidence that a QB actually does cause a defense to commit this penalty.

Penalties that football outsiders has found to be statistically predictive of QB play, like DPI, are included in DYAR.

So yes. I think Brady's negative EPA for that game is reflective of problems in the EPA/QBR model, on not reflective of a poor game on his part.

Now I could be mistaken, Cowboys-Patriots wasn't a game that I watched, but I tend to trust NYPA and DYAR more then EPA when evaluating quarterbacks and I think this game might serve as an example as too why.

41 Re: Solving the Peyton Playoff Puzzle: Part I

Their system is still extremely flawed, while better then previous seasons.

They judge the return as luck, but not the recovery.

There are three parts too a fumble:
1. Player loses control of ball: this is either due to bad play by the ball carrier to lose control or good play by the defender to force the fumble. While due to the rarity and high variance nature of turnovers a fumble should penalize a players individual stats less then they effect the final score (but should still be big negatives), this part is probably more skill then luck.

2. Team recovers the ball: No team in NFL history has been able to sustain a high recovery rate. The team with the fortune of landing on the ball got lucky. And this is the difference between QBR and DVOA. DVOA accepts that this part of the game is random, QBR rewards the QB when his offenses lands on the ball and penalizes the QB when the defense does. That's not really fair on he QB.

3. The return: Maybe the player was able to get up and go for the TD. Maybe because he was on the ground the play ended there. QBR is finally acknowledging that the QB has no control over where the returner ends up.

So QBR still penalizes turnovers more then it should, and DVOA also penalizes turnovers slightly more then they should (though nowhere near to the same extent). The problem is turnovers greatly effect the outcome of a game, however due to their rarity and high variance nature they're very unpredictable and therefore I think when building a predictive model, which is what DVOA is, they should weight turnovers less. Partially because their rarity means you're focusing too much on a smaller sample of plays, and partially because their high variance nature means there's a lot of randomness that goes into them and a good deal of them are not reflective on the talent of the player committing the turnover.

I should also note I didn't know about most of these updates to QBR (I don't actively read ESPN or their sad attempts to defend QBR as if it's a legitimate statistic), maybe QBR will be a relevant statistical model in a few years. Probably not given they've already decided to exclude YAC which makes it totally understandable that Brady does worse in QBR then in both the common sense eye-test and in superior stats such as DVOA/DYAR/TAY.

44 Re: Solving the Peyton Playoff Puzzle: Part I

How do you know QBR penalizes turnovers more than they should without knowing the weights they use?

I also raise a broader question - why the hell do people hate qbr so much? Strangely, no one bothers to question dvoa and how they weight statistics. OR Anya with their 30*tds, -45*ints.

Or what about pff and their grades? QBR has become a convenient whipping boy from the very people who love and defend DVOA. Again, you may hate the Yac subtraction, but that's based on statistics, not someone's personal opinion. You may hate EPA, but again, that is based on statistics.

If you're going to hate qbr, at least tell me why its bad vs dvoa or dyar? Other than personal opinion, why people hate it but love other statistics that attempt similar types of context correction?

51 Re: Solving the Peyton Playoff Puzzle: Part I

QBR says it evaluates plays based on EPA. EPA values turnovers too much so therefore a model based on EPA will.

ANYA does 20 for TDs, which was the result of extensive research, summarized here (http://www.pro-football-reference.com/blog/?p=633). They do 45 for interceptions because that's what the book 'The Hidden Game of Football' found. It admits the number is probably out of date (book is from 1986), but they haven't been able to develop a good new model.

I've complained about PFF grades frequently in the past, including earlier tonight on a different thread. Outside of offensive line grades, I give PFF grade little focus (pardon the pun). But at least as a whole PFF grades do mostly make sense, generally my view of the top quarterbacks/receivers does lineup relatively well with their PFF grades (DYAR does slightly better though). Meanwhile QBR having Brady 11th and Carr 26th is insane, and Smith and Rodgers didn't belong in top 10 (Smith being 8 in QBR is another reason I'm convinced they overvalue turnovers. He's good at avoiding interceptions, but is far below average at moving the ball downfield, and that's more important).

Of the advance stats, DVOA/DYAR is the most objective and, from what I can tell, the most predictive. That's why I like it the best. If I'm wrong in that regard, maybe I'll give the more predictive stat more thought.

And maybe of ESPN did a better job explaining what QBR is and how it's calculated I'd give it more thought. I don't like a lot of what it's vague about and a ton of its results make no sense.

It's not like I woke up one day and said lets hate QBR, I dislike QBR because it too often doesn't make sense.

65 Re: Solving the Peyton Playoff Puzzle: Part I

Apparently the transition to QBR from traditional passer rating is that the QB gets blamed for sacks and loses credit for YAC.

It's a lot easier for a receiver to get YAC when the ball is thrown in the right place. And I guess it's Brady's fault that Solder got hurt that day and the resulting line couldn't contain Greg Hardy.

73 Re: Solving the Peyton Playoff Puzzle: Part I

I'd argue the offensive line hurts YPA more then sack rate.

QBs matter more then offensive lines when it comes to sacks, Peyton Manning has almost never gotten sacked because of his ability to get the ball out quickly, meanwhile Rodgers attempts to extend the play has often gotten him sacked despite some of the great offensive lines he's played behind.

I hate how many people evaluate offenive lines by sack rate because getting the ball out in time is the quarterbacks responsibility, and even when his offenisve line is terrible, he can always make quick reads and throws.

Yes his traditional passing stats will get hurt if he hurry's up his throws (traditional passer rating is a terrible, terrible stat, worse then QBR simply because unlike QBR which people like to laugh at, people think passer rating has merit), but he can still avoid a sack.

66 Re: Solving the Peyton Playoff Puzzle: Part I

There is value in statistics with simplicity. For example: completion percentage, TDs, TD %, INTs, INT%, yardage, yards per catch, yards per attempt.

The problems arise when people think that having several statistics is too problematic, and there's a need to distill disparate statistics into one all-purposeful number.

I would prefer to have a handful of stats for QB play: cumulative stats like yardage, attempts, completions, TDs, and INTs, as well as a few rate stats like yards per attempt (which really is the most useful).

6 Re: Solving the Peyton Playoff Puzzle: Part I

Great article. As a long-time Colts fan, all of the Manning losses are seared into my mind. The Colts found a new way to lose in the playoffs every year, be dropped passes in the last seconds or letting Lamar Smith shred the defense or having Billy Volek having the game of his life. All of them so painful. Although, the most painful of all losses for me was the 2005 game against Pitt. First, Rothlisberger makes the game saving tackle after the Bettis fumble and then after Manning drives down the field, the idiot kicker shanks it anyway. Still have nightmares.

7 Re: Solving the Peyton Playoff Puzzle: Part I

I just got mad about 2012 all over again. There's going to be a 30 for 30 on it some day. What if Mike Adams had gotten his feet in on 2nd down and intercepted Flacco at the 20 in OT? What if Denver had just stopped Baltimore on the following 3rd and 13 at the Ravens' 3 yd line instead of Jim Leonhard getting lost and losing coverage on Pitta? Dammit.

12 Re: Solving the Peyton Playoff Puzzle: Part I

Jake Delhomme is the perfect symbol of playoff small sample sizes working both ways.

His first playoff starts are really good, coming very close to a Super Bowl win, then his final two are as deep as you can fall into an abyss, throwing for 2 TD and 8 picks. Sanchez has never been more than a replacement level player so barring an injury he will never sniff the playoffs again, Eli Manning, meanwhile, as mentioned by Scott, will be forever rewarded because of his immaculate 8-3 record, at this point of his career, he won't make it for 5 seasons and lose every single game to bring it to .500.

Even for players who reached over 20 playoff starts the sample size pales compared to the regular season. Brady has 31 playoff starts to 223 in the regular season, his greatness doesn't come from lucky breaks that allowed him to boost his playoff record, since it is actually very frontloaded because of a 10-0 start in a team whose identity was on defense. Since 2007, when he started to be in consideration to be one of the best QBs in football, rather than just a good starter, his numbers are quite similar to Manning's, playoff record itself included, as you would expect. RIng count? Also equal

Adam Steele's guest post on football perspective today is about it, on how the infamous narratives are built in the start of players' careers, despite how similar their performances look.

Manning winning his second Super Bowl will probably be interpreted as "the defense carrying him" like how people view 2006, but it is actually the nail in the coffin on championships as a measurement of individual greatness in football, since one of the best to play the game might win when his decline is evident, hence the fact that his playoff DVOA has only dropped but the Broncos defense has been able to get it done so far.

16 Re: Solving the Peyton Playoff Puzzle: Part I

Sorry to argue semantics, but Manning has 0 rings since 2007. That may change this Sunday.

That I do agree with your point. I hate all the Eli is a great playoff quarterback people. He'd have a much less impressive record if he had made the playoffs more then once in the last seven years.

Whenever I debate Peyton is a choker types, if I want to use a simple argument, I just tell them to go watch the 2006 AFC Championship game.

You can't play like that against the Patriots in the conference championship and be a choker.

69 Re: Solving the Peyton Playoff Puzzle: Part I

Yet it took until 2006 for Peyton to have that game.

Peyton earned the label of "choker" with his early playoff appearances. In particular, the Colts lost all three of those games. In the loss on 2000 to the Titans, Manning was 19 of 42 for 227 yards. That's a 45% completion percentage!

He was slightly better the following year in a loss to the Dolphins. His third playoff game was two years later, and it was a dreadful loss to the Jets where the Broncos lost and Peyton was 14 for 31 for 137 yards with 2 picks.

He didn't have a Manning-level game until the following year, when he had two very good games against the Broncos and the Chiefs. And then he had a dreadful AFCCG vs the Pats where he threw four picks. By this point he was 27 and the narrative was set.

Since then it's been a regular feature at FO for somebody to rehabilitate Manning's image. Why does this need to happen so often? Well it seems like every other year in the playoffs his team is 1-and-done, whether it be the Colts or the Broncos. There is no similar need to rehabilitate Brady or Flacco or Roethlisberger. And yes, expectations for Peyton have always been the highest, but he earned that with his regular season MVPs.

Articles like this one by Scott bury the reasons people initially viewed Peyton Manning as a choker. Has Peyton had good playoff runs? Sure. He's gotten to four Super Bowls, which is as many as Terry Bradshaw and Joe Montana and Jim Kelly and more than anybody else not named Brady. And he deserves credit for that. But he's just as many disappointing playoff runs. Some were clearly not his fault, like the lost to the Ravens three years ago. But many of them featured poor play on his part. So be it.