ESPN Offers Some QBR Analysis on Rodgers, Brady, Newton
Some of those crazy single-game results from ESPN's QBR confuse our staff just as much as fans, so this article from Friday was hopefully a sign of analysis to come from ESPN. They explain some of the elements of QBR for the seasons Aaron Rodgers, Tom Brady and Cam Newton are having.
It is fair to say Rodgers is having his roughest season since 2008, but he's still fifth in QBR (69.4) in what has overall been a down year for quarterback play. Rodgers gets a good boost from his scrambling and penalties (pass interference) drawn. Plays like the touchdown scramble on third-and-long against the Lions are valued very highly in QBR. It makes sense to some degree as you could argue scrambles are the closest thing to a quarterback making a play that's independent of his 10 teammates. You rarely get good blocking on a scramble since it's such a spontaneous decision (most of the time at least), and bad blocking is often a reason for the scramble in the first place. All the yards are the quarterback's yards too, so there is no shared credit. These are also often third-down plays, so they are important. Rodgers has had to scramble more than usual this season with the cast around him.
Brady ranks 11th in QBR (64.5), which would be his third-lowest QBR in a full season since 2006. Brady's season was much stronger statistically before all of the receiver injuries, but even then it was not shining in QBR due to so much YAC from his receivers. I always figured that was the case and this article confirms it. They cite our YAC+ stat, which adjusts YAC against expected YAC based on where the ball was caught. Brady's receivers have produced a league-high 260 YAC+. So when Brady threw a 59-yard touchdown to Julian Edelman against Dallas that had 41 YAC after some broken tackles, Brady only gets 26 percent of the total EPA (plus-5.6) on that play. When you have -- or when Brady had -- hard to tackle players like Dion Lewis, Edelman and Rob Gronkowski catching the passes, it's easy to see why YAC+ brought Brady's QBR down.
ESPN's explanation for Cam Newton ranking 14th in QBR (62.7) did not pass the smell test as well for me, but then again, I cannot wrap my head around the "Cam for MVP!" talk either this season. Basically, Newton's defense has helped him in more ways than usually considered. Newton has 12 turnovers and the EPA hit on those plays has been lessened by the defense allowing half as many points as they were expected to in that field position. That includes the opponent missing three field goals after Newton turnovers. Still, we are talking about a dozen plays. It should be noted that Newton's QBR has been 82.1 since Week 9, which falls in line with the eye test that he has recently started playing much better.
Carson Palmer leads all quarterbacks with an 83.6 QBR for the entire season, and he has been my MVP choice since midseason. He may not stand a chance if the Panthers finish 16-0, but a stat like QBR is definitely picking up on the right things about his season, giving credit for all the deep throws and penalties drawn from that aggressiveness. DVOA also agrees here as Palmer currently ranks first (34.4%) and Newton is 17th (-0.3%).
Comments
94 comments, Last at 17 Dec 2015, 1:01pm
#42 by Jerrytron // Dec 14, 2015 - 1:44pm
Here's the short version.
Pretty much all QB's have 50% +/-3% of their yards through the air (air yards).
Here is a sample from 2013.
51.05% Peyton Manning
50.10% Drew Brees
49.22% Philip Rivers
48.56% Matthew Stafford
49.65% Andy Dalton
50.47% Tom Brady
48.62% Cam Newton
47.31% Andrew Luck
48.35% Ben Roethlisberger
51.93% Tony Romo
56.45% Ryan Tannehill
53.74% Russell Wilson
56.62% Carson Palmer
37.18% Matt Ryan
56.27% Colin Kaepernick
44.61% Alex Smith
48.74% Nick Foles
52.04% Joe Flacco
51.30% Robert Griffin III
59.48% Eli Manning
57.35% Geno Smith
60.93% Jay Cutler
64.53% Mike Glennon
42.52% Chad Henne
52.20% Ryan Fitzpatrick
47.83% Aaron Rodgers
51.94% Josh McCown
51.67% EJ Manuel
58.31% Matt Schaub
47.39% Terrelle Pryor
44.46% Sam Bradford
46.95% Jason Campbell
I compiled this data as part of trying to create a new fantasy scoring system. I abandoned the idea of valuing air yards differently than YAC after seeing these results.
There is no real way to look at air yards versus YAC without going into a play-by-play breakdown and injecting subjective opinion and I'd lay money that it varies a lot from game to game.
On this list Rivers, Stafford and Dalton have a combined 134 more yards of YAC versus air yards. Manning, Brees and Brady have a combined 166 more air yards than YAC (115 is just Manning, by the way).
The top six QBs in my little alternate fantasy scoring had 32 more air yards than YAC out of a combined 28,406 passing yards. I think my cutoff for QBs was 100 attempts so qualifying QBs had 64,062 air yards, 59,988 YAC for a total of 51.64% passing yards being air yards in 2013.
That means if you throw for 4000 yards I expect ~2066 air yards and ~1934 YAC.
#5 by jacobk // Dec 12, 2015 - 11:27pm
Is it in your guys' contract with ESPN that you have to pretend QBR is a real statistic? I only ever see ESPN-affiliated people reference it, it's kind of sad. I look forward to the day that ESPN gives up on it and we can all just pretend it never happened.
#6 by theslothook // Dec 12, 2015 - 11:35pm
Reading some of the comments above - I am surprised so many people are against qbr. In fact, Qbr attempts to do something FO stats admittedly do not - separate qb play from the rest of the roster.
As far as I know, only pff and qbr attempt to do this, but unlike pff, qbr isn't based on the subjective decisions of individual charters.
It takes into account a lot of the factors people would love to take into, namely breakdown of YAC, dropped ints/tds, game situations, etc etc.
Is it flawed? Absolutely, but name me another stat that is supposed to be better. For god sakes, people still point to passer rating, even on THIS SITE.
#11 by Duff Soviet Union // Dec 13, 2015 - 1:53am
Count me in as another vote of support of QBR.
I'm actually curious as to what everyone's problem with it is, other than the fact that it was invented by ESPN and has some strange results from time to time, which is true for any rating system worth it's salt
#12 by Insancipitory // Dec 13, 2015 - 2:05am
I think it's fine. I think it just needs to be consistent to be useful. But a lot of people don't like the blackbox nature of it. A lot of people loathed the original "clutchness" factor which spit out completely ridiculous results early on (often associated with Tebow). First impressions are what they are. And not necessarily unreasonably so.
#18 by Insancipitory // Dec 13, 2015 - 7:21pm
I'm comfortable with the proprietary and iterative nature of it. QBR is just the 0 to 100 temperature value for the heatmap of interesting passer variables. I don't have to know what metals my thermocouples are made of, or how the voltage varies with temperature, or how diligently or expertly this was determined. I just have to know if it's close, especially not ridiculous. If I turn on my car, and it says it's 200 degrees, that's ridiculous and something is broken. Much in the same way, if Tebow flailing for 55 minutes then miraculously finding and not fumbling away the Lost Wallet of Opportunity on his way to church grades out much better than a competent Rodger's effort, it might be time to pivot the tables.
Putting in work and getting out garbage is just part of the process. But it sure doesn't make for a reliable first impression. Is QBR better? I can't answer that. At best I can say 0 to 100 is more aesthetically pleasing that 0 to 158.3. It's got a little better contrast than passer rating as well. The top 10 passers are separated by around 20 points of QBR but just 10 of passer rating. Of particular note, the ordering of QBs is very different. The contributions of Carson Palmer and Ben Roethlisberger definitely seem to be more faithfully represented by QBR than passer rating.
As Scott references below, I think one of the things that contributes to people embracing DVOA is the disposition of the site and writers. DVOA is sold as a work in progress. It's a mountain we can watch people discover as they climb it. It's not a glossy product of the disinterested Disney corporation. When people have questions they take to the threads and very often get well thought out and occasionally researched answers. Even with ESPN buying Advanced Football Analytics, I have trouble even imaging taking the time to ask them anything. Such are my expectations. Maybe QBR is a zlionsfan template away from widespread acceptance. I don't know precisely how much better it is than alternatives, but it seems that QBRs form necessarily follows from its function.
#23 by Thomas_beardown // Dec 14, 2015 - 1:19am
Comparing to old qb rating should not be the goal. How does QBR do against a modern QB stat like ANY/A?
Also, QBR has completely obscure inputs because it relies on scouts to grade each play. You have no idea what that grade is, how it's done, how uniform it is, etc.
#16 by theslothook // Dec 13, 2015 - 1:29pm
They did tell us what the inputs are. We don't know how they are weighted, so yes, there is a black box element to it. I accept that as a legitimate criticism, but DVOA is also blackbox and we trust it implicitly as a measure of team value because it is the best stat to assess teams out there.
#20 by Raiderjoe // Dec 13, 2015 - 9:40pm
Am regular poster sat this site and don't care about dvoa and sometimes post in dvoa threads. Not even sure what it stands for.
I now try to figure it out
Defvesive value over average
Defensive valuation overt analysis
Descrptive
I give up
#27 by theslothook // Dec 14, 2015 - 1:45am
In the end, charting is the only way you can possibly do the separation of individual players from team result. ANYA does some of that, but its still going to be inferior. THere are layers to how subjective the charting gets. Things like pressures/dropped passes / ints are one level of subjectivity. Deciding whether a throw was accurate or a receiver ran the correct route is a another layer of subjectivity.
Given the inputs they've mentioned and having charted games myself, I think the subjectivity they're relying on doesn't seem too difficult. So I trust it. Yes, its a black box but I maintain no more or less so than DVOA.
#34 by Lyford // Dec 14, 2015 - 8:50am
"m actually curious as to what everyone's problem with it is, other than the fact that it was invented by ESPN and has some strange results from time to time, which is true for any rating system worth it's salt"
I am skeptical of any QB rating which emphasizes YAC. It's certainly true that there are some YAC that are due to individual efforts from receivers, but the more I watch, the more convinced I am that most YAC results from quarterbacks getting the ball to the right receiver, in the right place, at the right time. A quarterback who is playing really well frequently makes it look as if the position is easy. He makes what look like easy throws and hits receivers who are wide open. But when that happens, it's almost always a result of the QB doing his job really well - setting up protections, reading defenses, getting the team into a good play at the line, understanding what the defense is doing, maybe looking the safeties off to open up a section of the field, getting the ball off to the right place at the right time - all of which are vitally important aspects of the position, and none of which he gets credit for if you give all of the YAC credit to the receiver.
I haven't followed any of the QBR debate closely, but that's one thing that I'm aware of that I think is a real problem for it.
#66 by theslothook // Dec 15, 2015 - 2:48pm
So you are saying QBR will lead you to worse conclusions of how a qb peformed than a stat that makes no attempt to do so? I disagree on that. QBR rankings generally jive with what you'd expect. Short of some strange results week to week(which is true for DVOA and DYARR both as commenters love to point out from time to time); its seems fine.
#22 by Scott Kacsmar // Dec 14, 2015 - 1:08am
Wow, I was hoping for another Raiderjoe typo there, but Bortles really did have a 3.8 QBR today. Makes sense for the first half, but the TD passes alone in the second half, even if you give most of the credit to Hurns for the YAC on the one, should at least get him into double digits. That's one they should explain in an article tomorrow.
#26 by Duff Soviet Union // Dec 14, 2015 - 1:30am
I thought Bortles was total rubbish regardless of how many touchdowns he threw for or how many points his team scored. 3.8 is maybe a bit extreme, but I don't think it's really that out of line with how he played.
#29 by ramirez // Dec 14, 2015 - 2:42am
I think these comments about Bortles illustrate what people don't like about QBR. Giving a guy a score of 3.8 out of 100 indicates that he really hurt his team, but I don't think Bortles did that today. By the Real QB rating formula, which is traditional passer rating adjusted for sacks, fumbles lost, and rushing stats, Bortles got a rating of 88.0. That's very good, though not spectacular. To give him a score of 3.8/100 suggests a very different type of performance, that I believe is not accurate. When he accounted for 4 touchdowns and his team scored 51 points, it's hard to believe Bortles was that bad.
Real QB rating uses inputs that are definitive statistics. QBR, because it's based on EPA and video analysis, contains a subjective element that goes above and beyond traditional stats. That would be OK, except it produces too many incomprehensible results game-to-game. It gave Wheeden a higher score than Brady in the Cowboys-Patriots game, and gave Rodgers a higher score than Brady in week 8. When you look at the results of those games, the box score, or watch the video again, there's nothing that suggests to me that those ratings are accurate. I think ESPN is trying hard, but there's something wrong when you keep getting examples like these, and pedestrian performances by Tebow and Charlie Batch rank among the best games in QBR history.
#30 by Scott Kacsmar // Dec 14, 2015 - 3:04am
"Giving a guy a score of 3.8 out of 100 indicates that he really hurt his team, but I don't think Bortles did that today."
I strongly disagree there. First, the 51 points are very misleading. They had two return TDs and a drive that started at the 1-yard line where he needed two runs to run in a TD with his team up 44-16 at the two-minute warning. I would apply some garbage time factor (can still be garbage time for the leading team) on that score to give him less credit than usual. So that's a good 20 points where he really doesn't deserve the credit. So it was more like a 31-point day.
Then you consider that he couldn't handle a snap that was recovered in the end zone for a TD, and that's a net 24-point day. Then you look at Allen Hurns breaking a tackle and gaining 60 YAC on his 80-yard TD, and that's a play where Bortles should be getting about 25% of the EPA credit. So his actual contribution on the scoreboard was definitely nowhere in the neighborhood of 51 points or +35 differential.
Just watch Bortles' first play of the day. The Colts had a diving INT attempt, and the ball bounced to Clay Harbor for a 17-yard gain. For me, if the QB's pass has to travel through the hands of a defender to reach his receiver, he gets credited with an incompletion on that play. I won't go INT, but he's not getting any credit for a completion from me. That's why the Charlie Batch QBR in 2010 Tampa Bay game is ridiculous, because his long TD to Wallace went off a Buccaneers' hands (Talib?) first. So we know they're not doing QBR that way, but that's how I would handle plays like that. I think charting will give Bortles 3 or 4 dropped INTs in the first half alone here. I only saw 2 receiver drops, and saw more plays than that with broken tackles and 10+ YAC.
Bortles had 3 sacks and he fumbled on two of them, losing one that took his team out of scoring range and put Indy into scoring range. The one he didn't lose should be treated harshly as we know here that fumble recoveries should be treated as random events. The only sack Bortles didn't fumble on came on a third-and-4, so his sack EPA (-4.8) is understandably horrible. His run EPA (-3.5) takes a massive hit from the botched snap fumble-six, so that makes sense too to me.
I think we may have just found the worst game of the last decade for a QB who accounted for 4 TDs. That might have only been pointed out by QBR, which I think proves there is merit to what they're doing with the stat, even if a number like 3.8 still feels so low.
#32 by ramirez // Dec 14, 2015 - 4:07am
First, I will admit that I didn't see the Jags game, thanks to Goodell's monopolistic TV policies. But let's look at Bortles' day in a more simplified form. Bortles advanced the ball 20 times in 37 attempts, for 243 yards with 4 tds and 2 turnovers. When you look at it like that, Bortles didn't have a weak peformance, far from it. So which of these two methods actually tells you more about the value he has produced for his team? If Bortles throws an 80 yard TD that has 60 yards of YAC, that counts just as much from the team's point of view as a 70 yard pass with 10 YAC. This is why I think it's silly to subtract points for players with a lot of YAC. If a QB can consistently gain yardage and TDs by finding open receivers who can run after the catch, he's helping his team.
One of the reasons why Brady got a low score against Dallas was because by halftime, NE had something like a 95% chance of victory according to the win probability model. So all of the damage Brady did in the 2nd half was almost meaningless. What was the insurmountable deficit the cowboys faced at halftime? 13-3. It's ridiculous to suggest that a team at home can't overcome a 10 point gap at halftime, but this is the sort of thing the EPA and WPA models do. Again, I'm not saying we should abandon metrics like QBR and EPA. But to present them as be-all end-all answers is stupid. Whatever you think of the passer rating and real rating formulas, to my knowledge, they don't produce baffling results like the Bortles rating, Wheeden over Brady, and Rodgers over Brady in week 8.
#33 by Scott Kacsmar // Dec 14, 2015 - 5:04am
But things like passer rating, real rating, ANYA aren't even trying to do the same thing as QBR, so it's almost pointless to compare the results. Years of research for thousands of plays goes into something like EPA, which is right in line with how DVOA works. We have an idea of what third-and-5 at the 50 yard line is worth based on historical baselines. The value of charting for a decade and using that in QBR is basically saying we can develop more baselines of what to expect. So not only are we looking at third-and-5 from the 50, but we can look at the difference between a pass that is thrown 1 yard versus a pass that is thrown 13 yards.
I have no idea if this is how QBR works, but let's say you had 2 QBs with identical stat lines: 20-of-27 for 250 yards, 2 TD. All 27 plays came in the exact same situations relative to down, distance, score and time left. The only difference between these games is how far the QB threw the ball on the seven incompletions. If QB A was missing passes under 5 yards and QB B was missing 7 passes thrown 10+ yards, then I would hope that QB B finishes the game with a higher QBR. How much higher? I couldn't say, but I would hope that gets recognized in the formula as throwing longer passes has a lower chance of success. Our REC +/- stat backs this up well.
The changes made to QBR this year make it much more of a QB performance metric than a results-driven metric. They are limiting the impact on what happens after the ball leaves the QB's hand, which I strongly believe is a worthwhile pursuit. Yeah, you'll take the result of an 80-yd TD for your team no matter how it comes, but in the sense that we don't treat a 10-yard pass on 3rd-and-15 as equal to a 10-yard pass on 3rd-and-10, why should we treat an 80-yard TD that's 10 air, 70 YAC as equal to one that's 50 air, 30 YAC when crediting the QB? We can do better than that thru charting.
Then on an interception, you now end up with an expected return based on where the pass was caught instead of getting hit with an 80-yard pick-six because the defender dipped two tackles and scored. That means QBR is going to have worse correlation with win% now, but don't we already have enough stats that do a decent job of looking at things that way? This stat is trying to tell us about the QB's contribution independent of those around him as best it can.
Brady's TD to Edelman in Dallas probably does lose further luster since it came with a 14-point lead in the fourth quarter with Dallas having a low WP. How low? That's up for debate. There is no one right WP formula, and what worked in 1999 is not going to be teh same in 2015. I seriously doubt NE was 95% WP at halftime with a 13-3 lead. That sounds like the crazy results you get from the PFR WP calculator that thinks comebacks are basically impossible. Maybe ESPN has changed their model with Burke's data. I know the FPI thing is new for them. But I would think under the old results, that 13-3 lead at halftime would not be 95% WP. The TD Brady threw to start the 3Q is another one where I would give him basically the credit for a 2-yard completion. He throws a pass in the flat to Dion Lewis, who does an incredible job to get past four defenders for a 10-yard TD. I still couldn't tell you why Weeden was higher that day, but I understand why Brady was well below the caliber of his traditional numbers.
I don't think the three of us here have the time or interest to do it, but I'm sure if we took every play and matched it up with YAR (QBR not adjusted for opponent) and things like WPA, drops, pressures, air yards and YAC, we could probably come pretty close to recreating ESPN's QBR results by game.
#40 by ramirez // Dec 14, 2015 - 12:07pm
I appreciate you giving me such detailed responses. I wasn't aware that QBR had been tweaked this season, but I still think it's producing results that if I had designed the formula, I would be uncomfortable with. You said that a QB who was missing throws further down the field would get a higher QBR. My question would be, why? How is is bringing value to the team to throw longer incompletions? This reminds me of one of the articles you wrote at your blog, where you claimed that one of the reasons Manning is better than Brady is because in Manning's losses, he has produced more WPA than Brady in HIS losses, and therefore Peyton has been more valuable. But even if that's true, how much impact can that have on Manning's career value, if the Colts and Broncos lost all of the games in question? How much value did Peyton create when his teams went 0-60 or whatever it was, in that sample?
I haven't seen any evidence that teams who rely more on air yards will win more games than teams that use more yac. If Brady and Alex Smith are winning games with yac, I see that as a good thing. This is why I've never bought into the hype over guys like Manning and Marino. Brady has won more games than Manning, and for most of his career has been just as good statistically. If you want to take points away from Brady because of yac, that's your choice, but I don't see how that's based in anything concrete. Marino is sometimes ranked ahead of Montana and Young, even though Montana had better rate stats, and Young's numbers in the 90s crush Marino's. It's like people give Marino bonus points for playing with worse defenses and rush attacks, despite the fact that when Marino had a D or a running game, he still didn't win. I also think it's noteworthy that Brady passed Marino in career TDs in about 700 fewer attempts, and is ahead of Marino in every era-adjusted + stat except sacks. I guess I'm just not impressed by gaudy career totals.
I'm getting off topic, so I'll leave it there. But those are some of the reasons why I'm skeptical.
#43 by Thomas_beardown // Dec 14, 2015 - 1:47pm
"Brady... is ahead of Marino in every era-adjusted + stat except sacks"
Brady's career ANY/A+ is 117 while pfr annoyingly doesn't give Marino's career, but he has only 2 years below 117 and he was when he was old and broken down.
#48 by ramirez // Dec 14, 2015 - 11:41pm
You can use the player season finder on pfr to get career + figures for retired players. You're right, Marino is at 119 ANY/A+, Brady 117. Brady's ahead in all the categories that don't include sacks or net yards, and they're tied in comp+ and ypa+. So it's closer than I thought. I bet if you included postseason stats, Brady would move ahead, as he's got better playoff numbers than Marino pretty much across the board.
If you take away the 3 weak seasons at the end of Marino's career, he improves to 122 ANY/A+. If you take away Brady's first 3 seasons, he improves to 120 ANY/A+. So it's close, and Brady generally has better passing stats, but including Marino's historic sack numbers puts the Dolphins QB ahead. Is that enough to overcome Brady's better career totals, which he's still compiling? I'm not sure. But I feel if you include postseason stats and achievements, it moves Brady ahead by a comfortable margin. I never agreed with the theory that Marino failed in the postseason because his teammates let him down, or his coaches were idiots.
#52 by ramirez // Dec 15, 2015 - 12:02am
I don't see what's controversial about saying that Marino didn't play real well in the postseason. I've heard a lot of Marino's fans make a litany of excuses for Marino's failure to win a Super Bowl, just as some of Manning's fans do the same to explain Peyton's poor postseason record. People will say that Marino/Manning played well in the playoffs, but were let down by their defense, or running game, or special teams, or coaches, or whatever. I just don't think that's a fair assessment.
#55 by theslothook // Dec 15, 2015 - 1:27am
My problem with it is...its always an after the fact comment. I hate to pick on tom Brady, but hes the most readily available example. Tom Brady maybe the greatest qb ever, but he's also a prime example of how a qb can play badly during the playoffs and still win and those moments never come back as a bullet point on his resume because the defense rose to the occasion. If the pats lose that sb last year, we may look at bradys two picks as examples of his failures. Brady playing lousy against a depleted denver squad in 2013 - its like it never happened. When he played badly against the ravens in 2011 - but won anyways, it doesn't come back to haunt him as a choker. And when he throws 3 picks in an afc champ game against an injured chargers team but wins anywaz - again, all is forgiven.
The point isn't to bash brady - hes had many great games too, its to point out how meaningless a single elimination game is at assessing credit. I don't do this just to pump up peyton(who is my favorite player), but as a deep football aficionado, I find that the playoffs mislead you far more than they inform you which players are great.
#61 by ramirez // Dec 15, 2015 - 9:53am
Who says Brady has played badly in the playoffs? You picked out several examples from his career that taken together, prove nothing. Brady put together two TD drives in the 4th quarter to win ths Super Bowl, Seattle gets a crazy catch from Kearse, and somehow you see this as Brady getting lucky. He threw two picks, but unlike Manning the year before, those picks did not give the other team great field position, and Brady made up for it with outstanding play later in the game. Brady had a 93.9 rating in the 2013 afc champ against Denver, and was working with a terrible receiving corps decimated by injury, the likes of which Peyton has never been forced to work with. He wasn't great, but the notion that Brady played poorly in that game is revisionist history.
I agree he played badly against Baltimore in 2011. I never said Brady has never had a bad playoff game. Peyton and Marino have also had playoff games where they were subpar, and still won. What's your point? Are you seriously suggesting that Brady has never been called a choker? Did you miss SB 42 and 18-1? And that was a game in which Brady had a good passer rating, and led a TD drive in the last 3 min to put his team ahead. Has your boy Manning ever done something like that in a SB? Contrast that to the reaction to Manning losing to Seattle, when we heard excuses for how his teammates let him down, and suddenly the Seattle D was the new version of the 85 Bears. I'm tired of the excuses.
I don't think postseason results over an entire career are meaningless. Go back and look at Marino's playoff career, and you'll see that in many games he was subpar, and his overall playoff stats are not impressive. That's why he never won a Super Bowl. Brady's postseason stats are better than Manning's, particularly if you throw out Wild Card games.
#63 by duh // Dec 15, 2015 - 12:17pm
Looking at Playoff QB performance post 1960 when their passer rating was under 75 shows this:
Tom Brady has 8 games in the playoffs with a game Passer rating of under 75, his record in them is 5 wins and 3 losses.
Dan Marino had 9 games in the playoffs with a game Passer rating of under 75, his record in them was 1 win and 8 losses.
Some others of note:
Favre was 1-6 in such games
Aikman was 1-5
Elway was 4-5
Montana was 2-4
The only other QBs I can find who had a winning record in 5 or more playoff games with a Passer rating of less than 75 were Steve McNair and Earl Morrall the other 26 all were .500 or less. (and only 4 of them were .500)
#72 by ramirez // Dec 15, 2015 - 9:06pm
It's important to remember that Brady has played more playoff games than any other QB. So that means that in 21 of 29 playoff games, his rating was 75 or better. In 14 of those games he was above 90, and in 10 he was 100 or above. So the notion that Brady has typically put up weak postseason numbers is not accurate. So let's look at the games where Brady has won with a weak passer rating, and see if it indicates that he keeps getting lucky. The two games I'll give you are the 2007 and 2011 AFC championship wins, in which Brady was not very good, although in the Baltimore game he was credited with a GWD. The other games are the Tuck Rule win over Oakland, the 2003 win over Tennessee, and the 2006 win over San Diego. In each of those games, Brady led a GWD late in the 4th quarter or in OT. That doesn't suggest to me that it was the product of lucky breaks.
Look, all QBs with a significant number of playoff starts have had games where they did not play well and still won. The fact that Brady has a couple of those type of wins, out of 21 total playoff wins, doesn't really tell us anything. He's also had a couple of games where he played well, and still lost. Those kind of outlier results don't tell us a lot about a guy's career.
#83 by theslothook // Dec 15, 2015 - 11:45pm
Can you name which games he played well in but lost? The 2006 Championship game he played well. Beyond that, I can't think of another one. Maybe the 2011 superbowl, but he was just ok(by his standards) in that game. Btw, Eli was just ok in that one too.
#68 by Duff Soviet Union // Dec 15, 2015 - 3:54pm
It's been shown quite often that Brady and Manning's play in the postseason is remarkably similar with one exception- wins and losses. When Manning has a bad game, his team pretty much always loses whereas when Brady has a bad game the Patriots still win more often than not.
Marino, on the other hand, really was a pretty poor postseason performer. I don't really hold it against him though since he never would have been in a position to under-perform in the postseason if he wasn't so awesome in the regular season. And no, I don't think it's "making excuses" for him to point out that his defenses were usually below average and his running game was a joke.
#73 by ramirez // Dec 15, 2015 - 9:23pm
I agree about Marino, but what you said about Brady and Manning is nonsense. Take away Wild Card games, and Brady has better postseason numbers, particularly in the Super Bowl. Brady's teams also outscore Manning's teams by more than 4 points per game in the playoffs. Brady throws a higher rate of TDs, and a lower rate of INTs. Brady has succeeded 9 times in 12 chances at postseason GWDs, Manning just once in 9 chances. These are some of the reasons why Brady has won more playoff games.
Peyton Manning was bad against both KC and Baltimore in the 2006 playoffs, but won both games. In the 4 playoff games that year, Manning had 3 TDs and 7 INTs, and a rating of 70.5. The Colts won the SB because the defense allowed just 16.3 ppg in the playoffs, and the running game averaged 151 yards per game. Peyton was good in the 2nd half against New England, and that's about it. Actually, that 2nd half comeback against the Patriots is pretty much the highlight of Manning's playoff career, unless you want to crow about some lopsided wins over Denver in the Wild Card round. Manning's a great player, but his playoff career leaves a lot to be desired.
#76 by ramirez // Dec 15, 2015 - 10:17pm
Do you honestly not see the significance of the fact that Manning's best numbers have come in the first playoff round? And Brady has only played 3 games in the Wild Card round, and in 2 of those games, he had a passer rating above 100. So I didn't take away all of Brady's bad games, far from it.
Please check the facts before you post next time.
#67 by RickD // Dec 15, 2015 - 3:04pm
After his sophomore season when the Dolphins made the Super Bowl, the Dolphins never won more than one game in any playoff run during Marino's career. These losses include several stinker games like hosting NE in the AFC championship game in 1986 (20 for 48 with 2 TDs and 2 INTs), losses to Buffalo in '90 and '92 (also below 50% completion rate with 2 INTs in each), and a dreadful loss to NE in '97 when he complete less than 40% of his passes and again had 2 INTs. And his last playoff game vs. Jax he again completed less than 50% of his passes and had two INTs.
In eighteen playoff games he had 24 picks to 32 TDs and had a total passer rating of 77.1.
Is is "mostly" Marino's fault they kept losing playoff games? I don't know about "mostly" but certainly his level of play tailed off in the playoffs.
Brady doesn't have a single playoff game with less than 50% completion percentage. Marino has five, and one of those was below 40%.
#74 by Scott Kacsmar // Dec 15, 2015 - 9:50pm
I think ramirez is way off here, but I'm not touching the Irrational now. I will in due time, because I solved the Peyton Manning playoff puzzle this offseason. Just have to put it into words.
I will comment on Marino. 23.4% of his playoff drives started with at least a 3-score deficit, which is just staggering. When the playoff game was close, he did his job every single time. The problem was he had a lot of blowout losses. How much of that is on him? Hard to say. We don't have those games as readily available via torrents/streaming like we do for today's QBs. But I don't think anyone else really would have stood a chance at an advanced age against the 98 Broncos or 99 Jags with the way Miami played in those games. And the Bills were always better, running all over Miami's defense in the early 90's. His SB loss was against an all-time great team that destroyed his defense too. Marino did not play poorly against them in 1990 loss. You could say 1992 AFC-C was a bad day. The 97 game against NE was certainly bad. He wasn't as good in the playoffs as most of his peers, but he was better than his losing record suggests.
#82 by theslothook // Dec 15, 2015 - 11:43pm
The convenient decision to selectively drop data to support a prior, like dropping Brady's wildcard games, thus really showing how he is > than Peyton Manning in the postseason. We all biases, but but few are willing to make such ridiculous arguments so plainly. When someone does, it usually means they aren't being objective at all.
Frankly, its on the level of me arguing that Brady can't win without some form of cheating. See...statistically, you can't argue my point, but that doesn't make it a correct argument or even one that's reasonably justifiable.
#84 by ramirez // Dec 16, 2015 - 12:12am
Brady has better numbers than Manning in the postseason to begin with. When you take away the Wild Card games, the gap gets even wider. Brady's numbers in the Super Bowl are significantly better than Manning's. In addition to better stats, Brady's other advantages are that his teams score more points per game and per drive, he throws a higher rate of TDs and a lower rate of INTs, and he's been much more successful at converting GWD opportunities in the postseason. Brady's record with a top 10 scoring defense in the playoffs is 18-6, with 5 Super Bowl appearances, and 4 SB wins. Manning's record with a top 10 defense is 2-6, with 1 SB appearance, a loss. I've also shown that Brady's 5-3 record with a rating below 75 doesn't tell us anything conclusive, and I"ve shown that Manning has won games in which he played poorly, too. Manning's playoff stats in 2006 were weaker than Brady's playoff stats in any of the four seasons he won the SB. How does any of that suggest that Manning has been as good as Brady in the playoffs?
By taking away the wild card stats, I removed Brady's worst playoff game (2009 Ravens) but also two of his best (2005 Jaguars and 2006 Jets). So it's NOT the product of bias in favor of Brady.
It's only in the analytics community that I run into resistance on these points. Among pretty much all other football fans, the claims that Brady is better than Manning, and that Brady has been better than Manning in the playoffs, aren't considered controversial.
#85 by theslothook // Dec 16, 2015 - 12:23am
I'm not going to go deep into this, but its been analyzed many times over. But for the sake of discussion - here's a small tidbit. Comparing qbs in the postseason is not like comparing two runners on an equal track and gauging who is faster. Neither plays in the same scheme/same circumstances. Once you try and correct for that, the differences essentially wash completely away. But even then, here's why its misleading...
The inherent flaw in trying to assess the quality of a player to a small sample size. When you also realize that the teams themselves are very different year to year and the circumstances they are in are different year to year, you then realize you're not just faced with a small sample size, you're faced with a small sample size that's not even independently and identically distributed. In other words, its a big mess. If you don't agree with that, there's a huge subject called statistics that will explain it in very great detail.
#86 by ramirez // Dec 16, 2015 - 12:43am
You're right, circumstances do matter. And in this case, Manning has played more of his playoff games in domes, and Brady has played more of his games in bad weather. So actually, this helps Brady's case even more, a point I hadn't previously considered, so thanks for that.
Apart from that, you're saying that we can't say Brady is better than Manning in the playoffs, because the sample size is too small to be meaningful, which is nonsense. It's time to give it up, dude. Brady has had a better playoff career than Manning. That's just the way it is.
#87 by theslothook // Dec 16, 2015 - 1:23am
Sigh, I guess its pointless to argue. Seriously, without sounding arrogant, statistics as a discipline has whole chapters on why the assertions you are making are simply invalid. Now you can toss those all aside if you want.
#92 by theslothook // Dec 16, 2015 - 6:49pm
Ok, I took the bait. You've essentially implied Tom Brady's numbers are better than Manning's anyway you look at it. Well, ignoring the more naunced arguments I made above, I will go with the very simple and naive version: Comparing their two ANYA's, Brady is worse than Manning. If we drop both of their worst games, the gap grows even larger. If I did the ultimate cherry picking and dropped only Brady's worst game and left Manning's career as is, he's still worse than PM. So no, I don't agree with your argument even on naive grounds.
Oh and to cement that clutch argument - Brady's career regular season ANYA is 7.0. Manning's is 7.17. In The postseason, Brady's drops to 6.1, Manning's drops to 6.48. Thus, Brady's drops by -.9 vs Manning's drop of -.69, thus showing Brady is worse than Manning when it comes drop off in the postseason.
Btw, for the rest of the patriot fans reading this - In no way do I take these numbers to suggest I think Brady is unclutch or a worse player than Manning either in the regular season or postseason. In fact, I have made my views of both players very clear in many other places.
#93 by ramirez // Dec 16, 2015 - 9:56pm
I appreciate you guys taking the time to respond to my posts. I think you are missing the point(s) I'm trying to make. I'm saying that Brady has had a better postseason career than Peyton Manning, and I think the majority of the evidence weighs in Brady's favor. I never said Brady was ahead of Manning by EVERY playoff metric, that would be absurd. Manning is ahead of Brady by anypa, and by a significant margin. He's also ahead of Brady in accuracy and yards per attempt. Manning takes fewer sacks than Brady, and Brady's only ahead of Manning by half a point of passer rating. I think we should also give Manning a small boost because he's never enjoyed the coaching stability Brady has had under Belichick. Brady has also generally had better defenses, and better offensive lines than Manning. Those are all significant factors that help Manning, and I considered all of them when making my assessment.
But now let's compare that set of advantages for Manning, to Brady's advantages. Most importantly, Brady wins far more games in the postseason than Manning. Partly because he's played so many playoff games, Brady is the career leader in most major passing categories, including attempts, completions, yards and touchdowns. But Brady's rate stats are also good, and Brady throws both a higher rate of TDs, and a lower rate of INTs. Brady's teams outscore Manning's by both points per game and points per drive. Brady has been much more successful at converting GWD opportunities, 9 for 12 to Manning's 1 for 9, and 4 of those came in Super Bowls. Now it's true that Brady has had a top 10 defense more often than Manning, but Brady's record of 18-6 with 5 SB appearances, and 4 SB wins, dwarfs Manning's record of 2-6, with an 0-1 record in SBs. Brady and Manning have both had a couple of games where they played poorly and won, but in the vast majority of Brady's playoff wins, he's played well. Brady's stats in 2001 weren't impressive, but Peyton's stats in his SB season were even worse, so you can't claim that Brady is the one who has relied more heavily on his teammates.
I've been emphasizing the point about removing the Wild Card stats because I think it's very instructive. Much of Manning's value in the playoffs has come in the first round, the only one in which either guy has a rating over 100. But Brady's best split by round is in the Super Bowl, with a 95.3 rating in 6 appearances. Why is that important? It proves that Brady doesn't decline in more important games, in fact if anything, Brady is at his best in the Super Bowl. I believe this is one of the major reasons why Brady's teams have more often advanced deep into the playoffs, and Manning's so often have not. By removing Wild Card stats, I've taken away two of Brady's best playoff games. Why would I do that, if my goal was to make Brady look as good as possible? I did it to emphasize the point that Brady has outperformed Manning in later playoff rounds. As a general rule, defenses get better and competition gets tougher the deeper you go into the postseason.
The way I see it, it's not close. Brady has clearly had a better postseason career than Manning, on both an individual and team level. As to the claim that playoff careers are too short to produce meaningful sample sizes, I don't agree. Brady's 29 career starts constitute almost two full seasons of data, and Brady and Manning have combined to play almost 50 playoff games. I don't think I can make my case for Brady over Manning in the playoffs any more succinctly than that.
#44 by ChrisS // Dec 14, 2015 - 4:52pm
A 3.8 with a minimum of zero means to me that literally he could not have played much worse, according to QBR it was a (an) historically bad performance. I did not see the game so I can't say, but it looks like he had a few good plays.
#31 by RobotBoy // Dec 14, 2015 - 3:42am
I'm curious about this YAC+ of which you speak. I understand giving long passes higher value than short passes, but are all short passes created equal? One would imagine that a better thrown ball will give a receiver a better chance at YAC, even of getting through the first tackle attempt. If a quarterback throws a three yarder on 3rd and 11, but hits the receiver in stride with a lot of green in front of him, is that rated only slightly better than throwing the three yarder on 3rd and 11 and leading the receiver in a decapitating hit? Does QBR give credit for anticipating receivers' breaks and going through progressions quickly, or the weakness of an OL?
From what I've gathered from QBR, the short passing QBs, even the most effective, are generally underrated. This bias seems to be highlighted where Kacsmar's writes: 'When you have -- or when Brady had -- hard to tackle players like Dion Lewis, Edelman and Rob Gronkowski catching the passes, it's easy to see why YAC+ brought Brady's QBR down.' Fair enough but do QBs throwing deep balls to the likes of Beckham and Brown get docked points because of the quality of the receivers also?
#35 by Hoodie_Sleeves // Dec 14, 2015 - 9:41am
Pretending that YAC is completely on the receiver is nonsense.
The receiver's ability to dodge/break tackles is a significantly smaller factor than the QB's ability to pick the receiver that has the biggest chance of getting a big play.
Like Robot Boy suggests above, QBR thinks that the quarterback throwing a 5 yard pass where the receiver gets decleated as soon as he catches the ball is the same as the QB dropping a ball over a linebacker to a RB on a route designed to give the RB lots of space behind the linebacker. It's nonsense.
QBR is a bit like ALEX - it's stating that a specific style of play (The long bomb) is better than another style of play (the entire NE patriots offensive scheme essentially).
#36 by Scott Kacsmar // Dec 14, 2015 - 10:31am
It's not stating anything. The numbers just show common sense. On average, it's harder to complete deeper throws than it is shorter throws.
If YAC was really a QB skill, then why wouldn't it consistently translate among most of a QB's receivers? The same guy is throwing these accurate passes, isn't he? Why would only certain ones have good YAC numbers? Oh yeah, because it's largely based on each receiver's ability and their role in the offense. If you're a deep threat, you're not getting much YAC. If you're a slot WR, especially in a NE offense, you'll have one of the greatest YAC rates in history because they'll hardly ever target you on a route beyond 10 yards. RB? Tons of YAC just because of where you're catching the ball most of the time and screen passes.
Watch receivers catch passes and one of the things that should instantly jump out to you is how quickly they adjust to gathering the ball and running. Even if the throw is off target, it doesn't take much effort at all to pull it down and run. The bad throws that cause a receiver to fall down to make the catch or do some really weird extension on a dive, those are very rare. I'd be very surprised if anyone could prove a certain QB gets a lot more of those catches every year than most QBs. And yeah, Eli Manning would probably be the guy most will target first. Highlight syndrome is a bitch.
#46 by Scott Kacsmar // Dec 14, 2015 - 5:27pm
You're not closing in on a receiver in a fraction of a second, and if there's a play to have some serious YAC on, the defender's not going to be that close in the first place. Sometimes an off-target throw actually leads to better YAC since the receiver makes a move that was unexpected.
#56 by theslothook // Dec 15, 2015 - 1:30am
The argument that a qb is specifically responsible for YAC. You mentioned the smith example as have others with tom brady. They are legitimate examples, but then if they were true, they would show up in the data. Fact is, yac seems to follow the receiver, not the qb according to the statistics.
#80 by theslothook // Dec 15, 2015 - 11:37pm
Fair enough. Since no one really staked a percentage, I inferred that most people felt that qbs are say 60% and above responsible for YAC. I assert that WR make up 70 percent of that, which is in line with what the research says.
#57 by Scott Kacsmar // Dec 15, 2015 - 1:36am
Good example of what? SD ultimately rushed five and the DB was beat on that play. Sometimes that'll go for a TD, but I bet you most of the time the CB will play it better and stop him at the 30 at worst.
What you picked is a good example of basically the one big passing play in a 10-3 game. Not exactly something to be proud of.
#59 by Scott Kacsmar // Dec 15, 2015 - 2:00am
No it's not. The YAC there is close to what the expected would be, but a broken tackle is what gives the play a boost. Alex Smith didn't break the tackle for Wilson.
I'm saying that's not some offensive performance to be proud of. They scored 10 points at home against one of the worst defenses in the league.
#62 by Scott Kacsmar // Dec 15, 2015 - 10:28am
You post as "PatsFan" and still feel content to call someone else biased?
I don't care for bombs. I like a vertical passing game. I like watching an offense where I might see a pass thrown over 10 yards with regularity, and not just on a pick play or a blown coverage or some extreme mismatch.
You can keep treating YAC as a QB skill. I'll stick with my research and the research of others I have no connection to that shows otherwise.
#37 by nat // Dec 14, 2015 - 10:45am
If you want to understand why QBR is a flawed stat, just look at the description of the Brady-to-Edelman TD in the article:
The ball traveled 18 yards past the line of scrimmage and hit Edelman down the left sideline. Edelman ran 10 yards untouched, juked a defender and broke two tackles before gaining a total of 41 yards after the catch for the touchdown.
Followed by...
In the case of the Edelman touchdown, the expected YAC was three yards, meaning he gained 38 more YAC than expected. Brady gets more credit for the first 21 yards than the final 38...
What happened to the other 7 yards untouched? And the expected additional yardage of a quality receiver set up to face a lone defender in the open field?
QBR simply denies the existence of QBs who set their receivers up to gain yardage after a catch. And ESPN's hand picked example shows how wrong that is. We can assume they picked an example to bolster their case, and that there are many worse examples out there.
Anyone with even a little knowledge of football knows that there is a world of difference between passes that lead to immediate tackles and passes that set up long runs, even for passes of the same air distance. QBR just ignores that basic fact of football and substitutes an average YAC when grading QBs.
That's just plain ignorant.
#38 by Scott Kacsmar // Dec 14, 2015 - 11:00am
It was a pick play. Let's not act like there was some tough throw to get him open. Amendola got him open.
You can't criticize a stat for not being objective and then complain that it didn't use a subjective measure (the extra 7 yards). You're just assuming he gets the extra 7 yards. The stat is using 3, because that's what the research shows you're expected to get in that situation. You don't expect someone to be that wide open usually. If you're going to dismiss that because "this play was different" then you might as well dismiss all of these stats, including DVOA and EPA, or anything that is built around the idea of what's expected based on historical data.
What you called ignorant is actually just trying to make the stat objective. You're working in fantasy land if you want to judge every play by how much YAC you think the receiver should have gained.
#39 by Lyford // Dec 14, 2015 - 11:21am
"Let's not act like there was some tough throw to get him open. "
That's the point - when a QB is really playing well, most throws are not tough throws. Great QB play is far, far more about running the offense than throwing the ball. There's a baseline ability to throw the ball required to play NFL QB, and all of them and hundreds more have that ability, but it's not even close to being the most important part of the job. QBs get throws that aren't tough because they're doing their job well. Most of the work is done before the ball is thrown, and YAC and QBR take no cognizance of it.
#47 by Scott Kacsmar // Dec 14, 2015 - 5:33pm
This is going down a road that stats just aren't equipped to travel. Without knowing the play call, how often does the QB actually pick the best target on each pass? Most of the time he'll have five options, so it's a 20% chance, but the most open guy is not always the best option. You have a down-and-distance situation to play to. You have a clock and scoring margin. Some plays beyond screens are clearly designed to go to a certain receiver, so the QB's hands are a bit tied with that decision. And one of the things I hate most about football analysis, especially in the all-22 era, is the "he should have thrown to this open guy" hindsight takes. So often people are saying that with ignoring the physics of the play. If the QB is moving to his left and you're pointing out a guy in the right flat, that's probably not a smart throw. Could end up as a pick-six if you're not careful.
I hope these Next-Gen stats expand to look at things like how open a receiver is at the catch point to see which offenses are doing the best job of finding the open guy. Find out which receivers get the best separation. Right now, I don't really care how many MPH a guy runs, but unfortunately by the time they get to doing stuff like this, we won't have any data on the best quarterbacks of this era.
#41 by theslothook // Dec 14, 2015 - 12:33pm
To address the yac comments...sure on the surface the qb(in this case everyone seems to be thinking of brady) is making the right read and so he deserves a good chunk of credit. As scott pointed out, if this were true, you would see it in the data. But you dont. Brian burke had a post all about it.
I saw last nights ne v texans game. Brady threw short of the sticks repeatedly but amendola wiggled out of tackles for first downs. We all remember plays a qb saw a hole in the zone and read it perfectly, while conveniently forgetting the million other ones that were simple throws where the receiver did most of the work.
#69 by Duff Soviet Union // Dec 15, 2015 - 3:59pm
The YAC is graded compared to the average YAC on a whole bunch of similar throws. That's why Bortles is getting penalised, or more accurately, not rewarded, for his 80 yard touchdown pass. I imagine that the average YAC on a 20 yard sideline out is pretty much nil, but in this case the Colts DB made a horrible tackle attempt and Hurns went the distance. The pass could have easily been defensed or intercepted (not sure if this is factored in or not), but I'm guessing that pass turns into an 80 yard touchdown roughly 1% of the time.
#70 by TXinsider // Dec 15, 2015 - 7:10pm
I find this entire discussion on QBR (or what's better!) boring.
When you get down to it, I'd say that the top 5-10 at any position are quite equal, with any deviation in stats owing to scheme, coaching, coach willingness to let loose, health, and other etc's. I don't give a rat's ass whose numbers are better today, this month or this year because, the real reason why is always because of things out of a given player's control.
Further, it's not even fun talking about a particular football player's stats. Baseball stats are fun, football stats are boring - we all know they're reliant on team functionality.
#71 by theslothook // Dec 15, 2015 - 7:43pm
Well, that's why advanced stats exist - to help bridge some of that ambiguity that you're describing. I suppose we can all just look at the macro statistics of team quality and give up the whole attempt at quantifying individual players, but the fact is the nfl is made up of individual players and rosters turn over so much. When thinking about how much to pay a player or who to sign in free agency or even if the coach/gm is doing their jobs, you have to consider the quality of the individual player and that's why we care about these statistics.
#88 by ammek // Dec 16, 2015 - 5:49am
Can anyone explain Derek Carr's QBR? He ranks 7th in VOA, 9th in ANY/A, 10th by passer rating – and 23rd by QBR. I can just about comprehend how a stat can mark Brady's receivers as being responsible for much of the production, but Carr's?
#89 by Scott Kacsmar // Dec 16, 2015 - 6:10am
Carr is 31st in run EPA and has fumbled 8 times this season. I would also imagine he gets hit for some of the biggest turnovers in the fourth quarter this season. He threw a pick-six vs. Denver in a 9-7 game with Oakland in FG range. He threw an INT on 3rd-and-10 at the PIT 11, down by 7 with about 4:30 left. He threw another INT from the 11-yard line in a 23-14 game against Minnesota. He had that brutal 4Q vs. Chiefs two weeks ago with 3 INT.
We talked about funky single-game results with QBR, but Carr's game log actually makes a lot of sense outside of maybe the Chicago game being a 25.9. http://espn.go.com/nfl/player/gamelog/_/id/16757/derek-carr
I also think we might be focusing on the YAC part too much. Carr's 4-TD game vs. Jets had a lot of big YAC plays from his receivers, but he still gets a 93.6 QBR for that game. Similarly, Carson Palmer had about that same QBR vs. Vikings, and I highlighted how his long gains of the night on the TDs were just YAC plays. So it seems like you can still put up a 90+ QBR even with huge YAC TDs.