Writers of Pro Football Prospectus 2008

23 Sep 2013

ESPN: Aaron Rodgers' Hidden Weakness

Why can't Aaron Rodgers seem to lead his team to a comeback in the fourth quarter? There's certainly a lot of regression to the mean on fourth-quarter comebacks, but a 5-24 record is pretty significant.

Posted by: Scott Kacsmar on 23 Sep 2013

53 comments, Last at 08 Jan 2014, 4:42pm by sac longchamp bandouli

Comments

1
by Thunderbolt of ... :: Mon, 09/23/2013 - 1:22pm

Is there any predictive, or even explanatory, value to this stat? A table listing 24 "notable" quarterbacks doesn't seem to be very conclusive evidence of correlation. This seems to be an ESPN-quality statistical analysis (noticing a single statistic and writing an article about it) rather than a FO-quality analysis (actually trying to figure out what is going on).

2
by DA (not verified) :: Mon, 09/23/2013 - 2:05pm

If Rodgers was clutch he could have prevented Franklin's fumble obviously

4
by Johnny Vicars (not verified) :: Mon, 09/23/2013 - 3:11pm

Could he have prevented the bad picks? Could he have prevented not getting it done on the last drive? Rodgers is not good in pressure situations when the game is on the line. In games where they are blowing out the other team and there is no pressure, he can do his little jump balls to Nelson and such, but when he has to perform or they lose, he is not good. Sorry.

9
by DA (not verified) :: Mon, 09/23/2013 - 4:41pm

This is not true at all. Rodgers actually has been very good overall in pressure situations. The 1st INT Jones even admitted afterwards was his fault for stopping the route.

27
by BJR :: Tue, 09/24/2013 - 8:05am

I remember it well - Rodgers collapsing under pressure against a mean defence in the biggest game of his life in Super Bowl XLV.

3
by tuluse :: Mon, 09/23/2013 - 2:26pm

"5-24 record is pretty significant."

Is it? If a baseball player got on base only 5 times in his first 24 at bats, would it signal what his career would be?

5
by Johnny Vicars (not verified) :: Mon, 09/23/2013 - 3:12pm

Do you understand, why that comparison you made is stupid?

6
by Keith(1) (not verified) :: Mon, 09/23/2013 - 3:26pm

I think it is rather good point, to be fair. There are a lot more moving parts in a football game, sure, and quite a bit fewer opportunities to turn things around, but this particular stat comes down on the side of "not indicative of a player's true skill."

With that said, having no idea the circumstances of all 29 games, I think most will agree that Rodgers' larger body of work (all other games and quarters) point to one of only a handful of possible issues, the most likely of which is the fact that other team's defense was able to shut down the offense throughout the rest of the game. I can think of only one or two examples where Rodgers has been unable to just simply outpace the other team -- vs. the Cardinals and vs. the Steelers -- which leaves a vast majority of the games where the other team had a really good defense.

It is an interesting little piece of information, but does not really say much about Rodgers himself.

8
by tuluse :: Mon, 09/23/2013 - 4:00pm

More moving parts makes it even less likely that it's a meaningful sample.

10
by DisplacedPackerFan :: Mon, 09/23/2013 - 5:05pm

I personally think it, and knowledge of the 29 games, says more about McCarthy and staff and their play calling than it does about Rodgers, if it says anything at all about anyone.

7
by mmeiselman :: Mon, 09/23/2013 - 3:47pm

It's somewhat hard to believe this article comes from Football Outsiders and not directly from ESPN

11
by theslothook :: Mon, 09/23/2013 - 5:51pm

I like scott normally, but the premise behind this article is flawed for a large number of reasons. First of all, we are essentially attributing the failure of winning a close game entirely to him. I realize qb is the most important position, but to assign 100 percent of the credit/blame to one player is absurd.

Furthermore, I'm not really sure what the stat ultimately points to. In a large sample of qbs, you're going likely going to end up with someone like rodgers if only by chance.

And finally - ultimately, are we really implying that rodgers becomes a different player when the game gets close to the end in the 4th quarter? I find this narrative is a completely after the fact kind of statement because people continually lambaste manning for his choker label and yet the vegas line continues to have him and rodgers a favorite. if it were really so true, then the lines would in fact reflect this.

13
by Anon (not verified) :: Mon, 09/23/2013 - 6:09pm

"we are essentially attributing the failure of winning a close game entirely to him."

Scott's response to this in a recent thread was that it's assumed that when he writes "Aaron Rodgers" he actually means "Aaron Rodgers and the Green Bay Packers," but then he turns around and writes articles like this. Kinda seems like he wants to have it both ways.

15
by DA (not verified) :: Mon, 09/23/2013 - 6:21pm

It is obviously Rodgers fault and is very meaningful. After all, with the same teammates, Matt Flynn was 1-0 as a starter when trailing in the 4th Quarter. :)

12
by Anon (not verified) :: Mon, 09/23/2013 - 6:02pm

Ugh, here we go with this again. That 5-24 stat includes games like:

GB vs. Steelers 09 - GB, down by 2 in the 4th, scores TD and 2-point conversion to take the lead on their last offensive possession. Aaron Rodgers then lines up at DB and fails to prevent PIT from marching down the field to score a TD with a couple seconds left on the clock. Packers lose by 1.

GB vs. Falcons 10 - GB, down by 7 in the 4th, goes on a 16-play, 5-minute drive to tie the score with 56 seconds left. Rodgers, lined up at gunner on the ensuing kickoff, gives up a 40-yard return. He then lines up on defense, allows the Falcons to gain 20 yards, and fails to block their go-ahead field goal with 9 seconds left. Packers lose by 3.

Fail Mary game - Packers, down by 1 in the 4th, go an 8-minute scoring drive. Aaron Rodgers, again lined up at DB, intercepts Russell Wilson's final pass. Packers lose by 2 anyway.

There are more like this, but from what I've seen so far, Scott's response is generally to mock folks who criticize him for this stuff as being Rodgers fanboys. It's bad-faith argumentation that ignores legitimate gripes with his methods, and I'm sure that digging through the stats he's compiled on other QBs will have examples like this as well. The problem is with how he defines his stats, not their application to Rodgers in particular.

24
by Bobman :: Tue, 09/24/2013 - 1:33am

Clearly, if he was a more complete football player, and not just some glory-boy QB, he'd have won those games for his team and the city of Green Bay. I've had the same problem with Peyton Manning for years. Hey, buddy, you're 6-5/230. You just scored the go-ahead points, great. So why don'tcha strap on some LB pads and defend your lead, dammit! These pansies expect other guys on the team to do all that messy tackling and crap and it makes me sick.

29
by Will Allen :: Tue, 09/24/2013 - 10:26am

I think Rodgers is a real choker, due to his failure to mentor John Jolly in a way to keep Jolly John off the codeine. More choking occurred, when he failed to prevent Charles Woodson from getting old. I hate guys like that.

14
by Roch Bear :: Mon, 09/23/2013 - 6:20pm

Do we even know if there is such a thing a consistent clutch performance? e.g., Do even year totals correlate well with odd year totals (for the same team compared to other teams)?

20
by Anon (not verified) :: Mon, 09/23/2013 - 8:35pm

No.

And, even given that he's "flawed" in the clutch, his superior production over 60 minutes of a football game is what ultimately matters. Tim Tebow is awesome in the clutch, and he's still looking for a starting job in the NFL.

16
by theslothook :: Mon, 09/23/2013 - 6:27pm

If we really wanted to discuss Rodgers hidden flaw, it should be his propensity to takes sacks. As far as I'm concerned, that may be the last flaw in his game. He does other things at above average levels and several more things at savant like levels. The taking sacks is the one area I think hes fairly below average in.

I also think a more interesting conversation would be to determine just how much hes being aided by the talent around him from a receiver perspective/good scheme design. Obviously, I think hes one of the best qbs in this game, but I've always wondered how much I should also be praising his receivers. I don't think individually they are great, but they work so well in harmony with one another.

18
by tuluse :: Mon, 09/23/2013 - 6:30pm

Rodgers and his receivers are very talented and very well coached.

Even when there is tight coverage, he just whips in that back shoulder throw that the DB has no chance to get at, and the receiver always turns at the exact right time to catch it.

It's infuriating.

I think I know how Titans/Texans/Jaguars fans felt during Peyton Manning Colts era.

19
by theslothook :: Mon, 09/23/2013 - 7:52pm

I've debated about who is the better qb, favre or rodgers. From a cursory stat examination, its rodgers, but part of me stirs at the idea of just going by nominal stats on this. The game has changed within even the last 5 years and so favre's numbers pale in comparison to todays evolution. I'm not an expert on gbs receiver quality over the favre years, but I suspect favre's receivers were not as good as rodgers' are today. The other issue is o line play and again I'm ignorant on the facts. Having watched both, its hard to argue against rodgers but then I'd probably differ again to a season packer fan. Or maybe an opposing bears fan :p.

22
by Will Allen :: Mon, 09/23/2013 - 9:28pm

As absurd as it sounds with regard to a guy who got way too much exposure (of a typically nonsensical nature) people tend to underestimate how great Stubbleface was at his peak, when he was coached very hard by Holmgren. The receivers were coached very hard as well, so even though Sterling Sharpe's premature exit left the receiving corps without dominant talent, they as a unit performed well. "He threw it too hard" was not an acceptable excuse.

25
by Keith(1) (not verified) :: Tue, 09/24/2013 - 1:49am

Favre's wide receivers were similar to Rodgers' groups of receivers, so much as neither really got many years of significant playing time with a truly transcendent player.

Favre had Sharpe, Antonio Freeman, a few good years of Robert Brooks and Bill Schroeder, a young and productive Donald Driver, pre-crazy Javon Walker, Greg Jennings, and then a slew of support guys like Mark Chmura, Don Beebe, Bubba Franks, and Robert Ferguson.

Compare that to Rodgers' group, which includes an older and slowing Driver, Jennings, Finley, Cobb, Nelson, James Jones, and minor support guys like Donald Lee and Tom Crabtree.

Add in Ahman Green, Edgar Bennett, Dorsey Levens, and a year of functional Ryan Grant to Favre's list. To be fair, add in two good years of running from Grant to Rodgers' list.

Favre certainly has more years (obviously, number of years played speaks towards this a bit) with near- or over-1,000 yard receivers, and many more years with 1,000-yard+ runners (though, not always). However, the general composition of talent on the team has remained fairly constant during both of their careers -- two or three 1000-yard+ receiver seasons for each one to two league seasons, some middling receivers between 400-600 yards, and a 1,000-yard runner.

If you figure that Rodgers could play as long as Favre did, I could see Rodgers being strictly better stat-wise with very similar career outcomes -- that is, long time contenders in their eras, a Super Bowl or two, in the discussion for greatest of their generation (or, at the very least, top-3 or top-5), and a few receivers on their coat-tails into the Hall of Fame.

They each have their place in my heart as a Packers fan, but if you told me I could pick either one in their prime to lead my team, I think I would go with Rodgers. It is really difficult to separate my current perception of how good I think Rodgers is, and how good he really is comparatively.

17
by nat :: Mon, 09/23/2013 - 6:30pm

At this point, Scott's use of GWD stats is so insanely bogus that he must be trolling, trolling for comment traffic. Only an idiot would fall.....

Oh, crap.

21
by kamiyu206 :: Mon, 09/23/2013 - 8:42pm

I also like Scott's work, but for this one I think he's constantly falls into a trap called 'Wyatt Earp'.

Chase Stuart's article about Ray Rice might help.

http://www.footballperspective.com/ray-rice-is-the-newest-star-in-what-a...

P.S. Scott's "Aaron Rodgers's tendency to take many sacks is the problem" claim is understandable, at least, but that makes me think why Ben Roethlisberger, another quarterback who takes many sacks, racked up some good 4QC/GWD numbers.

23
by usernaim250 :: Mon, 09/23/2013 - 11:55pm

I haven't read the ESPN piece but I've read Scott's other ramblings on this topic. I agree with the posters who find this line of analysis to be deeply flawed and frankly embarrassing to FO.

Before we even get to the very valid "what are the odds of that" critique, let's start with GWD. First of all, the small sample size lends itself to a fine grained drive by drive analysis rather than a simple headline number.

More importantly, there is no such thing as a game winning drive. There is only the drive that won the game. When you are in the midst of a drive, you have no clue whether "this is the game winning drive." It doesn't exist until you see what the other team's offense does. By contrast, you are aware of "late and close," for instance. That's a defined situation that really exists. An analysis of performance in a defined situation will have some actual value.

Additionally, the actual wins and losses are not exactly advanced stats. We look at those because giving "GWD" for a kneel down following a turnover and followed by a field goal has zero predictive value (possibly negative because a better QB might be trusted to advance the ball). There are very good reasons we use advance stats instead of W/L, and in particular why we scoff at the use of W/L for QBs. What's the WPA for the QB and the offense in late and close situations? What's DVOA? Even QB rating.

26
by td (not verified) :: Tue, 09/24/2013 - 2:19am

Not an insider, but I've assumed the point of running these articles was to deflate the PM/Brady 'winnersauce' argument (or the people who are hyper-critical about Romo while overlooking Eli's or Flacco's failings)

28
by dan s (not verified) :: Tue, 09/24/2013 - 9:55am

I feel bad about this, but the other commenters are right. This seems like garbage. Didn't the AFC lose the coin flip in the super blow like 14 times in a row? AT WHAT POINT DO WE START BLAMING THE AFC, SCOTT??? Also, if you look at 100 QBs, you expect one of them to be two standard deviations below, so that's silly.

Also, should you check the standard deviation on win percentage? It seems to me you're double counting games if you rely on win percentage. Note that 9-1 is .900, and 1-9 is .100, but it's really only 5 games of difference. Scott--I'm worried this article is long on numbers and short on wisdom.

Next--if your stat has Jay Cutler near the top and Kurt Warner near the bottom, maybe your stat is stupid? I love using this stuff as a counter to bad thinking--if someone says "Jay Cutler is bad in close games", you can say "Actually, he's fine." But that doesn't mean you should use what looks like a garbage stat to say "THIS GUY IS THE WORST." That's some real Phil Simms garbage there. Again, long on numbers, short on wisdom.

Finally, all this game-winning drive stats stuff. Is it at all useful in separating good QBs from bad? Scott says this is where cream rises to the top, and his evidence is that the top QBs in it go to the Hall of Fame. But most NFL analysis is garbage (FO is different...or used to be...)...people have been overweighting playoff success and shit like that for years, so of course these guys will be in the HOF. It's bad evidence to say "My stat is good because it tracks the beliefs of hall of fame voters like Peter King." Frankly, if 100 Peter Kings agreed with my stats, I'd take a harder look at them, and then I'd buy 100 giant Peter King traps to get rid of the infestation.

30
by Will Allen :: Tue, 09/24/2013 - 10:41am

This argument is about as sensible as one addressing Ted Williams' hidden weakness, based upon his hitting stats in the World Series.

Is FO dumbing things down for the ESPN Insider crowd?

31
by Independent George :: Tue, 09/24/2013 - 10:56am

I'm not an insider, but Scott's previous article on the subject basically (and convincingly) argued that Rodgers has a poor record was largely because he played well enough that the Packers were rarely blown out, and so they lost a lot of close games that they might not have otherwise been in.

34
by Will Allen :: Tue, 09/24/2013 - 1:11pm

Well, then it is the bane of all journalism; the crappily constructed headline, as a means to draw eyeballs. I can't blame somebody, I guess, for not writing, as a header, "Nothing to see here, move along".

(edited to get the intended meaning right)

32
by Scott Kacsmar :: Tue, 09/24/2013 - 12:57pm

Some things need to be clarified.

1. You cannot judge an article by its title. It's always easy to see who reads and who looks at titles. This is a business, so titles are going to be what they are. The writer usually has no say on them. Nowhere in the article did I ever say Rodgers takes the full blame of this record, and there's specifically a whole section where I express that.

2. The only thing I would change in what I've done is to not express it as much as "close-game failure" as it is "games where the team needed to score the winning points in the 4Q/OT."

You can't compare past studies of close games to what I've done, because it's a whole different method. I don't concern myself with the final score to decide how close a game was.

The Colts/49ers were tied or within one score of each other for 55:17 on Sunday. That's 93 percent of the game, yet because the Colts pulled away with two TDs in a 27-7 win, that game gets rejected as being close. Meanwhile the Redskins/Eagles slop from MNF in Week 1 was rarely ever close, yet since it ends 33-27 with Washington not getting the onside kick, that gets counted as a close game.

I go much deeper than the final score.

3. Everyone wants the one-line reason for why GB is so bad here. It doesn't exist, just as it doesn't exist for why the Patriots have been so good in the past or why the Carolina Panthers are even worse than GB here. It's always something different each game, but regardless of who was at fault, we shouldn't be ignoring how awful the record is for GB.

You want my one-liner on this? Here it is:

Aaron Rodgers has not been as bad as his 4QC/GWD record, but the Packers have been.

4. On Wyatt Earp - well wasn't Wyatt Earp a famous name at least? While someone is likely to be "that guy" near the bottom, how in the world could it be the QB with the gaudy stats and the 49-5 record in games that weren't this close late?

Let it be someone like Troy Aikman, who relied on the running game and elite defense and had bad years to start and end his career. Rodgers? It makes no sense how GB can blow so many teams out, yet be so bad in these games year after year. Great teams don't work that way, unless they were front-runners, which I laid out years ago with Kurt Warner and Len Dawson. I have stunning stats on them I will share eventually that show the front-runner difference.

As for the line about 5-24 being significant, I didn't write that, but I agree with it. I believe there's statistical significance and there's football significance. We're always dealing with SSS here, but in covering the game, sometimes we have to look at things that may only be football significant.

At what point does GB's poor record in 4QC/GWD become statistically significant? We're looking at a sixth consecutive season of this being a problem in the Rodgers era. We will work in the offseason to answer that question of statistical significance better.

5. Please, can we stop acting like Mason Crosby is the only kicker to ever choke or the defense is the only one to blow late leads? You want the 5,000-word article full of tables and data on how often such things happen to these other QBs? Give me some time to update things (it's been a one-man show going through thousands of gamebooks and manually entering thousands of rows of drive data) and I'll create it, and I'll do it on here as we are limited to 1,500 words on Insider.

6. This data is most useful at the extremes. Rodgers is one of those outliers, but he's on the side we'd least expect, much closer to Bill Kenney/Chris Weinke/Cam Newton so that makes him as interesting as any QB in history when it comes to this.

While Kurt Warner actually was not good at comebacks, no one should ever look at a table and say "QB X is better than QB Y because he's higher." Jay Cutler being higher does not invalidate this anymore than Philip Rivers being so high in passer rating or Doug Williams so high in sack rate. These things happen. We can make assumptions that the players are better than those they rank above at this specific thing, but that says nothing about overall value.

However, we should definitely not ignore when someone is considerably above or below his competition.

I'll end with one stat for thought.

If there was really nothing to any of this, then explain this result: In a 58-QB study, the four QBs who had the lowest rate of games and wins that included a 4QC also had the smallest average deficit overcame in their 4QC wins.

Some teams, with a specific QB behind center, are just not very good in these games.

33
by Will Allen :: Tue, 09/24/2013 - 1:09pm

"If there was really nothing to any of this, then explain this result: In a 58-QB study, the four QBs who had the lowest rate of games and wins that included a 4QC also had the smallest average deficit overcame in their 4QC wins."

Because weird stuff happens sometimes, and unless your sample size is really big, it's hard to assert strongly that it's not just some weird stuff?

36
by Roch Bear :: Tue, 09/24/2013 - 1:25pm

Indeed. That is point 1. Also, to read anything into the aforesaid 'anomaly' one would have to know the shape of the tails of those distributions.

37
by Scott Kacsmar :: Tue, 09/24/2013 - 1:31pm

Let's say you're right and it's just weird stuff. So is it not worth writing about and putting out there?

What I take from it is these teams managed to make very few 4QC wins in that QB's tenure. The few times they did, it was often in the most unimpressive fashion possible: the 1-2 point comeback and against a lot of bad teams.

Hell, Rodgers' first 4QC came against the 2008 Lions (0-16), it was a 1-point deficit and the Packers ended up winning by 23 points anyway. That's about as unimpressive as it gets. Of Rodgers' 9 GWDs, four are against the Lions. Is that just a factor of playing them twice a season, or is it the fact that the Lions are usually such a bad team, that they're the perfect team for GB to actually complete a GWD against? Matt Flynn did it too in 2011, and that was actually the winning Detroit team.

Peyton Manning had multiple three-score comebacks in the 4Q in his career. Is that just a coincidence or could anyone have done that? If the explanation for why that happened is "he's just really good" then that's fine with me. I don't write about the "clutchiness" of players. I write about what happened in clutch situations; success or failure. I've been able to find many patterns in that time too. Are they all significant? No, but I think you're making a big mistake if you continue to chalk it all up to random luck.

I write about what happens in these games every week. Some wins are more luck-based than others, but the skill and the decision making of these teams and players shines through more than anything.

38
by Will Allen :: Tue, 09/24/2013 - 1:43pm

Sure, put it out there, with the prominent disclaimer that there isn't enough data to make confident assertions. I know that goes against the grain of the typical path to getting paid in sportswriting, and I have no objection to people trying to get paid, and since my hatred of the ESPN website is so longstanding I'll never pay them, for I know, you did put in said disclaimer. If so, I will accurately self-identify as a ignorant critic.

I do fully appreciate the tension in writing headlines, however.

39
by Scott Kacsmar :: Tue, 09/24/2013 - 1:52pm

Do we need the disclaimer if it's assumed? I think most NFL writers are well aware of SSS issues, but the job would frankly suck if we only tried to be purely scientific about things. Joe Montana played in 23 playoff games (not even 1.5 seasons of games), but I like to think we did see enough to conclude he was pretty good against the top competition in single-game elimination.

When I see the Packers with Rodgers potentially being 0-20 at 4QC against teams .500+ (0-18 for sure), my brain does not jump to "this must just be random and meaningless." That's almost as many games as Montana played in the playoffs. It's roughly 22% of the games in Rodgers' career.

How much more do we need to see to say "yeah, this is a problem"?

40
by tuluse :: Tue, 09/24/2013 - 2:03pm

23 full games vs 29 * 3 or 4 drives at most is a huge difference in sample size.

Also, Montana was good in the regular season too. If you were just looking at 23 games and decided he was a good player, irrespective of when or how those games took place, I would be suspicious. Luckily he played in almost 200 games that we can look at.

41
by Scott Kacsmar :: Tue, 09/24/2013 - 2:22pm

That's just the thing. Is it only 3-4 drives in those 29 games? Sure, the game does not qualify until it comes to one of those 4Q/OT drives in a tied or 1-8 point game.

But what happens in the first three quarters sets the path for the 4th quarter. If Rodgers didn't get sacked 8 times in the first half against Seattle last year, that might not have been a one-score game in 4Q (then again, every Russell Wilson game ends up close eventually going back deep into his college days). It was a close game because GB never pulled away early, which is something they do as well as anyone.

This is why I hate when someone points out a "weak" GWD to me, ignoring all the other parts of the game that had to happen for that to even be the GWD. If you take a lead early in the 4Q, your job is not done. The credit comes for more than just the one scoring drive.

Do I think W-L record is the best stat for a QB for this? Absolutely not. But that first step is always to gather all of the applicable games, wins and losses, and create the record. I'd love to use something like EPA or WPA for just the plays the QB dropped back on tied or down 1-8, but I'm not equipped to calculate such things right now and the farther we go back in time, the harder it gets (play-by-play).

But do I think W-L is completely meaningless for this? No, not at all. While Rahim Moore screwed over Peyton Manning royally out of a GWD against Baltimore, I can't ignore that Peyton handed it off five times to Ronnie Hillman instead of ending the game with his arm. There is no Jacoby Jones/Rahim Moore play if Denver runs out the clock. I can't ignore the kneel down with 31 seconds left. I can't ignore the awful INT in OT that set up BAL's FG. So while Moore is the main reason Denver did not come away with the win, Manning's go-ahead TD drive does not fully absolve him of the loss. That's why giving him a GWD loss in that game feels reasonable.

Advanced stats (EPA/WPA/DVOA/QBR) on Manning in that game would be fine to use, but the loss is still legit too.

43
by Will Allen :: Tue, 09/24/2013 - 2:46pm

Well, I think w-l record in a small sample of games, as a means to analyze the performance of one player, is extremely, extremely, problematic, in a game where there are 21 other players on the field at any given time, 44 starters in all, excluding special teams, where in-game coaching has the most effect in any major sport, where randomness pertaining to official's calls, the bounce of loose oblong balls, tipped balls, and weather, play a huge role in the outcome of close games. Until you get to a really big sample size, you are just a confirmation bias accident waiting to happen.

I know it's not as much fun, for some folks, to say "Damned if I know", but it quite often more illuminating.

46
by TimJohnston (not verified) :: Tue, 09/24/2013 - 3:06pm

"But do I think W-L is completely meaningless for this? No, not at all."

Actually, I think that everyone in football (and team sports in general) would be better served if they stopped using wins and losses for the purpose of assessing individual performances.

Was that playoff game Manning's finest game? Nope (5.28 ANY/A against a so-so Ravens defense). But whether Manning stinks up the joint or throws for 8 TDs with an arm tied behind his back, he doesn't win or lose games strictly speaking; he can only add/subtract from your team's chances of winning. That's that. Only teams win and lose games at the end of the day.

Ben Roethlisberger himself can attest to this fact, since "his" first Super Bowl win came after his sparkling, legendary 9-21/135 yards/0 TD/2 INT/1 sack for 8 yards/1.14 ANY/25 rush yards on 8 carries performance. And the 1 rushing TD doesn't make up for that dud.

44
by dan s (not verified) :: Tue, 09/24/2013 - 3:01pm

Fully agree Tuluse--weird stuff happens. You can mine any stat to find weird anomalies.

Do you need a disclaimer? No. You need thoughtful analysis. This doesn't meet that standard...it's solely a "look at this weird stat for such a good QB" article. Again, I'm fine with you using your (good) data to rebut Peter King's nonsense. But to use it to argue that Rodgers is bad in these situations? You don't have enough evidence to show that there's something real here, and not just randomness.

And did you figure out if the standard deviation of winning percentage is the right way to go here? I know you say you don't like to dwell on statistical significance, but actually, that's what I think a lot of FO readers come for: thoughtful, statistical analysis. If you're using a bad measure of standard deviation that double counts games, that's a huge problem, especially in light of our Wyatt Earp criticism. If, after correcting for this (potential) problem, Rodgers is only 1 standard deviation from normal, that's a huge issue for your case.

47
by dan s (not verified) :: Tue, 09/24/2013 - 3:06pm

Great, great comment. This is what it's all about: http://xkcd.com/904/ .
The best commentary rises above this nonsense.

50
by Anon (not verified) :: Tue, 09/24/2013 - 7:02pm

"Please, can we stop acting like Mason Crosby is the only kicker to ever choke or the defense is the only one to blow late leads?

Can you please stop acting like this is what people are saying? This is the kind of thing I was referring to earlier when I said you argue in bad faith. Again, the problem is with your method, not how it's applied to Rodgers. The way you define success on a comeback/game-winning drive makes it possible means that a successful fourth-quarter, go-ahead drive on the final possession becomes a failed drive if the defense then gives up the lead, then I think your method is flawed, and judging from the comments so do a lot of others. Of course other QBs run into that stuff, and if your system is calling the same situation a failure when it happens to, ie. Brees then your system is screwed up in that case too.

51
by kamiyu206 :: Tue, 09/24/2013 - 8:07pm

4. On Wyatt Earp - well wasn't Wyatt Earp a famous name at least? While someone is likely to be "that guy" near the bottom, how in the world could it be the QB with the gaudy stats and the 49-5 record in games that weren't this close late?

--------------------------------------------------------------------------------------------

Doesn't matter Rodgers is a gaudy name or not, Scott. Isn't Ray Rice a famous guy, too? (I'm referring to Chase Stuart's article)

Important thing is, weird thing can happen to any guy.

Perhaps you're right and Rodgers actually doesn't have the ability to win close games. But your finding cannot prove it, because there is a possibility Aaron Rodgers is just another Wyatt Earp. I don't think it's that complicated.

35
by Ender (not verified) :: Tue, 09/24/2013 - 1:11pm

A bad defense with no run game and WR that drop tons of passes and a coach who is pretty bad at game management late in the game. This does not surprise me at all.

The Packers are bad at everything I'd expect in a team that comes back a lot. You win close games with clock management and they just stink at it.

This game specifically he did not play a very good game but it would be hard to hang the loss on him given how the game ended up going. He put them in the position to win and the 4th string RB who was only in because of a stack of injuries fumbled it away.

42
by Corey (not verified) :: Tue, 09/24/2013 - 2:39pm

Bad job outta you with this story, FO. Bad job, indeed.

45
by dan s (not verified) :: Tue, 09/24/2013 - 3:03pm

"Aaron Rodgers has not been as bad as his 4QC/GWD record, but the Packers have been."

Nope. We should start with the premise that anything we find--Montana's playoff record, Manning's success in close games--is random. Then, we should see if there's enough evidence to show otherwise. The problem with this article is that it stops at identifying an anomaly without presenting a strong enough case that the anomaly is driven by substantive factors.

48
by DA (not verified) :: Tue, 09/24/2013 - 4:24pm

A few quotes from Scott that I find amusing:

1)"You cannot judge an article by its title. It's always easy to see who reads and who looks at titles"

-When the majority of people can see little more than a title because the content is behind a pay wall, it is kind of silly to object to people focusing on the Title.

2)"The only thing I would change in what I've done is to not express it as much as "close-game failure" as it is "games where the team needed to score the winning points in the 4Q/OT."

-This is simply wrong. If this is the case, you should consider tied games too. In a game tied in the 4th Quarter, the winning points still need to be scored. Also, I assume Rodgers gets no credit for games where he scores what would be winning points and then the defense surrenders the lead again.

52
by Scott Kacsmar :: Tue, 09/24/2013 - 8:08pm

DA, tied games have always been considered. That's why there are two different stats (4QC and GWD). The GWDs include games where it was only tied in the 4th quarter or overtime.

And you're right, no QB gets "credit" for a game that is ultimately a loss. That doesn't mean I don't keep track of every time that happens and I have used tables for that in the past and will again on a larger scale in the near future.

49
by Aaron Schatz :: Tue, 09/24/2013 - 6:45pm

Hi everyone. I wanted to see if I could address some of the concerns that readers have about Scott Kacsmar's fourth-quarter comeback research. I'm going to post this in the discussion thread for both Clutch Encounters and for the ESPN Aaron Rodgers article.

I really appreciate that readers have only the highest expectations for Football Outsiders. It's the biggest compliment that you can give us. Scott's new to Football Outsiders, and I actually think the issue here may be more one of writing style and less one of disagreement about what stats are and are not important. I've always noted that Football Outsiders "leads the league in couching our opinions," but that wasn't really the style at the website where Scott wrote previously. It's also not the easiest thing to do in a 1,000-word article for ESPN Insider. Scott also knows perfectly well that to give a quarterback's overall win-loss record in close games is to paint with a broad brush, but a lot of the specifics where he has noted why certain quarterbacks have certain records are in articles on other websites. Over time, I'm very confident that even those of you who are questioning these Q4C articles will appreciate that he's a fantastic writer and historian of the game.

To be honest with you, the readers: I personally feel that a quarterback's W-L record on Q4C is *usually* not going to be predictive. That doesn't mean these numbers are not interesting, and these numbers are going to be better when they are accurate. You have to understand that before Scott came along to do this research, the numbers given out by team PR departments about quarterback comebacks were total crap. (I'm looking at you, Broncos.)

Also, just because something generally regresses towards the mean, that does not mean it *always* regresses towards the mean. A good example is injuries: in general, team injuries tend to regress towards the mean each season but there's no doubt certain teams like the Patriots have a higher level of injuries year after year after year. That's not random chance, there's something going on there even if we don't quite know what it is yet. So in this case, I do think there may be something to the idea that we learn something about certain quarterbacks who are either very high or very low when it comes to fourth-quarter performance. (And yes, performance in late and close situations does not necessarily equal wins and losses in late and close situations.) For example, a number of mobile quarterbacks have a better record than you would expect in Q4C: Vince Young, for example, or He Who Will Not Be Named. Is there something to the idea that a mobile quarterback is more likely to make a big comeback because he can take advantage of a defensive breakdown with a big scramble? I'm not sure right now, but it is an interesting question.

I will also say going forward that while Scott's work on this subject has always revolved around quarterbacks, I'm personally much more interested in questions about head coaches. I do think there's something there, and I do think it is predictive. (You all know how I feel about Ron Rivera.) I've encouraged Scott to work more towards looking at what we can learn about certain coaches and their records in close games. Is there a lot of randomness that goes towards deciding close games? Absolutely. But that's all the more reason to criticize bad decision-making by head coaches. You never know where the luck is going go, so you better at least get the process right. Random chance isn't an excuse for coaches to make stupid-ass decisions.

Finally, there is something to Scott's complaints that he keeps trying to say it isn't all necessarily the quarterback's fault, but that didn't really get into the ESPN Insider piece, and a lot of the people who are criticizing his work here seem to be willfully ignoring anything Scott says about it being more of a team issue than a quarterback one.

With that in mind… Scott's analysis in the past has mainly revolved around fixing all the errors in people's conceptions of past Q4C. Going forward, however, we have all kinds of ideas for making this research even better. We want to look at go-ahead/tying drives where we can credit the QB even when his defense gave back the lead (as in Super Bowls XXXVIII and XLII). We want to look closer at coaches. We want to work on using DVOA or EPA to separate out the quarterback's performance from times when he has his defense or special teams to blame. And we want to do better statistical analysis to figure out at what point a coach or quarterback's record in Q4C becomes statistically significant. Unfortunately, we can't do that in the next 24 hours or even the next two weeks. We've got all kinds of in-season responsibilities. But we want to get to it.

Finally, in case I haven't made it clear, I think Aaron Rodgers is awesome. There isn't another player I would rather have on my team right now. I don't think that his past record necessarily indicates that he'll blow it the next time he gets the ball with two minutes left and the Packers down four. But I do admit that I'll be watching a little bit closer because of the research Scott has done, and if he fails again, I'll be looking for the pattern that might explain why Rodgers does have such a poor record in these opportunities.

53
by sac longchamp bandouli (not verified) :: Wed, 01/08/2014 - 4:42pm

FOOTBALL OUTSIDERS: Innovative Statistics, Intelligent Analysis | ESPN: Aaron Rodgers' Hidden Weakness
sac longchamp bandouli