Writers of Pro Football Prospectus 2008

26 Sep 2009

ESPN: Do Clutch Quarterbacks Exist?

ESPN Magazine asked me to look at whether there truly is such a thing as a clutch quarterback, i.e. a quarterback who always plays better in the fourth quarter of close games. This article is the (very short) result of some (very short) research: After looking at both passer rating and DVOA, there doesn't seem to be any year-to-year correlation in the difference between a quarterback's performance in the clutch and his performance overall. (There's one surprising exception, but that's probably a fluke.) I certainly could do a more intensive study of this in the future, and perhaps we'll have a different finding, but the results of this simple study seemed pretty clear.

The ESPN folks are doing a good job of getting clicks on this with headlines that say I'm proving that "Tom and Ben are myths," but in reality, I'm just saying that what we think of as a clutch quarterback is generally a guy who is a good quarterback the rest of the time as well. You know, like Tom and Ben.

Posted by: Aaron Schatz on 26 Sep 2009

50 comments, Last at 30 Sep 2009, 12:11pm by RickD


by Joseph Adler (not verified) :: Sat, 09/26/2009 - 7:20pm

I did this analysis on baseball players a few years ago. Here's something to think about:

If you look at an individual player, you'd expect to see a distribution of different performances in clutch situations: some good, some bad. You could then average his performance out, deriving a "clutch DVOA" score for an individual player.

Suppose that you were to compare the clutch DVOA scores for a set of players. There would be some distribution to these performances. I'd guess that this was a normal distribution (because the individual clutch DVOA scores are a sum of random variables, the central limit theorem applies).

Even if there were no players who truly performed better in clutch situations, you would find that, simply as a matter of chance, a handful would appear to do so. I'd bet that Elli's performance is simply a matter of chance.

by DaveRichters (not verified) :: Sat, 09/26/2009 - 10:22pm

I think a problem with this analysis is the selection of clutch plays. There are many clutch moments in the "non-clutch" group and many non-clutch moments in the "clutch" group. It seems unlikely, even if there is a very large clutch effect for individual players, that this analysis could be fruitful. It would be far better, though not at all trivial, to have human observers rate just how clutchy each situation is and use the rating to select plays.

by Jerry :: Sun, 09/27/2009 - 1:51am

Human observers would be much too arbitrary. I'm personally fine with "late and close" as a proxy for "clutch"; if you can come up with a more comprehensive definition, great.

by DaveRichters (not verified) :: Sun, 09/27/2009 - 3:00pm

"Late and close" is a terrible definition for clutch. It actually isn't a definition at all, but a situation that is sure to contain more clutch situations on average than other situations, but probably only to a small degree, meaning that if the effect exists it will be very hard to see it with this analysis.

What's wrong with human observers? I don't know about you, but I can certainly identify clutch situations much better than the broad stroke of "late and close". In fact, the most accurate definition you could get for clutch would be through human ratings. I suppose you could build a model with a strict definition based on these ratings and that would only be marginally worse than using the ratings themselves. Moreover, if you would insist on using a simple broad definition, you could do much better than "late and close".

Why do you think human ratings would be arbitrary? Mine certainly would not be, but I do agree that if an observer were rating situations arbitrarily he would be unfit for the analysis.

by HostileGospel :: Sun, 09/27/2009 - 8:55pm

Do you have a concise definition of the word "clutch" that you would be prepared to apply to this analysis? I think "late and close" probably accommodates most people's definition of "clutch;" if you can more precisely replicate that, I'd be interested to see it. (Not trying to be snarky here; if you have a better way to describe clutch play, I'd actually like to hear it.)

Overall, I'd be kind of embarrassed to critique something when I didn't know what the hell I was talking about, but then, oh yeah, my NAME is on what I write, isn't it?

-Les Bowen

by DaveRichters (not verified) :: Mon, 09/28/2009 - 7:53am

I guess a concise definition would be something like "when the stakes are high", which doesn't really help. My point is that people already know what clutch is, so defining it isn't the goal; identifying clutch situations is the goal. I doubt that any concise definition of clutch can select the moments well. In order for these analyses to succeed the selection of plays needs to be as good as possible to maximize the effect that could be seen.

by Jerry :: Mon, 09/28/2009 - 2:23am

"Late and close" is/was a precisely defined sabermetric category that I came across years ago. It's like, if not exactly like (I don't remember), mlb.com's Late Inning Pressure Situations: "any at-bat in the seventh inning or later, with the batting team either leading by one run, tied, or has the potential tying run on base, at bat, or on deck." It looks like Aaron used something similar in his study, which strikes me as reasonable.

Human observers raise many questions, from "Are all 'clutch' plays identified before the snap?" to "Is the observer looking for a particular conclusion for the player/team she's tracking?" I strongly prefer a precise definition, even if it gets as complicated as what Bill James starts to describe here.

by DaveRichters (not verified) :: Mon, 09/28/2009 - 8:10am

The fact that a baseball analyst may have used a similar methodology isn't relevant. If they didn't measure a clutch effect I may have similar issues with that analysis, and if they did measure an effect then they had enough power so it's not a problem. Using a magnet to locate a needle may work for haystacks but not in iron mines, and its efficacy in the haystack situation is hardly evidence it would work in all contexts.

As to your second point, I'm proposing a controlled situation in which neither or your questions apply. The ratings would be done in the same way by all observers. There are details that would have to be worked out, but none are problematic. Do you think that 3rd and 7, on the opponent's 20 with 2 minutes remaining and a 6 point lead is a clutch situation? I certainly don't as an incomplete pass still results in a likely field goal and a 2-score advantage with very little time left. This situation is included, I assume, in "late and close". If you agree that this situation is NOT clutch, and I imagine you do, then we are really just disagreeing about to what extent this poor selection method limits the statistical power of the analysis. Noise propagates so any chance to eliminate noise in an analysis should be seized. There will already be so much noise in the analysis based on the performance of non-QB players on the field that I doubt this will ever work in football even with perfect selection of clutch plays (which makes it different from baseball).

by DrewTS (not verified) :: Mon, 09/28/2009 - 9:28am

"The ratings would be done in the same way by all observers."

Ratings are never done the same way by all observers. That's the primary objection people have to letting humans rate something versus just coming up with a definition. The definition will never be perfect, but at least it will be consistent.

The situation you describe above is an obvious one. That situation would probably not be considered "clutch" beforehand. But if the kicker misses that field goal, it might be considered a bigger deal in hindsight. Or, instead of putting that play on the 20, put it on the 35. You're still in field goal range, but it's a lot dicier. To me, that's a gray area, where a reasonable person could interpret it either way. The only way to ensure that all observers are going to interpret that situation the same way is to come up with a pre-defined selection criteria, which gets us right back to the humans vs formula debate.

by DaveRichters (not verified) :: Mon, 09/28/2009 - 1:11pm

Not true!

In the "dicey" area some observers would rate it very clutch and some would rate it not clutch, so the mean rating would be in the "dicey" area, which is exactly what you want.

I certainly would not assume that all observers would interpret every situation the same which is why multiple observers are necessary. What I mean by ratings being done the same way is not that each observer should have the same brain, but that each observer is given identical instructions.

Human ratings are a perfectly acceptable way to study something, and in this case would be clearly superior.

by dmb :: Mon, 09/28/2009 - 2:05pm

But if the notion of "clutch" play exists in the first place because of observation and confirmation biases -- a plausible hypothesis -- then it's definitely NOT the "clearly superior" methodology.

by DaveRichters (not verified) :: Mon, 09/28/2009 - 2:34pm

Not true! If it exists only because of confirmation bias that will be clearly visible in the result. Unless we only analyze data that confirm the theory, of course, which would be silly.

by Pat (filler) (not verified) :: Mon, 09/28/2009 - 7:38pm

"Do you think that 3rd and 7, on the opponent's 20 with 2 minutes remaining and a 6 point lead is a clutch situation?"

Why wouldn't it be? An interception/fumble gives the other team life, whereas an average play salts the game away. A sack could easily put a team out of field goal range.

"Clutch" refers to the situation, not the difficulty of the desired effort. If you restrict clutch situations to only ones where better-than-average QB play is required, you'll likely bias your selection. It's entirely possible that clutch performance exists, but every above-average quarterback is clutch, and only the mediocre/replacement-level quarterbacks have variations in pressure performance.

by DaveRichters (not verified) :: Tue, 09/29/2009 - 7:27am

Man, I have to watch games with you! There's nothing I would like more to hear than "What a clutch kneel-down to ice the game!"

Seriously, though, I was talking about the situation and not any level of difficulty. Surely you can't think that a situation is clutch just because a turnover or sack would hurt the team. I don't think I would ever call a situation "clutch" if kneeling down makes it extremely likely for the team to win, but if you would, fine, that just shows why ratings from multiple observers would be required. I find it doubtful that most people would agree with you considering how almost any result leads to a likely victory.

by Pat (filler) (not verified) :: Tue, 09/29/2009 - 2:40pm

"I don't think I would ever call a situation "clutch" if kneeling down"

Would you call a situation "clutch" if catching a long snap and placing it down basically ices the game? How is that significantly different than properly handling a snap? Because most people consider *fumbling* a long snap in a win-the-game situation a "choke" action. As evidenced by Tony Romo.

If you can choke, it's a clutch situation. The fact that the bar for success is low should be taken into account by the *statistics*, not by removing it from the sample.

Hines Ward would've iced the game if he had just went down in the Titans game. Jerome Bettis would've iced the game if he had just went down in the playoffs in 2005. If either of those players repeatedly fumbled in those kinds of situations, you'd want them to be called "not clutch."

And *removing* those screwups from the sample just means you're going to bias the result. Those were situations where the bar for success was very low. Ward just had to catch the ball and go down. Bettis just had to hold on. Romo just had to field a snap. They failed, dramatically, in all of those easy situations. Those results *offset* the situations where they succeeded, dramatically, in *hard* situations.

By removing plays that you think are "easy," you're already starting to generate the result that you want.

Implicit in the idea that "multiple observers is best" is the idea that the mass idea of "clutch" is unbiased, and I find that *extremely* unlikely.

by DaveRichters (not verified) :: Tue, 09/29/2009 - 8:45pm

You make an excellent point. I agree with nearly everything you say, but I have a quibble.

One is that you say:

"Implicit in the idea that "multiple observers is best" is the idea that the mass idea of "clutch" is unbiased, and I find that *extremely* unlikely."

I say:

What kind of bias are you talking about? I maintain that the mass idea of "clutch" is the best. Anything that differs from this is a bias. If the mass idea of clutch includes that the situation has a raised level of difficulty, then that is part of what clutch is. If you and I disagree with it, and I concede you've changed my mind, then it is our idea of "clutch" that is weird. You raise a great point, but the fact is that whatever the level of difficulty is for a given situation, we can still still measure success above or below a player's baseline, so I don't see it as an issue.

By removing plays that are "easy", if indeed that is what observer ratings would do, is appropriate. Either we spend time coming up with a definition of clutch that can give us enough statistical power to perform the analysis, or we select clutch plays in a different way. I'm suggesting a way that I think is the best and would isolate plays based on the consensus of "clutch".

by RickD :: Sun, 09/27/2009 - 2:02am

A poor p-value is not the same thing as displaying non-existence of a phenomenon.

This really needs to be a basic statistical fact taught to all sportswriters (esp. Rob Neyer).

We _know_ that there are players who have performed at a much higher level in high-pressure situations than others have. In baseball, the contrast between Manny Ramirez and Alex Rodriguez as postseason hitters is well documented. Given that they are roughly equivalent as regular-season hitters (if anything, ARod is a bit stronger), this disparity has to at least be considered _interesting_.

I would comment more about your DVOA analysis, but of course it's hidden in ESPN's pay-to-read section. But the general point is that: your failure to detect a phenomenon using statistical analysis is _not_ proof that the phenomenon doesn't exist. For all we know, your modeling is bad! Given our inability to examine the derivation of DVOA statistics, this problem is doubly bad!

by loneweasel (not verified) :: Sun, 09/27/2009 - 8:23am

First, the "A-Rod, post-season choker" thing is tired bunk. But there is little point to fight old baseball battles on a football site.

More importantly, there is a baseball analogy to be made with clutch quarterbacking. It's not clutch HITTING, but clutch PITCHING. There is little question among the saber community that clutch pitching exists. It makes sense. The guy with the ball should be expected to bust out moves in high leverage situations. While the guys waiting to react (hitters, defense) do what they always do (looking for a fat one to drive/getting on base, looking for a turnover/preventing the first down). The better initiator will have better moves. The more intelligent good ones will have disproportionately better moves.

I don't have any doubt that at least clutch offense exists. The most extreme example is the Boise bowl game, three plays they waited a whole season to run. One example of quarterbacking in clutch situations is the ones who usually never run, Brady, Peyton, Pennington, have scooted for first downs late in crucial games when they saw an opening. Eli probably just takes the sack if it were the regular season with the Jints up by 21 instead of down in the last drive of the superbowl.

The trouble in quantifying clutch quarterbacking is sample size. How many high leverage situation does a quarterback face who has a good offense and great defense? Compare that with the ace pitcher of say the Yankees, who will have a lot of high leverage at bats against in dozens of games even with a great offense behind him.

by Dunbar (not verified) :: Sun, 09/27/2009 - 5:16pm

Actually, Ramirez has always been better in the regular season, too, at least as a hitter. Rodriguez probably has better counting stats, since he doesn't come up with an excuse to sit out a month every year like Manny does, but in terms of rate stats Ramirez has been better in the regular season.

Ramirez's career OPS: 1.004
Ramirez's career OPS+: 155

Rodriguez's career OPS: .965
Rodriguez's career OPS+: 147

Ramirez's best seasons were better than Rodriguez's best, too.

Now, Ramirez is known for being a terrible fielder, and Rodriguez for being a pretty good one (especially when he was a shortstop), but if we're just talking hitting, Ramirez is superior.

by RickD :: Wed, 09/30/2009 - 12:11pm

"since he doesn't come up with an excuse to sit out a month every year like Manny does"

If by "every year" you mean "three times in a 17-year career", then this argument is sound.

Sorry, I know this is a football site, but I find that particular point to be really noxious and hard to kill. While he was in Boston, Manny Ramirez was by far the most durable player on the team.

by Anonymous Jones :: Mon, 09/28/2009 - 1:44am

Notwithstanding the title of the piece, I'm not so sure that the exercise is really meant to *prove* that clutch quarterbacking does not exist. You've got the null hypothesis wrong. The null hypothesis should be that no such thing exists, leaving the naysayers with the simple task of showing just one example to repudiate the null hypothesis.

What Aaron actually does do is make a good case that he has yet to find statistically valid observations of "clutch" quarterbacking. Proving something doesn't exist is very, very hard. The black swan may be just around the corner. That said, I hope we can agree that it's usually a waste of your time to keep searching for a black swan in London after you've just seen your 100,000th white swan in a row (not the best analogy here because color is a clearly observable phenomenon, but I hope the point is clear).

In any event, after reading Aaron's piece, it's pretty clear the null hypothesis should be that no such thing as clutch quarterbacking exists. It's up to someone else to prove that wrong.

Of course, it will likely be impossible to prove that null hypothesis wrong because of the inevitable sample size problem. It's totally irrelevant whether "clutch" exists or not (I still tend to believe that it does exist) if you can sufficiently establish that it is not possible to identify QBs good in the clutch *even if they exist.* With such small sample sizes (regardless of how you define "clutch"), it simply does not seem possible to prove that a series of so-called clutch performances were the result of prolonged luck or the result of skill. Ironically, the baseball postseason analogy of Ramirez v. ARod seems to be the perfect example of this. The sample sizes are still so small that they do not seem to establish anything other than for those x amount of at-bats, the results were the results (especially in light of the overwhelming evidence in the regular season to the contrary). Another perfect example would have been the seemingly endless Brady-Manning comparisons circa 2005 ("Manning is a choker. He will never, ever win a Super Bowl no matter what teammates he has." An idiotic prediction masquerading as fact even if it hadn't actually later been proven wrong.).

Frankly, the only way I see people "winning" this clutch argument is to do what so many other commenters have already done in this thread: defining clutch situations more and more narrowly. They don't realize that they are making it impossible to prove their own point by limiting the sample size so radically.

by Tracker (not verified) :: Sun, 09/27/2009 - 7:06am

I don't think DVOA and passer ratings are the best way to measure clutch quarterback performance. For example last year when Chicago scored a touchdown to go up by one at Atlanta with 11 seconds left you would think, baring a touchdown kick return, that Atlanta didn't have time to do much. But Matt Ryan got them in field goal position to win the game. Passer rating only sees his clutch performance as a 26 yard completion, and DVOA only it as a 26 yard completion on a 1st and 10. In terms of clutchiness I think it was off the charts.

I think you have to look at drive results instead of stats. Which quarterbacks get their teams reasonable FG attempts when they need to, which get TDs, and which flush their teams chances by fumbling or INTs (also consider a hail Mary INT is not the same as one with 1:30 left). Compare those rankings with DVOA or passer ratings. Those who rank signifigantly higher on the clutch list are your clutch QBs (raising their level), those who are about the same are your consistent QBs (good or bad), and those lower are you're chokers.

by Boo Cocky :: Sun, 09/27/2009 - 11:16am

Clutch stinks because a touchdown scored on the last drive of the game to win it is no more 'special' than if the team had scored on its first possession and never had to drive down the field to win it in the closing seconds. Instead they have the lead and kneel the clock out. Just like in baseball where a run in the bottom of the ninth equals a run in the bottom of the first, I think a touchdown in the final two minutes equals one in the first two.

by tuluse :: Sun, 09/27/2009 - 11:22am

If you have 1:40 and 2 timeouts left, it is in fact harder to score a TD compared to the beginning of the game.

by Still Alive (not verified) :: Sun, 09/27/2009 - 11:22am

Actually the TD in the first quarter is more valuable.

As for the article the answer is "no". I have seen enough research along these lines from enough different situations to think that it just doesn't matter on the professional level how stressful the situation is. The players are all such hypercompetitve people anyway I doubt they notice much.

by BobbyFlaysTouchdown (not verified) :: Sun, 09/27/2009 - 1:10pm

Of course later scores are more valuable. A go-ahead touchdown scored at the beginning of the game leaves nearly four quarters for a response. A comeback is a hell of a lot harder with only four minutes, or four seconds.

You can see this in the win probability studies, like in the The Hidden Game of Football. A first-quarter TD which creates a 7-0 score probably moves the scoring team's win probability to about 60%. That same score late in the fourth moves it closer to 90% or 95%, depending on time remaining, timeouts, etc.

by HostileGospel :: Sun, 09/27/2009 - 9:00pm

Kneeling in the fourth quarter is also more valuable than kneeling earlier in the game.

Overall, I'd be kind of embarrassed to critique something when I didn't know what the hell I was talking about, but then, oh yeah, my NAME is on what I write, isn't it?

-Les Bowen

by Rick (not verified) :: Tue, 09/29/2009 - 1:33pm

Of course that's true - that's just a time differential argument. But it's not an argument about whether or not the job is getting done.

If a QB steps up on the first play of the game and tosses an 80 yard strike for a TD, he just scored in (let's say) 15 seconds. So how is that LESS valuable than a QB who drives his team 40 yards in 1 minute 38 seconds for a winning FG? How is it less "clutch"? The concept is absurd. At any point in time something fluky can happen that allows a QB to be "clutch". The Bills/Pats game in week 1 was a perfect example - Brady was clutch while Edwards was not?

I'm sorry - Brady doesn't have much of a chance to win the game if the return fumble doesn't occur...and Edwards has NO shot without a decent O Line. In that situation, I heard the commentators mention several times "this is why you want a guy like Brady on your team". I laughed so hard when I heard that, because it's nonsense. Sure, the guy's a good QB, but I mean realistically how vastly different is he than Matt Cassel? It seems to me, given Cassel's success last year, that the Patriot scheme and personnel provides huge benefits to any QB. And given the performance of the Bills' O Line in that final drive - how likely is it that Edwards is EVER going to be a "clutch" QB?

by Quasibozo (not verified) :: Sun, 09/27/2009 - 11:22am

Competition should not be casual. It should be fierce. Every minute of the game. Clutch Quarterbacks only exist if Clutch Receivers, Clutch Defensive-Offensive linemen, Clutch Running Backs, and Clutch Special teams exist. Too often we see teams, seemingly running on one cylinder for most of the game, perform miracles in the "two minute drill". All of a sudden they're all running on four cylinders. Why they can't perform as a well oiled machine the whole game perplexes me. They're paid enough money to make the effort. Clutch Coaching staff too! I hereby rename the two-minute-drill the "Clutch Minutes", when everyone is clutch.

by Chris M (not verified) :: Sun, 09/27/2009 - 11:23am

The Eli thing is obvious to anyone who's watched him - hes simply much better in a no huddle offense, and particularly against prevent defenses. Why the Giants don't go no huddle more often is actually an oddity, no?

by Dunbar (not verified) :: Sun, 09/27/2009 - 5:27pm

My own (admittedly unscientific) view on clutchness is that it's not so much about finding a 5th gear in crunch time that you can't otherwise reach; instead, I think it's when players who normally play in 3rd gear or lower recognize the gravity of the moment and shift into 4th. Clutch, in other words, is the realm of the procrastinator. Examples include Robert Horry and (to a lesser extent, since he's always good) Manny Ramirez.

Of course, "clutch" is often more due to luck (see David Tyree) than anything else, but I don't really consider that clutch. People like Horry and Ramirez, though, who pretty clearly don't try as hard until they think it really matters, seem to be fairly repeatably "clutch." I'm basically talking out of my ass on the basis of long-ago-viewed stats, though, so I don't know how much we should trust my opinion.

by Xeynon (not verified) :: Sun, 09/27/2009 - 9:00pm

We _know_ that there are players who have performed at a much higher level in high-pressure situations than others have.

My understanding is that no evidence has ever been presented that "clutch" definitely exists, but there is evidence that suggests that choking does - so what we think of as clutch players may in fact be guys whose performance doesn't suffer under intense pressure, while their opponents' does.

Clutch Quarterbacks only exist if Clutch Receivers, Clutch Defensive-Offensive linemen, Clutch Running Backs, and Clutch Special teams exist

Going to disagree with this. Football demands different qualities from players who play different positions. Quarterback and center are much more mentally demanding positions than running back or defensive tackle, and require a more cerebral type of player. If clutch performance exists, it's likely almost purely a mental phenomenon, so it stands to reason that in some positions it will be more prevalent than others.

Examples include Robert Horry and (to a lesser extent, since he's always good) Manny Ramirez.

Actually, I don't think Horry's a good example - I recall reading an interview in which he himself pointed out that he didn't make potentially game-tying or game-winning shots at any higher a percentage than he did other open shots, and that his "Big Shot Bob" reputation was largely the result of the media focusing on his successes while ignoring his failures, i.e. selection bias.

Ramirez is *always* a very dangerous hitter - my guess is his "clutch" performance is as likely an artifact of choking on the part of opposing pitchers as it a result of improved performance on his. I'd like to see the stats from throughout his career on how he does when facing elite opposing pitchers in clutch as opposed to non-clutch situations.

by Sifter :: Mon, 09/28/2009 - 4:26am

Clutch Quarterbacks only exist if Clutch Receivers, Clutch Defensive-Offensive linemen, Clutch Running Backs, and Clutch Special teams exist

Going to disagree with this. Football demands different qualities from players who play different positions. Quarterback and center are much more mentally demanding positions than running back or defensive tackle, and require a more cerebral type of player. If clutch performance exists, it's likely almost purely a mental phenomenon, so it stands to reason that in some positions it will be more prevalent than others.

You may be right but it would be almost impossible to measure. As I see it, the only time you can see a QB truly being clutch or not is when he has a clear pocket and an open receiver. Otherwise his failure is due to other circumstances, a bad line, or excellent coverage (or poor route running) or poor play calling. So if you weed out all those plays and only select the ones where he has an open receiver and time to throw you could measure it against a standard and then see how his stats go in the clutch. To my mind it's much easier to measure a 3pt shooter's clutchness for example. There are less factors that he can't control.

And just let me add, I don't believe in clutchness I only believe in chokes usually in 0.0001% of players who don't have the mental toughness to be there in the first place.

by Pat (filler) (not verified) :: Tue, 09/29/2009 - 2:48pm

"So if you weed out all those plays and only select the ones where he has an open receiver and time to throw"

It's impossible to do what you're asking, because you *still* don't know what the quarterback was supposed to do.

What if the receiver runs the right route, but the quarterback makes the wrong read and throws to a different spot? How do you tell the difference between that and the receiver running the *wrong* route? How do you tell the difference between when the quarterback and the receiver physically see different things, and so are *both* right, but the result of the play is wrong?

You can't purely isolate performance like that in football. Obviously. How is a quarterback who has plenty of time to throw and makes the proper read just as clutch as the one who has half the time to throw and still makes the proper read? The *first* QB was under less pressure than the second - since clutch is performance under pressure, by definition the *second* QB should be more clutch than the first.

Rich said it best above: clutch is a crap concept.

by Sifter :: Tue, 09/29/2009 - 6:02pm

I totally agree with you. That's why I mentioned a 3Pt shooter in comparison. They have a hoop, that doesn't move and they just have to put it in. Sometimes there will be pressure from a defender, but that would be the only thing that would need to be accounted for. Football is WAY too complex with too many moving parts to single out even an open pass by a QB as you rightly point out.

But back to basketball for a sec. In those 3PT competitions at the All-star weekend they have balls worth one and then the last on the rack is worth 2. That's a 'clutch' shot ie. added importance to the shot - anyone know if the 2pt balls are sunk more often/less often?

by Joseph :: Mon, 09/28/2009 - 1:21am

I agree with #5 "Tracker". I think that the best way to measure "clutch" is to find players who do at least as good as normal in "late & close" situations. I think you could also include in a "clutch" study all 3rd and 4th down attempts, as well as inside the last two/three minutes of the first half.
Regarding whether the score is more valuable relative to the time on the clock: If you are down 7-0 as a result of the other team taking the opening kickoff and driving down for a TD, you as a fan nor any player or coach is worried--there are still many opportunities to score. If you are down 21-14 with 3 minutes left, 1 of your TD's was a non-offensive score, and your offense is 2-12 on 3rd downs, you are probably a LOT more worried. Less time remaining on the clock means less possession opportunities, means less time to play "field position", means the other team changes it's strategy to better use up the remaining time, etc.
The later the go-ahead score is, the more likely you are to win the game--it's that simple.
It also allows you to dictate end-game strategy. In baseball--THERE IS NO END-GAME STRATEGY--YOU STILL HAVE TO GET ALL 27 OUTS. As an extreme example--Miami/Indy game from last week. Indy made the most of it's very limited possession time. It was great strategy by Miami--and nearly resulted in a victory. I can guarantee that, if the Colts have the ball 45 minutes and Miami only 15, the final score is VERY different. That's why the baseball analogy is incorrect--dominating the "time at bat" does not deprive the other team of any of its outs. They still get the same chances.

by mrparker (not verified) :: Mon, 09/28/2009 - 8:41am

A study at advancednflstats.com found that the team who is leading after 1 quarter wins at a very high percentage(the exact percentage escapes me). Lets just call it 75% for the sake of argument though I think it is a higher number than that.

In that context Eli's performance in the superbowl does not look so clutch. And neither does Big Ben's. They merely did what they were supposed to do 3 out of 4 times. However Tom Brady's performance in the Rams Superbowl looks pretty good. They were down 3 points after the 1st quarter.

Since that only happens 25% of the time, yet the same teams/players always seem to be the ones to be able to do it, its those performances that I think have the meaning that we're searching for when we search for clutch.

by C (not verified) :: Mon, 09/28/2009 - 8:59am

I think the guy who pointed out Eli being better in the "muddle huddle" hurry up no huddle offense anyway is on to something. The guy just performs better in that setting at the end of the half, whenever they run it, AND at the end of games.

Clutch QB: John Elway
Not Clutch QB: Mcnabb

I know the Mcnabb comment will piss people off and I do think he's a good player and one of the best, but clutch he is not.

by Rick (not verified) :: Tue, 09/29/2009 - 1:24pm

As I point out in a later response, McNabb has had a few clutch games. I remember one against Pittsburgh, in his first or second year in the league, as well as the one famous "4th and 26" play against Green Bay. That I can think of few others is hardly an argument against him, it's more an argument against the concept of "clutch".

by beargoggles :: Tue, 09/29/2009 - 1:41pm

He also delivered a strike on 4th down last year in the Cardinals to keep the drive alive, except either the receiver fell or there was uncalled pass interference.

I would guess that a lot of it is based on how the offense runs as discussed: Eli Manning and Elway were better suited by that form of offense that runs in the two minute drill. I also think that there are chokers (A Rod, early Bonds), but that's just my personal opinion. In any case, the media tends to way overplay small sample sizes and these things are exceptions rather than rules. I also do agree that statisticians need to be very careful about making conclusions about things that they don't have enough information to make conclusions about. If choking/clutch does exist, it will be very hard to prove it because of sample size and definitions.

by Xeynon (not verified) :: Mon, 09/28/2009 - 10:19am

You may be right but it would be almost impossible to measure. As I see it, the only time you can see a QB truly being clutch or not is when he has a clear pocket and an open receiver.

A good point. It's interesting to note that Tom Brady was suddenly no longer Mr. Clutch when he was asked to win playoff games in the fourth quarter with poor receivers ('05, '06) or line play ('07). And it's hard to call Eli's Super Bowl performance "clutch" when his team would have lost the game if not for Asante Samuel whiffing on an interception and a wild heave-ho that probably succeeds 1 time out of 50 if you re-run it somehow coming down pinned to David Tyree's helmet.

I know the Mcnabb comment will piss people off and I do think he's a good player and one of the best, but clutch he is not.

Let me be the first one to take issue with this, which is the kind of substance-free jab you hear on sports talk radio all the time. In what way is McNabb "not clutch"? Please provide some evidence, statistical or otherwise, for this assertion. The fact that he failed to lead game winning playoff scoring drives on a few occasions does not count - Tom Brady, Peyton Manning, Drew Brees, Philip Rivers, and every other elite quarterback in the league right now have likewise failed on a few occasions. Unless you are willing to concede that none of them are clutch, you can't say McNabb isn't clutch either.

As for Elway, despite his numerous late-game scoring drives he was considered a big game failure because of his poor performance in his first three Super Bowl appearances for most of his career. Once he had Terrell Davis to hand the ball off to and Rod Smith and Shannon Sharpe to throw it to, he magically got over the hump and achieved unqualified clutchness in the popular imagination. More likely, he lifted inferior teams into the Super Bowl early in his career and was finally properly equipped to win it once he had better complementary players around him.

by C (not verified) :: Tue, 09/29/2009 - 12:13pm

On the flip side, do you think Mcnabb is clutch?

Do you think he gets better in big games or with the game on the line?

Do you think John Elways was not clutch?

by Rick (not verified) :: Tue, 09/29/2009 - 12:15pm

It's worth mentioning that McNabb did deliver a 4th down strike to Freddie Mitchell to keep the Eagles in the playoffs for another round....and while that play didn't WIN the game, it did prevent them from losing (actually he delivered that pass twice, if I remember correctly, due to a whistle or some such).

I'm a McNabb fan, and I take issue with the term "clutch" only because the "clutch" moment can occur many times in a game.

"Clutch" moments can occur often in the beginning of a game, particularly if it seems like something may get out of hand quickly. The Eagles, playing against New Orleans 2 weeks ago, tie the game on a nice pass to DeSean which goes for a long TD. If he doesn't make this (or any one of several other plays in the first half), the Eagles are probably down 2 TDs (based on how well Brees is playing) and Kolb is fighting for his life. Well, that happened ANYWAY, but not necessarily because of Kolb, more due to an untimely fumble on a return that altered the course of the game.... point is, if the game has the potential of falling into a state of disarray early, a "clutch" moment can occur early - but how can you define it?

Meanwhile, this past week, the Giants utterly destroy the Bucs. Had Leftwich pulled a rabbit from the hat early in the game, it may have been a closer match that we all watched. Would that have made him "clutch"? In most books, people would say no. But altering momentum is a "clutch" move, isn't it? Why are final drives considered the ONLY "clutch" moments?

The Linc, several years ago, Eagles leading 24-7 (I still shudder, since I was there that day), Westbrook fumbles the ball, leading to a massive shift of momentum. Eli starts throwing little out patterns to Plaxico, marching them down the field. Then he has one toss that was desperate which leads to a winning TD. Was Eli "Clutch"? I'd say not, but the job got done...mostly because of a height differential in coverage, a fumble, a minor shift in gameplan, and a lucky toss.

How you define "clutch"? I just don't see how you can.

by Martial (not verified) :: Mon, 09/28/2009 - 1:54pm

There is such a thing as “social science”. We social scientists can actually learn some amazing things by listening to people. For example, by asking enough people to tell us what they think about certain situations in a football game we could come up with a definition of “clutch situation” that is significantly more robust than “late and close”.

There is a trick to designing and then doing worthwhile and rigorous social research (i.e. not everyone can do it), but good qualitative studies could add to what FO is learning through its quantitative methods. The chief obstacles are, as usual, time and money. A single obsessed researcher can’t sit up nights in front of a computer and do regressions. You have to go out and talk to a lot of people across a lot of contexts. (I do a type of social research globally and my travel budget can be killer.)

That said, the internet allows for certain types of preliminary data collection. Hmm, I’m having a few ideas. Any social science grad students in a qualitative program out there who want to apply their listening skills to football? I’d be happy to chat about it.

by Tommy k (not verified) :: Mon, 09/28/2009 - 1:59pm

There is one shining example of being "un-clutch". The ultimate un-clutch player would be Peyton Manning in nearly every big playoff or bowl game he has been in dating back to college.

by DrewTS (not verified) :: Mon, 09/28/2009 - 5:22pm

I give this trolling a C- for effort, and an F for originality.

by daddyoscar (not verified) :: Mon, 09/28/2009 - 5:28pm

clutch qb's need clutch defenses clutch fg kickers and clitch receivers

without vinatieri the pats win zero superboiwls IMO

big ben needing one or two big plays a game then having the defense hold the lead after he scores with 3-4 minutes to go iunlike the last two weeks

clutch qb's are good qb's with good supporting casts

by MC2 :: Mon, 09/28/2009 - 7:29pm

I don't believe that there is any such thing as "clutch", other than a crutch that is used by clueless sportswriters to cover up their inability to provide meaningful commentary.

There is, obviously, such a thing as "choking", but it is irrelevant in pro sports, since anyone who consistently chokes will be weeded out long before they get to that level.

by The Irishman (not verified) :: Tue, 09/29/2009 - 2:12am

As someone pointed out above about Eli Manning running a no huddle, the other complicating factor with QB's in "clutch" situations defined as late and close is that offensive and play-calling can change.

There's a view that during the Reeves - Elway era in Denver, Denver's play-calling was too conservative during the first 3 quarters of games, which is a big reason that Elway led so many memorable 4th quarter comebacks. That's not knocking Elway and saying that other quarterbacks would necessarily have succeeded as often as he did. It is knocking the Denver coaching staff and saying that they didn't call plays that fully-utilized Elway's talents throughout the game. I haven't ever seen a statistical analysis that would confirm or refute that view, however.

by Rick (not verified) :: Tue, 09/29/2009 - 12:23pm

I agree with this totally. Game changing situations change game plans. Changing game plans MAY or MAY NOT suit the needs of the players on the field, particularly the QB.

Can a RB or a WR be "clutch"? Can an O Line be "clutch"? 2 weeks ago, Buffalo has a shot to knock off New England, and a fumble gives Brady the "right to be clutch" because he makes a go ahead TD. Then Trent Edwards is given the chance to be "more clutch than Brady". As I'm watching the game, I turn to my wife and say "Edwards is a good QB, but how clutch he is will be determined by his O Line". Sure enough, he gets sacked. Twice. One incomplete due to pressure. Is Edwards "unclutch"?

I'm sorry, just not buying into this "clutch" thing. Makes absolutely no sense to me until it is fully defined.
"Must be in the final 3 minutes of a game, down by less than a TD, with a running back and wide receiver who are more than adequate and an O Line that is average at best". MAYBE THEN you could determine if the QB is "clutch" because he's working on time constraints and surrouned by players who are only slightly above average. Anything he accomplishes would be amazing.

Oh yeah, and you're Irish, so you must be smart. Or something.