Writers of Pro Football Prospectus 2008

16 Dec 2010

Which QBs Handle The Blitz Best?

Our friend J.J. Cooper at Fanhouse has a look at the way quarterbacks are handling the blitz this year -- and whether the blitz really changes things at all.

Posted by: Bill Barnwell on 16 Dec 2010

39 comments, Last at 19 Dec 2010, 4:29pm by otros

Comments

1
by Yaguar :: Thu, 12/16/2010 - 11:09pm

Woah, it's ridiculous how close the rate stats are.

2
by BSK (not verified) :: Thu, 12/16/2010 - 11:15pm

Honestly, I think we'd need to see the splits for this based on context. Personally, I think teams blitz too much and, more importantly, often in the wrong situation. I would like to see is there a situation where you definitely should blitz or definitely should not blitz?

10
by Israel P (not verified) :: Fri, 12/17/2010 - 6:46am

Since a large part of "a situation where you definitely should blitz or definitely should not blitz" is based on what the offense expects, following a formula or a fixed set of rules seems counter-productive.

3
by drobviousso :: Thu, 12/16/2010 - 11:22pm

Whoa. Optimal mix!?!

4
by tally :: Fri, 12/17/2010 - 1:24am

That's the real implication, right?

The observation that blitzed and non-blitzed QBs perform about the same suggests that blitz frequency is close to optimal.

If, on the other hand, non-blitzed QBs perform a lot better than blitzed QBs, that would suggest that defenses should blitz more (or vice versa), but that's not the case here.

Too bad the article fails to draw this conclusion.

5
by some guy (not verified) :: Fri, 12/17/2010 - 2:31am

That's not true at all. If, purely for the sake of explanation, a blitz makes a quarterback 50% worse when it is used sparingly but it loses it's effectiveness as you increases its usage, the optimal mix would NOT be to use the blitz until its no more effective than a non-blitz, you've just ruined your blitz. Football Outsiders linked these articles awhile back, they arent about football but they explain this far better than i could:
http://gravityandlevity.wordpress.com/2009/11/30/an-example-of-braesss-p...
http://gravityandlevity.wordpress.com/2009/05/28/braesss-paradox-and-the...

6
by Kulko :: Fri, 12/17/2010 - 4:29am

Sorry, but this posts have an very obvious fallacy, which makes there result simply wrong.

When Patrick takes 50% of the shots his efficiency doesnt fall because of some mysterious power, but because the Def is concentrating on him. Therefore the Def is leaving his teammmates open, and there efficieny must therefore rise. When you consider this it becomes obvious, that the Team efficiency is above 45% at the Nash equlibrium, and very likely also above 48%.

7
by Jim Glass (not verified) :: Fri, 12/17/2010 - 5:21am

Kulko is right, and the guy admits it when after somebody raises this point in the comments he says he assumes the other players' shooting percentages "don't change much" when they face extra defense, only Patrick's does. Which is hooey.

34
by some guy (not verified) :: Sat, 12/18/2010 - 7:18pm

I think you are missing the forest for the trees but the point of the post was that giving more equal shot attempts to the other players improves the team play. Why more equal shot attempts would draw extra defenders defy's me.

37
by drobviousso :: Sun, 12/19/2010 - 1:02am
32
by drobviousso :: Sat, 12/18/2010 - 1:27pm

While the other critiques of your link are correct, you have backed into an important strategic point. By using a non-optimal mix of blitz/non-blitz, you can sacrifice overall efficiency for a better weapon in "high-leverage" situations, like 3rd and long when you can pin your opponent deep, or late in a game on potential game winning drives.

31
by drobviousso :: Sat, 12/18/2010 - 1:23pm

Well, the optimal mix is league wide. For many QBs, some are very clearly blitzed too much or too little. I wonder if that's noise, or if there some play callers don't adjust enough for opponent match ups. If play calls use the 'by the gut' method, the caller may have developed a feel for what works in general over many year, and my just go with it no matter who they line up against.

Or it could be something else. Who knows.

33
by some guy (not verified) :: Sat, 12/18/2010 - 7:13pm

What I was trying to say was that the optimal mix, if we assume that blitzes will yield worse efficiency for quarterbacks when the blitz is used rarely, would have the blitz as considerably more effective than non-blitz league wide. It might be that there are one or two teams that blitz far too much or it is a league wide trend. There are far to many variables i guess with the league wide data however so i guess you can't really make any conclusions.

38
by drobviousso :: Sun, 12/19/2010 - 1:11am

Any strategy, when used rarely in a dynamic equilibrium environment, will yield better results than if it is used often. Blitz, nickel, 11 angry men, doesn't matter.

The fact that an apparently optimal mix is observed, (i) along what we assume are decision making lines used by play callers, (ii) precisely because there are far too many variables for us to digest and (iii) even if it is league wide and doesn't hold true for all QBs or defenses, is an interesting observation all by itself. It implies a certain level of 'efficiency*' that we generally don't assume a non-market based system, like the NFL competition, should produce very often.

If any of the simulations I wrote in grad school produced spontaneous optimal mixes, I probably would have ended up published and working in academia, instead of slogging it out turning data and time into real, utilitarian value for my firm's clients. You could spend a lifetime trying to figure out "why?"

*As in efficient market theorem efficiency, not making-first-downs efficiency.

8
by Jim Glass (not verified) :: Fri, 12/17/2010 - 5:30am

Anybody who concludes that rookie and novice QBs handle the blitz best, and experienced top QBs -- Tom Brady as an example! -- handle it *worse* than the novices, really should re-examine his numbers and come up with a clear explanation for why it is so. Because either he's discovered something that should win him a big prize or he's made a howling mistake.

It's one thing to speculate that DCs are systematically stupid and over-blitz novices to make their life easy -- but what is the explantion supposed to be for Brady and the other top QBs being *worse* at handling blitz than the kids?

9
by tuluse :: Fri, 12/17/2010 - 6:08am

He interprets his data pretty wrong in a few cases I can see. 1) He just looks at raw QB rating delta--ignore for a moment the flaws of using QB rating--and doesn't adjust for what the rating is. Max Hall increases his rating from 35 to all the way to 59, in other words, still terrible. While Tom Brady falls from 110 all the way to 99, or still amazing. 2) He doesn't account for sacks probably. Hall goes down 1 out of every 5 times you blitz. So sure he might be more efficient on the other 4 passes when you blitz, but you get a sack 20% of the time.

Also, I'm not sure his data is correct. He has Brady being sacked 22% of the time when not blitzed, which sounds wrong to me.

The 2nd table is more interesting to me. Trent Edwards got blitzed on 92% of his drop backs!

11
by DavidL :: Fri, 12/17/2010 - 9:53am

If you look at the choice as "blitz or don't blitz", leaving the specific play out of it, delta is more important than objective rating. If Brady's efficiency does in fact fall 9% against the blitz, you want to blitz him somewhat more; yes he's still good, but he'll be less good, and that means your defense is working. If Hall's efficiency nearly doubles, you want to blitz him a lot less. It doesn't matter that he's going to suck no matter what, because you want him to suck more. Just like you want Brady to be less good, even if you can't make him outright bad.

22
by tuluse :: Fri, 12/17/2010 - 3:04pm

I'm not sure you can treat QB rating that way. Also, I think it's a lot easier to improve from ~30 to ~60 than ~100 to ~110. Small sample sizes also play a large part in this.

12
by JSA (not verified) :: Fri, 12/17/2010 - 9:59am

Yeah, some of that sack percentage data just seems off. For example he has Eli Manning as being sacked 12 times against non-blitz, for a sack % of 34.42% Seems hard to believe Elisha has passed and not been blitzed only about 35 times all season.

I'm not sure he makes the interpretation you say he makes though. I don't see him saying Hall handles the blitz better than brady, it's obvious he doesn't. I think he draws the conclusions the data suggests - that Hall, for example, is better against the blitz than not against the blitz, and Brady is worse against blitz than against non-blitz. He doesn't argue that Brady is worse against the blitz than Hall, nor could he.

15
by The Powers That Be :: Fri, 12/17/2010 - 10:42am

The Non-Blitz Sack% column is completely messed up. Notice that the numbers in there aren't even listed as %ages (as they are in the Blitz Sack% column). They aren't the deltas between blitz and non-blitz sack rates, either. I think he just copied over a completely different column of data.

18
by DZ (not verified) :: Fri, 12/17/2010 - 12:15pm

Yeah, the 'non-blitz' column is actually the player's yearly totals

The whole thing needs a rewrite. Sloppy job

20
by JonFrum (not verified) :: Fri, 12/17/2010 - 2:32pm

Are you talking about the column with decimal values in it? That's not totals.

13
by mrh :: Fri, 12/17/2010 - 10:28am

I suppose this is obvious but here's a key flaw with using passer rating in this analysis. Suppose it's 3rd and 12 on the offense's 20 yard line, 1st qtr, no score. The DC calls a blitz not just to sack the qb but also to force the QB to make a quick hot read throw underneath. The pass coverage is designed to accept a short completion but make a stop short of the sticks. Obviously the D has to execute, just like the QB.

If the QB completes the pass but the D makes a tackle, say a gain of 10 yards. The QB's passer rating for that play is 108.3, which looks good but it's a failed play as the offense will now punt in most instances.

Meanwhile, a QB on 3rd-and-3 in the same game situation goes unblitzed, completes a pass for 4 yards and a first down. Passer rating = 83.3. So in this analysis, passer rating makes it look like the QB performed better and blitzing performed worse, but in fact it was the unblitzed qb who did better - and the D did worse. It also doesn't mean the decision to forego the blitz in the 2nd case was wrong.

28
by Bobman :: Fri, 12/17/2010 - 6:58pm

Honestly, is there really THAT BIG a delta in those two passes?

Man, that system truly does suck.... The 108 might bre fair, depending on the pattern and the receiver--was he set up for YAC and failed, or was it a come-backer or a diving play where he had no chance for a YAC and a conversion? But such a mundane rating for a completion for a 1st down (just because it was short?)... that's just heinous.

14
by matt w (not verified) :: Fri, 12/17/2010 - 10:38am

I don't think he's asking the right question -- he's putting everything on the QB, as opposed to the o-line and the people who are supposed to pick the blitz up.

That's pretty common, I guess, but for some reason it especially bothers me here.

16
by CraigInDC :: Fri, 12/17/2010 - 11:15am

He kind of gets to that point in the article when he notices that teams with bad OLs get blitzed more often. Although, that even seems a little counter intuitive to me. If the offensive line is bad, then your normal rush should be more effective, so you shouldn't have to blitz as much. On the other hand the Steelers OL was particularly bad at picking up blitzes early in the season. It seemed to have to do with the guards not being very quick (both mentally and physically) so they couldn't adjust when a stunting player came into their zone. However, since Max Starks got hurt and Jonathon Scott is the left tackle, no one really needs to blitz. You can almost have an effective defense based on holding penalties on Scott.

17
by Bobby Wommack (not verified) :: Fri, 12/17/2010 - 12:04pm

So if Max Hall gets blitzed and simply checks down to his back two out of three times, does that mean he's successful against the blitz? Cause from watching someone like Sanchez all year, I can tell you, he loves checking down to LT whenever any pressure comes remotely close.

19
by Jim Glass (not verified) :: Fri, 12/17/2010 - 2:21pm

Another problem, related to using the awful passing rating number. He ignores the high risk-reward payoff of beating the blitz.

E.g., the Jets are the biggest blitzing team in the league. They are at the top of the league in allowing the lowest completion percentage to opposing QBs, but at the bottom of the league in allowing the most yards per completion.

And the passing rating system so over-weights completion % that a QB who beats the blitz with big passes is penalized.

Take two QBs who both hit 2 of 3 passes at 7 yards an attempt. That gives a rating of 87. Say QB1 when faced with three blitzes throws three safe checkdowns for 5 yards each, 3 of 3 for 15 is a rating of 88, up a bit -- even though his yardage falls to 15 from 21, down 29%. QB2 in response hits only 1 of 3 passes but by going deep for 30 yards. His rating plunges to 72 -- even though his yardage is 30 up from 21, up 43%, and double QB1's.

So QB1 is deemed to be beating the blitz, his rating up to 88 even though his yardage is down 29% -- and QB2 is deemed to be failing against the blitz, his rating down from 87 to 72, even though QB2 has *double* the yardage of QB1 and 43% more yards than against non-blitz plays!

Maybe this explains why by those numbers Favre and Brady supposedly do worse against the blitz and naif beginners do better?

The passer rating system is total crap and shouldn't be used for anything, but especially shouldn't be used for this analysis.

27
by Bobman :: Fri, 12/17/2010 - 6:54pm

But did this study compare Player A against Player B? I read it as more Player A in one set of circumstances vs himself in another set (the blitz). Their differentials can be compared from player to player, and I agree THAT is terribly flawed.

But the comparison for a single guy is probably useful--all the other variables are the same--teammates, opponents, playbook, etc. A bit like using the same rating to measure how well a guy is playing before and after a coahcing change, or an injury, etc. If all the other potential variables are held constant, it's probably an adequate measure of the one variable you are studying (in this case a blitz). Yes, DYAR/DVOA etc might be better, but using the stupid traditional rating system should be adequate to measure if a guy is doing better/worse/same under different conditions.

I don't really care how my guy stacks up against everybody else (he's better; I already KNOW that!) but I DO want to know where his flaws are and if there is a big delta in his blitz numbers--regardless of the metrics used to derive it--then it's worth digging further into to find out why, IMO.

29
by RichC (not verified) :: Fri, 12/17/2010 - 10:13pm

"Take two QBs who both hit 2 of 3 passes at 7 yards an attempt. That gives a rating of 87. Say QB1 when faced with three blitzes throws three safe checkdowns for 5 yards each, 3 of 3 for 15 is a rating of 88, up a bit -- even though his yardage falls to 15 from 21, down 29%. QB2 in response hits only 1 of 3 passes but by going deep for 30 yards. His rating plunges to 72 -- even though his yardage is 30 up from 21, up 43%, and double QB1's.

So QB1 is deemed to be beating the blitz, his rating up to 88 even though his yardage is down 29% -- and QB2 is deemed to be failing against the blitz, his rating down from 87 to 72, even though QB2 has *double* the yardage of QB1 and 43% more yards than against non-blitz plays!

"

Consistently getting first downs correlates highly to points. Passer rating is doing it right here. QB 2 is failing against the blitz, because those incompletions count, and can kill drives.

IE, if you had a QB who could get you 5 yards consistently every play, you'd win every game. You'd have the best offense in NFL history.

If you had a QB who got you 100 yards every 20th attempt, you'd have one of the worst offenses in league history.

35
by Jim Glass (not verified) :: Sat, 12/18/2010 - 10:46pm

Consistently getting first downs correlates highly to points. Passer rating is doing it right here

Bunkum. Passing rating is *so bad* that it gives a 79 to any completed pass that *loses* yardage, no matter how much, even 99 yards backward. How does that help get first downs? Change the example to...

"QB1 when faced with three blitzes throws three panic passes that lose 10 yards each -- he *loses* 30 yards total. QB2 in response to the blitz hits 1 of 3 passes but by going deep for 30 yards. His rating plunges to 72, because although he made +30 yards, he hit only one of three passes.

"QB1 loses 30 yards, rating 79. QB2 makes 30 yards, 60 yards net more. But his rating is only 72. So passer rating tells is QB1 with his -30 yards is *better* against the blitz!"

What is passer rating doing right there? By rewarding passes that lose yards and punishing incompletions that don't? If you have a QB who loses five yards every play forever, how many first downs and wins are going to come with his passer rating?

Passer rating is total crap.

39
by otros :: Sun, 12/19/2010 - 4:29pm

I agree, QB rating is bad. And it is especially bad when you're using it to rate one or two passes. But, i don't think that is the expected use of the QB rating.

30
by Scott C :: Sat, 12/18/2010 - 3:45am

Completion % is still useful. 1 for 3 for 30 yards is not as good as 3 10 yard completions on average.

But that is situational -- if its third and 15, a 33% chance at 30 yards is better than a 100% chance at 10. There have been proposals for 'fixed' QB ratings that use yards per attempt only and drop completion %. But that is not good enough -- consistency has value. Perhaps getting rid of the 'double counting' of completion percentage in qb rating by changing from yards/attempt to yards/completion and tweaking the weights of that and completion percentage would result in a metric that correlates better with success.

36
by Jim Glass (not verified) :: Sat, 12/18/2010 - 10:55pm

"Completion % is still useful. 1 for 3 for 30 yards is not as good as 3 10 yard completions on average."

But 1 of 3 for 30 is a lot better than 3 of 3 for 15, which is the example I gave. Average yards per attempt *dominates* completion percentage when correlating to win%.

And passer rating rewards completions that *lose* yards. It gives hitting 3-of-3 for *negative* 30 yards a rating of 79, and 0-of-3 for 0 yards a rating of 0. Would you really be that much happier losing 30 yards rather than none?

21
by JonFrum (not verified) :: Fri, 12/17/2010 - 2:40pm

I'd rather see completion percentage and first downs gained than QB rating - which is a clusterf*ck of data.

23
by GoDillos (not verified) :: Fri, 12/17/2010 - 5:48pm

After reading this site for so many years, I am constantly surprised that people actual use the standard QB rating. I mean complete honest surprise. Like, I will see it pop up during the TV broadcast, and think ... oh yeah I forgot that stat existed.

24
by tuluse :: Fri, 12/17/2010 - 6:04pm

Adjusted net yards per attempt would have been nice, and easy to calculate. Would have been fine for a quick and dirty look at performance. A stat that accounts for down and distance would be even better.

26
by Bobman :: Fri, 12/17/2010 - 6:48pm

I think passer rating is an okay proxy in this case because, despite its flaws, it's a general measure and if we're comparing it against itself for the same guy on the same team, it's probably accurate enough to say "He's doing better, worse, or about the same under the two different scenarios."

I don't think you can compare Player X on one team with Player Z on another team, because you're talking about too many different variables--playbook, surrounding talent, OL, different defenses faced, etc. But using it to compare my own performance with and without blitzes... yeah, that's fine with me.

25
by Bobman :: Fri, 12/17/2010 - 6:43pm

I'm also not entirely sure this is a mreasure of QBs, as opposed to their lines and their RBs picking up blitzers.

For a moment I will don my royal blue colored Colts glasses. Putrid OL, which is better at pass blocking than run blocking, but not by a whole lot. Let's call them avreage at best. And their best blitz pick-up RB has been out the last six-seven games, and the All Pro TE safety blanket is on IR. I'd say blitzing Manning is a great idea right now. Hear some more of those "Goddammit Donald!" quips. Or Mike or Javarris or whoever else is suited up in the backfield. I know if my QB didn't have a rotation of zombies on the OL and his main RB in the backfield, he'd be a lot better vs the blitz.

Longitudinally, I'd like to compare Peyton Manning's 2010 stats vs seasons when he's had a decent blocking RB and TE playing--I bet he suddenly "improves" how he deals with the blitz. (And other QB's as well) That's one reason the Colts relied so hevily on James his whole career, because he was so much better at picking up rushers than Rhodes, who was an okay runner and receiver. Same for Addai over Brown.