Trying to Improve Sack Rates

Trying to Improve Sack Rates
Trying to Improve Sack Rates
Photo: USA Today Sports Images

by Scott Kacsmar

Mainstream acknowledgement of sacks as a function of quarterback play has been a slow process, but nearly every advanced metric from DVOA to Win Probability Added to QBR factors in sacks. They are, after all, an attempt to throw the ball, and usually have a more negative impact than an incompletion.

How often have you heard an analyst revel about the way quarterbacks like Aaron Rodgers and Andrew Luck use their mobility to escape pressure and avoid sacks? Now what if we could quantify this with a number each season?

I actually did try that for Luck's rookie season, coming up with an absurd -- or is it? -- total near 70. Luck's actual sack total was 41. Could he really have been sacked nearly 111 times? Were the Colts like the 1986 Eagles, who did allow 104 sacks? This counted plays where Luck absorbed contact in the pocket and either got rid of the ball or scrambled. It also included plays where he was hit while throwing, which happened 51 times if you believe ESPN's data. Suddenly, 70 does not sound so hard to believe. But in this context, the number is meaningless without anything to compare it to.

Clearly, that stat could use a ton of work before going live. While this does tread down the path of subjective statistics, we still can do a better job of expressing a quarterback's sack rate.

Sack Rate: Flawed, Misunderstood and Dominated by One Player

Historically, sack rate has been nothing more than sacks divided by the sum of pass attempts and sacks. Matt Ryan threw 615 passes and was sacked 28 times in 2012. He had 643 pass plays. His sack rate (28/643) was 4.35 percent.

Game Rewind: Relive every NFL moment…subscribe to Game Rewind.

However, we have a calculation with a phony denominator. It's not accurate to say Ryan only had 643 pass plays last season. The pass ratio in the NFL is always misleadingly lower than the true number of pass plays. This is due to plays that start as passes turning into statistical runs, such as a scramble or a pass thrown laterally. Runs practically never turn into passes, because the play would likely be nullified by penalty for offensive linemen being illegally down the field.

Despite having a simple calculation, sack rate is not a stat you often hear -- actually, do you ever hear it anywhere? -- discussed in NFL coverage. For example, Eli Manning led the league in lowest sack percentage (3.42 percent) in 2012, yet when does anyone ever bring that up? (Note: of course ESPN Stats & Info just had to tweet this after I started the rough draft of this article.)

Oddly enough, those mobile quarterbacks known for escaping defenders are often among the league's worst players for accumulating sacks. John Elway and Dave Krieg retired after 1998 as the most sacked quarterbacks ever, and that is only because the league does not count the sack totals Fran Tarkenton had in 1961-62. Randall Cunningham falls right behind them and is the most notable player with a sack rate over 10 percent. Some called Roger Staubach "The Dodger" in Dallas, but that may have been heavily influenced by his rhyming name and highlight syndrome. Staubach had a career sack rate of 9.57 percent.

It's always entertaining to hear announcers talk about "how hard it is to sack" Ben Roethlisberger, who has an 8.38 percent sack rate and has gone down 344 times since 2004. Actually, he's fairly easy to sack behind that Pittsburgh offensive line. Bringing him down is what's hard sometimes. Someone like Ryan is hard to sack, but easy to bring down. That subtle difference in semantics matters.

Pocket passers do tend to rule sack rate, with Dan Marino and Peyton Manning currently the top two at 3.13 percent. The numbers were practically identical, yet one of these players stands out much more than the other.

Marino led the league in lowest sack rate 10 times, including seven straight seasons to start his career (1983-89). That's phenomenal, and I challenge anyone to find a NFL stat that one player has led the league in more total times (10) or more consecutive times (seven). Even Don Hutson only lead the league in touchdown receptions nine times, and that was a much smaller league. Marino's dominance of sack rate is unprecedented, which is a testament and statistical proof of his lightning-quick release.

It also makes a good argument that sack rate is just as much of a legitimate quarterback statistic as something like completion percentage.

2012's "True" Quarterback Sack Rates

Now if we adjusted Marino's sack rate the way we are about to do for 2012's quarterbacks, there likely would be no noticeable change given how he rarely ever registered a rush that was not a kneel down.

Using regular season data from the game charting project, we added scrambles (SCRAM) to the total number of pass plays (TOT Passes) to calculate the True Sack Rate, which is what the players are ranked by. This was done for all 32 qualified (minimum 224 attempts) quarterbacks in 2012 along with both San Francisco passers, who just so happened to each notch 218 attempts. While scrambles are not always 100 percent differentiable from designed runs, they are usually clear to pick out.

(We are calling this stat "True Sack Rate" for now because we already have a stat called "Adjusted" sack rate, which is sacks and intentional groundings per pass play adjusted for situation and opponent. This stat isn't adjusted for situation or opponent, but that stat isn't adjusted for scrambles. Also, this stat doesn't include intentional groundings, but we do plan on including those plays when we next revisit the concept of "True Sack Rate.")

The other interesting piece of information obtained from game charting was how many of the scrambles involved the quarterback being under pressure (PRES). Sometimes the hole just opens up and the running lane is there for decent yardage, especially for a mobile quarterback. We found that 54.2 percent of the league's quarterback scrambles last season were the result of pressure. That's lower than I expected, but pressure is very much an "in the eye of the beholder" statistic.

True Quarterback Sack Rates (2012)
Rk Quarterback Passes Sacks Sack% Rk SCRAM PRES Pct. TOT Passes True Sack Rate
1 Eli Manning 536 19 3.42% 1 9 5 55.6% 564 3.37%
2 Peyton Manning 583 21 3.48% 2 5 3 60.0% 609 3.45%
3 Drew Brees 670 26 3.74% 3 3 1 33.3% 699 3.72%
4 Matthew Stafford 727 29 3.84% 4 19 7 36.8% 775 3.74%
5 Tom Brady 637 27 4.07% 5 4 4 100.0% 668 4.04%
6 Matt Ryan 615 28 4.35% 6 20 6 30.0% 663 4.22%
7 Josh Freeman 558 26 4.45% 8 19 7 36.8% 603 4.31%
8 Carson Palmer 565 26 4.40% 7 4 3 75.0% 595 4.37%
9 Matt Schaub 544 27 4.73% 9 2 1 50.0% 573 4.71%
10 Brandon Weeden 517 28 5.14% 10 14 9 64.3% 559 5.01%
11 Tony Romo 648 36 5.26% 11 9 6 66.7% 693 5.19%
12 Ryan Fitzpatrick 505 30 5.61% 12 31 15 48.4% 566 5.30%
13 Christian Ponder 483 32 6.21% 16 40 21 52.5% 555 5.77%
14 Sam Bradford 551 35 5.97% 13 19 10 52.6% 605 5.79%
15 Andrew Luck 627 41 6.14% 14 35 22 62.9% 703 5.83%
16 Matt Cassel 277 19 6.42% 18 16 8 50.0% 312 6.09%
17 Ben Roethlisberger 449 30 6.26% 17 12 9 75.0% 491 6.11%
Rk Quarterback Passes Sacks Sack% Rk SCRAM PRES Pct. TOT Passes True Sack Rate
18 Joe Flacco 531 35 6.18% 15 3 3 100.0% 569 6.15%
19 Colin Kaepernick 218 16 6.84% 20 24 13 54.2% 258 6.20%
20 Robert Griffin III 393 30 7.09% 24 43 25 58.1% 466 6.44%
21 Cam Newton 485 36 6.91% 21 35 25 71.4% 556 6.47%
22 Ryan Tannehill 484 35 6.74% 19 12 3 25.0% 531 6.59%
23 Michael Vick 351 28 7.39% 27 39 20 51.3% 418 6.70%
24 Jake Locker 314 25 7.37% 26 30 13 43.3% 369 6.78%
25 Nick Foles 265 20 7.02% 23 7 3 42.9% 292 6.85%
26 Mark Sanchez 453 34 6.98% 22 9 5 55.6% 496 6.85%
27 Russell Wilson 393 33 7.75% 28 50 31 62.0% 476 6.93%
28 Blaine Gabbert 278 22 7.33% 25 9 6 66.7% 309 7.12%
29 Jay Cutler 434 38 8.05% 30 24 9 37.5% 496 7.66%
30 Andy Dalton 528 46 8.01% 29 19 10 52.6% 593 7.76%
31 Aaron Rodgers 552 51 8.46% 32 35 16 45.7% 638 7.99%
32 Chad Henne 308 28 8.33% 31 11 9 81.8% 347 8.07%
33 Philip Rivers 527 49 8.51% 33 11 7 63.6% 587 8.35%
34 Alex Smith 218 24 9.92% 34 12 7 58.3% 254 9.45%

Since scrambling is dangerous to a NFL quarterback's health, the True Sack Rate will not change too much. The average decline was 0.26 percentage points. The more you scramble, the more your sack rate will fall. No one benefits more than Russell Wilson (-0.81 percent), because he had a league-high 50 scrambles (31 under pressure). The four division title-winning veterans in the AFC combined for just 14 scrambles.

The goal was never to drastically change things, but to improve the denominator in calculating sack rate, which this certainly accomplishes without being controversial. The only issue is the lack of historic data for scrambles. We can only adjust for what we have.

Also, while you could argue many scrambles include no pressure on the quarterback, he still found a way to avoid a sack and not lose yards, which is a positive. (After all, some sacks don't include much pressure on the quarterback either, at least for the first 3-4 seconds; that's why we talk about "coverage sacks.") Some quarterbacks are so slow they will still get tackled behind the line on a decision to scramble, resulting in a sack. There are also many pass plays, such as quick screens, where the quarterback has virtually no chance of being sacked by the laws of physics, but he can juice his sack rate by doing this 80-plus times a season.

And True Sack Rate will still not help explain how a quarterback takes the sacks he does.

Can We Improve on True Sack Rate?

Coming full circle to the beginning about Luck and Rodgers, it does take game charting to create more data, which will not be as objective as passes, sacks and scrambles.

For the most part, this would be about studying pressure, as it is the root cause of all sacks. (Okay, most sacks.) But unless your quarterback does something Orlovskily -- Jared Allen actually received credit for the sack even though Orlovsky ran out of bounds pretty quick here -- there has to be pressure for a sack.

Another sack rate calculation could involve only the plays where the quarterback was under pressure; however, pressure is a tricky beast. Some quarterbacks, such as Michael Vick and Cam Newton, just take way too long to make a decision on whether to throw a pass or run, which can turn three seconds of sound pass protection into a pressured play that takes 5.5 seconds. Roethlisberger can turn a few seconds of solid protection into four more seconds of him fighting off the whole Cox's Army before going down on a sack.

For these reasons I have personally soured on average time stats for how long a quarterback had to throw or how long until he made a decision. I tracked snap-to-release time (also for the scrambles and sacks) for Luck last year, and while there can be some usefulness when studying the extremes, the stats do not really show what you expected.

If a quarterback has a low average time to throw, the protection was lousy, right? Not necessarily. The quarterback may have been throwing a lot of screen passes that took no more than two seconds, bringing down his average. He may have executed a dink-and-dunk game plan all day long, rarely testing the defense deep and hardly using play-action passing, which takes longer to set up. Maybe this quarterback also panicked at the hint of any pressure and quickly got rid of the ball, also helping to lower his average.

If a quarterback has a lot of time to throw, that means the protection was good, right? Well, not exactly. Much of the NFL passing game is built on timing patterns, so quarterbacks are looking to get that ball out within three seconds on most plays. When receivers are jammed and the quarterback holds it, things usually fall apart unless you have one of those mobile playmakers. The quarterback running for his life on several plays could easily boost his time despite it not being a good situation to be in.

When your quarterback is holding the ball for six seconds, it's not likely he is able to order a sandwich in that time. Unless it's Albert Haynesworth lying prone, the pressure will finally get there. This is why tying in pressure stats is so important to sacks and quarterback performance in general.

While it would be nice to have a stat that shows how many times a quarterback avoided a sack while under pressure, we cannot ignore just how many times he brings that pressure on himself too. Between the offensive line and the quarterback, the ability to make a play and knowing when to give up, it's a two-way street filled with double-edged swords. Only Takashi Miike might be able to make sense of that, but splitting blame is still too hard.

For now, we can settle on having an accurate number of dropbacks in the passing game that resulted in a sack as a way to express a team's actual sack rate.


10 comments, Last at 04 Sep 2013, 12:31pm

#1 by John Walt // Sep 02, 2013 - 3:07pm

I think as far as how long a sack takes, listing the mean, median and mode would give a full picture of the average. This way we could see if there are any outliers that would pull the mode in one direction or the other.

Points: 0

#2 by Chase (not verified) // Sep 02, 2013 - 4:03pm

On the last point, one thing that might be useful is the average time the quarterback had in the pocket on plays where he was sacked. That solves some of the problems you highlighted, although it obviously doesn't help us get anywhere w/r/t sacks avoided.

Points: 0

#3 by Dan // Sep 02, 2013 - 4:07pm

I think that there are useful ways to make use of the timing data, but it's better to use the timing for each individual play rather than averaging.

For example, if the QB throws the ball without pressure 1.8 seconds after the snap, then you give the pass blocking credit for holding up for 1.8 seconds. There should be some way of looking at the numbers leaguewide to determine how much credit the pass blocking gets for holding up 1.8s, compared with how much credit they'd get if the pass came 2.6s after the snap, and compared to how much of a negative it would be to allow pressure after 1.6s or 2.9s or whenever.

If I had this timing data for each play then I could try to figure out how to create this stat, but unfortunately I do not.

Points: 0

#4 by Will Allen // Sep 02, 2013 - 4:17pm

The ability to throw the ball away effectively is the most underappreciated aspect of NFL qb play. Archie's brats are pretty good at it.

The latter half of Don Shula's coaching career is greatly overrated due to the fact that his feet never touched the ground. Marino's back got pretty sore in the process.

Points: 0

#5 by theslothook // Sep 02, 2013 - 5:29pm

I'm more interested in finding ways to use sack and pressure data to control for the qb and thus measure o line success. Even the more flawed qb sack rate stat is pretty good already and indicating which qbs are pretty good and which are pretty lousy.

Points: 0

#6 by Andreas Shepard // Sep 02, 2013 - 7:23pm

I agree. You could do something with using sack timing to allocate blame to the O-line similar to adjusted line yards. For example:
<2 seconds: O-line gets 100% of blame
2-2.5: 80%
2.5-3: 50%
3-3.5: 30%
3.5-4: 20%
4-5: 10%
>5%: 0%

The real trick of course is figuring out what the right %'s are, but that doesn't seem impossible. You could use situations where there were multiple QBs in one season or where the QB changed between seasons but the O-line stayed the same. For the same O-line, quicker sacks should be more consistent across different QBs than longer sacks. If you can measure exactly how much more consistent, you can calculate a pretty decent "blame %" for each sack speed.

This still wouldn't be perfect of course (a shiftier qb might be able to dodge an unblocked rusher for a few extra seconds, turning a quick slack into a slow sack), but it would be a good start.

Points: 0

#7 by Scott Kacsmar // Sep 02, 2013 - 7:39pm

FO's own J.J. Cooper does the sack-time study articles. I'm talking more about the general timing stats like snap-to-release or snap-to-decision (run/sack). You really have to combine pressure into them to mean something interesting.

For example, I bet QBs who get 4+ seconds to throw with no pressure have an outstanding completion percentage.

Points: 0

#8 by D // Sep 03, 2013 - 2:19am

Accidental post.

Move along folks nothing to see here...

Points: 0

#9 by CBPodge // Sep 03, 2013 - 11:23am

I would have thought that using the game charting definitions of different reasons for sacks and pressures would potentially have some validity?

A sack that is blown block or rusher untouched should maybe have less assigned to the QB than a sack that is defined as coverage sack or ran into sack (I can't quite recall the categories).

Or just breaking it down to short and long sacks? If you completely removed all short sacks from a QB's sack rate, how would it look?

I think the problem with anything to do with sacks and timing is that it's very hard to distinguish between a play where the blocking was excellent and the QB stood in the pocket for 5 seconds doing his fingernails, and a sack where a rusher beat his man straight away and the QB managed to elude him for 5 seconds before eventually going down.

Points: 0

#10 by Ryan D. // Sep 04, 2013 - 12:31pm

What do you do for a guy like Jimmy Clausen that loved to scramble his way into a sack before there was any real pressure coming through the O-line? I can't imagine how many expletives I shouted at Clausen in 2010 when he would tuck the ball and run around behind his tackle, creating the direct path for the defensive player to tackle him. He wouldn't even try to throw on the run. Had he just stood in the pocket, and thrown near someone in the same colored jersey, he could have been half-decent.

Points: 0

Save 10%
& Support Scott
Support Football Outsiders' independent media and Scott Kacsmar. Use promo code WRITERS to save 10% on any FO+ membership and give half the cost of your membership to tip Scott.