Trying to Improve Sack Rates

by Scott Kacsmar

Mainstream acknowledgement of sacks as a function of quarterback play has been a slow process, but nearly every advanced metric from DVOA to Win Probability Added to QBR factors in sacks. They are, after all, an attempt to throw the ball, and usually have a more negative impact than an incompletion.

How often have you heard an analyst revel about the way quarterbacks like Aaron Rodgers and Andrew Luck use their mobility to escape pressure and avoid sacks? Now what if we could quantify this with a number each season?

I actually did try that for Luck's rookie season, coming up with an absurd -- or is it? -- total near 70. Luck's actual sack total was 41. Could he really have been sacked nearly 111 times? Were the Colts like the 1986 Eagles, who did allow 104 sacks? This counted plays where Luck absorbed contact in the pocket and either got rid of the ball or scrambled. It also included plays where he was hit while throwing, which happened 51 times if you believe ESPN's data. Suddenly, 70 does not sound so hard to believe. But in this context, the number is meaningless without anything to compare it to.

Clearly, that stat could use a ton of work before going live. While this does tread down the path of subjective statistics, we still can do a better job of expressing a quarterback's sack rate.

Sack Rate: Flawed, Misunderstood and Dominated by One Player

Historically, sack rate has been nothing more than sacks divided by the sum of pass attempts and sacks. Matt Ryan threw 615 passes and was sacked 28 times in 2012. He had 643 pass plays. His sack rate (28/643) was 4.35 percent.

However, we have a calculation with a phony denominator. It's not accurate to say Ryan only had 643 pass plays last season. The pass ratio in the NFL is always misleadingly lower than the true number of pass plays. This is due to plays that start as passes turning into statistical runs, such as a scramble or a pass thrown laterally. Runs practically never turn into passes, because the play would likely be nullified by penalty for offensive linemen being illegally down the field.

Despite having a simple calculation, sack rate is not a stat you often hear -- actually, do you ever hear it anywhere? -- discussed in NFL coverage. For example, Eli Manning led the league in lowest sack percentage (3.42 percent) in 2012, yet when does anyone ever bring that up? (Note: of course ESPN Stats & Info just had to tweet this after I started the rough draft of this article.)

Oddly enough, those mobile quarterbacks known for escaping defenders are often among the league's worst players for accumulating sacks. John Elway and Dave Krieg retired after 1998 as the most sacked quarterbacks ever, and that is only because the league does not count the sack totals Fran Tarkenton had in 1961-62. Randall Cunningham falls right behind them and is the most notable player with a sack rate over 10 percent. Some called Roger Staubach "The Dodger" in Dallas, but that may have been heavily influenced by his rhyming name and highlight syndrome. Staubach had a career sack rate of 9.57 percent.

It's always entertaining to hear announcers talk about "how hard it is to sack" Ben Roethlisberger, who has an 8.38 percent sack rate and has gone down 344 times since 2004. Actually, he's fairly easy to sack behind that Pittsburgh offensive line. Bringing him down is what's hard sometimes. Someone like Ryan is hard to sack, but easy to bring down. That subtle difference in semantics matters.

Pocket passers do tend to rule sack rate, with Dan Marino and Peyton Manning currently the top two at 3.13 percent. The numbers were practically identical, yet one of these players stands out much more than the other.

Marino led the league in lowest sack rate 10 times, including seven straight seasons to start his career (1983-89). That's phenomenal, and I challenge anyone to find a NFL stat that one player has led the league in more total times (10) or more consecutive times (seven). Even Don Hutson only lead the league in touchdown receptions nine times, and that was a much smaller league. Marino's dominance of sack rate is unprecedented, which is a testament and statistical proof of his lightning-quick release.

It also makes a good argument that sack rate is just as much of a legitimate quarterback statistic as something like completion percentage.

2012's "True" Quarterback Sack Rates

Now if we adjusted Marino's sack rate the way we are about to do for 2012's quarterbacks, there likely would be no noticeable change given how he rarely ever registered a rush that was not a kneel down.

Using regular season data from the game charting project, we added scrambles (SCRAM) to the total number of pass plays (TOT Passes) to calculate the True Sack Rate, which is what the players are ranked by. This was done for all 32 qualified (minimum 224 attempts) quarterbacks in 2012 along with both San Francisco passers, who just so happened to each notch 218 attempts. While scrambles are not always 100 percent differentiable from designed runs, they are usually clear to pick out.

(We are calling this stat "True Sack Rate" for now because we already have a stat called "Adjusted" sack rate, which is sacks and intentional groundings per pass play adjusted for situation and opponent. This stat isn't adjusted for situation or opponent, but that stat isn't adjusted for scrambles. Also, this stat doesn't include intentional groundings, but we do plan on including those plays when we next revisit the concept of "True Sack Rate.")

The other interesting piece of information obtained from game charting was how many of the scrambles involved the quarterback being under pressure (PRES). Sometimes the hole just opens up and the running lane is there for decent yardage, especially for a mobile quarterback. We found that 54.2 percent of the league's quarterback scrambles last season were the result of pressure. That's lower than I expected, but pressure is very much an "in the eye of the beholder" statistic.

Since scrambling is dangerous to a NFL quarterback's health, the True Sack Rate will not change too much. The average decline was 0.26 percentage points. The more you scramble, the more your sack rate will fall. No one benefits more than Russell Wilson (-0.81 percent), because he had a league-high 50 scrambles (31 under pressure). The four division title-winning veterans in the AFC combined for just 14 scrambles.

The goal was never to drastically change things, but to improve the denominator in calculating sack rate, which this certainly accomplishes without being controversial. The only issue is the lack of historic data for scrambles. We can only adjust for what we have.

Also, while you could argue many scrambles include no pressure on the quarterback, he still found a way to avoid a sack and not lose yards, which is a positive. (After all, some sacks don't include much pressure on the quarterback either, at least for the first 3-4 seconds; that's why we talk about "coverage sacks.") Some quarterbacks are so slow they will still get tackled behind the line on a decision to scramble, resulting in a sack. There are also many pass plays, such as quick screens, where the quarterback has virtually no chance of being sacked by the laws of physics, but he can juice his sack rate by doing this 80-plus times a season.

And True Sack Rate will still not help explain how a quarterback takes the sacks he does.

Can We Improve on True Sack Rate?

Coming full circle to the beginning about Luck and Rodgers, it does take game charting to create more data, which will not be as objective as passes, sacks and scrambles.

For the most part, this would be about studying pressure, as it is the root cause of all sacks. (Okay, most sacks.) But unless your quarterback does something Orlovskily -- Jared Allen actually received credit for the sack even though Orlovsky ran out of bounds pretty quick here -- there has to be pressure for a sack.

Another sack rate calculation could involve only the plays where the quarterback was under pressure; however, pressure is a tricky beast. Some quarterbacks, such as Michael Vick and Cam Newton, just take way too long to make a decision on whether to throw a pass or run, which can turn three seconds of sound pass protection into a pressured play that takes 5.5 seconds. Roethlisberger can turn a few seconds of solid protection into four more seconds of him fighting off the whole Cox's Army before going down on a sack.

For these reasons I have personally soured on average time stats for how long a quarterback had to throw or how long until he made a decision. I tracked snap-to-release time (also for the scrambles and sacks) for Luck last year, and while there can be some usefulness when studying the extremes, the stats do not really show what you expected.

If a quarterback has a low average time to throw, the protection was lousy, right? Not necessarily. The quarterback may have been throwing a lot of screen passes that took no more than two seconds, bringing down his average. He may have executed a dink-and-dunk game plan all day long, rarely testing the defense deep and hardly using play-action passing, which takes longer to set up. Maybe this quarterback also panicked at the hint of any pressure and quickly got rid of the ball, also helping to lower his average.

If a quarterback has a lot of time to throw, that means the protection was good, right? Well, not exactly. Much of the NFL passing game is built on timing patterns, so quarterbacks are looking to get that ball out within three seconds on most plays. When receivers are jammed and the quarterback holds it, things usually fall apart unless you have one of those mobile playmakers. The quarterback running for his life on several plays could easily boost his time despite it not being a good situation to be in.

When your quarterback is holding the ball for six seconds, it's not likely he is able to order a sandwich in that time. Unless it's Albert Haynesworth lying prone, the pressure will finally get there. This is why tying in pressure stats is so important to sacks and quarterback performance in general.

While it would be nice to have a stat that shows how many times a quarterback avoided a sack while under pressure, we cannot ignore just how many times he brings that pressure on himself too. Between the offensive line and the quarterback, the ability to make a play and knowing when to give up, it's a two-way street filled with double-edged swords. Only Takashi Miike might be able to make sense of that, but splitting blame is still too hard.

For now, we can settle on having an accurate number of dropbacks in the passing game that resulted in a sack as a way to express a team's actual sack rate.