Writers of Pro Football Prospectus 2008

Most Recent FO Features

ReplayIns14.jpg

» Scramble for the Ball: Getting it Right?

Instant replay review is one of the cornerstones of the modern NFL. The process and its myriad special rules have been internalized and constantly debated. Mike Kurtz wonders: is it worth it?

22 Dec 2003

Fun With Sacks

by Aaron Schatz

While Football Outsiders started in July, in some ways this week marks our birthday.  It was one year ago that, sick of reading about how the Patriots needed to establish the run more, I decided to spend my company Christmas hiatus putting together the statistics to see if establishing the run really mattered.  That turned into the 2002 play-by-play database, which turned into articles about the VOA system and adjusted line yards, which turned into this website.

Adjusted line yards, the statistic that we use to attempt to separate offensive lines and defensive lines from running backs, was the first statistic that I invented.  That's the statistic that's listed if you click over on the left for either offensive lines or defensive lines.  The problem, of course, is that those statistics have nothing to do with half of line play: pass blocking for the offense, rushing the passer for the defense.  So, how to develop a statistic that measures pass blocking, other than just counting sacks?

Part of my problem was that such a statistic has already been done.  The guys over at Football Project rank offensive and defensive lines in their annual book using sacks per pass attempt.  This keeps an offensive line from looking like it pass-blocks well simply because that team runs more often than usual.  A good example from 2003 is Baltimore: with only 34 sacks allowed, the Ravens are in the middle of the pack in total sacks.  But the Ravens have only 420 pass attempts this year, fewest in the NFL.  When you measure sacks per pass attempt instead of total sacks, the Ravens are actually one of the three worst teams in the league.  (Obviously, this may not entirely be the fault of the line; I've written previously about the feet-of-stone Raven quarterbacks.)

Here are the top five teams in the league and bottom five teams in the league in both sacks per pass attempt and sacks per pass attempt allowed.  Note that this is based on my database, which may have minor errors from the play-by-play logs that I won't get time to fix until the season ends, and if you are reading this after Monday night remember that Oakland and Green Bay numbers don't include this week's game.


OFFENSE: sacks allowed per pass attempt DEFENSE: sacks per pass attempt
  Team Sacks Pass Att. Sacks/Pass   Team Sacks Pass Att. Sacks/Pass
1 DET 12 559 2.1% 1 STL 41 513 8.0%
2 IND 18 551 3.3% 2 SFO 40 515 7.8%
3 GNB 15 436 3.4% 3 TAM 37 484 7.6%
4 KAN 19 526 3.6% 4 MIA 41 538 7.6%
5 TAM 21 573 3.7% 5 NYG 39 528 7.4%
28 HOU 35 451 7.8% 28 DET 23 503 4.6%
29 MIN 42 529 7.9% 29 JAC 22 505 4.4%
30 BAL 34 420 8.1% 30 HOU 19 484 3.9%
31 PHI 42 493 8.5% 31 CHI 18 512 3.5%
32 BUF 45 502 9.0% 32 ARI 15 487 3.1%


Yes, the Detroit Lions really have allowed the fewest sacks of any team in the NFL, although they don't sack anyone either.  Perhaps this isn't just a testament to the offensive line's pass blocking ability, but also to Joey Harrington's game smarts.  As Patriot fans know, part of avoiding sacks is throwing the ball away when necessary -- while not throwing it away to the other team -- and that's the difference between the guy who is our quarterback now and the guy who used to be our quarterback and is now at #32 on the "sacks allowed" list.  The list of the teams with the fewest sacks allowed is not only a list of good offensive lines, but also quarterbacks who are often considered high on game smarts: Manning, Favre, Green, Brad Johnson.  Steve McNair is #7.

McNair brings up another interesting point.  You wouldn't expect "mobile quarterbacks" to be among the most sacked in the league, yet both Daunte Culpepper and Donovan McNabb are sacked plenty -- ranking down with the Baltimore statues and the immobile Drew Bledsoe.

One team missing from this list gives a good example why sacks per pass attempt makes more sense as a measurement than total sacks.  The Giants, with 40 sacks, rank fourth in total sacks allowed, but they are in the middle of the sacks per pass attempt list because they have 611 pass attempts, most in the league.  Giants fans will recognize that this is what happens when you suck.  The flipside of that is the New England Patriots.  They have 37 defensive sacks, tied with Tampa and Baltimore for fifth in the league, but they rank #15 in sacks per pass attempt because they have faced 612 pass attempts, the most in the league.  This is what happens when you are consistently good.

So, giving all credit to the folks at Football Project, what are we missing here?  Well, knowing Football Outsiders, you know that we (and by we, I mean mostly me) are obsessed with situational bias and opponent quality.  The Giants are listed among the best sacking teams -- but they got two games against Philadelphia, and one against Houston, and two against Washington (who rank #27 in sacks allowed per pass attempt).  Doesn't that make it easier for them to run up big sack numbers?  And do teams who face more third-and-long situations allow more sacks, or fewer sacks, and could we adjust for that?

YARDS TO GO 1-4 5-8 9-12 13-16 >17
1st down 1.5% 5.4% 4.8% 3.3% 5.6%
2nd down 5.6% 4.7% 5.0% 6.1% 4.5%
3rd/4th down 5.9% 8.2% 10.5% 7.6% 11.1%

OK, let's take the second question first.  Yes, it turns out that sack rate does change based on down and distance.  The table to the right presents sack rate for the league as a whole in 2003, but it doesn't look much different from the table that Palmer and Carroll present on page 71 of Hidden Game of Football.  Third down here includes non-punting fourth downs.  That 1.5% sacks per pass attempt on first-and-goal from four yards away or less includes only 65 attempts, so I don't think it really counts for much compared to other first downs.  Simplified, the rate on first and second down are basically the same no matter how many yards to go, but the sack rate on third down is higher, and even higher if it is third-and-long.  It makes sense when you think about it: these are obvious pass situations, there is a lot of blitzing, and on third down a quarterback will wait until the last second and eat the ball rather than toss it away to avoid a sack, because there isn't (usually) another chance on the next down.

Adjusting sacks for these situations doesn't change things very much.  Buffalo goes from allowing sacks on 9.0% of pass attempts to allowing sacks on 8.8% of pass attempts.  The adjustment actually makes Detroit look even better than they did before, since they of course face tons of third-and-long situations and still don't give up many sacks.

The more important adjustment is for opponents faced.  St. Louis and San Francisco's sack rates, for example, don't look so swell when you consider that they got to face such poor pass-blocking teams as Seattle, Minnesota, and the entire NFC North.

So, adjust every team's sacks for number of pass attempts, situations, and opponents faced, and we should get a good idea of the best and worst offenses and defenses when it comes to pass blocking/rushing the passer.  Here is the resulting table, once again including all games through Week 16 except for Green Bay-Oakland.  By the way, the league average team sacks/allows sacks on 6.0% of pass attempts.




OFFENSE Adjusted
Sacks/Pass Att.
Sacks Pass Att. Sacks/Pass Att.
1 DET 2.7% 12 559 2.1%
2 IND 3.2% 18 551 3.3%
3 TAM 3.5% 21 573 3.7%
4 GNB 4.1% 15 436 3.4%
5 KAN 4.2% 19 526 3.6%
6 CAR 4.3% 19 451 4.2%
7 JAC 4.8% 26 511 5.1%
8 TEN 4.9% 24 499 4.8%
9 SDG 5.1% 25 519 4.8%
10 NWE 5.2% 29 532 5.5%
11 NYJ 5.4% 27 494 5.5%
12 SFO 5.4% 26 497 5.2%
13 DAL 5.8% 31 496 6.3%
14 DEN 5.9% 25 474 5.3%
15 ARI 6.2% 35 537 6.5%
16 NYG 6.2% 40 611 6.5%
OFFENSE Adjusted
Sacks/Pass Att.
Sacks Pass Att. Sacks/Pass Att.
17 STL 6.3% 36 591 6.1%
18 NOR 6.3% 35 535 6.5%
19 MIA 6.4% 29 449 6.5%
20 PIT 6.5% 36 538 6.7%
21 WAS 6.7% 38 532 7.1%
22 ATL 6.7% 33 470 7.0%
23 OAK 6.7% 32 487 6.6%
24 CIN 6.8% 35 523 6.7%
25 CLE 7.0% 37 525 7.0%
26 SEA 7.2% 36 519 6.9%
27 CHI 7.3% 36 507 7.1%
28 HOU 7.5% 35 451 7.8%
29 BAL 7.8% 34 420 8.1%
30 PHI 8.0% 42 493 8.5%
31 MIN 8.3% 42 529 7.9%
32 BUF 8.6% 45 502 9.0%


DEFENSE Adjusted
Sacks/Pass Att.
Sacks Pass Att. Sacks/Pass Att.
1 TAM 7.7% 37 484 7.6%
2 SFO 7.5% 40 515 7.8%
3 STL 7.3% 41 513 8.0%
4 CAR 7.3% 36 512 7.0%
5 MIA 7.3% 41 538 7.6%
6 ATL 6.8% 34 512 6.6%
7 DEN 6.7% 33 509 6.5%
8 BAL 6.7% 37 538 6.9%
9 IND 6.7% 29 448 6.5%
10 NYG 6.6% 39 528 7.4%
11 BUF 6.6% 33 512 6.4%
12 NYJ 6.2% 31 456 6.8%
13 TEN 6.2% 33 541 6.1%
14 MIN 6.2% 27 526 5.1%
15 NOR 6.1% 28 468 6.0%
16 GNB 6.1% 31 544 5.7%
DEFENSE Adjusted
Sacks/Pass Att.
Sacks Pass Att. Sacks/Pass Att.
17 PIT 6.0% 30 483 6.2%
18 PHI 5.8% 33 555 5.9%
19 CLE 5.8% 28 493 5.7%
20 SEA 5.8% 34 571 6.0%
21 NWE 5.8% 37 612 6.0%
22 KAN 5.5% 31 549 5.6%
23 DAL 5.3% 29 488 5.9%
24 CIN 5.2% 30 520 5.8%
25 WAS 5.2% 27 478 5.6%
26 SDG 5.1% 25 524 4.8%
27 OAK 5.1% 22 429 5.1%
28 DET 4.8% 23 503 4.6%
29 JAC 4.7% 22 505 4.4%
30 HOU 4.4% 19 484 3.9%
31 CHI 4.1% 18 512 3.5%
32 ARI 3.2% 15 487 3.1%

There is one other problem here.  Sacks, of course, are not the only indicator of pass blocking or pass rushing.  Sometimes a good pass rush will force the quarterback to scramble.  In Hidden Game of Football, Palmer and Carroll list "forced runs" for each defense.  I'm not quite sure where they found this number, because unfortunately current play-by-play logs use the word "rushed" to refer to both a planned run by a quarterback and a broken pass play turned into a scramble.  Occasionally the NFL.com logs will tell you when a run is a scramble, but sometimes they don't -- it's part of the frustrating inconsistency of NFL play-by-play logs.  You can't just count runs by a quarterback, because of course someone like Michael Vick will have more runs than someone like Tom Brady no matter how good the offensive line.

A good pass rush, of course, will also lead to more incomplete passes, but you can't really measure how much of that is caused by the pass rush either.  I do have one idea, which is to count the number of incomplete passes with no listed intended receiver.  If you think about it, these passes have to be passes thrown out of bounds or batted by the defensive line; either way, they represent a quality pass rush.

I can't give you definite numbers on these passes right now.  Since I haven't had a chance to double check all my numbers, I might have spikes listed still as incomplete passes with no intended receiver; I also only have intended receivers data worked out through Week 14 so far.  However, my mildly imperfect data lists these as the best and worst offenses and defenses measured by incomplete passes with no intended receiver through Week 14:



OFFENSE no intended
receiver
DEFENSE no intended
receiver
STL 2 KAN 22
GNB 2 CAR 15
DET 3 NWE 14
TAM 3 OAK 13
MANY 4 BUF 12
CIN/SDG 11 DEN 11
OAK 12 ATL/IND 3
ATL 12 NOR/SFO 3
BUF 14 ARI 2
DEN 17 BAL 2

Consider this a public request: If you have an idea for another statistic to measure pass blocking/pass rushing, please let me know.  The never-ending quest for knowledge marches forward!

Posted by: Aaron Schatz on 22 Dec 2003

3 comments, Last at 20 Sep 2011, 2:19pm by jeff Anderton

Comments

1
by TheKosmicMuffin :: Tue, 01/06/2009 - 9:01pm

If you want to use "forced runs" as a stat to measure pass blocking/rushing, there are other sites besides NFL.com that have play-by-play logs. I just browsed through an ESPN.com play-by-play log and it looked as if the word "scramble" was used every time the QB kept the ball and ran with it. I would recommend using the play-by-play logs on ESPN.com from now on instead of the ones on NFL.com.

2
by TheKosmicMuffin :: Sun, 01/11/2009 - 9:15pm

Also, another thing you could try is using the game-charting project to your advantage. One stat that I often see as I'm watching the game on TV is how many times a QB has been Sacked, hurried, and hit. As far as I know,these stats are only kept during the game, but never saved for people to see afterwords. With the game-charting project, you could save these stats and use them to measure pass-blocking and pass-rushing. For pass-blocking, all 3 of these things should have equal value, because the offensive line only has control over whether or not a defender gets past the line of scrimmage, but not whether or not the QB is mobile enough to escape the pass-rush. For pass-rushing, sacks can be given the highest value with hurries, hits on the QB, and forced runs that result in stuffs being equal to one another.

One other thing I'd like to see in addition to the adjusted sack rate is the non-adjusted sack rate. The reason being that what I like do with pass-blocking data is making judgments on a QB's skill by comparing his DVOA, and his offensive line's adjusted sack rate to see how much their offensive line contributed to their DVOA. The thing is, when doing this, it doesn't matter what the quality of their opponents was, all that matters is how much pressure the QB is actually getting. Also, it should be noted that by non-adjusted sack rate, I'm referring simply to a sack rate that isn't adjusted for opponent, but is still adjusted for down and distance.

3
by jeff Anderton (not verified) :: Tue, 09/20/2011 - 2:19pm

What about considering 1 step, 3 step, 5 step, and 7 step drops into the equation? For example, throwing a quick hitter to the Flanker, a WR Screen, a WR Hitch, a Quick Slant, or a quick TE release over the middle is a lot different than a 3 step drop with a pump fake then 2 more steps and a throw to a WR running an Out and Up or some other type of Double Move Route. There should almost be a time limit associated with each drop(1,3,5,7 step drops) and it should only "count against" an offensive line if they give up a sack before the alotted time total. Like a 3 step drop gets 2 seconds, and if a QB gets sacked after 2 seconds, the Offensive Line did their job. 5 Step gets 3 seconds, 7 step gets 4 seconds, etc. Also, what about designed Bootlegs and Rollouts, as I would guess there might be some trend/variance from the norm in those types of lays as well?