Guest columnist Zachary O. Binney fact-checks a story in a national publication and finds that everyone makes mistakes.
22 Dec 2003
by Aaron Schatz
While Football Outsiders started in July, in some ways this week marks our birthday. It was one year ago that, sick of reading about how the Patriots needed to establish the run more, I decided to spend my company Christmas hiatus putting together the statistics to see if establishing the run really mattered. That turned into the 2002 play-by-play database, which turned into articles about the VOA system and adjusted line yards, which turned into this website.
Adjusted line yards, the statistic that we use to attempt to separate offensive lines and defensive lines from running backs, was the first statistic that I invented. That's the statistic that's listed if you click over on the left for either offensive lines or defensive lines. The problem, of course, is that those statistics have nothing to do with half of line play: pass blocking for the offense, rushing the passer for the defense. So, how to develop a statistic that measures pass blocking, other than just counting sacks?
Part of my problem was that such a statistic has already been done. The guys over at Football Project rank offensive and defensive lines in their annual book using sacks per pass attempt. This keeps an offensive line from looking like it pass-blocks well simply because that team runs more often than usual. A good example from 2003 is Baltimore: with only 34 sacks allowed, the Ravens are in the middle of the pack in total sacks. But the Ravens have only 420 pass attempts this year, fewest in the NFL. When you measure sacks per pass attempt instead of total sacks, the Ravens are actually one of the three worst teams in the league. (Obviously, this may not entirely be the fault of the line; I've written previously about the feet-of-stone Raven quarterbacks.)
Here are the top five teams in the league and bottom five teams in the league in both sacks per pass attempt and sacks per pass attempt allowed. Note that this is based on my database, which may have minor errors from the play-by-play logs that I won't get time to fix until the season ends, and if you are reading this after Monday night remember that Oakland and Green Bay numbers don't include this week's game.
|OFFENSE: sacks allowed per pass attempt||DEFENSE: sacks per pass attempt|
|Team||Sacks||Pass Att.||Sacks/Pass||Team||Sacks||Pass Att.||Sacks/Pass|
Yes, the Detroit Lions really have allowed the fewest sacks of any team in the NFL, although they don't sack anyone either. Perhaps this isn't just a testament to the offensive line's pass blocking ability, but also to Joey Harrington's game smarts. As Patriot fans know, part of avoiding sacks is throwing the ball away when necessary -- while not throwing it away to the other team -- and that's the difference between the guy who is our quarterback now and the guy who used to be our quarterback and is now at #32 on the "sacks allowed" list. The list of the teams with the fewest sacks allowed is not only a list of good offensive lines, but also quarterbacks who are often considered high on game smarts: Manning, Favre, Green, Brad Johnson. Steve McNair is #7.
McNair brings up another interesting point. You wouldn't expect "mobile quarterbacks" to be among the most sacked in the league, yet both Daunte Culpepper and Donovan McNabb are sacked plenty -- ranking down with the Baltimore statues and the immobile Drew Bledsoe.
One team missing from this list gives a good example why sacks per pass attempt makes more sense as a measurement than total sacks. The Giants, with 40 sacks, rank fourth in total sacks allowed, but they are in the middle of the sacks per pass attempt list because they have 611 pass attempts, most in the league. Giants fans will recognize that this is what happens when you suck. The flipside of that is the New England Patriots. They have 37 defensive sacks, tied with Tampa and Baltimore for fifth in the league, but they rank #15 in sacks per pass attempt because they have faced 612 pass attempts, the most in the league. This is what happens when you are consistently good.
So, giving all credit to the folks at Football Project, what are we missing here? Well, knowing Football Outsiders, you know that we (and by we, I mean mostly me) are obsessed with situational bias and opponent quality. The Giants are listed among the best sacking teams -- but they got two games against Philadelphia, and one against Houston, and two against Washington (who rank #27 in sacks allowed per pass attempt). Doesn't that make it easier for them to run up big sack numbers? And do teams who face more third-and-long situations allow more sacks, or fewer sacks, and could we adjust for that?
|YARDS TO GO||1-4||5-8||9-12||13-16||>17|
OK, let's take the second question first. Yes, it turns out that sack rate does change based on down and distance. The table to the right presents sack rate for the league as a whole in 2003, but it doesn't look much different from the table that Palmer and Carroll present on page 71 of Hidden Game of Football. Third down here includes non-punting fourth downs. That 1.5% sacks per pass attempt on first-and-goal from four yards away or less includes only 65 attempts, so I don't think it really counts for much compared to other first downs. Simplified, the rate on first and second down are basically the same no matter how many yards to go, but the sack rate on third down is higher, and even higher if it is third-and-long. It makes sense when you think about it: these are obvious pass situations, there is a lot of blitzing, and on third down a quarterback will wait until the last second and eat the ball rather than toss it away to avoid a sack, because there isn't (usually) another chance on the next down.
Adjusting sacks for these situations doesn't change things very much. Buffalo goes from allowing sacks on 9.0% of pass attempts to allowing sacks on 8.8% of pass attempts. The adjustment actually makes Detroit look even better than they did before, since they of course face tons of third-and-long situations and still don't give up many sacks.
The more important adjustment is for opponents faced. St. Louis and San Francisco's sack rates, for example, don't look so swell when you consider that they got to face such poor pass-blocking teams as Seattle, Minnesota, and the entire NFC North.
So, adjust every team's sacks for number of pass attempts, situations, and opponents faced, and we should get a good idea of the best and worst offenses and defenses when it comes to pass blocking/rushing the passer. Here is the resulting table, once again including all games through Week 16 except for Green Bay-Oakland. By the way, the league average team sacks/allows sacks on 6.0% of pass attempts.
There is one other problem here. Sacks, of course, are not the only indicator of pass blocking or pass rushing. Sometimes a good pass rush will force the quarterback to scramble. In Hidden Game of Football, Palmer and Carroll list "forced runs" for each defense. I'm not quite sure where they found this number, because unfortunately current play-by-play logs use the word "rushed" to refer to both a planned run by a quarterback and a broken pass play turned into a scramble. Occasionally the NFL.com logs will tell you when a run is a scramble, but sometimes they don't -- it's part of the frustrating inconsistency of NFL play-by-play logs. You can't just count runs by a quarterback, because of course someone like Michael Vick will have more runs than someone like Tom Brady no matter how good the offensive line.
A good pass rush, of course, will also lead to more incomplete passes, but you can't really measure how much of that is caused by the pass rush either. I do have one idea, which is to count the number of incomplete passes with no listed intended receiver. If you think about it, these passes have to be passes thrown out of bounds or batted by the defensive line; either way, they represent a quality pass rush.
I can't give you definite numbers on these passes right now. Since I haven't had a chance to double check all my numbers, I might have spikes listed still as incomplete passes with no intended receiver; I also only have intended receivers data worked out through Week 14 so far. However, my mildly imperfect data lists these as the best and worst offenses and defenses measured by incomplete passes with no intended receiver through Week 14:
|OFFENSE|| no intended
|DEFENSE|| no intended
Consider this a public request: If you have an idea for another statistic to measure pass blocking/pass rushing, please let me know. The never-ending quest for knowledge marches forward!
3 comments, Last at 20 Sep 2011, 2:19pm by jeff Anderton