Aggressiveness Index Revisited
by Aaron Schatz
Football Outsiders introduced the concept of Aggressiveness Index back in Pro Football Prospectus 2006. The goal was to find a way to rank coaches based on their tendencies on fourth downs in a manner that was easy to understand but accounted for the different rates at which the average coach will choose to "go for it" in different situations. Although no NFL coach is as aggressive as the data suggests he should be, we discovered there is quite a wide range of fourth-down tendencies among coaches.
Aggressiveness Index numbers center around 1.0 and generally describe how much more (or less) likely each coach is to go for it on fourth down compared to his peers; for example, a coach with 1.20 AI is roughly 20 percent more likely to go for it than an average coach in equivalent situations. The Aggressiveness Index excludes obvious catch-up situations: third quarter, trailing by 15 or more points; fourth quarter, trailing by nine or more points; and in the last five minutes of the game, trailing by any amount.
This offseason, Jim Armstrong -- who created the original Aggressiveness Index and is responsible for keeping both drive stats and AI stats for Football Outsiders -- worked on a slightly updated version of the Aggressiveness Index formula. Changes include:
- Aggressiveness Index now accounts for plays on the offense's own side of the field. Previously, AI only looked at plays in opponent territory, but a number of controversial fourth-down calls in recent years (including the infamous Bill Belichick fourth-and-2 as well as Mike Smith's fourth-and-1 against New Orleans in overtime) suggested that we would get a more accurate picture by including plays before the 50-yard line. Because the average rate of going for it on these plays is so low, going for it on your own half of the field (actually, technically, going for it between your own goal line and your own 48-yard line) will make a coach look extremely aggressive compared to average. Therefore, we are giving these plays only half strength in order to prevent them from overwhelming all other plays in the metric.
- Plays in the final 10 seconds of the half were removed, to account for Hail Mary-type situations. For example, Leslie Frazier had previously ranked very high in Aggressiveness Index for 2010 because the Vikings had "gone for it" by throwing a Hail Mary pass on fourth-and-14 with five seconds left in the second half of a 2010 game against Chicago.
- Baselines were changed to account for the surprising fact that the average head coach has become less aggressive in recent seasons.
We wrote about this lack of aggressiveness a year ago and nothing really changed in 2012. The public profile of football analytics has increased significantly over the last couple years, and the need for coaches to be more aggressive on fourth downs is the one thing that every single independent analyst agrees upon. In fact, not only do we all agree that coaches need to go for it more often on fourth downs, we also all agree that this strategy is even more important now than it was a few years ago because of the continued rise of overall offensive levels across the league. And yet, despite a few aggressive high-profile decisions that spurred discussion around the Internet, on the whole NFL coaches have become slightly more risk-averse.
Perhaps the most shocking example of this de-evolution in NFL strategy is the man who has long represted the ideal of aggressive fourth-down decision making: Bill Belichick. Last year during the regular season, the Patriots didn't attempt a single run or pass play on fourth-and-2, and they went for it only two times out of eight opportunities in long field-goal range (between the 31- and 37-yard lines). This trend carried over into the playoffs; against Baltimore, the Patriots had fourth-and-2 twice and settled for a punt and a 31-yard field goal. They also punted twice in long field-goal range, although these were "no man's land" situations, i.e. not just in deep field-goal range but in deep field-goal range on fourth-and-long when none of the options are mathematically any good.
As a result, Belichick ranks a shocking 23rd out of 34 head coaches in Aggressiveness Index for 2012. This is a massive change from most of Belichick's career. Belichick has the fifth-highest career AI of any head coach with at least three full seasons between 1991 and 2012, and he ranked in the top six for AI every year between 2004 and 2010 before falling to 11th in 2011 (although he was sixth in 2011 in the older version of AI). 2012 was only the third season out of 18 when Belichick ranked in the bottom half of the league; the others were 1994 (0.76 AI, 21st) and 2003 (.84 AI, 23rd).
He doesn't seem like the kind of guy to ever react to the pressure of conventional wisdom, but is it possible that a few high-profile fourth-down failures have actually made Belichick more risk averse? That certainly seems to be the case with Mike Smith. Smith has a career AI of 1.31 but ranked dead last in 2012 with 0.62 AI. The Falcons only went for it once in 91 qualifying opportunities in 2012, and that opportunity itself barely qualifies. It was a handoff to Michael Turner on fourth-and-1, up 13 points on Philadelphia in Week 8 with only 20 seconds left in the game. They didn't convert.
With Belichick and (especially) Smith coming down with a case of the wussies, and Sean Payton suspended for the year, the 2012 leaders in Aggressiveness Index look much different than in other recent years. Jacksonville's Mike Mularkey finished the year as the league's most aggressive coach with 1.61 AI. That includes decisions like going for it on fourth-and-10 from near midfield against Houston in overtime and going for it on fourth-and-3 near midfield against Tennessee with a minute left in the second quarter. It doesn't include a lot of other aggressive fourth-down decisions which don't qualify for Aggressiveness Index because the Jaguars were losing by more than two touchdowns. I would love to believe that Mularkey was the league's most aggressive coach because the Jacksonville front office has taken a very public stance on using analytics, but you might be surprised to learn that Mularkey was nearly as aggressive during his first year as head coach in Buffalo, finishing second with 1.26 AI in 2004 before dropping to 12th with 0.85 AI in 2005.
(Digression: One thing I have learned in talking to a lot of front office people who are interested in analytics is that there is very little correlation between how much analytical work is being done in a front office and how much the head coach's on-field decisions seem to reflect the general precepts that have developed in the football analytics community over the last decade. For most teams doing analytics, the impact is coming in draft and free-agency decisions, and the difference that analytics can make between one free-agent signing and another can be very subtle. Eventually we'll get to the point where a lot of head coaches have buy-in, but we aren't there yet, even on teams where the salary cap analyst is regularly reading Football Outsiders and fully understands Brian Burke's fourth-down calculator. Digression over.)
Here are the Aggressiveness Index numbers for all head coaches in 2012:
|Aggressiveness Index, 2012|
A couple of other interesting notes here:
- Marvin Lewis was extremely aggressive in long field-goal range, going for it on seven out of 10 opportunities between the 31- and 37-yard lines.
- Ken Whisenhunt comes out near the top of AI even though he only went for it once out of 11 qualifying fourth-and-1 opportunities, primarily because of a fake punt against the Jets and a fake field goal against the Bears. Yes, we count fakes as "going for it" for the purposes of Aggressiveness Index; it's hard to think of a play that would qualify as more aggressive than a fake from fourth-and-long.
- Andy Reid, one of the least aggressive coaches of the past two decades, swam surprisingly upstream against the recent trend of coaches becoming more conservative; the Eagles went for it on eight of 12 qualifying fourth-and-1 opportunities.
- No, I still don't know what's up with Jim Schwartz. One of these weeks when I don't feel buried by book editing and the Lions aren't in OTAs, I will call him up and ask him.
[ad placeholder 3]
As far as I remember, Football Outsiders hasn't published a full list of career AI since that first article back in Pro Football Prospectus 2006. We now have a new updated version of the metric and a lot more years in our database, so this would be a good time to run some career numbers for you all to enjoy and discuss. I've long referred to the most aggressive coaches as the "Holy Triumverate of Bills," Parcells, Belichick, and Cowher. The Holy Triumverate doesn't appear quite as high in this list as they would have a couple years ago; Cowher in particular drops once we include more seasons, because he was much less aggressive in his early years in Pittsburgh. This matches a trend that we generally see and probably could use a little more specific research: coaches tend to become more aggressive as they gain more experience, perhaps because they are less worried about being second-guessed by the press and/or ownership.
While analytics folks generally believe that the better coaches in the NFL will tend to be more aggressive, this list of coaches from 1991 through 2012 clearly demonstrates that there's a lot more to being a great NFL coach than simply making the mathematically correct decision on fourth downs. Some of the best coaches of the last two decades have been among the most conservative, particularly Reid and the other coaches of the Mike Holmgren coaching tree such as Jon Gruden and Holmgren himself. Meanwhile, the most aggressive coaches of the last two decades include Bruce Coslet, Jim Haslett, and... uh, well, just see for yourself who comes out at number one.
|Aggressiveness Index, 1991-2012 (minimum three seasons)|
- Five coaches went for it on more than half of all qualifying fourth-and-1 situations in our database, led by Bill Parcells (57.4 percent) as well as Don Shula (56.4 percent), Sam Wyche (52.5 percent), Belichick (52.4 percent), and Jack Del Rio (50.4 percent). Jerry Glanville (19.4 percent) was the coach least likely to go for it on fourth-and-1.
- Coaches who went for it at least 20 percent of the time on fourth-and-2 include Cosley, Payton, Mularkey, and Mike Nolan. Four coaches never went for it on a qualifying fourth-and-2: Lindy Infante, Chuck Knox, Art Shell, and Tony Sparano.
- Dave Campo (61.1 percent) and Cosley (60 percent) were by far the coaches most likely to go for it in long field-goal range (between the 31 and 37) while Wyche almost always took the long field goal, going for it just 7.9 percent of the time.
- Although Jim Mora Sr. may not have been one of the most aggressive coaches of the last two decades, aggressiveness seems to run in other coaching families. Both Shulas are above 1.0 AI, as are both Rex and Buddy Ryan (1.50 AI in 1991-92) and both John and Jim Harbaugh (1.06 AI in 2011-12).