Guest columnist Zachary O. Binney fact-checks a story in a national publication and finds that everyone makes mistakes.
20 May 2013
by Aaron Schatz
Football Outsiders introduced the concept of Aggressiveness Index back in Pro Football Prospectus 2006. The goal was to find a way to rank coaches based on their tendencies on fourth downs in a manner that was easy to understand but accounted for the different rates at which the average coach will choose to "go for it" in different situations. Although no NFL coach is as aggressive as the data suggests he should be, we discovered there is quite a wide range of fourth-down tendencies among coaches.
Aggressiveness Index numbers center around 1.0 and generally describe how much more (or less) likely each coach is to go for it on fourth down compared to his peers; for example, a coach with 1.20 AI is roughly 20 percent more likely to go for it than an average coach in equivalent situations. The Aggressiveness Index excludes obvious catch-up situations: third quarter, trailing by 15 or more points; fourth quarter, trailing by nine or more points; and in the last five minutes of the game, trailing by any amount.
This offseason, Jim Armstrong -- who created the original Aggressiveness Index and is responsible for keeping both drive stats and AI stats for Football Outsiders -- worked on a slightly updated version of the Aggressiveness Index formula. Changes include:
We wrote about this lack of aggressiveness a year ago and nothing really changed in 2012. The public profile of football analytics has increased significantly over the last couple years, and the need for coaches to be more aggressive on fourth downs is the one thing that every single independent analyst agrees upon. In fact, not only do we all agree that coaches need to go for it more often on fourth downs, we also all agree that this strategy is even more important now than it was a few years ago because of the continued rise of overall offensive levels across the league. And yet, despite a few aggressive high-profile decisions that spurred discussion around the Internet, on the whole NFL coaches have become slightly more risk-averse.
Perhaps the most shocking example of this de-evolution in NFL strategy is the man who has long represted the ideal of aggressive fourth-down decision making: Bill Belichick. Last year during the regular season, the Patriots didn't attempt a single run or pass play on fourth-and-2, and they went for it only two times out of eight opportunities in long field-goal range (between the 31- and 37-yard lines). This trend carried over into the playoffs; against Baltimore, the Patriots had fourth-and-2 twice and settled for a punt and a 31-yard field goal. They also punted twice in long field-goal range, although these were "no man's land" situations, i.e. not just in deep field-goal range but in deep field-goal range on fourth-and-long when none of the options are mathematically any good.
As a result, Belichick ranks a shocking 23rd out of 34 head coaches in Aggressiveness Index for 2012. This is a massive change from most of Belichick's career. Belichick has the fifth-highest career AI of any head coach with at least three full seasons between 1991 and 2012, and he ranked in the top six for AI every year between 2004 and 2010 before falling to 11th in 2011 (although he was sixth in 2011 in the older version of AI). 2012 was only the third season out of 18 when Belichick ranked in the bottom half of the league; the others were 1994 (0.76 AI, 21st) and 2003 (.84 AI, 23rd).
He doesn't seem like the kind of guy to ever react to the pressure of conventional wisdom, but is it possible that a few high-profile fourth-down failures have actually made Belichick more risk averse? That certainly seems to be the case with Mike Smith. Smith has a career AI of 1.31 but ranked dead last in 2012 with 0.62 AI. The Falcons only went for it once in 91 qualifying opportunities in 2012, and that opportunity itself barely qualifies. It was a handoff to Michael Turner on fourth-and-1, up 13 points on Philadelphia in Week 8 with only 20 seconds left in the game. They didn't convert.
With Belichick and (especially) Smith coming down with a case of the wussies, and Sean Payton suspended for the year, the 2012 leaders in Aggressiveness Index look much different than in other recent years. Jacksonville's Mike Mularkey finished the year as the league's most aggressive coach with 1.61 AI. That includes decisions like going for it on fourth-and-10 from near midfield against Houston in overtime and going for it on fourth-and-3 near midfield against Tennessee with a minute left in the second quarter. It doesn't include a lot of other aggressive fourth-down decisions which don't qualify for Aggressiveness Index because the Jaguars were losing by more than two touchdowns. I would love to believe that Mularkey was the league's most aggressive coach because the Jacksonville front office has taken a very public stance on using analytics, but you might be surprised to learn that Mularkey was nearly as aggressive during his first year as head coach in Buffalo, finishing second with 1.26 AI in 2004 before dropping to 12th with 0.85 AI in 2005.
(Digression: One thing I have learned in talking to a lot of front office people who are interested in analytics is that there is very little correlation between how much analytical work is being done in a front office and how much the head coach's on-field decisions seem to reflect the general precepts that have developed in the football analytics community over the last decade. For most teams doing analytics, the impact is coming in draft and free-agency decisions, and the difference that analytics can make between one free-agent signing and another can be very subtle. Eventually we'll get to the point where a lot of head coaches have buy-in, but we aren't there yet, even on teams where the salary cap analyst is regularly reading Football Outsiders and fully understands Brian Burke's fourth-down calculator. Digression over.)
Here are the Aggressiveness Index numbers for all head coaches in 2012:
|Aggressiveness Index, 2012|
A couple of other interesting notes here:
As far as I remember, Football Outsiders hasn't published a full list of career AI since that first article back in Pro Football Prospectus 2006. We now have a new updated version of the metric and a lot more years in our database, so this would be a good time to run some career numbers for you all to enjoy and discuss. I've long referred to the most aggressive coaches as the "Holy Triumverate of Bills," Parcells, Belichick, and Cowher. The Holy Triumverate doesn't appear quite as high in this list as they would have a couple years ago; Cowher in particular drops once we include more seasons, because he was much less aggressive in his early years in Pittsburgh. This matches a trend that we generally see and probably could use a little more specific research: coaches tend to become more aggressive as they gain more experience, perhaps because they are less worried about being second-guessed by the press and/or ownership.
While analytics folks generally believe that the better coaches in the NFL will tend to be more aggressive, this list of coaches from 1991 through 2012 clearly demonstrates that there's a lot more to being a great NFL coach than simply making the mathematically correct decision on fourth downs. Some of the best coaches of the last two decades have been among the most conservative, particularly Reid and the other coaches of the Mike Holmgren coaching tree such as Jon Gruden and Holmgren himself. Meanwhile, the most aggressive coaches of the last two decades include Bruce Coslet, Jim Haslett, and... uh, well, just see for yourself who comes out at number one.
|Aggressiveness Index, 1991-2012 (minimum three seasons)|
56 comments, Last at 30 Aug 2016, 4:38am by Eva Trumper