Writers of Pro Football Prospectus 2008

Most Recent FO Features

HundleyBre15.jpg

» Futures: UCLA QB Brett Hundley

Beyond the immediate considerations of Hundley's potential, the quarterback's tape raises larger questions about the position.

29 Mar 2011

Aggressiveness Index Update

by Aaron Schatz

If there's anything that statistical analysis in football has proved, it is that coaches are far too timid on fourth down. In fact, statistical analysis has proved this over and over and over again. It's been done. So is anybody listening?

The answer seems to be "maybe," but 2010 actually provided more evidence that the answer is "no." Two months ago, Steven Levitt of Freakonomics fame had a post on his blog titled "Are NFL Coaches Starting to Listen to Economists," with a few pieces of evidence that analysis on fourth downs and the pass-run ratio is starting to sink in with NFL head coaches. They mention that NFL teams are now passing more and running less than they did a few years ago, and also give an example of Mike Smith as a head coach who has been going for it frequently in fourth-and-short situations in opposing territory.

(By the way, this is a strange common tick for economists, to talk as if statistical analysis is only done by economists, but I digress.)

There's no doubt passing is up, but when it comes to head coaches listening to analysis on fourth downs, evidence is mixed.  I went to look at fourth down "go for it" rates in what we'll call "qualifying situations." That includes all fourth downs in opposition territory, except for these situations:

  • Third quarter, trailing by 15+ points
  • Fourth quarter, trailing by 9+ points
  • Last five minutes of the game, trailing by any amount

It turns out there were only three seasons between 1992 and 2010 where head coaches went for it on more than 16 percent of fourth downs, and two of those seasons happened to be 2007 and 2009. However, this past season, fourth-down attempts were down to their lowest rate since 2006. Overall, it looks like coaches are being more aggressive now than they were five or six years ago, but it is hard to say that they are definitely more aggressive than they were durnig the period 1994-1997.

Speaking of fourth downs, here are the Aggressive Index numbers for head coaches in 2010. Aggressiveness Index was a stat we introduced way back in Pro Football Prospectus 2006, created by Jim Armstrong. The computation for AI is a little more complicated than just giving the overall rate of "going for it" on fourth downs. Instead, the head coach's record in different "going for it" situations is compared to the league average, with the numbers weighted by the number of opportunities the coach had in each situation.

Leslie Frazier ends up with the highest AI for 2010, but that's based on a single play that actually exposed a hole in AI that we'll need to fix for the future. Frazier is given a lot of "aggressiveness" credit for going for it on fourth-and-14 in the second quarter of the Week 15 game against Chicago... but of course, he only went for it because it was the final play of the half, and an incomplete pass wouldn't actually give the ball back to Chicago. We'll fix this element of AI before we publish numbers in FOA 2011. The real most aggressive coach of 2010 was... once again, Bill Belichick, who is the most aggressive coach of the last two decades with a career AI of 1.84. (As an aside, the original Aggressiveness Index article in PFP 2006 very presciently mentioned that Belichick "distinguishes himself by going for it twice as often as average on fourth-and-2." Oh, if we had only known.)

Rank Coach AI Overall 
Att.
Overall
Opps.
Rate
1 Frazier 1.90 2 13 15%
2 Belichick 1.75 13 43 30%
3 Haley 1.57 13 52 25%
4 Fox 1.37 9 41 22%
5 Carroll 1.30 10 54 19%
6 Payton 1.13 6 45 13%
7 Garrett 1.13 7 25 28%
8 Kubiak 1.07 8 41 20%
9 M.Smith 1.07 13 59 22%
10 Lewis 1.04 10 51 20%
11 Phillips 1.00 6 24 25%
12 Fisher 0.98 7 42 17%
13 Spagnuolo 0.97 6 57 11%
14 Caldwell 0.97 6 48 13%
15 McDaniels 0.95 6 33 18%
16 Coughlin 0.95 3 41 7%
17 Cable 0.92 10 58 17%
18 McCarthy 0.90 7 42 17%
Rank Coach AI Overall 
Att.
Overall
Opps.
Rate
19 Gailey 0.89 9 43 21%
20 Del Rio 0.88 8 46 17%
21 Harbaugh 0.85 5 61 8%
22 Tomlin 0.85 5 57 9%
23 Morris 0.82 4 47 9%
24 Whisenhunt 0.81 5 46 11%
25 Singletary 0.75 4 40 10%
26 Reid 0.75 6 50 12%
27 Turner 0.74 4 42 10%
28 Childress 0.73 3 21 14%
29 Schwartz 0.73 4 42 10%
30 L.Smith 0.73 5 49 10%
31 Mangini 0.73 2 42 5%
32 Shanahan 0.71 2 47 4%
33 R.Ryan 0.69 4 63 6%
34 Studesville 0.67 0 7 0%
35 Tomsula 0.67 0 3 0%
36 Sparano 0.57 3 59 5%

Here are the most aggressive and least aggressive head coaches for the total period of 1992-2010. While we generally favor more aggressiveness, it's worth noting that not all aggressive coaches were great coaches, and not all great coaches were aggressive. The second-most aggressive head coach, between Belichick and Parcells, is Rich Kotite. Meanwhile, Andy Reid, Mike Holmgren, and Mike Tomlin are among the least aggressive coaches of the past two decades. So is Rex Ryan, although I have a sneaking feeling that's related to his offensive personnel rather than his own personal preferences. After all, I can't think of a head coach who is more aggressive than Ryan on the other side of the ball.

Bill Cowher was second on this list when we ran it in PFP 2006, but it turns out he was a little less aggressive in his first few seasons as Pittsburgh's head coach, so he drops down a bit on our new list. So much for my frequent comment about "the holy triumvirate of Bills." Nothing with Rich Kotite in it can possibly be a holy triumverate.

Most and Least Aggressive Coaches, 1992-2010 (min. 100 attempts)
Rank Coach AI Overall 
Att.
Overall
Opps.
Rate Rank Coach AI Overall 
Att.
Overall
Opps.
Rate
1 Belichick 1.84 174 845 21% x 75 Sparano 0.76 18 151 12%
2 Kotite 1.77 37 222 17% x 76 Turner 0.75 68 560 12%
3 Parcells 1.68 147 622 24% x 77 Reid 0.73 62 586 11%
4 Seifert 1.63 85 377 23% x 78 J.Jones 0.73 14 138 10%
5 Haslett 1.56 51 308 17% x 79 R.Ryan 0.73 15 125 12%
6 M.Smith 1.52 35 171 20% x 80 Knox 0.72 6 113 5%
7 Payton 1.49 46 210 22% x 81 Fox 0.71 43 394 11%
8 Ditka 1.48 26 161 16% x 82 Ross 0.70 58 449 13%
9 Tice 1.48 23 174 13% x 83 Holmgren 0.69 106 849 12%
10 Campo 1.47 24 144 17% x 84 Tomlin 0.64 19 196 10%
11 Cable 1.45 19 117 16% x 85 Saban 0.61 7 108 6%
12 Cowher 1.42 127 803 16% x 86 Riley 0.59 8 133 6%

Posted by: Aaron Schatz on 29 Mar 2011

40 comments, Last at 04 Apr 2011, 1:06pm by Anonymouse

Comments

1
by Yuri (not verified) :: Tue, 03/29/2011 - 12:27pm

Interesting stats. Is it correct that the 3 exclusions are the "obvious" go-for-it situations?

I do not like the index name-it's too broad. Aggressiveness is by no means defined by 4th down performance. E.g., Andy Reid is an extremely aggressive coach, the 4th down timidity notwithstanding.

2
by Bobby Wommack (not verified) :: Tue, 03/29/2011 - 12:34pm

I demand to know where Marty Schottenheimer stands on this list lol

18
by tally :: Tue, 03/29/2011 - 4:57pm

I remember seeing this study in PFP 2006 (and shouldn't change much now given that he's been out of coaching), and Marty was actually more aggressive than average. He gets a bum rap for playing it safe when in reality, most NFL coaches are even more conservative.

25
by Bobby Wommack (not verified) :: Wed, 03/30/2011 - 9:24am

True. I think Marty just played it safe in big games, that's where "Marty Ball" came from.

34
by tally :: Wed, 03/30/2011 - 6:09pm

Yes, he certainly played it safe in big games (mainly b/c he managed to get a few teams into big games, while many other coaches could not), but no safer than most other coaches would or have, which is where the bum rap comes in.

Funny, because Belichick has made some gutsy calls that didn't pan out but were the right decisions statistically and was lambasted for being too risky.

In the end, it doesn't matter if the decision was correct or not; coaches are judged for the results, and a bad decision will be lauded if the eventual result is a win. Simple, stupid outcome bias.

3
by JIPanick :: Tue, 03/29/2011 - 12:39pm

"It turns out there were only two seasons between 1992 and 2010 where head coaches went for it on more than 16 percent of fourth downs, and two of those seasons happened to be 2007 and 2009." ?

Also, you misspelled Belichick right above the second table.

Great article...I'd love to see the full list from 1-86.

5
by Aaron Schatz :: Tue, 03/29/2011 - 1:01pm

Whoops, sorry, three seasons, as you can see from the chart.

4
by mm (not verified) :: Tue, 03/29/2011 - 12:49pm

5. Haslett, 7. Payton, 8. Ditka

That's every Saints coach in the time period except Mora (who certainly wasn't aggressive on offense with the Saints).

I knew Haslett and Payton would do well in this stat. I'm somewhat surprised by Ditka, but then I didn't get to see many of their games when he was the Saints' coach.

6
by Aaron Brooks' Good Twin (not verified) :: Tue, 03/29/2011 - 1:07pm

Based on the top-12, I see the line between genius and insanity remains a fine one.

Have you tried correlating AI versus DVOA?

7
by ChargerJeff :: Tue, 03/29/2011 - 1:28pm

Is there any way to correct this stat for the quality of the punter on the roster? This is not to defend Norv in any way, but I know from press conferences that he considers Scifres one of the best special teams weapons in the league (when he's not being blocked) - I imagine that would affect the decision making process.

26
by krugerindustria... :: Wed, 03/30/2011 - 10:30am

I think that comment touches on the limitation to this stat. i.e. aggressiveness is only measured around the variables of field position, game time and score. I think/hope that you might actually get a 'holy triumverate' if it measured coaches decision making abilities and therefore should take into account the decision vs. the risk - reward of taking the chance. i.e. his O vs. the opposition D. His punter and/or FG kicker, also the risk, opponents O (or rushing O) vs his rushing D (can I get the ball back in better position).
yeesh, this is quickly making my head swim, proving once again it is far easier to critize than to create, so please don't take this as such. FO does a great job of matriculating the ball down the field, this is another example of such.

8
by jbrown (not verified) :: Tue, 03/29/2011 - 2:12pm

Just to clarify, AI does factor in the success of going for it, right? The way I read it Garret "goes for it" twice as much as Payton, but Payton is more successful when he does so they both end up with a 1.3.

This is a really cool way of looking at coaches and I'm surprised to see some that local fans usually complain about being too conservative (i.e. Fox and Kubiak) being high on the list.

10
by Aaron Schatz :: Tue, 03/29/2011 - 2:19pm

No. It has nothing to do with success, only with how often you go for it in certain situations compared to average.

21
by Ryan D. (not verified) :: Tue, 03/29/2011 - 7:42pm

As a Carolina fan, I really liked John Fox. You can look at last year's AI rating when he had a historically terrible offense, and then look at his career rating in the second table and draw your own conclusions about what a weird year last season was for Carolina.

9
by andrew :: Tue, 03/29/2011 - 2:18pm

While I agree that going for it on the final play of a half... where you wouldn't give the ball to the other team... shouldn't be necessarily rewarded for being ultra-aggressive...

but there are coaches who wouldn't even go for it there. I'm not sure if we should have an inverse "Timidity Index", or just give some (albeit greatly reduced) credit for going for it in such situations.

11
by Danish Denver-Fan :: Tue, 03/29/2011 - 2:54pm

I might be missing something, but isn't this missing a huge point. It's easier for a coach to send out Tom Brady to try and convert, than sending in Derek Anderson. I guess you sneak this caveat in with the Ryan remark, but the whole excersize becomes a little moot, when you allow for disclaimers like that.

15
by are-tee :: Tue, 03/29/2011 - 3:16pm

Actually, Ryan's offensive personnel are probably better equipped than most to go for it on 4th and short; good OL and good power RB's. The Jets actually sealed the playoff victory over San Diego in Ryan's rookie season by going for it on 4th down with a 3-point lead late in the game. I suspect it has more to do with his confidence in the defense to get the ball back quickly with a fresh set of downs.

Gregg Easterbrook gets a lot of grief on this site (at least when TMQ is linked in Extra Points), but wasn't he a big proponent of coaches going for it more often years before it became fashionable to advocate that strategy?

16
by Aaron Brooks' Good Twin (not verified) :: Tue, 03/29/2011 - 3:55pm

Even a blind squirrel finds the occasional acorn.

17
by Mr Shush :: Tue, 03/29/2011 - 4:23pm

It's a bit of all of the above, and some other things besides. The rational level of aggressiveness is probably directly correlated with the quality of one's offense (and particularly but not exclusively one's offense in the given situation), inversely correlated with the quality of one's defense, inversely correlated with the quality of one's punt team, directly correlated with the quality of the opposing offense, inversely correlated with the quality of the opposing defense, directly correlated with the quality of the opposing punt return team, inversely correlated with one's overall team strength and directly correlated with the opponent's overall team strength. And some others.

But given that absolutely every coach should still be more aggressive than they are, all the above is probably kinda moot for now.

Also, I find it interesting that Saban and Sparano are so much less aggressive than Parcells and Belichick. There's obviously more to it than just coaching tree.

19
by dmstorm22 :: Tue, 03/29/2011 - 5:32pm

Easterbrook is more hated for his stupid theories, like "Cold Coach Wins" or his amazing disgust of any blitzing.

20
by JonFrum :: Tue, 03/29/2011 - 7:13pm

Look down the list and eyeball how 'going for it' correlates with offensive production.

29
by tuluse :: Wed, 03/30/2011 - 1:44pm

I was just thinking something like this. Lovie Smith ranks as very timid, but his teams are generally strong defensively and weak offensively. So it makes sense to punt and ask your defense to get the ball back more than risk a 4th down conversion which will most likely fail. While coaches like Peyton, Kubiak, and Bellichick (this last year) have the opposite team construction.

30
by Intropy :: Wed, 03/30/2011 - 3:05pm

That makes sense. But doesn't it also make sense that if your defense is good and your offense is bad you'd want to risk the bad field position (which your defense can handle) for the chance at maintaining possession (which your offense does rarely)? I'm not sure which way wins out.

31
by tuluse :: Wed, 03/30/2011 - 3:19pm

Well it depends where on the field you are--turning it over in field goal range is probably going to lead to points no matter how good your defense is--and what you think the odds are. Is a 10% chance at conversion worth risking 40 yards of field position?

I'd like to see a study at what the break even points are for chance at conversion vs field position.

33
by Jerry :: Wed, 03/30/2011 - 5:13pm

This is Part 1 of a 4 (short) part series by Brian Burke that addresses the question.

35
by tuluse :: Wed, 03/30/2011 - 6:17pm

He actually goes a step too far because he assumes historical averages for the conversion percentage.

I want to see a chart that tells me the break even point of each percentage increase of conversion rate. Which is still a step too far for specific game analysis because the opposing offense may or may not score at historical rates, but I'm just curious.

36
by BJR :: Thu, 03/31/2011 - 12:34pm

The reason most coaches don't go for it more is that they are conscious of making the decisions that will attract the least criticism, and that are less likely to cost them their job if they go wrong. The perception of how agressive coaches should be amongst the mainstream media and fans has changed a lot recently, thanks in part to sites like this, but the 'play-it-safe' mentality is still likely firmly ingrained in most NFL coaches.

With that in mind, we should probably factor in that Belichick has been a sure-fire HOF coach for at least the last 6 years, and has the most secure coaches job in the NFL. That's a bit chicken and egg I suppose; did he become that great coach in part because of his aggressiveness, or has his reputation and job security unshackled him from the pressures most coaches feel to behave conservatively?

37
by Fan in Exile :: Sat, 04/02/2011 - 7:22am

I know that's what TMQ always says, but I'm not so sure that we want to condemn most football coaches as cowards. Looking at the lists it's not like the correlation to winning is all that strong so maybe we should take a step back and say they know something that we don't.

12
by Dean :: Tue, 03/29/2011 - 2:56pm

I think you're a tad late to fix the AI before publishing FOA 2010.

13
by Independent George :: Tue, 03/29/2011 - 3:00pm

So does Belichick's 4th down attempt with the lead vs the Colts not count as a qualifying situation?

ETA: Nevermind. I re-read the paragraph.

14
by AnonymousA (not verified) :: Tue, 03/29/2011 - 3:13pm

A simple improvement: linearly regress overall "go for it" chance from "points per second needed to tie", then score each decision based on this relationship. That will give better perspective, and eliminate the need to remove specific categories of decisions.

24
by Theo :: Wed, 03/30/2011 - 8:23am

There's a coach who never punts in high school. He did the numbers and it works for him.
When players know it, the game changes a lot. You have 4 downs instead of 3 to get the first down.
It's how I play madden too.
Is there also thought about not kicking the FG and going for it?

22
by MJK :: Tue, 03/29/2011 - 7:57pm

While we generally favor more aggressiveness, it's worth noting that not all aggressive coaches were great coaches, and not all great coaches were aggressive.

The reason for this is very simple, even though it's often overlooked, even on this site.

The decisions that a coach makes on gameday, while important, probably account for 10% of what goes into being a head coach, at the most. Coaches like Andy Reid and Mike Holmgren (or, for that matter, Marty Schottenheimer), may make horrible game day decisions or be extremely timid on 4th down, but still be very good coaches because they're good at the other 90% of the coaches job, which happens on Monday through Saturday (breaking down opponents, gameplanning X's and O's, coaching players to improve their skills and techniques, preparing them for what they will see, setting up the organization to run smoothly and give the players all the support they need, hiring/grooming good assistants, making good personnel decisions, etc.).

23
by Joseph :: Wed, 03/30/2011 - 12:16am

I'm not surprised to see Sean Payton high on the list--I am somewhat surprised that he is not higher. I have always considered him to be almost too aggressive. However, I will give him this: he challenges his players (esp. the offense with Drew Brees) to go out and win the game, instead of waiting for the other team to mess up and hand it to him. And after reading Brees' recent book, he enjoys that aspect of Coach Payton. And like TMQ, I like the style. It obviously worked exactly halfway through an important game in February 2010.

27
by commissionerleaf :: Wed, 03/30/2011 - 11:01am

I'm shocked that Caldwell is almost average. He is a poster child for lack of testicular fortitude, so I would have expected to see him much lower. Maybe Manning called some plays without telling him.

32
by dmstorm22 :: Wed, 03/30/2011 - 3:27pm

His more famous failings were the weird timeouts at the end of the Jacksonville and Jets games. He needs to go for it more, but he doesn't usually have any ridiculous punts.

39
by Bobman :: Mon, 04/04/2011 - 3:38am

Actually, Caldwell infuriates me much less in this regard than Dungy used to (until maybe his last two years...). I sincerely think that Dungy's EXCLUSION from the least aggressive career list is solely because of a few plays Manning called to override the coach's decision.

28
by Joshua Northey (not verified) :: Wed, 03/30/2011 - 1:16pm

"By the way, this is a strange common tick for economists, to talk as if statistical analysis is only done by economists, but I digress."

I think this mostly has to do with two things.

1) How non-statistical and armchair based much of economics is/was.
2) How conservative the institutions surrounding economics are.

When economists find mathematical models or better yet empirical data that work they fall completely in love with them, and feel like this must be the very bleeding edge. That is the only explanation they have as to why so many of the colleagues/institutional structures they work with seem oblivious to their findings.

Little do they realize that they have just rediscovered naturalism, and that other scientists have long ago resigned themselves to the fact that much of the time a good slogan will get you more traction than a good justification for your argument.

I love when I see economics talk about "the economic approach", when really what they mean is "not using intuitions and a priori reasoning solely as a guide". A practice most other fields grew out of long ago.

Economics is still just a half step above philosophy in sophistication (not that there is anything wrong with that, every field needs to start out somewhere once it is spun out of philosophy).

38
by mamber-m (not verified) :: Sun, 04/03/2011 - 8:16pm

You need to control for the quality of the short-yardage runner and the quality of the OL (in terms of run blocking). Pass-oriented coaches possibly have players who are not best suited for 4th and short.

At least, put run yardage, or proportion of yards gained by runs (to control for the overall performance of the offense) in the right hand side.

Also, if you are relying more on passing, it should affect what you do in the 3rd down (and the first two). If you have 3rd and 5, run-oriented teams can run and hopefully they can get either a 1st or 4th and short. So they are more likely to face 4th and short situation (of course, it would be more complicated if you think about the choice in the first two downs...)

40
by Anonymouse (not verified) :: Mon, 04/04/2011 - 1:06pm

Can we add in a column on winning percentages (or can you make your data public so one of us can do it?)

I'd be interested to know how aggressiveness correlates with wins. You've got dozens of coaches over almost two decades. I see top coaches -- guys who win a lot of games -- at the top (Ditka, Belichick, Parcells) and the bottom (Tomlin, Reid, Holmgren). If there is no correlation between aggressiveness and wins, then as a strategy, aggression is less helpful.