Writers of Pro Football Prospectus 2008

Most Recent FO Features

RoethlisbergerBen04.jpg

» Member Berries: Flashback to 2008 Steelers-Eagles

Remember when the Eagles sacked Ben Roethlisberger eight times in 2008? Scott Kacsmar takes a stroll down memory lane with a look at the last time the Steelers played in Philadelphia, the No. 1 team in DVOA in 2008.

31 Mar 2015

Aggressiveness Index 2014

by Aaron Schatz

When you think of the most aggressive coaches in the NFL, you think of Sean Payton and Bill Belichick. Perhaps you think of Chip Kelly. You probably do not think of Jim Caldwell.

But when we ran our Aggressiveness Index numbers for 2014, we had a bit of a surprise. Not counting catch-up situations, Caldwell went for it more often on fourth down than any other head coach in the NFL. The Lions went for it 14 times in 107 possible fourth-down situations that qualify for the Aggressiveness Index, or 13.1 percent. Marc Trestman and his Chicago Bears were second in both categories, going for it a dozen times for 12.8 percent of opportunities.

Neither coach actually leads the league in Aggressiveness Index, however. Instead, that title for 2014 once again goes to Payton. The Saints had fewer qualifying fourth-down opportunities than almost any other team, but Payton went for it seven times compared to just 3.49 expected "go for it" choices, making him one of just nine coaches since 1989 with an AI over 2.0 in a full season.

Football Outsiders introduced the concept of Aggressiveness Index back in Pro Football Prospectus 2006. The goal was to find a way to rank coaches based on their tendencies on fourth downs in a manner that was easy to understand but accounted for the different rates at which the average coach will choose to "go for it" in different situations. Although no NFL coach is as aggressive as the data suggests he should be, we discovered there is quite a wide range of fourth-down tendencies among coaches.

Aggressiveness Index numbers center around 1.0 and generally describe how much more (or less) likely each coach is to go for it on fourth down compared to his peers; for example, a coach with 1.20 AI is roughly 20 percent more likely to go for it than an average coach in equivalent situations. The Aggressiveness Index excludes obvious catch-up situations: third quarter, trailing by 15 or more points; fourth quarter, trailing by nine or more points; and in the last five minutes of the game, trailing by any amount. AI was expanded two years ago to include plays when the offense is on its own side of the field, excluding those obvious catch-up situations. A slightly newer version of AI we are using now also adjusts to judge coaches on all fourth-and-short opportunities, even when the play doesn't actually record as fourth-and-short because of one of those bogus delay of game penalties that moves the punter back five yards.

Some of Payton's more aggressive moves this year included:

  • In Week 8, going for it on fourth-and-2 from the Green Bay 43 in a tie game in the third quarter; Mark Ingram got only one yard and failed to convert.
  • In Week 10, letting Drew Brees sneak on fourth-and-1 from the Saints' own 33, down 4 to San Francisco at the start of the fourth quarter.
  • The Saints never kicked a field goal from the 1-yard line; they went for it all three times on fourth-and-goal from the 1, converting on a Drew Brees sneak in Week 9 but missing on a Brees pass to Erik Lorig in Week 11 and a Mark Ingram run in Week 12.

Here's a look at Payton among the top single-season AI numbers since 1989. "Exp." is the number of times an average coach would be expected to go for it in these situations. Note that some fourth-and-1s or fourth-and-2s also qualify as "between 31 and 37."

Top Single-Season Aggressiveness Index, 1989-2014
  Overall Fourth-and-1 Fourth-and-2 31-to-37-yard line
Rk Coach Team Year AI Go Opp Rate Exp. Go Opp Rate Exp. Go Opp Rate Exp. Go Opp Rate Exp.
1 Coslet CIN 1999 2.49 10 98 10.2% 4.02 4 9 44.4% 2.17 1 4 25.0% 0.18 3 4 75.0% 0.68
2 Seifert SF 1994 2.34 10 60 16.7% 4.28 3 9 33.3% 2.90 3 5 60.0% 0.67 1 1 100.0% 0.87
3 Tice MIN 2003 2.31 13 95 13.7% 5.63 4 6 66.7% 1.98 1 9 11.1% 0.80 3 7 42.9% 1.69
4 Schotten
heimer
KC 1991 2.22 12 87 13.8% 5.40 3 12 25.0% 2.82 1 6 16.7% 0.85 3 5 60.0% 1.74
5 Payton NO 2007 2.22 16 92 17.4% 7.20 9 17 52.9% 3.31 3 13 23.1% 1.68 3 7 42.9% 1.87
6 Parcells NE 1994 2.12 16 80 20.0% 7.53 7 8 87.5% 4.46 1 10 10.0% 0.87 7 9 77.8% 3.86
7 M.Smith ATL 2009 2.02 16 101 15.8% 7.93 7 14 50.0% 4.22 1 5 20.0% 0.29 5 8 62.5% 2.01
8 Payton NO 2014 2.01 7 81 8.6% 3.49 4 6 66.7% 1.77 2 14 14.3% 0.70 0 6 0.0% 0.44
9 Fisher TEN 2005 2.00 17 107 15.9% 8.49 4 8 50.0% 3.49 4 15 26.7% 1.67 6 12 50.0% 3.00
10 Belichick CLE1 1991 1.99 14 105 13.3% 7.02 8 19 42.1% 4.61 0 7 0.0% 0.69 3 4 75.0% 1.47

What about Trestman and Caldwell, this year's other leaders in fourth-down aggressiveness? Trestman's aggressive playcalling was probably highlighted in the Week 11 win over Minnesota. Near the end of the third quarter, a missed field goal by Blair Walsh left the score 14-10 Chicago and gave the ball to the Bears on their own 28. On the ensuing drive, Trestman twice decided to hand the ball to Matt Forte on fourth-and-1 instead of kicking a field goal to make the score 17-10, first on the Minnesota 27 and then again on the Minnesota 6. Forte converted both times and the Bears ended the drive with a touchdown that made the score 21-10. (They won 21-13.)

The numbers for the previously conservative Caldwell are interesting because his aggressive calls almost all came in the second half of the year. In the first eight games of the year, the Lions only went for it on fourth down twice in non-catch-up situations. Both were in Week 6 against Minnesota, although the second was a strange call. The Lions were up two touchdowns and decided to go for it on fourth-and-5 from the Minnesota 29 right after the two-minute warning, rather than attempting a field goal or a punt that probably would have been a touchback. That's not the strange call; the strange call was the play call itself, a pitch left to Joique Bell instead of a pass.

Then in the second half of the year, Caldwell suddenly got very aggressive. It started with two attempted fake punts in Week 10, which do count in Aggressiveness Index -- after all, fake punts are a pretty aggressive call. Later in that game, the Lions got stuck in no man's land, fourth-and-6 from the Miami 40 in the third quarter. Caldwell passed on a 58-yard field goal try for Matt Prater -- of the three kickers Detroit employed in 2014, he was the one who was not terrible -- and instead let Matthew Stafford pass the ball. He converted with a 12-yard pass to Calvin Johnson. I wish this story had a happy ending where the Lions eventually finished the drive in the end zone, but a holding call on Brandon Pettigrew on the next play ruined things and Miami ended up blocking a 42-yard attempt by Prater and recovering on the 3-yard line to take a lead.

But wait... there is a happy ending. Not only did the Lions come back to win the game, but Caldwell didn't give up on his newly aggressive playcalling. The next week, he had Stafford throw on fourth-and-2 near midfield, losing by 8 with 6:50 remaining in the game. Two weeks later, he eschewed a 19-yard field goal and let Joique Bell take it in against the Bears on fourth-and-goal from the 1. In the Bears rematch in Week 16, Stafford threw the ball on fourth-and-6 from the Chicago 34 in the first quarter and on fourth-and-2 from the Chicago 31 with the Lions losing early int he fourth quarter. And against the Packers in the regular-season finale, the Lions went for it on fourth-and-10 twice, once in the second quarter and once in the fourth (which doesn't qualify for AI since they were losing by 14). Overall, the Lions went for it in only two qualifying stuations in the first eight games, then in 12 qualifying situations in the final eight.

Unfortunately, we didn't get aggressive Jim Caldwell in the playoff game against Dallas. Playing with the lead halfway through the fourth quarter, Caldwell decided not to let his offense go for it on fourth-and-1 from the Dallas 46. It cost him big time when, after a 5-yard Delay of Game penalty, Sam Martin shanked a punt and the Cowboys got the ball on their own 41.

As for the rest of the NFL head coaches for 2014, here are all the AI numbers:

2014 Aggressiveness Index
  Overall Fourth-and-1 Fourth-and-2 31-to-37-yard line
Rk Coach Team AI Go Opp Rate Exp. Go Opp Rate Exp. Go Opp Rate Exp. Go Opp Rate Exp.
1 Payton NO 2.01 7 81 8.6% 3.49 4 6 66.7% 1.77 2 14 14.3% 0.70 0 6 0.0% 0.44
2 Trestman CHI 1.80 12 94 12.8% 6.69 6 11 54.5% 4.36 1 7 14.3% 0.67 2 3 66.7% 1.07
3 Caldwell DET 1.60 14 107 13.1% 8.75 4 11 36.4% 2.95 2 9 22.2% 2.12 3 14 21.4% 3.39
4 John Harbaugh BAL 1.31 11 97 11.3% 8.40 10 13 76.9% 5.26 0 9 0.0% 0.94 2 10 20.0% 3.17
5 Whisenhunt TEN 1.30 5 98 5.1% 3.85 4 11 36.4% 1.85 1 14 7.1% 0.66 0 3 0.0% 0.53
6 Gruden WAS 1.28 5 94 5.3% 3.90 2 7 28.6% 1.17 1 9 11.1% 1.87 1 3 33.3% 1.24
7 O'Brien HOU 1.26 11 126 8.7% 8.76 7 17 41.2% 4.42 2 8 25.0% 1.73 2 10 20.0% 3.04
8 Pettine CLE 1.23 8 126 6.3% 6.52 5 15 33.3% 3.57 1 5 20.0% 0.69 1 9 11.1% 1.36
9 Belichick NE 1.13 8 106 7.5% 7.08 3 11 27.3% 3.68 0 12 0.0% 1.23 2 5 40.0% 1.84
10 McCarthy GB 1.11 7 90 7.8% 6.29 6 10 60.0% 3.38 0 6 0.0% 0.36 0 7 0.0% 1.29
11 Tomlin PIT 1.10 6 88 6.8% 5.45 3 12 25.0% 2.72 1 9 11.1% 0.93 1 4 25.0% 1.36
12 Lewis CIN 1.09 9 100 9.0% 8.26 4 15 26.7% 4.59 0 9 0.0% 0.80 5 11 45.5% 3.96
13 Jim Harbaugh SF 1.08 8 105 7.6% 7.43 5 10 50.0% 3.10 3 12 25.0% 1.33 2 9 22.2% 4.12
14 Fox DEN 1.07 7 99 7.1% 6.52 3 11 27.3% 3.52 1 10 10.0% 1.03 3 8 37.5% 1.95
15 Philbin MIA 1.07 7 96 7.3% 6.56 3 7 42.9% 3.42 0 9 0.0% 0.85 1 8 12.5% 0.88
16 Ryan NYJ 0.98 5 106 4.7% 5.09 2 5 40.0% 1.54 1 11 9.1% 0.93 1 5 20.0% 1.25
  Overall Fourth-and-1 Fourth-and-2 31-to-37-yard line
Rk Coach Team AI Go Opp Rate Exp. Go Opp Rate Exp. Go Opp Rate Exp. Go Opp Rate Exp.
17 Bradley JAC 0.92 6 105 5.7% 6.49 5 12 41.7% 4.30 0 10 0.0% 0.81 1 8 12.5% 2.86
18 Carroll SEA 0.92 6 104 5.8% 6.55 2 7 28.6% 3.00 2 15 13.3% 0.77 3 9 33.3% 2.40
19 Sparano OAK-2 0.88 3 81 3.7% 3.42 1 10 10.0% 1.45 0 3 0.0% 0.65 0 3 0.0% 0.98
20 Zimmer MIN 0.85 7 102 6.9% 8.26 4 13 30.8% 4.25 2 9 22.2% 1.42 0 8 0.0% 2.14
21 Rivera CAR 0.84 5 94 5.3% 5.97 3 11 27.3% 3.15 1 10 10.0% 1.13 2 6 33.3% 1.01
22 Reid KC 0.76 5 99 5.1% 6.56 3 10 30.0% 3.49 0 7 0.0% 0.76 0 4 0.0% 1.40
23 L.Smith TB 0.76 2 91 2.2% 2.62 2 4 50.0% 1.15 0 3 0.0% 0.04 0 7 0.0% 0.65
24 Pagano IND 0.74 6 102 5.9% 8.05 5 10 50.0% 4.71 0 10 0.0% 1.18 0 4 0.0% 2.24
25 Garrett DAL 0.68 4 79 5.1% 5.90 3 9 33.3% 3.65 0 3 0.0% 0.10 1 9 11.1% 2.89
26 Coughlin NYG 0.67 5 106 4.7% 7.45 5 10 50.0% 4.17 0 9 0.0% 0.49 0 6 0.0% 1.45
27 Kelly PHI 0.65 4 104 3.8% 6.12 3 9 33.3% 3.21 1 7 14.3% 1.14 0 4 0.0% 0.91
28 Arians ARI 0.60 4 121 3.3% 6.71 3 11 27.3% 3.63 0 6 0.0% 0.52 0 10 0.0% 1.95
29 Marrone BUF 0.55 4 116 3.4% 7.26 1 13 7.7% 3.00 1 5 20.0% 0.82 1 7 14.3% 1.74
30 McCoy SD 0.53 2 92 2.2% 3.77 1 7 14.3% 1.80 0 6 0.0% 0.38 0 3 0.0% 0.36
31 Fisher STL 0.43 3 100 3.0% 7.03 2 14 14.3% 3.77 0 8 0.0% 1.23 0 6 0.0% 2.17
32 M.Smith ATL 0.21 1 85 1.2% 4.85 1 11 9.1% 2.79 0 4 0.0% 0.29 0 4 0.0% 0.78
33 Allen OAK-1 0.00 0 23 0.0% 0.71 0 3 0.0% 0.19 0 2 0.0% 0.32 0 1 0.0% 0.07

Michael Smith ended his career in Atlanta with a truly remarkable tribute to pusillanimity, as the Falcons only went for it one time on fourth down in a non-catch-up situation. That came Week 13 against Arizona, on the first drive of the game, as Matt Ryan threw a 1-yard touchdown to Levine Toilolo on fourth-and-goal. The rest of the year, Smith had the Falcons punting twice on fourth-and-1 from their own 45 and kicking seven field goals with less than three yards to go, including two 20-yarders. Most head coaches over NFL history have gotten more aggressive with time; some have gone the other way, but the complete collapse of Smith's courage after a couple of failed Matt Ryan sneaks against the Giants in the 2011 playoffs is unique.

Michael Smith Aggressiveness Index, 2008-2014
  Overall Fourth-and-1 Fourth-and-2 31-to-37-yard line
Year Coach AI Rk Go Opp Rate Exp. Go Opp Rate Exp. Go Opp Rate Exp. Go Opp Rate Exp.
2008 M.Smith 1.22 12 9 96 9.4% 7.39 8 13 61.5% 4.61 0 4 0.0% 0.18 0 4 0.0% 1.00
2009 M.Smith 2.02 1 16 101 15.8% 7.93 7 14 50.0% 4.22 1 5 20.0% 0.29 5 8 62.5% 2.01
2010 M.Smith 1.35 7 13 115 11.3% 9.66 7 14 50.0% 5.10 3 7 42.9% 1.07 3 10 30.0% 2.45
2011 M.Smith 1.00 15 8 96 8.3% 7.98 5 17 29.4% 5.20 1 10 10.0% 1.22 2 4 50.0% 2.01
2012 M.Smith 0.15 32 1 92 1.1% 6.63 1 14 7.1% 3.44 0 16 0.0% 1.57 0 4 0.0% 1.18
2013 M.Smith 0.64 26 3 91 3.3% 4.66 2 9 22.2% 1.65 1 10 10.0% 1.38 0 7 0.0% 1.89
2014 M.Smith 0.21 32 1 85 1.2% 4.85 1 11 9.1% 2.79 0 4 0.0% 0.29 0 4 0.0% 0.78

You may also be surprised to see the allegedly super-aggressive Chip Kelly down near the bottom of the list. In his first season, Kelly was one of the league's more aggressive coaches. He went on 12 of 110 qualifying fourth-down opportunities, where an average head coach would have gone on 8.2, which gave him an AI of 1.46. That was second in the league to Rob Chudzinski. (We did not run an article on the site with those AI numbers from 2013, but they appeared in Football Outsiders Almanac 2014.)

At first glance, it looks like that aggression disappeared in 2014. Then I looked a bit closer, and Kelly's 2014 AI rating actually exposes some problems that we need to fix in the Aggressiveness Index formula going forward. The AI formula controls for teams being naturally aggressive when they fall behind, but it doesn't yet control for teams being naturally conservative with big late leads. As a result, we end up "punishing" Kelly for two third-quarter punts against Carolina in Week 10, both of which came on fourth-and-2 near midfield. They also came with the Eagles winning 31-7 and then 38-7, so it's hard to fault the decision. We also discovered that the AI formula doesn't adjust for tie scores late in the fourth quarter. So in Week 8 against the Cardinals, the "expected" odds that Kelly would go for it on fourth-and-1 from the 2 don't adjust for the fact that the score was tied with two minutes remaining and even the most aggressive coaches are going to send in the kicker to get a lead. Of course, that's not necessarily the best move... the Cardinals had two minutes to score a field goal to tie or a touchdown to win, and a 75-yard Carson Palmer touchdown pass to John Brown gave them a victory. (You can relive that play in the Week 8 Clutch Encounters article.)

We hope to work soon on some further improvements to Aggressiveness Index which will adjust for these sorts of situations, as well as an expanded AI metric that will measure other aggressive moves such as when coaches go for 2 and when they settle for conservative long field goals instead of aggressive touchdowns in late-game situations where a field goal ties or wins the game.

Posted by: Aaron Schatz on 31 Mar 2015

20 comments, Last at 02 Apr 2015, 10:21am by Aaron Schatz

Comments

1
by chemical burn :: Tue, 03/31/2015 - 2:24pm

Chip Kelly's totally conventional approach to 4th downs and 2 point conversions was probably the first disappointment I can remember experiencing with him as a coach. When he was first fired, all you heard about was how aggressive and statistically minded he was in terms of that stuff. Early on, he tried this ugly swinging-gate, failed 2 point conversion at a strange moment and got totally skewered for it in the media. After that, I'm hard pressed to think of a time he ever took a big risk on 4th down or made an unconventional 2 point attempt. I truthfully wasn't aware it was any longer a part of his reputation.

Also, I'm not sure that failing to be aggressive during a blowout should be taken out of the calculation - wouldn't a smart coach understand that the upside of putting the game away earlier outweighs the downside of maybe allowing a comeback. Or certainly that difference be exactly what this system measures? I'm not sure changing tweaking it to match up with a coach's reputation makes sense - tweaking the coach's rep makes more sense to me.

2
by Guest789 :: Tue, 03/31/2015 - 3:01pm

"When he was first fired" - some wishful thinking there?

3
by chemical burn :: Tue, 03/31/2015 - 5:34pm

Hahaha - man, is that ever a Freudian slip.

4
by electricmayhem :: Tue, 03/31/2015 - 8:24pm

So, chemical burn, how about the latest Eagles signing, Miles Austin?

(sits back awaiting the fireworks)

7
by chemical burn :: Tue, 03/31/2015 - 9:23pm

He's better than Riley Cooper and Josh Huff. I don't know how they're going to sign all their draft picks now though. DVOA-wise, last year he was about as good as Jordan Matthews and with far worse QB's throwing him the ball. He's an injury risk who hasn't made it through a full season since 2010 and is getting paid too much, but that's clearly the plan with Kelly. It's an alright signing. They shouldn't have gone into the season with Huff/Cooper in the #2 slot, so this addresses that.

10
by Alex51 :: Wed, 04/01/2015 - 1:30am

He's an injury risk who hasn't made it through a full season since 2010 and is getting paid too much, but that's clearly the plan with Kelly.

I'm not really bothered by overpaying him. I would be if this were a long term contract, but it's only for one year. As long as they have enough cap room this year to handle it, this doesn't really cost them anything. This seems like one of the few solid, sensible moves that Kelly has made this offseason.

15
by chemical burn :: Wed, 04/01/2015 - 11:01am

Yeah, it's totally fine. It doesn't solve their wr issues by any stretch of the imagination, but as a single move in a vacuum it's pretty ok.

12
by Aaron Schatz :: Wed, 04/01/2015 - 9:45am

The issue isn't tweaking it to match a coach's reputation, but rather tweaking it to reflect that the baseline for how frequently an average coach would go for it drops even further with a big lead.

14
by chemical burn :: Wed, 04/01/2015 - 10:59am

That makes sense. It really would be a super-duper aggressive move to go for it in that context...

5
by tuluse :: Tue, 03/31/2015 - 9:13pm

Marc Trestman correctly surmised he had a decent offense and a defense that couldn't stop anyone. So his numbers make perfect sense.

The burning question is why didn't Mike Smith get that memo? I don't know exactly how the tie breakers work, but it looks like one more win could have potentially put the Falcons in the playoffs.

11
by Vincent Verhei :: Wed, 04/01/2015 - 2:40am

The burning question is why didn't Mike Smith get that memo? I don't know exactly how the tie breakers work, but it looks like one more win could have potentially put the Falcons in the playoffs.

Atlanta would have made the playoffs if they had beaten Carolina in Week 17.

13
by dmstorm22 :: Wed, 04/01/2015 - 10:45am

Yes, but that's because they would have swept Carolina.

Considering they lost that game 34-3, I don't think it was a lack of 4th down agressiveness that made a 31 point difference.

I think it is more had they won some other random game, would they have made the playoffs.

16
by Travis :: Wed, 04/01/2015 - 1:49pm

If Atlanta had won some other random game, their 7-9 record still wouldn't have been enough to beat Carolina's 7-8-1.

6
by PerlStalker :: Tue, 03/31/2015 - 9:14pm

There may be an issue with fake punts though I don't know if you can control for it. I know that a couple of Denver's fake punts last year were audibles at the line rather than plays the coach called in. Should the coach get credit for an "aggressive" call that he didn't make or does he get credit for allowing his players to audible into a fake punt?

I don't have an answer but it's something to think about.

8
by tuluse :: Tue, 03/31/2015 - 9:25pm

Yes he gets credit for coaching his players for such a situation and allowing them the freedom to make such a call.

9
by PerlStalker :: Tue, 03/31/2015 - 9:32pm

Fair enough. I'm actually okay either way as long as it's stated as part of the assumptions.

17
by YoHoChecko :: Wed, 04/01/2015 - 2:58pm

I don't want to get too mired down in the weeds here, but is there any consideration given to kicker reliability?

Obviously, this primarily applies to the 31-37 yard line scenario, but as was noted, the Lions' kicking game was awful, and suddenly the typically conservative Caldwell got uber-aggressive. By limiting the window to the choice between 48-55 yard Field Goals and going for it, reliability of the kicking game obviously would play a large role in any coach's decision.

18
by Sablesma :: Wed, 04/01/2015 - 3:29pm

As if it wasn't hard enough to be a Titans fan, we didn't even make the Aggressiveness Index in 2014. In an extra twist of the knife, for this to be corrected it means we have to be reminded that Whisenhunt is again our coach and not still on ARI as the chart claims...

19
by nath :: Wed, 04/01/2015 - 11:38pm

lol, ouch.

20
by Aaron Schatz :: Thu, 04/02/2015 - 10:21am

Fixed. I add the teams manually, so my mistake.