Risky Business: Conference Championships

Matt LaFleur
Matt LaFleur
Photo: USA Today Sports Images

With just over two minutes remaining in the game, Matt LaFleur made what is sure to be one of the most scrutinized decisions of the NFL season. Facing a fourth-and-goal from the 8-yard line and trailing by eight points, he made a most puzzling choice to kick a field goal. It shouldn’t come as a surprise to anyone who has been following our analysis of fourth down decisions over the years that this was likely a blunder. In absolute terms it was only about a 3% game-winning chance (GWC) error (reducing their winning prospects from 10.8% to 7.8%) but that is due to the fact that the Packers were already a significant underdog in this situation. What is more telling is that, in a relative sense, they squandered about 28% of their available equity. When Matt LaFleur defended his decision in the post-game press conference, he referred to the trust he had in his defense. That line of reasoning has to be put in proper context. Here are some of the main considerations at the root of this fourth down:

  • How often can the Packers score a touchdown, and what is the required success rate?
  • How often can the Packers succeed on a 2-point conversion?
  • How often will the field goal succeed?
  • With three timeouts, how often can the Packers stop the Bucs and get the ball back?
  • What is the effect of the resulting field position when they fail on fourth down?

The beauty of the EdjSports customized simulation is the ability to weight all of these outcomes and their respective frequencies accurately. This is precisely what is captured in the model’s assessment of a -3% GWC error. Full consideration of the team strengths is also embedded in the simulation using Football Outsiders’ DVOA metrics. To provide some additional perspective on the decision we will calculate the required success rate for the Packers with some conservative assumptions.

The Packers risk (7.6 – 3) = 4.6% GWC to gain (25.3 – 7.6) = 17.7% GWC and therefore need to succeed on the fourth down 4.6/(4.6 + 17.7) = 20.6% of the time. It is certainly not a stretch to imagine that Aaron Rodgers and the top-rated passing offense in the NFL (No. 1 in Football Outsiders passing DVOA) can convert the fourth-and-8 more than one-fifth of the time.

After the successful, albeit misguided, field goal, the subsequent kickoff provided another fascinating choice. With 2:05 remaining and three timeouts, the Packers needed to get the ball back. A touchback would have preserved the two-minute warning as effectively a fourth timeout. Instead, Mason Crosby kicked to the 8-yard line and induced the Bucs’ Jaydon Mickens to attempt a run back. According to historical NFL statistics, a returned ball will burn more than 5 seconds more than 84% of the time and result in squandering the value of the two-minute warning. However, Mickens appeared to abort his return at the Tampa Bay 17-yard line and left the Packers with 2:02 on the game clock. As it played out, this was more advantageous for the Packers than a touchback. Small differences in GWC to be sure, but every amount of equity matters at such a crucial moment.

Finally, perhaps the most unusual strategy occurred after the Bucs picked up nine yards on the subsequent first down. Facing a second-and-1, LaFleur cleverly took an intentional encroachment penalty that cost five yards of field position and gave the Bucs a first-and-10.

This ploy improved the Packers GWC by 2.2% because of the clock preservation! As one of my colleagues noted, it was hard to believe that LaFleur could find this gem but botch the fourth-and-8 moments earlier. Of course, Arians should have declined the penalty but didn’t appear to notice that he had been duped. That fact will undoubtedly be lost on him as he celebrates his appearance at Super Bowl LV on his home field.

Comments

29 comments, Last at 28 Jan 2021, 6:46am

1 https://twitter.com/ben_bot…

https://twitter.com/ben_bot_baldwin?lang=en

It's interesting that 4th Down Bot not only liked the FGA on 4th-6 down 14-7, it thought the odds of converting 4th-6 were lower than converting the 4th-8 at the end of the game (where it wanted to go for it).

31% of converting 4th-goal from the 6 and 33% for 4th-goal from the 8. Small sample size theatre?

2 FYI

Aaron-sorry to bother you on this thread, but I wanted to let you know that Gwilly Gecko found and posted Marino's chart from that Chase Stuart study on P-F-R on the Audibles thread. Just wanted to give you a heads-up.

3 What data set??

This is the crux of the matter. It is logically impossible for 4th and goal from the 8 to be more difficult than 4th and goal from the 6. Yet, such data sets are determining analytics conclusions about when to go/when to kick. Back in my programming days, that was called 'garbage in, garbage out'.

So when a team goes for it on 4th and 8 down 20 from its own 20-yard line with 2 minutes left, does that get factored in to how historically successful teams are overall on 4th and 8? Like, for instance, now instead in a massively high-leverage situation? If so, that's 'garbage in' as far as 'data-based conclusioning' is concerned.

 

4 who cares what I opine

In reply to by BigRichie

To put my 2 cents in, LaFleur could've figured: 1), we have to make it; 2), we then have to convert the 2-pointer; 3), we then have to keep the Bucs from moving into field goal range and converting; 4), we then have to win in overtime.

So 3) is a necessary condition. Which, if we accomplish that, then scoring a TD after that stop only has to be > 25% (roughly, but pretty accurately) as likely as 4th and 8.

My guess is the math still works out to instead go on 4th and goal from the 8. But in finding 28% equity differences, my figuring is that the analytics guys are intentionally playing with a marked deck.

7 Your conditions are correct,…

Your conditions are correct, I just think it's bananas to be worried about a potential field goal when the other team can very realistically run out the clock too. 

I believe GWC doesn't consider OT (i.e. being tied at 0:00 = not a win = "loss") so the numbers are probably more gentle than the reality if anything. 

It's fascinating to me that many otherwise decent coaches just aren't willing to make a play 4th-and-game when there are minutes left on the clock. Instead they choose certain defeat spread out over several plays.

25 Thanks for clarifying

I was wondering about that and about to ask about it.

I usually find that, even when all my opponents fall into instant temporary comas, it takes me 33% more calories to run eight yards for the TD than just six. Unless those extra two yards are somehow downhill.... ;-)

6 I could see how in a goal…

In reply to by BigRichie

I could see how in a goal line set of data, how say 4th-10 is easier than 4th-6. Once it's hard to run, the less space you have to throw into the more it helps the D. So I could see the effect being real, but I also suspect the set is not sufficiently large to prove it, and the curve should probably be smoothed to fit the overall trend. 

29 Not logically

I can't imagine that 4th and 10 would be easier to convert than 4th and 8. Under the same logic than wouldn't 4th and 13 be even easier? So why not take an intentional false start?

If you graph it out it would be a pretty funky curve. We know that 4th and 1 is going to be the easiest simply because running plays and even a QB sneak are in play. 4th and 2 is a little harder but running is still possible .. including the Andy Reid inside shovel pass and jet sweeps.

We also know that 4th and 20 is going to much harder than 4th and 10. I don't see how you draw a  probability curve where these hold and there is some inflection where 4th and 6 is harder and 4th and 10 is easier.

Even if it were so, the difference is miniscule. The area to work with is 16 yards for 4th and 6 and 20 yards 4th and 10, So even though 10 is 67% farther out you only get 25% more to work with (I assume its a bit more complicated because we are dealing with 2 dimensions here not just one.)

Obviously the offset would be that at 4th and 6 a lot of plays like a swing pass to a RB would be much more likely to work on 4th and 6 than 4th 8. 

I'm a Chiefs fan and Andy Reid uses the pass to the RB frequently on about 4th and 5 or less (see game winning TD to Damien Williams in last year's SB).

Of course had Rodgers scrambled on 3rd Down going for it would have been a no brainer. Rodgers might have scored or at least gotten down inside the 2. This is the most confusing of all .. Rodgers alleges that he thought they would have 4 downs ... then why not scramble dude?

9 Its counter intuitive not to put it on your QB

Even if the odds favored kicking the FG, you'd think every HC with a great QB would put the onus on the QB. If Rodgers failed no one would ever blame LaFleur for making the wrong decision. And if Rodgers succeeded, then LaFleur shares in the glory (if only in a small way).

26 Belichick got blamed

That's because--and this is going back a long way--nobody genuflects when they mention BB's name, or when their comment is a multiple of 12, etc.

Man, that was fun times at old FO for about a three days before everybody got tired of it.  But I DID apologize to my wife one night in bed for not being Tom Brady, almost rolled to the floor laughing, and she replied with a "what the hell are you talking about?!?!"  Which almost made it funnier.

11 No matter what happens on…

No matter what happens on that 4th-and-8, the defense needs to get a stop on the ensuing drive. Maybe, if you're Matt LaFleur, you figure that you have a much higher probability of stopping the Bucs if they're going to spend at least two downs on running plays to burn clock. I would not have liked the odds of the GB defense stopping Tampa from getting to field goal range with two minutes and three timeouts. They moved the ball well in the passing game all day.

I also think that LaFleur had good reason to believe the offense's odds of converting that 4th down were poor. They had three possessions in goal-to-go situations that started beyond the five yardline, with series beginning at the six, the eight, and the eight yard lines. At the time of the final possession, they had attempted seven times to convert those, and succeeded once. Tampa was defending these situations well and it's possible that LaFleur didn't have any new plays to throw at them that they hadn't already successfully defended. What do you think the true conversion rate is there? I think it's reasonable to guess that it's less than what the win probability models assume.

Still though, as a fan, I was mad as hell when he sent the field goal unit on (and was pretty miffed when he did it in the first half, too). IMO it's probably still the wrong call, on balance. Your best asset as a team is Aaron Rodgers. Put the game in his hands, coach!

12 I'm surprised to see these…

I'm surprised to see these two situations so close
3.0% "Packers fail and Bucs take over on 8-yard line"
7.6% "Packers kick field goal and touchback on subsequent kickoff"

Both leave the Packers needing a stop and then a touchdown. The failed go-for-it leaves them also needing a 2 point conversion & a win in overtime. Both of those are about 50-50, so I'd think that failing the 4th down attempt would leave them with about 1/4 the GWC vs. making the FG. Instead it is 39% as big.

There is a 17 yard difference in field position, but that doesn't seem nearly big enough to account for that gap.

13 I suspect their model is…

I suspect their model is calibrated with the assumption that they would be more than 50% likely to get the 2-pointer, based on the other odds. Specifically it lists their win percentage as 25.3% immediately after a touchdown, before attempting the conversion. If you assume the conversion is 50% and overtime is also 50% (it's hard to forecast the latter any other way, at least), that's a baseline parlay of 25%, which then has to be modified to account for the following cases:

+The 2-pointer fails (Packers kick off trailing by 2), the defense gets the immediate stop they need just like in the down-5 or down-8 cases, but now a FG is enough and Rodgers manages to lead another drive for the game-winning score

+The 2-pointer succeeds (game tied), the Bucs either go 3&out or surrender another turnover, and the Packers now have a chance to freeroll with their last drive, where a FG will win in regulation

- The 2-pointer succeeds (game tied), but a now-aggressive Brady manages to use the final 2 minutes to lead a FG drive of his own and pull back ahead at the gun

If you believe that cases 1 and 2 combined are more likely than case 3, the net adjustment from the 25% figure may be positive, or they could be less likely (if it's particularly bullish on a crunch-time Brady with motivation to score some points) and the adjustment would be negative in that case. If it is negative, then to end up at 25.3 it had to subtract something from a baseline that was even higher, in which case either the model's estimated success rate on the 2PC or (less likely) its estimated odds of winning a game that goes to overtime would have had to be greater than 50%. Judging by the low probabilities (<10%) the model ascribes to getting a stop and a needed TD, in situations where they only need a stop and a FG the odds are definitely higher than that, but the pertinent "by how much?" is again opaque.

15 The fact that every model creator

is surprised by how "low" the chance was (despite still saying go for it) says a lot. The packers were more or less doomed one way or another when they faced a fourth and goal from the 8 with 2:09 left down 8. The chances of pulling out a win would've been historic so I think that's why it was so low to begin with. Again, still a monumentally dumb decision I'm still fuming at. 

The idea that they could get a stop, and with enough time left on the clock, to not only get the ball back but then a TD, was extremely low. Even after the dumb KR slide, the team struggled to stop the Bucs. The play call coming out of that though was great by the Bucs. I had mentioned how I would pass in such situations, here before. Got 9 instead of the 3(.2) they averaged on the ground. Then the Bucs helped the Pack again by accepting the encroachment penalty! So the Bucs did two dumb things and the defense still couldn't get a stop. This should be a prime example of how hard it is to stop a top 3 (5?) offense even when you're getting multiple breaks to go your way. And they didn't even get to the other hard part of getting a TD!

Now apparently Rodgers might want more money! He'll say it's because he's underpaid, which is correct, but you better believe them trading up for Love didn't help and (conspiracy theory) he didn't run on Sunday, because why would he put his body on the line for a franchise that ignored the present so much?

I am mad. Give me some time. 

16 This should be a prime…

This should be a prime example of how hard it is to stop a top 3 (5?) offense even when you're getting multiple breaks to go your way. And they didn't even get to the other hard part of getting a TD!

The rules have become such that the game is broken for all defenses trying to stop an opposing offense late and close.

It's essentially trivially difficult to advance into scoring position with no TOs and 0:30 left. It's essentially now assumed teams will successfully do this. This is the problem with DPI being an unlimited foul.

17 What

But it's easy for them to stop em with 2 minutes left?

Scoring position like from the 8 yard line? Instead of your own 25?

Ok

 

18 The ease of advancing means…

In reply to by ImNewAroundThe…

The ease of advancing means a 4-minute drill is also easier than historically the case, because so long as you are willing to pass, first downs are pretty easy.

21 The Bucs did...

Also you think a TD is >50% with no timeouts and advancing 75+ yards? 

Ok, I'm pretty sure the #5 defense knows they'll be passing. 

24 No; I think in late-game…

In reply to by ImNewAroundThe…

No; I think in late-game situations like this that TDs are >> harder than FGs.

I have come to garner a healthy respect for Rodgers' ability to convert Hails Mary, though.

19 One thing I haven't seen…

One thing I haven't seen discussed much, if at all, is that no matter what LaFleur decided, the Packers' defense was still going to need to stop the Buccaneers' offense at the end of the game.

If they had tied the game with a touchdown and two-point conversion, the Buccaneers had plenty of time - and all three timeouts - to get into field goal range.  I have to think, with Tom Brady under center, that Arians would have been fairly aggressive.

But with the lead, the Buccaneers were more likely to turtle, and they more or less did (outside of the play two seconds prior to the two-minute warning).  They had three first downs where the clock would be running after the play (one repeated due to penalty), and they chose to run on all three.

While I don't think this would move the field goal from "wrong call" to "right call", I think it's fair to consider that the Packers' chances of stopping the Buccaneers were higher if they kicked the field goal than if they had tied the game, and makes LaFleur's call merely "bad" instead of "OMG worst EVAR" like so many reactions seemed to say.

22 And despite "turtling"

they still couldn't stop em. Turtling might be harder to stop in this situation more than people think (despite a bonus "mess up" of taking the encroachment penalty). And that's just to get the ball back. 

Given the stakes, I wonder what's worse. The Seahawks INT in the SB? 

20 Lost in all the furor of…

Lost in all the furor of Bills and Packers settling for FGs... Late 1st Qtr, Chiefs down 9, face a 4th-and-1 at BUF 23.  Kick FG to make it a one score game?  No, Chiefs go for it, convert on a pass to Williams, the comeback is full on.

Not only does Reid trust his QB but at that point he knows that just like FGs won't beat KC, FGs won't beat BUF.  We know after that point the Chiefs dominated on offense and defense until late, but I think the going-in assumption was that BUF will get its (non-garbage-time) points.  Also, the play isn't to its all-world TE, WR, or 1st round RB, but to Darrel Williams.  He may be JAG in raw talent, but Reid trusts him in short yardage, pass-catching, and pass-blocking because Williams is a sound football player.

23 Chiefs are smart

That's why they've been to 3 straight AFCCGs and back to back SB (wins?)

Hard not to love their coaching staff even as a non AFC fan.