Austin Ekeler is pushed into the endzone

Bringing Them to Their Knees

With only 1:47 remaining in the game, the Los Angeles Chargers found themselves trailing the Browns by one point with a third-and-2 at the Cleveland 11-yard line. Just when Austin Ekeler appeared to have a clear path to the end zone, he slid to his knees 3 yards short of the goal line. It is not easy to coach an NFL running back to show such restraint when he has been programmed to reach the end zone for his entire career. However, Ekeler did the prudent thing. The Browns had just burned their final timeout prior to the snap and the Chargers could now ensure themselves of kicking a game-winning field goal with no time remaining on the clock. Now all the Chargers needed to do was safely take a couple of knees and run down the clock while the Browns defense watched helplessly.

Surprisingly, Ekeler ran into the center of the Browns' defensive line on the subsequent play with apparently no intent on crossing the goal line. The Browns defenders would have no part of this clever strategy as they provided a personal concierge service for Ekeler by literally carrying him forward for a touchdown. This was a brilliant maneuver by the Browns that could have snatched an almost certain victory from the Chargers.

Why the Chargers didn't simply take a knee behind the line of scrimmage is inexplicable. They were assured of kicking a field goal that was closer than an extra point for the win with very little risk. Their pre-snap Game-Winning Chance (GWC) was 97.5% with this conservative approach. After Ekeler was forced into the end zone and prior to the two-point conversion attempt, the Chargers managed to drop their GWC by a whopping 13.0% all the way down to 84.5%. To complicate matters, the two-point attempt failed, and now a custom simulation showed the Chargers' GWC to slip even further to 79.2% GWC. Independent betting market data showed even greater confidence in a Baker Mayfield game-winning drive with an implied GWC for the Chargers as low as 73.0% after the kickoff.

Although it surely was not the intent of Ekeler to score a touchdown, this was a completely avoidable blunder. In fact, it was one of the most egregious and costly errors of the 2021 NFL season to date. Unfortunately for the Browns, they were unable to capitalize on this very poor decision.

Big Risk vs. Big Reward

At a time when analytics is dominating the conversation in the NFL more than ever, it is easy to lose sight of just how far we still have to go. The largest error of the week, according to a custom simulation from the EdjSports model, is a punt the Browns chose to attempt on a fourth-and-6 from their own 18-yard line late in the game with a 42-41 lead. This is going to seem crazy, but a passing attempt would have improved their GWC by approximately 18% on average. It is highly doubtful Kevin Stefanski and his staff seriously considered such an aggressive action. However, this situation is highly leveraged and, while a failure to convert looks horrible, the upside of a successful first down is enormous. Let’s examine the possible winning paths of both choices.

Browns punt (actual play):

  • With an average punt, Jamie Gillan forces the Chargers to begin their drive between the 40- and 50-yard lines.
  • Chargers have two timeouts and the likely benefit of the two-minute warning to orchestrate a game-winning drive. If the Chargers execute properly (see analysis above), a single first down with an advancement of 20+ yards will ensure they can attempt a game-winning field goal and leave no time on the clock for the Browns.

Browns attempt first down (optimal play):

  • In midfield situations, an average NFL offense can be expected to convert this fourth-and-6 approximately 40% of the time.
  • A successful conversion does not completely seal the victory, but the Browns’ GWC will jump to about 90%.
  • It seems counterintuitive to attempt something at which you are a favorite to fail (about 60% of the time), but it is all about GWC leverage. If the Browns come up short on the fourth-and-6, the game is far from over. They still have three timeouts, and if they can force the Chargers to kick a field goal without gaining a first down, they will still hold onto significant GWC. Assuming proper clock management by both sides, the Browns would retain about 31% GWC after a failed first down attempt.

We are probably still a long way from any NFL coach following this recommendation, but the analysis reveals just how much value is still attainable for those willing to buck conventional wisdom.


18 comments, Last at 12 Oct 2021, 9:51am

1 Disappointed

Disappointed that you had nothing to say about all the Chargers decisions to go for it on 4th down.

”This is going to seem crazy, but a passing attempt would have improved their GWC by approximately 18% on average.”  On 4th and six from your own 18?  I would say crazy is an understatement ; insane seems closer.

2 I do think it's fair to…

In reply to by Raiderfan

I do think it's fair to criticize the model's specific outputs (is it really 18%? what are the error bars? etc), I do think we're getting value here, as the model is better at thinking through subsequent probabilities than just the outcome of a single play (which is how most people approach the decision).

The likelihood of the Chargers scoring on their next possession, whether following a punt or a failed fourth down conversion, were relatively high - I'd probably say greater than 50% offhand.  And since the goal is not to succeed on more plays, but rather have more points at the end of the game, there are a few ways to increase the odds of that.

The first is reduce their odds of the Chargers scoring on their next drive.  This is the option that suggests punting, as even if you believe they are more likely than not to score from their own side of the field, it's certainly less likely than if they get the ball deep in your territory.

The second is to get an even bigger lead before the Chargers get the ball again.  This option favors going for it, as punting definitely means you won't have more points before the Chargers get the ball again.

The third is to re-take the lead after the Chargers score again (if they do).  This option also favors going for it, as the Chargers are more likely to score quickly and leave you time to come back if they get the ball closer to your own end zone.

The fourth is to never let the Chargers have the ball back.  This option requires going for it, obviously.

The challenge is that those three scenarios all have different impacts on game winning chances, and individually are quite difficult to model.  If you think the first option reduces the Chargers' scoring odds by a greater amount than the other three options increase your chances of scoring again, then punting is the right option.  The EdjSports model figures it's the other way, though; that your likelihood of winning by going for it, which is the only way of scoring again before they get the ball or denying them the ball, and likely gives you more time to come back if they do score, moves the needle more.

4 Is that how these models work?

"... as the model is better at thinking through subsequent probabilities than just the outcome of a single play (which is how most people approach the decision). ..."

I thought these models just took the history of down, distance, field location and time in game and used those to say something like in x% of these cases decision A led to outcome B. You are describing a very different model. In either case, I don't think the models are simulating decision trees, like computer chess games. Or are they? Just guessing here.

10 I was inferring that from…

I was inferring that from some comments the author has made on some previous weeks' editions, I believe EdjSports simulates how the rest of the game could play out, not just look at WPA tables for before and after the play.

11 The "rest of the game"…

The "rest of the game" results basically all boil down to small effects, though. Fundamentally, if you punt, the highest-leverage portions are the first few plays. That first pass to Jared Cook basically did everything. Similarly, if you go for it and fail, the highest-leverage portions are those first few plays, again, except now all of those plays' contributions are deweighted by your chance of driving down for a field goal with under 2 minutes and no timeouts. So ignore them - the only play that significantly matters there is the 4th down conversion.

Really, if the Browns chances of converting that 4th down are 40% and you want it to be obviously the right decision, your chance of stopping the Chargers on 3 plays has to be significantly less than converting that down. The chance of the Chargers scoring after you punt has to be way higher than 50%. Probably more like 70%.

18 Not quite insane

In reply to by Raiderfan

I don't think it's as crazy as it sounds. The 2 main points that are most convincing* to me are 1) identifying the actual leverage in the scenario (high, in this case) and 2) not risking more than you can afford to lose at that particular time. Yes you're giving them the opportunity to take the lead but you get the ball back with time to score yourself, and in a 40-some-to-40-some game, I think that's probably more valuable than lead + other team has the ball (and no, I couldn't quantify that).

*Doesn't mean I'm saying it would've been the right call, only that I understand the rationale behind it and wouldn't dismiss it out of hand.

3 I am very skeptical of that…

I am very skeptical of that 18% number. Ben Baldwin's model wants to go for it but it says it's 1.2%, which seems much more reasonable.

5 It's also a bit weird…

It's also a bit weird because if you think about it in first-order effects, the difference is just (convert 4th and 6) versus (stop the Chargers from gaining a first down). And if you assume the Browns can convert 40% of the time, that means the likelihood is that the chance they can stop the Chargers must be really small. With an 18% difference, you'd guess you'd need 40% chance of gaining the 1st, 20% chance of stopping the Chargers from gaining a 1st down.

The additional "but if they stop the Chargers if they fail they retain 30%" is a bit silly, because, again, it's a second order effect. If you think it's likely they can stop the Chargers, the difference would be much closer in favor of punting. And if it's not likely, then it's a small effect.

So crudely you'd guess that one model thinks the difference between "4th down conversion percentage" and "stopping the Chargers from gaining a 1st down" is small, and the other one thinks it's large.

17 Spot the leverage

I think more important than the specific number is the idea that the situation is high leverage and merits considering an action that conventional wisdom would consider foolish. Once it's considered, and the various possibilities laid out, as Frank does, it doesn't sound like such a bad thing after all. And someone brave enough to try it and actually succeed wins, while everyone else is forced to reconsider what they're really so afraid of. Myths dispelled, the game gets a little more exciting.

6 Rule curiosity

Isn't the rule that if a player gives himself up the play is over? If the Browns had to drag Ekeler forward wouldn't that only be because he gave himself up on the play?

In that case, the ball should have been called dead where he was swarmed prior to the touchdown.

Now I ask, not because I think this should have been the outcome (feels like opening Pandora's box when there isn't a clear sign of giving himself up), but because it might be called like this in the future if this type of scenario happens again.

9 I think generally "giving…

In reply to by Ferguson1015

I think generally "giving yourself up" is called when the player kneels or slides. I don't think it was clear enough intent. An official could have reasonably thought Ekeler was just protecting the ball against a strip.

7 "Isn't the rule that if a…

"Isn't the rule that if a player gives himself up the play is over? If the Browns had to drag Ekeler forward wouldn't that only be because he gave himself up on the play?"

Specifically, Section 2, Article 1.d states:

An official shall declare the ball dead and the down ended...

(d) when a runner declares himself down by:

  1. falling to the ground, or kneeling, and clearly making no immediate effort to advance.
  2. sliding. When a runner slides feet or head first, the ball is dead the instant he touches the ground with anything other than his hands or his feet. 

In Ekeler's case, I guess the "making no immediate effort to advance" part would apply... but I'm not sure how an official would make any determination of that when in a scrum like Ekeler was. 

12 Comrade!

As to that 18%:

"But if you go carryin' pictures of Chairman Mao,

Ya ain't gonna make it with anyone anyhow"

famed analyst John Lennon

13 When you say improved their…

When you say improved their GWC by 18% do you mean an 18% increase in GWC from say 50% to 59%? Or do you mean an absolute increase in GWC from say 50% to 68%?

14 A few things

Ok, I would like to address a few things that have been raised by readers.

  • The -18% GWC is absolute not relative.  I expected this would be hard for people to swallow as it violates so many common precepts of 4th down decision-making.
  • The cost is generated by an extensive custom simulation that can be controlled for any parameter that defines game state and team strength.  100,000 simulations produces 100,000 unique box scores.  Each simulation plays the game to conclusion from the play in question with running clock, time out usage and accurately representative outcome distributions and play-calling behavior.  Most importantly, it is tuned using DVOA for team characteristics.
  • The decision tree and GWC benchmarks that are referenced in the article are only there for perspective and as supporting evidence of the model outputs.  The model is NOT a look-up table of average NFL performances.  The simulations are tuned to historical NFL performances but adjusted for custom match-ups.  For instance, the ~40% conversion rate on fourth-and-6 is a historical reference point but in the simulation the Browns would be converting closer to 50% due to their strong offensive DVOA and the Chargers DVOA adjusted defense.
  • Not only is the model calibrated against historical data points with high statistical significance, but it is also regularly compared to liquid, in-game, sports market pricing.  In fact, it is explicitly trusted by some of the largest and most sophisticated betting groups in the world to produce accurate in-game probabilities.
  • It is very important to consider that even after a failed 4th down attempt by the Browns the game is not over.  In the actual game, the Browns still had greater than 20% winning chances AFTER the Chargers scored a touchdown.  Independent verification from other analysts and betting market data support our simulations.
  • The beauty of the EdjSports' simulation model is that it only cares about maximizing GWC.  Near term set backs, and embarrassing play execution, are only considered in their proper context in the way they impact final outcomes.
  • Finally, we can stress test the models recommendations on any underlying team assumptions to address any possible counter-argument.  If you want to make the Browns offense as bad as the Texans, we can do that.  In fact we did, and it was still -11% GWC to punt!

15 for the 18th time

Regarding that 40% conversion rate for fourth-and-6:

Let us see the raw data. Let us see the raw data. Let us see the raw data.

And if it includes converting fourth-and-6s while down 20 points with 2:41 left in the 4th quarter, that data's only fit for wiping up my beer spills.

(I was going to be cruder, but, well ... )


16 Comparison time??

If ESPN's conservative model is open source, I'd love to see how it derives such lower estimates than all the ANALYTICS! ones. And yes, I do have suspicions that the ESPN model is just data-independent speculative mumbo jumbo.