Risky Business
EdjSports examines critical decisions and their impact on GWC (Game-Winning Chance).
Matt Rhule
Photo: USA Today Sports Images

With 2:08 remaining in the game and trailing by 11 points to the Green Bay Packers, the Carolina Panthers found themselves in the desperate predicament of needing two scores. Even though they had a first-and-10 at the Packers’ 15-yard line with one timeout and the benefit of the two-minute warning, this was a tall order indeed. Matt Rhule decided to take the unconventional approach of kicking a field goal on first down. This seems to play into the classic bias of risk aversion, or more specifically the postponement of disappointment. Knowing he needs two scores, a coach will often take the easier (higher percentage) option first. But this choice is worth a closer look.

A custom simulation in the EdjSports model reveals that an immediate field goal attempt reduces the Panthers’ Game-Winning Chance (GWC) from 3.3% to 0.5%. This difference of ‑2.8% GWC may not seem very significant, but it represents 85% of their available equity! Imagine the outrage if the Panthers were trailing by six points with 45 seconds remaining and no timeouts from their own 1-yard line and resigned the game. That would be essentially the same cost. We are not suggesting Matt Rhule intentionally threw in the towel with his misguided field goal attempt, but you get the idea.

It helps to break down this odd choice further with the aid of some empirical data and a decision tree. While historical averages do not speak directly to the Packers-Panthers matchup, they serve as a helpful referenceHere are some key assumptions, some of which are derived from NFL play-by-play data from 2000–present, along with model-generated assumptions per the custom matchup where noted in bold.

  • Scoring a touchdown from the 15-yard line with a full set of downs: ~54%
    • Carolina per advantageous matchup: ~74%
  • Successful 2-point conversion: ~48%
  • Forcing a three-and-out: ~49%
    • Green Bay per advantageous matchup: ~31%
  • Scoring a touchdown with one minute and no timeouts from own 30-yard line: ~15%
  • Scoring a field goal with 40 seconds and no timeouts from own 30-yard line: ~28%
  • Winning in overtime: ~50%
    • Carolina per disadvantageous matchup: ~42%
  • Successful 33-yard field goal: ~94%
  • Average clock usage of (multi-) touchdown attempt: ~20 seconds
  • Average clock usage of three-and-out:~one minute (with two min warning), 1:40 without
  • Onside kick recovery rate: ~11%
  • Panthers’ GWC down 3 after onside kick recovery (1:37 remaining): ~41%
  • Panthers’ GWC down 8 after onside kick recovery (2:01 remaining): ~9%
  • Panthers’ GWC down 8 starting at own 15-yard line (0:50 remaining): ~2%

Using these assumptions, we can get a rough idea of the comparative choices. We can then insert more customized matchup numbers to get a truer sense of why the model strongly disagrees with Rhule’s decision.

Field goal attempt parlay:

(successful field goal) x (force three-and-out) x (score last drive touchdown) x (successful 2PAT) x (win in OT)

NFL Average Assumptions (kick deep):
.94 x .49 x .15 x .48 x .5 = 1.7% GWC

Matchup-Based Assumptions (kick deep):
.94 x (.31) x .15 x .48 x (.42) = 0.9% GWC

Matchup-Based Assumptions (onside kick):
((successful field goal) x (onside kick recovery) x (Panthers’ resulting GWC)) + ((successful field goal) x (unsuccessful onside kick) x (force three-and-out) x (1:00 no timeouts at own 15-yard line))

(.94 x (.11) x (.09)) + (.94 x (.89) x (.31) x (.02)) = 1.4% GWC

Touchdown attempt parlay:

(successful touchdown) x (successful 2 PAT) x (force three-and-out) x (score last drive field goal) x (win in OT)

NFL Average Assumptions (kick deep):
.54 x .48 x .49 x .28 x .42 = 1.5% GWC

Matchup-Based Assumptions (kick deep):
(.74) x .48 x (.31) x .28 x .42 = 1.3% GWC

Note: Additional residual value exists for the Panthers to score a second touchdown rather than a field goal. In this instance, that would raise their GWC to ~2%.

Matchup-Based Assumptions (onside kick):
(successful touchdown) x (successful 2 PAT) x (onside kick recovery) x (Panthers’ resulting GWC)

.74 x .48 x .11 x .41 = 1.6% GWC

Additionally, the Panthers can now win on a failed 2-point conversion (52%) when they recover the onside kick down five and score a touchdown with a GWC of ~33%. This is a flashback to the value of optionality that we discussed in a prior Risky Business column.

.74 x .52 x .11 x .33 = 1.4%

This now gives the Panthers a GWC total of ~3%, which is consistent with the simulation.

Note: The decision tree explanations are approximations and do not account for the proper weighting of all the scenarios that are captured in the simulation.In some instances, such as after an onside kick attempt, the GWC values are recognized without all of the resulting branches.

As it turns out, the Panthers had four distinct paths to an improbable victory.

  • Kick field goal and kick deep
  • Kick field goal and onside kick
  • Attempt touchdown and kick deep
  • Attempt touchdown and onside kick

Interestingly, it was not the Packers’ offensive skill difference that was the distinguishing factor in this very complex decision. As indicated in the original customized simulation, the touchdown attempt was best, and the Panthers’ actual choice was the worst of the four. However, it was the model’s surprising revelation of the effectiveness of an onside kick in combination with a touchdown attempt that made all the difference.

Comments

6 comments, Last at 22 Dec 2020, 6:35pm

1 It felt like the quick FG…

It felt like the quick FG was the decision you make if you think an onside recovery is impossible or close to it - I wonder if that 11% chance of recovery is based on onside kicks after the rule changes a few years ago that seemed to make them significantly tougher.  The idea was to get a score on the board before the 2 minute warning so you had a chance to get the ball back.

All that said, the Panthers made what felt like an even worse decision on their previous drive: down 11 with 7-ish minutes to go they had 4th and goal from just inside the 4 yard line and kicked the field goal.  I'm curious how much that kick reduced their win percentage relative to the one before the 2 minute warning.

4 Yes!

Yes to both! Yes, the estimated chance of recovery is based on data from 2018-2019 only. Also, yes, that first field goal was a problem. 9.2% GWC with a go on fourth-and-goal from the 4 but 5.2% GWC if they kicked a field goal.

2 TDs are hard

I always figure any "we need two scores" logic is predicated on the ability to score a TD.  And it's much harder to score a TD in a limited amount of time than a FG.  With that in mind, it seems that the offense needs to make sure it can perform the more difficult task.  I'm not surprised that the model dislikes the choice to go for the quick FG.  The analysis shows the emptiness of the "two score" framing.  Not all scores are created equal, and scoring only 3 points to get "within one score" is really not all that much if one's  team is still eight points behind.

The trend among sportscasters to partition score differentials into bins based on "number of scores" is frustrating.  Points matter!  

5 The end game TD/FG…

In reply to by RickD

The end game TD/FG difficulty has a weird curve in the end game, where if you have very little time left, and a lot of distance, the TD is easier.

3 If You Had Seen Packers v Lions

You might have had different feelings about the onside kick. Packers watched the watermelon ball and Lion player was an inch-- if that--OB when he snatched it. OTOH, Panthers correctly determined they could stop Rodgers and Co from getting a first down, as the short passes GB has been using all year in many different situations were simply eliminated from their repertoire-- in this case Rodgers couldn't avoid a sack on 3rd down....

6 Examining Assumptions

There has been some discussion here and elsewhere regarding the underlying assumptions of this analysis. In particular, whether an 11% onside kick recovery rate is appropriate with respect to the  2018 rule change.  Also, the three-and-out  probability has come into question, particularly after the Panthers choose to go for a TD and lose the 2 minute warning stoppage on the defensive side of the ball.  We can debate the onside kick recovery rate as  the data is limited. Some have suggested it may be lower and closer to 8%.   With regard to the three-and-out, the point is fair, but it becomes mostly irrelevant with the onside kick approach.  By applying very generous counter argument assumptions to the comparison of Matt Rhule's actual strategy to the TD/Onside Kick strategy, we can get a better idea of why the simulation strongly favors the latter.

With the following adjusted field goal attempt parlay, two of the inputs are particularly generous. We are using a 49% rate for three-and-out (well above the model's assessment of 31% for the custom matchup and game state) and a  50% win rate in OT which is well above a more realistic 42%  (Green Bay was roughly 2:1 pregame favorite) 

(successful field goal) x (force three-and-out) x (score last drive touchdown) x (successful 2PAT) x (win in OT)

Favorable assumptions for Rhule's actual strategy:
.94 x .49 x .15 x .48 x .5 = 1.7% GWC  

Now let's use very conservative assumptions on the touchdown/onside kick parlay. We are substituting a 54% TD rate on the red zone first down  for the model projected customized value of 74%, and using only  an 8% onside kick recovery rate.  

(successful touchdown) x (successful 2 PAT) x (onside kick recovery) x (Panthers’ resulting GWC down 3)

+(successful touchdown) x (unsuccesful 2 PAT) x (onside kick recovery) x (Panthers resulting GWC down 5)

(.54 x .48 x .08 x .41) + (.54 x .52 x .08 x .33) = 1.6% GWC

As you can see, even twisting the underlying assumptions extensively in favor of Rhule's actual choice, it pulls the GWCs very close.  This is a pretty compelling argument that the original observation of the model in favor of the touchdown attempt followed by onside kick, is likely correct.