Risky Business: Divisional Round
Kevin Stefanski and Andy Reid finished the regular season as two of the highest rated coaches in EdjSports’ Coach Rankings. And when it comes to the Critical Call Index (CCI), which measures the specific merits of fourth down decisions, Stefanski and Reid finished fifth and sixth, respectively. Although the talented Chiefs were a clear pregame favorite in this matchup, it was not expected that either coach had a significant advantage with regard to in-game decisions. However, analytics only matter if you have the courage to act at the most crucial moments. In the final minutes of this game, there could not have been a clearer distinction between the seasoned veteran and the rookie head coach.
When the Browns faced a fourth-and-9 on their own 32-yard line with just 4:19 remaining and trailing 22-17, they chose the textbook punt. After all, it was a long fourth down, they were in their own territory, and the Chiefs were relying on backup quarterback Chad Henne. A solid punt followed by a defensive stop would put Baker Mayfield in a position to execute a game-winning touchdown drive. Stefanski had been very accurate with his fourth down decisions up to this point in the game. In fact, we credited him with three of the top seven fourth down gos of the weekend, including a fourth-and-1 from his own 29-yard line just moments earlier. Here, Stefanski had a golden opportunity to buck conventional wisdom and follow the math. Even accounting for the absence of Patrick Mahomes (resulting in a substantial downgrade of the Chiefs’ offense in the simulation model), this punting decision costs the Browns nearly 5% of game-winning chance (GWC). As always, it can be helpful to look at some exaggerated game states to see why Stefanski blundered.
- Conservative estimate of first down success. Browns gain only the minimum nine yards for first-and-10 on their own 41-yard line.
- Browns’ GWC = 32.5%
- Browns execute a hypothetically perfect punt and down the ball at the Chiefs’ 1-yard line
- Browns’ GWC = 17.6%
- Browns fail with an incomplete pass and Chiefs take over on Browns’ 32-yard line
- Browns’ GWC = 7.4%
Therefore, even under these grossly exaggerated assumptions, which are designed to support the choice of punting, the Browns would be risking 10.2% GWC (17.6 - 7.4) to gain 14.9% GWC (32.5 – 17.6).This means even if they knew in advance that the punt would be perfect and they would not get the benefit of any extra yardage on a successful first down, they would only need to convert the fourth down 10.2/(10.2 +14.9) = 40.6% of the time. To put this in perspective, the average NFL conversion rate of third-and-9 attempts (excluding red zone) is 38%.
While the Browns couldn’t find the right choice at a crucial juncture of the game, Andy Reid executed a mathematically sound play with extraordinary nerve to put the game away. Looking at a fourth-and-1 on their own 48-yard line, with 1:14 remaining in the game and a backup quarterback, many coaches would have tried to pin the Browns deep with no timeouts. However, Reid opted for a short pass that appeared to catch the Browns off guard and secure a third consecutive visit to the AFC Championship Game. Using a similar analysis to the Browns punting decision, let’s examine the merits of Reid’s choice to go for it.
- Chiefs gain exactly one yard for the first down
- Chiefs’ GWC = 100%
- Chiefs execute a perfect punt to the Browns’ 1-yard line
- Chiefs’ GWC = 92.5%
- Chiefs fail at line of scrimmage and Browns take over first-and-10
- Chiefs’ GWC = 64%
Again, assuming an absurdly perfect punt, the Chiefs would be risking 28.5% GWC (92.5 – 64) to gain 7.5% GWC (100 – 92.5) for a required conversion rate of 28.5/(28.5 + 7.5) = 79%. Historical data suggests an average NFL team would be expected to convert this fourth-and-1 at midfield at least 70% of the time. In the actual detailed simulation by the EdjSports model, which accounts for the proper weighting of a wide distribution of outcomes, Reid’s decision improves the Chiefs’ game-winning chance by nearly 11% GWC on average.
In both situations, the extreme counter-case assumptions illustrate just how clearly the analytics support going for it on these fourth downs. Having the nerve to follow the science when so much is at stake is another matter altogether.