McCarthy's Critical Coaching Errors
The 2021 NFL season kicked off Thursday night in exciting fashion. The Super Bowl Champion Bucs picked up where they left off as Tom Brady orchestrated yet another game-winning drive. Much of the postgame conversation revolved around the poor execution of Dallas Cowboys kicker Greg Zuerlein. He was quick to accept responsibility as he told the press that, "If I do my job, we win this game."
While Zuerlein's humility is admirable, there are many different jobs that contribute to the outcomes of NFL games, not the least of which is making critical, in-game coaching decisions. Zuerlein did miss two field goals and an extra point, which makes it easy to point the finger at him with some simple arithmetic. When you lose a game by two points, those missed opportunities to score are glaring. It is helpful, however, to look at his kicking woes through the lens of game-winning chance (GWC) and compare that cost to that of Mike McCarthy's suboptimal fourth-down decisions.
In the table at the end of this article, we can see that the raw cost on fourth-down decisions, as evaluated by the EdjSports simulation model, totaled -8.6% GWC compared to the raw cost of missed kicks at -12.1%. Of course, the kicks were tangible and obvious errors to the viewers while the fourth downs were barely discussed by the television commentators. It is also worth noting that Zuerlein's successful 48-yard field goal improved the Cowboys' GWC by 10.2% when compared to the pre-snap GWC. According to the EdjSports play-calling model, he was expected to convert this field goal attempt 75% of the time. With this consideration, his net execution cost improves considerably. None of this absolves him from what was clearly a subpar performance, but the broader context represented in the table is important.
Redirecting the critical eye to Mike McCarthy reveals a fascinating decision on fourth-and-6 prior to Zuerlein's successful 48-yard field goal. It may seem surprising that an extensive custom simulation revealed this to be such a large error, at -5.8% GWC, but a few data points might help to tell the story.
- ~75% field goal success rate for Zuerlein (this is without any adjustment for the in-game performance)
- Historically an average NFL team would be expected to convert this fourth-and-6 about 45% of the time.
- With only one timeout remaining, a successful fourth-down conversion would ensure the Cowboys leave no time on the clock for Brady's attempt at a game-winning drive (~56% success rate) while improving the field position for the success of a later field goal attempt.
- A failed fourth-down conversion is devastating, but it does not completely seal the Cowboys' fate. With two timeouts they retain some residual GWC (perhaps 2% or 3%).
- There is also a slightly advantageous starting field position for the Bucs after a failed Cowboys field goal vs. a failed fourth down attempt.
Since we can debate actual success rates for both the field goal and the fourth-and-6 conversion attempt, let's look at some comparative model-generated GWC values for a range of assumptions that might argue in favor of the conservative action.
| Field Goal Success Rate |
~GWC | Fourth-and-6 Conversion Success Rate |
~GWC |
| 75% | 33% | 45% | 39% |
| 80% | 35% | 40% | 35% |
| 85% | 37% | 35% | 31% |
Additional assumptions for the Cowboys are:
Successful field goal = 44% GWC
Unsuccessful field goal = 2% GWC
Successful fourth-down attempt (exactly 6 yards) = 88% GWC
Unsuccessful fourth-down attempt (incompletion) = 3% GWC
For example, the calculation of GWC for a 75% field goal rate is the following parlay:
The most realistic values, which are supported by both simulations and empirical data, are indicated in bold. We can see that even shifting both field goal and fourth-down conversion assumptions by 5% in favor of the field goal attempt still doesn't quite overturn the model's recommendation of going for it.
Some additional considerations in favor of the fourth-down attempt are the conservative 6-yard advancement of the ball on a successful first down and Zuerlein's kicking woes. The strongest argument against the model's recommendation is the game-winning drive success rate for Tom Brady. We would need to drop Brady's game-winning drive rate to 48% with the assumptions of 75% field goal and 45% fourth-down to flip in support of a field goal attempt. While it is always difficult to anchor on any underlying assumption, it does seem compelling that the fourth-down attempt should have been strongly considered and likely would have produced significantly more wins on average. This may not get the monkey off Zuerlein's back in the coming days, but it does suggest he shouldn't shoulder all of the blame for a disappointing season opener.
| Qtr | Clock | Ball Position | Down | ToGo | Score | Play Choice | Decision Cost (GWC)* | Execution | Execution Cost (GWC)** |
| 1 | 1:42 | 6-7 | 1 PAT | NA | good | NA | |||
| 2 | 6:41 | TB 13 | 4 | 3 | 7-14 | FG | -1.2% | missed | -5.7% |
| 2 | 4:25 | 13-14 | 1 PAT | NA | missed | -2.4% | |||
| 2 | 3:21 | TB 17 | 4 | 6 | 13-14 | FG | NA | good | NA |
| 2 | 0:20 | TB 42 | 4 | 31 | 16-21 | FG | NA | missed | -4.0% |
| 3 | 9:19 | TB 3 | 4 | G | 16-21 | FG | -1.6% | good | NA |
| 3 | 0:38 | 25-28 | 1 PAT | NA | good | NA | |||
| 4 | 1:29 | TB 30 | 4 | 6 | 26-28 | FG | -5.8% | good | +10.2% |
| * Decision Cost represents the cost, if applicable, in GWC of the actual play choice vs. the optimal play choice assessed prior to the snap ** Execution Cost represents the cost, if applicable, between the pre-snap GWC (assuming actual play choice) and the updated GWC after the completion of the play. |
|||||||||
Comments
43 comments, Last at 14 Sep 2021, 8:04pm
#1 by jheidelberg // Sep 11, 2021 - 12:03pm
I was expecting a few word answer yesterday regarding my question about team specific scenarios vs general scenarios and now an entire article to explain. Thanks! This is a great service to your readers.
Am I clear, the model used Zuerlein’s specific odds (other kickers would not be 75%) the 4th and 6 of
45 % were general NFL odds of converting and the 56 percent successful drive was specifically Brady (and I assume Succop as his success is required in all non TD scenarios).
#43 by Frank Frigo // Sep 14, 2021 - 8:03pm
The kickers are customized, so yes, this was our expected success rate for Zuerlein. It did not get adjusted down for his earlier misses. The 4th and 6 rate is derived from the custom simulation but as noted in the article it is supported by empirical data on mid field 3rd down conversions excluding garbage time. Brady's Game Winning Drive is again derived from the simulation but it is also well supported by in-game betting market figures.
#4 by jheidelberg // Sep 11, 2021 - 2:31pm
If McCarthy manages the game properly, Tampa Bay may never get the ball back and that play never happens. It is the process that we are looking for here, not the result. McCarthy managed this game poorly and the decisions pointed out in the article would still be poor decisions regardless at to whether Dallas won the game or lost it.
Dallas will win games this year in which McCarthy makes egregious coaching decisions as will many other teams when their coach makes poor decisions.
Of course some other team playing Dallas this year will likely beat Dallas when McCarthy commits more of his buffoonery.
#38 by RickD // Sep 13, 2021 - 11:05am
That's not really analysis. As Collinsworth said at the time, that kind of fall by a receiver can happen when the receiver is switching shoulders even with only light contact. And the no-call was consistent with how officials (fail to) call OPI in the NFL in general, and in that game in particular. Hell, the Bucs were called for a DPI face mask on a play where replays showed no violation (touching the face mask is not supposed to be sufficient for a flag, grabbing and pulling is required).
When I look at the no call on the last pass, I see contact, but I don't see whether the receiver is pushing hard or not. I don't see the arm extension that officials have said is usually required to get an OPI call.
#3 by BigRichie // Sep 11, 2021 - 1:43pm
'Tis obligatory I point out that the "Historically an average NFL team would be expected to convert this fourth-and-6 about 45% of the time" thing is based on garbage data. Defenses defend '4th and 6/up by 20' very differently than '4th and 6/this is ballgame right here right now'. Historical rates which don't include 'leverage' are basically garbage for these decisions.
(oh, and I don't get what that first table is saying at all)
#6 by Bryan Knowles // Sep 11, 2021 - 2:54pm
Since 1994, teams have converted 3rd-and-6 39.7% of the time. Since 2011, that number rises a bit, to 40.8%
I didn't filter out the few edge cases which might bump both numbers a bit higher -- Hail Marys on the last plays of halves, teams setting up for field goals, teams planning on going for it on fourth down, etc. -- but those are the raw numbers.
#10 by BigRichie // Sep 11, 2021 - 3:49pm
The raw numbers suck, Bryan. They don't apply.
Leave OUT the low leverage 4th downs. Leave OUT the low leverage 4th downs. Leave OUT the low leverage 4th downs. It is no more complicated than that.
And if that leaves a too-small data set, life in the Big City, boys and girls. Far more honest that than pretending we know something we don't know, by using irrelevant garbage data.
#14 by BigRichie // Sep 11, 2021 - 6:42pm
They're a poor proxy. Quite a difference between giving up a first down and so staying on the field, and giving up a first down and so losing rather than winning the game.
I'll grant that Bryan's info is relevant rather than irrelevant. But still a hugely different situation.
And one that would be easy to address. Compile what happens on (ridiculously) high leverage 4th downs in general, then start from that basis.
#16 by Pat // Sep 11, 2021 - 7:38pm
The exact 4th down make percentage honestly doesn't matter that much. It's not 10%. It's not 90%. 30, 40, 50%, it's gotta be somewhere around there.
The important quibbles you can make there are the difference between the Cowboys win percentage by kicking (44% - in other words, still favored to lose!) versus gaining the first down (88%). That's a massive difference. PFR would've had the Cowboys win percentage by kicking at more like 65-70% just by game state, which means the majority of the decision to kick is coming from the fact that it's a very good offense they're playing against (and the Cowboys aren't a great defense).
Which is why, to me, I just call that a wash of a decision. Any decision that's entirely based on how good you are relative to the other team is a wash in my opinion. It's not what I would've done, but I fully believe that they wouldn't've converted anyway.
#33 by Beavis // Sep 12, 2021 - 12:16pm
You seem to presume that the defense will be much better in a high leverage situation? Why? Because they're amped up more? Why isn't this true for the offense? All they have to do is convert and they win, shouldn't that be just as much an extra incentive for the offense as the defense? You're criticizing actual data as irrelevant based on a bunch of empty assertions you have pulled out of your...
#26 by ImNewAroundThe… // Sep 11, 2021 - 10:42pm
but what we have. Unfortunately the sample size is already probably small and what is "this is ballgame right here right now" anyway? 2 points to...apparently not 20 so...6? IDK probably arbitrary when you also have to factor in weather, time left, etc.
#39 by RickD // Sep 13, 2021 - 11:08am
Teams have their own estimation of success likelihood. Thus, when we focus on teams that go for it on fourth and 6, we are already looking at the subset of data defined by teams that think they are more likely to achieve the goal. Teams that are not likely to convert are more likely to kick there.
Thus, one cannot simply take a stat (conversion %) defined on a subset of the data (teams that try to convert on 4th and 6) and extend it to a larger set (teams facing 4th and 6). The success rate on the subset is likely higher than the success rate on the subset's complement.
#41 by Frank Frigo // Sep 13, 2021 - 3:02pm
The (Historical) figure of 45% for 4th and 6 conversions that was referenced in the article is based mostly on 3rd down data from 2000-present. Additionally they are limited to midfield scenarios and exclude late game desperation attempts and taking a knee.
#7 by The Powers That Be // Sep 11, 2021 - 3:27pm
I'm simply not going to complain about this particular decision, especially when other bots have it going the other way.
Now look at the call to kick the FG on 4th-and-goal at the 3 in the 3rd quarter. THAT's the one McCarthy blew.
#8 by rpwong // Sep 11, 2021 - 3:36pm
I looked at the clock, saw "1:24", and thought, "yep, Dallas is screwed". I'm not the only person who thought that.
When the opposing offense has moved the ball well through the entire game, you just can't leave them more than a minute to win with a field goal. Particularly when the QB is Brady, Mahomes, or Rodgers. Their two-minute drills are ridiculously efficient and they don't panic under time pressure. They'll almost always give their kickers a chance.
You'd think every coach would know this by now.
#11 by jheidelberg // Sep 11, 2021 - 4:00pm
Do not show the situation. All two minute drills are not alike. Of course Brady has failed at a horrific rate when getting the ball on the minus end of the field with under a minute left needing a TD. I must believe that the stat you saw was a “How to use statistics to support a faulty argument.”
#19 by OmahaChiefs13 // Sep 11, 2021 - 9:06pm
What was that bit you said a little further up, about how including garbage data (in this case, a large number of 2 minute comebacks where the circumstances are very different, and they're therefore not representative of the actual situation) invalidates a conclusion, and how if the only data available is garbage, it's better to say that, instead of pretending to know something we don't know?
Teams succeed at very different rates in 2-minute scenarios depending on how deep they are into that final 2 minutes, where the drive starts, how many timeouts they have, etc. You know...the leverage is different.
I mean, I understand that it's far more fun to make an appeal to one isolated call/no-call decision, especially when it lets us thumb our nose at math, but you do kind of need to decide whether leverage matters or not.
#20 by Duff Soviet Union // Sep 11, 2021 - 9:18pm
For all that you’ve ragged on misleading data, how is this not the same? How many of these drives did he need a touchdown compared to a field goal? How many of these were truly impossible situations (ie you need to go 80 yards in 10 seconds)?
#27 by BigRichie // Sep 11, 2021 - 11:07pm
The original writer gave no data supporting his argument that Brady, Rodgers, all the greats etc., are near-guarantees within 2 minutes. The only data given was in the broadcast. Which yes, only marginally applied to last night's exact situation. But last night's situation wasn't the issue of this particular thread.
#29 by rpwong // Sep 12, 2021 - 1:40am
What I wrote was, "They'll almost always give their kickers a chance."
You're right that I don't have stats to back that up, but all I meant was that you shouldn't give them a chance when they've moved the ball well all game. If your defense hasn't stopped them all night, there's no reason to think they would now.
So yes, last night's situation was specific to this thread, because the context of the gameplay mattered. Sorry that wasn't clear.
#15 by Pat // Sep 11, 2021 - 7:29pm
Additional assumptions for the Cowboys are:
Successful field goal = 44% GWC
I mean, there are a lot of assumptions going into this: just by pure game state (and ignoring timeouts), a successful field goal actually puts the Cowboys in a situation where like, 60-70% of teams have historically won. The fact that the model predicts a 44% GWC here means pretty much the entirety of the decision is just the assumption of how good the Bucs specifically are going to be.
Ben Baldwin's 4th down bot, for instance, had this at about 5% in favor of kicking, and I'm pretty sure the main differences between the two models are timeouts and team strength assumptions.
So in some sense you could say that, essentially, the decision's a wash: conservative estimates would favor kicking, aggressive estimates would favor going for it.
As I've said elsewhere I personally would've gone for it, but I also basically thought the Cowboys were screwed either way.
#17 by Bryan Knowles // Sep 11, 2021 - 7:41pm
It's worth noting that the BaldwinBot gives the Cowboys a 46% GWC with a successful field goal, so that's not the difference here. Both the EdjSports model and Baldwin's model take into account relative team strengths, and the Buccaneers were favored by 8.5 points coming in (and had the higher estimated DVOA, and so on and so forth), so that's why you're seeing a smaller win probability there than based on historical teams in average scenarios.
The main difference between EdjSports' decision to go for it and Baldwin's decision to kick comes on the GWC on a successful conversion. EdjSports has it at 88%; Baldwin has it at 57%. There's the fundamental difference, there. If you think that picking up the first down more or less ends the game, you have to go for it. If you think picking up the first down only gives you a 60/40 chance of winning, you have to kick the field goal on a fourth and long(ish).
It's worth noting that the Buccaneers would have presumably used their final timeout, assuming the Cowboys had converted without going out of bounds or otherwise stopping the clock. With about 85 seconds left in the game, the Cowboys could have then run the clock down to near-zero, and attempted the game-winning field goal of 40 yards or so. Whether you are comfortable with Zeuerlein kicking that sort of game-winner or not is a matter of personal taste, though McCarthy was apparently more than happy with letting him kick from just about anywhere.
#21 by Pat // Sep 11, 2021 - 9:18pm
Both the EdjSports model and Baldwin's model take into account relative team strengths, and the Buccaneers were favored by 8.5 points coming in
Oh, duh, I forgot that Ben's bot uses the point spread - which I hate. I really wish people would give ranges of estimates in those cases. Especially early in the season, the coaches sure as hell better be able to get a stronger feel for the actual difference in team strength between the two.
What I usually do is just use a zero point spread and then the point spread as well to treat it as a systematic error. If the decision stays the same in both cases, it's obvious. If it falls in between the two cases, it's a wash. I mean, think about it - if you go to a coach and say "that was a bad decision" and they say "hey, look at the math, historically this gave us a better chance to win," and you reply with "yeah, but you were the worse team" - you're gonna get punched in the face.
#40 by Frank Frigo // Sep 13, 2021 - 1:50pm
Bryan, I think you sum it up pretty well here. The conversion is a huge boost in GWC with proper clock management toward a game winning field goal attempt, presumably at a more advantageous distance. The assumption on the game winning drive for the Bucs was generated from our simulation which accounts for custom match-ups and I believe it was very close to the fair value in in-running betting markets. In fact, I believe those very informed wagering opinions may have had Brady's GWD prospects even a few percentage points higher than our numbers.
#22 by OmahaChiefs13 // Sep 11, 2021 - 9:29pm
So...when it's a wash, choosing to kick doesn't demonstrate a lack of faith in his offense's ability to pick up those 6 yards? Especially after 58 minutes of that offense performing well?
"I'd rather attempt a long field goal and leave time on the clock than let my offense pick up 6 yards" feels like it might indicate a sudden loss of faith in that offense.
Unless there's some faith-based component to kicking that I don't understand, that doesn't apply to other positions.
#34 by Pat // Sep 12, 2021 - 12:23pm
I hate the whole "have faith in your offense" argument, because there's no offense on the planet that can gain 7 yards every play. NFL yards/play is 5.6 yards on average, and highest in the league was 6.4 yards last year.
So for me, the question is really what the defense "state" was - does the coach think there's a weakness they can exploit with a decent chance, or have they been mostly lucking out on broken coverages, etc. And... really, in the previous plays it really did look like Dallas was lucking out on broken coverages. I don't necessarily think that the conversion to Lamb was anything other than a coverage breakdown, for instance. As opposed to the Bucs, for instance, who did look like they were just attacking overmatched players.
But really, to me if you're going to criticize the coach the biggest issues were really that once the coach thought he was in "I'm gonna kick it" range, he should've gone all-in and just stuck with burning clock. Two minutes left, run the ball three times, see what your super-expensive RB can get it, and get the Bucs totally out of timeouts. Think about it - that false start penalty would've forced a 10 second runoff and they'd be seriously time strapped.
#37 by OmahaChiefs13 // Sep 13, 2021 - 8:50am
I hate the whole "have faith in your offense" argument, because there's no offense on the planet that can gain 7 yards every play. NFL yards/play is 5.6 yards on average, and highest in the league was 6.4 yards last year.
Ok, but there's also no kicker on the planet who's going to make 100% of field goals from that distance either.
So why is "faith in your offense" a bad argument, but "faith in your kicker" is a decision-maker?
(For clarity, I don't think either argument is terribly valid...that was kind of my point. I'm just wondering why demonstrating faith in one's kicker carries so much more weight than demonstrating faith in one's offense. I'd think we'd do better without either faith-based argument, if you'll excuse the phrasing)
#24 by jtrucksis // Sep 11, 2021 - 10:22pm
Feels like the NFL is really back with the GWC breakdowns again! Is there an easy way to figure out if Tampa's decision to accept the holding penalty prior to the 4th and 6 was wise? I thought it was relatively insignificant from a FG attempt standpoint but gave the Cowboys an additional 3rd down chance to convert as well as costing Tampa another timeout.
#25 by ImNewAroundThe… // Sep 11, 2021 - 10:38pm
Compounding mistakes? After missing the 31 yarder (and then the extra point), I'd think to just going for it (or punting instead of the midfield miss) for the rest of the game. Gotta be aggressive anyway against the 3rd ranked DVOA offense last year that returned all it's starters. FGs wouldn't cut it. And they didn't.
Ironically Dallas also went for a FG on 4th and 6 earlier in the game. That's where I would've been aggressive but then again Ben Baldwins model said it was a strong FG. Dont think the model takes in the night he was having at that point though. Gotta figure the TB O is still strong and will score again (and they would). And you might not since TB also has a good defense (although they did). Believe Next Gen Stats made a new one that does that sorta thing but they didn't have it up for that play.
New slate for Zuerlein next week though. Wouldn't shy away from him as to further ruin his confidence.
#28 by alan frankel // Sep 12, 2021 - 1:25am
Hmmm
1st whats backing that 56% Brady game winning drive pct?
2nd the bucs kicker isnt automatic either so even if brady gets them in position thiers no guarantee he makes the kick
3rd i think were lowballing dallas chances of converting that 4th&6 as they project to be well above league average offensively.
4th is decision fatigue for a qb real?
Ive had a theory for awhile that as a qb makes more throws his ability to keep making correct decisions falls. It just feels like alot to ask of Dak to execute another do or die play for a few pct points that may or may not be extremely accurate. I understand that ypu pay him to win games but im not sure mccarthy made the wrong call here.
Can we try not have such clickbaity article leads especially when it was not clear cut and different stats systems are giving diff answers.
Its ok not to be so definite on these types of close calls
#42 by JacqueShellacque // Sep 13, 2021 - 3:56pm
I agree. The Cowboys faced 2 options, failure of either of them means an absorbing barrier (loss with certainty). I have to believe that even one such possibility breaks these models - it's one thing to get a general feel for how much you may help yourself converting a 4th and 2 from midfield in the 3rd quarter, an entirely different matter when the decision could lead to an action if, unsuccessful, ends the game. That simply can't be quantified. You can't take the arithmetic average of all the possible outcomes and assign 'losing the game' a 0. In reality it should be undefined.
#36 by JacqueShellacque // Sep 13, 2021 - 8:19am
There's an epistemological problem based on this model highlighted by this point:
"the fourth-down attempt should have been strongly considered and likely would have produced significantly more wins on average"
The model thus assumes that all possible outcomes occur at once, and that one can somehow "achieve" their average. Reality however occurs over paths based on decisions taken. In other words, it assumes ergodicity in a non-ergodic situation (ie, there's at least one outcome that results in certainty that all is lost). In fact the only thing that mitigates the model here is the fact that the Cowboys only had the field goal and 4th down conversion attempt to choose between (it's the only thing that allows for these 2 choices to be even theoretically comparable). It's unfair to McCarthy to call this a critical error, when really humans have an intuitive understanding that paths matter, and it strikes me as odd to present this as a cognitive or even intellectual limitation.