Guest column by Cole Jacobson
If it happens to the team you're rooting for, you're pissed. If it happens to the team you're rooting against, you get cocky. But no matter what side you've been on, you've seen it happen: the offense using the final seconds of the first half to take a knee or otherwise kill the clock rather than trying to score.
Recent years have seen NFL coaches improve vastly in terms of avoiding old habits of risk aversion, whether that comes down to being more willing to roll the dice on fourth downs or on two-point conversion attempts. But despite this overarching trend, we still regularly see coaches decline to take risks when the game clock approaches halftime. Often we hear the justification of keeping momentum, and often we hear the ridiculous notion of "well, the offensive team gets the ball to open the second half anyway," as if who receives that kickoff should make any impact on a team's goal to maximize its points during the first half. Observing this pattern leads to a natural question: should coaches be more aggressive with the ball in the final seconds of the first half?
To answer this, I started with all of the play-by-play data available via NFLScrapR for the past 11 completed seasons (2009 to 2019). I removed all timeouts, kickoffs, and PATs, and then isolated all plays to occur in the last two minutes of the first half. PATs are removed because before 2018, NFLScrapR had a data entry flaw in which all PATs following defensive touchdowns were omitted. Kickoffs were removed because a kickoff return touchdown does not at all reflect a coach's offensive decision-making, and therefore isn't relevant to our question.
From there, I grouped all plays by the drive on which they occurred, creating a new data frame called PointsScoredOnDriveTable, which includes 4,347 drives. This table includes core information about every offensive drive to start in the final two minutes of a first half, including points scored, time remaining at the start of the drive, how many timeouts the offensive team had at the start of the drive, starting field position, and more.
Summary of the project: in most instances, coaches are deciding to be too passive in late first-half scenarios when it would benefit the offense to stay on the field. However, when it comes to situations when there is only time for one play before halftime, coaches are actually extremely accurate in deciding when to take a shot at the end zone.
The Basics: Are Teams Wasting Opportunities When They Could be Scoring?
For brevity and common sense, we can limit the bulk of the project to drives that start in an offensive team's own territory. If you can't figure out that attempting to score is the proper move when your starting field position is at the 50-yard line or inward, your pure fundamental cowardice goes beyond anything this paper could address. Most NFL coaches agree with this stance; since 2009, there have only been three kneeldowns taken on non-scoring drives in the final two minutes of the first half at the 50-yard line or closer, and one was accidental.
One more key distinction before getting into the analysis: taking a knee isn't the only way for an offense to demonstrate no real intent of scoring. If the offense gets the ball at its own 25 with 40 seconds left and executes one or two run plays before the clock hits zero, everyone watching the game knows that the offense meant to run out the clock. Because of this, I created a new metric, called "Quit" -- any drive was labeled a "Quit" if it started in the offense's own territory in the final two minutes of the half and did not include any pass attempts, offensive timeouts, or offensive touchdowns. (This last modification was because of a Chris Johnson 76-yard touchdown run in 2010 on the opening play of a drive starting with 1:56 left; we can't necessarily say that the offense's goal was to run out the clock there.) It's not a perfect metric, but within reason, if the offense doesn't pass the ball or call timeout once in a two-minute drill situation, we can say there was no real intent of scoring. The numbers back this up; since 2009, there have only been three rushing touchdowns of 50-plus yards in the final two minutes of the first half. (The other two came on drives that also included passing plays, and therefore wouldn't be labeled as "Quits" anyway.)
With these disclaimers out of the way, we can start by looking at the most basic question of all: how often are coaches "quitting" in pre-halftime situations to begin with? I did make a linear model (which I can show to any true statistics nerds interested), but what's much easier to interpret are the following graphs, which display data from the past 11 years. Keep in mind that the "Starting Field Pos" refers to distance from the opposing end zone -- e.g., 90 on the graph means the offense's own 10-yard line.
Looking at four plots simultaneously can be a bit overwhelming. So, here is the same data, but without considering how many timeouts the offense had:
We can see that "quitting" on a drive is very uncommon in most areas of the field when more than 40 seconds are left on the clock. If we use the approximations that result from not considering timeouts, "Quits" occur more than half the time inside a team's own 20-yard line in the 21- to 40-second range, and inside a team's own 40-yard line with 20 or fewer seconds.
Are coaches leaving points on the board in these situations? To answer that, the simplest way is to look at the net offensive points on each drive. This is exactly what it sounds like, taken straight from the scoreboard; i.e., an offensive touchdown goes down as +7, and a field goal as +3, but allowing a pick-six or punt return touchdown would go down as -7. Below are graphs that display the average net points for all "non-quitting" drives in the last two minutes of the half, starting in the offense's own territory:
Using these graphs, it looks like a pretty open-and-shut case; in just about every situation, the offense is more likely to gain points than lose them. So, we're done, right?
Unfortunately, it isn't that simple. A large part of the reason that coaches kill the clock is that a turnover in their own territory could be deadly, allowing the opposing team to get quick points before the half even if the play itself doesn't result in a score. Strictly looking at the scoreboard after a turnover doesn't account for this, which is why we have to make a new metric.
A New Metric: Adjusted Net Points
We can already easily quantify how much better a touchdown is than running out the clock, and how much worse throwing a pick-six is than running out the clock. But if the offense punts or turns the ball over on a drive, and that play does not directly result in a touchdown, how much worse off is the offense than it would have been if it had just knelt out the clock? To answer this, we can turn to one of NFLScrapR's unique traits: expected points added, or EPA.
With the help of EPA, we can get a very good estimate of just how harmful those drives are. For turnovers, we can simply look at the average EPA lost on all relevant drives (i.e., last two minutes, own territory) that resulted in a non-scoring turnover, which turns out to be approximately 2.6 points. The process for punts not returned for touchdowns is slightly more complicated, and I will share methodology details to anyone interested, but it results in an average loss of approximately 0.7 points. As a result, we get a new metric, Adjusted Net Points, which lets us evaluate the impact of any outcome:
NetPoints - (2.584 * NoScoreTurnovers) - (0.6634 * NoScorePunts)
Now, we can finally get to the question we want to answer: when is letting the offense try to score more beneficial than running into the tunnel with the score intact?
We can see that the majority of situations are expected to result in a net positive for the offense, even when we're isolating drives that start on the offense's own side of the field. But is it enough of a majority to suggest that coaches should be more willing to take a gamble?
Looking at the graph without timeouts for the sake of simplicity, it's almost never a good idea to "Quit" with 41 seconds or more left on the clock. When it comes to the 21- to 40-second range, the average threshold of where drives become detrimental to the offense is their own 14-yard line. Based on the graph's trajectory, this 14-yard line approximation would also be fairly accurate for the 1- to 20-second range if it had a big enough sample size to register, but there have only been two drives to start inside the 15 with 20 or fewer seconds left that had an attempted pass play since 2009.
Does this match up with the rate at which coaches are "quitting?" We earlier established that "Quits" occur more than half the time inside a team's own 20-yard line in the 21- to 40-second range, and inside a team's own 40-yard line with 20 or fewer seconds to go. Put two and two together, and we have a clear result: coaches are indeed "quitting" on drives more often than they should.
Last Play of the Half: Do I Take the Shot?
For this isolated situation, we can forget the concept of drives altogether. If I only have time left for one single play before halftime comes, do I take the shot and try to score, or accept the score as it is?
I created a data frame called FinalPlayOfHalf_OwnTerritory_Table, which consists of the 1,775 plays since 2009 that started in the offense's own territory and ended the first half. I kept the same "Quit" metric as before, referring to any play that was either a rush attempt or kneeldown as a "Quit" (there has never been a rushing touchdown of 50-plus yards on the final play of a first half since 2009).
We can see that the own 39-yard line is right around when coaches become more likely to pack it in. Is this the right strategy?
Since we're only looking at situations where there's one play to go, we can throw out the "Adjusted Net Points" metric, because turning the ball over doesn't matter unless that turnover happens to get returned to the house. All that matters is whether the offensive team is more likely to score than get scored on for that one specific play, if they are throwing the ball.
The smaller sample size here leads to some unconventional-looking data, which is why I included the full-field version too. Both graphs suggest that approximately the offense's own 38-yard line is where a pass attempt on the final play of the half becomes ill-advised. Thus, when it comes to situations where there's only one play to go, coaches are hitting the nail on the head.
For Fun: Most Egregious Kneels and Quits
To give more concrete examples of how this logic can be beneficial, here are some of the most egregious decisions to kneel or "Quit" since 2009. We have to separate the two concepts, because a drive that featured a pass attempt and then a kneeldown wouldn't be classified as a "Quit," since the decision to "Quit" came during the drive rather than at the start.
The most egregious kneeldown by far was the aforementioned "accident." In 2015, the Redskins had the ball at the Eagles' 6-yard line with six seconds remaining in the first half. Kirk Cousins forgot how football worked, and took a knee when he was supposed to spike the ball to stop the clock. The Redskins lost out on 3.33 expected adjusted net points there.
As for kneeldowns that were actually called by the coaching staff, the worst comes from a surprising name: Bill Belichick. In 2016, the Patriots had possession with 1:00 left and three timeouts at their own 34-yard line against the Texans, but chose to take a couple of kneeldowns to enter halftime up 10-0. They lost out on 1.23 expected adjusted net points. This came during Tom Brady's four-game "Deflategate" suspension, with Jacoby Brissett making his first career NFL start. It's fair to say Belichick might have been less passive with his ideal quarterback under center.
If we strictly look at field position, the worst kneel besides Cousins' came in 2010, in Jason Garrett's first career game as the Cowboys' interim coach. The Cowboys had driven to the Giants' 44-yard line, but after a false start and an incompletion put them back to the 49, Garrett chose to take a knee on the final play of the half. The Cowboys lost out on 0.75 expected adjusted points according to a linear model for half-ending plays specifically.
As for the worst "Quits" rather than worst kneeldowns, two nearly identical drives stood out. One came from Jim Harbaugh's 49ers against the Houston Texans in 2013; the 49ers took possession at their own 29 with 1:32 remaining and three timeouts, but ran the ball three straight times for a combined 5 yards and then punted just before the half. The 49ers had 1.12 expected adjusted net points to start the drive. (San Francisco led 21-0 at the time, perhaps contributing to its conservative mindset). Similarly, in 2016, Belichick's Patriots had 1:24 left with all three timeouts and started at their own 31, but their drive consisted of five runs, two of them wiped out by their own penalties, for a combined loss of 8 yards before punting. This was in Jimmy Garoppolo's first career start, during the "Deflategate" suspension. The Patriots, who led 10-7 at the time, started the drive with 1.11 expected adjusted net points.
Possible Sources of Error/Other Comments on Methodology
Like any football analytics project, these findings shouldn't be blindly obeyed in all contexts. Game situations matter, and analytics are used properly when they're helping teams make informed decisions rather than forcing coaches to disregard all other factors at play. For example, player personnel makes a major impact. If a coach isn't particularly confident in his quarterback -- as we saw with Belichick for Brissett and Garoppolo -- he has more justification to take it easy. Similarly, score of the game plays a role as well; if one team is down 31-3 right before halftime, it has incentive to be more aggressive than usual.
On a similar note, the argument of morale can also come into play. Any of us who has ever played a defensive snap has felt that morale boost when we see opponents take a knee right before the half because they doubt they can score. But does this actually make a quantifiable impact on how the game will go after halftime? It would be an extremely small one, if any, and even though I'm all for coaches sending the right message to their players by playing to win instead of not to lose, this project was not the place to approach that debate.
Some readers may be wondering why EPA didn't consist of a bigger portion of the project. The answer is pretty simple; as informative as EPA is in many contexts, the core flaw of EPA as it pertains to this project is that it's based on an offense's expected production entering a drive. For example, say a team starts a possession at the opponent's 10-yard line, with an EPA of 5.0, but then settles for a field goal. This drive would have an EPA of -2.0 points, because it resulted in two points fewer than originally expected. But obviously, choosing to leave the offense on the field was a smarter decision than taking a knee would have been, since the offense still got three more points than they would have if they had taken a knee.
The above graph actually reaches similar conclusions as our earlier graphs (i.e. that "quitting" is not smart with 41-plus seconds left, and that it becomes beneficial at roughly the own 15-yard line with 40 or fewer seconds left). But it shows that being farther from the own end zone often leads to fewer EPA, which is very misleading, because the offense is still more likely to score as its field position gets better. WPA (win probability added) has the same issue, though to a smaller extent.
One more topic worthy of debate comes in the labelling of certain drives as "Quits." Needless to say, there's no one right way to label these correctly. For example, in 2011, Mark Sanchez lost a fumble on the opening play of a drive from the Jets' own 49-yard line with 0:53 left -- this was labeled as a "Quit," but can we guarantee the Jets wanted to run out the clock here? I decided that my criteria of "no pass attempts, no timeouts, no offensive touchdowns" were as accurate as we were reasonably going to get. The more qualifiers you add, the more unnecessarily complicated the project becomes to the average reader.
Looking at the graphs, an unfortunate concession I had to make was to stratify the time remaining into 20-second groups. This is because of the impossibility of having two continuous predictor variables displayed at once while having the graph make sense visually. There's a big difference between 21 seconds and 40 seconds being left on the clock, and the linear models of the project reflect this, but when it came to the graphs, this stratification was a necessity to make the images at all comprehensible.
Thanks for the read, and I hope to hear any positive or negative feedback. I'd like to give a special thanks to Keegan Abdoo of NFL Next Gen Stats and Bailey Joseph of the Oklahoma City Thunder for giving specific R tips.
Cole Jacobson is an Editorial Researcher at the NFL Media office in Los Angeles. He played varsity sprint football as a defensive lineman at the University of Pennsylvania, where he was a 2019 graduate as a mathematics major and statistics minor. With any questions, comments, or ideas, he can be contacted at email@example.com and @ColeJacobson32 on Twitter.
UPDATE: Predictive Graphs -- When Should Teams Try to Score?
While informative, the above graphs have a few key issues. One is that they can be difficult to interpret without much of a statistics background. Another is that I had to make the concession of stratifying, or binning, the times into 20-second groups, which unfairly treated 0:21 and 0:40 as the same. This was because the graphs would have looked even more convoluted if there were two continuous predictor variables displayed at once, rather than one continuous and one categorical. Lastly, the graphs look back at past data, rather than directly helping out with future decisions.
To combat these issues, I used linear regression models based on that past data to create predictive graphs. If you're looking for an all-encompassing graph telling you when you should be calling plays with the intent to score, in the style of the New York Times' famed 4th Down Bot, here it is:
While it should be self-explanatory, the blue area means that the offensive team is likely to benefit from trying to score (using the "Adjusted Net Points" formula), while the red area means it is better off going into halftime with the score as is. Along the same lines, we can use this format to display when coaches have usually gone for it in the past, with the blue area meaning that coaches tried to score at least 50% of the time in those situations, and the red representing under 50%:
We can see that the red triangles are way wider in the bottom set of plots. This means that when it comes to field position, coaches are being too passive; in other words, it's almost always a good idea to try to score when your field position is at your own 30-yard line or better, but coaches neglect to do so in some situations. However, the red triangles are also shorter in the bottom plot. This means that when it comes to time on the clock, coaches are actually being too aggressive. Coaches almost always try to score with more than one minute left on the clock regardless of field position, but this is an ill-advised move if the field position is particularly bad, e.g., inside the own 10-yard line.
To summarize, it is still true that coaches are slightly too passive on average -- though it's difficult to tell to the naked eye, there is indeed more blue space in the top set of plots than the bottom. But now we have a more nuanced view of the thought process; coaches overreact to seeing a lot of time left on the clock, while they don't react enough to the prospect of above-average starting field position.