Instant replay review is one of the cornerstones of the modern NFL. The process and its myriad special rules have been internalized and constantly debated. Mike Kurtz wonders: is it worth it?
14 Mar 2011
Guest Column by Daniel Lawver
The home team is down by a touchdown with eight minutes left in the fourth quarter. For the second drive in a row, its offense sputters and punts after a three-and-out. The defense gets back on the field, beaten, broken, and exhausted. If the offense had been able to sustain drives, the defense might have had the energy to prevent the opposing offense from scoring another touchdown, effectively ending the game.
This is a common story told repeatedly by announcers during fourth quarters. If the offense had played better and allowed the defense to rest more, the defense would have played more effectively.
For example, the Arizona Cardinals offense became far less efficient after losing Kurt Warner. Their offensive DVOA plunged from 10% in 2009 to -36% last season. In turn, the offense is running fewer plays per drive, meaning their defense has less time to rest. The Cardinals' defensive DVOA increased from 0% in 2009 to 8% this season. (Recall that a rise in defensive DVOA is a bad thing). So how much of the decline in the Cardinals defense can be explained by the decline in the Arizona offense's ability to extend drives and allow the defense to rest?
We all know the basic conventional wisdom here: Offenses that run more plays per drive should have defenses that rest more and perform better on average. Is it true? We can estimate the effect of the number of plays an offense runs per drive -- and therefore the amount of rest that the defense gets on average between drives -- on defensive performance. To do this, I compare the number of plays that an offense runs per drive to their defense's performance, measured by defensive DVOA.
This is an appropriate exercise for a few key reasons. First, while defensive performance affects the number of plays that an offense runs per game, it is not likely that defensive performance affects the number of plays an offense runs per drive. Second, performance of the defense affects the average starting line of scrimmage for the offense, and therefore potentially affects the number of plays that an offense runs per drive. Defensive performance does affect the average starting line of scrimmage for an offense. However, the average starting position for an offense has a very small effect on the number of plays the offense runs per drive. Over the course of the season, teams that start offensive drives 10 yards closer to the end zone than average run 0.1 fewer plays per drive than average.
One difficulty in doing this exercise is estimating the amount of time that a defense rests per game. Without explicitly measuring the amount of time between drives, there are two potential ways to measure the average amount of time between drives: game time elapsed per offensive drive and number of plays per offensive drive. In this article, I use the number of plays per drive as a measure of the amount of time that the defense rests between drives. Defensive rest, of course, depends of the actual amount of time that elapses between drives, rather than the amount of game time that elapses between drives. My best guess is that the number of offensive plays per drive is a better measure of the amount of time between drives. Regardless, my results do not change if I use game time elapsed per drive instead.
In Figure 1, I plot the relationship between offensive plays per drive and defensive DVOA for all teams between the 2003 and 2009 seasons. If there were a strong relationship between these variables, we would expect to see some correlation. It's clear that we don't. Teams at all levels of defense DVOA have a wide range in the number of plays that their offenses run per drive. There is no strong relationship between the amount of time a defense gets to rest between drives and its performance.
To determine the impact of the number of plays an offense runs per drive, I look at the relationship between offensive plays per drive and defensive DVOA. In Figure 2, I plot this data along with the line that fits the data best. These results actually suggest that an increase in the amount of time the defense gets to rest per drive slightly worsens the performance of the defense. If an offense runs one extra play per drive over the course of a season, we can expect an increase in defensive DVOA by about two percent (once again, meaning a decline in defensive performance).
This estimate, however, is not statistically significant. This means that it is reasonable to believe that the number of plays an offense runs per drive has no impact on defensive DVOA. The amount of time that the defense gets to rest per drive is a very unimportant determinant of defensive performance. Similar results are seen when doing this exercise using data on the performance of the pass and rush defense.
Let's return briefly to the recent decline of the performance of the Cardinals. These results suggest that the decline in the offense's ability to extend drives has not had a strong effect on the performance of the defense. Other factors, especially the loss of Karlos Dansby to free agency in the offseason, have likely played a much larger role.
One shortcoming of this analysis is that I estimate the impact of the amount of time the defense rests per drive on the performance of the defense over the course of the entire game. One might expect that this has the biggest impact on performance during the fourth quarter. I understand this view, but I doubt that the effect is much bigger in the fourth quarter than it is overall. Defensive DVOA measures the performance of the defense over the entire game. If there were a large effect on defensive performance in the fourth quarter, we would see some effect on defensive performance overall.
One important thing that I want to emphasize is that all of this analysis focuses on defensive performance on a per-play basis, as DVOA is a measure of per-play performance. The performance of an offense does have important effects on the performance of a defense over the course of a game or over the course of a season. The number of plays that an offense runs per drive affects the number of plays that the opposing offense runs per game. As a result, an increase in the number of plays that an offense runs per drive will lead opposing offenses to score fewer points per game because they will run fewer plays each game.
Conventional wisdom holds that offenses affect the performance of their defense through their ability to extend drives, and allow the defense additional time to rest. This conventional wisdom does not seem to be supported by the data. As shown here, the number of plays an offense runs per drive has very little impact on the performance of the defense. One simple explanation for this result is that a team's defensive players aren't the only ones that rest while their offense drives down the field. The opposing offense gets to rest too. The net impact, as shown here, is negligible.
Daniel Lawver is a PhD student in Economics at Arizona State University. You can contact him at dafrk3in-at-gmail.com or at this website. If you have a guest column, something that takes an unconventional look at the NFL or college football, you can submit your idea or rough draft at mailbag-at-footballoutsiders.com. The offseason is a great time to submit a guest column.
56 comments, Last at 29 Mar 2011, 6:49pm by Cyrus2