Rushing Success and Play-Action Passing

Guest column by Ben Baldwin

Why do teams run the ball so often? The average pass play gained 6.2 yards in 2017, compared to 4.1 yards for the average rush play. And yet, on first-and-10, teams ran the ball 53 percent of the time. On these first-and-10 runs, 44 percent of rushes and 52 percent of passes were successful.

A common response to the question of why teams run so frequently is that teams need to run the ball in order to maintain the effectiveness of play-action passing. When play-action is based on a committed rushing attack, the argument goes, the pass rush has to slow down and the linebackers have to respect the threat of the run, pulling them towards the line of scrimmage and away from their coverage responsibilities. Examples of this argument abound: The Ringer's Robert Mays recently wrote that "as much as our understanding of the sport has shifted in recent years, the belief that a play-action game's effectiveness is linked to a strong, high-usage running offense has remained steadfast." Former NFL lineman Geoff Schwartz stated that "teams have to commit to running the ball first to open up [play-action]."

Despite the pervasiveness of this argument, I have not seen much evidence about the extent to which the effectiveness of play-action passing is dependent on a team's rushing attack. With thanks to play-by-play charting from ESPN Stats & Info (from 2011 to 2013) and Sports Info Solutions (from 2014 to 2017), this piece takes a deep dive into measuring the empirical relationship between rushing and play-action passing.

Before we move forward, a few notes on the data:

• This uses play-by-play charting of play-action passes from the 2011 through 2017 NFL seasons.
• I group all dropbacks together when discussing the efficiency of passing, meaning that "yards per passing play" counts pass attempts, sacks, and scrambles.
• I do not count fake end-arounds as play-action unless also accompanied by a fake handoff to a player who began the play in the backfield.
• I count a successful rush as one that gains at least 45 percent of the yardage needed on first down, 60 percent on second down, and 100 percent on third or fourth down.
• This piece looks at the specific question of whether a team's play-action effectiveness is related to its rushing. For information on play-action generally, such as how often teams run it and how good they are at it, check out the annual Football Outsiders play-action articles.
• And finally, while it cannot be run without access to the underlying data, I am posting the code I used to generate the figures in this piece here for transparency.

Season-Level Results

This section counts one team-season as the unit of observation and asks the question of whether teams that run frequently or successfully tend to be better at play-action passing. Every team from every season is one dot on the plots below. For example, there is one dot representing the 2011 Denver Broncos and another representing the 2012 Broncos:

Recall, as noted above, that these numbers differ slightly from the official totals because I have re-classified scrambles and sacks as pass plays. The massive improvement in play-action efficiency for Denver between 2011 and 2012 is largely due to the quarterback switch from Tim Tebow to Peyton Manning.

In each figure below, the horizontal axis is a measure of rushing data, and the vertical axis is the average yards per play-action dropback. In the upper left figure below, using (X,Y) notation, the point (515, 7.0) represents the 2011 Broncos and the point (446, 10.5) represents the 2012 Broncos. There are also points for every team's season from 2011 to 2017.

In each figure, the R^2 is printed in the lower right. In the upper left figure, for example, the R^2 of 0.02 means that only 2 percent of the variation in a team's play-action effectiveness (as measured by yards per play-action dropback) in a given season can be explained by the number of rushes in that season.

The three plots above show that whether looking at total number of rushes, the proportion of plays that are rushes, or the proportion of rushes that are successes, no R^2 exceeds 0.03. In other words, knowing a team's rushing frequency or effectiveness in a given season tells one almost nothing about how effective that team was at play-action passing.

Because a team's season-long numbers can be influenced by game script (for example, a team might compile a bunch of rush attempts while salting a game away, when play-action is no longer relevant), I also checked to see whether there is any relationship between play-action effectiveness and these three rushing measures (total rushes, ratio of rushes, or rushing success rate) for plays that occurred while the game was within seven points. The results were similar.

Play-Level Results

The results above suggest that a team's rushing frequency and effectiveness in a season do not predict how successful that team was at play-action passing. But what about within an individual game? Are teams that have not run the ball recently or successfully able to take advantage of play-action passing?

The primary challenge here is constructing measures of rushing frequency and effectiveness for a given point in a game. There is no one number that captures everything about how well and how often a team runs the ball. Because a perfect measure doesn't exist, I tried a bunch of alternatives. As will be seen below, the good news is that they all tell the same story. Here is what I settled on:

1. The number of a team's rushes in the previous five plays (for plays six and beyond in a given game).
2. The number of rushes in the previous 10 plays (for plays 11 and beyond).
3. The number of successful rushes in the previous five plays (for plays six and beyond).
4. The number of successful rushes in the previous 10 plays (for plays 11 and beyond).
5. The ratio of rushes to total plays at that point in the game (for plays 11 and beyond).
6. The rushing success rate at that point in the game (for plays 11 and beyond).

Here are the relative sample sizes of play-action passes given each distinct measure of rushing, with each measure labeled at the top of each individual plot.

To take one example, the far left bar in the upper left chart shows that there were 1,585 play-action dropbacks from 2011 to 2017 in which the offensive team had not run the ball in the previous five plays. Most play-action dropbacks happened when one to three of the previous five plays had been rushes. As we go through the results, keep in mind that some of the sample sizes are very small (for example, there were only 265 play-action dropbacks where the team had zero rushing attempts in the previous 10 plays, about 1 percent of the sample).

Let's begin by looking at whether previous rushing has any relationship to the likelihood that a team showing a handoff will actually drop back to pass rather than handing off the ball. Expressed as a formula, this is:

As background, about 25 percent of shown handoffs end up as pass plays on first and second down, compared to 20 percent on third down and 15 percent on fourth down. Fifty-seven percent of play-action passes are thrown on first down, 36 percent on second down, six percent on third down, and one percent on fourth down.

Here is how each of the six measures of rushing are related to whether a team rushes or uses a play-action pass:

From here forward, each graphic will contain six different figures that show how various characteristics of play-action passing vary with the six different measures of rushing. For example, the far left point in the upper left box shows that teams pass about 22 percent of the time when they show a handoff in the case where they have not rushed once in the past five plays (meaning that they hand the ball off 78 percent of the time).

Looking through the six individual plots, there is some evidence that teams with fewer recent successful rushes are more likely to pass rather than hand the ball off (see lower left box). Pass rate also modestly decreases with rush ratio (lower middle box).

Next, here is the percent of play-action dropbacks that are from a shotgun formation, broken down by previous rushing:

With the exception of teams that have run the ball very infrequently in the last five or 10 plays (who are more likely to operate out of shotgun), there is not much of a relationship. The increase in shotgun rate with very few recent rushes is partially a game-script effect; the median play-action pass where the team did not rush in the previous 10 plays came with the team trailing by 11 points, compared to a median of zero for all play-action passes. But even with the score within seven points, teams with no recent rushes were more relatively likely to run play-action from shotgun.

The next outcome measured is pressure rate on play-action dropbacks. As described above, one of the arguments for rushing affecting play-action passing is that if the defense has to respect the run, then it will affect their pass rush on play-action dropbacks. However, there does not appear to be any support for this hypothesis, as pressure rate is mostly constant regardless of previous rushing, hovering in a range of 27 to 29 percent. Note that for pressure rate alone, I exclude 2017 because I do not have data from that year.

Next, depth of target. For this and the subsequent chart, I show the median (lower edge of each bar), 75th percentile (upper edge), and mean (circle). This is a modification of a typical box plot where I do not show the 25th percentile because it is zero for the yards gained plot and thus uninformative.

The mean and median depth of target is mostly constant regardless of a team's previous rushing statistics.

And finally, yards per play on play-action dropbacks:

This is the main relationship of interest. Regardless of which of the six measures of rushing one chooses, there is no meaningful relationship between the effectiveness of play-action passing and a team's rushing statistics in the game to that point. Aside from a couple extreme cases with very small sample sizes (zero rushes or eight rushes in the previous 10 plays), there is no relationship in the data between the median, mean, or 75th percentile of yards gained and a team's previous rush attempts. This is consistent with the scatterplots of team rushing versus team play-action passing for entire seasons that were displayed in the beginning of this article.

Here is the data in the lower middle figure in table form:

Between 2011 and 2017, 93 percent of play-action passes occurred when the offense had between one and four rushes in the previous five plays. In this range, yards per play and its standard deviation are remarkably similar for all values of previous rush attempts. Looking at the graph of rushes in the previous 10 plays, teams are at least as successful at play-action when they have rushed one time in the previous 10 plays as when they have run seven or eight times in the previous 10 plays.

Putting this all together, I cannot find any support for the success of play-action passing being related in any way to a team's rushing statistics, whether measured by frequency or effectiveness.

Final Thoughts

Coming into this, I did not know what to expect. Since play-by-play data on play-action passing is not readily available, it was something I had long wondered but never been able to look into. After measuring this every way I could think if, it appears that the conventional wisdom that running is necessary for play-action passes to be effective should be questioned. We have a lot of evidence that play-action passing is more effective than non-play-action passing, so the big question that remains is why teams run play-action so infrequently (the percentage of passes that are play-action has hovered around 20 percent since 2011). What would happen if teams started devoting a higher share of their plays to play-action passing? Would the advantage persist or would defenses adjust?

The recently-concluded 2017 playoffs may provide a glimpse into a future where play-action is more common. The Eagles attempted 21 play-action passes in the Super Bowl on 43 dropbacks (49 percent). Frequent use of play-action (33 percent of dropbacks against the Patriots and 54 percent against the Steelers) also helped the Jaguars score 65 points across two playoff games and nearly reach the Super Bowl. In the constant search for advantages in a competitive league, play-action passing appears to be an under-utilized edge.

An economist by trade, Ben Baldwin uses large datasets to try to learn about human behavior. His work can be found on Field Gulls and Grid Fe. Reach him on Twitter at @guga31bb.