Play-Action Passing and Game Conditions

Guest column by Ben Baldwin

With the Rams and Patriots each finishing in the top four in both play-action rate and play-action yards per play (subscription required) on their way to reaching the Super Bowl, play-action passing will continue to be a topic of interest.

In this piece, we're going to attack some of the questions generated in my last piece. As always, the data notes from my first investigation into play-action apply: this uses data from the 2011 through 2018 NFL regular seasons, and I'm counting all dropbacks, including those that ended in a sack or scramble. When calculating success rate, I use the 45/60/100 definition.

For the analysis in this piece, I'm restricting the sample to first and second downs only. Because 93 percent of play-action passes come in these situations, early downs constitute the vast majority of play-action use and the bulk of plays where both rushing and passing are threats.

With that out of the way, let's get to it.

Question 1: Is play-action less effective when teams are trailing and the run becomes less of a threat?

This is a question that I have been asked repeatedly and never had time to investigate, but it gets at what drives play-action effectiveness. With the threat of a run diminished for teams facing large deficits, are defenses less vulnerable to play-action?

Let's start with simple plots of yards per play (left) and success rate (right) for play-action dropbacks, with the horizontal axis representing the score differential from the perspective of the offense:

For both yards per play and success rate, we see an upward-sloping relationship, where having the lead is associated with more effective play-action passing.

However, looking at efficiency by score differential is somewhat tricky because the teams that fall behind in the first place tend to be bad passing teams. In other words, even if there were no causal relationship between score differential and play-action, we might expect trailing teams to be worse at play-action because those are the teams that are worse at passing to begin with.

There are a number of different ways we could deal with this, but a simple one is to measure the difference between play-action and non-play-action dropbacks. Here's what that looks like:

In the figure above, each point displays the difference between play-action and non-play-action dropbacks for a given score differential. For example, in tie games, early-down play-action dropbacks gain 7.1 yards per play, compared to 5.9 for non-play-action, yielding a difference of 1.2, which can be seen on the figure above.

Even after accounting for bad offenses being more likely to fall behind in the first place by subtracting out non-play-action efficiency at each score differential, play-action dropbacks gain more yards per play than non-play-action at all score differentials within three touchdowns (greater differentials face sample size challenges). At the same time, there is still an upwards slope, meaning play-action is still most effective for teams with a lead.

Success rate tells a somewhat different story, with lower success rates when facing big deficits, perhaps because these teams are taking more shots deep down the field on their play-action dropbacks. Because trailing teams usually need big chunk plays to get back in the game, we probably shouldn't take this as evidence that play-action should be abandoned when teams are trailing. Play-action is most effective when a team is ahead, but still effective -- as measured by yards per play -- when a team is trailing.

Question 2: Does play-action passing become more effective as game goes on?

This question was inspired by Josh Hermsmeyer's piece on FiveThirtyEight examining whether play-action passes become less effective at luring linebackers towards the line of scrimmage as teams use more play-action plays throughout a game.

Instead of looking at how many times a team has utilized play-action, let's look at how many early-down (i.e., first- and second-down) plays a team has run during the game. If play-action succeeds in part by calling plays in the present to set up other plays in the future, we might expect play-action passes to become more effective as the game goes on. Is that the case?

Looking at the figure above, play-action passes appear to be slightly more effective at the very beginning of the game. From previous work, we knew that teams don't need to run first in a game in order to set up play-action, and here is yet another piece of evidence supporting that assertion.

The drop in play-action effectiveness around the 40th play is a bit of a mystery. The drop is present even in close games, so it's not driven by teams with a big lead playing conservatively.

What about the difference between play-action and non-play-action dropbacks?

This looks pretty similar to the above figure, with play-action being just as effective at the beginning of the game -- if not more effective -- as in later stages. When looking at yards per dropback, play-action is substantially more effective than non-play-action throughout the game.

For fun, here's an example of a successful play-action dropback on the very first play of a game:

(Click here if you are having trouble loading the image.)

In Week 10 -- Freddie Kitchens' second week as Cleveland's offensive coordinator -- the Browns opened with this play. Though it had not been set up by the run, the linebackers were still sucked towards the line of scrimmage, opening a huge throwing lane for Baker Mayfield and an easy first down.

Question 3: If a team uses play-action substantially more than it hands the ball off, does play-action suffer?

This is a follow-up raised in the comments section of the last piece. Perhaps the relevant measure of play-action frequency isn't the percent of dropbacks that are play-action, but rather the percent of "shown handoffs" that are passes. In mathematical terms, let's measure:

-- the percent of time a team shows a handoff that leads to a play-action dropback.

Do teams that run the ball less frequently when showing handoffs have worse play-action efficiency? Here's what the relationship looks like at the team-season level:

In the above figure, each point represents a team's season (for example, the 2018 Rams). Perhaps not surprisingly, there's still not much correlation here (the R-squared values for both figures is 0.00; results are similar when looking at plays with the score within seven points), and discovering a relationship between rushing frequency and play-action effectiveness continues to prove elusive.

Notably, the five highest team-seasons in play-action rate by this definition all came in 2018: the Rams, Panthers, Chiefs, Eagles, and Falcons. In addition, 2018 was the highest average for the league at large by a fair amount. As teams rush less often and use play-action more often, the share of "shown handoffs" that are play-action has risen over time.

Conclusion

We have now covered a lot of ground on play-action across multiple pieces, so let's try to summarize everything we now know.

First, play-action passes are most effective when there's the greatest situational threat of a run. Examples include play-action being more effective from under center than shotgun, being more effective on early downs, and being more effective for teams with a lead.

Second, coaches appear to believe that rushing more often helps set up play-action: teams that run more often use play-action more often.

Third, despite the above, there is no relationship between how often a given team has actually run -- whether over the course of a season or recently in a game -- and the effectiveness of their play-action attack. In other words, the threat of rushing is sufficient to set up play-action; teams don't actually need to run first before taking advantage of it.

Fourth, as discussed above, play-action does not become less effective as the game goes on or as play-action is used more times in a game.

Fifth, play-action primarily works by increasing the frequency and success of passes thrown to the intermediate part of the field, with depths of target between 10 and 20 yards downfield. This is because linebackers are sucked towards the line of scrimmage, opening up space behind them. This effect lasts throughout the game.

Perhaps this will look silly later on, but I'm getting the sense that we're beginning to approach the limits of what there is to be learned about play-action without having access to the NFL's player tracking data. That would open up a whole new set of questions, including the exploration of route concepts, how each defender reacts, and other factors that drive play-action's effectiveness. In the meantime, we can continue to expect play-action usage to rise over time as other teams emulate the league's most successful offenses.

An economist by trade, Ben Baldwin uses large datasets to try to learn about human behavior. He can be found covering the Seahawks for The Athletic Seattle and on Twitter @benbbaldwin.