Stat Analysis

Advanced analytics on player and team performance

Situational Play-Action Passing in the NFL

Guest column by Ben Baldwin

Readers of Football Outsiders' annual play-action offense series will be familiar with a couple of basic facts about play-action passing: first, that play-action passes are generally more effective than non-play-action passes; and second, that teams rarely use play-action passes. In 2016, for example, play-action dropbacks gained 7.8 yards per play, compared to 6.2 yards per play for non-play-action dropbacks, but only 18 percent of dropbacks leaguewide used play-action.

Because the NFL does not include a play-action indicator in the play-by-play data that it releases to the public, there is little else we know about play-action. When do teams use it? When is it effective? Should teams be using it more often? This piece takes a first step in trying to get a better handle on these questions.

The notes from my previous Football Outsiders piece on play-action apply here: scrambles and sacks count as dropbacks in all yards per play calculations; success rate calculations use 45 percent of the yards to go on first down, 60 percent of the yards to go on second down, and 100 percent of yards to go on third and fourth downs; and all results are from data lasting from the 2011 through 2017 seasons.

Basic Patterns of Play-Action Passing

To start, here is a basic table showing how often teams pass, operate from shotgun, and use play-action in various down and distance combinations. For brevity and due to sample size issues, I do not show all situations (for example, no plays with more than 10 yards to go or plays on fourth down are shown). I group second-down plays into 5 to 10 yards to go and 1 to 4 yards to go because second-and-5 is the threshold for when teams pass more than 50 percent of the time. For third downs, I group 3 to 10 yards to go into one group because teams pass the vast majority of the time in this range (yes, even on third-and-3). The ratio of rush plays to pass plays is important because, as we will see, the frequency and effectiveness of play-action is related to the threat of rushing. Note here that I am being careful to say threat of a rush, because a team's actual rushing performance does not seem to matter when measuring a team's performance in play-action passing:

 Situational Play-Action Passing, 2011-2017 Situation Pass% Shotgun% % of passes from PA Pass Plays PA passing Non-PA passing Yards/Play Target Depth Yards/Play Target depth 1st & 10 52% 43% 31% 47512 8.2 11.0 6.4 8.3 2nd & 5-10 63% 55% 23% 30881 7.2 9.5 6.1 7.7 2nd & <5 42% 43% 25% 6816 6.0 8.1 5.5 7.1 3rd & 3-10 93% 93% 3% 28366 5.8 6.2 5.8 9.4 3rd & <3 48% 49% 19% 4381 6.7 6.3 5.1 6.5

There's a lot to take in here. Let's start with the obvious: in passing situations, teams are more likely to pass, more likely to operate out of shotgun, and less likely to use play-action. This is most pronounced on third down -- on third-and-long, 93 percent of plays are pass plays, and 93 percent of plays are from shotgun, while only 3 percent of passing plays use play-action. Focusing on play-action frequency, teams are most likely to use play-action when the threat of a rush is highest (i.e., first-and-10 along with second-and-short). Makes sense so far.

Comparing play-action to non-play-action passing, play-action carries a yards per play advantage in all of the situations shown in the above table except for third-and-long, when rushing is not a serious threat. Looking at target depth, the average play-action pass is thrown further downfield on first and second down, but not on third down.

Next, let's look at a similar table but restricted to snaps from under center:

 Situational Play-Action Passing from Under Center, 2011-2017 Situation Pass% % of passes from PA Pass Plays PA Passing Non-PA Passing Yards/Play Target Depth Yards/Play Target Depth 1st & 10 36% 58% 18701 8.3 12.0 6.6 8.7 2nd & 5-10 44% 49% 9713 7.3 10.0 5.9 7.8 2nd & <5 25% 54% 2295 5.8 8.7 5.0 6.3 3rd & 3-10 55% 23% 1190 4.9 7.1 4.9 7.2 3rd & <3 25% 56% 1179 7.2 6.6 4.5 5.3

Comparing this table to the previous one, there are striking increases in the percent of plays that are run plays and the percent of passes that are play-action passes. Again, this is consistent with play-action being used most frequently when the threat of a rush is strongest.

When under center, the advantage that play-action passing has over non-play-action passing remains large, with the continued exception of third-and-long. On first-and-10, especially, there are very large differences between play-action and non-play-action passes in both yards per play (8.3 to 6.6) and target depth (12.0 to 8.7).

Here is the corresponding table for shotgun snaps:

 Situational Play-Action Passing from Shotgun, 2011-2017 Situation Pass% % of passes from PA Pass Plays PA Passing Non-PA Passing Yards/Play Target Depth Yards/Play Target Depth 1st & 10 73% 14% 28811 7.8 9.2 6.4 8.2 2nd & 5-10 78% 11% 21168 7.1 7.8 6.1 7.7 2nd & <5 65% 10% 4521 6.2 6.6 5.7 7.4 3rd & 3-10 96% 2% 27176 6.3 5.7 5.8 9.4 3rd & <3 72% 6% 3202 5.0 5.1 5.2 6.7

When operating from shotgun, teams are very likely to pass and generally unlikely to use play-action when they pass. At first glance, this is a bit of a mystery because play-action passing appears to be more effective even from shotgun, with the exception of third-and-short. In addition to reduced play-action frequency, another notable difference from the under-center numbers is that the gap in target depth between play-action and non-play-action is not as pronounced from shotgun. For example, compare the first-and-10 under-center air yards differential (12.0 to 8.7) to the shotgun air yards differential (9.2 to 8.2).

Play-Action Passing by Target Depth

Let's continue with an investigation of the success of play-action from shotgun and under center on first-and-10. With thanks to Josh Hermsmeyer of AirYards for the inspiration behind the following figures that showcase target depth, the figure below shows yards per play by depth of target (all figures from here forward include both complete and incomplete passes):

The figure above shows data for first-and-10, which has by far the largest sample size. On each plot, the X's represent play-action yards per play by distance, while the diamonds represent non-play-action yards per play by distance, with best-fit lines for play-action (red) and non-play-action (black) also included. In addition, there are dotted lines showing confidence intervals around the play-action best-fit line to give a sense of how confident we should be about average yards per play at a given target depth.

The figure shows that yards per play increases with depth of target, a pattern which will likely not surprise regular readers of Football Outsiders. Thus, there are two potential reasons why play-action passing could be more effective than non-play-action passing -- first, because passes of a given distance are more efficient; second, because the average target depth is greater. The plot on the left in the above figure shows that on first-and-10, play-action passing is more effective on passes of a given distance (as measured by yards per play) starting on throws that travel more than 10 yards down the field, and appears to be more effective at all further distances. (To my knowledge, play-action becoming more effective at greater depth of target was recently pointed out for the first time by Hermsmeyer here.) The right plot shows that, despite play-action being used far less frequently on shotgun snaps than non-shotgun snaps, play-action passing is also more effective when used out of shotgun on first-and-10 on throws more than 10 yards downfield.

When looking at completion percentage by target distance instead of yards per play, we see a similar pattern for the relative effectiveness of play-action versus non-play-action passing:

Now that we know where play-action passes are more effective for given target depths on first-and-10, we can ask whether teams actually target those depths on play-action passes. The next plot shows the distribution of target depth under center and from shotgun:

These plots show the percent of passes thrown to a given target depth among throws between -4 and 40 air yards (sample sizes get very small outside of this range). From under center and, to a lesser extent, shotgun, we see greater density in the 10- to 20-air yards locations for play-action passes relative to non-play-action, which is exactly where play-action passing starts to become more effective. This is a wonderful case of data being able to capture something inherent to the complex game of football: play-action passes work in part by pulling linebackers closer to the line of scrimmage and then throwing it behind them to the vacated space. Play-by-play data show that play-action pulling the linebackers forward makes teams more likely to target the intermediate range of the field, and it makes those targets more effective.

Here is an example of play-action at work (on this play, depth of target is 15 yards):

Going back to the ways that play-action could theoretically help an offense (target depth and/or effectiveness at a given depth), it appears that on first-and-10, play-action increases both target depth and the efficiency of passes of a given depth (more than 10 yards down the field).

To this point, I have focused on first-and-10 because that is where the bulk of the sample size lies. Let's take a look at second down:

Play-action and non-play-action passing is split pretty evenly on snaps from under center on second-and-5 to 10, but the vast majority of passes from shotgun are non-play-action. However, there's still evidence that play-action is more effective to certain depths on the field, both under center and from shotgun. On second-and-5 to 10, there's some evidence that the benefit of play-action on shotgun snaps is further downfield, in the 20- to 30-air yards range. However, we do not see an increase in the percent of passes thrown downfield on play-action passes from shotgun on second down like we do from under center.

On second-and-short, play-action appears to be more effective than non-play-action when lining up under center, but not from shotgun, but sample sizes are smaller here.

Here is data for third downs:

On third-and-long, play-action does not appear to be any more effective than non-play-action, whether from under center or shotgun. (I have omitted the error bars on the completion percentage plot because they are so wide that they break the figures.) Perhaps this should not be surprising, because the defense does not need to respect the run on these plays. In an effort to understand how defenses might be playing these obvious passing situations differently, I reached out to Football Outsiders film expert Charles McDonald about what defenses might be doing differently on passing downs. Here is his response:

You don't need to play the run in third-and-long situations. Linebackers are taught to read the helmets of the offensive line to get their initial run/pass reads. If they fire out with their head down, play your run fit. If they sit back and slide into pass protection, drop to your coverage responsibility. However, on third-and-long, linebackers are always playing the pass first since they'll have enough time to recover and play downhill for a run stop on the off chance the offense tries to run it.

Looking at the distribution of target depth on third-and-long, there are a great deal of play-action passes close to the line of scrimmage, which are presumably screen passes or checkdowns. On third-and-short, play-action appears better from under center and worse from shotgun. I don't have a good explanation for this except to note that the sample size of play-action shotgun snaps here is very small.

The question of why teams use play-action so infrequently has risen in prominence recently. I believe we now have two partial answers, but not a full answer.

First, teams rarely use play-action in obvious passing situations, which is understandable because play-action does not appear to be any more effective in these situations. Thus, while it is true that teams use play-action on only 18 percent of dropbacks, a more relevant statistic might be play-action usage on dropbacks that are not obvious passing downs.

Second, when lining up under center, teams do use play-action frequently when they pass the ball (on first-and-10, 58 percent of passes from under center use play-action). The question becomes, as it often does, why teams run so frequently. On first-and-10 from under center, teams run 64 percent of the time, pass using play-action 21 percent of the time, and pass using non-play-action 15 percent of the time. The average yardage gained on these plays is 4.3 on rush attempts (39 percent success rate), 8.3 on play-action passes (48 percent success rate), and 6.6 on non-play-action passes (46 percent success rate). While it is possible that play-action would become less effective if the whole league collectively ran less frequently on first-and-10, it remains a mystery why teams are reluctant to try this.

After working on this piece, one question that stands out is why teams seldom use play-action from shotgun formations, especially on first-and-10 where the evidence is strongest that it works. On first-and-10 from shotgun, play-action passes have more yards per play (7.8 to 6.4) and target depth (9.2 to 8.2) than non-play-action passes. And yet, only 14 percent of first-and-10 shotgun pass attempts use play-action.

Another question that is not directly related to play-action passing concerns the relative rushing frequencies on first-and-10 from under center versus shotgun. On first-and-10 from under center, teams rush 64 percent of the time, compared to 27 percent of the time from shotgun. However, the average first-and-10 rush gains 4.3 yards from under center (33 percent success rate) and 4.8 yards from shotgun (39 percent success rate). It is possible that rushes from shotgun are more effective in part because they are less frequent, but the large difference in rush ratio from under center versus shotgun is noteworthy.

An economist by trade, Ben Baldwin uses large datasets to try to learn about human behavior. His work can be found on Field Gulls and GridFE; reach him on Twitter @guga31bb.

10 comments, Last at 15 Mar 2018, 1:31pm

1Re: Situational Play-Action Passing in the NFL

by Pat // Mar 12, 2018 - 12:33pm

"Thus, while it is true that teams use play-action on only 18 percent of dropbacks, a more relevant statistic might be play-action usage on dropbacks that are not obvious passing downs."

Great point. Awesome to see it completely fleshed out. One additional point here, though, is that there's an *additional* constraint here - you really would want to talk about it in terms of dropbacks that are not obvious passing downs *before you're in scoring range* (e.g. say, before the opponent's 30 or so). Once you start to get into scoring range and the field compresses, you're not really comparing apples-to-apples anymore. Pull the safeties closer to the line, and that vacated space isn't vacated anymore.

"While it is possible that play-action would become less effective if the whole league collectively ran less frequently on first-and-10, it remains a mystery why teams are reluctant to try this."

Well, I mean, the answer's basically in the question there. If they believe that it will become less effective, there's no incentive for them to try it. The only question, then, is whether or not they have more information than just the outcomes of games to inform their decision. Which, of course, they do. So it's not terribly easy to evaluate whether or not they're making a good choice.

"After working on this piece, one question that stands out is why teams seldom use play-action from shotgun formations, especially on first-and-10 where the evidence is strongest that it works."

Yeah, I dunno. Hermsmeyer's point is shown in the graphs there: play-action has more merit at larger target depths, and those are lower-percentage plays. So there's the "expected points versus yards" problem there: gaining 5 extra yards may not offset the increased risk of 2nd and 10.

Regarding shotgun versus under center - I don't see your point there: completion percentage from 10-20% is basically the same for play action from under center and from shotgun. Yards/play is basically the same between the two. Am I missing something? Where does under center vs. shotgun have an advantage? Or is it just that you'd expect the shotgun PA% to be the same as under center because their effectiveness is about the same?

In that case there's always the possibility that it's a talent problem (maybe only a few teams can effectively run play-action out of shotgun as easily as they do under center), but that would be a tough thing to figure out.

3Re: Situational Play-Action Passing in the NFL

by guga31bb // Mar 12, 2018 - 3:46pm

On your last question- right, if play-action is effective on 1st and 10 both from shotgun and from under center, why do teams run play-action so frequently from under center but not from shotgun?

5Re: Situational Play-Action Passing in the NFL

by Aaron Brooks Good Twin // Mar 12, 2018 - 3:51pm

It's harder to run play-action from shotgun.

There are usually fewer men between the tackles and via starting from farther back, shotgun gives LBs more time to recover from a missed first impression.

This can all be mitigated to some extent, via concepts like the diamond (a heavy-box shotgun), receiver or wingback options, pistol formations, or jump passes. But then, under center can use most of those concepts, too.

That it's harder to run it from shotgun is a big reason many people had problems with the Seahawks-Patriots play from the goal line. By lining up in shotgun, Wilson vastly reduced the odds the play would be a run or a PA.

6Re: Situational Play-Action Passing in the NFL

by Pat // Mar 12, 2018 - 4:50pm

"There are usually fewer men between the tackles and via starting from farther back, shotgun gives LBs more time to recover from a missed first impression."

Oh, that makes me think of another possibility I hadn't thought of, which is similar to what you're saying: starting from shotgun gives you *fewer options* to run play-action out of. As in, there are fewer ways to disguise the pass and have it still be effective.

Which would imply that you have to run those plays less frequently in order for it to be effective. If you have, say, 10 different ways to run play-action out of under center, and only 3 from shotgun, in the silly extreme case where running a play twice means it's completely ineffective, obviously you'll end up with significantly more play-action from under center even though the times when you *do* run it from shotgun, they're effective.

2Re: Situational Play-Action Passing in the NFL

by Raiderfan // Mar 12, 2018 - 3:32pm

TL;DR. It would have been nice if you would have started with a thesis paragraph that laid out your interpretation of what the data shows. I got lost about chart three.

4Re: Situational Play-Action Passing in the NFL

by Aaron Brooks Good Twin // Mar 12, 2018 - 3:47pm

"While it is possible that play-action would become less effective if the whole league collectively ran less frequently on first-and-10, it remains a mystery why teams are reluctant to try this."

Two thoughts:

1. Teams run for reasons not entirely captured by EYG.
This might be to give the QB or receivers a blow. This might be to let the offensive line be the hitter. This might be to keep the defense honest (converted into analytical terms, the expected efficiency of passing may decrease as the odds of a pass approach 1). It might be for all three.

2. While rule changes have made passing more effective than ever (potentially at the cost of running), this increase in passing efficiency has perversely decreased the penalty for poor rushing.
If you need three rushes for a 1st down or two completions, playing in an era of 50% completions means passing has variance problems. But in an era with 66% completions, it's golden. Basically, you can afford failed rushes when 3rd-10 isn't a disaster, but you cannot when it's an auto-punt. As passing has gotten easier, it's become easier to accept inefficient rushing.

7Re: Situational Play-Action Passing in the NFL

by zenbitz // Mar 12, 2018 - 5:53pm

I don't think you can consider more play-action shot gun passes if you don't consider the baseline shot gun running plays.

If anything you would expect a shotgun DRAW or Delay to gain the efficiency that a PA pass does.

8Re: Situational Play-Action Passing in the NFL

by Alex51 // Mar 12, 2018 - 8:41pm

"While it is possible that play-action would become less effective if the whole league collectively ran less frequently on first-and-10, it remains a mystery why teams are reluctant to try this."

How do we know they aren't trying? We don't know what plays they call in the huddle, so we don't actually know what percentage of the time they were planning to pass. We only see the percentage of plays that actually were passes. The actions of the defense aren't being controlled for here.

For instance, maybe they call a pass that they think would work really well if the defense were in man coverage, but would be a disaster against zone coverage. And when they see the defense line up, oops, looks like zone coverage. So, instead of risking an interception, they check to a run, gain a couple yards, and live to fight another down. That low risk/low reward alternative would be particularly appealing when they've got two more downs to play with (also a good reason to call that sort of feast or famine pass play on 1st down: if you don't see a favorable matchup, checking to a run is no big deal, but the upside if you see what you're hoping for can be huge).

Now, let's say they call a pass 90% of the time, but only get a favorable matchup/defensive alignment/etc a little over half the time. Then, you'd expect to see teams run about as much as they do, even if they really were trying to pass almost every time.

9Re: Situational Play-Action Passing in the NFL

by Pat // Mar 15, 2018 - 1:19pm

So, to explain this a bit better:

The reason why people argue that they aren't trying is because if you imagine football as a simple 2-person game, where each play is independent, and each person chooses "run" (R) or "pass" (P) - imagining one person playing 'offense' and the other playing 'defense' - and the result of the play differs depending on whether you get RR, RP, PR, or PP, then you would expect, when both players play optimally, to get the same average output per play regardless of P/R choice by a player.

This is the "minimax" solution, and it's a stable optimum - no one would want to try anything better/worse because it would never help.

Football *doesn't* look like they're playing minimax, because runs produce so much less than passes. So the argument is typically that teams should run less, and defenses should play pass more.

But, as you're probably guessing, the better argument is that football doesn't match the simplistic 2-player game model.

"Now, let's say they call a pass 90% of the time, but only get a favorable matchup/defensive alignment/etc a little over half the time."

Right. This would be one counter to why runs are so frequent in the NFL: because the players don't choose randomly, and the offense gets more information from the defense. Imagine a slightly altered game. There are 2 types of passes, A and B, and run play, C. AB/BA result in +5 yards. AA/BB result in 0 yards. CC results in 0 yards. CA/CB results in 2 yards, and AC/BC result in 10 yards. Both players choose plays, and present choices in front of them, but the offense can change from A or B to C after choices are made. Defense can't change.

If you play a game like this, the 'optimal' solution is obvious: defense randomly chooses A or B (and never C), and offense randomly chooses A or B, but switch to C if their opponent chooses the same. This results in 5 yards/pass, 2 yards/run, and runs occur 50% of the time, for a total average of 3.5 yards/play.

And there's no incentive for anyone to play differently. Defense has no incentive to change the 50% A/B rate (if opponent plays optimally, there's no gain), and no incentive to call C ever. Offense has no incentive to change their initial A/B rate (again, if opponent plays optimally, there's no gain) and no incentive to *not* choose C after seeing the opponent's choice.

The evidence that coaches aren't going for it on 4th down enough is *way* stronger than the idea that they're not playing optimally in run/pass.