Further Research on Play-Action Passing
Guest column by Ben Baldwin
The pair of pieces I wrote for Football Outsiders last offseason have generated a lot of discussion and follow-up questions. With the Rams' and Chiefs' offenses becoming unstoppable juggernauts in part because they use play-action at high rates, interest in what drives play-action success won't go away. Let's tackle a few questions here. I'll be using the same data that I introduced in the previous pieces, so check those out first if you haven't already.
1. We've seen that there's no clear relationship between a team's rush attempts and play-action efficiency leaguewide. But what if there's a real effect that's obscured by variation across teams' baseline passing effectiveness?
To this point, we have focused on looked at yards per play only on play-action dropbacks. But perhaps rushing increases a team's play-action benefit relative to what it otherwise would have been on other passing plays.
To test this, let's take the difference between a team's yards per dropback when using play-action and when not using play-action. If we think of non-play-action as a team's baseline passing effectiveness, then we can see whether rushing frequency or effectiveness is related to the increase in effectiveness when using play-action. Here's what the relationship looks like from 2011 to 2017:
As we saw when looking at play-action yards per play, there's no relationship here. The difference in the effectiveness of a team's passing game using play-action relative to without play-action is unrelated to the frequency or effectiveness of the team's rushing.
The most interesting part of these figures is the vertical placement of the dots. I've drawn a horizontal line at zero on each figure, and the vast majority of points lying above the line means that the vast majority of teams have more yards per play on play-action dropbacks than on non-play-action dropbacks. This isn't news -- Football Outsiders has documented play-action's efficiency for years -- but gives a visual for how rare it is for teams to fall below the zero line. And once again, there's no relationship between a team's rushing frequency or effectiveness and their likelihood of falling below the line.
Verdict: Unsupported.
2. We've learned that a team's effectiveness on play-action is not related to its rushing frequency or success. But coaches appear to believe that it does. Do teams use play-action at a higher rate when running more often?
The motivation behind this question stems from the first two weeks of the 2018 NFL season. During these two weeks, the Seattle Seahawks barely ran the ball (38 rush attempts to 81 combined pass attempts and sacks), and perhaps as a consequence, only used play-action on 12 of 81 dropbacks (15 percent). In subsequent weeks, the Seahawks would become one of the most run-heavy teams in the league. At the same time, they started using play-action frequently. Now at 31 percent, Seattle has risen to No. 3 in the league in play-action rate through Week 13. Offensive coordinator Brian Schottenheimer's comments later in the season implied that the Seahawks leaned on play-action more in later weeks because of their run game:
It helps when you're running the ball as well as we've run it the last couple of weeks. The play-action game is going to be there [...] It looks different these last two weeks than it has obviously the first two weeks when we weren't running it as well as we are now.
Oh, and if you're wondering whether play-action was still effective during those first two weeks when the Seahawks went with a pass-heavy approach, it was: Russell Wilson completed 80 percent of his passes at an absurd 18.5 yards per attempt.
Let's see whether coaches' beliefs about play-action drive how often they use it. The graphs below show the relationship between a team's rushing frequency and the percent of dropbacks that use play-action over the course of a season:
At last, we have finally found an avenue through which rushing appears to affect play-action passing: through coaches' beliefs about how often they can use play-action. (The numbers look fairly similar when looking at neutral game script situations.) While every piece of evidence we have points to those beliefs being misguided -- teams that don't run at a high rate could also use play-action frequently -- they translate into differences in play-action utilization.
Verdict: appears to be true
3. Rushing doesn't appear to set up play-action. But what about the reverse: could using play-action frequently set up a team's run game?
Frequent use of play-action could sow uncertainty in defenders' minds and lead them to be less aggressive in their run fits if they are concerned with defending the pass. For a supporting anecdote, check out Reuben Foster backpedaling here while Todd Gurley is in the midst of receiving a handoff:
.@RamsNFL offense takes all of the aggression out of defensive players. Watch #ReubenFoster be put into a deep freeze. #BaldysBreakdowns pic.twitter.com/JZxcPbynkS
— Brian Baldinger (@BaldyNFL) October 23, 2018
With the Rams using play-action on 37 percent of dropbacks through Week 7 -- the most in the league by far -- perhaps Foster's indecision is no coincidence.
Let's take a look at how often teams use play-action (as a percent of dropbacks) versus how successful they are at rushing:
Here we indeed see a positive correlation, but the direction of causation is difficult to get at. It could be the case that a heavy dose of play-action forces defenses to divert resources away from defending the run. But it could also be the case that having a higher rushing success rate leads teams to use play-action more frequently due to coaches' beliefs about what makes play-action successful.
Verdict: Worth further investigation.
4. Does using play-action too frequently diminish its effectiveness?
Already we should be skeptical here given the success of the Rams this year, but let's take a look anyway.
Whether looking at play-action yards per play (left) or the difference between play-action and non-play-action (right), there's not much of a relationship. If anything, teams that use play-action more often are slightly more effective at it, which should perhaps not come as much of a surprise. (If you're wondering about the outlier in the upper right of these figures, that's the 2012 Redskins, discussed more here.
Looking at the 2018 season specifically, four of the top five teams in play-action frequency through Week 13 -- the Rams, Seahawks, Patriots, and Chiefs -- are also in the top five in play-action yards per play. The outlier is the Eagles, who use play-action at the second-highest rate in the league but are only 16th in play-action yards per play. If there is a point where play-action has diminishing returns, we don't appear to have reached it yet.
Verdict: Unsupported.
Conclusion
We have an ever-growing body of evidence that teams don't need to run often -- or run well -- to set up play-action. Play-action works for teams that run frequently, infrequently, well, or poorly. For the vast majority of teams, it just works. From 2011 to 2017, 196 of 224 team-seasons had higher yards per play on play-action dropbacks than on non-play-action dropbacks. This includes teams like the 2017 Lions (9.4 yards per play-action play, No. 30 in rushing DVOA) and 2015 Jaguars (1.7 more yards per play on play-action dropbacks despite being No. 28 in rushing DVOA and only running 31 percent of the time).
For every team observed to have a strong play-action game and strong rushing attack, I can find an example of an effective play-action team that has a weak rushing attack. For every play where a successful play-action pass followed a series of runs, I can find a play where play-action succeeded despite not recently running. There just doesn't seem to be anything there.
Finally, there's no evidence of teams using play-action a lot seeing any less benefit to play-action. Coaches treat play-action as a fragile toy that can only be brought out under certain conditions -- only after running, and only if it hasn't already been used it too much -- but it's more like the tennis racket I've had since high school: always ready, and always effective.
An economist by trade, Ben Baldwin uses large datasets to try to learn about human behavior. He can be found covering the Seahawks for The Athletic Seattle and on Twitter at @benbbaldwin.
Comments
15 comments, Last at 25 Jun 2019, 3:02pm
#1 by Aaron Brooks G… // Dec 13, 2018 - 4:40pm
It's interesting that teams that rush more have a weakly better PA success on a per-play basis, but run PA more often.
The net effect is that they get a bump in passing on both a quantity and quality basis.
I'm curious about total rushes to total passes or total PA passes, though. Is the net combo less advantageous than just replacing (rushes+PA passes) with an equal number of non-PA passes?
#2 by guga31bb // Dec 13, 2018 - 7:03pm
That's a great question and I'll put that on the list for the inevitable follow-up piece (this is the author of the piece, thanks for reading). To make sure I understand correctly, the question is whether throwing on a higher percentage of "shown handoffs" (rush + PA pass) reduces the effectiveness of PA.
Another question I want to get to at some point that I'll write here so I remember is whether PA is less effective for teams that are trailing when rushing is less of a threat.
#5 by ammek // Dec 14, 2018 - 6:12am
Five of those are the bottom five offenses by DVOA, whose bad pass figure tends to be exascerbated by throwing interceptions. Four of the five are in the bottom ten in rushing DVOA too: it's highly unlikely that they'd win more games by running more often.
The only teams whose rush offense is good enough to warrant more runs are Denver, Dallas and perhaps Carolina.
#6 by Sifter // Dec 14, 2018 - 7:49am
It seems to me that defending play action is largely an instinctual exercise and the success of PA is based on that 'hack' - playing on defenders instincts. Defenders are taught from a young age to read their keys and react accordingly. Hard to turn off that instinct after 10-20 years of training just because a team is a little worse at running the ball. It reminds me of defending poor shooting teams in high school basketball. It took a lot of discipline to not step out and challenge a long range shot, even though our coach had told us 'let them shoot, they won't make many'. It's the same in the NFL. Defenders are gonna bite on a PA fake, even if that PA fake is from a team with a toothless running game.
I'd be interested to see the numbers filtered for only the absolute worst running teams in DVOA history. The teams where a DC might just say, 'let them run boys, they won't make many yards'. eg. This year's least effecitve running team is Arizona. And yet I don't think there's a DC out there who plays Arizona by clearing their LBs out of the box, sitting them on the first down line and playing the pass 100% of the time, since they can still run with at least some effectiveness.
#9 by Will Allen // Dec 14, 2018 - 2:26pm
Back in 2008, when the Williams Wall was.at it's peak (Pat was injured at the tail end of the season, and never was quite as good again), the Vikings would dabble with a 1 linebacker set as their base, so confident were.they that Pat and Kevin would dominate the interior, Jared Allen would play the run honestly, and dbs like Antoine Winfield were such sound tacklers. It was quite interesting, and if not for age, I think they might have expanded on it in 2009, when they had an offense which could reliably give them a lead to work with.
#10 by mrt1212 // Dec 14, 2018 - 4:55pm
As an anecdote, look at the Houston Hawks game from 2017. Seattle was ailing with running both seasonally and within that game itself but its PA passing was stronk for 2 reasons imho:
The instincts of the defense themself to treat PA at face value (despite evidence the Hawks couldnt run well) and the trash DBs who couldnt defend what their front 4 or 5 were ceding in time to find targets. There was just enough hesitation, confusion and ambiguity and really solid RO blocking to put all the fulcrum on the passes and catches that Houstons DBs could handle.
#14 by LionInAZ // Dec 15, 2018 - 6:38pm
This is a more sophisticated rendition of what I was thinking. Play action forces defenders into a decision tree that they don't always have time to process, so they make the decision based on instinct and training.
It would be very interesting to see if some defenders/defenses are better against play action.
#11 by robbbbbb // Dec 14, 2018 - 7:44pm
I may be grasping at straws, here, and this is a tough one to test. But perhaps the routes run off of play-action are just higher-value receiver routes than vanilla passing routes?
Maybe the route combos that offensive coordinators run off of play action would be just as effective if they were run as vanilla plays?
I don't know how to test that one, though. It could be a mess to evaluate it.
#12 by gomer_rs // Dec 14, 2018 - 11:16pm
This is not an unreasonable question. You could rephrase - are "shot" passes, passes designed to gain 20+ yards, sufficiently efficient that the benefit of PA derives from the increased rate of "shot" passes.
There could be support in this theory if you study the development of Art Briles' veer and shoot offense which reached its height at Baylor. A foundational concept in the early development of that offense is that "shot" passes were massively under used.
_______
I remember when they were the Sea-chickens.
#15 by JaredJen // Jun 25, 2019 - 3:02pm
Couldn't OC's call play-action passes when they see something from the defense that indicates it will have success? Running plays, perhaps even successful running plays, cause whatever tells the OC is looking for more often than passing plays.
For a super simplistic example: An OC wants to call a PA pass when he thinks the opposing SS will collapse hard on the TE. Running the ball at the SS a few times might do the trick, but so could a few shallow TE routes.
And then, for whatever reason, running plays do bring about the situation an OC is looking for more often than passing plays. This would explain why PA passes are so similarly successful following strings of runs or passes, but also explains why OC's seem to use the play type sparingly.
To add onto that, it could be that the Rams and Chiefs, through a clever scheme and personnel, force the defense into positions that make PA passes successful. It's not simply that Mcvay has decided to spam PA and ride that to the Super Bowl, but instead that the rest of the offense is designed to force the defense into positions where they're susceptible to PA passes.