A look at fourth-down decision making in 2014 highlights Sean Payton, Marc Trestman, and... wait, this can't be right... Jim Caldwell? Plus: Chip Kelly may be less aggressive than you think he is.
18 Aug 2014
by Scott Kacsmar
Last week we looked at play-action passing on the offensive side of the ball in 2013. Now we'll look at the 2013 defenses and review past data to see how much this strategy impacts the game.
We discovered 37.5 percent of offenses since 2008 fared better in DVOA without play-action passing. Similarly, 33.9 percent of defenses have defended play action better than they have regular dropbacks. So it's not the ultimate weapon that helps every offense and hurts every defense.
How much of an impact does play action have? We observed in 2013 that the average DVOA increase with play action was 11.1 percentage points. That's right in the ballpark of previous years. Can we comfortably credit this increase in offensive efficiency to the play-fake itself? Defenses instinctively tend to bite on the fake to play the run, and we see this even against opponents who aren't running the ball well. Instincts are hard to shake, and if the play-fake looks really convincing, the offense can set up some favorable passing windows. Even though defenses can recover quickly after realizing it's a pass, any hesitation can be deadly to a defense.
However, FO commenter Perfundle made a good point about the usage of play action by down. Offenses are only using it about five percent of the time on third down. Generally, it's harder to convert on third-down passes since they're usually obvious passing situations and pressure is applied. What if we looked at 2013's play-action success by down?
|2013 Regular Season: Play-Action Splits by Down|
|PA||No PA||Diff.||PA||No PA||Diff.|
Notes: Most numbers are over 0.0% because team efficiency DVOA includes both runs and passes, and for offense, some penalties. Since passes are generally more efficient than runs, passing DVOA is usually over 0% and rushing DVOA under 0%. And offensive DVOA for both passing and rushing is higher than defensive DVOA because total offensive DVOA includes false starts and delay of game, which are always negative plays. Also remember, a negative DVOA on defense is better than a positive DVOA.
As you may have expected, a play-action pass on first down has the biggest impact on offense and defense -- by a wide margin too. Less expected is the fact that offenses and defenses performed virtually the same on first and second down when there was no play action. Defenses are even better on third downs, but that down's rare use of play action has been proven to give a slight increase to the offenses. Conventional wisdom may suggest the element of surprise for play action is greater on a later down (especially in short-yardage situations), but for pure effectiveness, there's no better time to dial up a play-action pass than on first down. Otherwise, the impact is not that significant.
Honestly, why should play action be overly significant? Teams are only using it on 20 percent of their dropbacks, and most of that damage is coming on first down. Once you get past the fake, the play is essentially just another pass. So we would expect to see defenses that defend regular passes well fare similarly against play-action passes, but do they?
Since 2008, the correlation between a defense's DVOA on passes/scrambles with play action and DVOA on passes/scrambles without play action is 0.38 -- also 0.38 for only the 2013 season. That's not as strong as expected, but we know by now that a lot of football data doesn't have strong correlations. Some seasons the receiver hangs onto that 50-yard bomb, and some years he drops it. With 22 moving parts plus officials, there's always a lot of cause for variation.
There's still obviously some correlation there. There's also something we have neglected to this point. Even after ignoring the fake, the average play-action pass is different from your typical pass, because offenses are trying to go down the field. Not many offenses call a play-action pass with the primary goal being a three-yard route. Okay, maybe Brian Schottenheimer would do that, but that's not the norm. Offenses want the big play, and that's reflected well in the difference in the average depth of throw for the ten offenses that used play action the most in 2013:
|2013 Offense: Average Depth of Pass|
|Offense||PA Pct.||PA - Avg. Depth||No PA - Avg. Depth||Diff.|
Carolina was ass-backwards, but everyone else featured went deeper on play action with an average increase of 2.1 yards per throw. Hitting longer passes for big gains is a good way to boost DVOA. However, I'm not sure there are any special qualities a defense would need to be good against play action compared to regular dropbacks. It's still about getting pressure on the quarterback and covering receivers downfield. Maybe defenses with great linebacker corps are hurt the most if those guys keep biting on the fake. That's just one theory.
With that in mind, let's look at the defenses against play action in the 2013 regular season. Teams are sorted by the descending difference in DVOA with and without play action. The first two splits show the results for using play action with scrambles included and excluded. The third section is for dropbacks without play action.
|Defense||PA Pct||Rk||with PA (Pass/Scram)||with PA (Passes only)||No PA||Difference|
|Defense||PA Pct||Rk||with PA (Pass/Scram)||with PA (Passes only)||No PA||Difference|
On Friday, I asked if anyone wanted to bet on Seattle ranking first, and sure enough that defense was easily No. 1 against play action. Maybe just as impressive is the fact that the Seahawks still ranked second without play action, and with nearly the same DVOA. That league-leading pressure rate combined with the secondary's coverage rarely ever cracked. Somehow one of the best halves all year any player had against Seattle was Tampa Bay rookie Mike Glennon. Of course we only say a half, because Seattle adjusted and came back to win the game.
Similar to our results with defensive pressure, the Panthers and Rams provide examples of what a defense can do with a great front seven and a suspect secondary. However, we may not be giving the 2013 Panthers' secondary enough credit, because they still ranked second against play action and fourth without it. On the other hand, the Rams ranked 24th against play action and 13th without it. Perhaps this is the difference in experience. The Panthers were paced by veterans like Captain Munnerlyn, Mike Mitchell, Quintin Mikell and Drayton Florence. We'll see how things go with those players gone this year. The Rams mostly relied on players 25 and under like Rodney McLeod, Janoris Jenkins, Trumaine Johnson and T.J. McDonald. We'll see how they fare after another year of experience and with new defensive coordinator Gregg Williams.
No defense since 2008 faced a higher rate of play-action plays than the 2013 Bears at 30.8 percent. The previous high was 26.4 percent (2012 Bengals). The Bears certainly struggled too, ranking 22nd in DVOA against play action, but that's bound to happen when the middle of your defense is so soft. Henry Melton , Lance Briggs and D.J. Williams were lost to injury and the safeties really struggled. On the other end, the Chiefs faced the lowest rate of play-action plays (13.4 percent), which makes sense given how well the pass rush (namely Tamba Hali and Justin Houston) played before injuries.
Led by J.J. Watt, Houston had the second-best pressure rate in 2013, but it didn't matter when the secondary played the way it did in coverage. The Texans bring up the rear in DVOA against play action. Does Jadeveon Clowney really make D.J. Swearinger and Kareem Jackson better players? Watt didn't.
For the second year in a row the Ravens had one of the largest decreases in efficiency when play action was used. They ranked 32nd in 2012, but only moved up to 30th last year despite a ton of changes in personnel. Baltimore was strong without play action, ranking fifth, but slumped to 26th against play action. This is an odd look for the Ravens, given they ranked first, fourth, sixth and third against play action in 2008-2011. One thing we know that was lacking the last two years was a healthy, high-level version of Ed Reed. Safety Matt Elam could certainly get better in his second year, and we like C.J. Mosley as a smart, instinctive linebacker in the middle. The Ravens should break this play-action trend this year.
The defenses we really need to focus on for play action this year are Buffalo and Cleveland, making this The Mike Pettine Story. As Buffalo's defensive coordinator last year, Pettine coached yet another below-average Buffalo run defense (19th in DVOA), but the Rex Ryan pupil totally revamped the pass defense, which only trailed Seattle. Pettine brought more pressure than Ryan usually does, but the Bills were the only team to have negative DVOA without pressure, an incredible feat. That coverage paid off, because Buffalo had the best DVOA without play action since 2008. That gives them the biggest difference in DVOA relative to play action, but ranking 19th against it isn't so bad when you're first against the piece of the pie that's 79 percent of all dropbacks.
Of course the Bills just scream regression with Pettine in Cleveland, Jairus Byrd in New Orleans and Kiko Alonso on injured reserve. New Buffalo defensive coordinator Jim Schwartz is expected to use the wide-nine defense. He's had a wide range of success and failure against play action in Detroit, ranking 32nd in his first year (2009) and as high as fifth (2010).
Now Pettine has to see what magic he can pull in Cleveland, which had a totally opposite performance from Buffalo last year. The Browns were a miserable 30th when teams didn't use play action, but shot up to third against play action. That's the biggest improvement of any defense in the league. You might be wondering how this happened when the Browns allowed 5.8 yards per play with and without play action. One answer is touchdown-to-interception ratio. Using play action against the Browns produced six touchdowns to four interceptions, and the Browns even dropped three more picks. Without play action, passers had 23 touchdowns to 10 interceptions. Cleveland also did well against longer throws (passes thrown 21-plus yards) off of play action, allowing three completions on 23 attempts.
So while we anticipate different results in Buffalo and Cleveland in 2014, is there any reason not to expect different results for most teams? What's the consistency on these play-action numbers?
They're not good of course, especially since we're talking about defense. Since 2008, the year-to-year correlation on DVOA with play action is 0.07. Without play action, it's 0.20. For offenses, the numbers are 0.16 with and 0.55 without. In the same time period, defensive points per drive only has a correlation coefficient of 0.24, so what else would you expect? It's hard to sustain defensive success.
Just consider the case of aforementioned Buffalo. In 2008, the Bills had the fourth-worst DVOA against play action (70.2%) in the last six seasons. That's 189th out of 192 teams. A year later, the 2009 Bills had the best DVOA against play action (-49.2%) since 2008. From 189th to first with the same coaches, same scheme, a harder schedule and the only major player addition was a rookie safety (Byrd)? Yeah, whatever. We told you the 2013 Bills had the best DVOA without play action since 2008, but no one should be surprised to see a huge slide in 2014.
If anything, it's expected.
4 comments, Last at 23 Sep 2014, 2:22pm by