The Effect of Scrambles on DVOA
Scrambles are pass plays, by definition. The quarterback scrambles for positive yardage when he's under pressure or when all his receivers are covered. Sometimes he runs just because a big hole opens up in the pass rush and it's going to be easy to gain yardage. But every scramble begins as a pass play. Scrambles are essentially the positive version of sacks.
Scrambles are also run plays, by a different definition. They count as rushing yardage in the official NFL stats. They are listed in the play-by-play as running plays. Here at Football Outsiders, we've always counted scrambles as running plays because they are runs in the standard play-by-play. This is left over from the way things were when I first started breaking down play-by-play way back in 2002. Back then, all quarterback runs were just runs in the play-by-play. Scrambles were not marked specifically as scrambles until 2006.
If we're looking at pass/run ratios, scrambles should count as passes, because they are called pass plays. (This is how we do it in the Strategic Tendencies tables in FO Almanac each year.) If we're looking at the decision to run or pass on certain down-and-distances, scrambles should count as passes. And when we're looking at how efficient each team is passing or running the ball, on offense or defense, we really should be looking at scrambles as passes.
It's not quite that easy, of course. Especially in this age of run-pass options, there are plays that are difficult to categorize as scrambles or "designed runs." If the quarterback makes a little pass fake on his bootleg and then pulls the ball down to run, is that a scramble? Maybe it was run all the way and the fake was just part of trying to throw the defense off its game. Does it matter how the offensive line is blocking? Is a pass a scramble if the wide receivers are all running routes, or can those be designed runs too? FO gets data from ESPN Stats & Info and Sports Info Solutions as well as the official NFL play-by-play, and there are a 30-40 plays each year where the companies will disagree on whether a run is a scramble or not.
A brief digression to show some examples:
On the play above, the NFL marked scramble, but Sports Info Solutions marked QB draw, and I think Sports Info Solutions has it right. Still, you can see how it is difficult to categorize. I think this is a run because Dak Prescott goes as soon as he hits his back foot, and Jason Witten is blocking downfield the entire time.
Is this play a scramble on an RPO, or strictly a zone read that was always a running play? The NFL marked this as a scramble. The offensive line is run-blocking, but Parris Campbell (15) is set up for a "smoke route" and then starts running downfield on a scramble-drill route.
The NFL marked this play as a run, while SIS has it as a scramble. Ryan Tannehill has pass options but runs it in when there's a big hole in front of him on the bootleg. Is that a run play or a pass play?
Despite a few plays where it's difficult to determine pass or run, it's probably a good idea to count scrambles as pass plays. Among other issues, counting scrambles as running plays sometimes gives us a skewed view of how good a team's running game is. To give just one example, the Miami Dolphins quarterbacks averaged 7.48 yards per carry this year on 40 scrambles. That ranked ninth in the NFL. On all other runs (including quarterback carries not coded as scrambles), Miami players averaged 2.94 yards per carry. That was the worst figure in the league by more than 0.4 yards.
I wanted to take a look at the effect of scrambles on each team's running numbers and what would happen if we counted these plays as passes instead of runs. So I went and re-ran this year's DVOA numbers, only this time I marked all scrambles as pass plays. This is based on the official NFL play-by-play, rather than ESPN or SIS data. Of course, the number of affected plays differed significantly for each team, ranging from just three plays (New England) to 52 scrambles (Jacksonville). The numbers are closer together on defense, but range from Miami facing eight scrambles to three different teams that faced 33.
Despite a new set of opponent adjustments, since passes and runs are adjusted separately, this new version of DVOA does not make a lot of difference in each team's total offensive or defensive rating. For over half the units in the league, offense or defense, DVOA changed by less than 0.15%. The biggest changes on offense were Denver (-0.4% less) and the Giants (0.4% more). The biggest changes on defense were Denver again (0.5% worse) and San Francisco (-0.5% better).
However, when we look at run and pass DVOA separately, some of the differences from counting scrambles as pass plays are fairly significant. Ten different teams see their rank in run offense DVOA change by at least four spots when we take out scrambles. Here's a look at old run/pass and new run/pass DVOA as well as how many scrambles each team had and the DVOA on those plays. The initial sorting order is based on the change in each team's run DVOA from the old method to the new one.
Two things probably jump out at you right away. First, oh my god the Miami Dolphins. Hey, I told you that their standard running game was really bad this year, with less than 3.0 yards per carry. Change scrambles from runs to passes, and Miami's rank in pass DVOA moves up three spots. In run DVOA, the Dolphins drop from last place to even more last place. Sure, there's no ranking difference, but that drop of 12.5% DVOA is colossal. It's 70% larger than the change in run DVOA for any other team.
The second thing you notice immediately is just how good scrambles are, and the big effect of taking them out of run DVOA. This makes a lot of sense if you think about it. By definition, a scramble has to be positive yardage; even a scramble that gains zero yards is scored as a sack. But scrambles generally don't gain just one or two yards. The average for scrambles in 2019 was 7.1 yards per carry. That means we get a very high average DVOA as well. The league DVOA on all scrambles was 63.6%. Only three teams had negative DVOA on scrambles: Minnesota, New Orleans, and the Los Angeles Rams.
That also means that for 29 out of 32 teams, their new run offense DVOA (without scrambles) is lower than their original run offense DVOA. For the league as a whole, run offense DVOA drops -3.5%. I've often given the leaguewide pass and run DVOA numbers to demonstrate just how much passes tend to be more efficient than runs. It turns out that those numbers understated the efficiency advantage of calling a pass play because a whole bunch of positive called pass plays were being counted as runs. With scrambles counted as passes, the leaguewide pass DVOA goes from 10.3% to 12.3%. The leaguewide run DVOA goes from -3.5% to -7.0%. The average yards per carry (with kneeldowns removed) goes from 4.50 to 4.34.
Miami isn't the only team that gets an artificial boost to its rushing numbers due to scrambles. Seattle has the biggest change in rank, dropping from sixth in run DVOA to 13th when scrambles are removed. That's not good for a team that's so run-centric in its playcalling. Buffalo, Houston and Kansas City also drop at least four spots. And here's a surprise: Cleveland. It's a bit shocking to see Baker Mayfield with the highest DVOA on scrambles. Mayfield had 17 carries for 134 yards and 10 conversions, including converting 7 of his 10 scrambles on third down.
On the other end, the three teams with negative DVOA scrambling -- Minnesota, New Orleans, and the Los Angeles Rams -- all rank at least five spots better when we remove scrambles from running plays. The DVOA drops slightly for some offenses such as New England and San Francisco but their relative rank in run offense DVOA improves. (Tom Brady had one of the best DVOA ratings in the league when scrambling, but that's a whopping total of three plays. The big one was a 17-yard gain on fourth-and-6 from the Kansas City 29, losing by a touchdown in the fourth quarter.)
Let's run similar numbers on the defensive side of the ball:
The Seattle running game looked worse when we removed scrambles, but the Seattle run defense looks better because the Seahawks defense was one of the league's worst against scrambles. The Jets and Bucs run defenses look even more stellar when we don't count scrambles as runs. On the other side, the Rams and Packers are the only teams with negative DVOA allowed on scrambles, so their run defenses look worse when scrambles are removed. The Rams' run defense DVOA barely moves, but because of the run DVOA dropping around the league, their rank in run defense goes from eighth to 15th without scrambles.
Where do we go from here? If we want, we can probably switch over to start counting scrambles as pass plays in team DVOA starting in 2020. However, there admittedly a couple of issues if we want to always treat scrambles as pass plays.
First, there's a lot of hard work to re-run all 35 seasons of DVOA to change scrambles from runs to passes. And what do we do about the seasons before 2006 when scrambles were not marked in the play-by-play? It helps that option plays were so much rarer in the NFL 20 years ago. One possibility is to simply mark all quarterback runs as scrambles except for probable quarterback sneaks (third-and-1, fourth-and-1) and runs for a loss (which can't be scrambles, since a scramble for negative yardage is technically a sack). This would be an imperfect solution but might be closer than counting scrambles as runs for measuring the "real value" of passing and rushing for each team.
It's also a lot easier to change scrambles in team DVOA than in individual DVOA for quarterbacks. For team DVOA, all plays use the same baseline -- that's why the leaguewide pass DVOA is higher than the leaguewide run DVOA. For individual DVOA, passes and runs have different baselines, and runs have different baselines based on player position. So changing our individual stats would involve a lot of work figuring new baselines, whether that would be turning scrambles into pass plays or just combining all quarterback plays into one number no matter whether it's a run or pass.
All of these issues are under consideration, but it's clear that how we treat scrambles does change how we measure pass and run games separately.