DVOA Analysis
Football Outsiders' revolutionary metrics that break down every single play of the NFL season

The Effect of Scrambles on DVOA Revisited

Deshaun Watson
Photo: USA Today Sports Images

Regular Football Outsiders readers know that 2020 introduced a new version of our DVOA metric. One of the main changes of this new version was to treat quarterback scrambles as passes rather than runs. I wrote a long article about this last offseason. Now that we've counted scrambles as passes for a year, I wanted to revisit the topic and look at how that changed the way we measured running games and passing games in 2020.

A reminder of what I wrote last year about scrambles:

Scrambles are pass plays, by definition. The quarterback scrambles for positive yardage when he's under pressure or when all his receivers are covered. Sometimes he runs just because a big hole opens up in the pass rush and it's going to be easy to gain yardage. But every scramble begins as a pass play. Scrambles are essentially the positive version of sacks... When we're looking at how efficient each team is passing or running the ball, on offense or defense, we really should be looking at scrambles as passes.

Last year, to take a look at the effect of scrambles on each team's running numbers, I had to re-run DVOA marking all scrambles as pass plays. This year, I had to do the opposite: I re-ran DVOA but changed all scrambles back to run plays, and then compared that to actual DVOA using the new formula. Once again, the definition of which plays count as scrambles is based on the official NFL play-by-play, rather than ESPN or SIS data. It can be difficult to discern what is and isn't a scramble, especially in this age of run-pass options.

Houston had the most scrambles in 2020, with 62 of them, and the largest effect from having scrambles count as passes instead of runs in DVOA. Houston gained just 4.04 yards per carry on running back carries in 2020, compared to 6.37 yards per carry when Deshaun Watson was scrambling. Combine that with a huge efficiency difference, and the gap between Houston's run DVOA with and without scrambles is over 10 percentage points. Without scrambles, Houston was the worst running game in the league (-28.0%). If we counted scrambles as runs, the Texans would have ranked 27th instead (-17.8%). Houston is one of seven teams whose rank in run offense changes by four or more spots when we go back to treating scrambles as runs instead of passes.

The Texans' passing DVOA barely moves, however, because the efficiency on scrambles (30.7%) is almost the same as the team's efficiency on other passes (23.5%).

On the other side of the coin is Tampa Bay. Buccaneers quarterbacks only scrambled six times during the regular season (four by Tom Brady, two by Blaine Gabbert) and none of those runs converted for a new set of downs although Gabbert did have a 16-yarder on second-and-20 against Detroit in Week 16. Tampa Bay is the only offense whose run DVOA would have been worse if we were still including scrambles as runs instead of passes, and their rank in run offense would drop from 10th to 16th. That's because scrambles are on the whole very efficient plays. It helps that there's no such thing as a scramble that loses yards, since those are all sacks. Overall last year, quarterbacks gained 7.1 yards per carry when scrambling after a pass broke down. The league ended up with 65.8% DVOA on scrambles.

Other prolific scramblers besides Watson included Russell Wilson, Kyler Murray, and Lamar Jackson, who each ended up with 50 or more scrambles. The league's best scrambler of 2020 depends on which version of DVOA you want to use:

  • If you want to use team DVOA, counting scrambles as passes and comparing them to a baseline of all plays, the top scrambler was Buffalo's Josh Allen. Although he scrambled a bit less than he did in 2019 (going from 43 to 29 scrambles), the Bills had 110.8% DVOA when Allen scrambled. Allen averaged 7.9 yards per carry on scrambles with two scores and 10 other first downs.
  • If you want to use individual stats, then we count scrambles as runs and compare them to a baseline of all quarterback runs. With a minimum of 10 scrambles, Taysom Hill had the highest DVOA on scrambles (59.1%) followed by Cam Newton (58.3%). If we want to look at total value instead, Russell Wilson led the NFL with 141 DYAR on scrambles followed by Kyler Murray (120), a surprising Teddy Bridgewater (94) and then Allen (79).

Here are all the team numbers by the old DVOA method and the current DVOA method. Red numbers have scrambles as runs, while black numbers have scrambles as passes. The initial sorting order is based on the change in each team's run DVOA from the old method to the current one.

HOU 23.5% 8 -17.8% 27 24.1% 8 -28.0% 32 -10.2% 62 6.37 30.7%
SEA 26.2% 7 7.0% 5 30.4% 6 -1.4% 9 -8.3% 54 8.52 85.5%
PHI -18.6% 31 -2.2% 13 -12.1% 29 -10.2% 18 -8.0% 49 7.45 79.9%
BUF 40.3% 3 -7.5% 20 43.3% 3 -15.1% 22 -7.5% 29 7.93 110.8%
ARI 10.4% 19 -2.5% 14 15.0% 14 -9.7% 17 -7.2% 50 8.42 78.3%
CAR 8.1% 22 -1.7% 12 12.3% 20 -8.8% 16 -7.0% 38 7.18 85.9%
KC 47.3% 2 1.0% 9 49.0% 2 -5.7% 13 -6.7% 40 7.30 84.0%
NYJ -17.3% 30 -16.6% 26 -12.5% 30 -22.6% 27 -6.0% 31 7.45 74.4%
LV 18.8% 9 -12.6% 23 21.3% 9 -17.7% 24 -5.1% 28 5.39 79.4%
GB 50.1% 1 7.7% 4 52.0% 1 2.9% 5 -4.8% 24 6.21 91.3%
CHI -0.5% 23 -14.0% 25 1.6% 23 -18.3% 25 -4.4% 20 8.85 84.7%
DEN -15.8% 29 -18.0% 28 -12.9% 31 -22.2% 26 -4.3% 25 6.00 53.2%
MIA 10.6% 18 -13.4% 24 12.8% 18 -17.5% 23 -4.1% 34 6.65 63.1%
DET 13.4% 15 -9.8% 22 14.8% 15 -13.8% 21 -4.1% 19 7.95 71.9%
LAC 27.0% 6 -22.6% 29 27.5% 7 -26.7% 31 -4.1% 27 5.41 54.7%
CLE 18.5% 10 3.2% 7 20.9% 10 -0.8% 7 -3.9% 25 6.08 71.5%
JAX -11.0% 26 -7.4% 19 -8.1% 26 -11.1% 19 -3.7% 29 6.34 47.0%
BAL 9.4% 20 9.6% 2 13.9% 17 6.0% 3 -3.6% 50 7.28 52.5%
NYG -14.8% 28 -0.3% 11 -11.5% 28 -3.7% 11 -3.4% 26 7.19 73.6%
TEN 37.4% 5 9.5% 3 39.5% 4 6.1% 2 -3.4% 20 7.40 103.9%
WAS -30.0% 32 -2.6% 15 -26.7% 32 -6.0% 14 -3.4% 17 6.88 91.6%
ATL 15.0% 12 -22.8% 30 15.8% 13 -25.9% 29 -3.2% 16 6.50 62.8%
DAL -2.9% 24 -9.4% 21 -0.9% 24 -12.6% 20 -3.1% 26 7.12 60.2%
MIN 17.5% 11 3.0% 8 18.8% 11 0.7% 6 -2.3% 17 8.12 71.4%
NE -14.5% 27 0.7% 10 -10.6% 27 -1.2% 8 -1.9% 23 8.70 80.6%
NO 15.0% 13 11.1% 1 16.6% 12 9.5% 1 -1.6% 20 8.25 72.3%
LAR 12.5% 16 3.9% 6 12.4% 19 3.5% 4 -0.4% 24 5.08 9.4%
SF 8.8% 21 -7.1% 18 8.9% 22 -7.4% 15 -0.3% 7 5.43 36.9%
IND 14.6% 14 -5.2% 17 14.3% 16 -5.4% 12 -0.2% 2 2.50 -18.4%
PIT 11.5% 17 -26.2% 32 11.3% 21 -26.3% 30 -0.1% 4 5.50 21.6%
CIN -4.3% 25 -24.3% 31 -4.9% 25 -24.3% 28 0.0% 23 5.22 -26.8%
TB 38.2% 4 -3.0% 16 37.1% 5 -2.0% 10 1.0% 6 4.33 -46.9%

Baltimore stands out here because the Ravens had a lot of scrambles with a good DVOA rating, but you'll notice it doesn't have that much effect on their run offense DVOA. However, removing scrambles from pass plays would drop their pass offense rating from 17th to 20th. That's because the Ravens have such a strong run/pass ratio compared to most NFL teams. With many more runs and fewer passes, the effect of moving those 50 plays from runs to passes is stronger on the pass rating than on the run rating.

Once again as in 2019, only three teams had a negative DVOA when scrambling. It's three different teams than last year: Indianapolis, Cincinnati, and Tampa Bay. The Bengals are a bit of a surprise because there were a good number of scrambles there, not just single digits like Indianapolis and Tampa. The big problem for the Bengals was scrambles on third downs. Only two of Cincinnati's two scrambles on third down were able to convert to move the sticks, although one of those was a 19-yarder on third-and-18 by Ryan Finley against Washington in Week 11. Also, Joe Burrow fumbled the ball away on a scramble on fourth-and-goal from the 3 earlier in that same game before his injury.

On the other side of the ball, Tampa Bay is once again on the extreme. Just like the Buccaneers weren't good on scrambles, they also weren't good against scrambles, allowing 8.6 yards per carry on these plays. Because their defense was so stellar against the run otherwise, the Bucs see the biggest difference between DVOA with scrambles as runs and DVOA with scrambles as passes. Here are all the numbers for defenses, same structure as the table above:

TB -8.9% 5 -24.0% 1 -5.4% 5 -31.4% 1 -7.4% 23 8.57 93.2%
DEN -4.1% 8 5.1% 31 0.6% 11 -1.3% 25 -6.5% 28 10.04 100.0%
ARI -3.5% 10 -10.0% 14 0.1% 9 -16.3% 14 -6.3% 34 7.82 75.6%
CAR 11.6% 20 1.7% 25 15.5% 23 -4.5% 20 -6.1% 24 8.63 112.6%
JAX 26.3% 31 4.5% 30 30.1% 31 -1.4% 24 -5.9% 26 8.54 99.2%
ATL 9.2% 19 -15.6% 12 11.6% 19 -20.7% 6 -5.1% 29 9.10 65.6%
LAR -15.6% 4 -19.5% 4 -12.0% 4 -24.1% 3 -4.6% 26 7.50 97.5%
GB 1.4% 13 -1.2% 20 4.9% 15 -5.4% 18 -4.2% 32 6.81 64.3%
BAL -1.2% 11 -14.3% 13 0.4% 10 -18.4% 12 -4.1% 22 6.64 48.4%
NO -17.6% 3 -21.8% 2 -14.7% 3 -25.7% 2 -3.9% 18 8.83 75.1%
BUF -0.3% 12 -4.5% 17 2.2% 12 -8.3% 17 -3.8% 31 6.42 57.1%
CLE 14.5% 24 -1.6% 18 16.4% 25 -5.4% 19 -3.8% 26 6.69 64.4%
SF -5.6% 6 -15.7% 11 -3.4% 7 -19.3% 10 -3.7% 38 7.74 45.1%
SEA 11.9% 21 -16.9% 8 12.3% 20 -20.1% 7 -3.2% 37 6.65 27.1%
TEN 24.1% 29 -5.6% 16 25.3% 30 -8.8% 16 -3.2% 27 6.52 46.7%
CIN 17.0% 27 0.2% 22 18.6% 27 -2.9% 21 -3.1% 27 7.44 56.4%
NYJ 21.9% 28 -17.0% 7 22.9% 28 -20.0% 8 -3.0% 13 7.54 83.5%
NYG 12.0% 22 -7.0% 15 13.2% 22 -9.9% 15 -3.0% 32 6.53 45.9%
WAS -20.5% 2 -15.9% 9 -18.0% 2 -18.8% 11 -2.9% 26 7.12 45.2%
CHI 2.4% 14 -19.5% 3 3.4% 13 -22.5% 4 -2.9% 29 6.31 15.9%
DET 31.5% 32 2.3% 27 32.5% 32 -0.5% 27 -2.9% 18 7.06 62.0%
NE 6.9% 17 9.1% 32 9.1% 18 6.6% 32 -2.6% 37 6.95 46.9%
PHI 14.7% 25 -15.9% 10 15.9% 24 -18.2% 13 -2.3% 19 8.05 62.0%
LAC 7.0% 18 1.6% 24 8.2% 17 -0.7% 26 -2.3% 31 5.61 31.5%
IND -3.8% 9 -17.6% 6 -2.9% 8 -19.7% 9 -2.1% 30 5.37 13.5%
PIT -20.7% 1 -19.0% 5 -19.8% 1 -20.7% 5 -1.8% 24 6.71 13.5%
HOU 24.6% 30 3.4% 28 25.2% 29 1.6% 29 -1.8% 16 5.31 45.7%
KC 5.5% 16 3.8% 29 6.7% 16 2.5% 31 -1.3% 32 5.78 33.7%
DAL 12.9% 23 -0.9% 21 13.1% 21 -2.1% 23 -1.1% 35 5.97 24.2%
LV 16.4% 26 1.2% 23 16.5% 26 0.3% 28 -0.9% 25 6.48 17.5%
MIA -4.7% 7 -1.6% 19 -4.1% 6 -2.3% 22 -0.7% 22 6.14 20.3%
MIN 4.4% 15 2.0% 26 4.7% 14 1.9% 30 -0.2% 28 5.89 9.3%

The Vikings were very good against scrambles the last two seasons (9.3% and 6.5%) and the Broncos were very bad against scrambles the last two seasons (100.0% and 115.6%). But those seem to be exceptions, with the year-to-year numbers overall not showing much consistency. For 2015-2020, the year-to-year correlation of offensive scramble DVOA is .18 while the year-to-year correlation of defensive scramble DVOA is .13. Obviously, the small sample size affects those correlations. The worst offensive scramble DVOA of the last six years belongs to the 2017 Chargers because Philip Rivers only scrambled twice and fumbled on one of those carries.


21 comments, Last at 09 Apr 2021, 12:46pm

1 One of the big issues with…

One of the big issues with scrambles is they are fundamentally different from pass plays and run plays in many respects, but especially when comparing them to a baseline. This is probably also true with screens by the way.


Every team runs and every team passes to some lesser or greater extent, but the amount of variance in frequency between the teams is nowhere near as skewed as it is with scrambles.


How much sense does it make to compare the scrambling efficiency of the Ravens with a team like the Tampa Bay buccaneers? And even in this context, scrambles don't happen at random. they are baked into the team's offensive approach as well as the defense's expectations. In that sense making comparisons between the two teams is even more incorrect. A scramble from Tampa Bay is a fundamentally different play than a scramble for Houston or Seattle or Arizona. 

Pat mentioned this some threads ago, but maybe the proper way to do this is to scrap the whole play concept all together and do some kind of clustering. I resisted that idea because you lose a lot of interpretability going that route, but scrambles are definitely not a play you can categorize as a pass or a run in my opinion.

16 Disagree

The game situation that leads to a scramble is a failed pass play.  Either, the pass rush is getting home, the QB didn't see the open player, or no player got open.  A scramble is very clearly replacing a passing outcome: usually a check down, sack or intentional incompletion.  And, as to teams that plan for scrambles, they generally do so in which players they chose for WR/OL and by judging overall efficiency on called passes.

17 The issue is - when we lump…

In reply to by gomer_rs

The issue is - when we lump scrambles into passing and then use the standard passing statistics as a baseline, we are no longer comparing apples to apples. 

As you noted correctly, scrambles occur in the failed to pass portion of the game. They are fundamentally plays from the last resort, not part of the on schedule portion of the game. I suppose a better system would lump scrambles with sandlot throws or maybe check downs and dump offs. 

I do think teams with scrambling qbs design their offenses differently. They will probably not run many timing/option routes and will likely have receiver improvisation as part of the design. You may never see such a concept with Tampa Bay, hence even comparing scrambles between the two teams isn't the same thing. This isn't like comparing two runs up the middle between the teams. Those are pretty close to apples and apples. 

19 When looking at aggregate passing numbers:

I don't understand your concern.  When assessing passing DVOA we're attempting to identify whether and how much a play has an impact on the offense, when it's a called pass.

The outcomes are finite:


Completion or Incompletion

+passing yard or -passing yards

Scramble or Sack.

All of these together through DVOA create a percentage of yards gained over average (and success rate) per down and distance.

If you treat scrambling different then the other pass play outcomes then you are distorting the information.  Fundamentally, all passing stats have been distorted when describing the passing game BECAUSE they didn't treat scrambles as traditional passing outcomes.

Whether we're talking about Steve Young, Russel Wilson, Tom Brady, Peyton Manning, etc.  Of course if your QB is Young or Wilson you want WRs that can improvise when the play breaks down... but whether that is a net benefit or cost versus Dan Marino or Tom Brady is best assessed by a DVOA which includes scrambles as a normal passing outcome.

As long as we are comparing Net Yards +/- per passing play call it doesn't matter if we call it a scramble, sack, completion, bomb, dump off, check down, etc.

The only play outcomes where the description is actually of something fundamentally different are TD, INT, and Fumble.

2 I think counting scrambles…

I think counting scrambles as pass plays is a positive development. However, I think it would be most useful to have a number that aggregates scrambles with sacks, to show a complete picture of what happens when a QB drops back and doesn't end up throwing the ball.

For instance, the numbers show that Deshaun Watson scrambled (positively) 62 times for 395 yards. But he was also sacked 49 times for -293 yards. So in addition to his impressive passing numbers, he really had 111 dropbacks that didn't result in passes and gained a net of 102 yards. Since he threw 544 passes, this means that about 17% of the time he dropped back (111/655) he didn't end up throwing, and those plays gained an average of just less than a yard. This seems like a more helpful way to analyze his pass attempts than saying his 62 positive scrambles gained 6.37 yards per attempt, and also his 49 sacks lost an average of 5.98 yards per.

3 The distinction

is that not all sacks are trying to scramble though. Just like not all rushes are trying to rush...at first. Some of those sacks very well may be them getting sacked before they decide to do anything.

4 Scrambling vs Sacks

This is a very interesting article (and not just the DVOA portion).   I always wondered about Russell Wilson's (and other "scrambling" QBs) propensity to take sacks.   The argument was Wilson has a poor offensive line.  And while some of those sacks may be the OL's fault, from observing Wilson it seems that he just holds the ball longer and scrambles in order to extend the play hoping for the wide open receiver.  

But the scrambling data adds another dimension - if the scramble yards are added to the negative sack yards lost, Wilson (and the Seahawks) still gain yards overall.   From the data above - Wilson had 54 scrambles for 460 yards offset by 48 sacks with 304 yards lost (I looked that part up for him and other NFL teams) and is still a net overall positive 156 yards from these "extended" plays.  And that doesn't count the times that Wilson's gamble of holding the ball opens up a big pass downfield.  

This seems to be the case for the other "scrambling QBs" -- Ari, Hou, Bal, Buf.   More scrambles are accompanied by more sacks - but overall net positive yards when the yds gained from scrambles are netted against yards lost from sacks.   (Mahomes is in another category, of course - more scrambles, but less than average sacks - overall positive net yards).

At the other end - the immobile QBs who get rid of the ball quick (TB, PIT, IND) seem to have a different mentality.  Knowing that they can't move much, they focus on not taking a sack. (perhaps throwing it away or always dumping off to a back).

Perhaps this is a blinding glimpse of the obvious - but maybe sacks are more of a QB choice of what he wants to maximize vs what his ability is but if you have a scrambling QB it seems to me high sacks are a 1) cost but the benefits are 2) scramble for positive yards and overall net positive and 3) extending the play opens up a downfield pass that wouldn't have been there otherwise.

21 Lamar's sacks & scrambles in 2019

If you net "sacks & scrambles" together as times when the QB is "forced not to pass" – plays that are wins for the pass rush; dropbacks without a pass attempt – in 2019, Lamar gained 5.2 yards per play on those. 

His yards-per-attempt was 7.8, which was quite decent, tied for twelfth in the league.  But even when the pass rush was getting there and forcing him NOT to throw, he was still picking up 5 yds per play.  That's amazing.  Talk about keeping the offense on schedule.  The second place QB was Mahomes with 2.18.


(I haven't looked at the numbers for this past season, but the eyeball test suggests they won't be as good.)

5 Old Man Rodgers

Rodgers has absolutely lost a step, but he still knows how to pick his spots, apparently. GB was 4th in scrambling DVOA, but bottom-10 in yards per attempt (and middle of the road in number of attempts). Seems like his strategy is to get to the sticks if he can and get down.

6 First paragraph: "Now that…

First paragraph:

"Now that we've counted scrambles as runs for a year"

I think you have been counting them as passes then, no?

10 The league ended up with 68.5 DVOA on scrambles

In addition only 3 teams had negative DVOA on scrambles.  

Now, how to eliminate the seemingly great value of a scramble?  The problem is the definition.  You mention that scrambles are the positive version of sacks.  But some sacks are scrambles, once you take out all of the negative plays (sacks) and count only positive gains, of course DVOA is great on scrambles.  Scrambles are the success involved in avoiding the sack. Sacks are one of the worst offensive results in football.  If we discounted all RB plays that resulted in no gain or losses, especially those that resulted in fumbles behind the line of scrimmage, the rest of the run plays would also look very good.

I joked during the season in a post that the best way for the Ravens to covert 3rd and 7 was to not make any attempt to throw, have Lamar wait for the pressure, and take off.  He is such an extreme example, the drawback of this tongue in cheek strategy is that he has a good chance of getting sacked and worse yet fumbling.  There exposes another problem; most of the QB fumbles are sacks which are not counted as scrambles.  

If you have a play that averages 68.5 DVOA you would use it constantly.  The scramble is often an unintended consequence of poor blocking, the QB breaking containment, or shabby defense with a large open area with which to run; the scramble is not a planned play.

I have been a big critic of Watson on Houston as he is a sack machine. We know that sacks are more on the QB than on the line.  This article shows that Watson is also a scrambling machine too.

There must be some way to count the great positive impact of scrambles with the offset of the great negative impact of sacks.



18 Why is this a problem?

Now, how to eliminate the seemingly great value of a scramble?

Why do we need to do this? Why aren't we ok with just allowing for scrambles being very good?

The scramble numbers are just a subset of the overall passing DVOA, as are sacks. Yes, when you filter just for scrambles, the numbers are great...just like they would be for any filter that exclusively focused on positive plays. By the same token, if you filtered for only sacks, the resulting DVOA would be terrible. You know the results are "skewed" because you're purposefully skewing them with the filter....it shouldn't need a constructed numerical caveat.

I don't understand the need to offset purposefully filtered results with opposing filtered results....one would expect that somebody trying to answer a question with those filters would already understand the nature of the results.

Besides, a number that has sacks built in to offset scrambles already exists; we call it "passing DVOA" :)

I think it's perfectly OK to say "when a QB avoids a sack and takes off on a scramble, the results are usually extremely positive". Any risks or data distortions are self-evident within that statement already.

13 This is a step in the right…

This is a step in the right direction. But...

If you want to use individual stats, then we count scrambles as runs and compare them to a baseline of all quarterback runs.

... you are still screwing up QB run stats by lumping scrambles with designed run plays, while leaving sacks with pass plays.

You should go all in on scrambles as part of the passing game. QBs use the threat of the scramble to free receivers, and the threat of throwing to open up space for a scramble. And I think it is much more enlightening to say “scrambling is X% of QB Z’s passing game” than to treat scrambling as a part of the running game. Scrambling has little impact on the running game (for QBs or anyone else), and a lot on the passing game.

The biggest problem with including scrambles in individual running stats is that it pollutes the DVOA baseline. Scrambles are cherry-picked to be positive plays, both by definition, and because QBs can choose to make a throw when the scramble looks well defended.

Next season, you should treat scrambles as pass plays for all purposes.

14 minor nit to pick

".. you are still screwing up QB run stats by lumping scrambles with designed run plays,"

A lot of designed QB runs are successful to begin with (as long as they don't fumble, of course). QB sneaks are almost always high value plays b/c of the "bonus" of converting a 1st down or TD. There are other QBs who get designed runs in short-yardage situations too--those plays are also mostly successful. While there are some QB's who might be running in non-short-yardage/goal-line situations (QB draw, for instance), I would bet that there are many more QB runs that are either scrambles or short yardage. That's why almost all QBs have very high rushing DVOA--they mostly run in high leverage situations, and are mostly successful. Even some of their "failures" in a binary measure like success rate are mildly successful in DVOA. 

I also agree with others who have mentioned that it would be interesting to see all QB scrambles and sacks combined in one stat for both DYAR and DVOA. 

20 This matches the eye test…

This matches the eye test for me, as for some reason I saw a lot of GB and HOU games in addition to BUF and WAS. Allen's scrambles were deadly, and Rodgers' were nearly as much so. So many of Watson's seemed more about desperation than picking his spots and throwing it away when he wouldn't be successful, and of course WAS had success because, well, wait, why did they have success? Was Haskins especially good?