Stat Analysis
Advanced analytics on player and team performance

Why Successful Run Plays Work

New York Jets RB Bilal Powell
Photo: USA Today Sports Images

Guest column by Caio Brighenti

"Running backs don't matter." If you've read any analytics article about running in the NFL, odds are you've heard that phrase. But, if you watched the 2020 playoffs, then you saw firsthand how Derrick Henry against the Patriots and Raheem Mostert against the Packers proved that a quality run offense can win big games. So, what makes for a good run game?

I explored this question in my submission for this year's NFL Big Data Bowl student subcompetition. Instead of tackling the strategic question of whether it's worth it to run at all, I decided to investigate what separates runs that work from those that don't.

To start, let's look at two plays that were nearly identical from the runner's perspective, but that had vastly different results. In both plays below, the runner received the ball 4 yards behind the line of scrimmage, approximately one second after the handoff, and was moving towards the left end. However, Bilal Powell gained 12 yards, while Wendell Smallwood lost 4.

So, what happened? Why did the defense stop Smallwood almost immediately while Powell escaped for a first down? It's not always easy to tell from the film, so we can turn to the bird's-eye view offered by the dots. In these plots, the runner is shown as the white dot.

 

Here, it becomes clear where these two plays differed: Smallwood had two defenders moving into the space he intended to run towards, while Powell was fortunate to have wide open space ahead of him.

While looking at the dots visually gives us the information we needed, this approach isn't at all quantitative and doesn't provide any measurable way to describe either play. To quantify what's happening in both plays, we can borrow a concept from soccer analytics: pitch control, or field control for those offended by the word pitch.

The math behind field control can get complicated, but the concept is straightforward. If we assume that each player on the field controls some area around them based on where they are and where they're moving to, and we can find a way to calculate this, then we can arrive at each team's overall field control by calculating each player's control and adding it up across the entire team. Then, if we take the difference between the offensive and defensive control, we find which team has ownership over each point on the field.

With this concept of field control, we can get a measurable understanding of what made the two example plays different. In this plot, the yellow and purple areas represent areas of offensive and defensive control, respectively. Blueish-green areas represent neutral space, where neither team has an advantage.

 

While Powell has plenty of neutral or offensive-controlled space between him and the first-down marker, in the case of Smallwood the dark blue defensive control wraps around him, cutting off his path.

It's clear in these two examples that quantifying field control is a good approach for identifying differences between plays that work and those that don't, but if we're interested in making conclusions about running in general then looking at just two plays isn't enough. Instead, I calculated field control in a standard area around the line of scrimmage for all 23,000 plays available in this year's Big Data Bowl dataset.

Once I had field control calculated for each play in the dataset, I grouped runs based on the runner's direction of motion and looked at what the average field control looked like for each type of run. The interpretation for these plots is the same as before -- yellow represents offensive control, and purple defensive control.

Initially, these plots aren't particularly interesting. In all three groups, the defense has greater control to the right of the line of scrimmage, and the offense to the left of the line of scrimmage. But, if we instead look at the difference between successful and unsuccessful plays, the results are far more interesting. In this case, I define successful plays as those gaining more than 1 yard.

Note that the interpretation here is slightly different -- yellow represents areas where successful plays have greater offensive control than unsuccessful plays, and the opposite for purple.

Finally, we can answer the question we set out to address: what separates runs that work from those that don't? In these plots, it's obvious the biggest difference between successful and unsuccessful runs is control over specific spots on the line of scrimmage. The exact position of this spot also varies through each group, demonstrating a relationship between the runner's initial direction and where this gap should be.

Interestingly, successful plays actually have less control over the space past the line of scrimmage than unsuccessful plays. This suggests that the amount of space the offense can control is finite -- instead of aiming to control more space overall, teams might want to instead focus all resources in producing a single gap at the line of scrimmage.

In short, this application of tracking data confirms what the analytics has long suggested about the run game: the space created by the offensive line is what makes for consistently good runs. Derrick Henry wouldn't have gotten any more yards than Wendell Smallwood did, and I probably could've gotten the first down with the amount of space Bilal Powell had.

This isn't to say that running backs are irrelevant. In the words of Josh Hermsmeyer, running backs are just all good -- they're a solved problem.

Caio Brighenti is an undergraduate in his final year at Colgate University and a finalist in this year's NFL Big Data Bowl. You can follow him and his football analytics work at @CaioBrighenti on Twitter.

Comments

49 comments, Last at 05 May 2020, 3:50pm

1 Colors

I have nothing particularly insightful to say about the article (other than I think it's an interesting/useful idea on how to quantify control).

But as someone's who's (mildly) colorblind, I specifically wanted to thank you for using yellow/purple as the ends of your spectrum, instead of something like green/red (or green/orange, which I have seen and is even worse for me personally). I recall reading in an article that I can't find now that something very similar to the spectrum used here is ideal for covering multiple types of colorblindness, and it's certainly helpful for me.

6 Really glad to hear the…

In reply to by Salur

Really glad to hear the color scheme worked well for you. Early on this project used each team's primary color to signify their control and white as a midpoint and I got the feedback that it would be nearly illegible to people with colorblindness. Clearly the feedback paid off!

38 selecting colormaps

You made the right choice in using a perceptually uniform, colorblind friendly colormap. But, you can go one step further for creating clearer figures.

When you have a scale that has a natural midpoint, for instance around 0.5 in your field control plots and 0 in your difference plots, with interesting features at the extreme values, color theory suggests you instead use a diverging colormap. The python library, seaborn, has a decent explanation on these as a starting point. This type of colormap will allow you to quickly see where neither team has an advantage and where one team has more control of the other.

2 I think it also helps when…

I think it also helps when you block the guys in front of you (didn't happen against the Saints, the Falcons practically blocked themselves other than the guy who fell over himself trying to get to Powell.)

However, a different running back might fail to get the 12 yards Powell got by being decisive.  Leveon Bell could have stutter stepped himself into a 2 yard gain.

3 Interestingly, successful…

Interestingly, successful plays actually have less control over the space past the line of scrimmage than unsuccessful plays. This suggests that the amount of space the offense can control is finite -- instead of aiming to control more space overall, teams might want to instead focus all resources in producing a single gap at the line of scrimmage.

This may be an artifact of your definition of success -- 1 yard runs can be completely accounted for by initial line blocking, those are basically sneak plays.

If you defined success as positive DVOA, you may arrive at very different numbers. Otherwise you are selecting for Leroy Hoard:

Hoard reportedly once said to his coach, "Coach, if you need one yard, I'll get you three yards. If you need five yards, I'll get you three yards."

 

 

 

5 You're 100% right that the…

You're 100% right that the definition of "success" here (or anywhere) is subjective. From a game strategy perspective, positive DVOA or EPA is definitely a better measure of success. My objective here though was just to get at what's happening on the field, irrespective of context. 

Maybe it's more accurately phrased as what makes for a run that goes for a gain as opposed to what makes for successful runs. Knowing how to set your runner up for success isn't necessarily the same as knowing when to run the ball.

7 Exactly. On the oppposite…

Exactly. On the oppposite end of the spectrum is Adrian Peterson from 2012; 2097 yards, 6 yards a carry, and 1019 yards after contact. I don't think I fully appreciated that season as it was happening. The blocking was mediocre, and The Ponderous One had a overall poor group of receivers, so opposing defenses didn't really have to devote much scheming to stop the pass. The Vikings defensive backfield was atrocious. Yet Peterson dragged that roster to 10 wins. Whenever anybody says you should never give a rb big money, I think I mostly agree, even with rbs who pile up yards. But I also say never say never. Barry Sanders would be worth every penny today as well. Same with Tomlinson.

 

9 But would Peterson be worth…

But would Peterson be worth every penny? Don't get me wrong, as a runner, he's probably the best I've seen since I've followed the league. But he was a poor receiver and an even worse pass blocker. And while his 2012 season was the stuff of legends, there's no way he'd be able to rescue an offense by himself every year the way. 

If he wanted big skill position money, I'd seriously be mean and franchise tag him twice and let him walk. The NFL and its union have basically given a big finger to the running back position. 

12 His ability to turn any…

His ability to turn any handoff into a touchdown, from any spot on the field, really made his runs much,much, less distinguishable from pass plays, than can be said of all but a very rbs. Even among HOF rbs, that explosiveness, and how it makes defenses one dimensional, is very unusual. Oh, if only Favre had gone full A-hole in 2008, and forced his release, instead of accepting a trade to the Jets. That o-line was significantly better than the 2009 group, and Peterson was an acceptable blocker when he was protecting Favre.

Among the young guys today, I really don't see any who are as valuable as a Peterson, Sanders, or Tomlinson. Barkley has a chance to be.

31 Hard to put those guys in the same bucket

In today's game, CMac seems to be the closest comp to Tomlinson and he's plenty valuable. There are no Sanders or Peterson type backs, but there's very little about the numbers that suggests that one-dimensional runners can change the game or win Superbowls, not today, and not back in the 90s. 

4 Was anybody doubting that…

Was anybody doubting that superior blocking is the most reliable path to good results from running the ball?

8 The following questions come…

The following questions come to mind from your statement:

1) if you are right, then how much extra value does a good running back bring in today's NFL?

2) If you are right, then how easy is it to find a full team of good blockers? 

3) Even if you are right, just how much value is running the ball going to give when passing seems more effective and potentially easier to build?

 

Btw, its why I believe the QB has outsized importance in today's nfl. How much consistent production can you get when you are talent poor?

 

 

 

13 A merely good, or even very…

A merely good, or even very good rb, doesn't bring that much added value. But getting 5 blockers together without a single one being below average is a considerable task. Yes, given you can reliably expect the league to continually tweak the rules to make passing easier, leaning towards building a passing attack likely makes more sense.

However, one must always have enough of a contrarian ability to take advantage of market inefficiencies.

14 My 2 cents

1--Extra value from a good RB=understanding PASS-blocking schemes, ball protection, ability to break tackles (either by strength or shiftiness), and the immeasurable "heart"--that guy who is always fighting for the extra yard, refusing to go down, etc. And probably in this order.

2--I think the key is obtaining a team full of serviceable blockers. It's wonderful to have that DAL line of a few years ago, but that's hard to do. However, I think it is relatively easy to have 5-6 average-type blockers. The problem is that you need 5-6, versus 1-2. I think every team ought to try to draft at least 1 O-lineman every year, and at least one 1st to 3rd rounder every 3-4 years. 

3--Running the ball has value, just less than passing overall. However, as a regular here at FO, you know that there are situations where running the ball more effective than passing. And you have to have somewhat of a balance, at least in the earlier parts of a game and in certain situations. So, while drafting a RB high is not wise in general, there are some that were obviously worth it. If your team happens to obtain one, great! But I think we all agree that many RB's are equally good at getting the yards that the blocking provides, and not much else. And that these guys are fairly easy to find, and relatively cheap to employ. 

17 I disagree with point number…

In reply to by Joseph

I disagree with point number 2. How many teams have 5 serviceable offensive linemen? Its far fewer than you might think and even then, I don't think 5 competent run blockers gets you anywhere interesting, 

18 There's usually some…

There's usually some correlation between competent pass blocking and competent run blocking.

Put it this way -- it's an unusual year when none of the top-4 best rushing offenses make the conference championships.

42 Serviceable

Serviceable for me means: reasonably competent, not a penalty machine, not injury-prone. In other words, a guy that is kind of "meh"--not bad enough to get benched/replaced, but not good enough to necessarily warrant a pricey extension. Can a team get 5 of those guys? I mean, your team's best lineman can get injured in training camp and miss the whole year--but it's not as if the coach/GM can prepare for that. But the coach/GM can have a draft-pick/minimum salary vet for depth. 

IMO, an O-line should have 1 guy that is Pro-Bowl quality (since at least 16 make it, that's 1/2 per team for original selections--not counting alternates), another guy that has been/could become that good, and then the other 3 can be a mix of average veterans/young developing players. That's why I think a team should draft one every year.

Interesting exercise--pick two random teams. Look at their 5 best O-linemen. If you picked the best of the two at each position--you should in theory have 2 from each team, and the 5th spot being a tie. For this exercise, you would have 3 versus 2 (no ties). But if team A has 4, and team B has 1--then team B either has a problem or team A has a really great O-line.

15 I don't agree that superior…

I don't agree that superior blocking defines rushing success -- or that RBs are actually fungible.

Trent Richardson and Barry Sanders still existed. You can have success with no competent blocking, and failure with it.

23 You can disagree, but the…

You can disagree, but the numbers are pretty clear. The beauty of the Big Data Bowl is that it lets you separate the "who" (players/team) from the "what" (what's taking place on the field). What played out in the data is that the position/speed/acceleration/etc. of the players on the field was MUCH more important than who was actually running the ball...so much so that the best submissions didn't even look at who the RB was.

There is an interesting posting at the link below that compares players expected vs actual yards for 2017 and 2018. There was only a single significant outlier who was consistently good/bad both years...and that was Aaron Jones. Paying a guy like Zeke $15M a year makes no sense when someone like Isaiah Crowell would have gained 21 more inches per play (given the same blocking) for $12M less per year.

https://twitter.com/903124S/status/1200453012084252672

 

34 Numbers are only correct in aggregate

Anyone who watched Le'veon Bell and Bilal Powell last year saw that Powell was the better rusher behind that porous O-line, even though we have a career worth of data for both players saying the opposite is normally true. The argument about RBs is not that they don't matter, or that they are interchangeable, or that they are all equally useful. The argument is that except at the extreme margins, you CAN assemble a group of affordable RBs who give you a lot more cap efficiency. It's not an argument that anyone can run for 1400 yards. 

35 I wonder if style matters. A…

I wonder if style matters. A guy who blindly Leroy Jenkins forward behind a bad line will likely have more luck than a patient runner, because patience turns up nothing good under those circumstances.

If you put a QB behind a giant sack of wet crap throwing to a bunch of blind double-amputee, you'd probably have more luck with a Ryan Fitzpatrickgeraldsimmons than a Tom Brady, because precision and timing matter little in those circumstances versus damning the torpedoes and unleashing a dragon.

The trick is, it's hard to separate QBs from their lines or their teams, because they never change them.

37 You can disagree, but the…

You can disagree, but the numbers are pretty clear. The beauty of the Big Data Bowl is that it lets you separate the "who" (players/team) from the "what" (what's taking place on the field). What played out in the data is that the position/speed/acceleration/etc. of the players on the field was MUCH more important than who was actually running the ball...so much so that the best submissions didn't even look at who the RB was.

1. How you define your conditions has a lot to do with what results you find. This happens a lot in modeling.

2. "What played out in the data is that the position/speed/acceleration/etc. of the players on the field was MUCH more important than who was actually running the ball..." is an interesting finding. Especially that speed and acceleration do correlate to player identity. But it's not the sole factor, otherwise the best rusher would always be a Raider.

Then there is the Trent Richardson factor. Speed Score loved Richardson. His athletic numbers were stellar. He could not read a block to save his life. Speed and acceleration matter when a player can use that for something productive.

 

36 Lomas Brown was good. The…

In reply to by Raiderjoe

Lomas Brown was good. The rest were JAGs who could barely put competent rushers to 1000 yards before or after.

11 Plays chosen

One nitpick I have--not of your analysis, but the 2 plays chosen:

Against the Saints, it's 3rd & 6 with 21 seconds left in the half, and the Eagles need at least 50 yards to get in FG range. The Saints have no TO's left, and the Eagles only have one. So, the Eagles just want a running play to take the clock to the half, and go regroup, since they are losing by 17. It wouldn't surprise me if they were scheduled to receive the 2nd half kickoff. Also, the way their linemen are set up, they are trying to sell a pass play. This allows both Saints D-linemen an easy entrance into the backfield. Also, the double-team by the center and left guard is poor.

On the other play, it's literally the first play of the game. There is no special situation, nor any reason why either team might be guessing/planning a certain play type. To me, the initial personnel (2 TE & 2 WR, 1 RB) suggests either a run, or a play-action pass--especially considering the tight alignment of the WR & TE's on the left. Obviously, the Jets block this play better, and the Falcons have a poor run fit by their LB's. 

However, I agree with your conclusion--devoting more personnel to opening a hole probably makes any running play more successful, at least by your definition. We've all seen goal-line plays where the RB gets a sliver of a hole and knifes through just enough to have the ball break the plane..

16 Philly was actually kicking…

In reply to by Joseph

Philly was actually kicking off to open the 2nd. Not that it mattered.
https://www.pro-football-reference.com/boxscores/201811180nor.htm

One difference, too, is that NO had the 3rd best rushing defense in 2018. Atlanta had the 20th worst in 2017.

 

24 field control

field control is averaged over the whole length of the play?  I wonder if there a way to truncate the series after a certain amount of spacetime. I think the runs into the secondary are mostly going to be noise (they are obviously *value* but high variance and pretty rare.

40 It's actually not averaged…

In reply to by zenbitz

It's actually not averaged over the whole play. For the Big Data Bowl dataset, we only had access to a single frame of data right at the moment of the handoff. This is obviously a limitation and one way this work could be improved with better data. Tracking field control over the course of the play as opposed to assuming things will remain as they are in that moment would make for a more nuanced analysis.

25 This is why analytic people underappreciate the RB

They don't get the whole picture.  So basically all this article is saying is RB's get more yards if the OL opens up a good hole for them.  Well, welcome to football, that's always been how the running game works.

But they miss on several things and it would behoove them to study those areas to get a better understanding.

One of three things will happen:  The OL will open up a hole or the DL will disrupt the play. or it's a muddled draw.

If the OL wins, what is the difference per RB?  How much is that difference worth?

If the DL wins, what is the difference per RB?  How much is that difference worth?

If neither side wins, what is the difference per RB?  How much is that difference worth?

Then there are other data to be gotten.  What is each teams OL win rate?  DL win rate?  Not in yards but in did they open the hole or did the defense disrupt it?  How does each RB fare above average with or without the hole?  If you look at all that and you still find no difference I'd be amazed not all the great RB's had great  olines with high win rates.  Chicago was a crap team for most of Sweetness' career, for example.

I think you'll start to see RB's separating themselves if you study those factors.

26 Interesting post & paper…

Interesting post & paper.

What if you ran your model with a parameter, let's call it w, which is an angle defined relative to the RB's direction of motion at the time of the handoff. w represents how wide of an angle the RB can choose to attack.

In one limit, with w=0, the RB can only continue straight ahead in the direction that he is going at the handoff. So the model would just look at field control along that vector.

In the other limit, with w unboundedly large, the RB can go in any direction. So the model would look at field control along whatever path the offense has the most field control.

(If I'm understanding it correctly, these two extreme cases are what your paper looks at, with the labels "expected" for w=0 "ideal" for w=infinity.)

Intermediate values of w give you something like the Madden vision cone, a portion of the field that the RB can attack. It would look like a triangle coming out from the RB and opening up towards the line of scrimmage, with the assumption that the RB is free to go anywhere within the triangle but cannot go outside the triangle. So the model would look at field control along whatever path within that triangle has the most offensive field control. So it's like the paper's "ideal" analysis, but confined to a portion of the field.

Then instead of running one model with terms for both the expected and ideal paths, you can run the model for each value of w (I'm imagining it with only the terms for the "ideal" path within that angle, getting rid of the "expected" terms). Then see which value of w gives the best fit model.

That's one idea; there are probably other ideas in this space.

44 Great analysis

As much as I a) understand the whole you run when you win ethos and b) agree with the efficiency of passing over running.

However this article shows how much more complex and no well understood this issue is. 

I remember the old Bengals line that had Munoz and they literally drove the Oilers back about 7 miles in a game back in the day. My point here is that like Derrick Henry you can run a ball down a teams throat. That said it is hard to do. My prior is that we still haven't worked out the true value of the O line. Its like the forwards in rugby and especially the back row. If you are continually going forward and winning 50/50 ball then how much does that contribute to scoring a try versus what the backs deliver. The backs simply have to catch and pass if the forwards are controlling the gain line and keeping the defensive line on their heels.

An O line is the same. It may seem axiomatic that a line that keeps a QB clean and pass blocks well makes a difference. But if you are tearing up the pass game and can run block well, then, as the newer analysis says, you only need a replacement level RB. But that makes the point that we still don't fully understand the cumulative effect of the O line. 

Great analysis as always and it helps progress the body of thought on this subject. Luckily I am not colour blind.

 

45 Interesting look at the data…

Interesting look at the data. Would it be possible to isolate individual O-Line members and use their field control as a stat to measure O-line ability?

47 Ray Rice and Barry Sanders

Ray Rice and Barry Sanders are two running backs that fly directly in the face of this theory.

While it is true in these 2 plays that the number of defenders in the space mattered. There are plenty plays where backs avoid players in the backfield, or gain more yards after contact or make more defenders miss in the open.

 I chose these two backs because they both have unique qualities

Ray Rice was one of the best backs Ive ever seen at creating positive yardage on potentially negative plays, and Barry sanders had the unique ability to completely reverse field and outrun Safties once past the first level.

I find it hard to believe that Billal Powell would've gotten to the line of scrimmage in instances where Rice would have. I also find it difficult to believe that there were not runs where Rice got tackled after the line of scrimmage where Sanders would have scored. (Rice was historically frustrating at not finishing long runs for TD's)

 

As much as people want to rip running backs, I have personally seen certain Alpha traits that separate backs from their peers that are not offensive line related. Believing them all to be the same to me is just asinine and this isn't even talking about a back like Thurman Thomas who also provided receiving value.

 

I look forward to your response to this comment.

https://www.youtube.com/watch?v=PBhn1wMyzV4 Also as a fun exercise, tell me how many of these runs you would've expected Billal Powell to have the same result as Barry. (Btw I've owned Billal on my fantasy teams for several years so I'm not a Powell hater) Check out play #45 or #36 as a specific example where Barry provides clear Alpha value compared to Powell. 

49 If you think Ray Rice was…

If you think Ray Rice was good at getting something out of nothing, you would have LOVED to see Walter Peyton. One of my favorite players of all time.

In fact, it's disappointing that whenever the subject of great RBs comes up here, the examples are always Barry Sanders, Emmitt Smith, Adrian Peterson, even Terrel Davis. To me, Peyton is clearly and easily the goat RB. He was superb at everything.

48 Potential way to "value" running backs

It seems like most people are interested in how this analysis relates to running back value, so here's an idea on that front.

With the analysis already performed, for each run play, you have these different measures of field control (which I think we can take as a proxy for the characteristics and quality of blocking in front of the running back at the point of handoff). I don't think it would be too hard to treat these metrics as "inputs" into a model that tries to predict the outcome of each run play (e.g. number of yards gained, DVOA gained, etc.).

With that model, you can compare actual outcomes from each running play to predicted outcomes, and assign a residual/excess value generated by the back on each play. The idea is, controlling for the blocking in front, how many yards above expectation was the running back responsible for?