Introducing Running Back Success Rate

by Aaron Schatz

It started with the mind-blowing Terry Shumpert card [1].

A few years ago Wizards of the Coast, the same company that does Magic The Gathering and a ton of other games, produced a baseball-oriented collectible card game called MLB Showdown.  Well, my then-roommate Sean Benak and our good friend Ian Dembsky (a.k.a. Ian from Scramble for the Ball) both got some packs of cards and they would sit at the table at our house and play against each other.  You built a team using a salary cap system, so you would end up with a team mixing stars and scrubs.  Because the cards were based on the previous season, with no fix for park effects, the best player in the game was an undistinguished utility infielder named Terry Shumpert [2] who managed to have a really good 250 at bats the year before in Coors Field -- at near-minimum salary.

So back in March, Sean and I were driving out to a poker tournament and we got to talking about Terry Shumpert, who was in Red Sox camp at that point.  That got us to talking about Terry Shumpert's obscenely good MLB Showdown card, and that got to reminiscing about MLB Showdown in general.

Sean said to me, "You know, that game made you realize just how bad a player like Raul Mondesi [3] was.  On the surface, he looked like a star because he had a lot of homers and RBI and stolen bases.  But when you looked at his MLB Showdown card he had a low on-base number and that just meant he got out all the time."

And I said to Sean, "I wish I could do something like that for football, to point out to people how often running backs run for bad plays. With a pass, you know when it is incomplete, but with a running back you can run for 2-3 yards over and over on first down, and you'll rack up the stats even though you haven't really helped the team. Those runs are really 'outs' because they aren't much better than an incomplete pass, and a lot of times a running back will have a lot of these little pointless gains but look really great because of a few highlight reel runs."

From this conversation came the idea of "running back batting average," a statistic that could be used to measure the consistency of a running back by treating all runs as either hits or outs.  The length of the run wouldn't matter; the only important question was whether the run was a "success" or not.

I went home and started playing around with the idea.  At first, I used the same formula used in VOA to determine when a run was a success.  (What is VOA?  Explained here [4].)  On first down, success required 45% of needed yardage -- five yards on 1st-and-10, seven on 1st-and-15, and so on.  On second down, success required 60% of needed yardage.  On third or fourth down, success required converting for a new set of downs.

After some thought, however, I decided to fiddle with these benchmarks a bit.  Work on VOA has shown us that five yards on 1st-and-10 is a good indicator that the team will eventually convert for another set of downs, but teams aren't used to requiring five yards from their running backs on every carry.  Most teams consider a four yard carry on 1st-and-10 a success -- which it is, somewhat.  It isn't as good as a five yard run, but it is close, and it gets a partial credit in the VOA system to recognize this.  Since I was going to compare rush attempts to other rush attempts, not even thinking about passing, I decided to lower the standard to what people are used to expecting from the ground game.

There was another issue, though, and that was the question of running out the clock.  After all, if you are winning at the end of the game, you are probably satisfied with a shorter run as long as it stays in bounds; down by a couple of touchdowns in the fourth quarter, four yards on first down isn't going to be a big help.  So with some changes to make up for different expectations at the end of the game, the formula ended up like this:

• In general, a play counts as a "hit" if it gains 40% of yards on first down, 60% of yards on second down, and 100% of yards on third down.
• If the team is behind by more than a touchdown in the fourth quarter, the benchmarks switch to 50%/65%/100%.
• If the team is ahead by any amount in the fourth quarter, the benchmarks switch to 30%/50%/100%.

The resulting number isn't quite the same as measuring variation of yards per carry, because of the fact that the formula takes down and distance into account, but I think it is a better indicator of consistency because a back doesn't get penalized for plunging two yards ahead on 3rd-and-1.

The next problem was what the numbers looked like.  Let's take a running back -- say, Edgerrin James in 2003.  If you asked me how often Edgerrin James ran for a successful play in 2003, and I told you 49.5% of the time, is that good or bad?  You don't know, do you.  Heck, I don't know and I invented this thing.

The solution came from the same place as the initial inspiration for this statistic: batting averages.  If you took all 45 running backs who ran the ball 100 times or more in 2003, and looked at the percentage of their carries that resulted in "hits," you would have a range from Priest Holmes at .578 to, yes, DeShaun Foster at .363 (That man is just a glutton for punishment around here).  The total average for those 45 running backs is .458.

But subtract .200 from each of those numbers and you end up with a statistic that looks just like batting average.  The league average is .258.  The best player has .378.  The worst player has .163.  Over .300 is very good.  Under .200 is very bad.  It's easy to understand without needing to think it through too intensely.

In the same way that batting average does not tell you the best hitters in baseball, RBBA is not the best statistic to judge which running backs are the best in the league.  DVOA and DPAR do more to tell you how much a running back helps his team win, but RBBA helps to tell the story of how that performance is shaped.  While the best players in RBBA usually are among the best in DVOA, and the worst players in RBBA usually are among the worst in DVOA, that's not always the case.

For example, take James Jackson of the Cleveland Browns.  First of all, Jackson barely makes the minimum carries for our list with 102.  So it wouldn't be surprising if something was a little strange about his season.  Despite being the fourth-most consistent back of 2003, Jackson has a DVOA of -5.6%.  It turns out that Jackson in 2003 was the football equivalent of a singles hitter.  He had a lot of runs of 4-6 yards but didn't have a single run over 18 yards.  Some other backs like this in 2003 included Tiki Barber (.275 RBBA, -8.4% DVOA) and Jerome Bettis (.280 RBBA, -18.2% DVOA).  Often running backs fall into this category because they fumble the ball a lot, which is a big hit to their DVOA and DPAR but not to RBBA.  That explains Barber as well as Travis Henry (.302 RBBA in 2002, although only .249 RBBA in 2003).

The flipside of James Jackson is a running back who despite a good DVOA has a lousy RBBA, because he collects a ton of one and two yard runs and then every so often breaks out a really long gain.  The poster child for this type of running back is Deuce McAllister.  If you have wondered why our DVOA ratings don't have McAllister higher despite all his highlight reel runs, this should help explain.  In 2003, McAllister had a near-average -1.8% DVOA but a very low .210 RBBA.  In 2002, he had a slightly higher 3.4% DVOA and a slightly higher .220 RBBA.  Here is a breakdown of McAllister's runs on first down compared to all NFL running backs:

Every time McAllister is running for one or two yards on 1st-and-10, he is running into an "out" that leaves Aaron Brooks and the passing game in a more difficult situation on second down.  That leads to fewer conversions to first down and more punting by New Orleans, which counteracts the couple of long runs he gets in each game.  When all his runs are considered, McAllister helps his team no more than an average running back even though he usually ends up with a high yardage total and a high number of yards per carry.

Other backs who had even higher DVOAs than McAllister in 2003, but also had lower RBBA numbers, included Brian Westbrook (33.5% DVOA, .248 RBBA), Garrison Hearst (7.1% DVOA, .244 RBBA), and LaDainian Tomlinson (18.8% DVOA, .248 RBBA).

Looking at RBBA can also tell you a little bit about the different ways that two running backs on the same team get used.  (Rank here is out of 53 running backs with at least 75 carries in 2003.)

As you can tell, running backs who are considered short yardage, between the tackles power runners tend to do much better in RBBA when compared to running backs who are considered third-and-long, run to the outside finesse runners.  That's because the second group of guys is much more likely to get loose for a big gain, but also to get caught behind the line of scrimmage.  This distinction makes Tiki Barber's high RBBA look particularly out of place, and it also makes RBBA the opposite of baseball batting average.  In baseball, players who are better than their BA are usually power guys; in football, running backs who are worse than their RBBA are usually power guys.

The other distinction between players with high and low RBBA numbers is often offensive line.  Like a running back's DVOA, RBBA makes no attempt to separate a back from the quality of his blockers.  If a quality running back is running into a lot of "outs" but usually runs for a long gain when he gets a "hit," that is often a player who is often stuck with a short gain because his offensive line couldn't block well but manages to avoid lots of tacklers when he gets through the initial hole.  A number of the players who have lower RBBA but higher DVOA correspond to the teams in our offensive line yard ratings [5] who have lower rankings when it comes to line yards per carry but very high rankings when it comes to percentage of rushing yards collected more than 10 yards past the line of scrimmage.  In other words, LaDainian Tomlinson and Deuce McAllister.  That makes me wonder about Jim Schwartz's contention [6] that New Orleans has the best offensive line in the NFC South.  (What are line yards?  Explained here [7].)

While the center of the RBBA scale is the same as the center of the batting average scale, the extremes are a bit wider.  Each year you will end up with a couple of players up over .350 and a couple of players down below .180.  2002 is a better example of how extreme things can get.  Here are the top five and bottom five RBBA numbers from 2002 among running backs with at least 100 carries:

Expansion football: catch the fever!  The Houston Texans' search for a running game in 2002 was much like the Illinois Republican party looking for someone to run against Barack Obama -- just get someone in there, it doesn't matter how embarrassingly they get clobbered as long as they exist.  Also, like Alan Keyes, Jonathan Wells prefers to be addressed as "Ambassador."

RBBA will hopefully be a useful addition to our statistical toolbox for the 2004 season.  You'll find that it is now listed on the running back statistic sheets [8] and we'll be updating it as part of the 2004 numbers during the season.

One more thing I should note: As of now, RBBA is not adjusted for defenses faced.  My goal was an easy-to-understand statistic, not a perfect one.  If people feel that an opponent-adjusted statistic would be better, I can work on that for the future.