Writers of Pro Football Prospectus 2008

31 May 2013

RB Pro Days: When Pigs Fly (20 Yards)

There are a few positions in the NFL for which straight-line speed seems to matter. Chief among these is running back; does "Speed Score" ring a bell? Well, I'm putting together a running back projection model for Football Outsiders Almanac 2013 (only a month or so away!), and part of the research naturally involves how to handle 40-yard dash times.

More precisely, it involves figuring out how to handle 40-yard dash times at pro days. Conventional wisdom says that these times aren't trustworthy because they don't happen in an objective environment like the NFL Combine: Practically everyone in attendance (pro scouts, athletic department officials, college coaches, the players, etc.) has a clear motivation to engage in a shenanigan or 20.

Is conventional wisdom correct? To find out, I looked at every running back prospect from 1999 to 2013 who met the following criteria:

  • They played their final year in an automatic-qualifying FBS conference (or at Notre Dame).
  • They were not a fullback or a potential position convert.
  • They were either drafted or went undrafted despite being among NFL Draft Scout's top 50 at the position.
  • They ran the 40-yard dash at the combine.
  • They ran the 40-yard dash at their pro day.

That produced a sample of 99 running backs whose twin 40-times I could use to address the question, "Do backs have faster times at their pro day than they do at the combine?"

It turns out that the average 40-time at the combine for this group was 4.58 seconds; for pro days it was 4.55 seconds. That's not a huge difference, but a one-tailed, paired samples t-test suggests that the probability of it happening by random chance is less than 0.001 percent.

Now, you may have noticed I was careful to use the term "backs have faster times" instead of "backs run faster." Actually running faster is certainly a possible explanation, but so too are a whole host of other things (e.g., shenanigans). Obviously, I can't address this "why" question directly, but I did find something in the data that hints at potential explanation indirectly.

Namely, for 75 of the 99 running backs, we also have 10-yard and 20-yard splits for both venues; with some basic math, we also have the so-called "flying 20." If backs are simply running faster, we would expect that 0.03 average improvement to be spread out across the various splits in a systematic way. For all running backs (not included above) who ran at the combine since 1999, the 10-yard split accounted for 35 percent of their 40-times, the 20-yard split accounted for 58 percent (which means the second 10 accounted for 23 percent), and the flying 20 accounted for 42 percent. When divvying up 0.03 seconds, we're firmly in "distinction without a difference" territory, so I'm not silly enough to expect an average pro day improvement of, say, 0.02 seconds for the first 20 yards and 0.01 seconds for the flying 20. Nevertheless, we should see something approximating these splits.

The first half of the run is pretty much in line with our rough expectations. The 75 running backs had an average 10-yard split of 1.57 seconds at the combine and 1.57 seconds at their pro days. Similarly, the average 20-yard split was 2.61 seconds at the combine and 2.62 seconds at their pro days. Obviously, neither of these are statistically significant differences. And although we probably should have seen some improvement in those two splits, I'm not inclined to quibble over a hundredth-of-a-second or two.

The flying 20 is a different story, one worth gratuitous amounts of quibbling: The average drops from 1.96 seconds at the combine to 1.91 seconds at pro days. Per a t-test, the probability of that result being due to random chance is less than 0.00001 percent, which is even lower than our finding for the 40-yard dash as a whole.

Maybe, despite what a classical statistics test says, this is still just rounding error randomness; or maybe Chase Stuart is out there somewhere yelling, "splits happen!" My hunch, though, is that the conventional wisdom is right -- kind of. Yes, 40-times do appear to be faster at pro days than at the combine, and it's probably due in some part to the human element. However, our disbelief should only be focused on the flying 20, where the human element has tended to develop a spontaneous itch in its stopwatch trigger finger. Do I have definitive proof of this? Of course not. But I also don't think the average running back prospect can fly faster at his pro day when he just ran 20 yards at combine pace; it's only slightly more believable than seeing pigs fly beside him.

What do you think?

In case you're wondering about other timed drills, the average back from an automatic-qualifying conference ran his pro-day short shuttle 0.03 seconds faster than he did at the combine, and ran his three-cone a whopping 0.09 seconds faster. My guess (with tongue firmly in cheek): It's got something to do with the big cone.

Posted by: Danny Tuccitto on 31 May 2013

26 comments, Last at 05 Aug 2013, 12:27pm by brpjac

Comments

1
by Sifter :: Sat, 06/01/2013 - 5:35pm

I'm wondering if certain colleges have a more obvious difference between combine and Pro Day. Often you hear that a Pro Day was held on a 'fast track', and I wonder if there are a couple of colleges that are skewing the results a little. Anyone standing out??

Otherwise the times are not as different as I thought. Especially when you take into account that if you've run WELL at the Combine, there would be less of a need to run again at your Pro Day. Therefore I'd theorise that it's often guys who feel they've run badly at the Combine who would back up and run at their Pro Day too. And you'd think they'd: a) have a good chance perform better, and b) have been training their 40 a bit in the weeks between the combine and pro day.

2
by Danny Tuccitto :: Sat, 06/01/2013 - 7:00pm

The sample sizes are way too tiny to look at specific schools, but here's the slightly-less-tiny-sample-size breakdown by conference (name, # of RBs, average combine 40, average pro day 40):

ACC, 15, 4.58, 4.54
Big Ten, 14, 4.60, 4.55
Big 12, 17, 4.56, 4.54
Big East, 13, 4.59, 4.57
Pac-10, 18, 4.61, 4.58
SEC, 19, 4.58, 4.56

And since it was more so the point of the piece, here's the flying-20 breakdown:

ACC, 12, 1.97, 1.90
Big Ten, 10, 1.97, 1.92
Big 12, 12, 1.93, 1.92
Big East, 9, 1.96, 1.93
Pac-10, 15, 1.98, 1.92
SEC, 15, 1.94, 1.89

3
by fyo :: Sun, 06/02/2013 - 7:49am

Selection Bias

Couldn't the tiny (but apparently statistically significant) difference be completely explained by a slight selection bias? I.e. that backs who perform poorly at the combine (compared to what they believe they can do), are more likely to also attempt a run at their pro day. Similarly, backs who post an insane time during their combine might not have much of an incentive to try again at their pro day.

4
by Danny Tuccitto :: Sun, 06/02/2013 - 7:54am

For the entire 40, no doubt. But selection bias doesn't explain why these guys -- even assuming they're only running at their pro day because they're trying to post a better 40 -- run the first 20 yards in the same time they did at the combine, but can shave off .05 over the last 20.

6
by DisplacedPackerFan :: Sun, 06/02/2013 - 1:07pm

Well you need to look at this from the perspective of track and field, not football. The top sprinters in the world, once they get their block starts down generally only see an improvement over the first 20 meters (close enough to 20 yards) if they also manage to improve their reaction time. I thought the NFL 40 wasn't gun started but athlete started, the clock doesn't go until the the runner leaves the blocks, so reaction time isn't a factor. But it still takes 20 yards or so (which is about 15 or so steps) to sort out the start. At that point your speed is no longer dominated by how well you got out of the blocks and if you have real improvement you'll start to see it.

There are breakdowns of several of Usain Bolt's varions 100m times. His 9.69 in Beijing (2008) and his 9.58 in Berlin (2009) and is 9.63 in London (2012) when you factor out the reaction time are pretty much the same for the first 20 meters (2.87 or 2.88s), meters 50-90 are pretty much the same too as he is at top speed for those (taking about 0.82 sec per 10m). The difference, with the exception of Beijing where he coasted the last 10 meter (accounting for .07 sec of the slower time), and where the other .03 - .04 s comes from is in the 20-40 meter range. That's the point where he isn't up to top speed yet, but is no longer in the early hard acceleration that is dominated by how well a sprinter gets out of the blocks. I've seen similar for other world class sprinters, some hit their top speed faster than Bolt, some a bit slower, most can't hold the 0.81 - 0.82 quite as long either. But the anatomy of their races are all similar. The 20-50 range tends to determine if they are going to hit a personal best or not, assuming they aren't coasting the last 10-15 meters because they know they have won/moved on/etc.

I'm not saying that there aren't shenanigans going on with Pro Day 40's. Football players are great athletes, and they get a lot of coaching on block starts even if they don't have an extensive track background. Assuming they have the same race profiles as world class sprinters isn't a safe assumption neither is assuming that 100m runners are going to run the first 40 the same way for a 40 or 100 distance.

I just think you need to look at another data set covering similar situations to be as bold in your conclusion.

All that being said, I still believe that Pro Day numbers aren't as accurate as combine numbers. I'm just not sure we can assume that improvements should be more linear in a sprint.

8
by Danny Tuccitto :: Sun, 06/02/2013 - 4:49pm

Very insightful, thanks.

If I'm understanding correctly, you're saying 20-yard/meter splits for a world-class sprinter are pretty consistent because they've mastered getting out of the blocks as efficiently as possible; and 50 to 90-yard splits are also consistent because they've reached "cruising speed" for lack of a better term. Therefore, it's variability from 20 to 50 yards that really determine their actual times on any given day.

I'm willing to buy that, but have a nagging suspicion related to the caveat you mentioned towards the end of your comment. Seems to me that, although these backs are certainly world-class athletes, they're not world-class sprinters insofar as they haven't mastered getting out of the blocks. I always thought that one of the main things that backs do in prepping for the combine 40 is to have a track coach specifically train them on that first part of the run (i.e., getting out of the blocks, or their "in-the-blocks" stance in this case). By extension, further training on their starts between the combine and pro day would seem to be to be a logical way to post a faster time without actually being faster in terms of "true speed," so to speak. In other words, I would expect to see improvements at 10 and 20 yards for people who aren't world-class sprinters that have spent the past 10 years or so mastering how to get out of the blocks efficiently.

And since you seem very knowledgeable on the topic, one other thing:

I didn't bring it up in the XP because I was hoping someone mentioned it in the comments section, and you did, so here goes. What role does reaction time play here -- not on the part of the runner, but on the part of the timer? At the combine, a human manually starts the clock, but sensors spit out automated times for 10, 20, and 40 yards. At a pro day, it's all manual. Human fingertip reaction time is somewhere between 0.15-0.20 seconds, which means -- and this is just honest curiosity on my part -- shouldn't we expect the 10, 20, and 40 times to be that much slower at pro days (man) than at the combine (machine)? My thought is yes, although we can allow for a fudge factor related to the humans training themselves to hit their stopwatches a shade before they actually "see" the back hit a given yard line. Maybe they shave off that reaction time lag from the stopwatch reading after the fact?

If they don't, then logic would dictate that the human timers at pro days are jumping the gun at all points, but jumping it the most at the end of the run. This is actually one reason I leaned towards a shenanigans explanation. Of course, like I said in the post, curious splits aren't "definitive proof" by any means.

11
by CBPodge :: Mon, 06/03/2013 - 4:11am

The idea that a human being can run 10m in 4/5ths of a second continues to terrify me.

22
by Aaron Brooks Go... :: Tue, 06/04/2013 - 10:46am

An alligator can run 10m in 3/4ths of a second.

And that's through a swamp. Imagine their track speed.

24
by Bobman :: Fri, 06/14/2013 - 4:27pm

But they can't high jump or shot put for shit, which really crushes their decathlon scores.

16
by Danny Tuccitto :: Mon, 06/03/2013 - 5:49pm

To follow up on this, I ran a t-test to see if combine 40-times differed significantly between AQ-qualifying backs that ended up running again at their pro day (i.e., the group I included in the above study) and AQ-qualifying backs that didn't (i.e., the group I excluded).

Welp, looks like there's something to the selection bias argument. The guys who didn't run at their pro days averaged 4.52 at the combine, which is .06 faster than the guys who did. The two-tailed p-value there is < .00001.

Still doesn't explain the flying-20 split though.

5
by Chase (not verified) :: Sun, 06/02/2013 - 10:05am

Interesting stuff, Danny.

Do you have weight data at the combine and pro day? I know it's been rumored that some bulk up at the combine for the weigh-ins and then try to shed some weight and impress with a fast 40 time at pro days.

I'd take a look at say, the 15 biggest jumps in the flying 20 and see if there's a weight difference.

7
by Danny Tuccitto :: Sun, 06/02/2013 - 4:18pm

Thanks, Chase.

You make an interesting suggestion, but unfortunately NFL Draft Scout just duplicates the heights/weights from the combine in their pro day measurables. Indirectly, though, we do see that the subset of backs who bench-pressed at both venues actually increased their reps from 17.8 to 20.4, which suggests that the backs are getting bulkier, not lighter (assuming, of course, it's not because pro-day observers fudge the criteria for what qualifies as a rep).

For the sake of argument, though, let's stipulate that these guys are showing up lighter at pro days, and that's why they're posting faster 40 times. That still doesn't explain why the only part of the time that's getting faster is the flying 20...at least it doesn't to my untrained eyes.

13
by Chase (not verified) :: Mon, 06/03/2013 - 9:25am

Interesting. I agree that bench press is probably a solid proxy for weight.

Another thought could be that they simply have an extra couple of months to train for the 40. But the "timer error" seems pretty likely to me. We're talking about a measurement that humans can't really process.

9
by SackSEER :: Sun, 06/02/2013 - 9:48pm

This is an interesting study, Danny. It's something I struggled with a bit with SackSEER so hopefully my perspective is helpful.

I noticed early on that edge rushers pro day times were faster than Combine times. The tricky question is why.

One of the interesting things I found at the time was that pro day times were faster, but that they were no worse at predicting NFL success than Combine times. However, at bottom, if the problem with the "shenanigans" you reference is that if shenanigans are occurring, we would expect running backs with pro day 40's to underperform their projections under your new model. If it turns out that there is no underprojection, you risk running afoul of the old adage "if it isn't broke, don't fix it."

Also, I found some pretty strong anecdotal evidence for selection bias in pro day drills that represent Combine mulligans. Cliff Avril, for example, recorded a 4.51 forty, a 32.5" vertical, a 9'9" broad jump, a 4.51 shuttle, and a 6.90 3-cone. The interesting thing about his Combine workout was that he was well above-average in every drill but the vertical and the shuttle, which were a fair amount below average. Interestingly enough, Avril worked out at his pro day and only did two drills: the vertical and the shuttle. At that point, Avril recorded a 36" vertical and a 4.31 shuttle, which were more consistent with a player who records a 4.51 forty, 9'9" broad jump, etc., than his actual Combine times.

If you're interested in looking at selection bias further I have an idea of what you could try. Because the forty tends to be colinear with the vertical and the broad, you can try running a regression to predict Combine forty times from the Combine jumps. Then, you can look at whether those who choose to run a "second" forty underperform on their forty. It wouldn't tell you anything definitive, but it would at least give you a hint to the extent of the selection bias going on.

As to the splits, I don't have a good explanation. I can say that in general I'm very skeptical of the splits. Forty yard dash times tend to be pretty noisy, and I have always thought splits to be moreso.

-----------
Sorry JPP!

10
by Danny Tuccitto :: Mon, 06/03/2013 - 1:16am

Definitely some helpful insight, Nate. I'll try the 40-as-regression-estimate angle for sure.

Re the splits, yeah, I'm still waiting (with bated breath) to hear someone come up with any plausible explanation for the absurd flying-20 improvement -- let alone a good one. I'm open to it being noise, but I can't ignore what the t-test said.

15
by FanZed :: Mon, 06/03/2013 - 5:35pm

This is a fun discussion thread. SackSEER Nate kind of addressed the natural follow up question re: time differences. They may be statistically significant, but what do they mean in functional terms? Let's even say that on the whole, a group of RBs that run the flying 20 faster by .05 seconds out-performs a slower group on the football field. How can a scout or GM apply that information in the case of any one player?

Re: why is the flying-20 disproportionately faster at pro days. I always assumed that players who were motivated to improve on their 40 times would consistently do so. It would be interesting to expand the scope of the study to other positions where speed is at a premium, i.e., to the WR and DB groups. I would expect the same bias to appear, but if it doesn't, then there's something else to think about.

Secondly, if you want to test for human error, wouldn't you look for correlative evidence in the data? For example, I would expect variance at all splits to be much higher than the null hypothesis.

Third, you'd have to test for physical variables in the athlete and at the track. Given what the track guy said above (don't remember the name - sorry), it's possible if not likely for acceleration to continue, and thus to vary, in the flying-20. It could be that the 20-40 yard split is the easiest area to improve. We are talking about .05 seconds, on average. How much more muscle mass, better arm technique, and body lean are required to improve by that much? If PEDs are involved, wouldn't their effect primarily show up in the flying-20, which is not affected as much by starting technique?

Finally, factors other than the athlete might include the slope of the track, the surface, and weather conditions. However, if these were to come into play, I would also expect a fair amount of variance from site to site. I'm not sure how you would sort this all out without a lot more info than what's available on NFL Draft Scout.

20
by SackSEER :: Mon, 06/03/2013 - 9:15pm

Thanks, Danny. I was mulling this over earlier today and have a possible theory for the splits.

As I understand it, the forty times at the Combine are electronically timed "partially." Specifically, the clock is started manually when the player leaves the block, but the ten, twenty, and forty yard dash "end" is recorded via electronic sensor.

However, I think it's fairly safe to say that pro day times are not all timed the same way (a quick Google search revealed that Manti Te'o's Notre Dame pro day forty, at least, was 100% hand timed).

Now, imagine you've been tasked with timing various players' ten's, twenty's, and forty's at a pro day. When you're trying to time someone with a stopwatch, you don't wait for them to cross the finish and then press the button. What you do is you gauge the player's speed and try to time pushing the button to stop the stopwatch so that you press the button at the point the player crosses the finish line.

This process is relatively easy when timing the end of the forty-yard dash. The player has already reached top speed (and thus a constant rate of speed), and you've had four seconds and change to watch the player and gauge his speed. So it's likely that you can click the stopwatch at almost the same time he crosses the finish.

It seems to me that it is much more difficult to hand time a ten or twenty. At that point, the player is still accelerating, so if you time your trigger finger to the player's speed you're going to be too slow on the trigger. For example, it may be that a twenty yard dash that is really 2.65 becomes 2.70. The electronic sensors at the Combine, however, don't have this problem, because they just record when tripped. It's not a big difference, obviously, but we're dealing with hundredths of a second here.

If this were the case, these players' "Flying 20" would appear to be faster than they really were. So, that player whose twenty yard dash was erroneously recorded as a 2.70 and who records a "correct" 4.65 forty-yard dash, would have a recorded Flying 20 of 1.95 seconds, but a real Flying 20 of 2.00 seconds.

As a practical matter, there's no way to prove or disprove this is or is not the case. However, there seems to be some circumstantial evidence in your data to support this hypothesis. Your forty times were 0.03 seconds faster at pro days but the Flying 20's were 0.05 seconds faster, which means that the "First 20's" were 0.02 seconds slower. Other than human error, there's really no reason to expect the First 20's to be slower, especially because there is the occasional guy who doesn't do the speed training (which is all about getting a better start) for the Combine but gets it right at the pro day.

-----------
Sorry JPP!

12
by JMM* (not verified) :: Mon, 06/03/2013 - 6:12am

Is 0.05 seconds a significant difference? If you want to argue that in this case it is roughly 16 inches yada yada, I get it. But that 16 inches is accumulated over 40 yards. 40 straight line yards, in shorts, not securing a football and free to lean into a finish line, or not.

Then there is measurement error. Nothing Han timed can capture differences that small. Electric timing starts and ends when?

A more precise number is not necessarily more accurate nor more usefull.

21
by Revenge of the NURBS (not verified) :: Tue, 06/04/2013 - 8:00am

"Nothing Han timed can capture differences that small."

I hear Deion Sanders ran the 40 yard dash in less than 12 parsecs.

25
by Bobman :: Fri, 06/14/2013 - 4:33pm

Young Jedi, not only is that a great joke, but it perpetuates an error in the screenplay about the Kessel Run, with a parsec being a unit of distance and not time as Han implied. Since this is geek heaven, I am hoping you knew that and were using it ironically--doubling the joke!

Incidentally, Dion was not hand-timed, but droid-timed. And they can be bribed.

14
by Karl Cuba :: Mon, 06/03/2013 - 2:26pm

If there is a significant speed advantage from using a sprinter's stance then why don't receivers use it during games?

We've seen some edge rushers use it, most notably Jason Babin, so why not wideouts?

I can understand that it telegraphs the direction of initial movement but I would have thought it would be useful on a third and long where you know that the receiver is probably going to be running vertically anyway. Imagine the terror Mike Wallace would inspire in a sprinter's stance, or would it make it too easy to knock him over?

The other thought that occurs reading this article is why on earth are the sprints started by the player? Offensive players could be given a 3-2-1 countdown to simulate knowing the snap count but defenders don't have that information, why remove reaction time from the test?

17
by Dean :: Mon, 06/03/2013 - 6:09pm

In the old days, WRs normally lined up in a modified three point stance. Where a lineman might drop his outside foot back so the toe alligned with the heel of the inside foot, a WR might drop either foot back 18" or so, creating in effect a modified track stance. The problem, as you alluded, is that the further back you drop the foot, and the narrower your stance, the more pigeonholed you are as to your direction. The two point stance was originally looked down upon as the provence of the lazy and the hot dogs. It really came into prominence with the rise of the WCO (go figure). It's much easier to run slants and short (1-3 steps and break) routes out of a standing position. Not coincidentally, one of the last teams to insist that their WRs use 3 point stances exclusively was the Raiders, where Branch and Wells were routinely going 12-15 yards before their break and had time to both accellerate and rise out of the stance before they had to cut. The two point stance was also better for fighting off a jam, which became less imperitive with the rules changes of '78.

18
by Karl Cuba :: Mon, 06/03/2013 - 7:26pm

Thanks for the info. I still think there has to be a place for receivers situational use of a track stance. For example, if the corner backs off then the receiver responds by dropping into a four point stance so as to eat into the cushion more quickly, knowing that he won't get jammed. (This is all dependent on coaching your receivers so that they know what they're doing and can take advantage of the stance.)

19
by Greer (not verified) :: Mon, 06/03/2013 - 7:43pm

Can you write a very simple, "Running a T-Test for Dummies" post that breaks down exactly how the probability is calculated? Feel like most people would still say "oh, well .03 is still insignificant, screw your T-test" - it'd be really helpful to have a basic understanding of the math/stats behind it so that people can see just how improbable it is. Thanks!

23
by Aaron Brooks Go... :: Tue, 06/04/2013 - 10:50am

Something else at play here, especially given the Avril conversation, is that you're just looking at a compilation of single sprints. We have no idea what the variation is for 40-time for a given player. Some guys might be all over the place in a set of multiple sprints, and we're just not seeing it -- superimposed on the bias that good results aren't repeated, but poor ones are.

26
by brpjac (not verified) :: Mon, 08/05/2013 - 12:27pm

Where did you get your data set? Would love to do some of my own tests on it. Let me know if I can get it. Thanks.