Historical Draft Efficiency: College Rankings
by Danny Tuccitto
Based on the comments sections the past few weeks, I'm glad to see the Football Outsiders readership enjoying this series. As I stated in the first installment, my aim here has been to try to put numbers on things the best I can, and thereby drum up an entertaining discussion. With franchise rankings and an objective take on the best and worst drafts out of the way, we can now turn to more granular draft splits based on value above expectation (VAE) and return on investment (ROI). Today, I'll stoke the fire that is alumni passion.
As a reminder, like FO's distinction between DYAR and DVOA, VAE is a measure of total value relative to baseline (in this case, the expected career value of a given pick), whereas ROI is a measure of percentage value (or value per pick, given expectations). Interestingly enough, within the same college, there's significant divergence between the two stats, possibly because of the NFL's almost-blind ambition regarding players from certain schools.
Before showing the stats table, let me first discuss a few colleges that didn't make the cut. Easily the most surprising based on its sheer number of picks is Nebraska. From 1970 to 2007, 181 Cornhuskers were selected among the first 222 in a given year. In the aggregate, those picks yielded -52.3 VAE and -13.6% ROI. Among the 70 schools with at least one draft pick per year, Nebraska's VAE ranks dead last, and their ROI ranks 64th. Despite having the fifth-highest expected Career AV per Year (384.5), only 38.1 percent of Cornhuskers were positive-VAE picks (average among this group was 40.7 percent). Basically, for every Roger Craig (+4.8 VAE, +181.4% ROI) and Ahman Green (+3.9, +185.3%), there are twice as many Bruce Pickens (-5.0, -84.2%) and Trev Alberts (-4.0, -75.0%).
Though not as much of a disaster as Nebraska, draft picks from USC haven't fared much better in the aggregate. From 1970 to 2007, Trojans were picked the most (200 in the top 222 from 1970 to 2007) and had the highest expected CarAV/Yr (491.6), but were essentially a wash in terms of relative value (+1.2 VAE, +0.2% ROI). Although it's true that USC had an above-average percentage of positive-VAE picks (43.5 percent), for every Anthony Muñoz (+4.3, +72.1%), there was a Matt Leinart (-3.2, -72.3%). For every Marcus Allen (+2.5, +55.9%) and Hoby Brenner (+1.2, +54.7%), there was one Ricky Bell (-3.5, -48.9%).
Nebraska and USC aren't even some kind of big-school anomaly. Among the top 10 colleges in terms of total selections, seven more didn't make the table with respect to top VAEs and ROIs. Here they are, in order of selections: Penn State (180 picks, -6.9 VAE, -1.7% ROI), Ohio State (177, -11.7%, -2.8%), Notre Dame (168, -11.8, -3.2%), Michigan (162, -2.4, -0.7%), Oklahoma (157, -33.2, -9.7%), Tennessee (153, -16.1, -4.7%), and Florida (144, +4.1, +1.2%).
Now you know some of the familiar schools that weren't among the best at producing draft value from 1970 to 2007. Luckily (for ease of presentation, at least), the top 21 colleges in terms of VAE also ranked in the top 21 of ROI. So, below is a table of the top 20-plus-one (min. 38 picks):
|San Diego State||80||+12.6||14||+8.0%||14|
Miami is the only college among the top 10 in total picks to also be among the top 20 in VAE and ROI. It should come as no surprise that the most valuable Hurricanes come from the halcyon days of 1995-2001. Among picks that produced +3.0 VAE are Ed Reed (+5.2 VAE, +150.1% ROI), Frank Gore (+4.7, +204.4%), Reggie Wayne (+4.6, +143.4%), Ray Lewis (+4.6, +136.1%), Clinton Portis (+4.4, +173.9%), Warren Sapp (+4.4, +102.3%), Edgerrin James (+3.9, +88.3%), Vince Wilfork (+3.3, +91.2%), and Jon Beason (+3.0, +88.3%).
[ad placeholder 3]
Lest we forget, however, those that came before them. Suprisingly, the best pick from the University of Miami in terms of VAE was way back in 1973, when the Minnesota Vikings selected Chuck Foreman 12th overall (+5.6 VAE, +129.9% ROI). In his first five years, Foreman made five Pro Bowls, was voted first-team All-Pro once, and averaged an absurd 13.8 Adjusted CarAV/Yr; he also spearheaded three Vikings Super Bowl appearances.
In contrast to Miami's slew of household names, LSU boasts less-heralded picks that produced at least +3.0 VAE. I mentioned the impeccably named Tommy Casanova last week (+6.1 VAE, +189.3% ROI), but first on the Tigers' list is a running back who had one of the best three-year stretches of the salary cap era. Domanick Williams (nee Davis), selected 101st in 2003 by the Houston Texans, gained at least 1,300 yards from scrimmage in each of his three seasons, good for 8.5 Adjusted CarAV/Yr. Another blast from the past is wide receiver Eric Martin (+6.3, +562.5%), who the New Orleans Saints took at No. 179 in 1985. To this day, and despite the team's pass offense exploits over the past two decades, Martin still holds the franchise record for receptions and receiving yards.
To further feed the narrative of Pennsylvania football, the Pittsburgh Panthers rank in the top six of both VAE and ROI, and have produced the highest percentage of positive-VAE picks (53.0 percent). In the top 222 picks of drafts between 1970 and 2007, they boast six Hall of Famers, with Darrelle Revis poised to make it seven if he successfully returns from his 2012 ACL tear. Among other Panthers that produced at least +3.0 VAE were Giants defensive tackle Keith Hamilton (+4.4 VAE, +242.9% ROI), Oilers cornerback J.C. Wilson (+3.3, +354.3%), and 49ers cornerback Carlton Williamson (+3.1, +133.5%). If you're wondering about Larry Fitzgerald, he's produced +0.5 VAE and +7.9% ROI thus far in his career.
The only other college with an aggregate VAE above +30.0 is Georgia. But unlike LSU, Miami, and Pittsburgh, the Bulldogs don't have a single Hall of Famer to show for their relative draft value -- at least not yet. I'll get into the topic more when I present the best and worst individual picks since the merger, but for now I'll simply point out that Jake Scott produced 10.2 Adjusted CarAV/Yr as the 159th pick in 1970 and Terrell Davis produced 9.9 Adjusted CarAV/Yr as the 196th pick in 1995. Davis has several more bites at the Hall of Fame apple, but Scott's omission is a total mystery to me after having run the stats for this series.
Finally, we come to the "Luke, I am your father!" moment, a point at which we must talk about the thing you least expected. Tiny Jackson State, a member of the Southwestern Athletic Conference, boasts less than one-fifth as many students as my alma mater, the University of Florida. Yet Jackson State shows up in the top 21 while said alma mater does not. With a mere 39 picks, these Tigers have the fifth-highest percentage of positive-VAE picks (48.7 percent), which includes two Hall of Famers: Walter Payton (+4.0 VAE, +72.7% ROI) and Jackie Slater (+2.1, +107.5%).
Interestingly enough, neither Payton nor Slater are among Jackson State's best three draft selections in terms of VAE. Leon Gray (+4.9 VAE, +234.4% ROI), taken 78th overall by the New England Patriots in 1973, was an eight-year starter at left tackle for the Patriots and Oilers, made four Pro Bowls, and was voted first-team All-Pro three times. Rickey Young (+4.3, +354.9%) was San Diego's pick at No. 164 of the 1975 draft, started at either fullback or halfback during his first seven seasons for the Chargers and Vikings, and led the NFL in receptions in 1978.
And then there's Jimmy Smith. Chase Stuart has already made a convincing argument for why Smith is one of the best wide receivers in NFL history, so feel free to click that link. In terms of FO stats, the 36th pick of the 1992 draft had the following DYAR rankings from 1996 to 2001: 4th, 10th, 13th, 1st, 11th , and 7th. After that bit of information, along with his draft value stats (+4.1 VAE, +137.6% ROI), I can't say I disagree with Chase all that much.
I've detailed a mere one-quarter of the VAE/ROI table, so feel free to ask questions and offer opinions in the comments section. Next week, I'll be finishing off this series with a look at the best individual picks from 1970 to 2007. Here's a teaser: Tom Brady is not No. 1.
UPDATE: Comments requested it, so I set up this link, which shows the VAE and ROI stats for all 70 qualifying schools.
57 comments, Last at 27 Apr 2013, 7:23pm
#1 by trevor (not verified) // Apr 16, 2013 - 12:31pm
why is Tom Brady not #1? Seems impossible.
#3 by clark // Apr 16, 2013 - 12:58pm
My money is on Richard Dent
#43 by D // Apr 16, 2013 - 5:41pm
Problem with Dent is that he was only selected 4 slots lower than Brady (203rd vs. 199th) and is well behind Brady is wCAV, so year adjustments would have to be huge to push him ahead of Brady.
#5 by Danny Tuccitto // Apr 16, 2013 - 1:03pm
I'll certainly be going into that conundrum next week.
#12 by Aaron Brooks G… // Apr 16, 2013 - 1:39pm
Jake Scott, Keenan McCardell, Karl Mecklenburg, and Terrell Davis.
#34 by Theo // Apr 16, 2013 - 3:12pm
Shannon Sharpe was a 7th round pick and exceeded expectations; he held every record of the TE position when he retired.
#29 by IrishBarrister // Apr 16, 2013 - 2:26pm
While not impossible, any argument that a 6th round QB with Brady's level of historical efficiency (DVOA, DYAR, future HOF, etc.) isn't #1 must be backed up with some strong evidence to the contrary. Danny's been pretty sound in his logic historically, so I think the best approach is to wait and see.
#30 by Danny Tuccitto // Apr 16, 2013 - 2:33pm
Don't want to get into the issue here, but one general, purposefully vague point I'll make about it is that our memories tend to suffer from recency bias and the availability heuristic. The fact that it took several comments in last week's piece before someone finally guessed Terrell Davis to be among the most valuable picks shows that even the late 1990s are practically out-of-sight, out-of-mind already.
Of course, that's not the only reason (and probably not even the best reason), but it's an underappreciated one nonetheless.
#31 by Aaron Brooks G… // Apr 16, 2013 - 2:58pm
Part of it was how short TD's career was, relatively speaking, and a human tendency to note magnitude more easily than rate.
#33 by Danny Tuccitto // Apr 16, 2013 - 3:06pm
Yep. That too for sure.
#36 by Led // Apr 16, 2013 - 3:52pm
#40 by D // Apr 16, 2013 - 5:23pm
Moon went undrafted.
#2 by Israel P. (not verified) // Apr 16, 2013 - 12:36pm
"the Bulldogs don't have a single Hall of Famer to show for their relative draft value -- at least not yet."
Is that last bit a reference to Hines Ward?
#14 by Dean // Apr 16, 2013 - 1:42pm
It certainly shouldn't be.
AJ Green, Geno Adkins, and Matthew Stafford have long careers ahead of them, any of which could conceivably reach Hall of Fame level.
On the other hand, Wards career is done and he did not.
#15 by Aaron Brooks G… // Apr 16, 2013 - 1:48pm
Frank Tarkenton isn't a HOFer? Or are we just talking within this time window?
Because Bailey almost certainly will be, and Jake Scott and Terrell Davis have a shot.
#17 by Danny Tuccitto // Apr 16, 2013 - 1:52pm
Yeah, just talking about this time window.
Also, the "not yet" comment was about TD, not Hines Ward. Unfortunately, a little late for Scott I think.
#4 by Dice // Apr 16, 2013 - 12:58pm
I expected most of the names on the list(except Jackson State). I'm racking my brain about Maryland, though. Randy White and Boomer, OK, sure. But who else? Randy Starks and Eric Barton? Davis and Merriman were drafted pretty high, with Davis only coming on the last few seasons and Merriman out of the league.
#6 by Danny Tuccitto // Apr 16, 2013 - 1:08pm
Here's Maryland's top 10 in VAE:
Boomer Esiason (+4.7 VAE, +160.2%)
Walter White (+4.0, +190.1%)
Donald Brown (+3.3, +217.1%)
Frank Wycheck (+3.2, +259.2%)
Kris Jenkins (+2.8, +100.1%)
Eric Barton (+2.5, +184.6%)
Randy Starks (+2.2, +100.3%)
Randy White (+1.9, +30.1%)
Neil O'Donnell (+1.8, +71.2%)
Ferrell Edmunds (+1.6, +74.2%)
#53 by grady graddy (not verified) // Apr 17, 2013 - 1:02pm
Shouldn't that be J.B. Brown, not Donald?
#7 by ClemsonMatt (not verified) // Apr 16, 2013 - 1:12pm
8 ACC schools(9 counting Louisville) to 5 SEC schools.
And of the 7 in the top ten for # of picks that disn't make the cut, isn't that also the over rated in the preseason polls list?
#8 by J Oliver (not verified) // Apr 16, 2013 - 1:20pm
Where does Texas rank?
#9 by Danny Tuccitto // Apr 16, 2013 - 1:22pm
Out of the 70 colleges with at least 38 picks, Texas is 50th in VAE (-12.3) and 42nd in ROI (-4.2%).
#10 by Joseph // Apr 16, 2013 - 1:30pm
Danny, I'll ask again, since I don't believe my question was answered in the comments on the first installment of this series.
When calculating VAE, is the draft pick being studied taken out of the average before calculating his value over said average? In other words, is player X compared to the average of the other 36, or the average of the 37? For players within ~1 standard deviation of the mean, this probably does not change anything (it might move them up or down slightly overall in both measures, but probably does not matter). However, for the huge outliers--Tom Brady and JaMarcus Russell, to name one at each end--this would make a large difference. Brady surely inflates the average value of his pick--thus making his VAE compared to the average of all 37 lower than if his VAE was compared to the average of the other 36. Russell deflates the average of the #1's overall, making him seem "less" of a bust.
Here's an idea that would make you have to redo all the numbers, so I don't expect it done any time soon. Add the number of yrs contributing to Career AV/yr to that CAV/yr. This will give more value to guys who averaged 3 AV for 12 yrs vs those who averaged 3 AV for 5 yrs. (Currently, if I'm understanding your calculations, these two players would be rated approx. the same, depending on their year drafted and pick # overall. I'm basically thinking of an "average" 3rd round pick.)
This would also give kickers, punters, & long-snappers a value to go by, since they don't receive AV like the others. Jason Hanson has contributed much more value than Mike Nugent, for example. This could help reflect that.
#23 by Danny Tuccitto // Apr 16, 2013 - 2:09pm
Hmmm...sorry for not replying the first time (thought I caught everyone's questions). Yeah, I don't remove the player when calculating VAE and ROI. Know it's not the best excuse in the world, but doing that would take way too much time (as far as I know), and probably wouldn't be an efficient use of that time (i.e., it would be a ton of work, and yet only improve the model at the margins).
Re your second point, Career AV itself takes longevity into account by weighting a player's best years (usually at the end of his career) far less than his worst years. And yet, as I showed in the first piece, using Career AV instead of Career AV/Yr produces a slightly less desirable fit and, more importantly, seems a lot more sample-dependent.
The special teams issue, on the other hand, is something I'll definitely be fixing next time I give this project a fresh look.
#44 by Joseph // Apr 16, 2013 - 6:11pm
My comment was at the end of the first piece (like #70 or so).
Re: my 2nd idea, that's why I think you should use Career AV/Yr PLUS # of years. For an extreme example, let's take Robert Edwards, RB, NE. One good year, one horrific injury at the Pro Bowl, done in the NFL. He then has no fringe years to ever be discounted--his one good year gets counted at 100% in both systems. Someone like Terrell Davis, who maintained great play for 8/9 yrs, also gets helped because he had no "decline" years, plodding along at replacement level vs. previous years of star level. However, even though LdT's years with the Jets aren't weighted much in Career AV nor Career AV/yr, they are still less than his HOF-level years with the Chargers, and thus pull both numbers down slightly. If I understand AV math well, after about 10 yrs, a player doesn't add much to his Career AV unless he's still great, because these years will be counted at 50% or less. But anyone still playing good after 10 years is pretty great to start with, because he should be entering his decline phase (e.g. Ed Reed not re-signed by Ravens).
This brings me to my nitpick with AV (which I voiced in the PFR blog several years ago). The best years should be weighted 100-98-96-94-92-90, then falling off 5%/yr. This makes the best stars burn brighter, IMO. This is where I think adding # of years to Career AV/yr would help.
Re: my first point--really I think it only affects the guys whose VAE is outside of 1 or 2 SDs from the VAE mean. And let's face it--those all-time bests/worsts are the guys we care about anyway. You could probably do it for just the 100 best and worst guys, adjust them up or down as necessary, and the rest of overall big numbers (per team, per college, etc) wouldn't change a bit.
#27 by fb29 // Apr 16, 2013 - 2:20pm
I am also interested in an answer to the first question. Especially since next week we learn why Tom Brady is not #1.
#11 by CK (not verified) // Apr 16, 2013 - 1:31pm
LSU tops out VAE even with the recent string of under-performance (Glenn Dorsey, Tyson Jackson, Purple Drank, even Addai and Landry have under-performed expectations,) that's pretty impressive.
#13 by Aaron Brooks G… // Apr 16, 2013 - 1:41pm
So is Southern Miss just Stubbleface and 39 JAGs?
#19 by Danny Tuccitto // Apr 16, 2013 - 1:59pm
Heh. Not really. Here's their top 10 in VAE:
Adalius Thomas (+4.6 VAE, +427.0% ROI)
Brett Favre (+4.2, +134.2%)
Fred Cook (+3.8, +120.2%)
Michael Jackson (ABC, 123)
Hanford Dixon (+3.5, +99.0%)
Sammy Winder (+3.4, +226.7%)
Michael Boley (+3.0, +239.0%)
Louis Lipps (+2.8, +79.8%)
Patrick Surtain (+2.7, +97.2%)
Amos Fowler (+2.2, +139.1%)
#54 by Scott P. (not verified) // Apr 17, 2013 - 3:28pm
"Michael Jackson (ABC, 123)"
Ha ha ha.
#16 by wr (not verified) // Apr 16, 2013 - 1:49pm
Alabama is another one I'm curious about. I should say I'm not terribly
surprised they're not in the top 20.
#18 by po822000 // Apr 16, 2013 - 1:55pm
Glad to see Purdue is there. Of course, the college that produced the most passing yards (in aggregate) in NFL history should hopefully make the top-10.
#39 by InTheBoilerRoom // Apr 16, 2013 - 5:22pm
Agreed. I was expecting to see the Boilermakers rate highly, but not necessarily top 10 in both metrics. It passes the eyeball test, though. Their players don't get drafted based on school reputation, so they have to have proven to be good against generally stronger competition (since Michigan, Ohio St, ND can't play themselves) to get drafted. That may be why a decent number of historically weaker and/or smaller schools rate highly here.
Also, as you alluded to, Purdue certainly benefits greatly from Brees (and other QBs, although I believe Griese was drafted before the cutoff of this study), but I also think there aren't too many 1st or 2nd round picks from Purdue that have been busts, either. Add in the number of later round picks that have exceeded expectations as solid to great contributors over their careers, and their ranking makes perfect sense.
It makes me wonder why this is. There are other schools of similar size and historic quality to Purdue (and some of the other top 10 teams) that have not produced as well. Hall of Fame quality players alone can't prop up a team that has had 91 picks, especially since only one current HOFer from Purdue qualifies (Rod Woodson). Brees will be in the HOF, so that only really makes two HOFers drafted since 1970. The school seems to produce a lot of players that meet or exceed expectation without being stars, but why would that be unique to any particular school? I'd be interested in seeing the percentage of drafter players with positive VAE and ROI for each school.
#20 by The Only Cleo … (not verified) // Apr 16, 2013 - 2:05pm
Could we see a link to the whole table of schools with over 38 picks?
#37 by Danny Tuccitto // Apr 16, 2013 - 4:03pm
Believe this should work (click to zoom):
#51 by Aaron Brooks G… // Apr 17, 2013 - 9:37am
Looking at this list, I suspect there's a mild punishment effect for college teams that were dominant in the 1960s, but fell off some in the 1970s. They get the institutional push that hurts USC, Alabama, U-M, etc, but lose credit for their HOFers who were drafted in the 1960s.
#21 by Pottsville Mar… // Apr 16, 2013 - 2:05pm
How can Reggie Wayne be at +4.6%, but Dan Marino (drafted at almost exactly the same slot) be under +3.0%? Or am I misreading the paragraph about Pitt players?
#24 by Danny Tuccitto // Apr 16, 2013 - 2:12pm
Misread: I omitted specific mention of Marino because he's one of their Hall of Famers, not because he was below +3.0 VAE. His stats were +4.1 VAE and +124.4% ROI.
#35 by Pottsville Mar… // Apr 16, 2013 - 3:48pm
Thanks. Still seems strange that Wayne is higher than Marino.
#22 by CyrusN (not verified) // Apr 16, 2013 - 2:07pm
Where does Cincinnati fall among these rankings? I know we have had a lot of solid players come out the last few years.
#25 by Danny Tuccitto // Apr 16, 2013 - 2:16pm
Cincinnati only had 35 picks, so they didn't qualify for the rankings. Their stats, though, are +0.8 VAE and +1.6% ROI. Also, keep in mind that this only goes from 1970 to 2007, so post-2007 Bearcats like Barwin, Nakamura, and Kelce aren't included.
#26 by Dan in Philly (not verified) // Apr 16, 2013 - 2:20pm
Another way of putting this is this is a list of teams which underacheived in college given the talent they had to work with. Under the theory that playing on a winning team (Nebraska, Alabama) will overinflate your draft status, it would be prefectly expected that schools with great coaching and winning teams and great depth and walk on programs would outperform their players' talent.
#28 by fb29 // Apr 16, 2013 - 2:23pm
Zing! (likely from a Florida grad)
#32 by Aaron Brooks G… // Apr 16, 2013 - 3:01pm
That's kind of my takeaway, too. I suspect AdjAV also somewhat mishandles linemen, which works against Nebraska's strengths (that and their abysmal QB record in the pros). Probably Wisconsin, too.
It does make Miami's showing all the more impressive, though.
#38 by Danny Tuccitto // Apr 16, 2013 - 4:07pm
Everyone, it's kind of buried in the middle of the comments, so wanted to alert you to the fact that I linked up the full table if you want to see stats for the 49 colleges not included in the table up top.
See comment 38.
#41 by thok // Apr 16, 2013 - 5:23pm
You probably should edit the article (or get Aaron Schatz to edit it) to include the link at the end of the article.
Also, as a Cal fan, I'm not sure whether to be bummer or amused that we just missed both top 20's.
#42 by Danny Tuccitto // Apr 16, 2013 - 5:39pm
#46 by thok // Apr 16, 2013 - 6:58pm
#45 by Jim C. (not verified) // Apr 16, 2013 - 6:13pm
As a UVA grad, I am absolutely floored to see the Cavaliers ranked number 5.
#47 by CORNFED (not verified) // Apr 16, 2013 - 7:05pm
HOW DO THEY FACTOR IN GUYS LIKE PRIEST HOLMES, ETC (UNDRAFTED)
#48 by Theo // Apr 16, 2013 - 7:20pm
Maybe 'signing street free agents' is the next article.
#49 by Zerilan (not verified) // Apr 17, 2013 - 1:10am
Does WVU even have anyone of notable positive value?
#52 by Aaron Brooks G… // Apr 17, 2013 - 10:20am
I'd hazard Bulger and Talley.
#50 by FMJ3 (not verified) // Apr 17, 2013 - 8:06am
Guess some of these schools should start looking at their coaching staffs to figure out why all this "talent" isn't being used to produce more BSC trophies at their schools ;-)
#55 by dbostedo // Apr 17, 2013 - 10:06pm
But it doesn't necessarily have anything to do with how good the team is, or how much talent the team has. It really only has to do with whether or not players from a particular school perform in the NFL to the level they are drafted at.
Or maybe the better way to look at it is that this is simply rating whether or not players from a particular team are drafted earlier or later than they should be, based on their NFL performance.
It's possible that a bad/mediocre team could be that way because of poor coaching, and the players are revealed to be better than expected in the NFL.
It's also possible that a bad college team has a few outstanding players, and is well coached, but doesn't have enough good players to win in college. The outstanding players then could perhaps be drafted too low due to the bad team hiding their true ability.
It's also possible that there's a ton of noise and outliers and one-offs in these stats, and they aren't all that meaningful.
#56 by crw78 // Apr 18, 2013 - 3:35pm
I think the most interesting thing about this table (and sorry if anyone posted something similar already, I haven't read through them) is that during this time period San Diego State had more draft picks than Virginia, Louisville, Georgia Tech, Oklahoma State, Maryland, Boston College, Syracuse, and Missouri.
#57 by Ryan Quincy (not verified) // Apr 27, 2013 - 7:23pm
Tiny Jackson State, a member of the Southwestern Athletic Conference, boasts less than one-fifth as many students as my alma mater, the University of Florida. –ah easy to see why Florida State wasn’t included now.