Writers of Pro Football Prospectus 2008

Most Recent FO Features

manningbrees.jpg

» Scramble for the Ball vs. DYAR Fantasy Football

Mike and Tom finally get around to a candid discussion about the oft-requested and never-implemented DYAR fantasy football league.

26 Mar 2013

Historical Draft Efficiency: An Introduction

by Danny Tuccitto

I'm going to come right out and say up front that, as an avid member of the NFL stats community, you're likely to find the content in this column very familiar. Quite literally, people have been attempting to measure value and efficiency in the NFL draft for decades. For instance, Tony Villiotti of Draftmetrics.com published a study back in 1989, updated it in 2010, and now writes about it on both his site and National Football Post.

There's also, of course, the Jimmy Johnson draft value chart, which shas produced an entire cottage industry of statheads debunking it. (Seriously, check out these search results.) One thing that's often missed, or at best glossed over, in most of these "The Jimmy Johnson chart is wrong!" posts is that the chart was based on a historical analysis of draft-day trades. Rather than summarizing the actual value of players taken at a specific pick, its purpose from the outset was to summarize the trade market. The whole point was for Dallas to use trading tendencies exhibited by the rest of the NFL to their advantage. And you know what? Not only did it work (see "Herschel Walker trade"), but a study by Cade Massey and Richard Thaler in 2005 found that it remained to that day an "extraordinarily" accurate representation of the trade market.

Two related internet posts that thoughtfully explored differing notions of "trade value" and "player value" appeared on the now-defunct Pro Football Reference (PFR) blog: In 2007, Doug Drinen did a series responding to the Massey-Thaler study, and Chase Stuart made his own performance-based chart in 2008. (He's recently done an updated version on his new Football Perspective site.) Separately, they came to the conclusion that both types of value can be right, or close to right, without ripping a hole in NFL space-time.

And that was the end of things until a couple of years later, when I did a draft efficiency series on Niners Nation, and updated it shortly before joining Football Outsiders in 2011.

Bottom line: People have spent enough time and energy trying to establish "true" draft pick values, and ultimately they've found very similar results. So, it's high time that we leave to general managers whether they want to use the Jimmy Johnson trade chart, one of the myriad "true" value charts out there, both, or neither.

At this point, I'd rather use a draft efficiency model for entertainment purposes, catering to FO readers more than NFL general managers. So what I hope to accomplish over the course of the next month is a series that, as objectively as possible, answers interesting questions related to NFL draft history. For instance, from 1970-2007, which franchise has made the most efficient draft picks? What was the most efficient draft class for any franchise? Which university has produced the most efficient picks? Which draft added the most value at quarterback? Which draft had the least-efficient first round? Which team added the most value in the 1989 draft? And so on.

Draft Efficiency Model v3.0

With all of that said, even if we're now beyond establishing a "true" value for draft picks, we still need a model worthy of answering all those interesting questions about draft history. For reasons I'll detail shortly, the main thing we need to improve is the model's generalizability. And in order to understand why that's the case, I do need to provide a quick refresher on (or introduction to) how all these types of models work.

The basic idea is that you use PFR's career approximate value (AV) metric* (which weighs more valuable seasons more heavily) to find the average career values of players taken at each draft slot, and then have a computer find the logarithmic curve that best fits the data.

At that point, you have an actual value (i.e., career AV) and an expected value (i.e., what the curve formula says career AV should have been), which means you can then calculate whatever kind of draft efficiency measure you like based on an "actual minus expected" framework. In that 2011 piece on Niners Nation, I unimaginatively called my version of such a metric "value above expectation (VAE)." Equally unimaginatively, I called a separate metric, "return on investment (ROI)," which is a term any econometricians out there should be familiar with. The formula I used for ROI was (actual AV - expected AV) / expected AV. Essentially, the difference between VAE and ROI is analogous to the difference between DYAR and DVOA. Like DYAR, VAE is a measure of total value added, whereas ROI, like DVOA, is a measure of percentage value added.

My last model differed from Stuart's in only one fundamental way. Instead of using a player's career AV, I divided career AV by the number of years he was active in the league (i.e., CarAV/Yr). The main reason for this was ease of interpretation. At its core, AV describes a player season, and we know some benchmarks for the value of that season (e.g., Pro-Bowlers are around 10, MVP candidates are closer to 20, benchwarmers are closer to zero, etc.), so I wanted to keep the numbers on that scale. If I say that Terrell Davis added 62 points worth of career AV above what was expected from the 196th pick, even I'm not sure what that means without proper context. However, if I say Davis added nine points worth of career AV per year, we can readily interpret that to mean he was better than the average 196th pick by about one Pro-Bowler. Likewise, if I say that the Cincinnati Bengals added about 15 points of career AV per year with their 1971 draft, it means they added the equivalent of one MVP candidate above expectation.

For this latest iteration, though, I've made two necessary methodological improvements, the story of which starts with the following chart:

What you're looking at is the trajectory of actual CarAV/Yr for draft classes from 1970 to 2007. Because the number of picks in a given draft has changed over the years, it's more precisely the trajectory of actual CarAV/Yr among the first 222 picks. As you can see, the average draft pick has produced more value over time: The first 222 picks in 1972 produced about 380 CarAv/Yr in the aggregate, whereas the first 222 in 2006 totaled over 600 CarAV/Yr.

Similar to many other situations in sports analytics, this upward trend is a problem for a draft efficiency application that seeks to make comparisons across eras. Without era-adjusting** CarAV/Yr, picks in later years will appear to have higher VAEs and ROIs when in reality it's just an illusion created by the above trend.

Therefore, the first improvement I made in model version 3.0 was to create an index with CarAV/Yr in 1970 equal to 100, and adjust each player's CarAV/Yr based on the index associated with his draft year. For instance, the index for 1985 was 89.9 (i.e., lower than 100), so Bruce Smith had his CarAV/Yr adjusted upward from 7.7 to 8.6. In contrast, the index for 1993 was 120.1 (i.e., higher than 100), so Drew Bledsoe had his CarAV/Yr adjusted downward from 7.4 to 6.1. Essentially, Smith becomes a more valuable pick for having been a good player in a bad draft, while Bledsoe becomes a less valuable pick for having been a good player in a great draft.

The second improvement I made was to randomly split the 38-draft sample (i.e., 1970-2007) in half, and visually check whether there was a large difference in fit between the two 19-draft models. Again, for the purposes of being able to generalize across eras, I wanted to remove the (at this point very mild) concern that VAEs and ROIs were related to differences in draft years.

In the end, I tested six different models: two half-sample models and one full-sample model using PFR's raw CarAV metric, and three analogous models using the new adjusted CarAV/Yr metric I just described. Here are the R^2 values for these six models:

Sample Career AV Adjusted Career AV Per Year
Half A 82.0% 89.9%
Half B 84.1% 88.1%
Full 91.3% 93.2%

Looking at the row related to the full models, we see excellent fit using both value metrics. As I alluded to in the intro, pretty much any efficiency model you'll find out there explains around 90% of the variance in average draft pick values (including earlier versions of my own model). The more important findings for our purposes are that (a) all three models using adjusted CarAV/Yr performed better than their raw CarAV counterparts, and (b) the three adjusted CarAV/Yr models were more similar to each other than were their raw CarAV counterparts. Essentially, with these two results, we've achieved a level of generalizability across draft years that's more to my liking.

For the sake of transparency, here's the graph and equation for the full model using adjusted CarAV/Yr:

Example Application of Model v3.0

Just to recap, we now have an equation to give us era-adjusted expected player values, and we have era-adjusted actual player values, so we can now calculate era-adjusted VAEs and ROIs to compare draft picks from 1970 to 2007 in an era-neutral way.

In the next installment of this series, I'll exhaust a few thousand words on draft efficiency by franchise, so consider this example somewhat of a tease. Here are the franchises with the eight worst era-neutral ROIs (i.e., percentage value added) over the course of their picks from 1970 to 2007:

Franchise # Picks ROI Rank
STL 312 -3.1% 25
WAS 200 -3.4% 26
SEA 232 -4.2% 27
ARI 319 -5.1% 28
ATL 301 -7.8% 29
BUF 318 -8.0% 30
KC 285 -11.4% 31
CLE (new) 74 -14.5% 32

Right out of the box, our model passes the smell test. Come back next time for all the pungent details.

*I've deliberately side-stepped the entire debate about the validity of AV as a measure of player-season value. My view in the context of the present analysis is analogous to Churchill's famous quote about democracy: It's the worst measure of value across football positions and NFL eras ... except for all the others.

**I fully realize I'm technically "year-adjusting," not "era-adjusting." Just using the latter phrase because it's accepted shorthand.

Posted by: Danny Tuccitto on 26 Mar 2013

43 comments, Last at 03 Apr 2013, 1:13am by Scott C

Comments

1
by ts (not verified) :: Tue, 03/26/2013 - 1:00pm

One thing that seems off about this fit is the first round. Essentially, your model predicts that the top end (first ten picks) should be more valuable than the picks have turned out to be. And the list of poor drafters you present here is certainly populated by teams that drafted in the first ten picks a lot. Do the results differ substantially if you allow for a model without such a discrepancy?

2
by cisforcookie (not verified) :: Tue, 03/26/2013 - 1:31pm

bad teams always get less out of their draft picks than the average team would, and good teams probably often get more, because surrounding talent makes a player look better. And this on top of the higher probability that bad teams also draft badly and have bad coaching. A given pick is thus going to have different value both depending on the team that holds it and depending on every other team (in a league where everyone else drafts randomly, my 7th round picks are worth a lot more because of the likelihood that a star will fall to me). so basically you have to make a ton of assumptions about how everyone else will draft. as long as you state your assumptions, the actual value you spit out doesn't matter that much because you are going to have to correct it anyway for every different situation.

39
by Thunderbolt of ... :: Thu, 03/28/2013 - 12:22pm

This is especially true of AV (although I agree that it's pretty much the only feasible metric for this type of project). AV evaluates the performance of an entire unit (offense or defense) then splits up that performance into shares for the players who contributed. So if an entire unit is bad, there aren't going to be many points to go around.

6
by Brenton (not verified) :: Tue, 03/26/2013 - 3:52pm

"Essentially, your model predicts that the top end (first ten picks) should be more valuable than the picks have turned out to be."

I think this could be fixed by using LOESS or another fully nonparametric smoother to generate the curve, instead of imposing a logistic functional form in advance.

42
by Tom S. (not verified) :: Sat, 03/30/2013 - 8:56am

I would like to concur with this comment and propose a potential solution to the problem. The fact that the model misses so significantly in the lower part of the round is a concern. This is, of course, because the function being used is logarithmic. As this function appears to be arbitrary (at least there's no explanation given), I see no reason not to change it.

When presented with problems like this in the past where I need an arbitrary function to describe data (usually to "smooth" it), I've resorted to a program called TableCurve (Systat Software). This program fits the data to literally thousands of built in functions and ranks them by goodness of fit. The odds are extremely good that it will find an equation that will give you a much better fit while being only a little more complex. The program can be found here:

http://www.sigmaplot.com/products/tablecurve2d/tablecurve2d.php

3
by Revenge of the NURBS (not verified) :: Tue, 03/26/2013 - 2:44pm

What jumps out at me is how low the numbers are across the board. If I'm reading this right, the average #1 overall pick historically has a CarAV/Yr of about 5.5. What does that translate to, someone like Jeff George? Seems pretty low at a draft slot where the general expectation is a Pro Bowler, at the minimum.

Things fall off pretty quickly from there, and end up at a point where a 6th or 7th round pick looks almost worthless. That matches a little more closely with my perception. Not sure what this all means, but it makes me wonder if teams over-value draft picks in pick-for-player trades.

Seeing the new Browns as the worst drafting team doesn't surprise me. KC next does surprise me, as they are not historically a doormat franchise. Also surprised to see Washington on the list. I guess the recent history of trading away picks has some merit if those picks just would have been used poorly.

5
by cisforcookie (not verified) :: Tue, 03/26/2013 - 3:40pm

5-6 car/year is better than you think. also jeff george was a pretty good player when he wasn't being a total headcase.

the nfl draft hype machine has totally broken our expectations away from reality. there are fewer elite players in every draft than you think. in a given first round, 30-50 percent of the players will make the pro bowl at least once in their careers, at least if you look at drafts from the last 20 years. and that's great and all, but we all know lots of guys who make the pro bowl based on nonsense reputation not backed up by actual talent.

Plus the pro bowl isn't _that_ selective. Approximately 1 in 5 starters make the pro bowl. So it isn't THAT impressive that your average first round pick has a maybe 2 in 5 chance of making the pro bowl.

Also all those numbers also mean that 50-70 percent end up somewhere between pretty-good-but-nothing-special and total bust. Even in the first round, getting a good starter with a pick is a victory. There's a reason teams pay so much for even moderately talented veterans in free agency.

8
by Revenge of the NURBS (not verified) :: Tue, 03/26/2013 - 4:14pm

I agree with all those points (except maybe about Jeff George being pretty good -- I had season ticket for the Colts back then). But we're not talking about just any first round pick. We're talking about the #1 overall pick. When a team can choose from ANY player in the draft, the average outcome is (approximately) Jeff George. Seems like a fairly disappointing outcome to me.

16
by cisforcookie (not verified) :: Tue, 03/26/2013 - 8:06pm

let's agree that jeff george was a disaster with the colts, but the rest of his career was legitimately good until he went to the redskins. like I said, there's a lot of factors that go into whether a player will succeed. the magic of the number 1 overall pick can't turn a terrible front office/coaching staff into a good one, and it can't prevent injuries or dogfighting. just look at the list of recent number 1s

2010 - bradford - respectable but injury problems
2009 - stafford - amazing when throwing to calvin johnson, nothing special throwing to other players, plus injury problems
2008 - jake long - fun fact. only 3 number 1 overall picks since 1990 have been named first team all pro. the other 2 are first ballot hall of famers. it looks like jake long will break that streak thanks to, again, injury problems
2007 - jamarcus - amazing for sales of purple drank, otherwise disaster
2006 - mario williams - flashes of brilliance interspersed with injuries and mediocrity
2005 - alex smith - his career is a superfund site at this point, so maybe with some andy reid intervention it can be cleaned up. otherwise, disaster
2004 - eli - easily the most successful number 1 overall pick of the past decade so far, and he was probably only the third best quarterback in his own draft class
2003 - carson palmer - started out amazing, and then his career got totally derailed by injuries.
2002 - david carr - disaster
2001 - michael vick - probably would have been better as a cornerback, disaster
2000 - courtney brown - disaster
1999 - tim couch - disaster

it looks better for a few years there in the mid/late 90s, but then karma rears her head and steve emptman, russell maryland, and jeff george make appearances.

24
by Revenge of the NURBS (not verified) :: Wed, 03/27/2013 - 8:38am

I agree with all this. That's what I find so striking though. When a team is given a pool of ~400 of the best college football players in the country, and allowed to select any one of them, this is the result. It illustrates just how inefficient (if that's the right word) teams are at utilizing draft picks. Teams spend millions of dollars on scouting, and the team with the #1 overall pick is still just as likely to pick a JaMarcus Russell as a Peyton Manning. I know I'm not breaking any news here, but it's just fascinating to me.

29
by Andy G (not verified) :: Wed, 03/27/2013 - 1:22pm

Agree with much of this...but calling Vick a "disaster" is a bit harsh. Yes, the legal issues were a major problem, but when he's been on the field, he's had successful seasons. His weighted career AV is higher than Palmer's, who I also don't think was a "disaster".

30
by RickD :: Wed, 03/27/2013 - 1:40pm

I think you overuse the word "disaster". Surely there's a difference between Alex Smith and Tim Couch/Jamarcus Russell. And calling Vick a disaster is ludicrous.

35
by cisforcookie (not verified) :: Wed, 03/27/2013 - 6:06pm

You're kidding right? Alex smith spent 5 years battling to keep his job, sometimes losing it, and generally looking inept much of the time. Then he had 600some snaps over the last 2 years where he played much better, but still not great, on a team absolutely stacked with talent and coached by a mad scientist. Call me crazy for thinking he was a bust of a number 1 overall pick. As a mid 2nd round pick? Great pickup.

As for vick, yes, he's the greatest rushing qb of all time, but he's a terrible passer, he's always hurt, he was the center of the biggest scandal in pro sports since OJ, and his one great season occurred while playing for a team other than the one who drafted him. I don't think the bengals regret drafting carson palmer as much as the falcons regret drafting michael vick.

37
by LionInAZ :: Wed, 03/27/2013 - 7:53pm

Yes, but both Smith and Vick actually played well enough to get their teams to a conference championship, whereas Couch and Russell couldn't play well enougn to keep their teams out of the basement for years. Big difference there.

40
by Thunderbolt of ... :: Thu, 03/28/2013 - 12:26pm

Joe Montana couldn't have led the early-00s Browns or the 2007-09 Raiders to a conference championship.

13
by Scott C :: Tue, 03/26/2013 - 5:40pm

That doesn't surprise me at all. The expectation is Pro Bowler, but clearly reality and expectations are not the same.

Less than half of first round picks turn out to be pro bowlers in reality, and many are outright busts.

25
by Revenge of the NURBS (not verified) :: Wed, 03/27/2013 - 8:54am

That's kind of where I was going with the bit about teams over-valuing draft picks. It's not just the fans that have inflated expectations for the draft; it seems like the teams do too.

34
by Sifter :: Wed, 03/27/2013 - 5:54pm

Definitely. It's just part of the ego of a coaching/scouting staff. They will find some value in the draft because they are smart! It's like most people down at the horse racing track - everyone fancies their own theories, thinks they are smart, and thinks they can get good value from the bookies, yet only a very small percentage of people can say they've mastered horse racing gambling.

The same happens in the draft. Various studies have shown that no one NFL team has mastered the draft. There is no clear trend of one team ALWAYS doing well. Most of the time, teams like the Patriots, Steelers, Ravens etc. get smoke blown up their rear end for being great in the draft, but people really just remember their W-L percentages rather than truly going through pick by pick and analyzing their drafting skills eg. people claim Ozzie Newsome is a god among GMs, but he still drafted Kyle Boller. He'd be above average as a drafter, but it's not like he's fleecing the other teams like a big kid stealing lunchmoney. EVERY team misses in the draft - multiple times every draft in fact.

I, like you, wonder why draft picks are over valued. I'd rather trade them for players I like. That's why I personally think Alex Smith for a 2nd is a decent deal.

36
by Jerry :: Wed, 03/27/2013 - 6:55pm

If you're defining "mastering the draft" as hitting on every pick, then, yeah, everybody's overrated. Some teams have clearly been more successful than others, though, and it looks like the point of this series is to identify those teams.

23
by Danny Tuccitto :: Wed, 03/27/2013 - 1:34am

Will go into each team in much more detail during next part of series, but 26 of KC's 38 drafts (among picks 1-222) from 1970-2007 brought negative VAE & ROI, and their best draft in that time frame was nearly 2 decades ago.

More details soon.

26
by mrh :: Wed, 03/27/2013 - 9:47am

Since the Chiefs have been around, they've had two periods were they were one of the better franchises in football and two were they've been one of the worst.

Taking all teams w-l records since 1960, converting ties to half-wins, and then normalizing all seasons to 16 games, the Chiefs history looks like this:

1960-1973 - basically the Stram era - 10.1 adjusted wins per season, 4th best total and best among the old AFL teams. Only the Browns, Packers, and Colts were better (defined as averaging more adjusted wins per season) franchises in that period.

1974-1988 - the Steadman years or the Dark Ages of the Chiefs - this includes the last year of Stram - 6.1 wins, 27th out of 28 teams. Only the Bucs were worse (a lot worse, they averaged 4.8, the Chiefs were closer to 17th than last). But this still put the Chiefs below the other bad franchises of that era like NO, ATL, and DET.

1989-2005 - the Marty/Vermeil era (and the Gunther interregnum) - KC averaged 9.5 wins, again ranked 4th among all teams. Only SF, DEN, and PIT averaged more wins in this time period.

2006 to the present - the fall of Peterson and the Pioli era - the Chiefs have only managed 5.4 wins per season in recent years, ranking 28th. Only CLE, DET, OAK and STL have been worse. Missouri has been the black hole of professional football in recent years.

Anyone who started following football in the Marty/Vermeil era might perceive the Chiefs as least a decent regular season franchise with poor playoff form. But they've been really bad in recent years despite two playoff appearances.

So I'm not surprised the Chiefs had such a terrible draft record from 1970-2007, they were a bad franchise for much of that time. And even in the good years, I think their drafting was spotty. They famously have had an abysmal record drafting QBs in particular.

31
by Danny Tuccitto :: Wed, 03/27/2013 - 4:17pm

That breakdown meshes well with the timing of their best/worst drafts. Have an interesting nugget about their 1975 draft that I'll mention in part 2.

4
by Aaron Brooks Go... :: Tue, 03/26/2013 - 3:23pm

Have you checked how this looks with straight CarAV, instead of CarAV/Yr? The latter favors superstars with short careers -- guys like Davis, or Sayers, who never had a post-peak because their career ended during the peak. But it also punishes guys like Rice or Lewis, who may have stuck around for a season or two too long, and punished their career rate value, even though their total career was far more valuable than that of a Davis.

Basically, this method says that a guy who was a freak for three years, then exploded into a mess of bloody knee fragments was more valuable than a guy who was a freak for three season, and then just a run of the mill HOFer for seven more years. Which guy would you rather have?

7
by Revenge of the NURBS (not verified) :: Tue, 03/26/2013 - 4:07pm

Good point, but using straight CarAV has the opposite problem. It says that Kerry Collins provided about the same career value as Bert Jones. There's something to be said for a higher peak, even if it comes with a shorter shelf life. Bottom line is, boiling down an entire career into a single number is going to introduce a certain amount of ambiguity, whichever way you do it.

11
by Aaron Brooks Go... :: Tue, 03/26/2013 - 5:14pm

I realize. That's why I wanted to see both. They sort of box in the problem -- an upper and lower confidence bound, in a way.

14
by Scott C :: Tue, 03/26/2013 - 5:43pm

One could take the average of the top 6 years or so to prevent 'watering down' superstars that hang on for a long time after their peak.

Alternatively, a median or 75th percentile might be better than an average.

17
by Danny Tuccitto :: Tue, 03/26/2013 - 9:20pm

Glad you brought this up because it was something I forgot to address in the piece.

Without access to the actual PFR database (i.e., not just the various finders), it's impractical (i.e., time prohibitive) to try to split out specific years of of over 12,000 guys' careers. The draft finder spits out the weighted version of Career AV, not the simple, unweighted sum version of Career AV that you see at the bottom of the stat table on player pages. So, things like using X number of years or only using the X years a player played for his team, I'd either have to go to 12,000 player pages and get the data one at a time or (slightly less impractically) do a player season search on the PFR finder, grab over 51,000 lines of season data spread across over 5,100 search pages, and then do a bunch of post-data-collection gymnastics to get everything in its final analyzing-ready form.

And for what? To try to improve on a model that only leaves 7% of the variance unexplained by pick number? That seems like a really easy cost-benefit call to me; perfect being the enemy of really, really good, basically.

43
by Scott C :: Wed, 04/03/2013 - 1:13am

If you don't have access to the raw data, it makes sense that you'd be unable to easily try out these sort of things.

9
by Danny Tuccitto :: Tue, 03/26/2013 - 4:32pm

I put the R^2s for the Career AV model into that table up there. Fit's slightly worse in full sample, but more importantly (for me) seems more sample-dependent than the models using Adjusted Career AV/Yr

12
by jimbohead :: Tue, 03/26/2013 - 5:22pm

There may be a more fundamental reason to prefer players with high peaks. A team doesn't necessarily retain all the stars it drafts for their entire careers. Most of the star's value, however, seems likely to come in the early part of their career. Preferring high CarAV/Yr over CarAV is more likely to show value to the drafting team rather than value to the NFL in general.

18
by Danny Tuccitto :: Tue, 03/26/2013 - 9:38pm

Yeah. This is another reason I like using CarAV/Yr, and another reason why I don't mind being unable to split out "X years with drafting team." (See my comment 17.)

15
by Scott C :: Tue, 03/26/2013 - 5:44pm

It would be interesting to try out median or 75th percentile year AV for a player than the mean, and see how that fits. A mean of a limited number of top years may also be interesting.

10
by rageon :: Tue, 03/26/2013 - 4:44pm

But how often is injury risk really that highly included into an evaluation of high draft picks? Obviously when you have specific information (e.g., Bowers in 2011) it's going to be looked at, but I can't imagine teams really have a good idea of a players "injury-prone-ness" coming out of college.

My assumption is that teams are really looking for talent and then hoping for the best when it comes to injuries. I don't have the numbers in front of me, but I assume Curtis Martin had more career value than Terrell Davis, whereas I also assume Davis has a far, far higher value per year. Teams are going to be looking for the Terrell Davis or the Gale Sayers, the guys who are the _better_ players.

Looking at it that way, using value per year makes more sense.

And really, how many guys like Rice or Lewis are there who hang on a year too long? Those guys stuck around because of their names. If Jerry Rice wasn't named Jerry Rice in his last season, I doubt he would have even made a roster. There can't really be many of those guys who would mess up the entire system, can there?

41
by Thunderbolt of ... :: Thu, 03/28/2013 - 12:30pm

If you're trying to address how good teams are at drafting, then going with a yearly metric will probably yield more valuable results. You can't blame a team if it drafts a stud and then his knee gets turned into hamburger.

Thinking about it, maybe the best metric would be AV/Yr over the first 5 years of a player's career.

19
by nweb (not verified) :: Tue, 03/26/2013 - 9:58pm

Hope its not the Sierra Nevada speaking (actually dogfishead). But Isn't what's totally missing here the financial perspective. In the salary cap era (admittedly not the early part of this sample period, but the go forward state) the dollars are critical as the diminishing returns of player performance were not the same as the salary savings. This changed again with the new CBA. Today the salaries are predictable enough per position that this ought to be factored in.

20
by Danny Tuccitto :: Tue, 03/26/2013 - 10:50pm

Not trying to be dismissive here, but the financial perspective would be missing if not for the fact that, as I stated in the introduction, I'm not interested in trying to establish the "true" value of a draft pick. (It's been done to death already.) Just want to have some fun over the next month with objective evaluations of draft history.

Here's a 2007 article relating the draft to salaries, though:

http://www.thesportjournal.org/article/nfl-rookie-cap-empirical-analysis...

27
by asdf (not verified) :: Wed, 03/27/2013 - 9:55am

I think the salary issue would be tied with the long term lack of success for a franchise. Under the previous cba, you are a bad team, you draft high, you miss. You are now saddled with an enormous rookie contract, that eats cap space from all other other players on your team, in effect weakening the rest of the team. Now you are a worse team than before that contract, hence you are drafting high... AGAIN in essence compounding the problem. Even if you do succeed, you are still eating up huge chunks of salary cap that could be used to create a balanced team. I still firmly believe that picking high under the old cba was a pretty bad place to be, and sort of perpetuated the mediocrity of some of those franchises.

28
by Revenge of the NURBS (not verified) :: Wed, 03/27/2013 - 12:23pm

I believe this is known as "The Millen Model of Roster Construction".

21
by MJK :: Tue, 03/26/2013 - 11:21pm

One issue I have with this approach is that the draft isn't really about finding the best possible player...it's about finding a good player who you don't have to pay much for the first five years or so of his career. After that, you can (and indeed have to) pay top dollar for an elite player, whether you were the original team that drafted him or whether you get him in free agency.

For example, team A drafts Aaron Rodgers, who sits on the bench for three years, then has a decent season, then a really good season, at which point Team A has to pay top dollar to retain him. (Or maybe Joe Flacco, or better, Drew Brees, would be a better example. After all, the team that drafted Drew Brees only got one elite season out of him, and he went on to be a superstar for a different team). Team B drafts a pretty good CB, say, an Asante Samuel type, who plays OK his rookie season, really well for the next couple of seasons (going to a couple of Pro-bowls), and then leave to go elsewhere when his contract is up. I would argue that the two teams got roughly equivalent value out of their two draft picks. Yes, an elite QB is more valuable than a good CB, but one year of an elite QB is maybe not as useful as five years of a good CB.

I understand the challenge you ran into with trying to split out AV year by year, but if you really want to look at the value a team gets from the draft, it should be cut off after a player's rookie contract (or after he leaves the team that drafted him).

22
by Danny Tuccitto :: Wed, 03/27/2013 - 1:11am

"I understand the challenge you ran into with trying to split out AV year by year, but if you really want to look at the value a team gets from the draft, it should be cut off after a player's rookie contract (or after he leaves the team that drafted him)."

I don't disagree. Nevertheless, I think you'll find as this series continues that, even with the seeming limitations of the approach, the results clearly pass the smell test. If the approach I took ended up spitting out results that were clearly wrong (e.g. the top 10-most efficient QB picks of 1970-2007 included some random craptastic player), this series wouldn't have ever gotten past the idea stage.

38
by MJK :: Wed, 03/27/2013 - 11:15pm

That probably will end up being true, mainly because a player that is terrible in his first five years (or for a significant part of them) probably will not suddenly turn good later on or with a "change of scenery", and a player that is actually good will rarely be let go by a team before they manage to get him on the field for them (unless their personnel eval is terrible). There will be excaptions, of course, but you're looking at a large enough sample set that those exceptions will likely be washed out.

So your approach really seems like it could have some very good value and fun insight. But it still would be nice to be able to break it out by years with a team on rookie deals...

32
by ClemsonMatt (not verified) :: Wed, 03/27/2013 - 4:19pm

Did you consider normalizing each draft's CarAV/Yr. using the expected CarAV/Yr from the regression? It looks like some drafts just sucked, and normalizing based on the draft's value masks that suckitude.

It seems some franchises may have benefitted from giving up draft picks in down years to stockpile for better years, and any gains would be masked. Or, possibly, it would mask that there weren't any gains.

33
by bravehoptoad :: Wed, 03/27/2013 - 4:25pm

Can't wait to see the next article, Danny.