Feeling a Draft

Guest Column by Jason McKinley

Spring is here and a young man's fancy turns to Mel Kiper, Jr. As we look forward to the upcoming draft, it's always fun to look back at past drafts and see what information we can divine. What teams drafted well or drafted poorly? How did the good or bad drafts affect each franchise? How can we objectively measure success in the draft?

Evaluating a past baseball draft would be straightforward these days: Look at win shares by draft slot, or perhaps peruse win probability added. What does football have? DVOA is a wonderful thing, but it tells us little to nothing about a back-up defensive tackle who rotates in on short-yardage situations. Football might not have the extensive individual statistics that baseball has, but it does have one thing baseball does not: It has a salary cap.

This essay describes a period of four drafts, 2000-2003. We will set out to show who the biggest steals and the biggest busts were, and which teams drafted the best and worst over that period. We will use draft position and salary cap information to achieve this goal. We will then see how good or bad draft fortune affected teams on the field (if at all). The salary information used for this study came mostly from the USA Today salary database [1].

We calculated the average annual percentage of the entire NFL salary expenditures that each player accumulated. These totals formed a very nice logarithmic curve when graphed against draft position (Figure 1). For those of you who care about this sort of thing, the R-squared for this curve is approximately 0.60.

We used the equation of this line to determine the difference between the money that a player should have earned based on his draft position and the money he actually did earn. The basic idea here is that, while a player's first contract is generally determined by his draft position, good players will sign extensions and second contracts that better reflect their performance on the field. Players that exceeded expectations by at least one standard deviation are called "steals" in this analysis, while those who fell at least one short were "busts."

Obviously, this method works only if one believes that NFL executives operate a mostly efficient market when giving out contracts. The more fallible you think the general managers are, the less credible this study becomes. However, the initial results coincide with common sense. This methodology determines that the biggest draft steal in this period was Tom Brady. The biggest bust was Charles Rogers.

"Hold on!" you say. "I don't need some nerdy math to tell me that! I already heard Sean Salisbury say that Tom Brady has the guts of a burglar and that Charles Rogers is not someone he would trust to take with him in an alley!" That is true, of course, but the point is that the math results reflect the obvious, so perhaps it will do a credible, objective job in determining more subtle cases.

Let's start with first-round "steals" from 2000-2003.

The first-round steal is an important concept to introduce. These high picks worked out significantly beyond reasonable expectations. The most interesting example is Carson Palmer. He was the No. 1 overall pick in 2003, but he has played at a level that has earned him bonuses and contract restructurings that vaulted him well ahead of where a No. 1 draft pick normally lands. He has outperformed his draft position, because No. 1 picks rarely give a team the kind of production fans expect. Fans often expect All-Pro performance from high draft picks, but saviors come along too rarely to make that prospect realistic.

Other first-day steals break down like this:

Just look at all those fine wide receivers drafted in the second and third rounds. Come on Mr. Millen, just look. Lots of good value there. Just something to consider.

Of course, when most people talk about "steals," they really are looking at players drafted in the later rounds. To that end, here are the biggest day two steals from this four-year draft period:

Brady laps the field in terms of being a steal. His earnings place him almost eight standard deviations beyond his expected earnings based on his overall draft slot. Raheem Brock deserves a bit of an asterisk, as the Eagles made a smart draft choice and then didn't have room for him on the roster; the Colts signed him as a free agent right afterwards.

During the four years studied, only the Cowboys and the Raiders failed to acquire any of the 129 "steals" across all rounds.

Here are the 20 worst busts according to this method, ordered from the legendary worst (Charles Rogers) to the still pretty bad (Bruce Nelson):

All of the players on that list are first-day picks. Drafting is such a tricky, nuanced endeavor that it is difficult to label any second-day pick as a bust. The expected contribution for these players is already so low that to underperform, a player would have to get cut in training camp -- and then injure a star player while cleaning out his locker. Therefore, even though the system recommends Dennis Weathersby and Nate Dwyer as minor busts, they are somewhat exempt thanks to their second-day draft status.

Busts come in all shapes and sizes (Vive le difference!). Some simply lack NFL ability, some suffer unfortunate injuries, and some lack the necessary drive to make it. From 2000-2003, 102 players are identified as significant busts, and the only team that did not draft one is the New York Jets.

Yes, you read that correctly. The worst-drafting franchise in NFL history [2] actually drafted pretty well at the start of the 21st century. In fact, when we look at the total value over by this method, the Jets drafted better than any other NFL franchise during the years 2000-2003.

To figure out the best and worst teams, we used tallied up the total value over (or below) expected salary for all picks by each team, then look at the teams which were a standard deviation above the mean (five "good" drafting teams) and the teams a standard deviation below (six "bad" drafting teams).

After the Jets, the top five teams include the Patriots, Panthers, Ravens, and Bengals. The best single-year draft by any franchise is the 2000 draft where the Jets had four first-round picks. What's important is not that the Jets had these picks, but that they worked out. Chad Pennington, Shaun Ellis and John Abraham have all surpassed the average player taken in their respective draft positions, and even Anthony Becht -- the worst Jets pick of 2000 -- isn't that bad. The Jets also got Laveranues Coles in the third round that year. For the four years total, the Jets top our list not with spectacular picks, but by avoiding major busts and gathering a nice number of solid players. The Patriots are next after striking absolute gold with picks both low (Tom Brady) and high (Richard Seymour). Add that to guys like Asante Samuel and Matt Light, and they more than made up for guys like Brock Williams and Bethel Johnson.

The Patriots and the Ravens are the only teams that had overall positive value from each draft class in all four years. The Patriots were having an abysmal draft in 2000 until they nabbed Tom Brady. In fact, without that pick, the Patriots' 2000 draft would rank as the 15th worst of this era. With him, that draft ranks as the 11th best.

The six worst drafting teams were the Raiders, Cowboys, Chiefs, Browns, Rams, and Vikings, with the Raiders at the bottom. The worst single-year draft belongs to the Cardinals in 2002. They drafted eight players, one in each round and with an extra pick in the third. Six years later, only one player is still in the league: Josh McCown. The Raiders trudged their way to the bottom of the four-year list partly due to their complete lack of steals. Their best value pick was Shane Lechler. He is a punter taken in the fifth round. He has been an All Pro ... but he is a punter, and he's their best pick in those four years based on expected value. Meanwhile, they suffered five players that fell into the "bust" category. The Cowboys had a very similar profile but their busts were not as bad and they had more players that at least were slightly above expectations than the Raiders.

The Rams, Chiefs, and Browns were the anti-Patriots during this time. Those teams combined for zero draft classes that out-earned their expectations.

How did the good drafting teams identified above fare in terms of wins and losses? Did the good drafts affect positive change? In the four years prior to the four-year draft window, those teams averaged a 7-9 season. During the four years of superior drafting, they averaged an 8-8 record. In the four years after they averaged a 9-7 record. How did the poor drafting teams fare? They went from 8-8 to 8-8 to 7-9.

Many conclusions are possible from this limited data. One suggestion is that good drafting over a period of years can marginally improve a team in both the short and long term while poor drafting may not hurt a team at first but will eventually erode them. Another possible conclusion is that the numbers are not dramatic enough to conclude anything.

We use DVOA to try and further elucidate some meaning from the data. Both the five "good" and the six "bad" drafting teams showed little change between their 1996-1999 periods and their 2000-2003 periods, but experienced a very noticeable shift afterwards:

The good drafting teams enjoyed a ten-point boost in DVOA, and the poor drafters saw an eight-point swing in the wrong direction following the era of good and bad drafting. To give an idea what these numbers mean, consider that since 1995, 50 teams have made the postseason with a DVOA of 8.4% or less, but only six of those had a DVOA of less than -7.7%. These numbers seem to indicate that the draft is more of a long-term solution or detriment. Again, other conclusions may be equally reasonable.

We also looked at draft picks by position to see if there was a particular strategy for taking players at various spots in the draft. For the vast majority of positions, the answer is no. Steals come from all over the draft and every position on the field, as do players that fall by the wayside early.

The draft is important of course, but perhaps important enough to justify the time and energy people invest in it. It would increase substantially in importance if any teams were remarkably adept compared to their peers at divining who the next great star players are. As far as we can tell, no one is right or wrong all the time in the draft, and very few teams are even right more than half the time. Think about that. That is not a terribly high bar. Here we simply mean, for example, does a player drafted in the fifth round have at least an average career for a fifth-rounder? That career would not need to include being an important starter; just a serviceable special teams guy and backup would suffice for a few short years. In addition, it need not happen for the team that drafted him to reflect well on the original team's talent-selecting abilities. This study did not care where a player made their fortune, just whether they did or did not. Yet only three teams drafted well enough over those four years to yield a median pick of above average value. A team's fate hinges on player acquisition, and the draft is one part of that. Nevertheless, as scientific as front offices would have us believe they are, the secrets of the draft remain a mystery.

Jason McKinley is a future actuary and St. Louis resident; he wrote the essay "In Defense of Mike Martz" in Pro Football Prospectus 2006. Guest column ideas can be submitted at info-at-footballoutsiders.com -- and if you are interested in writing for FO, note that May and June are the best months for guest columns.