Writers of Pro Football Prospectus 2008

Most Recent FO Features


» 2018 Free Agency Cost-Benefit Analysis

Is Kirk Cousins the best free-agent quarterback in recent memory? Should Trumaine Johnson or Malcolm Butler have gotten the larger contract? And what makes a free-agent contract good or bad, anyway?

29 Jul 2007

Value under the Cap

Guest column by Zac Hinz

One of the problems with statistical analysis in the NFL is that more than half the positions have few or no official stats associated with them. Offensive skill players have touchdowns and yards, defensive backs have interceptions and pass defensed, and defensive linemen have sacks and tackles. Each of these stats at least gives you some idea about whether Player A is better than Player B -- although Football Outsiders readers know that these stats can sometimes be misleading. Nonetheless, think about the poor offensive linemen. The NFL.com Player Pages are basically worthless for them. Flozell Adams' page, for example, tells us that he has started 130 of the 134 games he's played in. That's all we have to describe his ability as an offensive lineman.

But there is one number that exists for every player in the league, the number that the player cares about most: his salary. Right away, I have to admit that there are a lot of cases where a player's salary doesn't accurately describe his ability. The most obvious involve players playing under their rookie contracts. Some of these players are underpaid and some are overpaid. But there are plenty of veteran players who are being underpaid or overpaid too. (Most of the overpaid players are on the Redskins.)

Another challenge in judging players by salary is that some teams use of roster bonuses in place of signing bonuses. The Minnesota Vikings are especially known for this.

If every player signed a new one-year contract every season, then we'd really have something here. Then the USA Today database, which I used for the salary information, would be a perfect resource. However, just because they don't, and it's not, that does not mean that trying to tie salary to performance is worthless. I compared the amount of money a team spends on each position to each team's DVOA (explained here) in various categories. The data runs from 2000 to 2006, and therefore includes 222 data points. Salaries from before 2006 are adjusted upwards by a small amount so that the total amount of all NFL salaries on each year equals the total of all 2006 salaries.

Money, That's What I Want

Let's start basic: Is there a relationship between a team's overall salary and its overall DVOA. Yes, in fact, there is. The correlation between a team's overall salary and its overall DVOA is .22. A correlation of 1 signifies a perfect match; a correlation of 0 signifies absolutely no relationship between the two variables. Most mathematicians will tell you a correlation of .22 is pretty small. Given the problems with using salary as a metric, and the relatively small data size, we're going to have to take what we can get. (With a 16-game season, Football Outsiders is used to dealing with smaller correlations when it comes to performance stats.)

Think of the information here as a suggestion of where a relationship may exist. If we had better data, we could very well come up with quite different results -- or a stronger relationship in the same direction.

A Passing Fancy

On the offensive side of the ball, the correlation between Pass Offense DVOA and salary is highest for Quarterback (.27), followed by Receivers (.21), Running Backs (.13), and Offensive Line (.05). That does not necessarily mean that this is the order of importance for these positions, nor does it necessarily mean that the offensive line has nothing to do with the success of the passing game.

First off, remember that we're looking at players' salaries. Quarterbacks have their success rewarded very quickly. Carson Palmer, for example, got a contract extension after three years. If you're a quarterback and you haven't been offered an extension after four years, then the writing is probably on the wall. At most other positions, you aren't going to get a contract extension until you have one year left on your old contract.

Second, the salaries of players on their rookie contracts are based on the positions they were drafted at. And the majority of starting quarterbacks, wide receivers, halfbacks, and offensive tackles are high round draft picks. Tight ends, centers, guards, and fullbacks are more likely to be later round picks. So 100 percent of quarterbacks, 66 percent of receivers, 50 percent of running Backs, and 40 percent of offensive linemen are high draft picks. Notice that this is the same order as the correlation of salary with Passing DVOA.

Run To Daylight

The positional salary that has the highest correlation with Rush Offense DVOA is Special Teams salary. Special Teams salary includes the salaries of the kicker(s), punter(s), and long snapper(s). It does not include any returners. This is because (1) some teams make it difficult to tell who their returners are, and (2) some returners play a role on the offense or defense as well. Kickers, punters, and long snappers, meanwhile, play a role only on special teams.

It's actually a negative correlation (-.18), which means that the more you pay your three special teams specialists, the worse your rushing offense is. This doesn't seem to make any sense to me. If you've got a possible explanation, feel free to post it in the comments. All the correlations involving special teams DVOA were very confusing and could appear on the cover of Correlation is Not Causation Magazine. For more details, see the section on special teams below.

Rush Offense DVOA in general seems to have very little correlation with any salaries at all. Other than special teams, no other position correlates with Rush Offense DVOA any higher than +/- .10. This could be because the strength of the running game is based more on the offensive line, and offensive line has the highest percentage of players who might be starting as late-round picks -- and therefore are being underpaid, as mentioned above.

In My Defense

So that I didn't have to deal with separating 3-4 defenses from 4-3 defenses, I calculated the salaries of the front seven all together. The numbers showed that the front seven has a larger impact than the defensive backs when it comes to both Rush Defense DVOA and Pass Defense DVOA. Perhaps it shouldn't come as a surprise that seven guys on the defense are more important than the other four guys.

For the front seven, the correlations are -.21 to Pass Defense DVOA and -.15 to Run Defense DVOA. For the secondary, the correlations are basically non-existent: -.12 to Pass Defense DVOA and -.04 to Run Defense DVOA.

Just for fun, though, the 4-3 numbers, in which defensive linemen are separated from linebackers, show that defensive linemen have the highest correlation with Pass Defense DVOA, and linebackers have the highest correlation with Rush Defense DVOA. This seems to suggest that the pass rush (provided by defensive linemen in the 4-3) is more important to Pass Defense DVOA than the coverage skills of the defensive backs.

Stranger is the case of the 3-4 defense. The 3-4 linemen and linebackers have the most ridiculously small sample sizes of anything in this study (there were only 33 teams running the 3-4 in the study's seven years) . However, the numbers for 3-4s are the opposite of those of the 4-3. The defensive linemen have the highest correlation with Rush Defense DVOA, and the linebackers have the highest correlation with Pass Defense DVOA.

I can't explain the relative unimportance of defensive backs. They do have a correlation with Pass Defense DVOA, but it's half as much as the correlation that 4-3 defensive linemen have on Pass Defense DVOA. Total Defensive Salary has the highest correlation with Pass Defense DVOA. That could mean that all parts of the defense need to be working in order to have a good pass defense.

Not So Special Teams

This section is pretty weird. The numbers show that teams that spend more money on special teams specialists (remember, that's only kickers, punters, and long snappers) have better defenses (both passing and rushing), and worse rushing offenses. There are three possibilities here:

1) The numbers are wrong.

2) The numbers are totally unconnected, and the correlation is meaningless.

3) The numbers are not saying that teams which spend more money on special teams have better defense. They are saying the reverse: teams that have good defenses and bad rushing offenses are the ones that are most inclined to pay big money to their specialists. When you are going to punt the ball a lot, and have difficulty scoring touchdowns, your punter and kicker become more important.

The other interesting finding is that Special Teams salary does not have the highest correlation with Special Teams DVOA. Rather, it is defensive backs and linebackers that have the highest correlation. This seems to imply that the coverage and return teams (which have large numbers of defensive backs and linebackers on them) are more important to overall special teams than the specialists are. Defensive backs also often double as return men -- although so do receivers, and the correlation between receivers and special teams is only half that of defensive backs and special teams.

In The Future

I fully admit that this project represents completely unfinished research. It is a simple first try at answering a complicated question. At a later date, I would like to divide these players up into smaller groups: separate halfbacks from fullbacks, cornerbacks from safeties, and if possible, tackles from guards from centers. I could also separate special teams DVOA into its five components to see if it is important to pay a kicker to get high field-goal value, and what position's salary correlates best specifically with kick and punt returns.

Zac Hinz lives in Oshkosh, Wisconsin, and is currently number 38,151 on the waiting list for Packers season tickets. He expects to receive them the same day he is tragically hit by a bus. If you would like to write a guest column for Football Outsiders, something that analyzes the NFL or college football in a new and different way, please send your idea or rough draft to mailbag-at-footballoutsiders.com.

Posted by: Guest on 29 Jul 2007

37 comments, Last at 14 Aug 2007, 6:00pm by steelersalarycap


by Theo, Netherlands (not verified) :: Sun, 07/29/2007 - 7:30pm

"the more you pay your three special teams specialists, the worse your rushing offense is."
Edgerrin, meet Neil Rackers. Neil, Edge.

by hooper (not verified) :: Sun, 07/29/2007 - 9:26pm

"The data runs from 2000 to 2006, and therefore includes 222 data points."

Just curious - how did you get the 222? In my mind, 7 years * 32 teams gives 224, but I can't get any closer than that.

Oh, and this is my first first. :)

by Brian (not verified) :: Sun, 07/29/2007 - 9:40pm


The special teams/run offense correlation may be due to coincidence. For n=220, you'll get a -0.18 correlation 6% of the time by chance even when there is no connection between variables.

It seems unlikely, but every 16 correlations you run will give you one (possibly) spurrious result just like this. It's especially likely when there is no theoretical reason for the outcome.

Great research. One reason for the low correlations is that player performance during initial draft contracts are, at best, educated guesses. Players in their follow-on contracts at least have an NFL track record.

by Brian (not verified) :: Sun, 07/29/2007 - 9:45pm

The significance level (p=0.05) for n=220 is 0.186 for those wondering. The correlation levels observed are very small and only barely outside that spec.

...and Hooper, the 32nd team was added beginning in 2002.

by Martin (not verified) :: Sun, 07/29/2007 - 9:48pm

re: no1

Texans weren't there in 2000 and 2001

Zac, great potential under this research

by Chris (not verified) :: Sun, 07/29/2007 - 10:20pm

Front 7, most important part of your defense. Front 4 is the most important part of your pass defense, linebackers are the most important part of your run defense. Somebody should tell this to the Washington Redskins.

I can see why the backers are the most important part of a 3-4 pass defense, because they are the 4th or more rusher ( not designated like in a 4-3). Instead of a DE rushing, you have an LT, Greg Lloyd, Kevin Greene, Shawn Merriman, Mike Vrabel etc.

by Zac (not verified) :: Sun, 07/29/2007 - 10:32pm

Thanks for the kind words, everyone. I agree (and Aaron said as much when I sent him my first draft) that the correlations are very small.

I agree that it doesn't work with players on their initial contract, which is a large percentage of players.

I started this idea a while ago. When I got done, I wasn't sure if I had found out anything useful or not.

by Optimistic Packer Fan (not verified) :: Sun, 07/29/2007 - 10:41pm


by IzzionSona (not verified) :: Sun, 07/29/2007 - 10:56pm

Would a logical extension of this be to look at teams that sign players to second contracts (either via FA or extending), and then seeing how that correlates to changes in and/or levels of DVOA (and possibly wins) in the next year(s)?

by Zac (not verified) :: Sun, 07/29/2007 - 11:02pm

Oh, I put all the raw data online. You can download a spreadsheet of it in .csv format by clicking my name.

by Not saying (not verified) :: Sun, 07/29/2007 - 11:08pm

Enjoyed the idea of the article. Seems like a good starting place. It's really hard dealing with the complicated system of salary information.

One small thing: "So 100 percent of quarterbacks . . . are high draft picks." That's not true. Am I missing something?

by Alex (not verified) :: Mon, 07/30/2007 - 3:07am

One small thing: “So 100 percent of quarterbacks . . . are high draft picks.� That’s not true. Am I missing something?

I was wondering about that, too. Did the Patriots move Tom Brady to RB, or something? I mean, his run against Chicago where he evaded Brian Urlacher was impressive, but I don't think it warrants a change of position. ;)

by Pat (not verified) :: Mon, 07/30/2007 - 3:26am

Regarding the intro: the introductory text says essentially "looks at what positions make sense to devote salary cap space to."

That's not quite what the article's saying. Zac's using salary as a proxy for talent. Clearly, giving Charlie Frye a $10M/year contract will not make the Browns a winning franchise.

Zac's essentially looking at "what positions are most valuable to a team?" What's interesting is that running backs come out as fairly valuable. From an injury analysis, Carl Prine had showed that teams don't do any worse when they lose their starting running back.

What that tells you is that running backs are important, but replaceable.


I've got a feeling that the special teams correlation isn't "really" a fluke - well, it is, as it's not causative, but the reason for it is because there's not a lot of variation in the amount of money that teams pay kickers and punters. Kickers get about 1.5M. Punters get about 1M. That's pretty much it.

In a case like that, you can get correlation if you've got one or two outliers, which is probably what's going on here. So if there's a case where you had an overpaid kicker/punter, and a team that utterly sucked at running, you'd get a minor negative correlation, just because they'd have large lever arm and that portion of the phase space wasn't well populated.

Oh! Hello, the 2005 Arizona Cardinals!

So it's not really random. It's just due to the fact that you don't have a uniform sampling across the range of salaries. That's probably true for offensive lines, as well, for instance - you simply can't afford 5 top-tier OL, and so there are no data points out there.

Just to comment on what #4 said: using p-values in a study like this doesn't really work, as the data isn't normally distributed. You'd really want to do a bootstrap analysis to see what the likelihood distribution for the correlation coefficients looks like. For the special teams one, it's probably a *big* peak at zero, with an asymmetric negative tail which the current data set happens to fall into.

by Zac (not verified) :: Mon, 07/30/2007 - 8:28am

Re: 11 & 12. That's a section that got rewritten, and it's not very clear. There are certain positions where the majority of the starters come from the top of the draft. On offense, they are QB, WR, Halfback, and OT. A team's QB salary includes 1 starter, so 1 out of the team's 1 starters tend to be high draft picks. That's the 100%.
Contrast this with the offensive line. 2 of the starting linemen (the OTs) play at a position where the best players are taken early in the draft. That's 40%. Receivers are at 66% (the two WR are in, and the TE is out) and RBs are at 50% (the Halfback is, the Fullback isn't).
I don't know if I'm explaining this very good, so let me know if you still don't understand.

by hooper (not verified) :: Mon, 07/30/2007 - 9:35am


I forgot to mention my appreciation in my first post. Thanks for the article, and I'm curious to see where you take this.

#4 and #5:
Thanks. I had forgotten about the Texans - shame on me. I knew there had to be a reason I was off, though.

As far as the special teams / rushing offense thing goes - are there any teams that are notable outliers in this? I'd first look toward the Oakland "First Round Kicker" Raiders and the Dallas "Vanderjerk" Cowboys as two teams who have influenced the special teams numbers over the last few years. Other than that, I can't really get my mind wrapped around a reason why kicker salaries should correlate to running effectiveness. It's not like anybody is paying their kicker 10 mil a year.

by Alex (not verified) :: Mon, 07/30/2007 - 9:41am

I guess I follow that. For those who are still interested, I count 8 QBs who weren't drafted in the first three rounds who are going to be starters next year, so really 75% of QBs are high draft picks.

by B (not verified) :: Mon, 07/30/2007 - 11:36am

2: The reason there are 222 data points is there were 31 teams in the league in 2000 and 2001

by BD (not verified) :: Mon, 07/30/2007 - 12:29pm


Great idea, and interesting to consider. One thing I'm thinking about would be that all of these numbers are inter-correlated, because teams have finite resources... in other words, a team that spends more on a big-time quarterback necessarily spends less somewhere else. I'm wondering how these results would change then if looking at partial correlations, or perhaps even a factor analysis to see if certain positions tend to "go together" when GM's are allocating salaries. The multicollinearity could also lead to spurious correlations as you think you have observed. Since you posted the data, someone here could run it or I could do it if I have time.

Sorry about all the statistical nerdiness in this post.

by Vern (not verified) :: Mon, 07/30/2007 - 12:37pm

One idea to improve the analysis (hopefully). In the spirit of DVOA, do not just use salary, but "Salary Above Average" for the given position (AND YEAR) as the comparison. Or more generally, whatever the number you use (Salary, Percentage of Cap, etc.) use it only "above average" when comparing.

Ideally, you would first calculate "salary above average" for each player first as some kind of percentage (e.g. what percentile of pay is that player in for that position and number of years in the league) and not by $ value. A "slightly highly paid" QB may be making a million more, whereas a "slightly highly paid" OL may only be a 100K. The issue here isn't $, but percentage relative to the position (and to the year of the player, see below).

The year of the player is key to deal with the fact that all players have a built-in pay scale due to their rookie deal, and the various FA stages they may go through. I think the simplest way to incorporate this is to just figure each salary average by position AND the year of the player, or perhaps using player year ranges of: 1-3 years, 3-6 years, 6-8 years, etc.

Doing it by year eliminates the need to go off and catalog what exact deal each player is/was under all these years. Since most rookie deals average 5 years, and some get re-done in year 3 to 4, I did the first cut-off at year 3, then 6 and so on.

With this, you should be able to then say things like: Does a player paid "more than the league average" deliver better production compared to others in the same position and year?

by Vern (not verified) :: Mon, 07/30/2007 - 12:39pm

Sorry, in the spirt of DVOA, that would be "Salary OVER Average" not "above."

by bravehoptoad (not verified) :: Mon, 07/30/2007 - 12:42pm

I wonder what would happen to your data if you factored out players still playing under their rookie contracts? Presumably veterans would be getting paid what they're "worth," and therefore their salaries would show truer correlations.

by Bobman (not verified) :: Mon, 07/30/2007 - 2:58pm

#21 Cutting the data set in half, I assume. I agree with your logic re: proven commodities, and it might be more precise, but less reliable.

#20: How about Salary Level Above avg Player, or SLAP? Using Over would be SLOP.

just trying to be helpful....

by David (not verified) :: Mon, 07/30/2007 - 2:59pm

Wow. This is one of the many things I love about FO.com. On most sites, sports sites in particular, reading the comments is a frustrating ordeal of nonsense and shouting and bad spelling. Here there's politely stated, well reasoned positions in every thread, and on occasions like this one, a fairly detailed discussion of some pretty advanced mathematical concepts. Awesome. Keep it up, all of you.

by Tom (not verified) :: Mon, 07/30/2007 - 4:01pm

I have some ideas on why money spent on the secondary doesn't correlate well with passing defense.

1) Safeties are one of the low pay positions, so you don't need to spend a lot to get a very good safeties.

2) Cornerback is one of those positions that rookies and young players can play at a high level, so a lot of the better corners on their rookie contracts. Just look at Chicago, New England, Buffalo, and Tennessee.

3) Last, but not least is that corner seems to be a high bust rate position. So you have a lot of high profile signings that turn out not to work so well, ie Washington and Minnesota.

by Thok (not verified) :: Mon, 07/30/2007 - 4:29pm

Maybe I'm being dumb, but wouldn't the correlation between special team DVOA and rushing DVOA simply be the observation that both factors relate (somewhat) to GM stupidity? I'd assume that bad GM's overvalue kickers and undervalue offensive lines.

by The McNabb Bowl Game Anomaly (not verified) :: Mon, 07/30/2007 - 5:55pm

Re: 24

Small correction- Washington has not signed any high profile busts at WR, although Springs has been injured a lot. In fact, Washington hasn't had any high profile secondary signings besides Springs and Archuleta since Mark Carrier and Deion Sanders, IIRC.

They have drafted Bailey, Smoot, Rogers, Taylor, and Landry over roughly that period.

by Ted Max (not verified) :: Tue, 07/31/2007 - 9:16am

I also appreciate the article and the thoughtfulness of the comments. I was gearing up to write another "geez, these correlations are small" comment, and then realized THAT'S THE STORY HERE!

There is almost no correlation between a team's overall salary and their overall DVOA (.22 R = .0484 R-squared, which means that 95% of the variation in DVOA is accounted for by something OTHER than salary). The same can be said for all of the other virtually nonexistent correlations: DVOA and salary are almost completely unconnected at any position.

This non-finding makes a certain amount of sense: If you are a fanboy of any team, you know that the money guys generally do a lousy job of assigning correct dollar values to talent, a situation that's made worse by salary structure issues like the cap and the various tags, etc.

I'd say let's step back and ask whether these tweaks we want to make to fit the data better even make sense. Maybe there just isn't a consistent connection between salary and DVOA: Both good and bad players [or units, in this case] get big paydays, and both good and bad players get small paydays.

Isn't that interesting in and of itself?

by Vern (not verified) :: Tue, 07/31/2007 - 11:12am

Re: 27
I think the correlations are small in the aggregate - which serves as a good baseline. But the real interesting part of this kind of analysis is looking for "market inefficiencies" by position or year or both where there is some correlation.

For one great example of this, check out figure 9 in the Massey and Thaler piece (linked on name) that shows where the "best" draft picks are actually between 15 to 80.

by Vern (not verified) :: Tue, 07/31/2007 - 11:14am

Re: 27
I think the correlations are small in the aggregate - which serves as a good baseline. But the real interesting part of this kind of analysis is looking for "market inefficiencies" by position or year or both where there is some correlation.

For one great example of this, check out figure 9 in the Massey and Thaler piece (http://faculty.fuqua.duke.edu/%7Ecadem/bio/massey%20&%20thaler%20-%20los...) that shows where the "best" draft picks are actually between 15 to 80.

by Phill Skelton (not verified) :: Tue, 07/31/2007 - 1:26pm

The weak correlations aren't all that surprising given the salary cap - at least that's my intuition (which may be wrong). Ultimately *all* teams have the same salary (give or take a little underspending), which would suggest that overall team spending and DVOA are almost entirely uncorrelated - with a small allowance for cheapskate teams who habitually underspend and field poor teams year in year out. Comapre that to baseball, where salary and something like DVOA (or whatever the moneyball equivalent is) are probably quite strongly correlated.

I assume market inefficiencies would show up in an analysis of money spent on a position vs DVOA of that position (team DVOA is essentially the sum of indivudual player DVOAs right?) rather than position by position correlations between salary and DVOA.

Look at the ratios of salaries of QBs, RBs, WRs, TEs and O-lines, then look at how much each group contributes to the overall offensive DVOA. Presumably there will be some positions (QB by any chance?) which eat a large fraction of the available salary, but don't contribute anywhere near as much of a fraction of the teams DVOA (maybe DPAR would be a better stat to use, come to think of it). And a position that is cheap in proportion to the DPAR generated is one that presumably a smart team could spend extra money on to lure better players, gaining a better overall team for less money.

In a completely efficient market, a position that (on average) contributed 10% of the teams DPAR would consume 10% of the teams salary. I doubt that happens. Partly because even DVOA and DPAR are stats of limited accuracy. Partly because coaches tend to over-rate their own players and also to go with 'gut instinct' (which, with the best will in the world, is going to be modified by who they happen to like). And also because football franchises are not simply concerned with getting the best team on the field. They also want o generate a profit by creating a marketable commodity, and so a big-name QB can get more money because they can generate more revenue for the franchise (I wonder how replica jersey sales break down by position, and how well that correlates with salaries...).

by Jimmy (not verified) :: Tue, 07/31/2007 - 1:59pm

Could the low cap figure for rookies in their first contract be compensated for by using a combination of cap figure and draft pick investment. The two methods of acquiring talent for a team are cap room expenditure (which this article already uses) and draft pick expenditure, both are finite resources and are used up in different ways by every GM. While some draft picks are busts, some free agents are over-paid, and some of both can turn out to be bargains. Would it be possible to factor in points value of draft picks (using the Miami chart or similar) over a three year period to the cap figures of players on their first contract.

If the question being answered is how teams can best use their available resources to acquire talent then draft picks need to be included. For example a guard taken in the first round will have a fairly low cap figure but it still represents a large investment of a team's resources in the position. Or the Colts corners; they may not be paid much but they have spent a first and two second round picks on them, again a big investment of finite resources.

by Troy Polamalu (not verified) :: Tue, 07/31/2007 - 4:44pm

1) Safeties are one of the low pay positions, so you don’t need to spend a lot to get a very good safeties.

Thank you for not telling Kevin Colbert.

by Pat (not verified) :: Tue, 07/31/2007 - 6:31pm

#27: It's incredibly naive to think that salary and DVOA are "truly" unrelated just because the correlation coefficient's small.

It's small because we're using a crappy measure. Even if two things are perfectly correlated, if you add completely random noise to one of the variables, you weaken the correlation. All this is telling you is that the noise is huge. But you'd expect that. Salaries are paid on multi-year timescales (when contracts are negotiated) - the year-to-year salary cap number is only weakly connected to the "true" salary. Moreover, the market changes from year to year.

The assumption chain:
Yearly salary cap value is a proxy for true salary.
True salary is a proxy for true worth (ideal market).
True worth is a proxy for talent (you pay more for more talent).
More talent results in higher DVOA for a given year.

Your latter conclusions ("if you are a fanboy of any team, you know that the money guys generally do a lousy job of assigning correct dollar values to talent") assume that the second assumption is adding the noise (i.e. teams over/underpay for talent).

In all likelihood, the first and third assumptions are probably adding virtually all the noise. Teams jitter around yearly salaries to fit everyone in - a player's salary cap value jitters by factors of 2 or more based on whether or not the team needs the cap space. There's a huge fraction of the noise right there.

The third assumption also adds a lot of noise: a player's worth isn't purely determined by his talent. It's a free market. A player's worth is determined by the supply of players at that talent level. A team which needs a guard in a year that's short on guards will pay more. This doesn't mean the team is stupid. It means they were simply in a poor spot that year.

Finally, a correlation can also be weak simply because the phase space isn't well populated, which is also true here.

Why? As an example: Teams aren't stupid enough to start a cheap quarterback unless they have to. This, unfortunately, isn't a lab. We have to accept that the sample's biased.

For instance: take all teams with a QB who makes less than $3M (in 2006 dollars). Take all teams with a QB who makes $9M or more. Correlate passing DVOA with that. The correlation coefficient will almost certainly be much larger.

by Zac (not verified) :: Wed, 08/01/2007 - 12:36am

Hey, all.
BD & Pat, I have to say that what little statistical knowledge I once had is leaving me. What you've suggested is beyond my abilities. Anyone who wants to build on this is free to do so.

I agree that there's a ton of background noise clouding the data. As of right now unfortunately it's the best we can do.

Here's the correlation between QB salary and passing DVOA, divided into 4 groups that each had 25% of the data. Remember this is the salary for all QBs on the team, not just the starter.
1st 25% ($11.6 mil or more): .17
2nd 25% ($8 mil to $11.6 mil): .09
3rd 25% ($4.8 mil to $8 mil): .04
Lowest 25% (less than $4.8 mil): .14
You are correct that the extremes show more collinearity. It seems strange that all 4 groups when taken by themselves have a lower correlation than the entire set of data as a whole (.27). Just small sample size issues?

The highest special teams salary belongs to the 2005 Indianapolis Colts, who paid Mike Vanderjagt a lot of money to be a bad FG kicker, while also being so bad at kicking off that they had to pay some other guy to do it.

The lowest special teams salary was the 2003 Vikings, who had Rookie K Aaron Elling, Rookie P Eddie Johnson (who played 14 games that year, and hasn't made an opening day roster since), and had P Leo Araguz for the other two games. Their special teams DVOA ranked 31st.

by Pat (not verified) :: Wed, 08/01/2007 - 2:05am

#33: Zac:

I wish I had a blackboard here. I could explain this in five seconds. :) It's too bad a statistics article would be too complicated - there are a few problems which show up all the time which a lot of people misunderstand.

I'll try to explain with crappy graphs that look like a child drew them (it's 'Kindergarten Nonparametric Statistics'!). Imagine a data set that looks like this (the red and blue are data, the black are axes, the dashed line is y=x). The data is clearly not normally distributed. Not anywhere close.

If you run a regression on that, you'll get a *very* strong correlation, since the spread will be dominated by the span between the two points (and, in fact, you should get one - because the two variables are clearly correlated).

Now, instead, consider doing a regression on just the blue, or just the red points. You'd get a nonexistent correlation on either. The correlation is there. It's just that the effect of the scatter is larger than the effect of the correlation over the range in the dependent variable that you're sampling.

What I was suggesting was using only the teams with the highest paid quarterbacks and the teams with the lowest paid quarterbacks, and run a correlation with that data set. It'll probably be noticeably bigger.

(Note: while I said 'nonparametric statistics', this isn't a nonparametric method. It's, in fact, a horrible method. The point, though, is that the correlation is there, and it is significant - hey, amazing, cheap QBs suck - but the spread seen in the actual NFL isn't large enough to make a difference. Which, again, isn't surprising. Teams don't play crappy quarterbacks. You could do a non-parametric test of the hypothesis that cheap QBs suck - do a t-test of the highest paid QBs versus the worst paid QBs, for instance. And it would almost certainly tell you 'yes, they do, very significantly.')

by jb (not verified) :: Tue, 08/07/2007 - 1:27pm

I think there is a correlation between the quality of comments posted about this article and the number of retards that would get past the first couple paragraphs.

Loved the article even if we don't know if actually found out anything useful.

Thanks Zac

by steelersalarycap (not verified) :: Tue, 08/14/2007 - 6:00pm

Interesting stuff. I would like to see more of the data.

Several years ago, I played around with linking DVOA and salary. I got busy, and never did much afterwards.

here is the link to some of my data (ps. I was using the older versions of DVOA, so some of the DVOA values might be different than th current updated DVOA values)


I was mostly trying to link unit DVOA with unit SALVOA (SALary Value Over Average).

Anyways, good stuff. Finding a way to link the management side of football to on field performance would be quite telling...