Writers of Pro Football Prospectus 2008

25 Aug 2009

Has Competitive Balance in the NFL Declined?

Last week when we debuted our full DVOA ratings for 1994, one of the issues I brought up in the commentary was the growing gap between the league's worst team and best team according to DVOA. A few people wondered whether that "growing gap" was actually a mirage that came from looking at only 1994 and 2008 without looking at the years in between, so I decided to go look.

The blue diamonds on this next chart represent the difference between the best DVOA and worst DVOA each season since 1994. The red squares represent the difference between the average DVOA of the three best teams and the average DVOA of the three worst teams, just in case a season happened to have one particularly extreme team that would skew the results. Each set of data points has a trendline.

The rising gap between the best and worst teams, at least according to DVOA, appears to be a pretty clear trend. I don't know what's behind the trend. I also don't know why teams were much closer together in 2001 and 2006 than in other years. One thing you will notice, however, is that there aren't any clear "jumps" on the table during expansion years. There's a big jump from 2001 to 2002, but the trend makes sense when you look at years before 2001. The same is true for the increase when Cleveland v2.0 came into the league in 1999.

Interesting, eh?

Posted by: Aaron Schatz on 25 Aug 2009

83 comments, Last at 28 Aug 2009, 7:09pm by Danny Tuccitto


by doctorjorts :: Tue, 08/25/2009 - 10:22pm

Two words: Matt Millen

Seriously, though, this is a nice analysis. When the patriots go undefeated and the lions go unvictorious in consecutive years that's enough to pique my interest, but comparing the top and bottom 3 really seems to give this idea some support. It's also kind of interesting that this appears to be a curved and not a linear trend.

by Raiderjoe :: Tue, 08/25/2009 - 10:26pm

maybe ebcausse some teams figure out fere agency better than others so work this to advantage. example is NEP who give T Brady crap contract and he dumb enough to like it but smart enough to know it make sense for team so he can get better players around him. if T Brady was jerk like E Grbac he would be highest paid player in league and his best players on offense would be Justin gage and Troy willaismon and some crappy linemen. Saso some teeams keep staying good like Pates and Colts and Toitans but other teams like chiefs and browns and Lions are dumb teams that dont know hwo to work system.

by JasonK :: Tue, 08/25/2009 - 10:56pm

There's actually some truth in there, I think. In the mid-90s, modern NFL free agency (i.e., post-"Plan B") and the salary cap were new, and nobody was all that good at managing it yet. It was a chaotic, equalizing factor, undercutting previous advantages that many teams had in scouting and developing players. But as teams started hiring smart guys to do cap analysis, and as older GMs who didn't adjust well to the new realities were replaced, the templates for strategies that worked began to emerge. As opposed to chaos, the free agency system now provided effective tools for intelligent managers to gain advantages over the less effective managers.

by drobviousso :: Wed, 08/26/2009 - 10:43am

Yup, we don't hear too much about the Pats, Steelers, Super-Chargers, Eagles, etc overpaying their best talent (Rivers signed for less than Eli?!?) like we do for some really bad teams.

by HostileGospel :: Tue, 08/25/2009 - 11:43pm

So, if the trick is to find a future HoF QB who is dumb enough to like being paid like crap... how are the Titans successful again?

Overall, I'd be kind of embarrassed to critique something when I didn't know what the hell I was talking about, but then, oh yeah, my NAME is on what I write, isn't it?

-Les Bowen

by dbostedo :: Wed, 08/26/2009 - 2:25pm

I believe that was just an example - not the only way to do well.

by Israel P. - Jer... :: Tue, 08/25/2009 - 10:27pm

To those who claim the competitive balance has worsened, isn't the issue that the same teams seem to be towards the edges more that what the parity folks wanted?

by Waverly :: Wed, 08/26/2009 - 8:13am

I agree -- it's turnover of the teams at the top that defines competitive balance, not the difference between the best and the worst teams.

I think most fans don't mind (too much) if their team is bad one year, as long as they have a plausible chance for being on top next year. [Insert standard jokes here.]

by Adam B. :: Wed, 08/26/2009 - 1:21pm

Agreed. When I think of "competitive balance," it's not whether the top teams are 15-1 as opposed to 12-4; it's the likelihood of teams at the top remaining at the top from year to year.

by Pat (filler) (not verified) :: Wed, 08/26/2009 - 10:42pm

The former, however, shows up in the quality of games - lack of in-season parity means that you get lots of garbage teams vs great teams. I think the NFL is probably more concerned about that than how long Lions fans might have to wait to see their team turn around.

by Insancipitory :: Tue, 08/25/2009 - 10:55pm

Eh ... I might agree on the "jumps" but it does appear that the trend started in 1995 (or so) and is accelerating. That might suggest there is a shortage of talent given the timeframe allowed for player development, and the exclusive rights teams enjoy when negotiating with young players they are developing. Larger rosters with teams allowed more offseason time to develop talent, greater opportunity for players to move around, or a viable farm system might be helpful. That is assuming a significant population of people who don't make 53 man rosters might be able to develop into NFL talent given more time.

by Rich Conley (not verified) :: Wed, 08/26/2009 - 10:58am

"Larger rosters with teams allowed more offseason time to develop talent, greater opportunity for players to move around, or a viable farm system might be helpful."

I think that would make the "problem" worse. Teams like New England, Indy, Pittsburgh, regularly cut guys young guys who go on to become starters or important role players on middle of the road teams. If you expand the rosters, NE/IND/PITT hang onto those guys.

by Dennis :: Wed, 08/26/2009 - 3:31pm

I agree. Smaller rosters are better for competitive balance than larger ones. Larger rosters would amplify the difference between the good and bad GMs.

by Insancipitory :: Wed, 08/26/2009 - 9:48pm

Notice that it was coupled with greater opportunity for players to move around. There are a lot of guys who we'll never know about. Lots of conversion projects, sure there are the Devin Hesters and Antonio Gates, but there are also Jordan Kents. After 3 years, if they can't crack the 53 man roster, what are their odds?

That said, some teams do do a better job at player development, with the new influx of talent they'll turn over players insearch of more diamonds in the rough, they have enough role players. What are they going to do, sit on their hands, "Well.... now we're good enough."

A good example of a team hanging on to a player too long to perhaps the detriment of the player and the league but obviously not the team are the Seattle Seahawks and Seneca Wallace. He's not going to start, but there are NFL teams where he could. Because of his numbers last year, the Seahawks excercised an option forcing him to remain a backup. He'll leave, because he wants to show everyone he can start. There are teams who's fortunes he could change. It's good for the Seahawks, and me, but not him, not the NFL.

If my view of the trend is accurate, the NFL needs more quality players. Waiting for them to be crapped out of the aether isn't going to make it happen. Develope them or go without seem to be the choices.

by The Ghost of Tiki Barber (not verified) :: Tue, 08/25/2009 - 10:59pm

Here's a parallel for you. I used to play competitive collectible card games. (Go, Crab Clan!) The cards on the tournament lists create the environment that every player had to operate in, and certain environments were considered more or less balanced than others. If there was a killer card combo, then players of all skill levels had about the same chance in a tournament, since dumb luck played a large part in success--who would draw the winning combinations first? But a more balanced environment actually favored more skilled players--since they had to be better at assembling their decks and playing the cards they drew, skill was more important than dumb luck.

So while balance in terms of the environment in which NFL teams operate may have been achieved, that has done nothing to affect the skill with which these organizations function.

by Mike Kurtz :: Wed, 08/26/2009 - 9:23am

Foolish, uncouth crab! How dare you suggest that any environment is not perfectly and sublimely balanced! Seppuku for you.

by The Ghost of Tiki Barber (not verified) :: Wed, 08/26/2009 - 9:57am

Careful, or you'll find yourself on Deadly Ground... ;-)

by JasonK :: Tue, 08/25/2009 - 11:00pm

Also, the increase from 28 teams to 32 teams in the League plays a part here.

But that can only explain the decline in parity through 2002. The developments since the expansion to 32 franchises must be coming from some other cause.

by Key19 :: Tue, 08/25/2009 - 11:02pm

Here's my theory (with no evidential backing whatsoever):

While the Salary Cap and Free Agency were created to establish more balance throughout the league, they really have done the opposite.

The worst teams (SF, DET, etc.) are constantly overturning their head regimes. As a result, coaches (and GMs) know they only have a short window of opportunity to get the ball rolling in the right direction. Therefore, they look to free agency for experienced and proven players. Drafting is not only much more difficult, but also takes longer to reap the rewards. New-age coaches don't have time for these things for the most part. As a result, they try and hit on their lottery picks, possibly trade a bunch of picks for more high ones, brew excitement, and hope that their new rookie phenom can carry the team along with some castoffs that were acquired in free agency. These teams are then stuck with paying top dollar for both mediocre free agents and unproven lottery picks. With no room under the cap to really build any depth or get any players worth developing (aside from the lottery guys), the team falls to shambles quickly and the regime is fired. Then a new regime comes in and tries the same thing all over again.

However, the best teams are in very powerful positions. They typically have the picks that the lottery teams want (after spending their first 1st rounder). They then trade down for more picks and stockpile things. They have enough picks that they can afford to miss on some guys who are really only going to provide depth anyways. No superstars needed here (for the most part). They also have an established heritage of winning. Players take less money to join them, the players they already have (and are worth keeping) stay because they know they've got a great situation. The coaches/GMs essentially have tenure because the owner trusts them after such long streaks of success.

The teams that become exceptions to this (Falcons, Dolphins) are the ones that are lucky enough to hit on great personnel guys, but also give them the time and space required to build something special. Hence, they can make up a lot of ground very quickly.

Just an idea. Feel free to poke holes in it. I'm actually pretty interested by the trend and I want to know the reason why it exists.

by Will Allen (not verified) :: Wed, 08/26/2009 - 12:08am

Actually, there was one head guy, Millen, in Detroit who couldn't get fired no matter how hard he tried. I suspect that strengthened the trend. There was another head guy, Al Davis, who has completely lost his grip over the past seven years, and can't be fired, and likely only leaves the job in a supine position. That strengthens the trend more. This may be simply an anomaly brought about by unique cricumstances.

by Bill (not verified) :: Wed, 08/26/2009 - 12:18am

You can substitute Mike Brown for Al Davis, and it fits too. Well, except he never had a grip to lose.

by Karl Cuba :: Wed, 08/26/2009 - 7:47am

I think that there's a lot to what you've proposed there. Two additional factors that you missed: the free agents leaving the more successful teams are more likely to be over paid and so those teams are more likely to receive decent compensatory draft picks for them, allowing them to trade their own picks for higher picks in the future or simple trade up in the draft. Secondly, the more stable franchises with 'tenured' staffs can stockpile picks in the manner the Pats have been doing so well recently

by The Ghost of Tiki Barber (not verified) :: Wed, 08/26/2009 - 9:39am

How much of a factor is the draft in all of this?

Think about the onerous rookie contracts losing teams are saddled with as a result of the way the draft works. Sure, it sometimes works out, but more often than not the high money comes with high expectations and, worse, high pressure to be the "savior" of a losing franchise. It's a recipe for failure more often than not, if the failure rate of first-round QBs is any indicator.
Whereas a rookie going to a successful team (Giants, Patriots, Steelers, Eagles) is going into a much more stable environment where they're usually not expected to be The Star Player! right away.

Maybe it comes down to ownership. Since the amount of money spent on an organization is more-or-less even across the league, that shouldn't be a factor. Each team has access to same pool of talent via the draft, so that shouldn't be a factor. We're left with organizational structure and ownership priorities.

Some owners are good at building organizations--your Patriots, your Giants, your Steelers, your Eagles--and prioritize winning as well. Other owners have different priorities--Al Davis seems to prioritize "doing things his way" over winning. Dan Snyder seems to prefer playing George Steinbrenner to building a stable organization. The Fords...uh...oh, god, the Fords...

An era with more balance is going to be much, much harsher in exposing the flaws of these organizations, as they have a much slimmer margin for error in any given year. The chart could show the accumulated effect of several years' worth of such behavior as the negative consequences accrue.

I don't know how you can fix this problem. Perhaps you could identify the link between failing teams and the flaws in the rules that help perpetuate losing, and address those rules. (A hard slotting system for rookie contracts, for example, or expanding a perennial loser's roster by a few extra slots to allow them to gamble more on devleoping talent at the big-league level.)

Or perhaps the NFL could commission a study on the optimal way for owners to build an organization; but unfortunately, you can't legislate stupidity out of existence.

by mm (not verified) :: Wed, 08/26/2009 - 11:04am

While the Salary Cap and Free Agency were created to establish more balance throughout the league, they really have done the opposite.

I just want to remind everyone that Free Agency was created because the Player's Union forced it on the league, not because they thought it would add balance. The Salary Cap (along with the floor) was included to try and keep competetive balance, not because they thought the new system would improve it.

by jimbohead :: Tue, 08/25/2009 - 11:06pm

any chance we could see variance, and not just the gap? I'm really interested if this is just an artifact of crazy and consistent outliers, or something more systemic.

by the silent speaker (not verified) :: Wed, 08/26/2009 - 12:09am

Eyeballing the graph, it looks like the effect of averaging three teams is a consistent 10% leveling. You'd expect a leveling from the average, of course, but the consistent size of it suggests that the effect is linear and not curved, no? A curved effect would also have its leveler affected by the multiplier. If we relocate the 2007 Patriots to where they're 'supposed to' be based on the brown datapoint -- around 80% rather than 90% -- and pretend the outlier years of 2001 and 2006 aren't there, how closely does a linear trendline match? Better than the current curve, or not?

P.S. You're missing the 2005 blue datapoint.

by Waverly :: Wed, 08/26/2009 - 8:19am

Maybe the 2005 blue point is underneath the red-brown point, i.e. the same value.

by Temo :: Wed, 08/26/2009 - 1:09am

Because Aaron didn't do it, here's the trend of standard deviations of DVOA, Offensive DVOA, Defensive DVOA, and Special Teams DVOA over time. I've found that the standard deviations of DVOA over time exhibits an upward trend-- except for Special Teams DVOA, which has been remarkably the same in variation over time.

As already suggested, I did find a more linear trend than an exponential one, as you can see. The output I achieved can be viewed here:


Edit: Actually, because of the mathematical properties of standard deviations, I suppose (or I think) that my assertion of "a more linear trend" is false. That is to say, my results are not in conflict with Aaron's, considering I'm dealing with Standard Deviations and he's dealing with actual DVOA. So nevermind that part.

by Karma Coma :: Wed, 08/26/2009 - 1:52am

Nice work, Temo. Thanks.

"Profit is limit ONLY by your ability to BANG SPORK"

by Karma Coma :: Wed, 08/26/2009 - 1:53am

double post. sorry.

by Danny Tuccitto :: Fri, 08/28/2009 - 7:09pm

Temo...good work. Any thoughts on why the SDs seem to "wobble" so much? It looks almost like a kind of heartbeat trend wherein there's a between-team expansion and contraction of DVOA variability every season or two. Do you think this is just regression to mean variability, or could there be a more substantive explanation? Might it be another feature of the "adaptation explanation" that most have offered for the raw DVOA trend?

by Bobman :: Wed, 08/26/2009 - 1:13am

Post-cap, as Ghost and Key19 point out in 6 and 8 above, the level of on-field talent is potentially closer across most teams, suggesting that the key varying factor is upstairs. It was amazing when Gruden got $7M to coach (was it him?) but then everybody realized, hell, the cap is only on players, not the staff... That was a smart, proactive move. Didn't lead to too much success over and above what Dungy and Co had achieved the previous 5 seasons, but still, it was a move in the smart direction.

More to the point, if the on-field talent pool has been diluted by 32 teams (which I don't quite buy) the pool of top-notch GMs/player-personnel guys, cap managers, is even more diluted. Since there are only a few great ones to go around, the teams with a Polian, Pioli, etc are the ones that will establish and sustain quality. While the teams that have Millen, or churn through coaching staffs every couple years, are just treading water until they happen to land a future great. One set improves and one declines.

What could the NFL do to re-engineer parity (if they want it)? Oh God, I hate to even suggest this in writing, but limit all non-players to a 5-year limit with any one team. Kind of a Bill Parcells model.... It is purely nuts, but could provide enough turnover of those few who really matter in a post-cap world so that everybody gets a chance. Until the owners figure out some way to overcome that--maybe making their GM a part-owner and thus exempt from the move-along requirement!

by Will Allen (not verified) :: Wed, 08/26/2009 - 1:40am

Hell, Bobman, why not just pay Parcells 25 million a year to go on three year rotations with the worst franchises, with a guarantee of no interference from ownership? Detroit to Oakland to Cincy to Cleveland etc., etc.? And also require that the Master of Sarcasm do weekly press conferences! And Terrell Owens goes with him!!

by Karl Cuba :: Wed, 08/26/2009 - 7:51am

Isn't it a bit soon to crown Pioli's ass, let him win something away from Belichick first.

by Eddo :: Wed, 08/26/2009 - 11:39am

By that logic, you could also say, "Isn't it a bit soon to crown Belichick's ass, let him win something away from Pioli first"?

by Karl Cuba :: Wed, 08/26/2009 - 11:56am

Well you say it then, I know who my money's on to succeed in the future. I've not been very impressed with Pioli, Belichick has a history of being flexible with his schemes to suit the players he has. He's had a 4-3 and then a 3-4 and even now is reportedly considering a 4-3 again because he has better personnel for it. Pioli arrived in KC with 4-3 players, including the 5th overall pick last year and is installing a 3-4 despite noone in the front seven suiting it. He then drafted Jackson 3rd when he could have traded down at least 4 or 5 spots and took another end with his second pick, so it looks like he's given up on Dorsey already. I think Belichick would have done the exact opposite.

in a similar manner, every coach that has left BB has been dire and McDaniels doesn't seem to be off to the greatest start. I think the evidence is mounting that Belicheat is the brains of the outfit. I can just see him watching his young proteges go out into the NFL (and college game) to fail and saying, "How do I reach these Keeeds?"

by Temo :: Wed, 08/26/2009 - 10:41am

But what about teams like the Cowboys and Raiders, where the Owner is already the GM?

I guess what I'm asking is, can we require that Jerry Jones spend 5 years with other teams as well? I'd like this very much.

by tally :: Wed, 08/26/2009 - 1:21am

Yeah, I think variance is probably a better measure of competitive balance.

by A6757575 (not verified) :: Wed, 08/26/2009 - 3:38am

Wrong statistic?

From my impression, the blue graph is next to useless due to the fact that your sample size (one team at the top, one at the bottom) is too small.

Another point might be that any rule changes over the past years have favored the offense (combined with the fact that dominant teams clobber weaker opponents this generates a bigger spread in DVOA[???]).

Look at it from another point of view: The competition in the top - say AFC the past years, which has had at least four very good teams (Steelers, Pats, Bolts, Colts ...) - I think got closer. I think the NFC saw the same, with a group of teams at a similar level at the top. Maybe the gap between the top group and the bottom group (my take: players quit earlier these days on bad teams - do you have a stat between first eight weeks of a season and last four to six weeks?)

I think this statistic is simply crap. You need to use bigger sample sizes and combine a little more different combinations - top eight teams vs. bottom eight et al. It doesn't tell much this way.

by Jeff Feagles is God (not verified) :: Wed, 08/26/2009 - 3:50am

There won't be any competitive balance in the AFC this year either. It will be owned by the usual suspects Indy, NE, Pittsburgh and Baltimore. SD and Miami may sneak in among the elite. Everyone else is the JV squad.

by Jerry :: Wed, 08/26/2009 - 4:29pm

At this point last year, Cleveland was a popular pick to win the AFC North, while (virtually) nobody expected anything from Baltimore's offense. Someone from the "JV squad" (beside Tennessee, which I assume was an oversight) will find their way into the playoffs.

by Scott C :: Wed, 08/26/2009 - 4:07am

It will be very interesting to get a good 4 to 6 years prior to the salary cap in there.

Some have suggested that the salary cap has made things worse, but that is an untestable hypothesis for now.

Perhaps there is a sharp decline between 1991 and 1994?

If thinking about the effect of the salary cap, isn't it most wise to look at the upper 2/3 or so and drop out the bottom 1/3?
The competitive balance induced by the salary cap has more to do with preventing overly powerful teams, than making poor ones good.
You can't fix an incompetent GM with a salary cap, but you can make top teams give up their top players more often.

Another aspect that is of interest WRT competitive balance is the % of seasons a single team has in each performance bucket. How many teams can say that they have spent about equal time in different performance buckets (top 20%, next 20%, etc)?

by edswood (not verified) :: Wed, 08/26/2009 - 5:15am

Fascinating stuff. I most agree with Key19 (post 8) theory. I also think this trend will continue and get worse during the uncapped season, when some teams will spend a ton and all the cheap teams (I am looking at you Cincinatti, Tampa, Jacksonville) spend way less since there is no cap floor. If the cap goes away for good, then I see football ending up like baseball with certain rich teams dominating or at least making the playoffs every year. If this scenario takes place it will be interesting to see how genius certain GMs who wont have the money to keep up like Bill Polian actually are. Will they still be competitive without having the same money to spend as the Cowboys, Redskins, Patriots, and Giants? I just hope this scenario does not come to pass. I would hate having to watch the same teams in the playoffs every year.

by edswood (not verified) :: Wed, 08/26/2009 - 5:19am

BTW Jeff Feagles, you forgot to throw the Titans into your usual suspects list. They seem to be a forgotten team, but they seem to at least make the playoffs on a consistent basis. I like how you threw Miami in there, but even as a Dolphin homer I do not see them moving into elite status with our schedule this year.

by Ed Schoenfeld (not verified) :: Wed, 08/26/2009 - 7:27am

Are the teams at the top in 1994 the same teams that are at the top in 2008?

Wouldn't an assessment of a system that is supposed to promote "competitive balance" by encouraging player movement between teams need to consider that?

by Anonymous Coward (not verified) :: Wed, 08/26/2009 - 9:10am

It would be interesting to overlay this with a line showing the diverging average contract amount given to the top 5 picks in the draft versus picks 27-32.

by nat :: Wed, 08/26/2009 - 9:13am

The trend lines for the best DVOA, second best DVOA, and third best DVOA are essentially flat. The trend for the worst DVOA is steeper than the second and third worst DVOAs, although all three have dropped since 1994.

That's not a surprise, as we've added 4 teams to the league since then. If we compare apples to apples, we get a different story. The trend for the 30th best DVOA shows an improvement since 1995. The gap between the best DVOA and the 30th DVOA has shrunk by about 5%.

So, the answer is "No". Competitive balance has not declined. The league just got bigger.

by Dave B (not verified) :: Thu, 08/27/2009 - 3:27am

I still think Nat's point makes the most sense.

This is the point I brought up in the original 1994 DVOA thread. Sure maybe the increase didn't happen exactly in the expansion years but I think just the fact that there are 4 more teams is the cause for the majority of the difference. Bad management probably accounts for the rest.

If you have more teams, the more likely you are to have ones that hit the farther ends of the distribution. If the size of the league stays constant, I don't think this is a trend that can keep going up indefinitely.

As to what competetive balance means to me is the following:

1. The same teams don't always end up above .500 every season.

-it kind of ruins the excitement of the season if you can predict the majority of the playoff teams before the season even starts.

2. The gap between the best and worst teams is small.

This is tough to acheive unless you have a really large sample of teams and can then regroup them based a on a ralatively contstant skill level( think chess or tennis). It's not really a realistic goal in team based professional sports where so many things are constantly in flux.

Try this thought experiment. If the players on every nfl roster were randomly assigned to teams every season fantasy draft style would the league be more competitive? Probably. We'd definitely see how much coaching matters.

by bravehoptoad :: Thu, 08/27/2009 - 11:19am

No, man. The reason is that the salary cap is four times the size it was in 1994.

by dbrude@gmail.com :: Thu, 08/27/2009 - 7:59pm

so? I don't have the exact numbers but I'll throw out examples.

If he cap was $20 million in 1994 maybe a star player got 4mil a year and a scrub got $100,000

Now the cap is $80 million and a star player costs 16 mil a year and a scrub gets 400,000

It doesn't matter if the cap is 4 times as large if the players are still getting the same percentage of the pie.

Also check out this data


by smilerz (not verified) :: Wed, 08/26/2009 - 9:29am

Not sure how to back it up, but my guess is that it has more to do with coaches than the players.

by Old Europe (not verified) :: Wed, 08/26/2009 - 9:50am

And other than in stocks everyone can name the subprime teames.
Exception: or at least ervery one with no part of part of the teamname in his nickname like denverjim or ...

IMHO it relates to skillfull GMs aka. brainbugs (Bill P, Bill B, etc.) who understand how to play the system and to the rest, maybee as to compare poker tournaments where thousands take part and the usuals suspects end in the top 100.

by nat :: Wed, 08/26/2009 - 10:06am

Aaron's original analysis essentially asks this question:

Why is the gap between the 1st and 32nd place DVOA bigger than the gap between the 1st and 28th place DVOA used to be?

The answer is clear. 32 is greater than 28. No other explanation is needed.

The more interesting question is this one:

Why is the gap between the 1st and 30th place DVOA smaller than it used to be?

by Temo :: Wed, 08/26/2009 - 10:16am

I posted the standard deviation of DVOA over time above. While this can also be explained by expansion (possibly), it's hardly just the difference between 1st and 32nd, or 1-3 and 29-32, that matters.

by nat :: Wed, 08/26/2009 - 11:12am

I agree, up to a point. But it still helps to compare apples to apples.

For example: the integers 1-30 have a standard deviation of about 8.8. The integers 1-32 have a standard deviation of about 9.4. Have the numbers spread out? It is useful to compare the two standard deviations at all?

So the more interesting question about standard deviation is How has the standard deviation of the top 30 (or 28) DVOAs changed?

Also: given that the range of the top 30 DVOAs has decreased, what would an increased standard deviation mean? I think it would mean there are more teams near the top and bottom, and fewer near-average teams. That would mean more competition for playoff spots, and better matchups in the playoffs.

But is that balance? It's matter of taste. For me, balance means more teams with a legit chance at the playoffs and championship this year, and every team having a realistic hope of reaching the playoffs within a few years.

by Temo :: Wed, 08/26/2009 - 1:29pm

In creating your example with the 32 integers, you've already biased the data to have a larger standard deviation.

In the NFL you have (had) 30 different entities that have discrete DVOA values. Lets say these values range from 1-30. In your example, when you add two new entities, their DVOA value is necessarily 31 and 32. In reality, they could have a value of 15, 21, 4, whatever.

In other words, you have created two new values (31 and 32) which are farther away from the mean, whereas the new values could just as easily be very close to the mean.

Now, there are simple statisitical tests to check the difference in standard deviations between two different sized samples (null hypothesis: difference=0). I have the data set at home and not in office, obviously so I couldn't perform those tests at the moment.

by Independent George :: Wed, 08/26/2009 - 11:20am

This borders on a political theory and therefore is in danger of violating the Prime Directive, but the entire debate seems to center on whether you define parity as 'equality of opportunity' or as 'equality of outcome'.

The salary cap pushes the league towards an opportunity model; team operates under the same salary constraints, but not every team has the same management. More importantly, the structure of the cap and dead money charges tend to create a positive feedback loop; for example, the guaranteed money on a single overpriced free agent or 1st-round bust reduces the amount you can spend on other needs, and requires additional resources to correct the original mistake. Over time, the best organizations will get better, and the worst will get worse, even if the difference in ability is very slight at the start.

The draft is intended to push towards the outcome model; by granting the worst teams the highest picks, the idea is that they have an easier time stockpiling young talent. The problem is, once again, that talent evaluation and organizational structure still matters more than draft position. The amount of money reserved for the top picks means that the worst teams might have a deeper pool of talent to select from, but they also face much more downside risk due to the salary effects (hence, the Loser's Curse theory).

I prefer the opportunity model, but think that the rookie salaries really puts the worst teams in an unnecessarily difficult position. The worst teams typically need superstars less than they need depth; the way rookie salaries are structured, that means the very worst teams essentially have to devoted larger portions of their payrolls to untested rookies, and less to veteran backups. The badly managed teams are still going to find a way to screw up no matter what, but the rookie salaries gives them more opportunities for bigger screwups.

by Karl Cuba :: Wed, 08/26/2009 - 11:47am

Interesting, that would suggest that the most equitable solution to the burden of having a high draft pick would be to instigate a slotting system where rookies are paid standardised contracts on a sliding scale and then not counting those contracts in the salary cap.

by Rich Conley (not verified) :: Wed, 08/26/2009 - 11:50am

"I prefer the opportunity model, but think that the rookie salaries really puts the worst teams in an unnecessarily difficult position. The worst teams typically need superstars less than they need depth; the way rookie salaries are structured, that means the very worst teams essentially have to devoted larger portions of their payrolls to untested rookies, and less to veteran backups. The badly managed teams are still going to find a way to screw up no matter what, but the rookie salaries gives them more opportunities for bigger screwups."

Have you noticed that when the better teams happen to stumble onto top 10 picks, they either grab QBs, or they trade down? Losing teams can trade down. They just chose not to. Its not an issue of the system, its just more bad management.

by Mr Shush :: Wed, 08/26/2009 - 1:01pm

QBs like Richard Seymour and Jerod Mayo? Or like Jake Long? Terrell Suggs? The point is that good front offices tend to pick better players on the rare occasions that they have top ten picks, not (primarily) that they have more sophisticated strategies.

by Jon Coit (not verified) :: Wed, 08/26/2009 - 4:09pm

Sure, it's important not to minimize incompetence (although an earlier post pointed out that coaches/GMs on bad teams have perverse incentives to prioritize short-term and neglect the long term). But isn't it also true that, precisely for the reasons identified, teams are reluctant to trade up in the first round?

by Ben Rapelesberger (not verified) :: Wed, 08/26/2009 - 8:16pm

"But isn't it also true that, precisely for the reasons identified, teams are reluctant to trade up in the first round?"

I've seen no evidence that its actually the case. What I believe is that bad teams are overly reluctant to trade OUT of the early first round, whether thats because of fear of fan revolt, or whatever.

Bad teams think the first 10 picks are worth much more than they actually are.

by The Ghost of Tiki Barber (not verified) :: Wed, 08/26/2009 - 12:30pm

This is probably a crazily bad idea, but hear me out...

We've brought up a slotting system for rookies. I have a similar idea--why not pay them all the same, and have the league itself pay an additional bonus depending on where you're drafted? So while the #1 pick and Mr. Irrelevant will have the same base salary for 3 or 4 years, the #1 pick has a fat supplement (also slotted) courtesy of the league. This would accomplish two things: It would preserve the financial incentive/windfall for rookie talent, while at the same time it would keep teams from getting saddled with financial pain for a high-pick bust.

So, let's say the rookie minimum was $1 million. The 1st overall pick might get a bonus of $6 million, while Mr. Irrelevant might get a $250K bonus with a bag of fries, or something. Either way, their team is only on the hook for the rookie minumum thanks to the league bonus.

by Jimmy :: Wed, 08/26/2009 - 1:32pm

Nice idea. An alternative might be to pay each rookie from a round the same amount (ie all first rounders get say $4m per year) and have to sign three year deals and then have the league top up the salary based upon the amount of playing time each player gets. The league top up bonus would not count against the cap and would work similarly to the existing scheme by which very low paid players are currently given additional compensation.

Or simply have all rookies get three year deals for the minimum salary and top up on a per play basis. The vast majority of NFL players get best paid on their second deals anyway and all players would reach free agency after three years. That way teams would want the top picks in the draft and it could actually drive parity. The players would get paid for what they did in the NFL and not college. If you want teams to stop stockpiling talent they don't need then take all restrictive tags away - if the player felt he will be better used elsewhere after three years he will find somewhere where he could play.

by The Ghost of Tiki Barber (not verified) :: Wed, 08/26/2009 - 10:59pm

The only problem with paying players on a per-play basis is that it introduces a pernicious influence on the process. Ideally, you want players being developed by the trained coaching staff on a properly determined schedule. Players who accrue X number of dollars for every play will be understandably anxious to play as many snaps as they can, whether they're ready or not. It would add another layer of consideration for coaches--can we literally afford to play this rookie? Don't think agents won't do what they can to affect the process. Players themselves would have an incentive to lie about their health or their readiness to play. And how do you handle injured players? What about a rookie QB who would be better served by holding a vet's clipboard for a year? Players should play, coaches should coach, and GMs should worry about money.

by Eddo :: Wed, 08/26/2009 - 11:36am

Two points about the graph itself:

1. It would be nice to have vertical lines at the three expansion points.

2. Rather than using top three and bottom three for the reddish squares, why not use the second, third, and fourth best/worst instead? The idea, try to get rid of potential outliers, is good, but the potential outlier still contributes 33% of that average. Using teams 2-4 at the top and bottom would remove the effects of the best team and worst team entirely, and could potential show less of a trend.

by Flounder :: Wed, 08/26/2009 - 11:42am

My guess is it's mostly a reflection of a growing difference between the quality of the best and worst starting QBs in the league. I don't think QBs have gotten worse at the bottom, however. I do think that the QB position has gotten significantly more difficult in the last 15 years.

by bravehoptoad :: Wed, 08/26/2009 - 12:04pm

The explanation is simple: the salary cap has gotten higher.

In 1994 the salary cap was $34.6 million. Today, it's almost 4 times that. If the salary cap is so large that good teams can afford to keep all their good players, then it's no longer a balancing force.

That seems to be the case today. Good free agents hitting the open market is a once-in-a-year event. In 1994 that happened all the time.

Teams are essentially today working without a salary cap. If you believe the salary cap was a levelling factor, then that right there accounts for the less competitive league.

by Mr Shush :: Wed, 08/26/2009 - 1:07pm

Bingo. I was going to make exactly the same point. What proportion of teams actually spend up to the cap, or anything like? Insofar as any do, it tends to be those who are carrying tons of dead money thanks to past incompetence, not those with the best players.

by Thok :: Wed, 08/26/2009 - 12:08pm

So, I'm wondering how much of this is actually a pace measurement rather than an actual trend. Since DVOA is an aggregate stat, if teams tend to play faster, they'll accumulate more plays over a season, and the good teams will get more DVOA (since their typical play is above average), while the bad teams will get a lower DVOA (since their typical play is below average.)

(An explanation for why games would tend to have a faster pace seem easier to find, see rule changes, for example.)

by tuluse :: Wed, 08/26/2009 - 12:58pm

DVOA is a rate stat, are you thinking of DYAR? That's just for players.

by Monkey Business (not verified) :: Wed, 08/26/2009 - 12:57pm

What the graph, and the comments, show is that the draft and a good front office are more important than ever. Simply put, everyone was iffy when free agency and the salary cap was introduced. Good franchises adapted and mastered the system, and bad ones didn't. Teams like the Steelers have had remarkable consistency in their coaching, and ownership willing to give them opportunities to prove themselves. Teams like the Lions haven't. It's easy to blame this on the slotting system for first round picks and players like Stafford getting contracts that dwarf established NFL starters before ever picking up the football, but that's the game. If you don't want to pay the guy at the top of the draft Peyton Manning/Tom Brady money, then don't. Give him low guarantees, but plenty of incentives, so if he stinks up the joint you can cut your losses and move on, but if he's a superstar he gets paid like one too. The problem is that everyone kowtow's to the agents, and is unwilling to let a top draft pick go unsigned.

by Theo :: Wed, 08/26/2009 - 7:26pm

There's a better way of researching this:
Play off fluctuation.
Do more teams go from non playoff to playoff and what round. That's why the system is put in place.

by ELTV :: Wed, 08/26/2009 - 8:03pm

Couple thoughts come to mind...

First, as others have suggested, I'm not clear on the definition of 'competitive balance'. Does this mean parity? Every team getting a title (as Waverly suggested above)? Absence of dynasties? Given that this isn't particularly clear, it's tough to answer the question at hand. This is likely worth a post on it's own.

Second, why not just do a median split? Compare the average of the top half v. the average of the bottom half. This addresses your outlier issue completely.

Finally, to address some of the money questions raised above (see Independent George) couldn't this in some way be scaled by dollar to give a better picture? Maybe salary/DVOA/players? You'd be getting into a groovy Money Ball-esque analysis. Do some teams overpay for DVOA? Hoard DVOA? Is DVOA evenly distributed around the league? Market efficiency, baby, yeah!

by Ben Rapelesberger (not verified) :: Wed, 08/26/2009 - 8:21pm

"Finally, to address some of the money questions raised above (see Independent George) couldn't this in some way be scaled by dollar to give a better picture? Maybe salary/DVOA/players? You'd be getting into a groovy Money Ball-esque analysis. Do some teams overpay for DVOA? Hoard DVOA? Is DVOA evenly distributed around the league? Market efficiency, baby, yeah!"

They ran that article like a month ago. Teams pretty much all spend the same amount of money, so all it tells you is who has the highest DVOA.

by ELTV :: Wed, 08/26/2009 - 8:32pm

Excellent. New to the site, so I'll go look for it. If'n you got a link, I'll take it. Thanks.

by tuluse :: Wed, 08/26/2009 - 9:53pm


Most all the under the cap articles are very good.

by Danny Tuccitto :: Thu, 08/27/2009 - 12:58am

Hey Doug...it's Danny from Niners Nation. Just wondering why you used ranges rather than standard deviations of yearly DVOA. Plot the SD of DVOA for each year and tell us if the graph looks the same. I'm not assuming the answer's going to be different from what you've presented here, but just think it's a little better methodologically. Unless of course normalizing DVOA creates some artificial SD. If that's the case, then just ignore me. :-)

by Danny Tuccitto :: Thu, 08/27/2009 - 12:58am

Hey Doug...it's Danny from Niners Nation. Just wondering why you used ranges rather than standard deviations of yearly DVOA. Plot the SD of DVOA for each year and tell us if the graph looks the same. I'm not assuming the answer's going to be different from what you've presented here, but just think it's a little better methodologically. Unless of course normalizing DVOA creates some artificial SD. If that's the case, then just ignore me. :-)

by Danny Tuccitto :: Thu, 08/27/2009 - 1:08am

Ooops...sorry Aaron. Was replying to one of Doug's posts when I started writing this one.

p.s. I should add that, to me, the most interesting part of your graph is the accelerating trend. It would make perfect sense if the trend was a simple positive linear one given that it's obvious to attribute decreasing competitive balance to some kind of assimilation effect whereby certain teams have adapted better to changes in rule/league structure over the past 15 years. However, the acceleration in the trend is a real mystery, but it begs a very obvious question: Is the acceleration due to the best teams getting increasingly better from year to year, the worst teams getting increasingly, well, "worser" from year to year, or both?

by Bowl Game Anomaly :: Thu, 08/27/2009 - 10:42am

I think the acceleration of the trend is overstated. It's a shallow curve, especially looking at the 3-team averages, and it's not hard to see that a linear function could fit equally as well.

by Dan :: Thu, 08/27/2009 - 10:12pm

Temo did that - follow his link in comment 14.

by Danny Tuccitto :: Fri, 08/28/2009 - 7:02pm

Thanks for the heads up.