Writers of Pro Football Prospectus 2008

07 Sep 2010

FO Basics: Regression Towards The Mean

by Bill Barnwell

Please click here for the other articles in the "FO Basics" series:

  • August 30: Where our stats come from, and the difference between charting stats and play-by-play stats.
  • August 31: A summary of research from our first seven years.
  • September 1: Our college stats, how they differ from our NFL stats and from each other.
  • September 6: The importance (and limitations) of watching games on tape.

Regression towards the mean. It's a phrase we use a lot at Football Outsiders, describing a concept that's probably easier to explain in a sentence than in two paragraphs. It's most often employed when looking forward at how a team, unit, or player will do in an upcoming season, which makes it a wonderful little lightning rod for controversy: What happened last time is unlikely to happen again.

In this latest entry in our "FO Basics" series, I'm going to take a step back and review the concept of regression towards the mean, explaining what it means and how we apply the concept to football. I'll also try to address some of the reader comments I've seen about regression towards the mean and how it's discussed and implemented at Football Outsiders.

Let's start with the phrase itself. Regression towards the mean implies that a data point is unlikely to happen again, and that the next instance of whatever that data point is representing is likely to exhibit a level of performance closer (regress) to average (the mean).

The simplest example of regression to the mean is flipping a coin. Go flip a coin 10 times. Let's say you get one head and nine tails. Perhaps you got tails 90 percent of the time because no one believed in you, you worked extra hard implementing your coin flipping strategy in the offseason, or your hand wanted tails more than the other one.

Now, go flip a coin 10 times again. Maybe you'll get one head and nine tails again. Chances are, though, that you'll get something closer to what we know is the true mean of flipping an unweighted coin 10 times -- five heads and five tails. You might roll five and five, or three and seven, or eight and two.

Something to clarify here is the difference between regression towards the mean and regression to the mean. It's a subtle difference, but one that we have too often been guilty of misstating. (We're going to try to fix that in the future.) Regression to the mean in a sample of 10 coin flips would suggest that you're likely to pull up five heads and five tails on that second set of flips. That's not very likely at all. Using the binomial distribution, we know that the odds of you getting exactly five heads and five tails in 10 flips is just 24.6 percent. The odds of you getting more than one coin heads-up in 10 flips, though, is better than 98 percent. Regression towards the mean is extremely likely; regression to the exact mean is pretty unlikely.

In football, the concept is more complex. Very few things are a 50-50 shot, the way that flipping a coin is. Fumble recoveries are a good example. While what the league refers to as aborted snaps are recovered by the offensive team about 80 percent of the time, the vast majority of fumbles have been proven to be about a 50-50 proposition. Take sacks: In 2009, 94 of the fumbles caused by sacks were recovered by the offense and 93 were recovered by the defense. On running plays, the offense recovered 73 fumbles, while the defense got to 108. That seems like the defense is recovering a larger portion of the fumbles than expected, but the year before, the offense recovered 88 fumbles on running plays and the defense picked up 91.

That randomness also extends to teams. Although there are myriad stories about teams that place an emphasis on flowing to the football (notably the Lovie Smith defenses in Chicago before they started to suck), teams that recover a particularly small or large percentage of the fumbles that hit the ground in their games in a given season will often see that figure regress toward the mean in the subsequent season.

The past three years provide plenty of examples. In 2007, the Cincinnati Bengals recovered a league-high 70.6 percent of the fumbles in their games (24 of 34). In last were the Baltimore Ravens, who only recovered nine of the 25 fumbles in their games, for a dismal recovery rate of 26.5 percent. (Not that these numbers are based on standard plays, so we're not counting the occasional special teams fumble.)

A year later? The Bengals did regress to the mean, albeit very slightly. They recovered 57.5 percent of their fumbles, which was good for third in the league. No. 1, though? Those Baltimore Ravens. They got 63.3 percent of their fumbles, nabbing 19 of the 30 loose balls. In 2009, Baltimore was 12th (15 of 28, 53.5 percent), while Cincinnati was 17th with an even 50 percent (15 of 30). Last year, Tampa Bay led the league by recovering a staggering 78.8 percent of fumbles, while the Bills recovered a league-low 35.5 percent. The year before, Tampa was 30th in the league and Buffalo was 13th. The year-to-year correlation for fumble recovery rate in 2008-09 was -0.01; in 2007-08, it was -0.07. That suggests that last year's fumble recovery rate has absolutely no predicative value whatsoever.

That leads to another point of clarification: The difference between "regression towards the mean" and what's known as the Gambler's Fallacy, the idea that something is "due." When reading comments about different aspects of our projection system that rely on regression towards the mean, I've seen this come up as a criticism of such a model. I'd like to point out the difference.

As an example, take those 2009 Buccaneers that led the league in fumble recovery rate. When we say that the Buccaneers' fumble rate will regress towards the mean, that doesn't meant that they're due to finish at the bottom of the league this season. We just established above that a team's fumble recovery rate is random from year to year, so there's no way to predict what the Buccaneers' actual recovery rate will be. We do know that teams with that sort of recovery rate in the past have had no ability to maintain that rate, and that the average rate of fumble recoveries for a team is 50 percent, though, so we can safely say that a team with an outlying rate of fumble recoveries is very likely to regress towards the mean in the subsequent season.

This is an example of how difficult it is to build predicative formulas for football. Although the odds of Tampa maintaining that 78.8 percent fumble recovery rate are remarkably slim, we can't project them for anything beyond a league-average fumble recovery rate of 50 percent. They might actually recover 65 percent of the fumbles on the ground and gain an extra two wins because of it. They could also recover 35 percent of the fumbles that hit the turf and lose two extra games.

If this seems too obtuse of an example, consider the coin flip example again. Let's say you're going to flip a coin 10 times. Each time you flip the coin and it comes up heads, I give you $100. Each time the coin comes up tails, you give me $100. In the first run, you flip nine heads! You get $800, and I'm left checking the coin.

Now, let's say that we're going to run the same bet on a second test of 10 tosses. As you might suspect, you shouldn't be counting on $800 again. Although the chance exists that you might flip nine heads again, it's very slim. As mentioned earlier, the chance is less than two percent, so there's better than a 98 percent chance that the coin flipping will regress towards the mean. The most likely scenario -- which happens 24.6 percent of the time -- is that you'll flip five heads and five tails, and we'll each get zero dollars. In the other 75.4 percent of trials, even though you should expect nothing, you may end up getting $200, losing $400, or losing $1,000. Add up all the probabilities and amount of money you'd get from flipping a coin at said probabilities and your expectation is ... zero dollars.

Another somewhat controversial way that we apply the concept of regression towards the mean to our projection system is with injuries. Research we've done on injury rates has suggested that injuries play a dramatically important role in team success, but that the injury rate of teams from year-to-year is random. Recently, though, we've seen teams like Tennessee and Kansas City (and to a lesser extent, Dallas) stay towards the top of the health charts, while teams like Detroit, Cleveland, and St. Louis wallow in injury pity. How should our projection system view the injury rates of these teams going forward?

One way to check this is to see if the year-to-year correlation for injuries has changed. That is to say, do teams that were hurt in a given year stay hurt in the following season? Do they regress towards the mean?

In early 2007, when we started publishing injury research on what later became the Patriots Daily site, the answer was that injuries were almost unquestionably random. The year-to-year correlation for Adjusted Games Lost (our injury metric) by a team's starters from 2002-2006 had ranged from 0.14 (2004-05) to -0.04 (2002-03). This suggests that there's virtually no year-to-year consistency for AGL. If we plug in the actual number of games missed by a team's starters, the range goes from 0.08 to -0.02.

Since then, things have been mostly similar. The table below charts the correlation of starter AGL from year-to-year, including those subsequent seasons.

Table 1: Injury Correlation
Years AGL True Starter GL
2002-03 -0.04 -0.02
2003-04 0.10 0.05
2004-05 0.14 0.08
2005-06 0.01 -0.09
2006-07 -0.01 -0.09
2007-08 0.07 0.19
2008-09 0.32 0.37

Whoa. There's virtually no predicative ability with the previous year's injury rate ... until 2009. We've written about the remarkable weirdness of 2009 before, but here's another example of why it was so strange: For the first time since 2002, injury rates actually stayed somewhat consistent from year to year.

(For the statistically skeptical, I chose 2002 as a cutoff point because it was the year Houston joined the league, not because 2001-02 had anything to do with or against this analysis. And if you're confused about the difference about Adjusted Games Lost and True Starter Games Lost, AGL is a measure that incorporates players who are listed as Probable, Questionable, and Doubtful and their position's historical rate of missing time with those injury designations.)

I don't believe that individual teams are throwing off the research. Dallas is the example most commonly given since we talked about their unlikely run of health in Pro Football Prospectus 2008 and then in Football Outsiders Almanac 2010, but Dallas has really just been a hit-or-miss team as far as health goes. They finished second in AGL in 2009, one of four finishes in the top two over the eight-year span ... but in the other four seasons, they ranked 25th, 21st, 18th (2007) and 17th (2008). Tennessee has ranked in the top three for three consecutive seasons, but they ranged from 11th to dead last over the five previous years. Although the chances are slim that Tennessee's health would be a statistical fluke, it's certainly possible. Detroit had back-to-back top 10 finishes in team health before injury-riddled seasons in 2008 and 2009. Before a legendary FOMBC afflicted them in 2007, the Rams' health had been consistently middle of the pack.

Although we'll never be able to tell Tom Brady to watch out for Bernard Pollard in Week 1 of the 2008 season, we can employ this information at a more micro level. Take offensive linemen. Before the 2009 season, we suggested that the Giants and Jets would suffer more injuries up front. Both teams had just made it through two consecutive years where their five starting linemen had put up 80 starts. From 2001-2008, there had been 25 other instances of a team's five offensive linemen doing that. Only two teams (the 2002-03 Chiefs and Vikings) had been able to pull off two consecutive years of perfect offensive line health. Neither team had made it to three years.

In 2009, the Giants' offensive line suffered injuries and cratered in value, causing a dramatic decline in their running game. Meanwhile, the Jets managed to pull off an unlikely third season of health, with their offensive linemen starting all 80 games for a third consecutive year. The result was an effective rushing attack (11th in DVOA) despite a DOA passing offense.

Does this mean that the Jets have broken the system and that we're expecting them to have zero offensive line injuries again this year? No. From 2001-2009, the year-to-year correlation for offensive line injuries is exactly -0.01. Again, that means that there's just no predicative value in using the previous year's injury rate for the next year's rate. Those 2004 Chiefs only missed one game up front, but the 2004 Vikings missed 18. In 2005, the Chiefs were up to six, and by 2006, they were at 19. On average, those teams that had a perfect season of offensive line health had nine games missed by their offensive line in the next season. The excuses that might come up for why the Jets were able to stay healthy won't pass the most basic of B.S. detectors -- it's not like they were the first offensive line that was nasty or had veterans that knew how to keep their bodies healthy.

Although we were wrong about the health of the 2009 Jets offensive line regressing towards the mean, the odds are very high that a Jets offensive lineman will miss time with an injury this year. Nothing is ever a sure thing when it comes to projecting football teams, but with regression towards the mean, we can say with a reasonably high amount of certainty that something is likely to happen. Or, more accurately, that something isn't likely to happen again.

Posted by: Bill Barnwell on 07 Sep 2010

68 comments, Last at 24 Jul 2012, 1:10pm by CoolAromas

Comments

1
by Jon Frum (not verified) :: Tue, 09/07/2010 - 3:24pm

"Perhaps you got tails 90 percent of the time because no one believed in you, you worked extra hard implementing your coin flipping strategy in the offseason, or your hand wanted tails more than the other one"

That is a terrible analogy - just plain wrong. I can only assume that the writer has no idea what he's talking about. To make that statement you really have to be suffering from the 'a little knowledge is a dangerous thing' syndrome.

3
by Will Allen (not verified) :: Tue, 09/07/2010 - 3:29pm

Yes, it really has to do with how much fun the coin is having out there.

5
by Alexander :: Tue, 09/07/2010 - 3:46pm

How much swagger the coin has is far more important.

7
by Will Allen (not verified) :: Tue, 09/07/2010 - 3:52pm

You are really stuck in an outmoded, cliched, paradigm of coin-flipping analysis, aren't you?

15
by Andrew Potter :: Tue, 09/07/2010 - 4:41pm

Don't be so flippant.

(Ow. It hurt me even to type that, never mind actually click post.)

19
by Micranot (not verified) :: Tue, 09/07/2010 - 4:56pm

Too bad you didn't change your mind, would have made more cents.

23
by Bobman :: Tue, 09/07/2010 - 5:13pm

This has been awesome. You guys certainly can coin a phrase. I saw where this was headed and it didn't tail off... until now.

25
by Will Allen (not verified) :: Tue, 09/07/2010 - 5:35pm

That and three quarters will get you a cup of coffee.

31
by Phoenix138 :: Tue, 09/07/2010 - 6:08pm

If this keeps up I swear I am going to flip out.

33
by Led :: Tue, 09/07/2010 - 6:18pm

Then I bet this thread would stop on a dime.

47
by frievalt :: Wed, 09/08/2010 - 12:20am

You've coined a new way to use that phrase.

63
by dbostedo :: Wed, 09/08/2010 - 12:57pm

This is getting ridiculous. You should all be drawn and quartered.

66
by Tofino :: Wed, 09/08/2010 - 7:14pm

Look guys, cut it out. Let's get back to football discussion using proper football terms like "nickel defense".

8
by DGL :: Tue, 09/07/2010 - 3:54pm

Actually, it's how much swagger the coin flips with.

Edit: Darn it, I knew I shouldn't have wasted my time reading the rest of the article, and just should have jumped down and read the comments for swagger-worthy opportunities.

13
by tuluse :: Tue, 09/07/2010 - 4:23pm

You could say Alexander has deceptive speed.

10
by RickD :: Tue, 09/07/2010 - 3:59pm


The simplest example of regression to the mean is flipping a coin. Go flip a coin 10 times. Let's say you get one head and nine tails. Perhaps you got tails 90 percent of the time because no one believed in you, you worked extra hard implementing your coin flipping strategy in the offseason, or your hand wanted tails more than the other one

It seems more likely that the coin is unbalanced than that one got 9 tails and 1 head in 10 tosses purely by chance. Indeed, the odds of getting either 9 or 10 heads or tails out of 10 tosses is 22/2^10, which is about 2.15%.

The problem here is that one is making assumptions about the property being measured and automatically dismissing unexpected results as necessarily resulting from random chance. You might be on solid ground doing this when talking about coin flips (presuming prior knowledge that the coin was evenly balanced), but that's really not the case when discussing most measurables in sports statistics. One opens the door to fallacies galore.

In sports, the true values of the underlying variables being measured are constantly in flux. For example, if you see a player get 10 sacks a year five years in a row, it may or may not be reasonable to expect 10 more sacks the following year. That could depend a lot on whether the player had a knee injury, or has simply gotten older. The problem with "regression to the mean" is that who knows what the mean is supposed to be??

Let's consider the mean value of sacks achieved by a defensive linebacker. What would that be, 2? 3? 1? Then we see DeMarcus Ware get 11 sacks in 2009. Will he regress to the mean of all linebackers over the NFL? That would imply a decrease in the number of sacks. Or will he regress to the mean number of sacks in his own career? That would imply an increase in the number of sacks.

Unless you're considering a statistic that you have a good reason to believe is determined by chance, there's no reason to expect any regression to the mean.

14
by Wolf (not verified) :: Tue, 09/07/2010 - 4:23pm

What would you consider 'good reason to believe is determined by chance'? It sounds like FO has that reason for the statistics they're measuring. They're definitely not measuring linebacker sacks this way.

18
by Karyn (not verified) :: Tue, 09/07/2010 - 4:56pm

It really matters little what the mean is "supposed" to be - taking your premise that the true mean for each player is not known and that performance is not purely random, that still isn't really their point. I don't think they're saying that all players will have the same average production per year, and that every player has an equal chance of being above or below average in a given year. Clearly some players are more talented than others on a consistent basis.

However, the point is that for some metrics (such as the ones they mention) past performance has little to do with future expectations. For other metrics, such as the sacks number you mentioned, while a player can be above the average consistently, extraordinary performances are unlikely to be replicated consistently.

Think of it as a normal (bell) curve. That may not apply to the distribution of sacks, I have no idea, but it's just an example so work with me. Everyone falls somewhere on the shape of the bell - some high, some low. Maybe one guy (such as Ware, though again I don't know if this is true) records an insane number of sacks in a year - he's at the very top of the bell curve, 3 standard deviations from the mean. Is he likely to be at the very top of the bell curve again next year? No. That's an extraordinary performance that needs a lot of other things to go right.

Is he likely to still be above average in the next year? I would say yes, but his performance is much, much more likely to be closer to the average than it was the year before.

It matters little what the average is, or if outcomes are random. You can have players consistently outperform the average. You are unlikely to have players who consistently outperform the average by an incredible margin, and that's what the regression towards the mean axiom (especially for statistics that have little year-to-year correlation and ARE mostly random) is all about.

24
by PB (not verified) :: Tue, 09/07/2010 - 5:24pm

Does FO's use of regression to the mean apply to player performances? The article mentions injury rate and fumble recovery, and that both those things have no year-to-year correlation. The inference (I think) is that if teams had any control over those two things, then the teams that are good at recovering fumbles one year would recover more fumbles the next year. Since we don't see any such correlation, we can assume that fumble recoveries are as random as a coin flip.

I don't think that applies to player performances. We may assume that Chris Johnson will not rush for 2,000 yards again, because rushing for 2,000 yards is really hard and no running back in NFL history has been good enough that we should expect them to rush for 2,000 yards, but Chris Johnson's performance is not as random as a coin flip and we should not expect him to become an average running back next year just because he was unusually good last year. Likewise, the 2007 Patriots may have been lucky, but they were also really good, and assuming they would regress towards the mean assumes the Patriots were the same as every other team.

So a team that went 12-4, but had average fumble recovery rate and an average number of Adjusted Games Lost (like the Manning-era Colts, maybe, since they go 12-4 or better every year) might not be expected to regress towards 8-8, since they were not a particularly lucky 12-4 team. If such a team gets average luck again, they would finish 12-4 again. They might be expected to finish worse than 12-4 because of Bill James's Plexiglass Principle, which I think says it is hard for a professional team to stay above or below .500 for an extended period of time, but I don't think that is regression towards the mean in the way the article uses the term.

36
by Sander :: Tue, 09/07/2010 - 6:59pm

You've just described exactly what FO does. Regression toward the mean is everpresent - but the mean is different for every player. And that's how FO looks at its statistics, from what I know. Peyton Manning is a great player, but he was extremely unlikely to repeat his 2004 performance. So he regressed toward the mean.

FO is not going to predict Chris Johnson to regress and become a league-average back. They are predicting that Chris Johnson will be one of the top backs, but that won't be enough to hit 2,000 yards again.

53
by bravehoptoad :: Wed, 09/08/2010 - 10:30am

...but the mean is different for every player.

This is my least favorite thing about regression to "the" mean -- it implies there's only one. I wish everyone instead used regression to "a" mean, because then the writer would feel obligated to state which mean we're regressing towards.

Almost never, on this site or anywhere else, do people ever say which mean they're talking about when they use "regression to THE mean." Sometimes it's obvious, and sometimes it's a mask for fuzzy thinking. I often feel like I'm grading freshman comp papers again, wanting to circle and ask for clarity in the writing.

Take something like: "Denver's line play is bound to regress to the mean this year." Sometimes this mean might be the Denver line's expected rate of play if you ran them through 10,000 seasons. This can be interesting if the writer thinks there's some reason Denver's line outplayed itself. Sometimes the writer is just talking about regressing toward the league-average mean of offensive line play. This is usually just sloppy thinking.

The next time you see "regression to the mean," ask yourself which mean the guy is talking about. In my experience, it's usually a coin flip that anyone knows.

22
by GnomeChumpsky (not verified) :: Tue, 09/07/2010 - 5:02pm

You make the most important point of all when modeling something as complicated as sports performance. What is the underlying probability? Players get better and worse, from week to week and year to year. We will never know if the coin is biased or it just experienced an unlikely result based on 10 flips.

28
by Briguy :: Tue, 09/07/2010 - 5:38pm

The simplest example of regression to the mean is flipping a coin. Go flip a coin 10 times. Let's say you get one head and nine tails. Perhaps you got tails 90 percent of the time because no one believed in you, you worked extra hard implementing your coin flipping strategy in the offseason, or your hand wanted tails more than the other one
It seems more likely that the coin is unbalanced than that one got 9 tails and 1 head in 10 tosses purely by chance. Indeed, the odds of getting either 9 or 10 heads or tails out of 10 tosses is 22/2^10, which is about 2.15%.

This is my particular favorite statistical fallacy, and is the crux of the argument someone is making when they say things like "Lovie Smith's teams just want it more." "Since it's extremely rare that this would happen by random chance, there must be something different about the case where it does happen." This isn't true, though. For each individual flipping a coin 10 times, yes, there is a very small chance of getting heads 9 times. If you line 10 people up and have them each flip 10 coins, the chance is the same for each individual, but the chance of it happening to someone increases dramatically. By the time you have 100 people in a row each flipping 10 coins, there's a very good chance SOMEONE is going to hit heads 9 times. When that happens, he's going to give a shout, and the people watching are going to ooh and ahh and try to explain why that person is better at flipping coins than everyone else. Then he's going to try again and go 5 and 5.

In football terms, this is what happens when the talking heads say "Team X is better than Team Y at effect Z." Sometimes effect Z is a real skill, and Team X is actually better at it. In that situation, there is no expected regression toward the mean, and FO doesn't incorporate regression in its projections. Sometimes, Team X is the 2009 Bucs, who just flipped heads 9 times in a row. They're not likely to do it again, but that doesn't stop non-statisticians from saying they're better at it than the other teams.

FO has all these statistics for something like 550 teams over the last 17 years. There are going to be outliers. There are going to be times when the same team is an outlier two or three times in a row. That doesn't mean there's something different about those teams with respect to the random occurrence you're measuring.

This is one of the reasons FO is so valuable when evaluating teams. They can research whether an effect is a true skill (linebacker sacks) or a result of random chance (fumble recoveries) in a way that television commentators using standard statistics can't--to use your words, they have "good reason to believe [effect Z] is determined by chance." A reader savvy in statistics who can understand the difference between skill and chance is going to get a lot of value out of that kind of analysis, while someone who doesn't understand worthwhile statistical analysis is going to pooh-pooh them and say there's something happening that FO's statistical models don't understand.

59
by MJK :: Wed, 09/08/2010 - 11:29am

Thank you. This is one of the biggest things that most people don't understand about statistics.

Outliers are actually expected (and indeed, often an indication that a process is truly random). But they get noticed more, because they're outliers, so people tend to ascribe too much value to them.

If injuries are truly random, then, out of 32 teams in the league, you EXPECT a couple of them to have very few injuries each year, and a couple to have a lot. If every team had close to the average number of injuries, this would be evidence that something systematic was causing a certain number of injuries.

But because no one notices when a team has a typical number of injuries, they always are drawn to the outliers and think it's pointing to something systematic, when in fact it implies nothing of the sort.

44
by JR (not verified) :: Tue, 09/07/2010 - 10:47pm

The "mean value of sacks achieved by a defensive linebacker" is an NFL-wide statistic, so it's invalid to compare it to the mean value of sacks achieved by DeMarcus Ware. Those are two very different variables with different means. And while the "sacks achieved by DeMarcus Ware" is a highly correlated (year to year) and non-stationary variable, the NFL-wide number of sacks achieved by a defensive linebacker may be at least weakly stationary (meaning that the underlying mean and variance do not change from year to year).

You could try to predict the number of sacks achieved by DeMarcus Ware, but you would not use the league average to do so.

27
by Theo :: Tue, 09/07/2010 - 5:37pm

Lets try prayer.

68
by CoolAromas (not verified) :: Tue, 07/24/2012 - 1:10pm

Dear Jon Frum:

He is being sarcastic when he states that he somehow influenced the flip.

And it is clear that you know nothing about probability as opposed to chance, or the law of large numbers.

In every aspect of life, performances always head towards an "average" or mean level.

2
by Will Allen (not verified) :: Tue, 09/07/2010 - 3:26pm

Everybody knows that injuries depend on whether The Great and Terrible Injury God is angered or pleased.

4
by DrewBrees4MVP :: Tue, 09/07/2010 - 3:43pm

Football has too many factors to determine with any level of intelligence the outcome of a season or statistics that may or may not happen. You can tell me that it's more likely for the Saints not to have 39 takeaways again this year, but that doesn't amount to a hill of beans if they get 40. The bottom line is, statistics in the short term mean nothing. If they meant something, then you would be living on your own private island sipping pina coladas and not writing a column about regressing towards a mean.

6
by Will Allen (not verified) :: Tue, 09/07/2010 - 3:49pm

I really don't know what you are trying to say, but anybody who drinks pina coladas should really just give up booze altogether.

37
by Sander :: Tue, 09/07/2010 - 7:03pm

You mean - statistics can't predict things with 100% accuracy?
Yeah, that's not news to anyone on this site (I hope).

The point of statistical analysis is to create a model of likely outcomes - but likely isn't the same as certain, and people who dislike statistics often dislike them because they can't see a prediction as a probability instead of a certainty.

40
by zlionsfan :: Tue, 09/07/2010 - 9:11pm

You've been living in the world of binary outcomes for far too long. Come, join us in the land of probabilities.

43
by Will Allen (not verified) :: Tue, 09/07/2010 - 9:58pm

Is whisky served?

45
by drobviousso :: Tue, 09/07/2010 - 11:35pm

Mabye!

46
by Will Allen (not verified) :: Wed, 09/08/2010 - 12:06am

Good enough for me!

9
by markomahony :: Tue, 09/07/2010 - 3:55pm

Great series of articles for those of us new enough to the site. Much appreciated.

Although, all the talk of coin flipping makes me wonder if teams keep track of their records in them and base heads/tails calls on reggression towards the mean!!

26
by Fontes of Wayne :: Tue, 09/07/2010 - 5:36pm

That would indicate that they do not understand regression towards the mean.
Indeed, they'd be falling prey to the Gambler's Fallacy.

11
by Holdie (not verified) :: Tue, 09/07/2010 - 4:03pm

What about people who like getting caught in the rain?

21
by Marko :: Tue, 09/07/2010 - 5:01pm

If they're not into yoga and have half a brain, they should escape.

41
by zlionsfan :: Tue, 09/07/2010 - 9:12pm

That's not half bad.

12
by Samson151 :: Tue, 09/07/2010 - 4:15pm

Being from DC, I'm curious what regression predicts for Albert Haynesworth, if he stays in Washington, or if he returns to Tennessee. Just glancing at his career as a starter, appears he has two exceptional seasons ('05 & '08) and a bunch of years where he performed at a much lower level. Albert would blame this on his coaches, of course, but I'm wondering if it's mostly about Albert's famed lack of motivation.

16
by drobviousso :: Tue, 09/07/2010 - 4:47pm

In your team preview interviews, I wish you would state it less like "This team's injury rate will regress to the mean" and more like "This team's past health isn't an indicator of future health, but it did help them win x extra games compared to similar teams with average health". They both say the same thing, but the second is a lot more palatable to, say, Dallas fans.

Tacking on "If you think Dallas really does have some magical anti-injury power, then you should add X wins onto our predictions" would be even better.

I think people are more willing to take the leap of faith that you can explain why someone won or lost more easily than how much they will win or lose in the future.

In fact, I wish more of your predictive content was presented in the form of an trend. "Team X was [good/bad] at [this/that]. This [has/has not] show itself to be a trend, so we [do/do not] expect it to continue next [week/year]."

38
by The Ninjalectual :: Tue, 09/07/2010 - 7:42pm

AGREE 100%!!!!!!!!!!!!!!!!!

48
by rfh1001 :: Wed, 09/08/2010 - 4:28am

AGREE 110%!!!!!!!!!!

54
by bravehoptoad :: Wed, 09/08/2010 - 10:32am

AGREE 90.9090909%!!!!!!!

(This gets us back to 100% from 110%).

60
by MJK :: Wed, 09/08/2010 - 11:31am

But only on average...

58
by roguerouge :: Wed, 09/08/2010 - 11:09am

I'm frankly starting to get curious if magical anti-injury power is a proprietary advance in sports medicine. I'm not saying that there's a magic power that eliminates all injuries, but one that reduces the likelihood of injury -- changes in stretching and warm-ups for example -- or the ability to play while injured -- such as the use of pain killers in a way more unusual and medically unethical than other teams use them -- would also explain the anti-injury power.

So, yes, an outlier may herald a new way of doing things. But you bet on it being an anomaly. Just like in fantasy sports: you don't pick up players the year after their career year, because they're at the peak of their value and you can't show a profit on them based on the draft slot you have to expend to get them.

I really don't see how fantasy football hasn't prepared people to bet against injury luck, really.

65
by Tracy :: Wed, 09/08/2010 - 1:48pm

If there is a magical anti-injury power that Dallas has stumbled onto, I wouldn't expect it to stay proprietary for very long. There's just too much personnel movement from year to year, and word would get out very quickly that Jerry Jones has turned his workout facility into a gigantic hyperbaric chamber, or whatever it is they're doing in Dallas.

17
by nat :: Tue, 09/07/2010 - 4:53pm

I'm disappointed by this article. Regresion towards the mean is an important concept, and Bill got it wrong. It's not that the next instance of a measurement is likely to be closer to the mean. It's only outliers that are likely to be closer to the mean. The farther out the score, the more likely it is to "regress" towards average next time.

It comes up in any experiment that combines an inherent quality with some random factor. The really high scorers are likely to be both good and lucky. The Patriots' undefeated regular season required a great team AND good luck. You didn't have to watch the games to guess that. Even without Brady's injury, you would have guessed against a repeat 16-0.

The same thing happens for a 0-16 team. That's a team that is bad AND unlucky. Next year they might just be bad. Who knows, they might be bad and lucky.

Not so for an 7-9, 8-8, or 9-7 team. They could be bad and lucky, good and unlucky, or just average for skill and luck. You have no basis to guess at their luck from just their record. Even with no change in skill they might move up or down. They don't "regress" toward a mean, because their record doesn't tell us much about how lucky they had been.

The challenge is figuring out the size of the non-predictive, random factor. How far off is a team's record from it's quality of play? Are interception returns more random than repeatable skill? Is there an injury-avoidance skill?

50
by Kulko :: Wed, 09/08/2010 - 7:00am

Totally Agree.

Horrible Presentation of a effect that is complicated enough as it is.

67
by DeltaWhiskey :: Tue, 11/02/2010 - 8:42am

Hey they've said it before, they're writers, not statisticians.

20
by NJBammer :: Tue, 09/07/2010 - 4:56pm

Suggesting for another article: What statistcal analysis cannot tell us.

You have frequently referenced the fact that stats and scouts should go together, and there are some (in fact, many) things which you cannot learn from stats. Why not put all these together in one article?

The limits of number analytics:
1) you cannot predict injuries (inspired by this article)
2) you cannot predict third down conversion
3) you cannot predict red zone conversion
4) you cannot predict FG precentage for or against
5) you cannot predict when an aging WR/RB/OL will break down to the season, just give odds of when it will happen

Maybe if you put all of these "known unknowns" into one neat package, it will clarify for newbies the usefulness of what our "known knowns" are?

29
by Bobman :: Tue, 09/07/2010 - 5:39pm

Wait wait wait....
1) agreed.
2 & 3) Not agreed. What if Team A has a 60% conversion rate for ten tears running and the league average is 50% during that time. I'd say that, all personnel and rule things being held equal, it's pretty likely that Team A will be close to 60% on conversions and the league will be close to 50%. While it's not a psychic-quality prediction, it's pretty useful, no? The outcomes are not random.

This is where I think the RTM discussions were way too loose here in the past. When the Colts would lead the league in 3rd down conversion for a few years running, people would harp about RTM--"they have to come back down soon, right?" But 1st downs are not remotely random. The O calls plays, the D calls plays, based on advance information about their individual skill sets, and their knowledge of the other guys. It's not a coin toss. If it were, then they might randomly call a QB sneak on 3rd and 20. But that doesn't happen. When you sustain excellence in one area for a while, why would a regression to the mean of the greater population be required? Do baseball's greatest hitters start batting 275 in their final seasons? I used to argue that 60% 3rd down conversion (or whatever) WAS Manning's mean. I think the "wisdom of the crowds" series here at FO is a pretty useful exercise--there's a lot that is random and not predictable, like injuries (to guys NOT named Bob Sanders), but general production is somewhat predictable.

4) Half agreed. Assuming no injury or age issues, if Kicker X has hit 87%, 90%, 92%, and 89% the last four years, are you saying next will be totally random for him, with an equal likelihood of 40% as 100%? Or that he has to regress in the direction of the league mean (assume 84%)? I'd say he's on target for 88-91%, despite the league mean being 84% (or whatever). Again, this is not random, if only because of the way kickers are used differently. How often do you hear that Kicker X has not had an attempt beyond 45 yards tis year? Yet another guy, maybe Bironas, has 6 attempts over 50 yards every year? Now which kicker has a higher likelihood of having a lower success rate, even if he is better, simply because he's used differently? Because a team has control over how and when and where a kicker is used, I'd say that's not random.

FG% against you I'd say is relatively random and should track the league mean, with minor differences for different venues--domes SHOULD (no data to back itup, just instinct) have higher success rates than windy ice-bowls like Soldier Field or Meadowlands. If you are a Chicago Bears 1985 style defense, then teams will try to kick and scratch out points from anywhere on the field, resulting in lower FG rates against (especially if your O is suspect and cannot be counted on to take advantage of good field position on a long miss). If you are a T-2 bend-don't-break D, chances are teams will penetrate the 30 and 20 a bunch but stall out before the goal line, making for more short-kick chances and a probability of a higher FG success rate.

5) Agreed, unless injuries are a factor. If a Player X seemed fine up to age 33, then had a major injury at age 34, I'd feel pretty confident predicting a demise at 35.

Now as I re-read your comment, maybe my back of the envelope analyses above are just adjuncts to pure statistical analysis, and your point holds true that statistical analysis alone can't provide these answers.

55
by bravehoptoad :: Wed, 09/08/2010 - 10:35am

Nice idea! A related idea might be "how statistics can be used to measure luck."

30
by Joseph :: Tue, 09/07/2010 - 5:52pm

For the negative, uneducated commenters:

It's like nat (comment #17) said--regression towards the mean is for OUTLIER numbers. If you went to Vegas and placed a bet on Chris Johnson rushing for 2,000 yards this year, I bet you would get about 100/1 odds. Why? Because it's only been done, what, SEVEN (iirc) times in the history of the NFL--and NOBODY has done it twice! It's REALLY HARD, and VERY UNCOMMON.
Now, some player tend to get above-average numbers regularly in a certain category--that's called TALENT, and after about 20 years, we call them HOF'ers. Those who are constantly below average are called "unemployed."
There are stats that are relatively consistent from year-to-year, and FO bases many of their projections/predictions on those. Those which over time have been proven INCONSISTENT (injuries, fumble recoveries, etc), are based upon THE MEAN. Teams/players which had an extremely high or low number/percentage in one of these stats are deemed to "regress towards the mean"--because it is improbable to sustain such a level of greatness/awfulness. Every once in a while, someone does--that only means that their likelihood to "regress towards the mean" is even likelier.
For an example, Drew Brees (accounting for playing all 16 games) will probably throw for over 4,000 yds. & 25 TD's. Why? TALENT. Now, will he complete over 70% of his passes? Probably not--that's regression towards the mean--even though, because of his talent, he will probably complete over 65%--still more than the mean, but not an NFL record.

Cheesh--some of you guys just need to learn how to read--it's not that hard.

32
by Led :: Tue, 09/07/2010 - 6:09pm

There seems to be a disconnect between the empirical fact that average injury rates are not correlated from year to year and the conclusion that particular players who did not suffer any debilitating injuries are likely to suffer such injuries. You can see the problem more clearly if you limit it to one player. At this point it's pretty clear that Favre and Manning have a significantly above average ability to avoid and/or fight through injuries. Let's call this "durability" for lack of a better word. Now 4-5 years ago it would've seemed reasonable to say those guys had been very lucky given the rate of injury to QBs and this luck is unlikely to continue. In fact, given their above average durability, that conclusion probably would not have been reasonable. Is the reasoning any more sound when you're talking about 5 players (the starting o-line) rather than 1 player? Yes, probably. But there's still a chance that the 5 guys happen to be particularly durable. A smaller chance than when you're judt talking about 1 guy, but it's not zero. I think all of this illustrates the risk of applying regression to the mean to particular situations where the "mean" is unknown and unknowable.

34
by tuluse :: Tue, 09/07/2010 - 6:20pm

Just ask the Bears how durable Columbo has been.

Or look at Damien Woody, he has been pretty durable in his career, but he still missed a number of games for Detroit, but stayed healthy for the Jets. Does a light green jersey keep you healthier or was it just luck that he played every game for the Jets while missing some for the Lions?

35
by Led :: Tue, 09/07/2010 - 6:51pm

Right. With Woody's health, it could be good luck for the Jets, could've been bad luck for the Lions. Could be a little of both. A precise answer is probably unknowable. That's why it's dicey to try to project injuries to particular players.

39
by The Ninjalectual :: Tue, 09/07/2010 - 8:17pm

This is interesting. We all agree that avoiding injury is a skill. Perhaps some teams focus on drafting/signing players who they feel also have that skill. Then, we don't have to make wild guesses like "the Cowboys trainers must be above average." Maybe the Cowboys focus on players who know how not to get hurt!

"Just look at that pumpkin."
-John Madden, looking at the moon.

42
by tuluse :: Tue, 09/07/2010 - 9:15pm

There is a theory that Bill Parcells knows how to scout toughness and seeks players with that trait. Which means teams built by him would have lower than average injuries.

49
by Theo :: Wed, 09/08/2010 - 5:47am

I think the use of the phrase 'regress towards the mean' will regress towards the mean, from now on.

51
by JR (not verified) :: Wed, 09/08/2010 - 7:34am

My problem with "regression towards the mean" is that it seems to impute a correlation between past and future events where none exists. Take the coin flipping experiment. If I flip 10 times and get 9 heads then, if the coin is fair, this is an outlier result. It does not mean that I expect this result any less next time than I did this time! The probability of this result does not change if I flip a coin another ten times. So, it is no more likely than the first set of flips to yield the average number of heads.
On the other hand, again if the coin is fair, say I flip a coin 10 times and get 9 heads. Then I flip a coin 10 more times and record all 20 results. I flip again and record all 30 results. If the first result was an outlier (e.g., 9 heads) then we would expect the observed mean over 20, 30, or 40 flips to regress towards the actual mean. That's not to make any prediction about whether any one of the 10-flip experiments produces close to 5 heads. Rather, if in the first experiment I flip 9 heads and in the second I flip 2, then out of 20 flips I have produced 11 heads, or very close to the expected value of 10. Both 9 heads and 2 heads are outliers (with probability differing by a factor of 2). Since the flip experiments are perfectly uncorrelated/independent, I can't make a prediction about the next series of 10 flips. I can, however, make the prediction that as the number of coin flips gets larger (10, 20, 30...) the observed mean will regress to the actual mean. That's the law of large numbers.

56
by GnomeChumpsky (not verified) :: Wed, 09/08/2010 - 10:39am

Very correct (as though correct isn't a binary state)! Regression is too often characterized as having causality, which is a form of the gambler's fallacy (that previous independent results affect the next outcome). It's just as wrong to say that a slot machine is due for a payout as it is to say that a slot machine that just paid out is less likely to pay out again. The underlying probability is the same!

64
by tuluse :: Wed, 09/08/2010 - 1:43pm

The NFL has been around for a while now. It's not a brand new coin every year.

Also, what they're saying is that average luck is predicted every year (5 heads, 5 tails), when you get better or worse luck than that it's a sign of unsustainable success or failure.

52
by dryheat :: Wed, 09/08/2010 - 9:48am

[redacted - wrong thread]

57
by jmaron :: Wed, 09/08/2010 - 10:56am

As an underachiever I prefer the term progression to the mean.

61
by MidnightAngler (not verified) :: Wed, 09/08/2010 - 11:58am

"the odds are very high that a Jets offensive lineman will miss time with an injury this year."

Has any effort been made to estimate injury rates for individual players and/or individual positions? I don't think it would surprise anyone to learn that things like offseason training regimens, genetics, toughness and playing style would significantly impact the injury rate from one player to another. Obviously if you average this over a team or league, these effects should disappear. However, an offensive line is only five guys. If you have 3-4 guys who are particularly well conditioned and willing to play through almost any injury, the injury rate for their line shouldn't be regressing to the league mean. The Jets line has also had a few young guys on it for several years, and youth is a huge advantage in terms of injury recovery and management. Age adjusted injury rates could possibly be quite useful....

The randomness of injuries is probably best exemplified by guys like Peyton Manning and Brett Favre. Due to Manning's playing style and rigorous offseason conditioning program he rarely sustains significant injuries. Due to Favre's stubborn will to play through any injury, he doesn't miss games. These players must be less likely to miss time compared with guys like Ben Roethlisberger who take 50 sacks a year. The same basic ideas ought to apply to other positions.

The difficulties, as I see them, are many. First off, how do you generate a big enough sample size to actually make predictive estimates for individual players? If a player has played two NFL seasons and hasn't been injured, that doesn't provide you with very compelling evidence he won't be injured in year three. You also have the difficulty of quantifying things like "playing style", which is troublesome in itself. Perhaps a starting point would be to estimate injury rates on a positional basis, and work from that. Any thoughts?

62
by justanothersteve :: Wed, 09/08/2010 - 12:43pm

Go flip a coin 10 times. Let's say you get one head and nine tails. Perhaps you got tails 90 percent of the time ... because Phil Luckett was flipping the coin.