17 Jan 2011
No, we aren't suddenly linking to articles about Paul Pierce. I saw this in an old New Yorker today and I found it fascinating. The article discusses how scientists are discovering that statistically valid studies which are well-accepted sometimes can't be replicated with the same results. I think the suggestion is that even the generally accepted standards for statistical validity may be too low, and there's way more randomness involved than any of us ever realized.
This probably has a lot of application to what we do around here. There's no question that some of the things we think we've discovered, we may eventually discover that we didn't really discover. A good example is the "third down rebound," which seems to have disappeared for offenses over the last couple seasons. In a lot of cases, we're already starting with particularly small sample sizes -- and as this article points out, even good-sized samples may be too small.
You also have to remember that when we're studying football, you can't keep all the variables constant. I think it's pretty clear at this point that the Lewin Career Forecast isn't quite as valuable as it was a couple years ago because the screens and high-percentages throws in college football have risen to a ridiculously high level. That doesn't mean it doesn't have value; it just means that it isn't the be all and end all. And it doesn't mean that it wasn't more valid when David Lewin came up with it five years ago; it is possible that circumstances have simply changed.
So I'll have this in mind when I'm writing about things for the next few months, certainly. The problem there is that I can't say that I have this in mind in every single article. It will get really tiring for regular readers. I've always said that I take pride in the fact that Football Outsiders leads the league in couching our opinions. But you can't stick two paragraphs of opinion-couching "what ifs" in every single article. That's a particular problem with picking games. I think 90 to 95 percent of readers would get angry if we characterized every single playoff game as a toss-up, but effectively, even if we think one team has a 65 percent chance of winning, that's really sort of a toss-up. The team we don't pick will win one out of three times, which isn't particularly rare. And frankly, most predictions are even closer than that. Upsets happen. Randomness happens. We may not repetitively write that in every single article, but trust us, we know it to be true.
80 comments, Last at 30 Jan 2011, 6:49am by erniecohen
How much do we tend to know after five weeks? Bill Connelly compares five-week data to full-season data to find out if we should be worried about TCU and Baylor.