Writers of Pro Football Prospectus 2008

Most Recent FO Features

FitzgeraldLar04.jpg

» Scramble for the Ball: The DVOA Schism

Mike and Tom try to figure out what kind of secret sauce Arizona is feeding the media to sit at the top of the power rankings and in the middle of our DVOA rankings.

06 Sep 2005

2005 DVOA Projections

by Aaron Schatz

Welcome to the second annual Football Outsiders DVOA projections. These 2005 predictions are based on a complicated system that combines a number of variables based on performance over the past two seasons in different splits (by down, passing vs. rushing, red zone vs. whole field) plus variables based on recent draft history, coaching experience, quarterback experience, and even weather. The goal is to get the best estimate of which teams will improve and decline in 2005 based on trends from 2000-2004.

(Here's the requisite link to an explanation of DVOA, which stands for Defense-adjusted Value Over Average and measures a team's performance on every play of the season compared to league average in the same situation, adjusted for opponent. I know a lot of people are coming here from various message boards and this is just going to look like a jumble of pointless numbers. Trust me, there is a method to the madness, and over the past five seasons DVOA has been a far more accurate predictor of future performance than wins or points. If you want to get past the nuts and bolts and just see the projections, click here.)

How does the projection system work? Offense, defense, and special teams DVOA are all projected separately using a system based on 2000-2004 numbers. The correlation between projected DVOA and actual DVOA for the past four seasons is .73 for offense, .63 for defense, and .48 for special teams. (For those unfamiliar with statistics, correlation is explained here.)

(You may notice that the "new and improved" system actually ends up with correlation coefficients lower than last year's system. That's because more data means more variation, and we're now building the system off of 127 samples instead of 95. The projections given by last year's system are slightly more accurate for 2001-2003, but far less accurate when you look at 2001-2004 instead. The correlation between projected DVOA from the old system and actual DVOA for the past four seasons is .62 for offense, .52 for defense, and .43 for special teams.)

Then we figure strength of schedule based on the average projected total DVOA of all 16 opponents for 2005. Note that strength of schedule prediction does not include any adjustment for which games are played on the road or at home.

Finally, we project wins based on offense, defense, special teams, and strength of schedule. Last year, I only could project a single "average wins" figure. But the projection system is really giving a range of possible performances, not a single definite prediction. This year, thanks to Dr. Benjamin Alamar from PROTRADE, we have a new system that runs the projections 1,500 times with random variation to determine a range of possible outcomes. This showed up in the book as "Philadelphia has an 80% chance of being a Super Bowl contender." Here, Dr. Alamar has run the season 1,500 times and given us a percentage representing the number of times each team won their division or made the playoffs in that simulation. You can read more about these playoff percentages in this article on the PROTRADE website.

The problem with a purely objective prediction system, of course, is that it can't take into account major changes that aren't reflected in statistics, or major issues that aren't represented by the variables in the system. Last year, I attempted to correct for that with some manual adjustments. This year, there are no manual adjustments -- the numbers we are presenting here are exactly what the projection system spit out. A few of them will look strange to you.  A few of them look strange to me. Of course, last year the one that looked the strangest was the prediction that San Diego would have one of the league's top offenses, and we know how that turned out.

The projections here are the same as the ones in our book with a few changes. I've changed the "QB experience" variable for a few teams, such as Chicago. There are also two changes related to the New England Patriots. There's a variable for offensive coordinator stability, and I had the Patriots at the minimum since we didn't know who the coordinator would be. But Belichick is in effect his own offensive coordinator this year, running the exact same system, so I've changed that variable to the maximum. At the same time, I must have done the book projections at a time when people thought Tedy Bruschi was coming back, and the "Pro Bowl defenders" variable had to be reduced. The effect was to increase the projection for New England's offense and decrease the projection for their defense, which seems realistic. Note that changing the projection for a few teams also changes schedule strength for all the teams.

This year, instead of presenting one big table of projections, I'm going to list each division and then give a little explanation. I'm also going to give the projections in the new "adjusted points per game" format introduced in Pro Football Prospectus 2005 rather than in the old DVOA percentage format. I would like to know if readers prefer this format and find it easier to understand. Adjusted points per game translates DVOA into points using the same ratio as DPAR for individual players, with 0% DVOA corresponding to the 2004 league average of 21.5 points per game. (It says 23.5 in the book, which is a misprint. My fault on that one.) The resulting number gives the number of points per game that a team would score if you separated out the offense from all outside factors. It assumes the team has an average defense and average special teams, running an average number of 61.5 plays per game against an average schedule of opponents. The same formula is also used for the defense's adjusted points per game. Special teams is just listed as points per game above or below the NFL average. The total is then:

Offense's adjusted points - Defense's adjusted points allowed + Special Teams = Total Team Rating


Explanation of Tables


Division: How often this team won division after simulating season 1,500 times.
Playoffs: How often this team made the playoffs after simulating season 1,500 times.
Mean Wins: Average wins for team based on DVOA projections and schedule strength.
Std Dev: This number represents how predictible the team is for 2005, according to the projection system. The lower the number, the more stable the projection. Jacksonville is the lowest team at 61. Arizona is the highest team at 115. (For math geeks, this is standard deviation of wins divided by mean wins, times 100. Arizona's standard deviation was actually higher than its mean wins.)
Adjusted Offense: Projected level of offense, separated from defense, special teams, schedule, and luck, along with rank among the 32 teams.
Adjusted Defense: Same for defense.
Adjusted Special Teams: Same for special teams.
Total: Theoretically, point spread for this team if it played a perfectly average team on a neutral field, with rank among the 32 teams.
Schedule: Average DVOA of all 16 opponents, without considering home field. Ranked from hardest schedule (1: San Diego, 7.8%) to the easiest schedule (32: Seattle, -5.7%).

Team Division Playoffs Mean
Wins
Std
Dev
ADJ.
OFFENSE
ADJ.
DEFENSE
ADJ.
SPECIAL
TOTAL SCHEDULE
NE 39% 56% 9.7 69 26.9 2 21.5 11 0.2 14 5.6 2 4.9% 5
BUF 29% 42% 8.7 93 20.0 23 17.4 3 1.1 1 3.7 6 4.5% 8
NYJ 27% 45% 8.7 83 24.6 7 22.3 18 0.3 10 2.7 10 4.6% 6
MIA 5% 8% 5.6 78 17.4 29 21.8 15 0.6 5 -3.7 28 5.9% 2

There's nothing too shocking about the projections for our first division, the AFC East, except perhaps that the Miami defense only projects to be average. New England comes out the best team, but not by as much as last year, and standard deviation says that Buffalo is the hardest team to project accurately. You might be surprised that New York's offense is so high and its defense so low if you don't either a) own a copy of Pro Football Prospectus or b) read our website regularly. It's one of those subjects that gets beaten to death around here. According to average DVOA of opponent, four of the eight hardest schedules in the NFL are in this division, and the other four are in the AFC West. Of course, those two divisions play each other this year, plus the AFC East plays the NFC South which is probably the best division in the other conference (especially if we are wrong about Atlanta).

Team Division Playoffs Mean
Wins
Std
Dev
ADJ.
OFFENSE
ADJ.
DEFENSE
ADJ.
SPECIAL
TOTAL SCHEDULE
PIT 35% 50% 9.2 84 21.9 15 19.7 6 0.3 9 2.6 11 -1.6% 23
CIN 33% 46% 8.9 73 23.7 9 22.5 19 0.4 8 1.6 13 -0.8% 18
BAL 26% 38% 8.5 89 17.4 30 16.3 1 0.5 6 1.6 12 -1.0% 19
CLE 7% 11% 5.5 95 17.9 27 23.3 27 0.1 17 -5.4 30 -0.7% 17

The projection system likes the Steelers, but only by a small margin. Many people are predicting that Baltimore's offense will take a step forward this year, but the projection system expects the opposite to occur. Cleveland sucks.

Team Division Playoffs Mean
Wins
Std
Dev
ADJ.
OFFENSE
ADJ.
DEFENSE
ADJ.
SPECIAL
TOTAL SCHEDULE
IND 49% 62% 10.4 69 28.4 1 23.2 24 -0.2 21 5.0 3 -1.2% 21
HOU 23% 34% 8.2 72 23.0 10 21.8 16 -0.4 28 0.8 14 1.1% 14
TEN 15% 28% 7.5 76 21.0 20 21.6 14 -0.4 29 -1.1 17 -1.2% 20
JAC 13% 31% 7.5 61 20.7 21 21.6 13 -0.2 20 -1.1 18 1.8% 12

Those percentages for Indianapolis to win the division or make the playoffs seem a little low, but the reason has nothing to do with the Colts. The projection system thinks this is the one division in the NFL where no team is really out of it, so even though the Colts have the highest mean projected wins in the AFC, all three teams have a chance to surprise if things go right. (Numbers aside, I don't expect that to happen.) This is also the reason why the schedule numbers aren't lower even though these four teams all get to play the NFC West.

For most people, the surprising team here is Tennessee. For the second straight year, the few personnel change variables that are in the system underestimate the depth of the Tennessee salary cap cuts, and so the system sees a team that was good in 2003 and expects some rebound. Jacksonville has the lowest standard deviation of all 32 team projections, which is why it has the lowest chance of winning the division but a higher chance than Tennessee to make the playoffs when we include the wild card.

Team Division Playoffs Mean
Wins
Std
Dev
ADJ.
OFFENSE
ADJ.
DEFENSE
ADJ.
SPECIAL
TOTAL SCHEDULE
KC 39% 54% 9.5 64 25.9 3 22.2 17 0.3 11 4.0 5 5.0% 4
DEN 31% 47% 9.3 71 25.9 4 21.5 10 -0.2 23 4.1 4 4.5% 7
SD 20% 29% 7.8 76 24.7 6 23.5 28 -0.4 30 0.8 15 7.8% 1
OAK 11% 19% 6.8 64 22.6 13 24.2 31 -0.3 25 -1.9 22 5.7% 3

The system really likes Kansas City, which was the best team in DVOA in 2003 and tenth last year despite a losing record. However, the team projection system doesn't yet include some kind of team age variable, which is why the individual projections for all the Chiefs in Pro Football Prospectus add up to something a lot lower than this projection of team offense. That's a problem that I will need to fix in 2006. No, I'm not sure why Denver comes out with a lower chance of winning the division and fewer mean wins despite its average opponent being slightly easier. Could this be related to Jacksonville's low standard deviation? They are on Denver's schedule and not Kansas City's schedule. Or perhaps it is just a random quirk in the simulation we ran, and the two teams would have been tied if we ran the simulation another 1,500 times.

San Diego has the hardest schedule in the entire league and the system thinks the defense will fall apart again, even though the offense stays strong.

Team Division Playoffs Mean
Wins
Std
Dev
ADJ.
OFFENSE
ADJ.
DEFENSE
ADJ.
SPECIAL
TOTAL SCHEDULE
PHI 66% 83% 11.7 91 24.1 8 16.3 2 0.8 3 8.6 1 -4.0% 27
DAL 14% 39% 8.0 77 20.4 22 20.3 7 0.2 16 0.3 16 1.8% 11
NYG 13% 30% 7.0 72 22.1 14 24.3 32 0.2 12 -2.0 23 1.4% 13
WAS 7% 22% 6.5 80 18.9 25 21.1 8 -0.2 24 -2.5 24 2.1% 10

It's Philadelphia's world and we're just living in it. They are the only team with more than ten projected mean wins. They are the only team that projects to be in the top ten with all three units. They have one of the league's easiest schedules. As we noted in the book, the DVOA win projection system gives Philly a 4.6% chance of a perfect season. (This is part of why the Eagles have such a high standard deviation; nobody else has a realistic chance to win 15-16 games.)

Dallas comes out pretty well, because its defense was good two years ago and some rebound is expected, plus the draft variable likes those two first-round defensive draft picks (though that doesn't seem to help San Diego much). The Giants are interesting. We've been down on Eli Manning but excited about the return of Michael Strahan and the signing of Antonio Pierce. The numbers seem to expect the opposite, that New York's offense will improve but their defense will become the league's worst. One big part of that: poor red zone defense against the pass, which is something that tends to carry over from year to year. Washington has a shot at a wild card, but doesn't everybody in the NFC?

Team Division Playoffs Mean
Wins
Std
Dev
ADJ.
OFFENSE
ADJ.
DEFENSE
ADJ.
SPECIAL
TOTAL SCHEDULE
DET 33% 42% 7.8 75 21.7 17 24.0 29 0.6 4 -1.7 21 -5.7% 31
GB 32% 43% 7.7 69 22.6 11 24.1 30 0.2 13 -1.2 19 -2.2% 24
MIN 24% 32% 7.0 72 18.7 26 21.5 9 -0.6 31 -3.3 26 -5.2% 30
CHI 11% 15% 5.5 95 16.7 31 23.1 22 0.2 15 -6.2 31 -5.0% 29

Yeah, this is a mess. The mean projection is under 8-8 for all four teams in the NFC North. Those easy schedule ratings are mostly caused by the teams all playing each other. The weird projection here is Minnesota. All summer long I've been saying that the problem with the projection system is that it tends to miss teams that make far more personnel changes than normal. But the Vikings project to have a huge improvement on defense and a huge collapse on offense despite this problem. On defense, the draft variable comes out high and the system likes that the Vikings were actually better than average last year against passes in the red zone. On offense, the projection system foresees trouble because the Vikings were the best offense in the league on third downs last year, an indicator for decline in 2005. The Vikings also have a new offensive coordinator.  This brings up the question, if I was going to consider Bill Belichick to be a continuation of Charlie Weis in New England, why not consider Steve Loney to be a continuation of Scott Linehan in Minnesota (since he was on Linehan's staff, and the system will stay the same).  In retrospect, perhaps I need to look at whether the "new offensive coordinator" issue only applies when the head coach is also new. But even if you gave the Vikings credit for "veteran offensive coordinator," that would just move them up to the same place as Detroit and Green Bay.

Team Division Playoffs Mean
Wins
Std
Dev
ADJ.
OFFENSE
ADJ.
DEFENSE
ADJ.
SPECIAL
TOTAL SCHEDULE
TB 42% 63% 9.8 80 22.6 12 19.0 5 -0.2 22 3.4 8 -4.6% 28
CAR 40% 61% 9.7 77 21.6 18 18.0 4 -0.3 27 3.2 9 -3.1% 26
NO 14% 31% 7.4 72 21.1 19 23.2 25 0.8 2 -1.3 20 -1.2% 22
ATL 4% 10% 5.2 77 17.9 28 23.2 23 0.5 7 -4.9 29 4.4% 9

OK, which of these wacko projections do we start with first: Tampa Bay being this good, or Atlanta being this bad?

Let's start with Tampa Bay. As we acknowledge in the book, Tampa Bay is the one team that, throughout the two years that we've been doing this, has always come out better in our metrics than on the field. The projection system sees a team that was 16th in DVOA last year and 8th the year before. That team also has a stable coaching staff and a veteran quarterback who improved over the course of the 2004 season. It all comes out to give the Bucs a high rating and, thanks to a schedule with all those mediocre NFC North teams plus the 49ers and the Redskins, a very high chance of making the playoffs. Nobody around here thinks Tampa Bay is really this good, but you can't write them off.

It is hard to understand why Tampa Bay's numbers continue to be this good, but there are a lot of easily explainable reasons why Atlanta's numbers are this bad. Start with the fact that Atlanta wasn't really that good in 2004: 19th in DVOA despite going 11-5, and they only outscored their opponents by three points. The offense projects to be terrible because passing is more important than rushing, and there's nothing in the numbers from 2003-2004 to suggest that Michael Vick is suddenly going to "get it." The defense projects to decline because it was so much better on third down last year than it was overall. But Atlanta's biggest problem is that schedule. Carolina is going to be better. Tampa Bay is, at least according to our projection system, going to be better. Atlanta plays Philadelphia and Seattle while its division rivals each get two much easier opponents from those divisions. Plus, like the other teams in this division, they play the AFC East. This schedule rating doesn't even capture the true difficulty of Atlanta's schedule because it is just an average of each opponent's DVOA, and doesn't recognize that even Atlanta's easy games are hard: It goes to Detroit on Thanksgiving and Chicago on a cold December night.

Is there reason to believe that Atlanta will not collapse dramatically? Well, the offensive projections only include variables from 2003 and 2004, and Vick was pretty good as a passer in 2002. (Seriously, what the hell happened?) Ed Hartwell is a pretty nice addition to that defense to make sure that it improves on first and second down even while it probably declines on third down. Carolina always seems to have trouble with Vick, Tampa Bay is not really this good, and beating Atlanta isn't going to be the most important problem for the New Orleans Saints this season.

Then again, if Vick takes one too many hits while he's freelancing, and Matt Schaub is starting by Thanksgiving, Atlanta might actually be worse than this.

Team Division Playoffs Mean
Wins
Std
Dev
ADJ.
OFFENSE
ADJ.
DEFENSE
ADJ.
SPECIAL
TOTAL SCHEDULE
SEA 60% 68% 10.1 85 25.5 5 21.6 12 -0.3 26 3.6 7 -5.7% 32
STL 19% 30% 7.0 83 21.7 16 23.3 26 -1.0 32 -2.6 25 -2.4% 25
ARI 14% 22% 6.4 115 19.4 24 23.0 21 0.0 18 -3.5 27 -0.5% 16
SF 7% 10% 4.7 81 16.2 32 23.0 20 -0.1 19 -6.9 32 0.1% 15

The projection system thinks the Seahawks win this division easily, although you'll notice that all four teams have high ratings for standard deviation. To be honest, I don't think Seattle's defense is really above average, nor do I think the Rams really will have an average offense. The system sees that Seattle declined on defense last year, while the Rams were actually better on offense compared to 2003, and therefore expects some reversal by both teams. Arizona's standard deviation of projected wins is actually higher than its mean projected wins. San Francisco projects to be the worst team in the NFL yet again.

Note that these are not the "official" predictions, per se.  There really aren't "official" Football Outsiders predictions. Like last year, there will be an article on Thursday morning with predictions and commentary from the entire Football Outsiders staff to go with these statistical projections.

Posted by: Aaron Schatz on 06 Sep 2005

130 comments, Last at 05 Oct 2005, 3:38pm by james

Comments

1
by Pat (not verified) :: Tue, 09/06/2005 - 11:03am

Here, Dr. Alamar has run the season 1,500 times and given us a percentage representing the number of times each team won their division or made the playoffs in that simulation.

Ah, yes, the beginnings of real statistical football studies. Monte Carloed seasons. Gotta love it.

I can imagine future improvements taking into account the relative frequency of injury of major players, and the quality of their backups. Interesting possibility: I wonder if injury rates are constant versus opponent, or if they're correlated with the strength of an opposing defense. Is it less dangerous if you played St. Louis last year, when they couldn't tackle a tackle dummy?

That's an intriguing possibility. Could mean that difficult defenses are worse to face than strong offenses.

(If you’ve been reading us for a couple years, you may notice that the "new and improved" system actually ends up with correlation coefficients lower than last year’s system, despite the fact that the new system is built on more data and therefore, supposedly, more accurate. The reason for this is the constant shifting of the overall NFL offensive environment over the last couple of seasons

OK, so granted that special teams are ridiculously variable anyway. But shouldn't the special teams predictions be immune from this effect? Special teams seems to be disjoint from offense/defense, so I don't see how a pass-heavy offense could make a team's kicking game go down.

Or is the special teams correlation coefficient actually steady?

2
by B (not verified) :: Tue, 09/06/2005 - 11:42am

Monte Carlo? I'd much rather have a Monte Christo.
Considering the fact that special teams vary from 1.1 to -1, I don't thine they're having much effect on the overall projections.
How did Detroit's projection get so high? I guess we can assume the projection system doesn't watch pre-season games. Seriously, maybe three years of sucking at QB shouldn't count as "vetran QB experience" Maybe it doesn't, that would explain why Baltimore is still projected to be lousy on offense.

3
by BlueStarDude (not verified) :: Tue, 09/06/2005 - 11:58am

re: "Interesting possibility: I wonder if injury rates are constant versus opponent, or if they’re correlated with the strength of an opposing defense."

Good question, that would be very interesting to find out. While the Dallas defense stunk it up last year, it did seem like Roy Williams knocked someone out of the game every other week (the most visible being TO).

re: "Dallas comes out pretty well, because its defense was good two years ago and some rebound is expected"

Does the change from 4-3 to 3-4 factor in at all? Though 6 of the starters from 2003 remain, only 3 of them will start in the base defense.

4
by Ken (not verified) :: Tue, 09/06/2005 - 12:19pm

I think you're underestimating the loss of Weis. The NE offense is going to be vastly different. If you want see what a Belichick offense looks like, you have have to look to the 91-95 Browns. And yes, while I'm here... Weis made Tom Brady, 'the' Tom Brady... the 1998 Vinny Testaverde, 'the' 1998 Vinny Testaverde and he will make Brady Quinn, 'the' Brady Quinn... That said, Go Irish.

As for the argument that the Jets have a tougher schedule, so they won't bode as well this year... doesn't the same apply to the rest of the AFC East?

5
by Aaron (not verified) :: Tue, 09/06/2005 - 12:26pm

Nope, 3-4/4-3 is not factored in. That's one of those reasons why these are not the "official" predictions.

Ken, you might be right about Weis, we'll see. Not sure where you see the argument that the Jets have a harder schedule than the other three AFC East teams.

Re: Pat's comment. After checking a little more, it turns out the correlation is in fact a *lot* stronger this year than last year. The problem was that I was comparing how this year's system correlates with 2001-2004 to how last year's system correlated with 2001-2003, not 2001-2004. Change has been made above.

6
by pawnking (not verified) :: Tue, 09/06/2005 - 12:51pm

A few questions regarding the Monte Carlo:

Why only 1500 simulations? Why not a million? It may not have changed much, but then again, you referred to the fact that "Maybe if we had run 1500 more, things would have evened out" with regards to KC Denver.

Are you considering a running M.C. projection for the remainder of the season, similar to BP?

I understand (I think) the St Deviation column, but wouldn't it have been clearer if you had two other columns, one labled +1 dev wins, and -1 dev wins? That would give (I believe) a clearer picture of how high or low we can expect each team to go, espicially Arizona.

7
by Pat (not verified) :: Tue, 09/06/2005 - 12:57pm

Aaron:

Cool. That looks better. The special teams correlation in particular is just hopping around 0.5+/-10% or so, which is probably its error in the regression anyway.

oh:

It’s Philadelphia’s world and we’re just living in it.

go eagles!

Honestly, I'm amazed that Philly still has all of its coaching staff. Yah, they haven't won a Super Bowl yet, but seriously: winningest team since 2000, continually improving record since 2000. You'd think that the Dolphins or the 49ers would kill to have these problems.

So how much does "coaching changes" play into the prediction? Is there an FO article on the basis of this?

8
by Digit (not verified) :: Tue, 09/06/2005 - 2:04pm

Re: Belichick and offense

Maybe so, but I can't imagine that Belichick would be using a 91-95 Browns offense, not with the pieces he has in place. Belichick, unlike most coaches, has actually seemed to adapt and change over the years, so I think assuming that he's just going to pull out the 91-95 Browns playbook is a dangerous assumption to make.

I've pointed this out to Sid, but I think the unofficial offensive coordinator of the Patriots isn't Belichick, but Josh McDaniel, the current QB coach... he seems to be the one taking the greater lead on offense playcalling in these preseason games, including wearing a distinctive sweater/shirt from the other offensive coaches.

My guess is Belichick is working with the coaches, but is leaving it as a committee, with McDaniel as the 'team leader'.

Honestly, I think the addition of Ben Watson and Andre Davis to the Patriots' offense will offset whatever was lost by losing Weis.

9
by geoff (not verified) :: Tue, 09/06/2005 - 2:12pm

This is great stuff, and a great way to present the information in PFP 2006

10
by John R (not verified) :: Tue, 09/06/2005 - 2:38pm

I am new to this site - and I love it. I have the book and I am going through all the stats and methodologies. As I read, I think of general comments or questions, is this the right forum for them?

I really think that you could charge money for your fantasy projections. It would be very cool if you could talk a little in the book about how you do them (or have I missed something?).

related to fantasy stuff - it would be real cool for you to do a standard deviation or a confidence interval on a particular player's projection. As I get to the later rounds, I might want to take a flyer on guys that aren't that great - but their standard dev is high - just in case.

Sorry if this is the wrong forum for this. This is just my fantasy ramblings. I will get back with more ramblings about DVOA and methodologies later. Maybe a general forum for a discussion of your methods would be good? Is there one already and I just missed it?

-

11
by Arkaein (not verified) :: Tue, 09/06/2005 - 2:40pm

Re: #6

I'm just speculating, but I'd imagine that they stopped at 1500 iterations because the system had already stabilized at that point, and more iterations would have made a negligible difference.

In addition one million trials would likely have been prohibitive in time and computation costs (and this may have played a role in limiting the number), but likely would have been massive overkill anyways.

12
by Jake (not verified) :: Tue, 09/06/2005 - 2:43pm

On Pats O-Coord:
Word is that the offense will actually be a committee of Brady, BB and QB coach Josh McDaniels with McDaniels so far actually calling the plays. And they'll be using the Weis offense (I watch all the Pats pre-season games and the passing game was classic Weis and the running game was the wham play heavy rushing attack they began moving to last season).

13
by pawnking (not verified) :: Tue, 09/06/2005 - 2:44pm

#11, one of my points is why use few iterations as an excuse if the system had stabilized at that point? It seems odd. And BP runs a million projections every day, even when there are thousands of games left. Why not that many for the 16 football games?

14
by Jeff F (not verified) :: Tue, 09/06/2005 - 3:01pm

I think you’re underestimating the loss of Weis. The NE offense is going to be vastly different. If you want see what a Belichick offense looks like, you have have to look to the 91-95 Browns.

The NE offense might turn out to be quite a bit different (I wouldn't be terribly surprised if they do significantly more passing, for one), but to compare the 2004 Pats to the early 90's Browns is a bit silly. Bellichick is a more learned coach now than he was then, and to say the Pats have a much more talented team today is an understatement, and they aren't going to force themselves into a style of play that would hamper themselves, would they?

Maybe an improvement for the projection system could be made by taking the KUBIAK projections for all of the players involved, and counting them partially against the DVOAs of the players they are replacing, and the result being an extrapolated DVOA estimated by improvement/decline in QB/RB/WR/TE/defensive players - IE your #2 receiver had X number of touches last year, a DVOA of +15%, and he is moving up to the #1 position where his performance is expected to drop slightly, but his participation is supposed to increase to 22%, and the outgong #1 WR got less play and had a worse DVOA than the new #1 receiver is projected to have.

That came out quite poorly, and it'd take some doing, but you could even make several sets of projections giving different things different weights.

15
by ToddCommish (not verified) :: Tue, 09/06/2005 - 3:30pm

I'm rather astounded that the 49ers are actually projected to win the division in 7% of the simulations (around 100 out of 1500 times). I'm wondering if these simulated seasons were more of the Seahawks and Rams being at the bottom end of their normal distribution (around 6-7 wins)and the 49ers being at the top of their curve (8 wins would be an amazing year). I can't imagine any simulation run where the 49ers top eight wins.

16
by Rodney Harrison (not verified) :: Tue, 09/06/2005 - 3:47pm

9.7 wins smells of disrespect.

17
by David Papazian (not verified) :: Tue, 09/06/2005 - 3:55pm

Any chance of seeing un-adjusted offensive and defensive DVOAs to assess true level of expectation regardless of schedule?

18
by Bruce Stram (not verified) :: Tue, 09/06/2005 - 3:59pm

Aaron,

Your slip is showing bit on your stats. First, more observations may mean more variation, but doesn't lead to a lower correlation coefficient unless you're not making an adjustment for degrees of freedom (i.e. making a systematic adjustment downward in the coefficient for fewer observations). Further you should also be judging your correlation stats and other by their statistical signficance in addition to their size. A higher correlation coefficient, especially one not adjusted for degrees of freedom, could be less signficant than a lower one. More observations should mean more signficant stats in general (though not necessarily more explanatory power, i.e. R^2).

Finally, you don't take preseason into account. Maybe for good reason, but I did a small study a couple of years ago, and found that the four preseason games had as much predictive power (r^2 60%) as 4 game stretches in the regular season. While preseason games don't count in the standings, at least by this small study, they are as much a harbinger of the future as regular season games.

19
by TomC (not verified) :: Tue, 09/06/2005 - 4:12pm

Warning: Serious math geekery to follow.

Aaron -

Either I'm missing something fundamental, or those standard deviation numbers are way too big. Let's take Dallas as an example. If I take your numbers at face value, then the standard deviation of projected Cowboy wins is 77% of the value of the mean projected Cowboy wins, which is 8. This gives a number close to 6 for the standard deviation, which is pretty strange on the face of it, because the intuitive way I understand standard deviation is: "A fair number of samples in the population are one or more standard deviations from the mean." In this case, that would have lots of seasons in which the 'Boys were 2-14 (or worse) or 14-2 (or better). (Precisely 38% of all seasons would be this good or this bad if the distribution were Gaussian.)

To be more quantitative, let's imagine Dr. Alamar's system produced a distribution of Cowboy seasons with 8.0 mean wins and the largest possible standard deviation -- namely one in which the Cowboys only ever went 16-0 or 0-16. In this case the standard deviation would be 8.0, or 100 in your units. To get 77 in your units, you'd have to have a distribution that was not much less crazy than the extreme one I just mentioned. For example, a Dr. Alamar run that had the Cowboys going 8-8 500 times, 16-0 500 times, and 0-16 500 times would have a standard deviation of 6.54, or 82 in your units. Any reasonable set of Monte Carlo'd seasons -- where the 'Boys went 8-8 a whole bunch, 9-7 or 7-9 alot, maybe as bad as 5-11 or 4-12 a few times, but never 0-16 or 16-0 -- would have a standard deviation more like 2, or 25 in your units.

And, of course, bless you and your FO brethren for doing analysis about which one can even dream of having this sort of discussion.

20
by Pat (not verified) :: Tue, 09/06/2005 - 4:13pm

Bruce:

Strangely enough, I did a similar study just a week ago, although I looked at the predictive power of first-game point spreads to predict second game point spreads.

Preseason actually ends up having a stronger correlation, though MDS pointed out to me that this could be due to the fact that regular season games exchange home and away games, which would tend to flatten any point spread correlation.

But the statistical significance of the two seems to be pretty much the same. If you beat the snot out of an opponent in the first half of the preseason, you're very likely to beat them again. And much less likely to get the snot beaten out of you as well.

Note that I said "first half". If you look at just the second half point spreads - uh, there's no correlation anywhere near as strong as the first half spreads.

21
by MadPenguin (not verified) :: Tue, 09/06/2005 - 4:15pm

And now I understand why TMQ chose TB. :-)

22
by Pat (not verified) :: Tue, 09/06/2005 - 4:21pm

(Precisely 38% of all seasons would be this good or this bad if the distribution were Gaussian.)

38%?? Bad TomC! It's 68%!

Don't forget you're capped at 16-0 and 0-16, so nominal fluctuations are going to stack at that point. That'll skew both the mean and the standard deviation.

You'd expect a high stdev for Philly, for instance, because that 4.6% chance of a perfect season pulls the mean higher, which means that the standard deviation gets larger to accomodate the tail.

But I do agree that the stdevs seem a little high, but hey, this is football.

23
by TomC (not verified) :: Tue, 09/06/2005 - 4:31pm

Pat -

Sorry about that typo, but your correction isn't right either. I should have written "32%", because I was talking about the # of samples outside of 1 sigma from the mean.

But keep up the stat police work.

Tom

24
by Pat (not verified) :: Tue, 09/06/2005 - 4:43pm

Oh, whoops, your comment made it a little awkward to understand if it was inside or outside.

And I just realized that my comment above is backwards logic: a truncated distribution has a lower stdev than an untruncated one. So I really don't understand the stdevs.

25
by Kevin (not verified) :: Tue, 09/06/2005 - 5:00pm

I figured Minny would drop on offense as well . . . but the 1st team has looked superb in preseason . . . unless the loss of Birk hurts the OL significantly, look for the Minny offense to keep rolling . . .

26
by the fumble (not verified) :: Tue, 09/06/2005 - 5:34pm

the line about Cleveland is priceless, especially since it comes from a non-steelers fan.

27
by Catfish (not verified) :: Tue, 09/06/2005 - 5:47pm

For math geeks, this is standard deviation of wins divided by mean wins, times 100. Arizona’s standard deviation was actually higher than its mean wins.

I'm no statistician, but does that explain it?

28
by Larry (not verified) :: Tue, 09/06/2005 - 6:14pm

From the PROTRADE article:

Based on their win probabilities, the simulation randomly selects a number of wins for each team. The simulation then determines which teams make the playoffs based upon the rules for winning the division and/or receiving a wild card birth.

Taking this at face value it doesn't seem to account for the schedule and the fact that the total number of wins for the season is fixed at 256. I'm sure the simulation did account for this fact that each team's win distribution is not an independant random variable, but it'd be nice to have seen it stated explicitly.

I'd also love to see a plot for each team with the predicted distribution of wins. It could be linked to, rather than in the article, but it would be nice to be able to find. For non-gaussian distributions, which these certainly are, the standard deviation isn't exactly a great measure anyway. Just seeing the distribution would be more meaningful, rather like the book does with the 5 categories, but a graph is always nice.

And I echo #19's final thought.

29
by Ryan Mc (not verified) :: Tue, 09/06/2005 - 7:15pm

I'm sure the numbers are correct, but I'm still surprised at the high rating for Buffalo's D that I keep seeing on this site. Surely, if any unit in the league benefitted from weak opposition it was the Buffalo D, or were teams like the Browns, Dolphins, 49ers, Cards, Ravens, Bengals (in John Kitna's first start of the season) and Jags just much better offensive units than I remember them being?

The Pats seemed to handle the Bills D fine, and, with the season on the line, the Bills D allowed Pittsburgh a FG drive of almost ten minutes in the 4th quarter when the drive consisted entirely of the third-string QB handing the ball off to the third-string RB and second string FB. Not fair to judge on just one drive, but that doesn't seem like a big-time defense to me

30
by Bruce Stram (not verified) :: Tue, 09/06/2005 - 7:49pm

Catfish,

Your analysis suggests Arz wins weren't statistically signficant, i.e we can't reject the hypothesis that they won zero games.

31
by DavidH (not verified) :: Tue, 09/06/2005 - 8:05pm

From the PROTRADE article:
All ties in the system are settled via random choice, as the rules for tie-breakers involve more than can be estimated. For example, when two teams in the same division have the same number of wins, the division is given to one of those teams at random in the simulation.

Aren't most of the more important tiebreakers based on records vs. certain subsets of teams, i.e. head-to-head, conference, division? I had assumed that the simulation involved looking at every game in the NFL season and simulating a winner based on your projections. But if they can't determine the tiebreaker records, it really doesn't appear that that's the case.

So, basically, what Larry said in #28.

And I agree that a graph of the distributions would be WAY helpful. Although I would understand if that was featured in your book so you're keeping it off the site.

32
by TomC (not verified) :: Tue, 09/06/2005 - 8:06pm

re #30: Your analysis suggests Arz wins weren’t statistically signficant, i.e we can’t reject the hypothesis that they won zero games.

I don't think that's Catfish's analysis; I think that's the face-value interpretation of Aaron's standard deviation statistic. As Aaron writes in the definition: "Arizona’s standard deviation was actually higher than its mean wins."

33
by Pat (not verified) :: Tue, 09/06/2005 - 8:14pm

Re: AZ's wins:

Yah, but does this surprise anyone? I mean, anything could happen this year. If Kurt Warner falls apart, that could be even worse than last year.

34
by DavidH (not verified) :: Tue, 09/06/2005 - 8:23pm

I don't see a place to comment on the PROTRADE page, so I'm posting this here, since it seems these two articles are intrinsically linked. If there is a better place, somebody please point it out to me...

From the PROTRADE article:
To measure a division’s overall quality, we calculated the median probability that each of its teams will make the playoffs (see table 3). The higher the median probability, the better the division, as the higher probability indicates that the division has more teams that will be involved in the playoff race.

That's just blatantly incorrect. You could say that a higher mean probability means more teams involved in the playoff race. That is defensible, although I would say that having the most number of teams above a certain cutoff (say 40%) or having the best 3rd-ranked team is a better indicator. Regardless, more teams in the playoff race does not have to mean higher quality. In some cases it does (see AFC West, AFC East), but in others it means that division has easy schedules (see NFC South). I mean, does he really think that a division with the 8th, 9th, 20th, and 29th ranked teams (according to the projections his simulations are based on) is better than a division with the 2nd, 6th, 10th, and 28th ranked teams?

OK, I'm done complaining. The footballoutsiders article is good. The PROTRADE one, not so much.

35
by MME (not verified) :: Tue, 09/06/2005 - 8:25pm

While points may be easier to understand at a glance, I like DVOA and DPAR because they are centered around zero. I would prefer "adjust points above average" with a comment saying average is 21.5 then having to look at every number and subtract 21.5 to find out how much better (or worse) than average the projection is. IMO, systems centered around zero are better.

36
by DavidH (not verified) :: Tue, 09/06/2005 - 8:25pm

And when I say "ranked" I mean by the Total column in the projections, not by wins or some other garbage. I mean, who cares about wins? The Super Bowl trophy goes to the team with the best DVOA, right? Riiiiight?

37
by DavidH (not verified) :: Tue, 09/06/2005 - 8:27pm

I agree 100% with MME.

As for "adjusted points above average" vs. DVOA, I'd say it's a toss up.

38
by TMK (not verified) :: Tue, 09/06/2005 - 9:41pm

If this is the same system that projected Baltimore to be the worst team in the AFC last year, I'm not worried. At least this year it's calling for 8+ wins, so by those standards, I should book a trip to Detroit. :)

And yes, I know that the prediction was somewhat vouchsafed in the commentary, but that is the result the system generated. Any similar reservations this year? Please?

39
by Arkaein (not verified) :: Tue, 09/06/2005 - 9:48pm

Re: #13 (a little late)

Pawnking, I'm not sure where the small number of repetitions is used as an excuse for the high variability in the projections, can you point this out?

I can't say anything about the BP method, but sounds like a pretty reliable number to me. Even if the simulations are fairly variable between runs, 1500 gives a pretty large population size for doing statistical analysis.

Finally, no way BP runs a million full season simulations per day. There are less than 100,000 seconds in one day. Running 10 simulations per second sounds ridiculous (maybe 10 single game simulations per second on a decent computer cluster, but no way 10 full seasons per second unless the simulation is extremely simplistic).

40
by Pat (not verified) :: Tue, 09/06/2005 - 9:53pm

Nah, it's trivial, Arkaein. A million simulations a day is easy. It's not exactly complicated in baseball. You're not talking about advanced math. Team A scores X runs on average, team B scores Y runs on average, jumble X by its stdev, jumble Y by its stdev. No problem.

41
by Charles Kaneb (not verified) :: Tue, 09/06/2005 - 9:54pm

"No other team has a realistic chance of winning 15-16 games".

Without significant changes for the worse, past performance is a very good indicator of current capabilities. The Steelers are better at every position where there was a personnel change. They were 15-1 last year.

If there is one disadvantage to current statistical predictions of football, it is that they don't move the great and awful teams far enough away from the median.

There were 5 teams 12-4 or better last year, including one at 15-1 and one at 14-2. There were 6 in 2003, including a 14-2 team. There were 3 in 2002, which was the only time there were no teams better than 12-4. In 2001, there were 5 including a 14-2 team. If the past five years are anything to go by, it is unlikely that no team will win 13 games, and very unlikely that no team will win 12.

42
by Pat (not verified) :: Tue, 09/06/2005 - 9:55pm

TMK:

Nope. It projected an offensive DVOA of -27% for Baltimore, and -27% for defense. Baltimore's offense was actually -2.5%, and defense was actually -16.6%. If you assume that the offensive environment for the league was, say, 10% higher or so, the projections were pretty bloody close.

Note that not all of them turned out that well. But there's a reason that the projections have a correlation coefficient of 0.6-0.7. They work - on the whole.

43
by Charles Kaneb (not verified) :: Tue, 09/06/2005 - 9:56pm

In 2000, there were four teams with 13 or more wins.

44
by Pat (not verified) :: Tue, 09/06/2005 - 10:01pm

#41:

Aaron said "realistic". The Steelers got lucky last year on several games - hitting opponents who had serious injuries at times when they were themselves quite strong as well.

If there is one disadvantage to current statistical predictions of football, it is that they don’t move the great and awful teams far enough away from the median.

I disagree. I don't think the great and awful teams are great and awful. I think they're lucky and unlucky. You can't predict luck.

45
by J (not verified) :: Tue, 09/06/2005 - 10:09pm

Interesting stuff.

Aaron, you must hate Ron Mexico, errr Mike Vick. His X factor screws up not only his own team predictions, but also those in his division.

Having said that, does this not show how important of a player Michael Vick is? Many on here have blasted Vick for being over hyped, but these predictions clearly show how dynamic of a player he is.

TB will not be as good as these predictions show...just like last years numbers.

Atlanta will not be as bad as the projections...just like last year..unless Vick goes down with an injury.

Vick is no Manning, and he is not Brady. However, he is not over hyped. Just look at these numbers.

46
by Catfish (not verified) :: Tue, 09/06/2005 - 10:14pm

Your analysis suggests Arz wins weren’t statistically signficant, i.e we can’t reject the hypothesis that they won zero games.

Actually I just copied and pasted from the article. I bolded the part about how the Std Dev is multiplied by 100 because TomC was wondering why the number was so high.

Without significant changes for the worse, past performance is a very good indicator of current capabilities. The Steelers are better at every position where there was a personnel change. They were 15-1 last year.

It's true that past performance is a good indicator of future abilities (I sure hope so, or this whole discussion is pointless!), but Pythagorean Wins (linked in sig) and Estimated Wins (see below) are better indicators than just plain wins and losses. Pitt's Pythagorean wins were 11.5 and their estimated wins were 11.9 . Generally teams that outperform either of these measures by a game or more tend to decline the next season. I do agree with you that the great and awful teams seem to close together.

Explanation of Estimated Wins (the methods have since been updated): http://www.footballoutsiders.com/2003ratings14b.php

47
by Zac (not verified) :: Tue, 09/06/2005 - 10:30pm

Arkaein, pawnking is talking about this from the article (note the bold):

No, I’m not sure why Denver comes out with a lower chance of winning the division and fewer mean wins despite its average opponent being slightly easier. Could this be related to Jacksonville’s low standard deviation? They are on Denver’s schedule and not Kansas City’s schedule. Or perhaps it is just a random quirk in the simulation we ran, and the two teams would have been tied if we ran the simulation another 1,500 times.

48
by Lou (not verified) :: Tue, 09/06/2005 - 10:51pm

I agree with those above who question the standard deviation column. There is no way that the (stdev/average) can be close to that high. A perfectly random distribution of 1-16 wins only has a value of 56 using this method. It looks to me like most value are roughly 3x too high.
Plus, barring Culpepper missing at least 8 games, there is no way that the Vikings won't have the most points scored in the NFC North.

Still though, facinating projections and I look forward to seeing how they match up with the results at the end of the season. I assume we'll see a new set of tables comparing the projected to actual. I also like seeing a SOS forecast based on something other than last years record. It certainly looks like Atlanta is in serious trouble.

49
by Zac (not verified) :: Tue, 09/06/2005 - 10:57pm

I agree with TomC, I don't understand these Standard Deviation numbers. I don't really understand Monte Carlo simulations either, but this is how I would simulate the season given these numbers.

For each team in each game, a random number would be generated. Let's say it's something simple, like a decimal number between 0 and 1. This would then adjust the team's total by the standard deviation. For example, let's say we are predicting the outcome of the opening Oakland-New England game. The random variables turn out to be .4 for Oakland and .6 for New England. So we assume that Oakland will score whatever a normal distribution of its standard deviation would be at 40% of total (with 50% being average), which according to the numbers is probably something like -2.5 points or something. We do the same for New England, except we adjust upward according to the normal distribution of their standard deviation at 60%, which again is probably about 7. We then compare total numbers, and that gives New England the win (by about 10 points). We do the same for every other game. When we're finished, we figure out which teams make the playoffs, then run the simulations another x times, and we have our numbers.

Maybe this isn't possible. Or maybe it doesn't make sense. I don't really know. But that is how I am picturing it.

50
by Tom W (not verified) :: Tue, 09/06/2005 - 11:10pm

Just a quick reaction before actually digesting the material: I much prefer the points/game format. In general, anytime you can translate percentages into points, it's helpful, especially for those of us who occasionally like to place friendly wagers on games.

51
by Moses (not verified) :: Tue, 09/06/2005 - 11:15pm

I always enjoy these things, though they crack me up. And it's not crack up from maliciousness or "football is played on the field" cliche'd superiority. I did them myself in the late 1970's and early 1980's and had a lot of fun with the concept. My less sophisticated models never did work out like I'd have liked, though it was a lot of fun. It cracks me up is that is a huge problem with the concept, which is an unfortunate, but very obvious simple fact of the NFL:

The team you have today is NOT the team you had last year. And that goes for moral, luck, the conditions, team health down through the season, etc., plus those same issues your not-the-same-as-last-year opponents are going through.

Look at luck, Detroit beats Chicago on a blocked FG returned for a TD. Wow, just returning a blocked FG for TD is about a 500-1 (per kick) occurence. Happens about once or twice a year (with some years skipped) in the NFL. And how much rarer does it provide the points necessary to turn a loss into a win? Bad luck for the Bears, good luck for the Lions. That in anyone stats models of victory prediction? How many rare wins like will the Lions get this year? :)

Who believes the Falcons will return 4 INT for TDs again this year? When most NFL teams have 2 or fewer? Probably the same people that believed the first half of 2003 Vikings defense was for real.

So, I like the work. I love the DVOA as a rating tool. I even love these "official non-predictions." I even hope that someday someone will figure out a system better than the talking heads that just make stuff up based on their biases and incomplete, and frequently outdated, assumptions.

As for me, until someone comes up with better, I've learned to use a very safe system. I call it SALY, which means Same As Last Year. And, by gosh, I do pretty well picking division winners. At least as well as the talking heads do, and instead of sounding pompus and self-important like Theisman or Salsibury, I get a good laugh. And it takes virtually no time at all. :)

52
by Bruce Stram (not verified) :: Tue, 09/06/2005 - 11:32pm

Moses,

Actually your point is part of why I'd suggest bringing in the preseason. Just to be a bit clearer about my (admitidly limited) analysis, it was preseason for a given year compared to regular season for the given year. In other words, preseason has explanatory power for this year's team as much as any other 4 game set played by this years team.

53
by Larry (not verified) :: Tue, 09/06/2005 - 11:32pm

I don't like using points instead of percentages. DVOA is situational, which means that you could have a good defensive DVOA, but with a bad offense and bad special teams, they'll have a short field to defend all the time, meaning more points allowed. So the 0% = 21.5 points isn't really accurate. Perhaps this averages out over the course of the season, but there's no strong reason to believe that is true (it isn't or there'd have been no reason to bother inventing DVOA). So, I'd stay away from the DVOA = points. I won't give the realted thoughts on (D)PAR, but I think this objection isn't as strong there for a number of reasons.

As for #49, that's pretty much how the book describes it. The Protrade article seems to say something entirely different as I quote in comment #28. But, as others have pointed out, there seem to be several objections to the PROTRADE piece, and I'm not sure if that's the writing or the execution. Hopefully, someone who knows will drop in here and spread some knowledge.

I guess this it what happens when we get to look behind the curtain a bit.

54
by Tom W (not verified) :: Tue, 09/06/2005 - 11:34pm

A second quick reaction after reading some of the comments: I wish people would keep in mind that this is a FOOTBALL website, not the M.I.T. student lounge. While I'm interested in how you come up with your projections, if I wanted a debate/discussion of the relative merits of various statistical methodologies, I'd take a night class.

55
by King Kaufman (not verified) :: Wed, 09/07/2005 - 12:23am

Hey, TomW, here's a suggestion: When you come to a comment where someone is discussing the relative merits of various statistical methodologies, skip over it. Read the next one instead.

I hope I'm not going too fast here, dazzling you with technical stuff, but that's what I do, and it has the magical effect of never making me feel like I'm in the MIT student lounge without depriving others of the opportunity to talk about the things they want to talk about.

56
by Arkaein (not verified) :: Wed, 09/07/2005 - 12:27am

Pat, I guess I severely overestimated the complexity of the simulations that were involved. When I heard DVOA I thought that maybe entire games were simulated on a play-by-play basis (which would be a more accurate usage of the term "simulation" IMHO). Running a few hundred billion simple formula calculations per day is certainly feasible.

I'd still guess that 1500 trials is sufficient for stable results in any case. The number one million used by BP sounds somewhat conveniently chosen rather than chosen based completely on it's scientific merits (possibly for a bit of marketing hype?)

57
by M (not verified) :: Wed, 09/07/2005 - 1:11am

Re: #40 & $56

I work for an insurance company, and somewhat regularly have to do Monte Carlo simulations for various purposes. We usually find that 10,000 simulations is enough for the results to stabilize.

I agree that one million for BP is probably due more to marketing hype than any theoretical basis.

#55

Part of what I love about the discussions on this site is that you can go high or low in intellectual content, and still be entertained.

58
by Scott C (not verified) :: Wed, 09/07/2005 - 1:59am

#54

This is a lot more fun and interesting than a night class. :)

#53
I like points a bit better than percentages, but only if they are + and - 0 and not some league average. One can still adjust for special teams or offense when looking at a defensive value in units of points easily if it is centered around zero.

Looking at a column of numbers centered around 21.5 and having to subtract before mental adjustment for opponent or other aspects of the same team is hard. So is conversion of percentages to points.

So if my vote counts for anything, I vote for points centered around 0 (with or without a negative for defense). The 21.5 point average adjustment is a distraction when analyzing a team or a matchup.

------------
I have to concur with the others that the standard deviation numbers have to be wrong -- they just don't make sense.

If one assumes the Eagles have an average of 12 wins predicted and a 5 percent chance of 16 wins, which is near the listed 4.7% and 11.7 in the prediction, one can maximize the standard deviation with this distribution:

Wins -- % chance
16 -- 5
15 -- 45
12 -- 35
5 -- 5
0 -- 10

Average 12 wins, std dev 4.78 or in the units of the tables above (divide by average wins, 12, and multiply by 100) a meager
>>>> 40 !!!!

The above distribution is completely insane. Philly doesn't have a 45% chance of 15 wins and a 10% chance of zero. But given the conditions of about 5% chance to win 16 and an average of 12 have a MAXIMUM possible deviation of 40 in these units. Meaning the 91 listed in the chart for the Eagles has to be wrong, or the description of the measurement unit is wrong. The maximum possible for a team with an average of 12 wins is 50% 16 win seasons, 15% 0 wins, 5% 8 wins, and 30% 12 wins yielding only 47 in these units.

That is, it cannot possibly be (100 * std deviation in wins / average wins). Hopefully these are all off by the same ratio so that the numbers are still valid.

However --
I think units of +- games at 1 or 2 stddev would be clearer. 1 for those familiar with statistics and 2 for those who aren't since 1 std dev leaves reasonable common sense probability of an outcome beyond 1 std dev. Either way, dividing by the mean occludes IMO.

After all, if a team was predicted to be 2 wins +- 1 game that is a value of 50 in these units which I would argue is alot less statistically significant than a value of 20 for a 10 win team (meaning 10 wins +- 2 games). That is, raw +- game deviation is more meaningful.

59
by Glenn (not verified) :: Wed, 09/07/2005 - 2:14am

Off the stats talk for a second.....

Jake (#12), as a Pats fan who devours all coverage, I haven't seen anything at all to indicate that Brady is now part of an "offensive committee" to call the plays with BB and McDaniels. Brady has latitude at the line to change the playcalls he's given as he judges fit based on the defense...that's it. Although he obviously has some ideas, he is not mapping out strategy as part of an offensive troika of game-planners. There's also no clear indication that McDaniels alone is "calling the plays"....the only clear indication is that he's physically communicating the plays to Brady. But I do believe that McDaniels is being groomed for the job and absolutely has lots of input into things, but BB right now is telling the media to blame him (BB) if there's a call that doesnt work.

Statistics 101 may now recommence....

60
by Glenn (not verified) :: Wed, 09/07/2005 - 2:33am

Wait!
Here is Wednesday's Rodney Harrison No-Respect Quote Of The Day:

From The Boston Herald: Safety Rodney Harrison on whether he was excited about squaring off against Moss: ``Any time you get on the field it's exciting. But considering no one thinks we can do anything with him? Yes, it's an added challenge. He's the best in the game.''

Good to see Rodney's in mid-season form.

61
by Tom W (not verified) :: Wed, 09/07/2005 - 2:58am

Hey King: Thanks for the suggestion. Since I went to law school instead of getting a P.H.D. in math, I guess I was just too stupid to think of that. The actual point I want to make here is that, while some of the comments expressing a preference for using DVOA rather than points have merit, here's the fundamental problem with DVOA: football games are decided by points not percentages. Betting lines are stated in points. Knowing that Team A's offense is 8.62% better than Team B's defense is, therefore, relatively worthless in practical terms. It has no predictive value and not a lot of analytical value, since it can't be tested for accuracy without somehow converting it into points. One of the great things about Bill James' research was that the basic metrics he used were contextualized so they could be understood in terms of bottom line impact on a team's ability to score runs and prevent opponents from doing same, the object of the game. I recently read an article by Aaron (I think) about using points/play (or plays/point) as an analytical tool, and he concluded that it didn't have much value as a measure of offensive or defensive efficiency. Thing is, it's not supposed to measure the efficiency of anything. People who use it, do so to factor in variables that clearly affect the outcomes of games, but aren't easily quantified, like special teams, turnovers, starting field position, 3rd down performance, etc. I haven't studied it enough to know how well it works, but the point is, if it can't be quantified, it's pretty much just theory. Sorry this was so long.

62
by roggermann (not verified) :: Wed, 09/07/2005 - 4:01am

I checked my old college Stats textbook. 100 * stddev / mean is defined as the "coeficient of variance". I have the word "useless" written next to it, so apparently I wasn't very impressed with this measure back in 1983? Working backwards, Arizona's stddev = 115 * 6.4 / 100 = 7.36. KC is at the low end at 64 * 9.5 / 100 = 6.08. A normal curve = 1, so these are very fat tailed distributions. This is evidence of a very competitive league where a single play/bounce/call can sometimes determine the outcome. I don't know if I should cry or get out the ouiga board?

63
by Larry (not verified) :: Wed, 09/07/2005 - 4:12am

There are lots of sites out there devoted to predicting performance against betting lines. The attraction of this site, to me at least since I speak for no-else, is that here things are devoted to understanding how teams and players contribute to winning and losing, which is ultimately what matters in the standings. I couldn't care less about betting lines. And I'd hate to see things overly processed and obscure the details of what these numbers really represent, which is the ability to make the play needed at the time. This leads to points, but exactly how many is influenced by a lot of factors, and is very different from baseball where all games have an equal number of out events. So, there just isn't a 1-1 correspondence. Just my opinion, of course.

The MIT student lounge comment probably has potential in some form as a slogan for this site.

Anyway, welcome to the discussion. Have fun.

64
by Robert (not verified) :: Wed, 09/07/2005 - 5:02am

Hey, great work. I want to repeat what many people have said: bless you guys.

A few thoughts:

First, I'm a Niner fan—sigh—and I'm not predicting we'll win the division. Honestly, I think 4.7 wins is pretty fair, although I would love to see what switching to a 3-4 does to teams. But I also don't think it's insane to think that they could win the division 7% of the time. Teams don't make big leaps that often, but they do make them. And, my being a homer aside, it's not that hard to see how they do it. The Niners were terrible last year, but they also suffered several key injuries. If Julian Peterson and Jeremy Newberry play all season that is realistically worth several wins right there. A generally improved offensive line, a few more developing players, and a few lucky bounces, and suddenly we're this year's Chargers.

Second, I would really like to add in the preseason to this analysis. It would surprise me if a win in the preseason was as good a predictor as a win in the regular season. But I have noticed that if you take the time the ones play each other in the first three games, it seems to match the teams that end up being good. This season, if I did it right, the Pats and the Steelers looked the best, the Titans and the Bengals the worst. The Niners, I'm glad to report, ranked a stratospheric 23rd in victory margin.

Finally, I find points a much more useful, intuitive measure, but I agree that it would make more sense if they were normalized around zero.

65
by Tom W (not verified) :: Wed, 09/07/2005 - 5:08am

Larry:
I realize that because football is much more of a team sport than baseball, it's much more difficult to place point values on different events. But whether you're interested in this stuff as a bettor, an N.F.L. G.M., or just a fan trying to gain a better understanding of how games are won and lost, it seems to me that to be meaningful, statistical analysis should at least try as much as possible, to quantify those events. I don't know. It's really late, and I should probably go to bed instead of posting messages on websites.

66
by Aaron (not verified) :: Wed, 09/07/2005 - 10:49am

Thanks everyone for their comments. I will try to respond to some of them in a mailbag article later this week or early next week. Unfortunately, because of the end of the book tour and some other things I'm dealing with from the business end, I don't quite have the time to respond to everything in full -- I still have email from a month ago I have not had a chance to respond to. For those complaining about randomness, well, welcome to the NFL. We can't change the way things are in this league. If you think that it means that there is no point in doing analysis of the upcoming season, why the hell are you reading this site? For those complaining about the methodology, points taken, but there are only 24 hours in a day, and I've got a lot of other stuff going on, so chill and enjoy the beer of your choice.

This site doesn't need more slogans. We already have three slogans:

Tackling Football from Outside the Hashmarks
The Best is the Enemy of the Better
-- and --
Feeding Mirinae Schatz Since 2003

67
by B (not verified) :: Wed, 09/07/2005 - 10:51am

Regarding Buffalo's high defensive ranking. Yes they played a weak schedule of offenses, but they dominated them. They had one lousy game against the Pats offense, but in the other game against the Pats, they hung with them pretty well, and it was a turnover by Bledsoe late in the 4th quarter that ended the Bills chances. For the Steelers game, the DVOA system doesn't have any way of reflecting the fact that the Steelers rested most of thier starters, so it looked like the Bills D struggled against the 7th ranked offense. Maybe those games were a better test of the Bills defense, but I don't think you can weigh thier performance in two games more than thier performance over 14 games.

68
by B (not verified) :: Wed, 09/07/2005 - 10:53am

Hey Aaron, if you see this, check your email. I sent you something important about my assignment in the game-charting project.

69
by Aaron (not verified) :: Wed, 09/07/2005 - 11:00am

One more thing. People realize that DVOA being centered around zero is exactly what people have been complaining about, right? That's the whole "I don't understand how defense can be negative" thing. If I were to center adjusted points per game around zero, doesn't it have the exact same problem where better defense is negative? That's the part I'm trying to make easier for people to understand.

70
by DavidH (not verified) :: Wed, 09/07/2005 - 11:44am

When I first ran across this site, there was a while where I kept forgetting that a negative defensive score was a good one, but it wasn't a long while. It's not that hard of a concept. But if you're really worried about "negative=good" confusing people, and you still want it centered around 0, then ... well... bear with me here, because the math is gonna get complicated...

multiply defensive DVOA by -1

Seriously, you could just say that defensive DVOA is a measure of "percent better than average" or something. I don't care either way, as long as it is clearly stated on the site which method you are using. It's not like this would make it any harder to use. If some of us at home have spreadsheets that use defensive DVOA, all we have to do is add in some -1 multipliers.

Now, if I had my druthers, I'd leave it as is, or convert it into points but leave it centered around 0 with "negative=good" But I'm not gonna cry either way.

71
by B (not verified) :: Wed, 09/07/2005 - 11:44am

I never had a problem with the DVOA % centered on zero, but I do like the new method of points per game. I think it makes things easier, but I think the defense should still be negative, since higher numbers are bad and lower numbers are good on the defensive side. Of course, since it's equivalant to the points allowed stat, maybe it should be positive.

72
by Ken (not verified) :: Wed, 09/07/2005 - 11:49am

Perhaps the comparsion of the current Belichick to his Browns days is a bit unfair... however everyone is understating the loss of Weis. Every offense that Weis has touched has turned to Gold. Bledsoe's Pats, '98 Vinny and the Jets, Brady's Pats. He prepares an offense like no other. Focus on fundamentals, exploits weaknesses, creates mismatches, develops players - that's Charlie Weis.

Watching the Jets religiously as I do, the offense changed dramatically when Weis left. The detail that he prepares for his game plans can not be replicated by a committee approach. Come on, remember how well the last committee based approach worked for another Mass. based team.

To statisically model the NE offense to be the same is denial.

73
by Shawn (not verified) :: Wed, 09/07/2005 - 12:16pm

Is there reason to believe that Atlanta will not collapse dramatically? Well, the offensive projections only include variables from 2003 and 2004, and Vick was pretty good as a passer in 2002.

So we're basing their projections for 2005 on the assumption that Doug Johnson will be behind center for half the games?

74
by GBS (not verified) :: Wed, 09/07/2005 - 12:47pm

A 7% chance of winning the division for the 49ers is right in line with the current Vegas odds of 14-1. After comparing all the chances of winning a division here with the current odds, it looks like the FO best bets are:
Tampa Bay - 42% DVOA; 5-1 odds
Buffalo - 29% DVOA; 13-2 odds
Houston - 23% DVOA; 7-1 odds

The worst bet is Atlanta, which only pays 13-10 to win the NFC South.

75
by Rick (not verified) :: Wed, 09/07/2005 - 1:10pm

Aaron is on 610 WIP in Philly right now! :)

76
by pchase (not verified) :: Wed, 09/07/2005 - 2:50pm

how does dvoa translates to points? if 0% dvoa = 21.5, what is 5% dvoa, 10%, etc.?

77
by John R (not verified) :: Wed, 09/07/2005 - 3:11pm

On page 4 of the book, you talk about how you rate special teams. There is a detailed analysis about the value of field position. If a team starts inside their 10 yard line, they are expected to get between -1 and -1.5 points. If a team starts at the 50, they are expected to get +1.5 points, etc.

Why isn't this information used to 'sanity check' your ratings of offense and defense? It would seem to me to be extremely useful. If the offense starts on their 10 and gets it to the 50, they have theoretically gained the team 2.5 points - not on the scoreboard - but in 'field position points'. Similarly, with defense.

This would at least be interesting info to know, yes? Wild variations between these numbers and DVOA would need to be reconciled somehow.

78
by Pat (not verified) :: Wed, 09/07/2005 - 3:30pm

pchase:

5% DVOA would be 22.575 adj. ppg.
10% DVOA would be 23.65 adj. ppg.
-5% DVOA would be 20.425 adj. ppg.
-10% DVOA would be 19.35 adj. ppg.

It's just percentages. 5% DVOA is 21.5*1.05 ppg, etc.

79
by Pat (not verified) :: Wed, 09/07/2005 - 3:33pm

Feeding Mirinae Schatz Since 2003

Aw, dang. We haven't moved up to paying for her future college tuition yet?

80
by B (not verified) :: Wed, 09/07/2005 - 5:40pm

That won't be untill Aaron replaces Skip Bayless on Cold Pizza (Oh, please god, make it happen soon. I know I haven't been in church in a very long time, but I don't ask for much.)

81
by Jim A (not verified) :: Wed, 09/07/2005 - 5:49pm

John R (#77): What you described is essentially the Power Points system used by Ryan Early on ESPN.com a couple years ago. He doesn't publish these any more, but I remember it matched up pretty well with an older version of DVOA. But I agree, there is some potential there.

One thing I believe is still true about Aaron's special teams rankings is that, unlike DVOA, he does not adjust for strength of opposition, maybe not for score, either. Analyzing special teams play-by-play is somewhat different than offense and defense because, except for FGs, the success markers are not as clear.

82
by princeton73 (not verified) :: Wed, 09/07/2005 - 6:00pm

Meaning the 91 listed in the chart for the Eagles has to be wrong, or the description of the measurement unit is wrong.

I was going to say the same thing--as written (and explained), the system predicts Philly to win 11.7 games plus or minus 10.6

that can't be right

83
by Pat (not verified) :: Wed, 09/07/2005 - 6:32pm

Yah, I'm actually wondering whether or not there's a missing square root in there somewhere. Given a distribution from 0-16, with a mean of 11.7, it's actually impossible to have a standard deviation of 10.6. The biggest spread you could have is if you had 1100 16-win seasons, and 400 0-win seasons. The standard deviation from that season is 7.07. You can't get a larger standard deviation than that if you've capped things to a 16-game season. The other possibility is if that quoted standard deviation is more than just the simple standard deviation of wins.

If there was a missing square root there, it'd mean a standard deviation of 3.26 games, which would be more consistent with Aaron's statement of a 5% chance of a perfect season for the Eagles, as well.

84
by John R (not verified) :: Wed, 09/07/2005 - 7:02pm

oh yes - Bayless is an absolute nitwit. Nitwits by themselves are OK, but nitwits that think that they aren't nitwits in spite of overwhelming objective evidence showing their nitwittitude are really, really annoying. Bayless fits that bill.

I think the real problem is that he is enabled by Woody Paige - who is dumb beyond belief. Anyone debating Woody looks like a genius. This is what emboldens Skip to think he is smarter than he is.

Ugh...I feel dirty just talking about him.

Anyway, it sounds like I should look into the powerpoints system - it sounds interesting, thanks Jim.

85
by G Eugene (not verified) :: Wed, 09/07/2005 - 9:34pm

As far as translating into point spreads, it's mostly intuitive. Dallas is getting 4 1/2 at San Diego. Dallas is projected to win 8.0, San Diego 7.8. Take Dallas and the points and pray you're not sabotaged by a bad call, a blocked punt returned for a TD or some other "lucky" play. Having read the book and articles on this site, luck has a huge amount of impact on an individual contest and it would be foolish to wager the mortgage on a single game.

86
by Pat (not verified) :: Wed, 09/07/2005 - 10:41pm

First off - DVOA won't convert to point spread well. It wasn't designed to do that. But...

#85: Those wins take into account strength of schedule. I wouldn't bet on that game at all. SD's offensive/defensive DVOA is +15%/+9%, Dallas's offensive/defensive DVOA is -5%/-5%.

If you stupidly just "subtract" the two, SD's got a slight advantage because their offense is much better than their defense is poor, and Dallas's defense isn't that much better than their offense is bad. SD OFF+DAL DEF = 10%, DAL OFF+SD DEF=4%.

So a naive stupid prediction would be SD 24, DAL 22. Given that SD's at home, if there's something like a 10% advantage to SD, they could easily cover the point spread.

I wouldn't bet on this game. I wouldn't bet using DVOA anyway, especially the projections, but not on this game.

87
by pawnking (not verified) :: Thu, 09/08/2005 - 10:26am

I think that by the time we develop a system which can truly predict seasons results, we will have solved world peace and hunger.

Football is a wonderful game of organized Chaos. Baseball's variables are boring and predictable in comparison. Any statistical model is better than none, but an accurate one is beyond the scale of anything we can do right now.

Maybe in another 5 years, when computing power has roughly tripled, and chaos and games theory continues to develop, we'll not have to actually play the games on the field.

Here's a good article talking about, among other things, developments in predictive models. A good read, I must say.

88
by B (not verified) :: Thu, 09/08/2005 - 10:37am

What about free will? Even if we account for the Chaos, random chance, injuries, fatigue, anything else that affects the game, we still can't account for a coach who inexplicabally punts on 4th and 2 or a WR who runs the wrong route, or a back who shies away from contact and doesn't get the crucial yard on a 2nd down play.

89
by Digit (not verified) :: Thu, 09/08/2005 - 10:57am

Re: Watching the Jets religiously as I do, the offense changed dramatically when Weis left.

Two things here:

1) The Jets' offense changed when Parcells took playcalling away from Weis in 1999, not a year later, like you suggested. (And since when did Weis make 'gold' with Bledsoe, anyway? Remember, Weis had Bledsoe for 2000/two games of 2001, and Bledsoe, quite frankly, stunk. He was never offensive coordinator in Bledsoe's glory years.)

2)Of course the whole Jets offense changed- after all, they brought in a whole new system of plays, didn't they?

But the Patriots' offense style hasn't changed, not in the preseason, not with Tom Brady there running it exactly, not with the addition of Ben Watson, (the jury's still out on Andre Davis.) So yeah, I'd say the offense will be just fine without Weis.

What the Pats' offense won't be fine without will be Tom Brady.

90
by B (not verified) :: Thu, 09/08/2005 - 11:03am

This isn't a totally fair comparison, but in 2000, the Pats Pass offense was ranked 14th by DVOA, in 2001, the Pats Passing offense (under Brady & 2 games of Bledsoe) was ranked 13th. Unfortunatly we don't have the 99 data online yet, so we can't see what Bledsoe looked like before Bellicheck/Weiss showed up.

91
by elhondo (not verified) :: Thu, 09/08/2005 - 11:05am

Re: 88

Free will is statistically insignificant.

92
by Led (not verified) :: Thu, 09/08/2005 - 11:05am

Not sure where to put this link, but this thread seems like the best place. Congrats to Aaron, et al. When a chucklehead like Rich Cimini of the NY Daily News is citing FO statistics, you know you've made it:

"[The Jets] finished 14th in passing yardage allowed and 18th in average per pass play, a more accurate barometer. This may sound overly simplistic, but the Jets didn't do a good job of covering wide receivers. According to a statistical analysis by Footballoutsiders.com, they ranked 28th in defending the opponents' No.1 wide receiver, 28th against the No.2 receiver.

That is an indictment of the secondary, infamously labeled the 'weakest' part of the defense in a midseason assessment by Henderson."

93
by Pat (not verified) :: Thu, 09/08/2005 - 11:37am

Aaron:

You've constantly said that the projection system is constantly high on TB. But that's not the only thing: TB has overperformed its record for the past two years. They had 10.1 estimated wins in 2003, and went 7-9 with a slightly easy schedule (-1.9%), which means they *really* were better than their record. They had 8.0 estimated wins in 2004, and went 5-11, with a *very* easy schedule (-7.3%), which again means they *really* were better than their record.

In fact, the 2004 projections for TB's offensive/defensive DVOA were actually pretty accurate.

TB: offense projected -0.3% actual offense -6.1%
TB: defense projected -17.3% actual defense -8.8%

That's not bad. Plus or minus about 8% on both. If you assume that the entire league had an offensive "boost" of about 5-10%, that makes it even more accurate except for the offense, and you can't predict Brad Johnson being yanked for Griese in the middle of the season. Plus a projection of 11 wins, when in "estimated wins" with the schedule correction, it's about 9 wins - again, that's pretty good.

So I think it's clear to say that your projection system is working great: it's reality that's screwed up.

94
by B (not verified) :: Thu, 09/08/2005 - 11:41am

We all know that TB signed a devil's bargin to win the superbowl in 02, and part of the deal was no matter how good thier team should be on paper, they would finish under .500

95
by pawnking (not verified) :: Thu, 09/08/2005 - 11:53am

Pat, I love ya, baby. Reality is what has failed, not the predictive model. I know you're speaking with your tongue in cheeck, but that's just a great line. Well done.

So I think it’s clear to say that your projection system is working great: it’s reality that’s screwed up.

96
by Pat (not verified) :: Thu, 09/08/2005 - 12:04pm

I think it might be from some movie, actually, but I can't remember.

Actually, though, from an analytical point of view, it could be telling you something else: it could be telling you that the conversion from DVOA to estimated wins/predicted wins is off.

Mostly I just think that B's right. Helps that it means that Philadelphia only lost in 2002 due to supernatural influences.

97
by Led (not verified) :: Thu, 09/08/2005 - 12:17pm

Yikes. I just read Cimini's Jets preview article and a disturbing amount of it is lifted from Pro Football Prospectus without attribution. Not exact quotes mind you, but thoughts, opinions, factoids. He DOES give FO credit for the passing DVOA numbers in the OTHER article, but still. I think the FO staff has a reason to be a little annoyed.

98
by B (not verified) :: Thu, 09/08/2005 - 12:54pm

Just look at the evidence: The Bucs, after firing the only decent coach in team history, hire a guy that looks like a possessed doll, whom they obtained through a deal with the Devil's representitive in the NFL (Al Davis). They win thier superbowl, beating the Raiders to do so (And the Eagles, another cursed team, not really sure what's going on there.), but then the team is beset by bickering and in-fighting (see Keyshawn and Sapp), star players are cast off, a series of underwhelming QBs are installed, back to back (to back) losing seasons are endured. Of course, the final disgrace will be watching the discarded Dungy win his ring with the Colts. Untill that happens, the Bucs will be left to wander the darkness.

99
by DavidH (not verified) :: Thu, 09/08/2005 - 1:01pm

#87:

Cool article, thanks for pointing it out. Just wish it was a little longer.

100
by jhutt (not verified) :: Thu, 09/08/2005 - 10:23pm

Re: Injury Probabilities

If you attempt to include injury probabilities in these numbers, you should not only include likelihood of injury occuring against certain teams but also likelihood of injury occuring while playing on certain fields. Is a team playing the Ravens more likely to get an injury playing on the Ravens turf or on their own playing surface? While no stadium is as notoriously bad as playing at the old Vet, I expect there are a few venues that incur more injuries.

101
by Miles (not verified) :: Thu, 09/08/2005 - 11:45pm

I wonder if the 3rd down concept should be slightly modified when dealing with 3rd and a mile. It seems to me that coaches sometimes view sucess on 3rd and a mile as 'not losing yardage.' As a result they will sometimes run a dive play up the gut -- clearly knowing this will not gain 16 yards. I have no idea how frequently this happens (just saw the Raiders do it), or if it *really* affects DVOA calculations...

So, the suggestion: for runs on 3rd and really long (15+ (?) yrds), change the definition of sucess to something more like 1st down --45% of yards would likely be way above the coach's expectations.

The loss of yards on 1st and/or 2nd will likely serve as a sufficient penalty -- increasing the penalty for failing on this kind of 3rd down play result would seem remove some of the stat's value.

102
by Capt Phil Martin (not verified) :: Fri, 09/09/2005 - 2:13pm

Redskins get back key injured players at Linbacker, Safety, RT. Teams always perform better the second year under the same system and the team upgraded at both center and in thier special teams. They should have better field positon and the Special teams may even win one for the skins this year. All that plus a more mature Ramsey at QB using the shot gun and Portis being used more like Gale Sayers than Jim Brown and maybe even a little at H-Back should enhance the Skins chances at the playoffs....this year. Besides the new wideouts are getting a lot of knocks but the fact is that they keep getting open and that may be the difference in a 6-10 or 10-6 team this year.

103
by Fhbrown (not verified) :: Fri, 09/09/2005 - 8:44pm

I should have known that my new favorite group of football geeks comes predominantly from my alma mater. It drives my wife crazy. I spent many happy hours in the basement of Zeta Delta Xi in the late 80s...

It seems to me that the choice of SD is rather arbitrary and doesn't have as much meaning as a classic Poisson distribution giving a range of +/- the square root of the mean. Although in this case that might be confusing as well. Recently, I came across the concept of the "Full width half maximum" which gives a width of the Poisson curve at its half value peak -- the bigger the number the more variable the curve, etc., etc. I suppose it essentially accomplishes the same thing, but the range is more in line with +/- wins rather than a larger number whose origin baffles me and means something only in relative terms.

My two cents, for whatever it's worth.

104
by james (not verified) :: Fri, 09/09/2005 - 10:20pm

Computers have never been good at predicting anything.

It's a nice exercise and its good to see what teams would be good ina perfect world.

The other stats offered on this site are awesome. This information is useful. Taking into account info from previous coaching staffs just doesn't make much sense.

The idea is great, the inputs are wrong.

With parity the average nfl team should be around 5-3 at home and around 3-5 on the road. When you start to figure out which team are deviated from that then you start to figure out which teams have been lucky and what teams have been good.

My idea computer system would have all the teams with hof qbs automatically in the playoffs and then figure out everyone else. GB, Philly, Indy, and NE would be automatics. Everyone else would be variable. I think that would be a good start.

You run a good site, this article is almost useless.

105
by Jeff F (not verified) :: Sat, 09/10/2005 - 5:16am

james - If "computers are bad at predicting things", then why do we use them to predict things all the time? Also, I have no idea what this "parity" in the NFL is that you speak of. The salary cap makes it harder to have a continuously great team, but the Eagles are a shining example of how that is possible, and McNabb just had his first truly great year, numbers wise, he has been supported by fantastic defense and special teams through the past few years.

The "Average" team will not be "5-3 at home, and 3-5 on the road". They will have probability (x) to attain those home/road records, and that probability is likely to be under 20-30%. An average team could be 7-9, 9-7, 6-10, 10-6, or even 11-5 or 5-11. Hell, they could go 13-3, however unlikely.

If an average team has a 55% chance to win a home game, and a 45% chance to win an away game, assuming a schedule of average difficulty, the chance of them going 16-0 is even there, if astronomical: they have a 0.84% chance of going undefeated at home, a 0.17% chance of going undefeated on the road, and a .0014% chance of going undefeated in the regular season, where they are likely to lose in the playoffs. Conversely, they have about the same odds to go 0-16. Yeah, in the real world, this isn't likely to occur. But, an 11-5 season probably has a 1-3% chance of happening, which would mean that every few years, an "average" NFL team makes the playoffs, which seems like a reasonable assertation to make.

So, just because the Packers have Farve, they will be making the playoffs? This is what we are trying to get away from. Why not just pick the previous division winners? It has been established that one can do a bit better than that. As with last year, the projection system is merely an "experiment" if you will, they are trying to project things more accurately.

The NFL will always be unpredictable. Small sample size, incorrect estimation of capabilites, and variance, in particular, will kill you (Here's one, for you. I play poker, and I *expect* to win money at it. I play in such a manner that follows computer generated probability numbers that determine the value of a hand in a given situation, even having it worked out to the expected value to me, positive or negative. But, as poker is unpredictable, even if I play perfectly, the random way the cards fall will always lend a level of unpredictability to my results. I can expect, in the very long term, to make a couple bucks an hour playing poker. In the short term, the apparent losses and gains are MUCH more significant than my expected return. I can go up a couple hundred bucks in a week, and drop it in the same amount of time. Historically, I've been earning about 50% more than I expect to, so I'd expect that to creep downwards a little bit in the very long term, because it will (sample sizes in the 10s and 100s of 1000s will do that for you).

There's no such thing as attaining perfect predictions of apparently chaotic events, but in the case of sports, if you can even get a moderate edge, you are in good shape.

Maybe we should run 1000 trials of Madden 2006 seasons to see how the stats compare ;)

Then, we should add 1000 trials of Madden 2006 seasons with edited players with stat values more in line with what is appropriate, and see if/how that changes things?

I'm only half kidding.

106
by Jay (not verified) :: Sat, 09/10/2005 - 10:58pm

How to you matchup 2 teams to make a prediction on a game with a point difference?

107
by Chris (not verified) :: Sun, 09/11/2005 - 11:34am

You can't just say "Average teams are 8-8, teams with good QBs instantly get in the playoffs" - Many times, they don't. and many of the teams that do make the playoffs are not teams with superstars at QB, but are teams with a QB whose competent and simply isn't a screw up.

If you flip a coin 10 times and it lands heads all 10 times, what are the odds it will land heads on the 11th? 50%. You can't just say "Oh! Parity and law of averages, it'll be 5 heads, 5 tails"

108
by Larry (not verified) :: Sun, 09/11/2005 - 6:01pm

J

E

T

S

SUCK SUCK SUCK

Way to start on your journey to the Super Bowl.

109
by DavidH (not verified) :: Sun, 09/11/2005 - 7:22pm

Well, you can't blame them for low offensive output when they're facing a defense as consistently dominating as the Chiefs.

110
by Paul K (not verified) :: Sun, 09/11/2005 - 9:46pm

Re:108

Don't worry...Nugent will bounce back and help this team finish the year off 15-0, on their way to the Super Bowl, while scoring all of Gang Green's points.

111
by Larry (not verified) :: Sun, 09/11/2005 - 11:13pm

Yeah, it's a good thing they didn't use that second round pick on a CB.

At least they will be able to get rid of Herm after this season.

112
by GBS (not verified) :: Mon, 09/12/2005 - 10:31am

How about that? Those 7% 49ers stand alone atop the NFC West.

113
by John R (not verified) :: Mon, 09/12/2005 - 5:17pm

I am new the site this year. I was wondering if you adjust the DVOA projections as the season goes along?

The folks over at baseball prospectus have a very crude simulation run every day that computes a team's probability for division champ, wild card, etc. I am pretty sure it is very crude and full of holes - but it is an interesting jumping off point.

Doing something similar with these projections would be great. Is there a way to use past results during the season to update your future projections?

114
by B (not verified) :: Mon, 09/12/2005 - 5:53pm

John R: They will run a new set of projections at mid-season, based on the first half of the season's data. And they compile each team's actual DVOA throughout the year starting at week four. For the first three games we just get the teams VOA score, cause there's not enough data for defensive adjustments.

115
by james (not verified) :: Mon, 09/12/2005 - 9:36pm

You can predict wins by the number of points a team is projected to score and give up. This is how Beane puts together those teams in Oakland every year.

Take out all int, and special teams tds of the teams score from last year before you start. Add them to the points against if intercepted from the red zone. This is arbitrary but it gives an indicator of how lucky a team may have gotten. Add to points given up for schedule differences. For instance SD plays a first place schedule so add 40 points automatically for NE and Indy and subtract 20.

Give Oakland 370 automatically since Moss has never played on a team that averaged less than 23. Then give them more credit for playing last place defenses twice more than the rest of their division.

Go on and on but just use common sense. You cannot use a computer. You have to almost go through every team play by play and substitue lucky plays for regular plays.

Here's what I came up with:

Pats, Indy, Cinci, Bal, Oak, KC in AFC

Phi, Atl, Sea, Stl, GB, Was in NFC

116
by james (not verified) :: Mon, 09/12/2005 - 10:48pm

re 105
I follow football religously and am pretty successful at predicting outcomes for the season for the past 3 years. DVOA and all that stuff are great for measuring players and coaches but not very successful at understanding something as "chaotic" as football. It takes the human touch to get a handle on these events.

Computers are useful for analyzing past performance. Hence you know that Champ Bailey is actually an average player. However, using them to predict the future is not better than using the old human mind and a little common sense.

Watch the games or poor over the data to predict. Use computers to analyze past performance and make calculations that the mind cannot make.

117
by Chris (not verified) :: Tue, 09/13/2005 - 11:04am

One of the reasons I like DVOA is it generally matches up with what I see on the field. It's not always 100% accurate, but it lets me confirm suspicions.

Like a few years ago when the Packers D was doing pretty well under Donatell and people were still saying "Typical Packers D, getting thrashed." Whenever I watched them, I thought they were playing pretty well - and DVOA confirmed it.

A lot of people here have been saying you can't predict something as chaotic as football - however, just saying "you can't" is never really a valid reason. Also, the goal here isn't to predict the seasons but moreso to supplement info we already know.

and I know also that DVOA is a hell of a lot better than the idiotic stats the NFL uses to keep track of Offense and Defense (Yards gained/Yards allowed respectively)

118
by B (not verified) :: Tue, 09/13/2005 - 12:13pm

james: If your system comes up with Cinci, Oakland, Green Bay and Washington as playoff teams, I'm going to have to go ahead and say there's something wrong with your calculations. Seriously, did you watch Collins throw the ball 5 feet over Moss' head last week? Or Washingtons' continued offensive woes, or Green Bay get crushed by the Lions? Cinci is a good wildcard team, but I don't see them overcoming the Steelers D just yet.

119
by james (not verified) :: Tue, 09/13/2005 - 3:56pm

B,

Flawed teams always make the playoffs. It's fine to say that certain teams are bad such as Raiders, Skins, and Green Bay.

About 4 bad teams make the playoffs every year. Instead of picking apart my selections why don't you explain why there other teams so much better that this wont happen.

120
by MilkmanDan (not verified) :: Tue, 09/13/2005 - 6:17pm

Yes, James, it is certainly odd that a site dedicated to hardcore statistical analysis of football would use statistical analysis to predict the results of next season. That does seem strange.

"Bad teams always make the playoffs" is a pithy statement, not a projection. Hey, if bad teams make the playoffs, I've got Cleveland and San Francisco meeting in the Super Bowl. What the heck.

121
by DavidH (not verified) :: Tue, 09/13/2005 - 6:27pm

Come on now, no need to pick on James. He's right - bad teams do make the playoffs. How do you think Minnesota and St. Louis made it last year?

122
by james (not verified) :: Wed, 09/14/2005 - 12:45pm

Milkman you are just playing semantics. You said nothing to disprove my point that some bad teams are gonna make the playoffs. Not the worse teams, bad teams. San Francisco is gonna give some teams fits this year. I know why. Do you? No, you probably let Sean Salisbury give you your opinion.

Not knocking the site for using stats to predict the season. I'm saying you have to take every stat with a grain of salt. You have to know what happened on every play to appreciate what stats represent. I'm saying the way they are using it is wrong. Including special teams is an especially big error since these guys are the young guys who start to deserve more money and are the first to go off of teams the next seson.

Why not do this for teams last season and take out the lucky plays? Take out long touchdown runs when the other team stopped trying. Take out plays called off by false penalties. Take out all returns for touchdown(these performances are never repeated..see KC last year for an example). You get a much better look at the shell of the team this way. It takes forever I know.

Remember what I said here and eat your words at the end of the season.

123
by B (not verified) :: Wed, 09/14/2005 - 4:06pm

Sadly, I don't have enough salt to take with james' "stats."
Anyways, I've been asked to make some predictions:
AFC: NE, Pitt, Indy, SD, Bal, KC
NFC: Phi, Min, TB, Sea, NYG, Atl

124
by james (not verified) :: Wed, 09/14/2005 - 4:45pm

B,
Think we might agree on something ever?

Think Pitt gets that lucky again?

Winning games with running backs throwing? Winning games bc the D does all of the dirty work? Winning games throwing less than 20 times a game?

If they repeat that performance then I will have the utmost respect for Pittsburgh. Lets not forget we have never seen Cowher have two straight awesome years. Every other year we drink the Pittsburgh Kool-aid and get sick. Luck played a big part in their season and the playoff game stats were more indicative of what they can expect this season.

S.D has two automatic losses plus Philly and Pittsburgh.

Minnesota won't score 20 points a game this season. There defense will be dog tired by game 8.

Can't disagree with Tampa Bay or New York Giants on paper. My problem with both of them is scheduling:
Washington schedule is easier than NYG on account of Bears and Tampa Bay have no offense. NYG drew the short straw and finished second so they draw New Orleans on a night basically honoring them and Minnesota.

Tampa Bay plays AFC East(Pats, Jets, Buffalo, Miami) and four dog fights against four tough defenses.

Washington plays AFC West(oak,kc,den,sd) and four cat fights against maybe 1 tough defense.

Teams that scored 20 points against DC last year, please note that defense is improved: Phi(28), GB(28), NYG(7 turnovers,20), Dallas(21). Thats it thats the list. Some questionable pi calls helps two of the teams reached that plateua and one had 7 turnovers for a gimme.

Tampa Bay defense last year(8 times). Defense improved? Not really.

Nyg defense last year(9 times)
defense improved? not really.

times New York scored 20(8 times). Gonna go out on a limb and say 8-8 this year

times tb scored 20 last year(8 times)
gonna go out on same limb and say 8-8.

Here is the kicker. Redskins dont have to score 20 bc there defense doesnt give it up. out on same limb skins better than giants and tb with a easier schedule.

The other 5 NFC playoff teams from last year did nothing to become worse
GB- new D coordinator plus still Favre and Green = points on O and less on D. same record
Sea-Problem was attitude last year not players. Got rid of Robinson king of all drops
StL-moved Jackson to starting running back-400 points here we come
Phi-Still McNabb TO and Westbrook plus vaunted D
Atl-Vick Crumpler run game defense excitement

If you read all the way to here you have stamina and you are smarter for doing so.

125
by B (not verified) :: Wed, 09/14/2005 - 6:09pm

james: As it turns out, we do agree on some things, just not about Pittsburg, I guess. I don't think they'll be as good as last year, but 11-5 should be enough for that division. You have a lot more faith in Cincy than I do, but I figure that Pittsburg's D will be just as ferocious, and thoer O-line will be opening up big holes for thier running backs that they can with throwing the ball 15-20 times a game. They already one one throwing 11, so I guess I'm onto something.
I agree with your evaluation of Tampa Bay, but don't forget that the rest of thier divison has just as brutal a schedule, they might end up with a worse record than the Skins, but they'll still win the NFC South. SD was a stretch, and I didn't like picking 4 div winners to repeat, but somebody has to win the West, and I don't beleive in Denver anymore and KC is to old to keep it up through the year. On the other hand 7-0 + 3-6 = 10-6, maybe KC can do it. If you think Oakland can actually win the west, you must have more faith in Kerry Collins than I. And i think you're missing an important point on GB, they still have Favre and Green, but they lost thier two tackles, so Green won't have big holes to run though, and Favre'll be under pressure, which is when he makes bad descisions, which turn into turnovers. And they have a new DC, but they're still missing thier top corner from last year, cause Sherman cut him. And an injured J-walk doesn't help the situation much.
For the NFC West, honestly I have no idea who will win that division. SF isn't ready, Arizona cant do it as long as Warner is under center. Saint Louis wasn't nearly good as thier record last year. Seattle's defense is still suspect and they replaced one guy with stone hands with another guy with slightly less stony-hands.

Anyways, I'll revise my picks, just for you:

AFC: NE, Pitt, Indy, KC, Cincy, Buf.
NFC: Phi, Min, TB, StL, Atl, NYG.

I really want to give the Redskins that last wildcard spot, but I just can't do it until they find a real QB (Brunell and Ramsey just won't cut it), not that Eli is much better, but still....

126
by james (not verified) :: Wed, 09/14/2005 - 8:21pm

wow,

I'm stupid. I totally didn't realize that GB lost both tackles. Where have I been? Thanks B

Guess we can also both agree that the NFC is just brutally bad.

AFC is brutally good.

127
by B (not verified) :: Wed, 09/14/2005 - 9:29pm

See, we can agree. Now I understand why you were so high on Green Bay before.

128
by Cael (not verified) :: Thu, 09/15/2005 - 1:05pm

From PFP 2005 entry for Kerry Collins, "Part of the KUBIAK projection system involves trying to square the receiver projections with the quarterback projections, and the difference between the projection for Collins and the projection for his receivers was by far the greatest."

That sounds like it could be an indicator of increased uncertainty for overall team performance. Is there some sort of bump upward for StdDev of DVOA due to wide disparity in projected performance of individual players?

129
by GBS (not verified) :: Thu, 09/15/2005 - 4:26pm

Green Bay lost both of their GUARDS. They still have Clifton and Tauscher.

130
by james (not verified) :: Wed, 10/05/2005 - 3:38pm

revision of playoff teams

original nfc phi,atl,sea,gb,was,stl

changing it too phi,atl,sea,gb,was,stl

just for arguments sake later when the bears win and I get to say I told you so