Writers of Pro Football Prospectus 2008

Most Recent FO Features

ProctorSpe15.jpg

» VN: Hits, Misses, and the Year Zero Effect

Bill Connelly looks at the college offenses, defenses, and overall teams that have improved (Air Force!) or regressed (North Texas!) the most in 2014. Year Zero is a real thing (sometimes).

05 Sep 2007

2007 DVOA Projections

by Aaron Schatz

OK, folks, here they are: the 2007 DVOA projections. However, before we get to the projections, let's clear up some issues about DVOA in the upcoming season.

DVOA (and its early-season cousin, DAVE) will still be updated on Football Outsiders every Tuesday, and will remain absolutely free. Our new premium database will include all the splits of 2007 DVOA beginning in Week 2 -- by down, by zone, by single game, etc. -- but the general DVOA ratings will still run for free, just as they have every week since Football Outsiders started four years ago.

DVOA commentary will no longer appear at FOXSports.com. DVOA commentary, and some of the other material that used to appear on FOX, will probably be appearing somewhere else, but I can't quite say where yet. For now, DVOA may go back to the old style of commentary, otherwise known as "Aaron rambles about whatever he feels like that week." Some of our articles will continue to run on FOX, including Rundown and Quick Reads.

OK, enough digression. Here's the requisite link to an explanation of DVOA, which stands for Defense-adjusted Value Over Average and measures a team's performance on every play of the season compared to league average in the same situation, adjusted for opponent. I know a lot of people are coming here from various message boards and this is just going to look like a jumble of pointless numbers. Trust me, there is a method to the madness, and over the past seven seasons DVOA has been a far more accurate predictor of future performance than wins or points.

Offense, defense, and special teams DVOA are all projected separately using a system based on 2000-2006 numbers. The equations include a number of variables based on performance over the past two seasons in different splits (by down, passing vs. rushing, red zone vs. whole field) plus variables based on recent draft history, injury history, offensive and defensive pace, coaching experience, quarterback experience, and even weather. Strength of schedule was then figured based on the average projected total DVOA of all 16 opponents for 2007 (yes, projected performance, not 2006 performance).

There are no manual adjustments. The numbers we are presenting here are exactly what the projection system spit out. A few of them will look strange to you. A few of them look strange to us. At the bottom of the page, we'll talk a little bit about the reasons for some of the projections that disagree with conventional wisdom.

The projections here are updated from Pro Football Prospectus 2007 based on changes in some of the variables, usually related to injuries, offensive line continuity, and quarterback experience.


TEAM TOTAL
DVOA
TOTAL
RANK
OFFENSE
DVOA
OFF.
RANK
DEFENSE
DVOA
DEF.
RANK
SPECIAL
DVOA
S.T.
RANK
SCHED SCHED
RANK
NE 31.9% 1 25.7% 1 -5.1% 9 1.1% 7 1.6% 9
PHI 26.0% 2 22.2% 3 -3.9% 12 -0.1% 18 0.2% 12
JAC 24.7% 3 8.2% 6 -16.6% 1 -0.1% 19 -2.5% 28
BAL 12.0% 4 6.0% 11 -6.5% 7 -0.5% 22 -0.9% 17
WAS 11.2% 5 -1.8% 15 -11.5% 4 1.5% 3 1.8% 7
TB 10.5% 6 2.9% 14 -7.5% 5 0.1% 17 -1.4% 21
CAR 10.5% 7 -2.8% 17 -12.0% 3 1.3% 5 -1.8% 23
PIT 10.2% 8 -3.5% 19 -14.9% 2 -1.2% 26 -0.1% 13
SD 8.8% 9 18.4% 4 10.8% 29 1.2% 6 -2.0% 25
NYJ 7.8% 10 7.8% 7 0.4% 19 0.4% 12 1.7% 8
IND 7.0% 11 22.8% 2 15.2% 32 -0.6% 23 2.1% 6
GB 5.7% 12 5.0% 12 -5.9% 8 -5.2% 32 -3.0% 29
CHI 3.8% 13 -5.6% 20 -6.6% 6 2.8% 2 -2.2% 27
DEN 0.2% 14 6.5% 9 5.9% 24 -0.4% 21 -3.4% 30
ATL 0.2% 15 -2.2% 16 -3.9% 11 -1.5% 28 -1.0% 19
SEA 0.1% 16 3.0% 13 3.0% 22 0.1% 16 -2.1% 26
TEAM TOTAL
DVOA
TOTAL
RANK
OFFENSE
DVOA
OFF.
RANK
DEFENSE
DVOA
DEF.
RANK
SPECIAL
DVOA
S.T.
RANK
SCHED SCHED
RANK
SF -1.5% 17 -6.2% 22 -4.4% 10 0.2% 14 -4.5% 32
CIN -3.5% 18 6.9% 8 10.0% 27 -0.4% 20 -1.9% 24
NYG -3.5% 19 6.2% 10 7.7% 25 -2.0% 29 4.1% 4
NO -4.5% 20 9.1% 5 13.6% 31 0.1% 15 2.2% 5
BUF -4.7% 21 -6.3% 23 2.0% 21 3.7% 1 6.8% 1
TEN -6.6% 22 -3.5% 18 4.0% 23 0.8% 9 1.3% 10
MIN -9.0% 23 -9.0% 25 -1.1% 16 -1.1% 25 -0.4% 15
DET -9.5% 24 -12.9% 28 -2.8% 13 0.5% 11 -0.8% 16
CLE -9.7% 25 -10.7% 27 -0.4% 17 0.6% 10 -1.0% 18
DAL -9.8% 26 -9.1% 26 -0.1% 18 -0.8% 24 4.1% 3
OAK -12.2% 27 -14.4% 29 -1.3% 15 1.0% 8 -3.6% 31
MIA -14.3% 28 -17.1% 32 -1.5% 14 1.3% 4 4.6% 2
HOU -17.3% 29 -5.7% 21 10.3% 28 -1.2% 27 0.9% 11
ARI -18.6% 30 -17.0% 31 1.8% 20 0.2% 13 -1.5% 22
STL -24.3% 31 -7.4% 24 13.2% 30 -3.8% 31 -1.2% 20
KC -25.6% 32 -15.2% 30 8.1% 26 -2.3% 30 -0.2% 14

Here are some updated projections for the season, including each team's chances to make the playoffs according to the DVOA projection system. Just as we did for the book, this projection plays out the season 10,000 times, with some adjustments to lower the number of teams with records of 0-16, 1-15, 15-1, and 16-0. We give the mean projected wins for those simulations, as well as the standard deviation, the percentage of time each team won the division, and the percentage of time each team made the playoffs. The higher the standard deviation, the stronger the chances this team will play either better or worse than its projection. Thank you to Dr. Ben Alamar, who wrote the code to create the simulations.


AFC
East Division Playoffs Wins StDev
NE 55.0% 91.6% 11.8 2.77
NYJ 30.0% 38.2% 8.7 2.07
BUF 8.0% 9.3% 6.2 2.48
MIA 7.0% 8.2% 5.8 2.35
North Division Playoffs Wins StDev
BAL 34.2% 53.4% 9.2 0.92
PIT 32.0% 48.6% 8.9 2.82
CIN 22.4% 34.3% 8.3 1.66
CLE 11.4% 17.2% 7.4 1.61
South Division Playoffs Wins StDev
JAC 53.0% 88.7% 11.3 0.89
IND 28.0% 36.2% 8.8 0.77
TEN 11.0% 13.4% 6.8 1.71
HOU 8.0% 9.4% 6.2 1.45
West Division Playoffs Wins StDev
SD 44.1% 66.1% 8.9 2.18
DEN 37.6% 56.7% 8.6 2.47
OAK 14.3% 22.4% 6.9 1.63
KC 3.9% 6.1% 5.1 1.40
NFC
East Division Playoffs Wins StDev
PHI 65.3% 96.4% 10.9 1.26
WAS 20.7% 30.7% 8.5 1.60
NYG 9.7% 14.6% 7.2 1.33
DAL 4.3% 7.2% 6.5 1.79
North Division Playoffs Wins StDev
GB 46.7% 69.5% 9.4 1.32
CHI 25.8% 38.8% 8.1 3.59
MIN 14.4% 22.2% 7.3 1.79
DET 13.2% 19.5% 7.2 0.74
South Division Playoffs Wins StDev
TB 35.8% 53.9% 9.1 1.96
CAR 34.2% 51.0% 9.0 2.05
ATL 18.0% 27.7% 8.0 3.45
NO 12.0% 19.1% 7.5 3.11
West Division Playoffs Wins StDev
SF 40.3% 60.5% 8.3 1.42
SEA 35.2% 52.8% 8.0 0.96
ARI 14.7% 20.9% 6.2 3.30
STL 9.8% 15.2% 5.9 2.75

Here's a look at the teams whose mean projected wins have changed by more than 0.25 since the book, and why. First, the teams that have dropped:

  • Jacksonville: down 0.51 wins based on quarterback change and Brad Meester injury
  • Cincinnati: down 0.47 wins based on injury issues on the offensive line
  • New England: down 0.29 wins based on Richard Seymour injury
  • Cleveland: down 0.27 wins based on adjustments to 2006 injury variables
  • Pittsburgh: down 0.26 wins because new center and right tackle change OL continuity variable

Now the teams that moved up:

  • Oakland: up 0.43 wins, primarily due to quarterback change
  • Atlanta: up 0.33 wins after I tried to approximate what a Vick-free offense might look like by removing Vick from rushing numbers and merging Harrington's numbers with the Atlanta team passing numbers -- this actually ended up improving their projection
  • San Diego: up 0.31 wins; see below
  • Baltimore: up 0.26 wins, mostly because the other three AFC North teams were moving down

Finally, let's see if we can answer your questions about the teams where our projections differ most from the conventional wisdom.

Tampa Bay and New Orleans: Read here. Short version: we don't believe these projections either, and the system doesn't understand that two-year variables may not be useful for the Saints because one of those seasons was destroyed by a hurricane.

Green Bay, Jacksonville, and Washington: Read here.

Indianapolis: Other than a small drop on offense, this forecast comes close to the Colts' actual 2006 regular-season DVOA. Remember, they were a completely different team in the postseason. That being said, I don't think a single FO staff member submitted a set of picks that didn't include the Colts as one of the six AFC playoff teams.

Chicago: Simply put, the odds they can continue to dominate on defense and special teams are not good, and it seems unlikely that the offense would improve enough to make up for that.

Dallas: A quick summary of the biggest issues here: 1) Third-down offense far better than overall offense; 2) Romo regressed at the end of the season; 3) worst defense in the NFL last six weeks of 2006; 4) new coordinators means learning new systems; 5) extremely unlikely that entire offensive line will remain healthy for all 16 games again.

San Diego: This is an interesting one because I've made some changes between the book and now. The main reason for the projected San Diego decline is not the new coaching staff, but rather the very specific problems that the defense had last year. The Chargers had the worst red-zone defense in the NFL and ranked 30th in third-down run defense. Third-down run defense is a negative indicator, separate from the overall "third-down rebound" trend. However, I realized over the last couple of months that it might be worthwhile to project San Diego using only the weeks when Shawne Merriman was in the lineup. The defense was significantly worse without him, and he should be there for the entire 2007 season, barring injury. This change improved San Diego's overall projection by a small amount -- not enough to suggest that they might match last year's 14-2 record, but enough to move the Chargers ahead of the Broncos as the most likely winner of the AFC West.

Why didn't removing Weeks 9-12 have a larger effect? While the overall San Diego defense was much better with Merriman in the lineup, both of these negative indicators were actually even more negative once we removed the weeks without Merriman.

Overall
Defense
3rd Down
Run Defense
Red Zone
Defense
San Diego DVOA with Merriman: Weeks 1-8, 13-17 -6.9% 48.0% 65.0%
San Diego DVOA without Merriman: Weeks 9-12 12.3% 29.5% 6.7%

Tomorrow, we'll have our annual look at staff predictions for 2007.

Posted by: Aaron Schatz on 05 Sep 2007

171 comments, Last at 27 Sep 2007, 12:51pm by Joe

Comments

1
by D (not verified) :: Wed, 09/05/2007 - 7:13pm

Any answer to why Chicago and Arizona have such high standard deviations? Atlanta and New Orleans don't surprise me, but does Rex Grossman really cause that much variation in the simulations?

2
by Kal (not verified) :: Wed, 09/05/2007 - 7:20pm

I'm still confused about Carolina's rise to power. How is their defense projected to be so great? Heck, why is their offense projected to be anything better than mediocre? Or is this again an artifact of Katrina and NO?

3
by AlexDL (not verified) :: Wed, 09/05/2007 - 7:27pm

I can't get over how it's possible for the AFC South to be projected in such a manner. Is Indy's defense and special teams really that bad?
Is the Jags defense really that good?
WOW.
The Jags really the third best team in the league?

I read the article and I understand that they were an inconsistant team last year, but what other than the "projection to the mean" would explain the belief that they won't be as inconsistant as last year. They have the same coach and he is the person that is ultimately responsible for how a team plays week to week.

4
by James G (not verified) :: Wed, 09/05/2007 - 7:27pm

Is it really smart to add a variable to minimize 0-16, 1-15, 15-1, and 16-0 teams? I would guess that would artificially deflate (or inflate) the mean records of the best teams (worst teams).

5
by JCRODRIGUEZ (not verified) :: Wed, 09/05/2007 - 7:32pm

AFC North Title will be a race worth watching, I think that a soft early schedule for the Steelers could help the o-line to gel before being truly tested by the others minefield-like AFC contenders

6
by clonmullin (not verified) :: Wed, 09/05/2007 - 7:32pm

This is why I love this site. Everyone else predicts 6/7 repeats for Divisions whereas your projections give you Tampa & Green Bay. And you roll with them ...

7
by Richie (not verified) :: Wed, 09/05/2007 - 7:43pm

Does this mean I should rethink my suicide pick of Seattle over Tampa Bay?

8
by Jimmy (not verified) :: Wed, 09/05/2007 - 7:44pm

I love this site and almost everything you write, but I don't know where to begin with all the places where these projections look screwy. I would like to say more but the message board curse compels me to stop.

9
by name withheld to avoid FOMBC (not verified) :: Wed, 09/05/2007 - 7:53pm

I agree with #8. I'd also like to add that I fear for my team this year. I've learned from the Falcon's fans. I dare not complain, but it still looks screwy to me.

10
by Raiderjoejoe Joe (not verified) :: Wed, 09/05/2007 - 7:54pm

Rairds at =12% a joke. Defense will be great, but so are Cullpepper and entire offense do for improvement. Win the division by Saturday, first rond bye by week 4. Entire offense rto prow bpwl!

11
by Isaiah C (not verified) :: Wed, 09/05/2007 - 7:59pm

I think it's amazing how wide of a variation there is between all the other sites projections and DVOA. Of course, this is the only statistically based one, but it still makes me scratch my head. For instance, Dallas at most other websites in the top ten, DVOA has them at 26. Wow.

12
by brasilbear (not verified) :: Wed, 09/05/2007 - 8:00pm

Being in Brazil, I don't have access to the book and recent things shipped haven't arrived so...I have a Favre question. I'm thinking we will see a decline in skills this year, anyway we can get some similarity scores?

I like the standard dev for the Bears, gives me hope.

13
by Peder (not verified) :: Wed, 09/05/2007 - 8:06pm

I'm not so much surprised at Indy's projected win total as I am by their very small standard deviation. Very surprised at how confident DVOA is about them.

14
by iapetus (not verified) :: Wed, 09/05/2007 - 8:09pm

How does this level of screwiness compare with previous years? How does it compare to the difference between reality and the predictions that this looks screwy against?

Every year the 'standard' predictions are just too conservative - they don't have enough changes at the tops of divisions, and they ignore the fact that you regularly do see teams jump from one end of the division to the other. I'd expect an accurate prediction to look somewhat screwy at first glance - and less so as the season progresses.

That's not to say I believe in the accuracy of this prediction - it has things that I find hard to stomach too - but I'm more open to a prediction that goes against common belief to this level than some people appear to be.

15
by RMoses (not verified) :: Wed, 09/05/2007 - 8:10pm

DVOA loves Philadelphia!

16
by Yaguar (not verified) :: Wed, 09/05/2007 - 8:19pm

"Is Indy’s defense and special teams really that bad?"

Defense: They could probably be mediocre if Bob Sanders stays healthy.

Special Teams: Have you watched the Colts special teams? Ranking them 23rd is charitable.

17
by Irishfan (not verified) :: Wed, 09/05/2007 - 8:22pm

If an offence including Pace, Bulger, Holt, Bruce, Jackson and Bennett is 24th best in the league defences are in for a tough time this year.

Love the site and will grin and bear it until the season at this stage but very frustrating for rams fans.

18
by Chris (not verified) :: Wed, 09/05/2007 - 8:27pm

Is the Wash prediction based on their defense improving, for the most part? Who was injured last year and coming back on their D?

19
by Tom (not verified) :: Wed, 09/05/2007 - 8:32pm

It seems weird to me that the best projected defense is only going to be -16.6% DVOA. Looking through the last 5 years there has one been one defense better than that every year, and usually about 3.

20
by chip (not verified) :: Wed, 09/05/2007 - 8:39pm

#8/9 - I completely agree. I love the site, but I am beginning to wonder how data-mined these projections are. It's one thing to build a predictive model based on your core football knowledge, it's completely another to datamine past trends and assume they will hold true in the future. Comments like these from Schatz really scare me:
(from the NO / STL projection comments link) "When the projection system spits out these weird numbers, that gives me ideas for new variables I might try. The goal is to find out where we are missing something. Believe me, I tried TONS of variables that I hoped would make Tampa Bay lower and New Orleans higher this year. Nothing worked. (I also had tons of ideas to make the Indianapolis projection higher and the Jacksonville projection lower, and those didn’t work either.)"

And this: "The Chargers had the worst red-zone defense in the NFL and ranked 30th in third-down run defense. Third-down run defense is a negative indicator, separate from the overall "third-down rebound" trend".

21
by Ben Riley :: Wed, 09/05/2007 - 8:39pm

At least one FO staffer didn't pick the Colts to make the playoffs.

22
by Ole (not verified) :: Wed, 09/05/2007 - 8:43pm

If the 9ers pan out like this I'll be thrilled, but it may be the last straw in terms of trying to take anything meaningful from the preseason.

In our four games, our passing offense looked sharp, our running game looked very poor, but then there was no Gore, so it's hardly unreasonable to expect it to be somewhere around the level it was last year once he comes back. On the other hand, our passing defense looked mediocre at best, and our run defense ranged from being a flat out abomination at times to being very poor at others. It looked like another season of hoping our D wouldn't get us buried so deep that our offense couldn't dig us out.

But the projected DVOA seem to suggest that the strenght/weakness balance will be exactly the other way around, so who knows?

23
by Mike (not verified) :: Wed, 09/05/2007 - 8:45pm

Four categories, and my team tops two of them. Shame it's special teams and strength of schedule. I LIKE special teams, too, but there's only so much Brian Moorman and Terrence McGee can do for the Bills this year.

The big hope, it seems, is that Losman, Lynch, and Posluszny, likely our biggest variables, bust the projection.

24
by Yaguar (not verified) :: Wed, 09/05/2007 - 8:52pm

On the Redskins:

I made a post detailing my statistical and subjective reasons for why I thought the Redskins would easily pass the Cowboys and Giants to be the 2nd best team in the NFC East. I was dismissed as a Redskinjoe. (Click my name for the link.)

Obviously they've got a lot of depth issues, and I could easily see their season going bad, but the fact is their starters are pretty solid almost everywhere. I think their offense will probably be slightly better than the projections give them credit for. They have monster run-blocking, and I think a strong RB tandem will set Campbell up to be an efficient passer. (That's exactly what happened to Campbell at Auburn, incidentally.)

Obviously, I think 4th is considerably too high for the Redskins defense, because they are weak at CB and DL.

The Redskins defense is an interesting projection because of their two-year trend. They were 4th in defensive DVOA in 2005, and then 32nd in 2006. Generally a team that suffers a dramatic decline over two years improves, often dramatically, the third year. The 2004-2006 Bills ranked 1st, 26th, and 19th in defense those three years. The 2004-2006 Patriots ranked 6th, 27th, and 8th. 1996-98 Raiders: 10th, 29th, 2nd.

Maybe the Redskins might have a good defense, but once the inevitable Shawn Springs injury occurs, I can't imagine them being 4th.

25
by Paralis (not verified) :: Wed, 09/05/2007 - 8:54pm

18:

I'm as puzzled as you about the Redskins defensive injuries. Pierson Prioleau went down on the opening kickoff, and Springs missed about half the season. A few other players were banged up (Cornelius Griffin and Marcus Washington, particularly), but nobody else missed significant time.

The problem with the Redskins' defensive personnel wasn't depth--it extended into the starting roster. Kenny Wright was always going to be the nickel back. Warrick Holdman was expected to start, and so was Lemar Marshall. The improvement's going to come from promotion and free agents. There are 4 new starters on defense this year (5 if you count Smoot), and 3 (4) have at least a year in Williams' system, so the transition should be easy.

Is that enough to get them up to a top-5 defense? I don't have any idea. But I grew watching games at RFK, so I'm not going to be the one to argue it.

26
by brasilbear (not verified) :: Wed, 09/05/2007 - 8:58pm

That’s not to say I believe in the accuracy of this prediction - it has things that I find hard to stomach too - but I’m more open to a prediction that goes against common belief to this level than some people appear to be.

I'm very open to this prediction, thats what has me worried as a Bears fan. My head tells me FO is right more than they are wrong, but I still can't see the Packers coming out ahead of the Bears. Would I be surprised? No, these are the Bears we are talking about and we are still starting Grossman. But,...the Packers? Favre at age 37/38? With their running backs? I just have a hard time seeing a 12th ranked OFF there. (And I'm not debating the Bears OFF at 20, it seems about right.)

27
by Jacob Stevens (not verified) :: Wed, 09/05/2007 - 8:59pm

Screwiness: doesn't look that bad to me. I wish it looked screwy for the Seahawks, but it's definitely not far offbase.

Only things that look real offbase is Carolina being high, and Pitt not being a little higher. I'd expect Green Bay to be a little higher, too, now that you've all gone & sold me on em.

28
by Yaguar (not verified) :: Wed, 09/05/2007 - 9:01pm

19: "It seems weird to me that the best projected defense is only going to be -16.6% DVOA. Looking through the last 5 years there has one been one defense better than that every year, and usually about 3."

It's a mean projection. Jacksonville will certainly do better or worse than -16.6%, but the projection system thinks the average of all possibilities -16.6%. It's the same reason that only two teams have a projection of 11 wins.

29
by zip (not verified) :: Wed, 09/05/2007 - 9:07pm

#7

Does this mean I should rethink my suicide pick of Seattle over Tampa Bay?

Yes.

30
by blacksuit (not verified) :: Wed, 09/05/2007 - 9:07pm

re 22

I think the niners wisely played a pretty vanilla preseason. It will take a few games for the defense to really get going due to the inconsistencies up front, but the linebackers should be excellent. The offense didn't display any of the Smith/Davis/Gore combo that will be the foundation of this offense over the next several years. Lots of unknowns, but that's what makes nfl football fun.

31
by Mr Shush (not verified) :: Wed, 09/05/2007 - 9:08pm

Disclaimer: I firmly believe that DVOA is by far the best measure of team/unit quality out there, and that the DVOA projections are very likely to thrash my intuitions or those of any professional expert.

That said . . .

1. Unless Gaines Adams is a dominant (not good, dominant) pro pass rusher from the word go, I cannot believe the Bucs will rebound to that extent, given the loss of Rice and McFarland and the continued aging of other key contributors.

2. The Redskins still have horrendous depth problems. I'll defer to DVOA on the likelihood of them being good, but the standard deviation looks too low given how devastating even a few injuries could be.

3. Carson Palmer is too good for the Bengals offense to be only pretty good if he is healthy.

4. I find it very hard ineed to believe that replacing Vick with Harrington will not affect the performance of the Falcons' running backs. Both Dunn and Norwood will probably get more carries than they would have with Vick under centre, but their yards per carry and DVOA be lower than they would have been. There is virtually no chance the Falcons will be able to field a middle of the pack DVOA offense. If they can, we will know that Vick was not merely hurting but killing the production of their receivers.

5. Not a criticism, but we should all note the high variance on the Bears projection. The biggest reason for projecting decline for the Bears is the year-year variability of defense, but teams have been known to field elite defenses for three successive years (notably the Bucs in their heyday), and if the Bears can keep their key men healthy they seem to me to have a better chance of this than most such teams, given their comparative youth and the lack of major losses in free agency.

6. Loathe as I am to bring the wrath of the Curse upon my own team, I expect the Texans to have strong special teams DVOA this year, where the projections list them at 27th. Jerome Mathis is healthy, two preseason touchdowns suggest Jacoby Jones may be an elite punt returner, Kris Brown is good at kickoffs, which should carry over, and the awful Chad Stanley has been replaced by the excellent Matt Turk.

32
by Larry (not verified) :: Wed, 09/05/2007 - 9:33pm

The NO pick is definitely one to question, but I think the staff here is perhaps backing off too much by saying they don't believe it. They're still projected as the #5 offense in the league, and that's an offense highly dependent on two second year players (Colston, Bush) who may not be as good this year as last year. The NO defense wasn't that much better last year than two years ago, so there may be something to the stats from two years ago to suggest it will still be bad this year, hurricane or no hurricane.

33
by Pat (not verified) :: Wed, 09/05/2007 - 9:37pm

Is the Wash prediction based on their defense improving, for the most part? Who was injured last year and coming back on their D?

Oh yes. Washington's defense was pretty much near the worst in the league last year. They heavily - and I mean heavily - overhauled it, and when you have the worst defense and replace a bunch of the players, good chance it's going to be better, because it can't get much worse.

I touted Washington last year as a "better than they looked" team - the projection's probably a bit optimistic, but when you've got a team that's as thin depth-wise as the Redskins are, preseason projections are often optimistic.

And this: “The Chargers had the worst red-zone defense in the NFL and ranked 30th in third-down run defense. Third-down run defense is a negative indicator, separate from the overall “third-down rebound� trend�.

What the heck is so odd about that?

Here's what the third-down rebound trend means. A defense's performance on first, second, and third down isn't going to be that different. They're still the same players. They don't get magically better just because the down marker says "3". But third down has much more leverage on the effect of the game than first or second down, and so if a team manages to do better on third down than first or second, their defense will look much better than it actually is.

This idea is a recurring theme in all sports. Teams have a "true strength" and an "observed strength," and "luck" is when a team plays over their head. Sometimes it truly is luck! Imagine if a team has the worst punter known to man. If they face a team who has such a poor performance on defense that the punter never came into play, their poor special teams won't affect the game. They had no control over that fact, but it makes them look better than they are. That's luck.

The third-down run defense is also a leverage issue - why? Because a team with a poor third-down run defense can't stop another team from running out the clock when they're ahead - so third down run defense has more leverage to win/lose games. If they manage to avoid games where the other team gets ahead, it won't kill them, but again - that's just getting lucky with opponents.

It's like with Rex Grossman last year. It was noted a while ago that Bad Rex comes out versus competent pass rushes/defenses. Last year, they faced an abnormal number of poor pass defenses. You'd be crazy to bet on them getting that lucky again.

Some teams will get lucky two years in a row (someone has to) but in any projection, you have to assume that they won't.

34
by flash (not verified) :: Wed, 09/05/2007 - 9:39pm

God, Shawne Merriman is a monster.

35
by Mr Shush (not verified) :: Wed, 09/05/2007 - 9:54pm

Strewth, yeah. Obviously there's a major sample size issue and a number of other variables involved, but still, a 19 point DVOA swing? I imagine Manning and probably a few other quarterbacks would be worth that much or more, but . . .

36
by throughthelookingglass (not verified) :: Wed, 09/05/2007 - 10:12pm

Some observations
-I don't really understand why NE has a better projected offense than Indy
-Doesn't DVOA penalize for lack of depth? I'd think that would drop Washington
-Tampa Bay is not the 6th best team in the NFL, etc
-San Diego's 'bad' projection is fueled entirely by the 29th ranked projected defense. It's hard to imagine that a team with that defensive line, Merriman, Phillips, and a couple decent corners (Jammer and Cromartie) could be that bad.
-The Jets come out pretty well. Does the system think Pennington will stay healthy the whole year?
-I think Manning & co's ability to consistently move the chains may not be dealt with effectively by the system because of the lack of comparable reference points.
-The system is surprisingly optimistic about Atlanta, especially the defense.
-Has there been a 'Bengal arrest factor' addition to DVOA? The offense seems under-predicted even after accounting for oline problems. It's not like there weren't problems last year, and they were still pretty good.
-I wouldn't be surprised to see NO come close to that prediction (within 5%). This is a team that still employs Fred Thomas, after all.
-Why is Detroit predicted to have an above-average defense? Are they the Cardinals of '04 (underrated D, aging qb, good young receivers, no oline)?
-No love for Oakland's defense? I thought that with their young talent, especially concentrated in pass defense, they would be one of the few teams expected to overcome defense variability.
-DVOA hates Matt Leinart, apparently.
-This has been said before, but the STL offense should be better than that (even though I still think it's a 6-7 win team).
-The projections (especially on the defensive side of the ball) are very clustered.

37
by PackMan (not verified) :: Wed, 09/05/2007 - 10:13pm

26. I think our Defense is where we can make up. I don't think our offense will be the 12 best in the league either, but I think the Packers are not unlike the Bears about 2 years ago. Thomas Jones was no superstar runner, and Benson has been underwhelming. So expecting Morency/Jackson to be close to as good is not too much of a stretch, and our O-line should solidify some more with experience this year. Favre is still significantly better than Orton was as a rookie, that is for sure. And our receivers are as good if not better if Driver stays healthy. So it comes down to our defense. They are young (other than the corners) and our front 7 will certainly be the strength of our Defense if not the whole team. So if we can keep things close with our D, Favre should be able to minimize the mistakes if we're not down by 14 in the 4th quarter every week.

Also, rewatch the week 17 game from last year.

38
by Jason Mulgrew aka The Mul-Dawg (not verified) :: Wed, 09/05/2007 - 10:14pm

The Eagles.

You remember it.

The Eagles.

39
by BHold (not verified) :: Wed, 09/05/2007 - 10:14pm

Why do you guys discount the Giants being able to improve their record conciderably like the Jags (as top 10 DVOA teams at 8-8)? They were a top five DVOA team before injuries and all else struck at mid-year. You say because they got significantly worse in the 2nd half of the season. You do realize in the 2nd half they also lost their LT, most consistent WR (though not the most dynamic), 3 of their top 4 DEs, 2 starting LBers (although neither were very good), their top CB for a time (again, not a great player anymore, but their best starter last year (RW did better than Madison but he was nickel)). I realize theres no way you guys can add a "Got rid of 2 deadbeat coordinators" factor, but they did that this offseason as well. But back to the main point, shouldn't your 2nd half rise/decline numbers account for injuries?

40
by Just Another Falcon Fan (not verified) :: Wed, 09/05/2007 - 10:31pm

There are too many other variables (new coach, amazingly better offensive scheme, change in offensive line style) to attribute any change in the Falcons offensive DVOA this year solely to the change to Harrington from He Who Will No Longer Darken My Jersey.

I personally would not be surprised to see the Falcons finish 8-8, or possibly better if the key rookies (Anderson, Blalock, Houston, possibly Robinson) hit the high end of expectations. IF the Falcons get to 8-8, Petrino will win Coach of the Year in a runaway.

41
by sam (not verified) :: Wed, 09/05/2007 - 10:31pm

It's a bit disheartening to see Jacksonville's special teams at 28 with a new ST Coordinator, new punter, new punt returner and Maurice Jones-Drew returning kicks. And a very strong-legged kicker.

Other than that, I am keeping my mouth shut. This Jacksonville team is loaded even with Dave Garrard at QB. If they aren't a top-5 team at season's end, send Jack Del Rio packing (and I am a big fan of Del Rio's).

42
by cbm (not verified) :: Wed, 09/05/2007 - 10:32pm

"A defense’s performance on first, second, and third down isn’t going to be that different. They’re still the same players"

Often they are not the same players.

"But third down has much more leverage on the effect of the game than first or second down,"

Absolutely untrue.
First of all, third down plays only account for about 20% of all plays.
Second of all, The success rate on first and second downs corralate higher with points that third down conversions.

43
by Jerry (not verified) :: Wed, 09/05/2007 - 10:55pm

Re #20:

I love the site, but I am beginning to wonder how data-mined these projections are. It’s one thing to build a predictive model based on your core football knowledge, it’s completely another to datamine past trends and assume they will hold true in the future.

Long time reader response:

In 2004, the projections showed a major improvement in San Diego's offense, and nobody, including Aaron, quite believed it. Lo and behold, the Chargers went from 0.8% offensive DVOA to 19%. That was what convinced many of us to take the projections at face value, rather than try to fit them into what we think we already know. They won't all be right (that's why they play the games), but they're real information that you won't get elsewhere in many cases.

And you should take some comfort in the idea that Aaron's looking for ways to correct the system's overrating of Tampa Bay and underrating of St. Louis, even if this particular statement wasn't as strong as you might like.

44
by BD (not verified) :: Wed, 09/05/2007 - 11:01pm

Re: New Orleans
I don't understand why everyone is shocked at the Saints projection of 7.5 wins. Last year they were 10 - 6. 1 - 3 against the much better AFC. They won one playoff game against the marginal Seahawks. Essentially they are still the same team this year, and will be less likely to have as strong "unmeasurables" (i.e., emotion, etc.). Media types will say they are taking a step back but I think New Orleans in 2008 will be quite scary.

Re: Jacksonville
There is no correlation year to year with inconsistency. As much as I think Del Rio is a buffoon, I don't think this prediction is unreasonable.

As others have said, these predictions far beat most media outlets who seem to be picking the same 6 division winners.

45
by throughthelookingglass (not verified) :: Wed, 09/05/2007 - 11:09pm

BTW, did the general luddity (not a word-I know) of the FOX Sports clientèle prompt them to drop the DVOA ratings, or was this an FO decision?

46
by Gerry (not verified) :: Wed, 09/05/2007 - 11:10pm

I get to be GiantsGer.

I believe the numbers have the Giants number all wrong. I think the offensive and special teams DVOA numbers look about what I expect, but I think that our defensive DVOA will end up being closer to -10% this year than 7.7%. If so, that should have us in the top 10 for DVOA again, just like last year.

If one can't be a dreamer the day before the season starts, when can one be? But I simply do not see a below-average defense barring another season of multiple injuries all at the same time to our linebackers, or our DL, or our secondary.

47
by Bronco Jeff (not verified) :: Wed, 09/05/2007 - 11:11pm

I kind of expected the Broncos to be a middle-of-the-pack team, but I sure wasn't expecting a 24th ranked defense. I'm not sure how that happens with the personnel they've assembled--though they weren't impressive in the preseason.

48
by Erik (not verified) :: Wed, 09/05/2007 - 11:18pm

#44 The Saints beat the Eagles in the playoffs.

As a Saints fan, I recognize that a regression in results this year is not unlikely (and I *really* hate that so many "experts" are picking them to have a big year) - they're not nearly the best team in the NFC (if I had to pick today, it'd be the Eagles). On the other hand, I have more hope for them than I have for a while. Also, I just feel like they're in for a sustained run of respectability.

49
by jonnyblazin (not verified) :: Wed, 09/05/2007 - 11:18pm

My 2 cents as a Ravens fan:

I'd be perfectly content with an 11th ranked offense, I think that is a reasonable projection. I'm a bit worry about their O-line right now, but inserting Ogden in the lineup should make a huge difference.

I'd be shocked if their defensive DVOA is a mere -6.5%, but FO's defensive projection of the Ravens D last year was about a mile off, so I'm not that worried.

The special teams DVOA projection is going to be flat out wrong. Their 2nd half Special Teams DVOA tanked when Sams got injured last year, this year they have Sams plus Yamon Figurs (fastest player drafted) who is a terror returning kicks. Not to mention the import of Lloyd Rhys (sp?) from England, I didn't see a single kickoff of his that didn't land in the endzone, and his 55 yard FG in Atlanta landed halfway up the netting. He's got a monster leg. Factor in old reliable Stover, who is the exception to the FG% doesn't carry over from year to year rule. Also relevant is their roster is tilted heavily towards LBs and secondary players who excel on special teams.

50
by Scott (not verified) :: Wed, 09/05/2007 - 11:21pm

I don't even understand how NE's offense gets in the top 3 (a place they've never been in the Belichick era). They're just not that kind of offense that's going to light it up every week.

They're also relying on a RB to carry the load that looks pretty injury prone and hasn't been able to show how he does in goal line situations. Sammy Morris is hardly a good player.

Donte Stallworth is always hurt. Randy Moss hasn't shown anything yet in NE, nor the last 2 seasons really. Wes Welker catches mostly just short passes. Ben Watson has some shaky hands. These guys like Washington and Gaffney can't be relied on for much. Kyle Brady is just a blocker at this point.

How's that going to be better than the more cohesive unit in Indy that is probably the most consistent offense the league's ever seen? I also would take Sean Payton's offensive mind over the Patriots to field a better offense.

The Chargers are loaded with weapons, Rivers is only going into his 2nd year as a starter, Norv does know offense, and Vincent Jackson should emerge for them to go along with the best RB and TE in the league.

The Eagles could be very good too if McNabb stays healthy.

That's the top 4 there.

Oh and Jacksonville winning 11-12 games with Garrard is laughable. They'll do their usual: split with Indy, beat some other good teams, get swept by Houston, lose some games late in the season they need to win and finish with a disappointing 8 or 9 wins. A lot of talent there but a fraud at QB mixed with bad WR's won't win that division this year.

51
by kevinNYC (not verified) :: Wed, 09/05/2007 - 11:23pm

#39... It's not surprising considering I saw Aaron Schatz go on ESPNNews and act like Luke Petitgout to David Diehl was as similar a transition as Anthony Munoz being replaced by Guy Whimper. Plus, I'm sure the projections believe Bob Whitfield is going to start at LT. I agree 100% regarding your point about the injuries... it really doesn't get as much publicity as the Tiki-Coughlin relationship but was 80 billion times more important.

My question for Redskins fans (I don't know much about their roster)... isn't the defensive line essentially the same players as last year's group? If that's the case, how is their run defense and pass rush projected to improve?

52
by Raiderjoe (not verified) :: Wed, 09/05/2007 - 11:44pm

Jaguars 3, Raiders 27

Something is rong with that computer DVOa thing. Jaguars can't be 3rd best team,. QB sucks, WRs suck, rest of team good. Some good players on defense. Raiders defnitely better than 27. More like 10-15 area. That would majke them be about 10-6.

53
by TracingError (not verified) :: Wed, 09/05/2007 - 11:52pm

The issue for the Redskins last year wasn't depth, it was that their worst starters didn't belong on the field.

Starting safety Archuleta was a bust for the role the assigned him, and they may have improved with a street free agent.

Starting DT Salave'a (much as I loved him) was not as good as his injury replacement Golston.

Nickel back/fill in starter Wright didn't belong on the field.

Starting MLB Lemar Marshall was marginal to begin with and a bit nicked up.

Starting OLB Warrick Holdman was awful, and his backup, Rocky McIntosh would likely have been better if he had seen the field.

Starting DE Philip Daniels is not starting quality--well maybe on a really good line (too bad he is back in the same role). On the other side, Andre Carter did have a good sack rate in the second half of the season, after being totally unproductive in the first half (and not great against the run).

They did have some trouble when Griffin went down (as they did the year before), and Springs hurt, as did Prioleau, but the significant problems were the starters, not the depth.

54
by milo (not verified) :: Thu, 09/06/2007 - 12:01am

Let's look at these projections:
Superbowl champion (Ind)defense DVOA rank: 32
NFC championship game loser (NO) defense DVOA rank: 31
NFL best regular season record 14-2 (SD) defense DVOA rank: 29

Let's just say you screwed up.

55
by TracingError (not verified) :: Thu, 09/06/2007 - 12:07am

Several other reasons to look for Redskins improvement:

a) The Redskins created a record breaking number of turnovers last year--12. That just won't happen again. Even if they were just really bad, they'd get about 20. They will catch more interceptions and recover more fumbles. [They will also likely give up a few more with Campbell under center, but the net turnovers should improve.]

b) They were afraid to blitz much because the secondary was so bad and they kept getting burned on their run blitzes too but improved safety play should allow for more pressure, even though the D-Line is still a major weakness.

Even if it doesn't, they could hardly give up more big plays than last year, so blitzing more as in Williams 46 roots will at least create more negative plays for the offense.

c) Campbell is virtually guaranteed to improve from his first seven games played, and the offense, based on Saunders history with the Rams and Chiefs, is likely to improve in its second year.

56
by Will Allen (not verified) :: Thu, 09/06/2007 - 12:13am

The Patriots defense isn't going to crack the top 10 if Seymour doesn't get on the field, and even missing six games is going to make it harder.

I hope Aaron is once again more accurate than I regarding the Vikings. I only have them projected at 6 wins, although I think their upside is bigger than their downside. I don't think they can fall much short of four or five wins, while I think 9 isn't totally out of the question. It all depends on whether any receiver they have can end up punishing a defense which puts eight or nine in the box, and blitzes incessantly. I have doubts, but I'd be happy to be proven wrong.

I agree that the Packers win the NFC North. I hope the Cowboys and Chargers lose 13 games apiece; it really feels good to re-adopt my traditonal hatred of all things Jerry Jones.

57
by Chris (not verified) :: Thu, 09/06/2007 - 12:19am

I had 5/6 playoff teams last year, and I'd bow down to the guy who had all 6 ( that would mean Naw Awlans). It took juevos to have the NFC East send 3, but it happened.

ESPN and the likes had the Redskins winning the east last year, because Brandon Lloyd and Randel El were supposed to solve their passing game, but I correctly had the Redskins in last, when everybody else had them in first.

I think you guys are way too high on Washington again, and I think this is your way of covering up last years mistake. If any team had Peyton and Harrison they would be good. You can't call it dumb luck that an old corner with an injury history, and Clinton Portis ( who got run into the ground 2 years ago) got injured.

58
by Chris (not verified) :: Thu, 09/06/2007 - 12:24am

Write down these projections all you want, but Vegas is still king. The guy picking season O/U win totals was 17-15 last year. You could flip a coin and be 17-15.

59
by hooper (not verified) :: Thu, 09/06/2007 - 12:25am

Re: 47

I'll bow to your expertise for the Broncos, since I haven't been able to see them yet. However, I'm pretty nervous about the defense; that's about as much reshuffling of the front seven as I've even seen at once. My other worry is age; some of them are clearly one or two year fill-ins, and will need to be replaced. I hope it works.

Other than that, I'm pleased with Denver's offensive rank. Certainly, I wouldn't put it higher without some proof to back it up. But I'd be most worried if the offense ranked 24th. Lastly, a quick look at defense predictions in the AFC West puts Denver's D behind only the Raiders. If SD's coaching works its magic, this may be good enough.

60
by Jake Schumaker (not verified) :: Thu, 09/06/2007 - 12:32am

I'm a Chiefs fan, and the great thing about this pre-season it's pushed my expectations into the Marianas Trench. First, I see that FO projects the Chiefs to win 5 games. Then, I watch Hard Knocks, which makes the team look about as professional as an elementary school recess.

Now, I'm going to be delighted as long as we finish ahead of the Raiders.

61
by Becephalus (not verified) :: Thu, 09/06/2007 - 12:35am

# 19 Don't predictions of this sort auto-mediate. If you were predicting the outcomes of 32 coins flip series you would predict them to have the correct distribution of outcomes.

BUT if you were predicting the outcome for any individual one you would need to say it was 50% despite your knowledge that more likely than not some would drastically over or under perform.

Same thing with these projections I am thinking.

62
by Pio (not verified) :: Thu, 09/06/2007 - 12:39am

@47: As much as I hate to say it, I think FO may be right about the Broncos D. Our secondary is incredible (tho I don't think Bly will quite live up to expectations), but our LBs are suspect in coverage and our defensive line has been a disaster in the pre-season - even against the run, many of our stops have been the LBs or even Lynch and Bailey playing the line. Dumervil and Rice make for a good pass-rushing pair of ends, but our tackles have been horrible so far. I'm willing to be proven wrong tho...

63
by throughthelookingglass (not verified) :: Thu, 09/06/2007 - 12:40am

57,58
So, what doesn't DVOA account for that you do?

64
by Pat (not verified) :: Thu, 09/06/2007 - 12:42am

“A defense’s performance on first, second, and third down isn’t going to be that different. They’re still the same players�

Often they are not the same players.

Hence the reason I said "that different" other than "not different at all." Often they aren't the same players if it's nickel, etc., but nickel shows up in 2nd down plays as well.

Second of all, The success rate on first and second downs corralate higher with points that third down conversions.

Yoda says "Means not what you think it does, this." Or something like that.

I said third down conversions have higher leverage, not higher importance. The fact that they do not have higher importance, while having higher leverage, is the entire point.

A team with poor success on third down relative to their success on first and second will necessarily significantly underperform what you would expect from their overall average performance. Switch "third" and "second" there, and the answer is much, much less because second down has less leverage than third down.

65
by Harris (not verified) :: Thu, 09/06/2007 - 12:47am

If 2007 is going to mirror 2004 (Eagles and Patriots the best two teams in the league, Eagles lose Super Bowl to Patriots) just let me know now. I'll have to move my family because, and I'm serious, Philadelphia might be put to the torch.

66
by Pat (not verified) :: Thu, 09/06/2007 - 12:47am

Write down these projections all you want, but Vegas is still king. The guy picking season O/U win totals was 17-15 last year. You could flip a coin and be 17-15.

You do not judge projection systems by success like this. You judge them by least mean square error. Two systems can be exactly identical in terms of how they do versus an O/U win total, and one can be vastly better than the other.

There are other uses for projection than betting.

67
by Independent George (not verified) :: Thu, 09/06/2007 - 12:52am

#39 - I think it's because all the problems that arose from those second-half losses are still with the Giants. Their LT spot is being filled by their LG, Toomer & Strahan's injuries were less about luck than about age, and this year's starters at LB and CB aren't any better than last year's starters who were lost to injury.

68
by chip (not verified) :: Thu, 09/06/2007 - 12:53am

#43 For every projection FO got right (SD 2005), they got one (or two) wrong (PHI 2005 projection = 12.1; actual wins = 6; HOU 2005 projection = 8.8; actual wins = 2). Whoops.

I think it's been shown that FO's projections are no better or no worse than Vegas lines:
http://www.bbnflstats.com/2007/09/pre-season-predictions-are-worthless.h...

http://footballpredictionnetwork.blogspot.com/2007/08/accuracy-of-vegas-...

http://footballoutsiders.com/discussion/viewtopic.php?t=146&sid=ad09c9bc...

69
by Bill Barnwell :: Thu, 09/06/2007 - 12:56am

The guy picking season O/U win totals was 17-15 last year. You could flip a coin and be 17-15.

And I picked against the projection system a number of times. This is really a fundamental lack of comprehension of both how we project wins and how Scramble works, and I've mentioned that more than once now, and you've ignored it.

70
by Independent George (not verified) :: Thu, 09/06/2007 - 1:05am

*sigh*

Don't you kids know anything? This is how it's done:

Washington is clearly ranked too high because they still haven't hired coach Janky Spanky. RaiderJoe is way better than this.

Falcons are gonig all the way this yeear!!!.

71
by Yaguar (not verified) :: Thu, 09/06/2007 - 1:08am

I'm curious whether a mix of Vegas lines and FO projections would do better than either one by itself. The FO projection system is obviously really good for a non-subjective system, but the Vegas lines probably include subjective elements.

Most people's subjective judgement would probably make the FO projection system worse, but I wonder if Vegas's could make it better.

72
by admin :: Thu, 09/06/2007 - 1:15am

It's amazing how many comments above boil down to the following statement:

"Every year, your readers wait with eager anticipation for the DVOA projections. The DVOA projections are one of your most popular articles. DVOA projections are imperfect. How dare you continue to publish imperfect projections that your readers keep asking you to publish???"

The main use for the DVOA projections is that PEOPLE LIKE THEM. We are in the business of publishing articles that people enjoy reading. We know people enjoy reading them more when they are more accurate than they were a year ago, but we are in the entertainment business, not the perfection business.

73
by Becephalus (not verified) :: Thu, 09/06/2007 - 1:22am

I am just going to say squat this year as last year Will Pacifist and I cursed the Vikings into oblivion :)

You were 100% right about them at least.

74
by Alex DL (not verified) :: Thu, 09/06/2007 - 1:27am

Hey Schatz,
Just because everyone is questioning the dvoa doesn't mean that it's not loved. We question and we critique because we love.

btw...#52 did have me laughing my butt off.

75
by JoshuaPerry (not verified) :: Thu, 09/06/2007 - 1:33am

Okay, so the Chiefs will have paid LJ for nothing when they get that top pick in the draft? This year is remarkably deep at RB(probably 5 1st rders)including the next big thing in Run DMC Darren McFadden of Arkansas.
Im going on record. The Redskins will fall flat on their face.
These projections just feel off. QB play isn't factored strong enough. Most of the top 12 have serious questions at QB, will Jason Campbell, Jake Delhomme, Jeff Garcia, and Chad Pennington really not get replaced for poor play?

76
by kevinNYC (not verified) :: Thu, 09/06/2007 - 1:43am

#67... The Giants LBs and DBs are almost meaningless to how the Giants defense has performed the past few years. The defensive line (when HEALTHY) was 2nd best in adjusted line yards and the defense 6th overall a week before Michael Strahan's injury (Umenyiora and Tuck were already out 2 weeks each). By the end of the season, they had dropped to 5th in adjusted line yards and 13th overall. Strahan's injury may have been caused by age, but that surely can't be the case for Umenyiora and Tuck?

Diehl actually moved to LT the last 2 games of 2006 and the Giants run offense was the best it had been since Petitgout's injury. I'd be a complete homer if I didn't mention the 2 teams they faced were among the league's worst run defenses. His pass blocking was poor in the first game and excellent against the Eagles in the playoffs. Let's not judge him solely on the Baltimore preseason game.

I don't expect Jacobs to replace Tiki, but he's no journeyman. He was third in DPAR last year.

77
by sam (not verified) :: Thu, 09/06/2007 - 1:51am

Fraud of a quarterback? So Garrard has no chance to improve after his 22 career starts? He's just done, that's all. Ouch. OK then. Well, I guess you might want to apply for a job as a jaguars talent evaluator because you clearly are qualified and they might as well all quit right now.

Or maybe, DVOA actually can predict offensive success (despite all the crap their pass offense hastaken this offseason, they weren't last. They were in the low 20's. Not good, but not abysmal/black-hole-of-suck either).

78
by Alex (not verified) :: Thu, 09/06/2007 - 2:02am

Write down these projections all you want, but Vegas is still king. The guy picking season O/U win totals was 17-15 last year. You could flip a coin and be 17-15.

For now, let's ignore the fact that this isn't how a projection system is correctly judged. The reason Vegas isn't still king is that Vegas doesn't publish any predictions for each teams offensive, defensive, and special teams performance. If they did, FO would absolutely blow them out of the water. Remember, what makes DVOA great isn't that it tells you who will win (although it does a good job of that), it's that it tells you why they'll win.

79
by TomC (not verified) :: Thu, 09/06/2007 - 2:10am

Not that anyone cares, but here's what I came up with going down the full schedule and picking a winner for every game:

AFCE:

NE 11-5
NYJ 6-10
BUF 5-11
MIA 4-12

AFCN:

BAL 13-3
CIN 11-5
PIT 10-6
CLE 7-9

AFCS:

IND 13-3
JAC 10-6
TEN 7-9
HOU 3-13

AFCW:

SD 13-3
DEN 12-4
KC 5-11
OAK 4-12

NFCE

PHI 12-4
DAL 11-5
NYG 8-8
WAS 7-9

NFCN:

CHI 10-6
MIN 7-9
DET 5-11
GB 5-11

NFCS:

NO 12-4
CAR 10-6
TB 6-10
ATL 3-13

NFCW:

SEA 9-7
STL 8-8
SF 5-11
ARI 4-12

80
by Becephalus (not verified) :: Thu, 09/06/2007 - 2:13am

Yeah my friend and I do a contest each year. This is my wild guess.

NE 12
NYJ 7
BUF 5
MIA 5

CIN 11
BAL 10
PIT 9
CLE 6

IND 11
JAC 10
TEN 7
HOU 5

DEN 11
SD 10
KC 6
OAK 6

PHI 12
WAS 9
DAL 8
NYG 5

NO 9
CAR 8
TB 8
ATL 5

CHI 10
GB 8
MIN 7
DET 6

SEA 9
SF 8
STL 7
ARI 6

81
by Martin Collinson (not verified) :: Thu, 09/06/2007 - 2:25am

Re the suggested improvement in the Redskins D and questions about what has changed to give any reason to think this will happen.

First off the secondary has MUCH better depth at corner with Smoot and Macklin the 3rd and 4th corners. Even if Springs goes down we have people back there who can cover man on man. Add in Landry who has looked very good in preseason at SS and Taylor being asked just to play FS not try to make up for lack of coverage elsewhere.

At LB London Fletcher taking over at MLB has/will make a big difference and Rocky MacIntosh has looked very very good in camp and preseason - he could be breakout player this year.

The question mark is still the D line. It is the same guys give or take. The difference is Griffin is healthy which is huge and we are hoping we see the Carter who layed the last 7 games last year not the one who started the season.

Also two DTs who were rookies last year have really developed - Montgomery and Golston. Golstin started a lot last year and made some nice plays but also got pushed around. Montgomer saw time but had conditioning issues. Montgomery will start this year and has realy looked good in preseason - not many fans around the league knwo who he is yet but they will by the end of the season.

Big hole is still LDE where Wynn was released and the starter Daniels is decent versus the run but is not a threat th rush the passer anymore.

All said though I think the D will be top 15 and maybe top 10 again. If it is that translates to 2 to 3 more wins than last year without any improvement from the O in its second year in the system and with more production - we hope - from the QB position.

9 to 10 wins is realistic.

82
by Brian (not verified) :: Thu, 09/06/2007 - 2:27am

In defense of FO, I think people forget just how unpredictable the NFL is, especially before the first snap of the season.

It's not that FO predictions stink, it's that ALL pre-week 1 predictions stink. FO's aren't any worse or better.

At least they stick to their system and didn't flinch when TB and WAS were in the top 10 and DAL, STL, and KC were in the bottom 10.

83
by Pippin (not verified) :: Thu, 09/06/2007 - 2:27am

See, this is why I hate this site.

I'm a Dallas fan and geeze that's a depressing prediction to look at. Makes me wince. And yet, the analysis that goes on here is always so impressive and so...

that's why I love this site.

84
by MC2 (not verified) :: Thu, 09/06/2007 - 3:30am

#75 - QB play is certainly important, but it's not always easy to predict. At the start of last season, the Chargers had a major "question mark" at QB, yet things worked out fine. And, of course, we can't forget the Bears, who made the SB despite having a QB who was considered a mjor weakness at the start and the end of the year (but briefly considered a strength near the middle of the season). Finally, we have the Eagles. Everybody and his brother wrote them off when McNabb went down.

85
by pharmboyrick (not verified) :: Thu, 09/06/2007 - 3:32am

I think JAX is the team that is the most over-rated and off in the book and web-site

The Vegas o/u factor in subjective data like public perception. For example there are teams that always generate more action than others. Popular teams like Dallas, Denver, Pittsburgh, NYG all have generally inflated numbers for both point spreads and o/u because there are a lot of people who want their favorite team to win.

86
by MC2 (not verified) :: Thu, 09/06/2007 - 3:36am

82 - I agree that people underrate the unpredictable nature of the game. Most people's preseason "predictions" are really just a reprint of last year's standings, with maybe one or two teams where the predictor "goes out on a limb".

I bet at the end of the year, when we look back on these, many of the strangest predictions will come true, while some that seemed obvious will turn out wrong. It happens every year.

87
by Richard (not verified) :: Thu, 09/06/2007 - 4:32am

Crap. 29th ranked defense? Really? Crap...

88
by David (not verified) :: Thu, 09/06/2007 - 4:53am

So how does week-by-week variance shape up from year to year? I'm wondering if we can expect an actual standout Jacksonville this year or another schizophrenic team that's just even better in the games where it doesn't suck.

89
by Richard (not verified) :: Thu, 09/06/2007 - 5:04am

I'm amazed that the Patriots come out as the favorite in every game on their schedule even when applying the 17.0% home field advantage. 16-0 maybe?

90
by Bobman (not verified) :: Thu, 09/06/2007 - 5:08am

My main comments concern standard deviations and consistency. Anybody else find it odd that the SB winner (and all-time standard deviation champ) and the Lions, one of the consistently worst teams over the past 5 years, both lead in StdDev? The system thinks they're pretty predictable.

The flipside is inconsistency--Jax's bugaboo. I am of the opinion that it is an inherent trait, like penalties, attitude, aggressiveness, pacifism, etc. If a team is highly inconsistent, unless wholesale changes are made, they'll generally remain so. Some traits are like a virus that infects a team. Some teams get flagged, others do not. Some play up or down to their opponents, others are consistent. So actually, Jax's low StdDev surprises me more than their wins projection.

NO in last place shocks me, but that's a tight cluster in the NFCS--it's really anybody's division.

What quirk of the system results in ZERO teams projecting out to 2, 3, or 4 wins and only one team at 5? Reality will probably have a couple teams at 4 and a couple at 5, maybe one below that. (and a couple at 12-13 wins) Which means some of the other teams will likely have higher win totals--my money is on some of the subjective favorites like Indy, SD, NO, and the winners in the AFCN and NFCW to add a couple games.

Aaron, take heart, we're a demanding bunch, which is why we're here in the first place. Because this is the best NFL site. We trust you to do this right and tweak as needed.

Jeez, 16 hours til the season revs up.

91
by Jake Schumaker (not verified) :: Thu, 09/06/2007 - 5:26am

#90

I'm sure that is some of the season Dr. Alamar simulated, some teams won 3 or fewer games. What you see are the average number of wins each team won throughout all the seasons that were simulated. In other words, the simulation says that it's very likely that some team will win very few games, it's just not sure which. And how can it be, if you give that team some possibility of playing over its head or getting a bunch of lucky bounces?

92
by Bobman (not verified) :: Thu, 09/06/2007 - 5:32am

Now to a topic closer to my heart: Over the past three seasons, Indy's worst Offensive DVOA was about 28% (Best was almost 40%), yet the projection is for 22.8 this year. Is Tarik Glenn the reason? With a healthy Dallas Clark and slot receiver, plus the usual cast, I just don't see it--it's not just predicting a regression from last year, but a big detour from the past three seasons from the most consistent team in the league. Curious. Dubious. Doubtful.

And on Defense, their DVOAs ranged from roughly -9% to about 11% IIRC (last year they lost 3 of their 4 best 2005 run-stuffers to injury and free agency, then added Booger, who is now on IR, and had Sanders out for 12 games and Gilbert Gardiner inexplicably starting for another 10 or so. All of that is unlikely to reoccur in 2007.) I don't see how the D is going to regress farther--again, outside the range of the past 3 years, especially when last year was a defense of historically bad proportions. Odd. Not ikely, but maybe....

And special teams..... yeah they bite, but I still say theyir value is over-weighted here and I have for years. I'm a broken record that way. Pig-headed of me, I know.

So if Indy had their cherry-picked WORST offensive and defensive DVOAs of the past three seasons combined, their total DVOA would end up around 17%, placing them 4th ranked among this year's projections. Maybe a little high, but probably closer to reality than 11th. Now I understand that I am looking backwards at past performances (no guarantee, etc), but I am also taking the worst ones combined, which we don't think is likely to happen, do we? If I took the best combined, we'd be looking at a total DVOA of about 40%, which is well ahead of the Pats and nearly laps Jax ranked at #3. We surely don't expect that, either.

Aside from losing Tarik Glenn, I don't see what has changed that much for them. The guys they lost were competent, but hardly high VOA guys.

93
by MC2 (not verified) :: Thu, 09/06/2007 - 7:27am

#90,91: Whenever you see teams finish with really good or really bad records, it's rarely an accurate reflection of their true quality, especially in the current age of parity. In order to win 13 or 14 games, you have to be good, but you also have to get lucky. Similarly, to lose 13 or 14 games, you have to be bad, but you also have to be unlucky.

I'm sure that if the NFL season was as long as the MLB season (or even the NBA season), there would be fewer teams with extreme W-L records. Of course, the simulation is effectively a 16,000-game season, so you would expect the effect of luck to virtually disappear, leaving a lot of teams that win somewhere between 6 and 10 games.

94
by MC2 (not verified) :: Thu, 09/06/2007 - 7:31am

Uh, make that a 160,000-game season.

95
by Gerry (not verified) :: Thu, 09/06/2007 - 7:34am

"The Giants LBs and DBs are almost meaningless to how the Giants defense has performed the past few years. The defensive line (when HEALTHY) was 2nd best in adjusted line yards and the defense 6th overall a week before Michael Strahan’s injury (Umenyiora and Tuck were already out 2 weeks each). By the end of the season, they had dropped to 5th in adjusted line yards and 13th overall. Strahan’s injury may have been caused by age, but that surely can’t be the case for Umenyiora and Tuck?"

kevinNYC, we are coming from nearly the same place, but I don't agree with the first part of your statement. Everything after that was true for last year, but two years ago, it was the decimation that occurred to our linebacking corps that really caused the collapse, especially in the playoffs. That means that LB play has mattered quite a bit to our defensive performance the last two years.

It is no secret that our DBs suck. We make up for it by stopping the run with our DL and LBs, and by putting pressure on the QB. This equation gets killed when we lose 3/4 of our starting DL all at the same time, or 100% of our starting linebackers all at the same time.

Obviously, we are injury prone. But even if we get the same number of injuries, we should be ok as long as they don't all happen at the same time-- the tail end of the season-- as has been the case the last two years.

96
by morganja (not verified) :: Thu, 09/06/2007 - 9:16am

I'm sure it is just an oversight, but I didn't see Appalachian State Mountaineers in the top 32 NFL teams. I have them winning the NFC South, the AFC East and tied with the Raiders for the AFC West.

App State 34 Michigan 32

97
by sam (not verified) :: Thu, 09/06/2007 - 9:29am

86:

The problem is that there are some things which cannot be predicted -- injuries to specific players, death in a player's family, a hurricane wiping out your city. As somebody already pointed out, this isn't so much that "FO thinks xxxx team will win yy games" but provides valuable trend insight.

Where many experts say the Colts defense will resume where it left off in the Super Bowl the FO believe it is going to be dead last. It's likely that will translate into enough losses that they won't win the division. Jacksonville's offense projects to improve. Many national pundits believe that the only way an offense can improve is if you sign an old veteran who hasn't done anything in a couple years (Randy Moss) or draft a pretty good offensive player who they've already decided can turn your team around (Calvin Johnson). Apparently, players don't get better with coaching and experience.

98
by Independent George (not verified) :: Thu, 09/06/2007 - 9:38am

#76, #95 - You don't have to sell me on Brandon Jacobs - I'm actually not too worried about the running game. I think LT is a problem, but we'll see how that goes - you're right, Diehl wasn't awful last year, but I'm still not convinced he's a good solution.

The defense, though, scares the heck out of me. The secondary looks as bad as ever. I think the real center of the defense isn't the DL, though, but Pierce - I don't have the numbers to back it up, but, subjectively, the Giants D has looked the worst whenever he's been out. Moving Kiwanuka to LB is probably a good idea in the long term, but I think it's likely to weaken both positions in the short-term.

Worst of all, though, is projecting the Giants as having the #4 schedule in the league. That actually reminds me of a question - should VOA be more predictive of records than DVOA? It doesn't matter if you get blown out by a great team - if you have a cupcake schedule, just beating the weak teams can sometimes be enough to get you into the postseason.

99
by Cmos (not verified) :: Thu, 09/06/2007 - 9:59am

Aaron, you are placing way too many varibles on Dallas. Dallas might finish with only an average win total (9-10 wins) but there is no way they are going to be as bad as the projections (we know they burned you mid-season). Also STL and ARZ worse than Cleveland? No way...

100
by Ole (not verified) :: Thu, 09/06/2007 - 10:08am

#87 Wow, didn't notice that. If a team with that quality D-line and Merriman and Phillips coming off the edge can manage to produce the 29th overall defense, then their secondary will have exhibit crapness of epic proportions. Either that or DVOA is predicting that the plague will descend on San Diego and they'll be playing scrubs from week 6 on.

101
by Jim M (not verified) :: Thu, 09/06/2007 - 10:10am

Hey Schatz,

Don't be so sensitive. A lot of the criticism is coming from paying customers.

I just bought your 2007 book. Great stuff. Reminded me of the reading the old Bill James Handbooks.

102
by dave thomas (not verified) :: Thu, 09/06/2007 - 10:48am

FO talks about ways to adjust for better accuracy. I can see one major flaw. coaching changes are weighted FAR too heavily. Two prime examples are SD and Dallas. Unlike say, the Raiders, these teams do not have coaches taking over with a talent-poor roster. (though even the raiders come with half the equation!) If you analyze the effect of new staffs, keeping in mind what the expectations would have been with the old staff and the same roster, you can see that the effect is minimal for the first year, and not always negative. (2006 Saints, anyone??)
Anyways, that is easily biggest flaw I see in the system. I would think shrinking or eliminating the effect of staff changes (excepting perhaps drastic changes, like AZ going 3-4 with not enough of the right players) would level out the projections to something approaching reality for teams like SD and Dallas.

103
by steelberger1 (not verified) :: Thu, 09/06/2007 - 10:49am

Re 89:

Yeah, bet on that.

Worry about 1-0 first.

104
by James G (not verified) :: Thu, 09/06/2007 - 10:51am

Actually, I think FO projections are better than other projections. Not no better, no worse. Didn't Easterbrook keep track of these things and demonstrate that?

In addition, you have to realize that even if the projections are statistically perfect that 9-10 teams should be expected to have win totals outside of their standard deviations. That is, only about 70% of projections should be within the standard deviation.

105
by Arkaein (not verified) :: Thu, 09/06/2007 - 10:51am

This question will probably be lost in the noise, but what's with the Packers' ridiculously low special teams projection. I know GB's STs were bad last year, but the one thing they still have is a strong legged kicker. Though a different kicker and a rookie to boot. Still, with regression to the mean shouldn't the Packers at least be in the ball park of other teams. I find it hard to believe that the Packers special teams will hurt the team as much as an 8th ranked defense could help them.

106
by vanya (not verified) :: Thu, 09/06/2007 - 10:55am

I find it interesting that FO and Bill Simmons coming from completely opposing methodologies (scrupulous statistical analysis vs. application of hunches and overgeneralized "rules") are both flying in the face of overwhelming conventional wisdom and picking Atlanta to improve. Time to go put some money on the over.

107
by Chris (not verified) :: Thu, 09/06/2007 - 11:01am

Bill B- but if dvoa is so great, then why go against it? It seems like a better tool to show what happened, as opposed to predicting the future.

If vegas had to deal with the "emotional aspect" then wouldn't a strictly numbers DVOA be even more effective? 17-15 is nothing to brag about.

Just don't think that since DVOA was created, that Vegas is in trouble or anything.

Last year Vegas had a Saints win total of 7.5, and people pounded the under, but they still didn't adjust their line ( because they believed in it), and you guys would have lost a lot of money if you bet UNDER on the saints 7.5, and over on the Redskins. Two of the biggest variances last year.

I like the stuff, but if your going to make clains that " we get it right more than anyone online", then be ready for criticism.

108
by Gerry (not verified) :: Thu, 09/06/2007 - 11:01am

#98-- That is why, despite thinking that DVOA is under-estimating the G-men, I am skeptical that we can hope for anything more than 9-7 or 8-8. We can be a top-10 DVOA team and still easily pull off such a mediocre record.

If we are truly just the 19th best team, then I think that the 7 win projection listed above is optimistic.

109
by Gerry (not verified) :: Thu, 09/06/2007 - 11:02am

"Dallas might finish with only an average win total (9-10 wins)"

10 wins for any team is above average. Or, if you want to say that 10 wins is average, then it stands to reason that 6 wins would also be average.

110
by mactbone (not verified) :: Thu, 09/06/2007 - 11:27am

First of all DVOA was never intended to be used as a betting device. Second, you're taking Bill's (or someone) results and using them as a proxy for DVOA - they're not DVOA's results, they're Bill's.

King Kaufman at Salon tracked expert predictions and compared the results, the 2004 season edition is linked in my name.

The staff of Football Outsiders is the winner of this column's 2004 NFL Preseason Predictions Contest, which needs a better name. Before the season I collected the preseason predictions of 27 experts, defined as more or less national typists and chatterers whose picks I could find without too much effort. My predictions, an aggregate of the 27 experts' picks, and the aggregated picks of you, the readers of this nonsense, brought the pool to 30. The point of all this is to prove that there is more than one way to be bad at making predictions about the NFL.

Here are the standings, with point totals followed by the number of division winners correctly predicted, in parentheses:

Football Outsiders consensus -- 18 (5.5)
Sal Paolantonio, ESPN -- 15 (4)
Randy Mueller, ESPN -- 14 (5)
Aaron Schatz, Football Outsiders -- 14 (4)
This column's readers -- 14 (4)

Dan Pompei, Sporting News -- 14 (4)
Mark Schlereth, ESPN -- 13 (5)
Experts consensus -- 13 (4)
Joe Theismann, ESPN -- 13 (4)
Mike Golic, ESPN -- 13 (4)

Sports Illustrated -- 13 (3)
Dr. Z, S.I. -- 13 (3)
Jim Buzinski, Outsports -- 12 (5)
King Kaufman -- 12 (4)
John Clayton, ESPN -- 12 (4)

Tom Jackson, ESPN -- 12 (4)
Cyd Ziegler, Outsports -- 11 (4)
Paul Attner, Sporting News -- 11 (4)
Vinnie Iyer, Sporting News -- 11 (4)
Peter King, S.I. -- 11 (3)

Sean Salisbury, ESPN -- 10 (4)
Eric Allen, ESPN -- 10 (4)
Brian Baldinger, Fox -- 10 (4)
Pete Prisco, CBS SportsLine -- 10* (4)
Don Banks, S.I. -- 10 (3)

Tuesday Morning Quarterback, NFL.com -- 10 (3)
Cris Carter, Yahoo -- 9* (4)
Clark Judge, CBS SportsLine -- 9* (4)
Len Pasquarelli, ESPN -- 8 (3)
Merril Hoge, ESPN -- 5 (1)

2005 Season
Schatz won with 14 of a possible 24 points. The contest awards two points for each correct pick of the eight division winners and four wild-card teams. A single point is awarded if a panelist picks a team to win a division and it takes a wild card, or vice versa. Fourteen is an all-time low.

In 2003, Don Banks of Sports Illustrated won with 17 points, and last year the consensus of all of the Football Outsiders writers won with 18. Banks finished in a six-way tie for 20th this season with six points. The Outsiders consensus was tied for third with 10.

Schatz was also the leader in picking five division winners, also an all-time low. He was the only entrant who correctly picked all four division champs in one conference, sweeping the AFC.

Banks picked six division winners in 2003 and the Outsiders consensus had five and a half last year, having split on one division. The panel averaged 7.29 points and 2.29 division winners per entry this year, compared with 10.93 points and 3.93 division winners in 2003 and 12.03 and 3.85 in 2004.

111
by coldbikemessenger (not verified) :: Thu, 09/06/2007 - 11:28am

Third down
Shmird down
Romo is awesome
Dallas wins 10+

112
by The McNabb Bowl Game Anomaly (not verified) :: Thu, 09/06/2007 - 11:48am

Re: Chris #58

If you are referring to the "scramble for the ball" column, he doesn't make picks based on DVOA, he makes picks based on his personal opinion. His picks have nothing to do with the accuracy of DVOA.

Re: #4

I agree, I'm not sure that eliminating extreme records helps because it all it does is squeeze the predictions into a narrower range. Of course those records are rare, but if the season was actually played 1500 times, some teams would have extreme records. The whole point of looking at the mean is to see the central tendecy of the possible outcomes, which should include the extreme ones.

Re: 68

Three comments:

Did anyone predict that the 2005 Eagles would win 5 games? I doubt Vegas did.

Also, Vegas "predictions" aren't really predictions at all. They are assigned values intended to balance the betting equally on both sides. That's not the same thing as trying to predict the outcomes.

Finally, the fact that generic predictions perform equally to DVOA in mean absolute error is interesting but not damning. Generic predictions are not useful even if they have a low error rate, while actual attempted predictions are useful if they have a low error rate. Comparing to generic predictions is pointless. He should have compared the MAE to other "expert" predictions. (Suppose, for example, that you wanted to predict division winners. Generic predictions can't do that as well as DVOA.)

113
by throughthelookingglass (not verified) :: Thu, 09/06/2007 - 11:54am

92-I take all the predictions for the Colts offense with a grain of salt, as it is harder (I think) to predict them because of a lack of comparable teams.
A couple reasons for this.
1. Manning is so good at getting the right amount of yardage for a first down, which is why his DVOA has been stratospheric. I don't know if DVOA considers that a skill or luck, but in this case I think it's a skill.
2. Ridiculous QB-WR continuity. I think DVOA already adjusts for it, but the Manning-Harrison-Wayne 'telepathy' is hard to account for statistically.
3. The Colts seem to consider Olineman more replaceable than most teams. This is partly because of Manning's quick release, and partly because Howard Mudd is a demigod. Maybe the system overcompensates for the loss of Glenn because of that.
4. If the Colts have at least a 25% DVOA again this year, they will clearly have the best offensive run of any team in the DVOA era. The only team to have a DVOA of at least 20% in 4 straight seasons (which is what the Colts are trying to do) is the 2002-2005 Chiefs, and they had a streak-worst 23.7% DVOA the last year of the streak.
Bottom Line: We haven't seen a team like this before (what would the best historical comparison be?), so capturing that in a model is very, very difficult.

114
by silentbob (not verified) :: Thu, 09/06/2007 - 12:09pm

If you're making adjustments to San Diego's projection based on the Merriman suspension, then why not make adjustments for major injuries/suspensions that every other NFL team experienced as well?

115
by Benjy's Cousin (not verified) :: Thu, 09/06/2007 - 12:20pm

As a Bears fan, I am still hoping Rex shows positive develoment and performs ahead of what DVOA projections are. If that happens, the Bears run away for the NFC Norris. Yes - that's wishful thinking - but worst case scenario this team is still more talented than Green Bay at every position except QB, on both sides of the ball.

Green Bay's running back situation is extremely dangerous. History shows Favre's performance degrade when he is forced to force it.

116
by bub (not verified) :: Thu, 09/06/2007 - 12:28pm

mactbone -
For the 2006 season, either King Kaufman didn't note FO / Schatz's picks (even though they won the prior year) or they were just down right awful. I'll go with awful. See link:

117
by Nate Dogg (not verified) :: Thu, 09/06/2007 - 12:51pm

Why can't I use DVOA for betting purposes? Seems like that would be the ideal application for such hard work. Is it not possible to use DVOA as a basis for my own power rankings? Then all I would need is to scale them to point values.

118
by White Rose Duelist (not verified) :: Thu, 09/06/2007 - 1:11pm

#116

One, it's not like the 2006 DVOA predictions and NFL standings aren't public knowledge. Why not look up whether they are awful or not?

Also, even if they were that bad in 2006, what other predictor can be among the top three in three out of four years?

119
by jonnyblazin (not verified) :: Thu, 09/06/2007 - 1:16pm

I'm curious about how many previous season's data get computed into these projections. Look at Baltimore's defensive DVOA with Billick/Newsome:

99: -21.5
00: -30.0
01: -17.2
02: -6.5*
03: -28.5
04: -19.3
05: -11.1
06: -25.6
*This was the salary cap hell year where they axed almost every vet and started from scratch.

So according to these projections, this year will be Baltimore's worst defensive team in the past 8 years. But 6 of the past 8 years they've fielded a defensive DVOA of -17.2% or better, and 5 of the past 8 years they've had the best or 2nd best defense in the league. It seems like the odds favor Baltimore fielding a top defensive team given their track record.

I'm just not sure why the Baltimore projection is so low, given their extensive history of fielding great defensive teams, combined with the fact that they had very little roster turnover this past year.

120
by mactbone (not verified) :: Thu, 09/06/2007 - 1:17pm

Re 116:
He doesn't give a full listing like he did for 2004 so I can't be sure - the best bet would be to look at his beginning of the season article and see if he mentions using FO for the expert predictions. Salon hates my work computer so I can't check it myself.

Re 117:
You can use DVOA however you want but it wasn't designed for betting purposes and might not be the best tool you could use.

121
by gerry (not verified) :: Thu, 09/06/2007 - 1:18pm

In the early days of FO, an avid reader had a pretty good understanding of FO's stats (DPAR, DVOA) and season projection methodology. A few sensible adjustments for down and distance, league average performance, adjusted for the opponent strength, were made to create DPAR & DVOA. Projections were driven by a few major factors including out / underperforming the Pythagoran win estimate, the third down rebound effect, etc.

Now the process to construct both the metrics and the projections is a black box process. Only FO staffers have any idea what ingredients are used to make the sausage. When readers don’t understand the process and the projections don't "feel right", people will complain. FO will respond that if it doesn't "feel right", use the zlionsfan template. And then readers will look at the projections from the last few years and wonder if FO was right then. They'll look at MSE and critique FO. But the critique is not of the projections, but of the black box nature of the projections. If readers had any idea what the most important coeffecients were, they'd better choke on DAL's 6.4 mean estimated wins and STL's 5.8 wins.

Here's is a great example:
GB outperformed their Pythagoran win estimate by ~2 games last year. GB's defense also played better on 3rd down (ranked 1st) than 1st (21st) and 2nd (11th) down. DEF (6th) is a team strength relative to OFF (20th) & ST (29th). CHI outperformed their Pythagoran win estimate by ~0.5 games last year. CHI's defensive performance is mixed on a down basis (1st: 13th; 2nd: 1st; 3rd: 4th; overall 2nd). Both teams have identical projected SOS. Both Pythagoran win estimate and down/distance splits are major reversion to the mean signals and are basic FO precepts (see the "Pregame" page). Schatz continoully points to these metrics as being some of the most important. Thus, one would think that CHI would continue to be projected as a better team than GB (even if CHI mean reverts). Yet GB is projected 2 games better than CHI, while PFP chapters do not focus on these major drivers.

The CHI chapter points out that the DEF & ST will mean revert (I don’t disagree) and that Grossman sucks (I could hit the Fox's message board for that info). Meanwhile the GB chapter focuses on Favre breaking records (thanks again), a subjective review of recent draft history and comments of defensive depth. Nothing to address various OFF & DEF splits or Pythagoran wins (except to note that GB's OFF was horrible in the red zone and should improve).

If this weren't a black box process and readers had a better understanding of the major drivers / coeffecients, there would be less complaints about the projections.

122
by White Rose Duelist (not verified) :: Thu, 09/06/2007 - 1:21pm

#116 Actually, I'll check for you.

2006 DVOA projections:

Divisions: Philadelphia, Chicago, Carolina, Seattle, New England, Cincinnatti, Indinanapolis, San Diego
Wild Card: Washington, Atlanta, Kansas City, Pittsburgh

Actual:

Divisions: Philadelphia, Chicago, New Orleans, Seattle, New England, Baltimore, Indianapolis, San Diego
Wild Card: Dallas, New York, New York, Kansas City

That's 6 division winners and a Wild Card, for 14 points. That would be... ok, look, top 3 again.

Looks like DVOA is doing something right.

123
by Brett (not verified) :: Thu, 09/06/2007 - 2:25pm

I find DVOA fascinating, but at some point you have to step away from the computer, calculator and sliderule and go with your gut...

Trends, theroms, adjustment to the mean, etc., are all well and good, but a team's overall talent, to me, is still the greatest predictor of success.

The numbers have Green Bay winning the North, but does anyone outside of Green Bay honestly believe the Packers are a deeper, more-talented, better-coached, well-rounded team than the Bears?

I don't doubt that the Bears might slip some (their defense and special teams really have nowhere to go but down), but at the end of the day, when you match their talent level against the Packers, or any other team in their division, there's really no comparison. I feel other talented teams, like N.O. and Dallas, should similarly outperform their predicted ranks.

I think DVOA is useful all things being equal..but in the NFL not all teams are created equal and I don't need a computer algorithm to tell me that.

124
by Bobman (not verified) :: Thu, 09/06/2007 - 3:13pm

Jonnyblazin (119) I wholeheartedly agree. I know it's distasteful to have an Indy fan agree with you, but these things happen;-)

Now our respective teams streaks CAN fall off a cliff with a couple key injuries, but those are not necessarily projected/assumed. So barring some major weirdness (an asteroid wipes out Canton, Ohio?), the Bal D should be better than projected above and the INd O should be as well. I have not dug down to find other teams in the same boat, but I suspect there are a few.

In my mind, you regress to the mean after a flukily excessive bad/good year. But these two teams have shown consistency in management, personnel, and execution over years. Not sure why they're projected to have their worst season in years. Of course, as an Indy fan, I am only too happy to have people project them low. I hate it when they're picked to be the best--only sets us up for a fall.

125
by Pat (not verified) :: Thu, 09/06/2007 - 4:18pm

If vegas had to deal with the “emotional aspect� then wouldn’t a strictly numbers DVOA be even more effective? 17-15 is nothing to brag about.

You DO NOT judge projection systems like this!

I've said this before in this thread. I've said it before to you in previous threads. If Vegas has an over/under of 8 wins, and DVOA says "8.1 wins", and you pick the over, guess what? You just placed a 50/50 bet, so why would you be surprised when you lose 50% of the time?

This is why you use mean square error, not "picking percentage" or "winning percentage."

126
by MDZ (not verified) :: Thu, 09/06/2007 - 4:19pm

Bobman(124)
I agree with you and johnnyblazin. It seems that the "mean" for the Colts O and the Ravens D is excellent, so I'd expect both to remain so. Part of the reason I think the Ravens D is consistently underrated by projections is that they draft players for 2 or 3 years down the line that can train quietly until they get an opportunity. Therefore the system penalizes them for roster turnover, but their subs are better fits for the system than a random replacement player. Just look at Bart Scott and Adalius Thomas. That said, the DVOA system has to work leaguewide and it is improper to mess with the system to account for outliers (Balt O/Indy D/the Rams) at the expense of overall accuracy. People complain about their team being underrated, but it sucks much worse when your team is overrated.

127
by Brian (not verified) :: Thu, 09/06/2007 - 4:21pm

I'm curious what most people think. What's the best way to grade win total predictions?

-mean absolute error
-standard deviation
-mean squared error
-# of picks dead-on

Personally, I think mean absolute error is best. If you use squared errors or st deviation, you penalize "out on a limb" errors far more severely. It rewards very safe predictions closer to the mean (such as between 6-10 wins).

Mean absolute error doesn't penalize a prediction for going out to 12 or down to 4 wins for some teams.

128
by Pat (not verified) :: Thu, 09/06/2007 - 4:39pm

#127: Why wouldn't you want to penalize out on a limb predictions more?

If you're off by one game, that's not a mistake. That's accurate. You can't win 7.2 games - you can only win 7 or 8.

The best way to judge a prediction system is to figure out what the error distribution should be, and come up with a figure of merit based on that. MSE assumes Gaussian, mean absolute error assumes something flatter than Gaussian, so realistically, it's going to be something like "mean error to the 1.5."

That's still a little unfair, though. Suppose the game has "high leverage" unpredictable events (which it does - injuries). Should a projection system even bother to try to predict those? No. Therefore, if it results in a wider distribution than a more conservative prediction (which will naturally have a less wide distribution), is it necessarily worse?

Welcome to dealing with the statistics of games.

129
by Pat (not verified) :: Thu, 09/06/2007 - 4:53pm

I should also note that judging systems by how well they predict wins/losses at all is a poor measure. It's too imprecise, and it's got massively aggregated error. Take 2005, for instance - one team fell due to a high leverage unpredictable event (PHI), and suddenly something like 5 other teams jump upwards one win (or more).

DVOA isn't trying to predict the NFL. It's trying to predict teams. The best way to judge it is how well the projections predict their own true strength variable (i.e. how well these predictions predict DVOA).

130
by Becephalus (not verified) :: Thu, 09/06/2007 - 4:55pm

90 read 61 vis a vis why there are no projected 3 win teams etc.

131
by David (not verified) :: Thu, 09/06/2007 - 5:07pm

So, in case anybody's interested in how DVOA, as opposed to Bill Barnwell, actually did against Vegas last year, I went through and crunched it. 10 wins, 11 losses, and 11 pushes, which I defined as any expected win value where you'd get different results depending on whether you bet on the higher number or the lower.

132
by David (not verified) :: Thu, 09/06/2007 - 5:13pm

That should be "any non-whole-number expected win value". Obviously if the expected wins were 7 and the O/U was 8, that's not a push.

133
by Brian (not verified) :: Thu, 09/06/2007 - 5:49pm

Re:#128

You wouldn't want to exponentially penalize "out on a limb" predictions (e.g. 13 wins, 4 wins) because that rewards excessively safe prediction ranges.

Using statistical squared error methods (r-squared, mean squared error, std dev, etc. favor predictions that stay in a very narrow band such as between 6-10 wins. Excessively narrow prediction ranges would always beat predictions in a realistic range.

134
by evil otto (not verified) :: Thu, 09/06/2007 - 6:17pm

Much crunchy, nutritious food for thought here. Should be amusing eight weeks from now if all the pundits are exclaiming: "who would have thought the (Bucs/Panthers/Redskins/Jags) would be doing this well?"

135
by MC2 (not verified) :: Thu, 09/06/2007 - 7:45pm

2 years ago, the Saints sucked, while the Bucs and Redskins both made the playoffs. Who's to say that last year wasn't an aberration? Maybe the DVOA projections simply represent a return to normalcy.

136
by MC2 (not verified) :: Thu, 09/06/2007 - 7:51pm

#133: It depends on what you're trying to predict. If you're trying to predict the actual outcome of the season, then you should predict quite a few outliers. However, if you're trying to predict the true quality of each team, then the vast majority of your predictions should be in the 6 to 10 win range, which is the point I was trying to make in post #93.

137
by Pat (not verified) :: Thu, 09/06/2007 - 9:35pm

#133: It's not exponentially, it's quadratically. But in any case:

"Realistic" range? What's a "realistic" range for the average wins of all teams? There is none. You don't know it. It certainly isn't the actual range of wins, considering that would imply there's no randomness in the game whatsoever when in truth we know the game's about 30-40% luck or so. You can measure that.

If you had a National Coin-Flip League, consisting of 32 teams flipping coins 16 times, the CVOA (coin-adjusted value over average) Predictions would predict everyone at 8-8, even though the actual season would have a team or two down at 2-14 and up at 14-2. Yet those predictions would be fundamentally correct, and better than every other one out there. And how would you measure that? Mean-square error.

You're not predicting the actual NFL season. You're predicting the mean NFL season.

What you called "statistical error methods" aren't statistical error methods at all. They're methods that assume normally distributed errors (i.e. the probability density function is Gaussian). The PDF of a team's win is almost certainly not Gaussian, but mean average error assumes something that falls off like e to the -x, which is also certainly wrong (it predicts way too high a probability for zero and 16 wins for any team).

The actual probability density function of wins is probably a sum of a bazillion Gaussian PDFs. Considering it's a truncated distribution, that means that it's probably Gaussian with a long tail.

Which means that realistically, a simple error construction like "mean squared error" or "mean average error" would never work. You'd need some custom function to weight each point.

138
by zlionsfan (not verified) :: Thu, 09/06/2007 - 9:36pm

Must be football season. The helpful tips for improving DVOA are already arriving.

Some things I know that I know:

Vegas doesn't predict results. It predicts money.

I was going to ask why the Lions may not be as bad as I think, but I see throughthelookingglass already covered that.

You don't build a good system by generating predictions and then changing them to match your gut. You get the predictions, compare them to results, look for differences, and then see if there are ways to account for the differences within the system.

Pay attention to what the system is saying. It isn't saying the Bears won't win the division. It's saying they're most likely to be around .500, but they could end up anywhere on the map. It isn't predicting a specific win total for Detroit. It's saying there's a 95% chance that the Lions will win between 6 and 9 games (roughly).

0-, 1-, and 15-win teams are so rare that it would be very odd to see a team projected for that record, especially at the upper end of the curve. Teams have much less incentive to win a 15th game than they do a 14th, or a 12th, or a 6th.

139
by Sid (not verified) :: Thu, 09/06/2007 - 10:33pm

In his latest Page 2 article, Bill Simmons mentioned Pro Football Prospectus twice.

140
by billj (not verified) :: Thu, 09/06/2007 - 11:32pm

I looked at NFL stats from every which way for a number of years--gamblers do that. I would guess that most of your readers look at what you present from a gambling aspect. Your look at things are very innovative (love the 3rd down theory--who'd u thunk?) But trying to break it all down takes the joy out of it. Surprises are great. Play Belmont on Friday and enjoy the NCAA on Saturday and the NFL on Sunday.

141
by The McNabb Bowl Game Anomaly (not verified) :: Thu, 09/06/2007 - 11:45pm

One thing nobody has mentioned yet about DVOA and win predictions:

DVOA is not designed or intended to predict wins and losses. It is designed to predict team performance, which is usually but not always reflected by wins and losses.

DVOA is the best system out there for predicting how well or badly teams will perform. It is only good at predicting wins to the extent that wins actually reflect how good a given team is.

Further, more important predictions than actual wins and losses are division and conference standings (except to gamblers betting the over/under). DVOA and FO have been excellent at predicting which teams will win divisions and make the playoffs, which ultimately matters a lot more than how many wins those teams earned.

142
by The McNabb Bowl Game Anomaly (not verified) :: Thu, 09/06/2007 - 11:47pm

Re: 140

You're wrong, most FO readers are not hardcore gamblers, they are hardcore NFL fans.

143
by QB (not verified) :: Fri, 09/07/2007 - 1:33am

I happen to be both of those things.

144
by Scott (not verified) :: Fri, 09/07/2007 - 1:45am

See, a game like that is exactly why I can't see how Indy's offense can't be ranked anything but #1. Granted, they won't play a defense like the Saints every week, but come on...

Tony Ugoh had some struggles, Wayne fumbled for a TD, Manning and the receivers looked a little out of sync, but by the time the game was over, they put up 34 pts, 452 yards, Manning nearly with a 300 yd/3 TD game, and 164 yards on 5.7 ypc for the running game.

Too much talent on the field, too much consistency with the coaches, too much Manning for defenses to handle.

145
by Rich Conley (not verified) :: Fri, 09/07/2007 - 2:14am

145.

Scott, from what I saw, Indy's offense had a lot of trouble, except when exploiting CBs they released in the last year.

They just violated Jason David, and kept doing it. They wont find many teams like that.

146
by Becephalus (not verified) :: Fri, 09/07/2007 - 3:22am

137 Yay for euridition!

147
by Pat (not verified) :: Fri, 09/07/2007 - 3:48am

#145: It was hilarious how polite Manning was regarding David. I also found it bizarre how they basically turned on the afterburners after the half. It's like the halftime adjustments were Reggie Wayne telling Manning, "Um. In case you didn't notice, that's Jason David over there. I own him. Just throw deep to me all the time."

148
by Pat (not verified) :: Fri, 09/07/2007 - 4:40am

#137: substitute "mean absolute" for "mean average." Obviously, mean average was an oops.

149
by Rich Conley (not verified) :: Fri, 09/07/2007 - 10:44am

Pat, the worst part was that it wasnt even like he was getting juked or anything, they were just running by him. He looked like he was trying to play zone while chasing someone.

150
by Brian (not verified) :: Fri, 09/07/2007 - 12:16pm

#136: Good point.
#137: I get what you're saying and don't claim that I'm right and you're wrong. I just think MAE is more meaningful for how most people would judge a prediction, and it doesn't encourage excessively timid prediction ranges.

I disagree that MAE requires the error distribution to decay at e^-x (for this purpose). It's just a more convenient way to conceptualize "total games wrong." It's essentially just summing the differences of every prediction from the auctual wins. But 112 games wrong or 128 wins wrong aren't as intuitively meaningful as 128/32 = "4 wins off per team," which is MAE.

By the way, I recently did a little research on season win distributions. http://www.bbnflstats.com/2007/08/luck-and-nfl-outcomes.html I'd appreciate your thoughts/criticisms. From what I've seen, the season win distribution is a function of one big Guassian curve (luck like the coin flip league) acted on it by lots of other smaller Guassian functions (team skills). This "flattens" the resulting distribution, as if someone sat on a bell curve.

151
by Sammy3469 (not verified) :: Fri, 09/07/2007 - 1:14pm

I also wouldn't get totally gaga about the Colts offense just yet If DVOA is accurate at all, it was saying the Saints defense will be atrocious this year. Their LBs and CBs flat out suck.

I'm going to hold judgement until they play Jax.

152
by mediator12 (not verified) :: Fri, 09/07/2007 - 1:30pm

In regards to BAL dropping in DVOA this year:

DVOA is dependent on the opponents. BAL plays a HIGH number of higher value DVOA offenses than they have played in the past. Will they still be a great defense, absolutely. Will they be as dominant against much better offenses, probably not.

It all depends on how they execute this year. They could be better, but its not as likely that they will drop more towards the average playing better oppposing offenses, right?

153
by DVOA Projection Computer (not verified) :: Fri, 09/07/2007 - 3:25pm

Greetings, carbon-based life forms.

There's still plenty of room left on the "Saints Might Not Be As Good As Everyone Thinks This Season" bandwagon. As you've probably noticed, New Orleans' defense allowed a guy named Kenton Keith to get 6.4 yards per carry off them. Anyhow, I realize it's just one game. I'm just saying.

Kenton Keith. Guy went to New Mexico State.

154
by Pat (not verified) :: Fri, 09/07/2007 - 4:40pm

I disagree that MAE requires the error distribution to decay at e^-x (for this purpose).

I'm sorry, but in this case, you're simply wrong. You're using it as a metric for how good something is - that is, you're saying one distribution is somehow more likely than another. That is the same thing as defining an error distribution! You're saying that if the mean value is 8 for 4 teams, then predictions of 8, 8, 10, 6 are identical to 7, 9, 7, 9, which means the PDF integrated from 9.5-10.5 (and 5.5-6.5) is one half that at 8.5-9.5 (and 6.5-7.5). Work out the math, and you get what the error distribution is (it might not be e^{-x}. Not sure what it is, but it's certainly flatter than Gaussian).

Saying "oh look, it's four wins off per team" - so what? You can also quote RMS win deviations as well, and those are "games" as well. Saying that mean absolute error is better than RMS error is making a value judgment on the error distribution.

You made it again in that comment by saying "excessively timid" prediction ranges. Who decides what "excessively timid" is? See the above with the National Coin Flip League. There, the "best" prediction is everyone at 8-8! No range whatsoever! And yet still, it's fundamentally correct. So what's the "correct" range for NFL predictions? Certainly not the same as the actual season range.

I thought I've commented on the second part elsewhere. It's cute. You can show it other ways pretty easily, too. Just look at rematched NFL games, and you can easily see that games within about 7-10 points are essentially unpredictable (the score difference in the first game has no predictive power over the score difference in the second game) - that corresponds to something like half the games in the NFL.

You can also do it a lot simpler just by making a maximum-likelihood ranking of the entire NFL, then going back and evaluating the likelihood that each team would win its game. You could also get really clever and do a directed acyclic graph of the entire NFL season and look at the minimum percentage of games which cannot be contained within (a la beatpaths.com). In the end, you get the same answer - somewhere between a quarter and a half of NFL games (it depends on the season/team) are unpredictable.

Note that that says nothing about the predictability of the teams themselves. All it tells you is the range of true strengths in the league (so saying "it's luck" is a bit much - what you're really saying is about half of all games in the NFL are played between teams of roughly comparable true strength).

155
by Raiderjoe (not verified) :: Fri, 09/07/2007 - 9:44pm

re153 Raiders Qb J McCown (very athletic Qb) also go t Nex zmexico state. What does that eman about anything? ny gujy from any schoool can be good player in Nfl. Nothing stops him. from succesidnh.

156
by Raiderjoe (not verified) :: Fri, 09/07/2007 - 9:46pm

checked book. xctuLLT j mCcOWH GO TO sAN hOUSTON sTATE. 2 much Sieera Nevada at bar before workingon some Coros now at home,,,

157
by James (not verified) :: Fri, 09/07/2007 - 10:35pm

Wow, awesome stat predicting machine.
Ouch, even in virtual simulation my Lions are still losing. Is there no hope?

158
by Brian (not verified) :: Sat, 09/08/2007 - 12:42am

Pat-They really must take their statistics seriously at State! FO could definitely use a professional statistician like you.

Let me try to explain it this way. What I mean by excessively timid predictions is limiting win estimations to 7,8, or 9 wins. I guarantee that by doing that, I would beat ANY other predictions using a quadratic/exponential scoring method. But I'd call that range (7-9) unrealistic.

And the 7-9 predictions aren't really useful information. They would be best mathematically, but do not add much if any advance knowledge, (not to mention boring).

Let's say I'm forecasting an equities index, the Dow for example. If I limit my forecasts to within 1 point up or down, I'd probably be the best forcaster ever using squared errors. But that's not helpful to anyone. The same would go for a weatherman whose daily forcasts predict the temperature is either over 55 degrees or under 55 degrees. He'd be the most accurate weatherman ever according to you!

159
by Pat (not verified) :: Sat, 09/08/2007 - 2:43am

Jeez, after 10+ years of higher education, I'd hope that I know statistics. (Incidentally, I have no idea what that last two paragraphs meant. They don't make any sense to me - the forecaster didn't make sense, because it's a binary projection, and I don't get the Dow thing.)

I guarantee that by doing that, I would beat ANY other predictions using a quadratic/exponential scoring method.

Try it. Do it randomly, and you'd be surprised how bad you end up. Do it intelligently ("all good teams have 9 wins, all bad teams have 7 wins", and subjectively pick the good/bad teams) and you will beat a lot of predictions.

But picking the good/bad teams is half the point, and yes, it is in fact true that the majority of the deviation from 8-8 is a random walk. In a 16-game season, this isn't surprising!

And the 7-9 predictions aren’t really useful information. They would be best mathematically, but do not add much if any advance knowledge, (not to mention boring).

If the "true" mean wins are distributed from 7-9 wins, then any predictions that fall outside that aren't adding information either! They're just chasing the wind. They might seem "more exciting", but you can't call them more accurate, as the 7-9 predictions would be hit more often than not.

How do you judge how good a prediction is? Simple. How often they are right. For 4 equal strength, "true" 8-8 teams, a prediction of 8, 8, 6, 10 is less likely to occur than 7, 9, 7, 9 if the PDF is really Gaussian-ish. Yet the mean absolute error is equal between the two.

I'd also like to stress, by the way, that season predictions are completely separate from the DVOA predictions. Season predictions suffer from the fact that they convolute all the teams into one giant conglomerate mess. In 2006, the Seahawks go from powerhouse to abysmal thanks to injuries, and suddenly all the projections for ARI/SF/STL all look low by 2 games or more - so now all 4 teams look mispredicted, when in fact only one was mispredicted and it demolished the others.

To be honest, I don't really see the point of a season prediction. Just predict the number of wins that a team would get, facing an average schedule. That's the best you can do, and then the variation of other teams will all wash out.

The other thing that's important to note is that you're also neglecting that these predictions, specifically, give you more than just win totals. They give you errors. Using just RMS deviation assumes the errors are 1, when in fact, the prediction never claimed they were. If Baltimore wins 4 games, is that the same as Chicago winning only 3? Not at all. What you should be judging it by is ((predicted wins)-(actual wins))^2 divided by the predicted error squared. If you want to assume the errors are Gaussian.

Or you could just say predicting season win totals is stupid, and just look at the DVOA predictions themselves. Those have been remarkably accurate, by any measure.

160
by Brian (not verified) :: Sat, 09/08/2007 - 2:41pm

Ok, it's coming into better focus for me now. By the way, is your background in math or economics?

You're definitely right about the conglomeration mess. But given the Seattle example, wouldn't everyone's predictions suffer equally, as long as they used the same scoring method?

I'm interested in your thoughts on DVOA in general. As someone with twice the the statistics background as most people around here, I would think you'd have some severe criticisms of DVOA. From what FO publishes about it, it's full of arbitrary "success" points. To me, it seems like a poster child for an overfit model of performance. It's like saying a team is successful because it has succeeded.

When you say DVOA has been remarkably accurate, what do you mean? That pre-season DVOA projection matches up with actual DVOA? Or that DVOA predicts game winners well? Or that DVOA matches up with season win totals after the season is complete?

Only the FO people have a clue what all the points and coefficients are, and how they were determined. But beyond that, it seems like DVOA is not much better at ranking teams, or predicting winners than a very simple points for/against method, such as Sagarin's. DVOA seems incredibly complex and requires so many man-hours but doesn't do much better than 90 seconds of downloading stats from nfl.com and some analysis in Excel.

And although the play-level data allows us to analyze passing performance vs running, offense vs defense, and other things, I'm not convinced DVOA is any better than simply calculating offensive/defensive run/pass efficiencies. And it requires so much more work and complexity.

Don't get me wrong, I think DVOA has value, but it's not that much better than far simpler methods.

What are your thoughts?

161
by NY expat (not verified) :: Sun, 09/09/2007 - 8:06pm

re: 1, 13 and the other std dev comments

Maybe this is too late too find the eyes of any FO staffers, but if "plays out the season" means what I think it means, you could print out predictions for every game with what the simulator thinks is the likelihood of victory for either team. We could find out who the simulation think should beat Indy and maybe which games it is unsure of for Chicago or Arizona.

162
by Mr Shush (not verified) :: Sun, 09/09/2007 - 9:09pm

On today's evidence, the offensive projections for Atlanta and Tampa look rather optimistic. The Chiefs offense and Packers defense, on the other hand . . . right on. Well, actually, #30 might be flattering the Chiefs offense a bit.

I just watched my team in a game I was confident it would win for the first time in more than two years. It was a nice feeling. However, watching that game did very little to convince me that Houston has turned a corner, although Schaub did look pretty good, in the main. The Chiefs are awful. Pencil them in for the #1 pick right now.

163
by Scott de B. (not verified) :: Mon, 09/10/2007 - 12:02am

See, a game like that is exactly why I can’t see how Indy’s offense can’t be ranked anything but #1. Granted, they won’t play a defense like the Saints every week, but come on…

Tony Ugoh had some struggles, Wayne fumbled for a TD, Manning and the receivers looked a little out of sync, but by the time the game was over, they put up 34 pts, 452 yards, Manning nearly with a 300 yd/3 TD game, and 164 yards on 5.7 ypc for the running game.

Care to change your opinion after the New England game? New England also had some offensive issues, but on the road against a defense predicted to be better than the Saints put up 431 yards. Brady had 297 yards and 3 TDs. #1 isn't a lock for the Colts.

164
by Scott (not verified) :: Mon, 09/10/2007 - 5:24am

Nope, nothing to change at all yet. That's one game against Mangenius, a team they played 3 times last season, and a team they've pretty much owned since Brady became the starter (11-2 record against them since 01). The Jets may be predicted to have a better D, but do you really think a secondary with David Barrett, Darrelle Revis in his 1st career game, Kerry Rhodes, Erik Coleman, Hank Poteat, and Drew Coleman is going to scare anyone? And the front 7 got absolutely zero pass rush. That was probably the most impressive part of their offense today. The o-line gave superb blocking all day.

Let's see NE do it for a whole season first. The Colts have a bunch of games like that almost every season. Manning was putting up those kind of numbers in 99/00 before he became a super efficient QB and when the offense was just him, Harrison and Edge.

165
by Scott (not verified) :: Mon, 09/10/2007 - 7:47pm

Wait a second, this is probably part of why the Pats offense was good yesterday:

http://sports.espn.go.com/nfl/news/story?id=3012989

I've said for years Belichick bends the rules, if not flat out cheats, more than any coach in the league and I bet this is another tactic of his.

166
by Pat (not verified) :: Tue, 09/11/2007 - 10:27pm

Ok, it’s coming into better focus for me now. By the way, is your background in math or economics?

Physics, math, and statistics.

I would think you’d have some severe criticisms of DVOA. From what FO publishes about it, it’s full of arbitrary “success� points.

Why would the success points need to be arbitrary? With a sufficient dataset, they wouldn't need to be arbitrary at all. A play is successful if the team is more likely to gain a new set of downs than if the play hadn't been run. The yardage at which that happens is zero. The scaling with yardage can be determined similarly, although the "overall scale" is arbitrary.

That pre-season DVOA projection matches up with actual DVOA?

Yeah, that. That's an indicator that you're actually measuring the true strength of the team. Yards gained minus yards allowed are also a strong indicator of the true strength of a team, but they don't correlate well with points.

or predicting winners than a very simple points for/against method, such as Sagarin’s.

Win prediction is overrated. It's not that big a deal. It's a game, there's a game output function between two teams with true strengths, and therefore the maximum prediction can be easily figured out. Most predictions fall close to, but shy of, that maximum.

I don't know why you're putting so much stock in "prediction." It's a game. It can't be predicted that well. DVOA's just a measurement.

It's also really nice because DVOA separates out offense and defense correctly. Everyone who makes up "offense" and "defense" stats completely ignores the effect of field position and special teams, and so you get an "offensive" stat that is actually cross-correlated with defensive performance.

but doesn’t do much better than 90 seconds of downloading stats from nfl.com and some analysis in Excel.

Sure it does. Just depends what you're looking for. DVOA stabilizes much, much faster than maximum likelihood rankings, and therefore you actually have enough data to pull out sub-season trends. Take a look at the midseason rankings from last year, and the end-of-season rankings are +/-5% in virtually all cases.

This is absolutely not true for maximum likelihood rankings, which live and die based on wins/losses, and so cutting the data set in half would murder them.

It's a pretty fundamental problem - in football, wins come from points, but plays gain yards. How yards translate into points seems simple at first, but it's really not. Yards are consistent from year to year, which you want (it's a necessary condition of the true strength of a team), but points actually win you games. You want a statistic which correlates with points, but has the year-to-year consistency of yards.

Do I think DVOA is overly complex? Maybe. It has the advantage of attempting to put everything on an equivalent scale, which is ambitious, but intriguing (it gives the relative contribution of special teams, for instance). But if you wanted to get an equivalent statistic, you'd need to break down the games drive-by-drive, at the least ... and you'd end up doing mostly the same things.

167
by Alex (not verified) :: Wed, 09/12/2007 - 12:30am

Re 160:
It’s like saying a team is successful because it has succeeded.

I really don't want to sound mean here, but...what you just described is the very definition of successful. In fact, here's a dictionary definition of successful from Merriam Webster:

"Main Entry: suc·cess·ful
Pronunciation: -f&l
Function: adjective
1 : resulting or terminating in success *a successful attempt*
2 : gaining or having gained success *a successful investor*"

There's absolutely nothing circular or illogical about saying someone or something is successful because they succeeded. In fact, to say they are successful for any other reason would be incorrect.

When you say DVOA has been remarkably accurate, what do you mean?

This is explained a little in the FO FAQ:

"Q: Does DVOA really work?

A: Yes. The goal of DVOA is to balance two things:

* The correlation of the opponent-adjusted statistics from year-to-year, representing the intrinsic quality of a team irregardless of luck and random chance, and
* The correlation of the non-opponent-adjusted statistics to wins.

DVOA -- at least, the team version -- does these things better than any other statistic available. Here are some correlation coefficients to demonstrate:"

And then it lists some correlation coefficients, and DVOA and VOA are at or near the top of the lists.

168
by Pat (not verified) :: Wed, 09/12/2007 - 3:38am

One last point, though - if you look at the correlation coefficients, they're not really significantly higher than some more conventional stats. But DVOA can be improved. Points or yards can't.

But really, the entire idea is that there isn't a conventional stat that has the year-to-year consistency of yards and a strong correlation to winning.

Incidentally, above, I said "more likely to gain a new set of downs" - it should've been "more likely to score".

169
by James G (not verified) :: Wed, 09/12/2007 - 11:50am

But there's also another huge advantage of DVOA - it's what the "D" stands for. Breaking it down play-by-play give us VOA. But figuring it out who had hard or easy schedules gives us DVOA. It's possible to make defense-adjusted drive rankings, but even then, we'd be missing out on some of the most interesting things DVOA has shown - like the 3rd down rebound effect, which shows the advantages of breaking it down into plays.

Brian - actually more than just the FO people have a clue what goes into DVOA (even if we don't know the exact details). Read The Hidden Game of Football and you get the beginnings of the VOA, even if you don't end up with the exact formula. It's really the secret "D" adjustment that we don't know, although my guess is that there are ways of approximating it.

170
by Pat (not verified) :: Thu, 09/13/2007 - 2:47am

#169: DVOA isn't the only opponent-adjusted measure of a team out there - maximum likelihood rankings are (by definition) opponent adjusted.

The one thing that DVOA does that no one else does, I think, is separate offense, defense, and special teams strictly. Doing that requires at least a drive-level breakdown of games, because you need to isolate field position, and averages are useless in that case (starting on your 1 one drive, then their 1 averages to the 50, but it ain't the same as starting on the 50 twice).

You could probably do it by parsing the drive-level stats from NFL.com, which might be easier than parsing the play-by-play.

(The "defense adjusted" part is actually very simple - it's spelled out in the Methods page).

The benefit of play-by-play breakdown is that you can compare disparate portions of a game with an equal scale.

171
by Joe (not verified) :: Thu, 09/27/2007 - 12:51pm

The AFC South is undefeated outside the division (6-0)

Very impressive