Writers of Pro Football Prospectus 2008

Most Recent FO Features

KerriganRya11.jpg

» Week 2 DVOA Ratings

Stomping the Jags leaves Washington No. 2 behind only Denver. But what can we really learn from one big win early in the season, before we are applying opponent adjustments?

21 Oct 2005

FO Mailbag

by Aaron Schatz

So, you all may have noticed that we installed new code on the website this afternoon. If all goes well, this will solve our server overload problems from all our new FOX-related visitors, plus return some of the old bells and whistles and allow us to add more bells and whistles. Cowbell, perhaps. Anyway, comments didn't work for a couple hours but they should be fine now, and there are going to be some hiccups, so bear with us.

Now that folks can comment again, time for another look at the Football Outsiders mailbag. Most of these questions came via the contact form which you can use to e-mail any of the writers. While we will answer questions posed in discussion threads -- including a couple below -- we are much more likely to answer a question asked through the contact form. When those discussions get filled up with comments, they get hard to follow.

The point of the mailbag isn't just for asking me questions, but for any question you might have about any FO article. This week, however, it's all me. Feel free to ask the other guys some questions too, ya know! (Well, you do that for Al and Vivek, but those fantasy questions get answered in Scramble each week, not here.) This week, I'll finally answer a few questions from 1998 and 1999, and, yes, there's more about Denver. I'm getting burnt out on the whole Denver thing -- there's only so many times you can get the same insulting e-mails from people who clearly didn't actually read your commentary -- but it is a good way to explain how the DVOA system works.

Speaking of DVOA, this mailbag is pretty much all about that little innovative metric of ours, so if you are a recent addition to the readership you might want to read this explanation first. You also probably want to check out this week's DVOA rankings, which are discussed in most of these questions.

Brian Boorman: First of all let me say that I'm a Steelers fan, which will explain why I'm asking the questions I am, and I think ranking us fourth is not only fair but accurate. But my question is: Is there any way to incorporate injuries into the rankings? I understand that you don't go by gut feelings like other ranking systems, which is nice, but being a Steeler fan, that loss to Jacksonville, though difficult to take, doesn't scare me at all (and obviously not you guys either, considering they're still fourth). Hell, I was shocked it was that close with Maddox in there. But I wonder about other teams, especially now with several key players having been injured (Cadillac Williams comes to mind, Julius Jones, and I'm sure there are others).

I mean, these guys aren't going to be out for the rest of the season, but let's say a key player on the #5 team is out for games against the #20 and #25 ranked teams, and the #5 team loses both? Their ratings will slide, but when the player comes back they're no longer the same team that lost those two games. But the ranking reflects those losses.

Aaron: A valid point, Brian. Injuries are a huge part of why teams win or lose football games. The problem is that -- as of right now -- there is no way to quantify the effect of an injury. Someday there might be, but that's a long way off -- probably around the time NASA develops the interstellar warp drive.

If I were to drop a game from a team's rating because of an injury, I would be stuck having to make decisions about the importance of every injury -- is a missing lineman worth dropping a game, how about a receiver, how about a cornerback, etc. Everything in my system is a number, and I have done research on every change I've made.

This is why I say in each power rankings commentary, "Remember, of course, that any statistical formula is not a replacement for your own judgment, just a tool to use in analyzing performance." And occasionally, I will see what the numbers say without a certain game. But I always emphasize that I've removed that specific game or two because of a specific injury, and that these numbers are not the total stats.

Trust me, I'll be doing that in every playoff preview article I write for as long as the Steelers are playing. "The numbers without the Jacksonville loss are..."

ABW: OK, this is kind of a stupid request. But just for laughs, I'd like to see what LaDainian Tomlinson's passing DPAR is for this year, and how favorably it compares to some of the "quarterbacks" that have been lining up behind center this year. For instance, is LDT worth more as a passer than Michael Vick?

Aaron: Heh heh, fun e-mail there. Tomlinson has 3.6 passing DPAR this season. Yes, that puts him ahead of Michael Vick as a passer, because Vick has 2.1 passing DPAR. Tomlinson is also right ahead of Kurt Warner and far ahead of all the negative quarterbacks including Carr, Harrington, and yes, Daunte Culpepper.

As an aside, Culpepper will shatter the record for biggest year-to-year decline by a quarterback even if he is average the rest of the season. I wrote about that for the New York Sun this week and plan to run an expanded version of that research on the site. Maybe I'll run it when I run another set of Eli Manning similarity scores and we'll call it "Trading Places."

Steve Diehl: I've noticed that if you take pass offense DVOA and simply average it with run offense DVOA, you can pretty much hit the total offense DVOA or very close to it. To my unwashed, uneducated brain this implies that run offense is just as important as pass offense to offensive success or winning games. Can this be true? I thought there had been ample proof shown (don't ask me where) that the passing game was more valuable, over an entire season, to winning games than the running game. I sort of figured we had a genius here in Philly with coach Andy Reid. Could Andy and I both be wrong?

Aaron: Well, Andy might want to consider running the ball, you know, occasionally, since those play fakes look completely absurd when you know they are never going to run the ball. To answer your question, each play is considered separately. Therefore, offensive DVOA will only be an average of passing and rushing if the team passes exactly half the time. Most teams pass more than they run; this year, the average team passes 57 percent of the time.

I could run correlations proving that passing is more important than rushing and I'm sure I have done so in the archives somewhere, but instead let me use DVOA ratings to demonstrate why passing has more impact than rushing. Look at offensive ratings for 2003. The passing numbers go from -41.8% to 42.8%. The rushing numbers go from -18.8% to 22.9%. In other words, the scale for passing is almost twice as big as the scale for rushing. (2004 might confuse people because #1 Indianapolis and #32 Chicago were significant outliers.) A good passing game will contribute to scoring more than a good run offense, and a bad passing game will keep you from scoring more than a bad running game. Very few teams can build a good offense out of strong running and below average passing -- the last team that probably qualifies is the 2001 Kansas City Chiefs. Atlanta sure is trying this year -- third in rushing, 17th in passing through Week 6 -- but while their passing rank may be mediocre their passing DVOA is still above average (thanks in large part to Matt Schaub).

Ryan Pump: YOUR FOOTBALL RANKINGS SUCK! HOW CAN YOU RANK TAMPA BAY SO LOW WHEN THEY ARE 5-1. YOU ARE NOT GIVING ANY PROPS TO THE BUCS AND THERE FANS. NO WONDER I HAVE NEVER HEARD OF THIS SECOND RATE WEBSITE OR YOU FOR THAT MATTER! YOU HAVE 4 TEAMS RANKED AHEAD OF THE BUCS THAT ARE ONLY ONE GAME OVER 500 THAT IS TERRIBLE. FIX THIS PROBLEM!

Aaron: That's the Football Outsiders Johnny Come Lately Try Your Caps Lock Key Hate Mail of the Week. I'll tell you, if there's anything Football Outsiders is known for, it is underrating the Tampa Bay Buccaneers.

Michael Romero: How long do you plan to use this inferior rating system? Your readers obviously don't like it. The ranking of the Denver Broncos proves it is not a good gauge of how good a team is. All you do every week is use your hole space explaining why it is so inaccurate in team placement. Washington gets kudos' for winning close games Denver gets castrated for it. Come on we all think its time to get rid of this joke of a rating system you can't rate a team on last year and you can't drop a team on a four game winning streak at the time now 5. Its ok to admit it was a bad experiment

Aaron: I tell you, if there's anything Football Outsiders is known for, besides underrating the Tampa Bay Buccaneers, it's overrating the Washington Redskins. We would never, ever write about them the way we write about the Denver Broncos.

By the way, you guys are the readers. Do you like the system and the way I use my hole space?

Randy Childs: I'm trying to understand why the Cowboys are ranked 9th and two teams that they've beaten are ranked higher. The Cowboys beat both the Chargers and Giants this year, yet both teams are ranked higher (3rd and 5th, respectively). Is it due to games at the end of the 2004 season?

I'm particularly confused by the fact that the Giants are ranked higher than the Cowboys. The G-men have beaten Arizona, New Orleans, and St. Louis. Of that group, only the Rams stick out as a decent team and all of them have a losing record. The Cowboys have beaten San Diego, San Francisco, Philly, and the Giants. Throw out SF and they have beaten 3 teams with records of .500 or above.

I understand that your ranking method takes into account each individual play, not overall records. However, it seems to be reasonable to assume that the teams with winning records are making the plays to win games.

The three blemishes I can see on the 'Boys record are: 1) they lost to Oakland, 2) they lost a 13-point lead to the Redskins in the final minutes of a game in which they were otherwise dominant, and 3) they had to come back from a 12-point deficit against SF. On the last one, I would think that all of the plays that were made to overcome the 12-point deficit would at least cancel out the bad plays that led to being behind in the first place. And could the two long bomb TDs from Brunell to Santana Moss wipe out how much the Cowboys had dominated the Redskins for the first 55 minutes?

Aaron: See, people, it isn't impossible to question the DVOA numbers and still write a well-reasoned and respectful e-mail.

The Giants are third in DVOA, fourth on FOX. Dallas is eighth in DVOA, ninth on FOX, so 2004 isn't a large issue here. It really just comes down to the fact that, according to the system, the Giants' wins were more impressive than the Cowboys' wins, except for the Philadelphia win. Dallas has a positive DVOA on the Washington loss, since they outplayed Washington for almost the entire game. They have a negative DVOA on the San Francisco win, because it was a close win against the league's worst team.

The high rating for the Giants is largely special teams, and that's the element that is most inconsistent, so there's a good chance they are not this good. I picked Dallas to make the playoffs, and I still think they are the better team, although I'm not sure how important the Flozell Adams injury will be.

Here are the ratings for all three teams by week:


Team Week 1 Week 2 Week 3 Week 4 Week 5 Week 6
DAL 30.1% 27.9% -35.9% -38.2% 110.5% 51.8%
NYG 64.2% 59.0% -40.9% 85.0% BYE -25.9%
SD 0.9% 7.2% 80.1% 62.8% 14.0% 39.5%

With four games involving teams in my top 12, this is an important week. Each of these games will see the winner get an extra boost from beating a team that has been good the rest of the year.

KScottish: Your blatant bias towards the Broncos is evident ... I am sure your pajama loving computer geek friends love all of those other numbers and are trying to figure out a way to screw up the NFL like NCAA Football.

Aaron: Pajamas?

Before we get to a couple more questions about the 2005 Broncos, I need to get this in writing: I think the Broncos are going to win this week. I think the Broncos are a better team than the Giants. Objectively, based on this season's play-by-play, the Giants are for real and the Broncos are not as good as their record. But subjectively, based on my own opinion, the Giants are playing over their heads and the Broncos are actually not playing up to their level of talent, despite the close wins. Bias? Remember, I picked them to win the division and praised Al Wilson in our book. But instead of making up the power rankings each week depending on how I feel and which side of the bed I woke up on, I stick with the numbers, and then I use the commentary to explain where my personal subjective views differ from the stats. This somehow gets taken as "making excuses."

We've got a couple good questions here that tie into Denver and address the DVOA methodology, but I swear I am so sick of e-mail after e-mail about this team that I may never write about Denver again until we get to the playoffs.

Larry: You wrote in the Power Rankings: "What makes [Denver's] failure to convert third downs even more problematic is the fact that, because of their great running game, Denver faces an average of just 6.2 yards to go on third downs, the third-lowest number in the NFL." This strikes me as exactly backwards (or at least inconclusive). Since DVOA compares plays to average performance in the situation, then a low DVOA when facing favorable situations doesn't necessarily mean anything bad at all. It could indicate good third-down performance in terms of number of conversions. Not as good as it should be given the first and second down performance, but potentially still good in terms of converting the third downs. This all depends on exactly how much better than normal that 6.2 yards is, as well as the baseline performance in such situations.

Aaron: Yes, that's true, although so far this season Denver has converted third downs at a rate worse than the NFL average from every distance. The Broncos have faced more third-and-short situations than the average team, raising their general third-down conversion rate. This means they've been successful on first and second downs, and that does, of course, show up in their DVOA -- just not their third-down DVOA. And you've got to be able to convert third downs in order to win over the long haul. Blowing second down means you get third down, but blowing third down usually means you have to give up the ball (or four points). This is where the table was supposed to go, but then tonight I got this e-mail ...

Brian Walsh: We know about Denver's offensive problems on third down, but I just heard from the announcers while reviewing the Giants/Cowboys game that the Giants' defense is the worst in the league at stopping third downs. I was wondering if DVOA agrees. If so, it could make for an interesting game on Sunday, kind of "a resistible force meets a movable object."

Aaron: DVOA doesn't quite agree, because the Giants have five takeaways on third down. So their DVOA is 26th on third downs. But, yes, the resistible force is meeting the movable object. Now we can run the table (which also includes rushes and passes on fourth down):

3rd/4th Down
Yards to Go
NFL Average
Conversion Rate
Denver
Conversion Rate
Conversion Rate
vs. Giants

NFL Average
Opportunities*
Denver
Opportunities
Opportunities
vs. Giants

1-3 59% 57% 60% 22 28 18
4-6 46% 35% 94% 19 17 19
7-9 32% 16% 43% 17 19 17
10+ 20% 11% 31% 24 18 35
All 3rd/4th 39% 33% 53% 82 82 89
*Note: Teams with five games (including Giants) pro-rated to match Denver's six games.

Kibbles: Aaron, any word yet on why Denver has the third hardest schedule to date according to DVOA, yet their VOA is higher than their DVOA (meaning they're getting adjusted DOWNWARD when you account for opponent, as if they've played an easy slate)? It seems very … strange, to say the least.

Aaron: There are actually three differences between DVOA and VOA. One is the opponent adjustments. The other two are the difference between special teams with and without the adjustment for weather and altitude, and the difference between counting all fumbles as equal and counting only fumbles lost. The Broncos are affected by the weather and altitude adjustment more than any other team (ST DVOA: -2.0%, ST VOA: 0.1%). The Broncos have also had better-than-average luck so far recovering fumbles, recovering four of six fumbles by their own offense and six of nine fumbles by the opposition. As for the issue of opponent adjustments, I love it when a reader question is correctly answered by another reader:

Trogdor: I've been doing a little more thinking on that, and I've thought of ways that it's possible. I'll use a completely exaggerated example to illustrate.

Let's say the Broncos have played the 2003 Chiefs this year. Those Chiefs finished the year ranked #1 in DVOA (though only #6 in weighted DVOA). So by playing them, their schedule strength should go up, because “past schedule� appears to be just the average DVOA (or VOA) of teams previously played.

Now this is where it gets fun. Remember that KC team had a craptacular run defense (28th). So Denver decides to run the ball on nearly every play. They manage to hold the ball for about 50 minutes that game, running about 85 times and passing maybe 10. You see what's going to happen? The vast majority of plays in this game are being run against a nearly league-worst unit. So every successful run gets downgraded because it only came against KC. Also, by holding the ball so long, they don't defend many plays against KC's top-ranked offense. So when all the plays are totaled and weighted, there will be a ton for Denver that are downgraded, and very few that are upgraded. The net result is, even though their “past schedule� will go up, their DVOA for this game will be lower than VOA.

I hope that explains how this phenomenon is possible. The question now is, how is it happening to Denver? Do all/most of the teams they've played have some big unit/situational weakness that Denver has exploited and abused? If so, isn't that good coaching, to identify and exploit weaknesses? If that's the case, I wouldn't get too upset if I was a Denver fan.

Aaron: Denver's opponent adjustment is all about the terrible pass defenses they've faced. (The Denver defense actually gets a small bonus for facing difficult offenses.) DVOA rates Jacksonville as the number one pass defense right now, but all the other teams Denver has played are ranked 16th or worse.

Born a Bronco Fan/Die a Bronco Fan: Aaron, this may be a stupid question but I am still learning what the DVOA is all about. Does the system take into account the situation at all – for example, a third-and-long from the 22-yard line at the end of a half and behind four instead of behind three? So much of success in football is situational – it seems to me this is maybe the biggest factor for second half and in particular fourth quarter performance. As to the Broncos winning on the road, Jacksonville is a favorite for the rest of the way according to DVOA and Denver dominated them on the road.

Aaron: The answer to your situational question is that what you describe is actually what DVOA is meant to do, although not at quite such a specific level of granularity. If the play is "a third-and-long from the 22-yard line at the end of a half and behind four instead of behind three" it gets compared to "third-and-long from between the 20- and 30- yard lines in the first half of a close game."

The end of the half thing is an issue. When I started doing this analysis, I was using ESPN play-by-play data that did not include clock time, so I could only separate time by quarter. This summer we worked on changing our 2002-2003 ESPN data to official NFL.com data so I could have those clock times, and it isn't quite done yet. It will be done before I create the next upgrade to the system, and more granularity based on the last five minutes of a half, last two minutes of a half, etc. is on the "to do" list.

Also on the to do list are adjustments to the ratings based on whether games take place on the road or at home. Originally, this didn't make any difference, since every team plays the same number of home games as everyone else, the adjustments would all cancel each other out. With so much attention now paid to my week-to-week ratings, however, it is clearly important for me to take into account when one team has played four home games and two road games while another team has played two home games and four road games (not to mention that I never expected that a team would get nine home games in a season).

Ah, but here's the irony: if I did include such adjustments, it would move Denver down in the ratings, because the Broncos have played four home games and only two road games.

By the way, I'm glad your name is not Born a Fan of Nobody/Die a Fan of Whoever is Winning That Year.

QUESTIONS ABOUT 1998 DVOA

James G.: I'm wondering where did Antonio Freeman ended up in the wide receiver rankings. He was definitely high in the raw yards category.

Aaron: Last night, I was able to stick up the 1998 and 1999 team numbers -- total, offense, defense, and special teams -- but I haven't had a chance to lay out the individual players yet. Sorry about that. Freeman led the league in receiving yards in 1998, but because he had a low catch percentage he ranks eighth in DPAR and 14th in DVOA. (As I noted in the original 1998 commentary, the top receivers that year were Eric Moulds, Terrell Owens, and Rod Smith.)

Josh: Regarding the Jets in ‘98, recall that Vinny Testaverde wasn't the starter at QB at the beginning of the season. Glenn Foley started the first two games, both losses. Vinny then played the next two games, both wins. Then the fifth game (third loss), Foley started, was pulled in the middle of the game, Vinny finished. After that all Vinny. So you take out those three games Foley started and Jets were 12-1, and probably an even higher DVOA.

Aaron: You are correct. In fact, you are more correct than you realize, because the Jets improved on both defense and special teams over the course of the season, so during those first couple games they were worse at everything:


1998 New York Jets DVOA, with and without Glenn Foley starting
OFF DEF ST TOTAL
Foley: Weeks 1, 2, 6 -19.2% 13.3% -4.6% -37.2%
Testaverde: Weeks 3, 5, 7-17 23.2% -27.7% 1.1% 52.0%

Kibbles: Aaron, was Terrell Davis' rushing season the best you've encountered so far? Also, I'd be really interested to see Denver's DVOA for the first 12 weeks, and then its DVOA for the final 4 weeks (after they had clinched up home field advantage). Second, you mentioned that you might have overcompensated in the adjustments for how horrible SD's and Oaktown's offenses were. Could we perhaps see Denver's defensive DVOA vs. Oakland/SanDiego compared to its defensive DVOA vs. the rest of the NFL?

Aaron: Egads, more Denver questions? Well, this is about a different Broncos team, I guess. First, Terrell Davis has the highest rushing DPAR (65.7) of any running back since my stats begin in 1998. Priest Holmes is second with 64.5 DPAR in 2002, Marshall Faulk is third with 60.0 DPAR in 2000. However, Davis did not have the highest total value of any running back. Holmes had 22.2 DPAR receiving in 2002, and while rushing and receiving DPAR aren't exactly equal, let's pretend for a moment like we do for the Quick Reads column and say that gives him 86.7 DPAR. That's followed by Holmes again in 2003 (56.6 rushing, 19.9 receiving, 76.5 total), Marshall Faulk in 1999 (37.8 rushing, 36.9 receiving, 74.7 total), Faulk again in 2001 (43.1 rushing, 29.8 receiving, 72.9 total), Davis (65.7 rushing, 4.3 receiving, 70.0 total), and then Faulk yet again in 2000 (60.0 rushing, 8.7 receiving, 68.7 total).

You know, if Tomlinson steps it up a little, he could make this list thanks to that 3.6 passing DPAR.

But I digress. From what I can tell, the 1998 Broncos did not clinch home field throughout the playoffs until Week 14, so you're actually talking about comparing Weeks 1-14 to Weeks 15-17. It's also interesting to note that the Broncos didn't rest their starters: Terrell Davis and John Elway played all three games, and of course the Broncos were still working on a perfect season going into that Week 15 game against the Giants.


1998 Denver Broncos DVOA, before and after clinching top seed
OFF DEF ST TOTAL
Weeks 1-14 34.8% 0.2% 2.3% 36.9%
Weeks 15-17 21.6% 22.0% 1.7% 1.3%

To answer your final question (with an additional note):


  DVOA
(adjusted)
VOA
(no adjustment)
Denver defense vs. San Diego -3.4% -47.0%
Denver defense vs. Oakland -8.3% -35.4%
Denver defense vs. everyone else 7.0% 4.0%
Denver defense vs. everyone else, not counting Weeks 15-17 0.2% -12.1%

QUESTIONS ABOUT 1999 DVOA

Charles: Hey Aaron, do you have the DVOA of the 62-7 Jags-Dolphins playoff game from that year?

Aaron: Unfortunately, not yet. I still need to break down the playoffs for the years 1998-2002. That's on the schedule for February 2006 along with the 1997 play-by-play.

Josh: I'd like to see the Jets breakdown with Rick Mirer as QB and Ray Lucas as QB, I imagine it's fairly large.

Aaron: Oh, you bet it is.


1999 New York Jets DVOA, Lucas vs. Mirer
OFF DEF ST TOTAL
Mirer starting: Weeks 1-5, 7-9 -14.3% -8.9% 1.3% -4.1%
Lucas starting: Weeks 6, 10-17 7.3% -15.7% 2.1% 25.1%

Ted: Second, do you have rankings for variance? I reckon the Raiders were the least consistent team in the NFL that year. You mention the Tampa Bay game, but they also beat the Bills, Vikings and Chiefs (once) but lost to the Chargers, the Broncos twice and the Packers.

Aaron: Ready for something strange? Oakland is 21st in variance for 1999. They were losing to worse teams and beating better teams, but they were always almost doing it by a close score. The three wins you mention came by a combined 14 points. The four losses you mention came by a combined 16 points. So with the exception of that 45-0 annihilation of Tampa Bay, the win over Buffalo, and a 16-9 loss to Miami, Oakland's DVOA for each game fell between the narrow range of -25% and +35%.

Posted by: Aaron Schatz on 21 Oct 2005

48 comments, Last at 24 Oct 2005, 11:45pm by James

Comments

1
by thad (not verified) :: Sat, 10/22/2005 - 12:09am

Aaron,
after reading all the comments by various readers about Denver, I am really confused and have no idea of what to make of them? Can you explain? Thanks!
kidding, I generally have the attention span of a bug, I admire yours

2
by Andrew (A.B.) (not verified) :: Sat, 10/22/2005 - 12:10am

re: Opponent adjustments and Trogdor's explanation

It sounds to me like DVOA is missing something by only looking at one play at a time and not on the relative quantities. Trogdor's 50 minute rush example would be a great example of an OC doing exactly the right thing: attaching an opponent's weakness. And yet the offense is essentially getting punished for this in DVOA numbers. Alternatively, an offense might have inflated numbers by not sticking with what works -- in other words, an above-normal passing performance against a top pass D will improve DVOA even when, from a game perspective, the offense would have been better off with more plays on the ground against a bad run D (even if each individual run is no better than average against that team).

3
by KL (not verified) :: Sat, 10/22/2005 - 12:40am

Do you like the system and the way I use my hole space?

I like the system but could you point me to where this hole is?

4
by Ted (not verified) :: Sat, 10/22/2005 - 3:49am

Thanks for answering my question Aaron. I, er, stand corrected. I have another question though (apologies if you have already answered this question somewhere else). Do you know what the best performance by any team in the

5
by Nathan (not verified) :: Sat, 10/22/2005 - 3:57am

Can we have a thread devoted to talking about Denver and then never mention the subject again?

6
by Kibbles (not verified) :: Sat, 10/22/2005 - 4:28am

Thanks for the answers, Aaron. A couple of things...

I don't think you're overcompensating Denver's defensive adjustments because of Oaktown and SD. I mean, Denver's defensive DVOA during those 4 matches was better than its DVOA against the rest of the league, so it's not like their numbers are getting dropped due to overcompensation there.

Second, I understand that Denver still had something to play for late in the season. I just think it'd be hard for the team to get up pretty much any intensity when it's abundantly clear to them that they've got HFA already wrapped up. I think that's when the coaching staff starts planning more for the playoffs and less for the weekly games, showing more vanilla schemes and giving away less of what they're going to do during the postseason. I'm glad to see the DVOA bears this out, because I like patting myself on the back.

Also good to know that DVOA includes the special teams adjustments, while VOA does not. That certainly explains a large part of the drop.

Thanks again for another bang-up job.

7
by Kibbles (not verified) :: Sat, 10/22/2005 - 4:30am

A note: The 1998 team efficiency rankings are incorrectly labeled as the 1999 team efficiency rankings. Just FYI.

8
by Kibbles (not verified) :: Sat, 10/22/2005 - 4:35am

Hey triple post. Man, you talk about a bad year for rushing offenses... in 1998, the top 3 team rushing DVOAs were SF: 23.4%, Denver: 22.4%... and Buffalo: 6.2%. And only 4 other teams had positive rushing DVOAs (if you can count .2% as positive).

Consider this post just another kudos on a job well done, Aaron. I'm a numbers junkie on a stats high right now.

9
by Björn (not verified) :: Sat, 10/22/2005 - 8:36am

Yeah, FO has ALWAYS been underrating the Buccaneers and giving major kudos to the Redskins.

Aaron, why don't you publish the funniest hate mails you recieve every week in their own column?

10
by admin :: Sat, 10/22/2005 - 11:17am

Sorry about the strange comment spasms, they are related to the re-code and using quote marks. We're working on the issue.

11
by Eric (not verified) :: Sat, 10/22/2005 - 11:20am

The pajama reference is in the link. The guy was probably kidding, and offering fake complaints...if he's following the spirit of the 'pajama-wearing stathead/blogger' crack.

12
by Scott C (not verified) :: Sat, 10/22/2005 - 11:47am

Yall remember, the geeks shall inherit the earth.

Seriously, do these people even read the whole Power Rankings column? You had a perfectly good explanation about how the Miami loss impacted Denver and at what point it would matter less in DVOA rankings (after Week 8). But I do love the all caps emails, though I generally get them as forwarded right-wing rants about JANE FONDA from assorted relatives.

13
by admin :: Sat, 10/22/2005 - 11:49am

The idea raised in comment #2 is an interesting one, and I'll look into it. On the other hand, if you remove the opponent adjustments, you are penalizing teams for playing difficult schedules. For example, which of these do you believe is true about the 2004 Cincinnati Bengals:

1) Carson Palmer is a below-average quarterback.
2) Marvin Lewis is a bad coach who does not understand how to exploit matchups.
3) A schedule featuring the AFC East and AFC North meant that the Bengals faced many teams with quality pass defenses.

At the same time, the split in quality between run defense and pass defense (or run offense and pass offense) is very real.

When Kevan Barlow ran for 95 yards against the Colts, but Alex Smith threw all those interceptions, did this mean that Kevan Barlow is a good running back who knows how to exploit matchups, or that the Colts can't stop the run but are good against the pass?

When the Falcons run for 170 yards and Vick passes for 120 yards, does this mean that the Falcons are a one-dimensional offense or that they know how to exploit matchups? If all of a sudden Matt Schaub shows up as quarterback and throws for three touchdowns against the Patriots, is this a sign that the Falcons are great at exploiting matchups, or that Duane Starks sucks? The following week, with Vick at quarterback, the passing went way down even though the Saints and Patriots are both poor pass defenses. Did the Falcons forget how to exploit matchups, or is Vick just not as good a passer as Schaub?

Not that the issue of exploiting matchups is unimportant. But I think the question that I need to study is "Do teams dramatically change their run/pass ratio when playing a team that stops one better than the other." To be honest, I'm guessing the answer is no. Remember that in last year's playoffs, the Steelers did not change their run/pass ratio against the Jets even though the Jets were one of the league's top run defenses and one of the worst pass defenses.

14
by Arkaein (not verified) :: Sat, 10/22/2005 - 12:59pm

This idea of DVOA matchups is very interesting. I agree that teams should not be rewarded in DVOA for exploiting opponent weaknesses (because this does not improve predictive power of a unit against teams without such weaknesses), but I wonder if looking at run and pass DVOA individually would be a useful tool in predicting game outcomes.

I don't remember the exact methodology used for this year's FP predictions, but I wonder if they could be improved by bolstering a team's odds when it has offensive units that matchup favorably with the other team's defense.

An example: two teams meet, each with 0.0% DVOA on both offense and defense. Each team is equally balanced on defense but team A has positive rushing offense DVOA and equally negative passing offense DVOA. This should give a theoretical advantage to team A because the offense gets to decide what type of play to run, not the defense. Oddly, this gives the advantage to the unbalanced offensive team, but would favor balanced defensive teams (fewer weaknesses to exploit).

Of course, maybe none of this matters if the teams don't actually exploit these advantages, but maybe it matters enough to enhance prediction.

(BTW, the comment preview shows no paragraph breaks and gives no place to edit, might need to retool the new code a bit.)

15
by Ron Mexico (not verified) :: Sat, 10/22/2005 - 2:07pm

Washington gets kudos’ for winning close games Denver gets castrated for it.

If any team is getting "castrated," I'd say it should probably be the Vikings...

16
by princeton73 (not verified) :: Sat, 10/22/2005 - 2:27pm

Do you like ... the way I use my hole space?

I wouldn't touch that with a set of bolt-cutters

17
by cjm (not verified) :: Sat, 10/22/2005 - 2:56pm

Culpepper will shatter the record for biggest year-to-year decline by a quarterback even if he is average the rest of the season.

After all the hoohah about Randy Moss leaving, you know what's hurting Daunte a lot more than that? Matt Birk being injured. Gets my vote for the single biggest reason the Vikes have imploded this year. Birk is the real leader of that team.

And you know what? Birk would have forgone the surgery and played this year if the Vikes had been willing to guarantee his contract through next year.

18
by karl (not verified) :: Sat, 10/22/2005 - 3:02pm

Anytime I read one of these douchebags railing on DVOA (and FO in general) without bothering to understand what it is, I flash to the endscene in Jay & Silent Bob. Aaron, you and MDS should really consider it.

19
by Will Allen (not verified) :: Sat, 10/22/2005 - 3:21pm

I agree that losing Birk was huge. However, if neither party was willing to shoulder the risk of further injury, by playing through the torm hip labrum, it seems pretty likley that sitting out the year was the best option for all parties. Sure, the year may be lost due to the decision, but attempting to play through it may not have saved the season in any case, and may have meant that Birk's future services would have been lost as well.

I don't think trading Moss per se will be viewed as a terrible error in the future, because I think Moss is likely to be a chronically injured player for the rest of his career. Having said that, the Vikings receivers this year have been horrible, and have led significantly toward Culpepper's statistical fall-off. From guys tipping catchable passes, to failing to jump more than 2 inches in an attempt to catch the ball, a significant percentage of Culpepper's interceptions have not been the result of terrible throws (I can think of four off the top of my head). Toss in the interceptions when he has been trying to force a comeback while three touchdowns down, due in part to lousy defense (see Bengals game), along with poor interior line play, and Culpepper, who certainly has not played well, should not be seen as the main problem.

The only Viking who is having an outstanding season is their punter.

20
by TroyF (not verified) :: Sat, 10/22/2005 - 5:10pm

Stop complaining about the Denver mail. you have a system where a team who has beaten five consecutive teams with winning records is ranked as a middle of the pack football team.

I'm not saying the ranking is wrong. But you have to know that when there is an anomoly like this, it's going to gain attention. The attention is a win/win. If it turns out that Denver falters again and does end up a fringe playoff team, it give you some ammo to talk about how DVAO showed us something others didn't see. If it turns out that Denver continues to win/win/win and the DVOA doesn't catch up for some reason, it's a tool to determine potential flaws with the system.

Either way, your readership is going to continue to move up because of not only the controversey, but the way you deal with it.

So relax and get used to writing things about the Broncos. :)

21
by Matthew (not verified) :: Sat, 10/22/2005 - 5:29pm

As a lifelong Denver fan, I should apologize for the ignorance of so many others. It's terrible, Broncos fans are starting to take the place of Vikings fans as the craziest people on earth.

Denver hasn't shown me nearly enough to be in the discussion as a top 5 team. I think they play enough good teams in the next few weeks that they'll answer the question for us soon enough, but for now, I'll believe DVOA over the latest random power poll on other websites.

22
by fyo (not verified) :: Sat, 10/22/2005 - 6:40pm

I don't WANT to know the way you use your hole space. That's entirely up to you and certainly shouldn't be shared on a Football site. Please!

23
by fyo (not verified) :: Sat, 10/22/2005 - 6:43pm

bah! I posted that, or tried to rather, many hours ago, but there's a new (rather annoying) preview bit. Unlike the old one, where the preview occupied a seperate page, this one is hidden way down at the bottom of a large article. (And, for some reason, my browser didn't skip down there). Please bring back the old preview page! It should be easier on your server and bandwidth as well, since the article doesn't have to be retrieved from the database (plus any other SELECTs you may be performing, such as looking up poster and email, if not part of the original SELECT).

24
by Jere (not verified) :: Sat, 10/22/2005 - 6:54pm

"Do you like the system and the way I use my hole space?"

Aaron, are you talking dirty to us all???

25
by Sean (not verified) :: Sat, 10/22/2005 - 7:01pm

What a harrowing bunch of e-mails (not all of them- we know which in particular I'm referring to). The price of success. (Shudders)

26
by MJK (not verified) :: Sat, 10/22/2005 - 8:35pm

I think it's interesting that in a couple of the cases Aaron describes above where he compares DVOA for teams with two QB's, we see that there is an improvement in all three types of DVOA with a better QB, not just offense.

For instance, the Jets Foley vs. Testaverde: Offensive DVOA went up (OK, Testaverde is better), but defensive DVOA also went up (maybe because they found themselves on the field less often, with longer breaks? And got to play from ahead more?) but SPECIAL TEAMS? Why would a different QB affect special teams play? The only think I can think of is that when you have a better QB, you have more kickoffs and fewer punts. Also, I'm surprised that DVOA is this affected by a QB change.

It almost implies that there is far greater coupling between the offensive performance of a team and the defensive (and special teams) performance that maybe is traditionally believed. This begs two questions: (1) does this trend hold for other teams that changed QB mid season over the last number of years, and (2) if so, why?

27
by MJK (not verified) :: Sat, 10/22/2005 - 8:37pm

Sorry, in my last post I meant to say that I'm surprised that DEFENSIVE DVOA is this affected by QB play.

I second not liking the new preview format. It also doesn't seem to be showing my paragraph breaks properly...

28
by Andrew (A.B.) (not verified) :: Sat, 10/22/2005 - 10:33pm

Re: 13

Aaron, I think you formulated the question formulated correctly. But I highly doubt that the answer is a simple no. Isn't the whole job of an offensive coordinator to design a gameplan and call plays that take advantage of the mismatches in talent?

The only question is whether this may be masked in the data. Arkaein (14) has a point in that offenses have the advantage in that they determining the play mix, but it's not as one-sided as that. Example: A good running team may abandon the run if facing a team with an ostensibly bad run D, but who decides to overstack the line. That then opens up a deep passing game, until the D adjusts again.

29
by Andrew (A.B.) (not verified) :: Sat, 10/22/2005 - 10:34pm

And btw, the following line (above the comment field) is incorrect:

"Line and paragraph breaks are automatic"

30
by Andrew (A.B.) (not verified) :: Sat, 10/22/2005 - 10:36pm

Wait, now it's working again. Maybe it's my browser. Sorry about the bandwidth.

31
by admin :: Sat, 10/22/2005 - 11:02pm

Obviously, there are some matchup issues we'll never be able to address in DVOA. I meant to mention this in that previous comment, but I think "exploiting matchups" is usually about something more specific than just run/pass ratio: which cornerback to attack, which defensive lineman to run at, etc. Not how much to pass, but where.

Actually, Andrew, you're an economist, if you've got some time and want to formulate some sort of way to test the data to see if we think that teams change their run/pass strategies in ways that will cause the opponent adjustments to make DVOA less accurate, drop me an e-mail, and I'll get you the data.

Re comment 26: We did a cursory look at this a year ago, click link on my name.

32
by C (not verified) :: Sun, 10/23/2005 - 12:38am

Why is everyone so upset about Denver? It seems to me that this is a perfect application of Bill James' famous 80/20 rule of thumb. If you invent a new statistical measure, you want it to give you about 80% what you expect, and 20% things you don't.

James felt that if the numbers gave you *more* than 80% of conventional wisdom then it was probable that you were in some way or another putting a thumb on the scales, biasing the results of your study. If you had less than 80% then it makes you wonder whether you accurately depicting reality, since conventional wisdom does have a lot of truth.

So Denver could well be an outlier, but that's not a problem with DVOA, necessarily, any more than than middling stats that Derek Jeter records is a sign that there is something wrong with those formulas either.

33
by bobman (not verified) :: Sun, 10/23/2005 - 1:25am

Dear Diary: Why, oh why doesn't anybody ever ask questions about Denver?

Also, note to Aaron: Be careful what you do with your hole space in public. I had an uncle who got arrested one time for that. No, twice.

34
by jebmak (not verified) :: Sun, 10/23/2005 - 9:25am

It is kinda sad, I used to love looking at all of the different power rankings online, but now, looking at them and seeing all that gut feeling crap makes me a little ill.

On a side note Aaron, you need to get a 'Jimmy' of your very own to sort through the hate mail.

35
by Catfish (not verified) :: Sun, 10/23/2005 - 2:25pm

TroyF, I don't think Aaron is complaining about the quantity of Denver-related mail, but the quality of it. Aaron has always been very good about accepting valid criticism and working to address it. However, most of the emails he's gotten seem to have been sent by people who don't bother reading his explanations and immeadiately decide that Aaron is a pajama-clad geek living in his mom's basement.

36
by Walt Pohl (not verified) :: Sun, 10/23/2005 - 8:36pm

Aaron: You are clearly a level-headed and polite person, which in some ways is too bad, since you have an awesome opportunity for some trash-talk. A less well-adjusted person would put up an article entited "Denver Loses. I Called It!"

37
by Nelphonious (not verified) :: Mon, 10/24/2005 - 11:21am

Some of the Denver anomolies regarding DVOA and Carolina too, look to 2004 drive stats.Starting LOS,red zone defense,giving up huge amt. yards between the 20s and still forcing punts,one year luck of turnover margin, in certain combos on any given year skew(screw up) the stats.

38
by Sammy3469 (not verified) :: Mon, 10/24/2005 - 11:40am

Aaron I tend to agree with you on the exploiting matchups though there are some caveats that should be noted (though I have no idea how DVOA would ever pick them up).

Just yesterday in the Den-NYG game the announcer said that Eli holds the ball longer than just about any other QB in the league. Thus the Denver D coord decided to blitz early and often. I'll assume most teams will do this against Eli this year. How that shows up in DVOA is up for debate.

39
by billvv (not verified) :: Mon, 10/24/2005 - 1:27pm

No Denver posts since 10/22. Hum...makes you wonder where they went.

40
by Richie (not verified) :: Mon, 10/24/2005 - 3:26pm

YOU ARE NOT GIVING ANY PROPS TO THE BUCS AND THERE FANS.

Aaron, he's right. You aren't giving props to their fans. In fact, you aren't giving props to any of the fans. I think it's time to start work on your fan DVOA.

I'm guessing Arizona's fan DVOA will be close to zero, since they have none, and the fans they do have don't care. The Raiders will be highly negative, because their fans are the most myopic in any sport.

41
by James (not verified) :: Mon, 10/24/2005 - 4:31pm

The matchup issues brings up an interesting point.

Yet another reason why football is not played on paper. DVOA is the best tool I've ever found as far as evaluation. The fact that has been tweaked more than Fred Taylor's hamstring is another example of why it is great.

It's not perfect but it is a work in progress, that seeks to be as close to perfect as possible.

What's great about this site is I can use alot of the ideas, go to NFL.com, and evaluate each and every game myself to form my own opinion if I feel like. Or I could just write hate-mail in capital letters just like every Bronco fan. Somehow Jake Plummer and Mike Shanahan are on the same sideline yet they have managed to win 5 games. That is a miracle and Bronco fans should be in church praying instead of complaining to Aaron that his "rankings" are wrong.

42
by EorrFU (not verified) :: Mon, 10/24/2005 - 5:51pm

I was board and playing around with the Defensive DVOA numbers and it backs up the idea that a pass offense creates more scoring than a run offense.

I did a regression of the total DVOA and charted it against lines of best fit for passing and rushing. The slope for the passing defense was much higher than Total DVOA where the run slope was smaller.

Basically it shows that a great secondary does more for a defense in DVOA terms than a good run defense.

The teams that went against this trend were TB, MIA, and CAR. Those seem to me to be the three teams with serious upset potential.

Just some food for thought.

43
by Mr Shush (not verified) :: Mon, 10/24/2005 - 5:59pm

Assuming all five of Carr's runs were broken passing plays, the Texans called 28 runs and 14 passes against Indy on Sunday. I'm guessing that it's pretty unusual for a very bad side to have a 2:1 run:pass ratio in a loss to a very good side, so I'm thinking that's a case of matchups affecting play-calling. Whether that's attributable to the Colts' bad run defense or the Texans' horrible pass offense I'm less sure.

On a side point, note that Carr had 7 yards per attempt rushing but only 5.3 passing - and only 50% more total passing yards than rushing. That's just daft. This is not Mike Vick we're talking about here.

44
by EorrFU (not verified) :: Mon, 10/24/2005 - 6:01pm

Btw, I have been using Defense and Offense matchups for wagering purposes and have found that it works better than a strait DVOA comparison. I have tweaked it every week so far and have been rather successful. It seems much better with predicting the total points scored than a straight out win.

They are usually just a guide but have gotten a few games right where concensus was different than the results. I still want to figure out where the holes are. It is hard because I currently value passing and rushing offense equally when in fact they do not seem to equally contribute to scoring.

I know when there are 2 great passing mismatches here will be more scoring than I originally hoped.

I am also figuring out how to integrate the Offensive/defensive line matchups into my system. which can be revealing.

Like last week where it showed that MIA was horrible at 10+ yard runs and that TB was the best in the league. These stats seemed to foreshadow the fact that TB would have more running success than a strait DVOA comparison would suggest.

45
by Pat (not verified) :: Mon, 10/24/2005 - 6:10pm

#44: You could try weighting run/pass by their actual run/pass ratio. In other words, net off = (pass off+pass def)*(pass fraction) + (run off+run def)*(run fraction). You can get run/pass ratios from the DVOA numbers, since the total offense is just the rush and pass offense weighted by the run/pass ratio of the team (so the run/pass ratio is just (TOT-PASS)/(RUN-PASS) - for example, for the Eagles, it's 28.3%). That still doesn't take into account the possibility that running translates to winning better than passing, though.

46
by MTR (not verified) :: Mon, 10/24/2005 - 7:55pm

Re: matchups, the point is that you can't stand all the NFL teams in a row and say "you should beat everybody else, you should beat everybody but the first guy" and work your way down the row. Even if you limit matchup analysis to run/pass you get circles.

47
by Pat (not verified) :: Mon, 10/24/2005 - 9:43pm

Yah, but you can say "70% of the time, you'll beat him" and play the percentages. If you could actually guarantee things, um, the game'd be rigged.

48
by James (not verified) :: Mon, 10/24/2005 - 11:45pm

It'd be awesome if Aaron could do a week by week Pro Bowl report so all the readers would have a good idea who actually deserves to play in the pro bowl. Then people making it on name alone could stop getting invite and deserving players could actually get recognition.