28 Oct 2005
by Aaron Schatz
Time for another look at the Football Outsiders mailbag. Most of these questions came via the contact form which you can use to e-mail any of the writers. (Once again, this mailbag is all questions for me about the DVOA ratings, but I do mean to include questions to other writers in the future.) And while we will answer questions posed in discussion threads, we are much more likely to answer a question asked through the contact form. When those discussions get filled up with comments, they get hard to follow.
Be aware that we reference plenty of our innovative FO stats here, not to mention their unfamiliar terminology, so if you are a recent addition to the readership you might want to read this first.
By the way, no funny hate mail this week. There was less of it, but it was all just annoying.
Let's start with a question that has literally been sitting around in my e-mail for months. It didn't take long to answer, either, so I'm an idiot.
John Grenci: Aaron, you probably have stated this somewhere before, but is there a correlation of red zone efficiency from year to year? What about defense?
Aaron: I assume you are asking about DVOA red zone numbers, not the league's red zone efficiency numbers. I checked the correlation of red zone DVOA from year-to-year during the period 2000-2004. The correlation coefficient for offense was .409, and for defense it was .329. (Correlation coefficient is explained here for those unfamiliar with the term.)
By comparison, the correlation coefficient for offensive DVOA on the entire field was .555, and for defense it was .397.
Mark Padden: Do you plan on updating individual stats through Week 7?
Aaron: I hope this doesn't really upset anyone, but no. Something weird was going on with NFL.com and the site just posted the gamebook from Monday's Jets-Falcons game late this afternoon. So I had to manually break down the play-by-play from that game, and I saved time by not doing the players involved. That's why I haven't finished the individual stats. Since we're already at the weekend, I'm just going to wait until next week to post updates on the individual stats through Week 8.
Darren Willett: Is special teams data used equally with offense and defense in determining a team's overall efficiency? If not, how is the overall weighting calculated to determine overall team efficiency?
Aaron: Offensive and defensive DVOA use a complicated system of "success points" that measure both yardage and yardage towards a first down. Since special teams plays only involve raw yardage (or field goals), the special teams system is based on actual points, not arbitrary "success points." Then I had to figure out the right formula to turn that number into DVOA so it could be added to offense and defense.
The result is that a team's total value is roughly three parts offense, three parts defense, and one part special teams. I should note that I've made a few different adjustments to the multiplier that turns special teams points over average into DVOA, and I'm still not sure it is exactly right, so that's something that could change again in the future as the system is continuously refined.
Sean Desmond: I was wondering if you could tell me how to break down your future schedule portion of the DVOA ratings into a future "rushing" schedule and a future "passing" schedule. I think this would be a great option for fantasy players.
Aaron: Well, the easiest way to do it is to jot down the run defense and pass defense DVOA ratings for the remaining opponents of a specific team and average them. That's probably a lot of work, though, isn't it? It's work for me to add more stats to the regular reports too, but I can run this one here so you have an idea of how things stand as of Week 8. Here is the average defensive DVOA of remaining opponents for each team, split into passing and rushing. Just like the normal DVOA tables, ranks go from hardest (1) to easiest (32). Go get yourself some Jaguars and Seahawks and ditch Warrick Dunn:
|Week 1-7 Opponents||Week 8-17 Opponents|
|Team||Pass Defense||Run Defense||Pass Defense||Run Defense|
|Week 1-7 Opponents||Week 8-17 Opponents|
|Team||Pass Defense||Run Defense||Pass Defense||Run Defense|
Golly, whatever might be wrong with Kevin Jones? He's not the only one; you might notice it really isn't a very good overall year for the running game.
Just for fun, I also took a look at remaining offensive opponents vs. pass offensive opponents. I won't run the whole table, but here are a couple teams that stood out:
David Nazaret: I have a question regarding the defensive DVOA ratings vs. #1 WR. I noticed that following Week 6 the Jets were ranked 29th (+49.0%) vs. opposing #1 WRs. That seems somewhat inconsistent with the performances by opposing #1's when you check out the box scores. Isn't it possible that vs. a â€œshutdownâ€? corner like Law the QB only tends to throw when the guy is wide open? So while number the total instances vs. the shutdown CB is low, their rate of success tends to be higher, which unintentionally skews their DVOA. I look at how Law absolutely shut down Michael Clayton in Week 5 and wonder how that jives with a 49.0% DVOA. He caught zero passes that day and was only thrown to once. However, since there is only one iteration in Law's favor, the shutdown performance only has a very minimal impact on the overall DVOA calculation right? Or am I missing something?
Aaron: Obviously when we compile these rankings, we have to decide who counts as the #1 and #2 receivers. And the hardest one this year has been Clayton/Galloway. Clayton was amazing last year. But there's something wrong with him this year, we don't know what it is, and they aren't throwing to him. That's not against Law -- that's against EVERYONE. It's pretty clear Galloway was Griese's new fave receiver. So I made the decision to tag Galloway as "1" and Clayton as "2". (The other difficult choice is Keyshawn Johnson vs. Terry Glenn in Dallas.)
In addition, the Jets don't really assign Law to the opposition's best receiver. He plays one side and usually stays there. Against Jacksonville, for example, a lot of his coverage was on Reggie Williams.
Here are the Jets' stats vs. number one and number two receivers this year:
Jeff Finkelstein: Could you please explain how exactly you adjust DVOA rankings for fumble recoveries? I understand how/why recovery percentage is random, but not how it affects the rankings.
Also, I thought the fumble recovery for a TD would have significantly raised the Eagles special teams rating this week, but they only improved by about 1%. Their coverage teams were also much improved this week. Any thoughts? Thanks!
Aaron: If you've read the description of how DVOA is computed -- which is still too convoluted, and I need to re-write it and make it clearer, but I digress -- you know that DVOA starts with each play being given a value in "success points" based on how many yards it gets towards the goal line as well as the first down. Before any adjustments, turnovers are penalized -8 success points, and -9 success points when they take place between the 20-yard line and the goal line (on either side of the field).
The same is true for interceptions in the adjusted numbers, but fumbles on different types of plays are penalized different amounts based on how often that type of fumble is recovered by the offense:
These penalties are also multiplied by an additional 12.5% between the 20-yard line and the goal line. In addition, the fumble penalty is added to the positive value of the play prior to the fumble, so a fumble after a 15-yard run is worth a lot more than a fumble after a one-yard run.
Now I get to admit two flaws with the system. First, I don't do this for the length of a pass before it is intercepted, and that's something that should be added, and it will be added the next time I upgrade the system. (This is another leftover from the first two years where I was parsing a different style of play-by-play: it didn't tell you where a ball was intercepted.) Second, the "all fumbles equal" deal doesn't apply to special teams, because I haven't had time yet to separate the muffs from the fumbles on runbacks.
When you say "fumble recovery for a touchdown" I assume you mean the blocked field goal. Blocked field goals don't count in the special teams ratings because they are rare, non-predictive plays. Non-predictive does not mean random: blocked field goals require skill and timing. But one blocked field goal is not an indicator that the team will block another field goal in the future. So the blocked field goal only counts as a negative for San Diego (as a missed field goal) and not a positive for Philadelphia.
It isn't just blocked field goals, however. I don't count touchdowns on interception or fumble returns either. Once again, the length of the return on a turnover is often based on random bounces of the ball, who happens to be standing where when the ball comes out. Not to mention the fact that I don't want to be judging an offense based on its tackling ability or a defense based on whether the ball is recovered by the speedy cornerback or the big fat lineman. The turnover penalty sort of assumes an "average" return, and when I put in the adjustment to take into account the length of an intercepted pass, it will take into account the fact that a three-yard interception is more likely to be returned for a touchdown than a 45-yard interception.
These issues tie directly into today's last question:
Freddie: Falcons win and drop 6 spots to 18th. Am I the only one here that thinks this has to be too low? Obviously, the system needs some fine tuning. A 5-2 team worse than 2-4 team. And one of those loses was without Vick. This system is just not a good predictor of how good a team is.
Aaron: This comment appeared in the DVOA power rankings discussion. When I responded three comments later, I was somewhat flippant. Let me see if I can explain this one better, although I don't think it is going to make the Atlanta fans any happier.
Atlanta's DVOA for the Monday night game was -31.6%. In fact, according to DVOA, it was the worst game Atlanta has played all season.
First of all, remember, the Jets are ranked 28th. So the rating was penalized by opponent adjustments. You don't need DVOA to know that beating the Jets right now does not exactly qualify as "impressive."
Second, let's look at each of the turnovers and the subsequent Atlanta drives:
At this point, the Falcons are leading 17-0 and the offense has really none nothing impressive. The defense has made some big plays that turned into colossally big plays thanks to some luck regarding the bounce of the ball. They have done this, I repeat, against the 28th best team in the league. What have the Falcons done in this 20-minute span to convince us that they are a good team? They look like a mediocre team with a very good pass rush.
And then, for the rest of the game, the Jets outscored the Falcons.
That's two straight weeks where the Falcons have played a mediocre game against one of the worst teams in the league and won thanks to a couple of lucky breaks and bounces. It was just less obvious against the Jets because those breaks and bounces all came early and because most people don't understand that fumble recoveries and defensive touchdowns are mostly random even though the sacks that lead to them are not.
Before these two games, the Falcons were a 3-2 team ranked 10th in DVOA. Frankly, that ranking made sense, and so does this one. Most fans just don't want to accept the role of random chance in determining winners and losers in individual ballgames.
24 comments, Last at 02 Nov 2005, 4:22pm by Andrew (A.B.)