Writers of Pro Football Prospectus 2008

28 Oct 2005

FO Mailbag

by Aaron Schatz

Time for another look at the Football Outsiders mailbag. Most of these questions came via the contact form which you can use to e-mail any of the writers. (Once again, this mailbag is all questions for me about the DVOA ratings, but I do mean to include questions to other writers in the future.) And while we will answer questions posed in discussion threads, we are much more likely to answer a question asked through the contact form. When those discussions get filled up with comments, they get hard to follow.

Be aware that we reference plenty of our innovative FO stats here, not to mention their unfamiliar terminology, so if you are a recent addition to the readership you might want to read this first.

By the way, no funny hate mail this week. There was less of it, but it was all just annoying.

Let's start with a question that has literally been sitting around in my e-mail for months. It didn't take long to answer, either, so I'm an idiot.

John Grenci: Aaron, you probably have stated this somewhere before, but is there a correlation of red zone efficiency from year to year? What about defense?

Aaron: I assume you are asking about DVOA red zone numbers, not the league's red zone efficiency numbers. I checked the correlation of red zone DVOA from year-to-year during the period 2000-2004. The correlation coefficient for offense was .409, and for defense it was .329. (Correlation coefficient is explained here for those unfamiliar with the term.)

By comparison, the correlation coefficient for offensive DVOA on the entire field was .555, and for defense it was .397.

Mark Padden: Do you plan on updating individual stats through Week 7?

Aaron: I hope this doesn't really upset anyone, but no. Something weird was going on with NFL.com and the site just posted the gamebook from Monday's Jets-Falcons game late this afternoon. So I had to manually break down the play-by-play from that game, and I saved time by not doing the players involved. That's why I haven't finished the individual stats. Since we're already at the weekend, I'm just going to wait until next week to post updates on the individual stats through Week 8.

Darren Willett: Is special teams data used equally with offense and defense in determining a team's overall efficiency? If not, how is the overall weighting calculated to determine overall team efficiency?

Aaron: Offensive and defensive DVOA use a complicated system of "success points" that measure both yardage and yardage towards a first down. Since special teams plays only involve raw yardage (or field goals), the special teams system is based on actual points, not arbitrary "success points." Then I had to figure out the right formula to turn that number into DVOA so it could be added to offense and defense.

The result is that a team's total value is roughly three parts offense, three parts defense, and one part special teams. I should note that I've made a few different adjustments to the multiplier that turns special teams points over average into DVOA, and I'm still not sure it is exactly right, so that's something that could change again in the future as the system is continuously refined.

Sean Desmond: I was wondering if you could tell me how to break down your future schedule portion of the DVOA ratings into a future "rushing" schedule and a future "passing" schedule. I think this would be a great option for fantasy players.

Aaron: Well, the easiest way to do it is to jot down the run defense and pass defense DVOA ratings for the remaining opponents of a specific team and average them. That's probably a lot of work, though, isn't it? It's work for me to add more stats to the regular reports too, but I can run this one here so you have an idea of how things stand as of Week 8. Here is the average defensive DVOA of remaining opponents for each team, split into passing and rushing. Just like the normal DVOA tables, ranks go from hardest (1) to easiest (32). Go get yourself some Jaguars and Seahawks and ditch Warrick Dunn:

  Week 1-7 Opponents Week 8-17 Opponents
Team Pass Defense Run Defense Pass Defense Run Defense
ARI 22.5% 32 -4.4% 11 -2.7% 11 -0.4% 23
ATL 2.5% 20 3.1% 27 -5.8% 7 -15.9% 1
BAL -14.2% 2 -1.6% 18 -3.7% 10 1.3% 28
BUF 10.5% 27 0.3% 21 5.7% 23 -6.3% 8
CAR 8.7% 26 -1.9% 16 -9.9% 2 -0.1% 24
CHI -9.7% 4 1.3% 24 6.3% 27 -2.8% 15
CIN 0.7% 16 -2.9% 13 -6.1% 6 -2.4% 18
CLE -18.5% 1 -1.7% 17 2.3% 17 -5.3% 12
DAL 11.6% 29 -4.8% 10 0.5% 14 -8.3% 4
DEN 3.3% 21 -10.3% 2 -1.9% 12 -3.6% 14
DET -1.9% 12 -12.0% 1 -4.5% 9 1.0% 27
GB -4.1% 11 -4.9% 9 -13.4% 1 -6.2% 9
HOU -9.6% 5 0.7% 23 1.0% 15 -2.6% 17
IND 17.7% 31 4.1% 30 3.8% 21 -0.5% 22
JAC -13.9% 3 -1.9% 15 21.8% 32 6.3% 31
KC 0.7% 17 -10.0% 3 3.5% 20 4.8% 30
  Week 1-7 Opponents Week 8-17 Opponents
Team Pass Defense Run Defense Pass Defense Run Defense
MIA -6.5% 8 -9.0% 5 11.5% 30 6.4% 32
MIN -4.9% 10 -0.7% 20 -4.7% 8 -5.7% 11
NE 1.4% 18 -6.3% 8 -7.8% 4 -2.8% 16
NO 7.9% 25 3.9% 28 -7.0% 5 -11.7% 2
NYG 1.6% 19 1.7% 26 5.5% 22 -8.5% 3
NYJ -7.0% 7 -8.5% 6 10.8% 29 -2.4% 19
OAK -1.8% 13 -3.3% 12 5.9% 26 -7.4% 5
PHI 11.1% 28 -1.0% 19 5.8% 25 -3.7% 13
PIT 5.7% 24 4.0% 29 -8.2% 3 0.3% 25
SD 4.3% 23 -7.3% 7 -0.7% 13 -7.2% 6
SEA -0.2% 14 4.4% 31 18.0% 31 1.4% 29
SF -8.4% 6 -2.5% 14 5.8% 24 -6.1% 10
STL 12.4% 30 1.5% 25 2.8% 18 -1.1% 21
TB -5.9% 9 0.6% 22 7.3% 28 0.5% 26
TEN 0.2% 15 4.5% 32 3.3% 19 -1.6% 20
WAS 3.8% 22 -10.0% 4 1.9% 16 -6.5% 7

Golly, whatever might be wrong with Kevin Jones? He's not the only one; you might notice it really isn't a very good overall year for the running game.

Just for fun, I also took a look at remaining offensive opponents vs. pass offensive opponents. I won't run the whole table, but here are a couple teams that stood out:

  • Indianapolis has faced the easiest average offense in Weeks 1-7 (-15.8%) but ranks seventh in average offense left on the schedule (5.4%).
  • Buffalo's defense is also going to see things get a lot tougher, going from -5.9% in Weeks 1-7 to 7.3% in Weeks 8-17.
  • Baltimore is screwed. If the 46 couldn't look good against an average offense of -9.3%, how will it look against an average offense of 3.9% the rest of the way?
  • The league's worst defense, the Houston Texans, at least gets an easy schedule the rest of the way. So far, their opponents have averaged 11.0% on offense. The rest of the way, opponents average -6.6%.
  • Think the Pats are ready to start that division schedule? They go from an average offensive opponent of 11.2% in Weeks 1-7 to an average offensive opponent of -7.3% in Weeks 8-17.
  • Finally, the Jaguars. As you see above, the Jacksonville offense has the easiest remaining schedule. So does the Jacksonville defense. By a ton. Over the final ten weeks, Jacksonville gets to play ten offenses averaging -13.7% instead of six offenses averaging 12.6%.

David Nazaret: I have a question regarding the defensive DVOA ratings vs. #1 WR. I noticed that following Week 6 the Jets were ranked 29th (+49.0%) vs. opposing #1 WRs. That seems somewhat inconsistent with the performances by opposing #1's when you check out the box scores. Isn't it possible that vs. a “shutdown� corner like Law the QB only tends to throw when the guy is wide open? So while number the total instances vs. the shutdown CB is low, their rate of success tends to be higher, which unintentionally skews their DVOA. I look at how Law absolutely shut down Michael Clayton in Week 5 and wonder how that jives with a 49.0% DVOA. He caught zero passes that day and was only thrown to once. However, since there is only one iteration in Law's favor, the shutdown performance only has a very minimal impact on the overall DVOA calculation right? Or am I missing something?

Aaron: Obviously when we compile these rankings, we have to decide who counts as the #1 and #2 receivers. And the hardest one this year has been Clayton/Galloway. Clayton was amazing last year. But there's something wrong with him this year, we don't know what it is, and they aren't throwing to him. That's not against Law -- that's against EVERYONE. It's pretty clear Galloway was Griese's new fave receiver. So I made the decision to tag Galloway as "1" and Clayton as "2". (The other difficult choice is Keyshawn Johnson vs. Terry Glenn in Dallas.)

In addition, the Jets don't really assign Law to the opposition's best receiver. He plays one side and usually stays there. Against Jacksonville, for example, a lot of his coverage was on Reggie Williams.

Here are the Jets' stats vs. number one and number two receivers this year:

Player Catch Passes Yards
87-E.Kennison 4 5 76
84-C.Chambers 3 7 21
82-J.Smith 2 4 41
85-D.Mason 5 6 54
84-J.Galloway 5 10 87
80-E.Moulds 7 9 63
12-M.Jenkins 1 4 8
Player Catch Passes Yards
18-S.Parker 2 3 23
86-M.Booker 1 5 21
11-Re.Williams 5 6 81
89-Ma.Clayton 1 4 4
80-Mi.Clayton 0 1 0
83-L.Evans 3 6 22
84-R.White 2 3 20

Jeff Finkelstein: Could you please explain how exactly you adjust DVOA rankings for fumble recoveries? I understand how/why recovery percentage is random, but not how it affects the rankings.

Also, I thought the fumble recovery for a TD would have significantly raised the Eagles special teams rating this week, but they only improved by about 1%. Their coverage teams were also much improved this week. Any thoughts? Thanks!

Aaron: If you've read the description of how DVOA is computed -- which is still too convoluted, and I need to re-write it and make it clearer, but I digress -- you know that DVOA starts with each play being given a value in "success points" based on how many yards it gets towards the goal line as well as the first down. Before any adjustments, turnovers are penalized -8 success points, and -9 success points when they take place between the 20-yard line and the goal line (on either side of the field).

The same is true for interceptions in the adjusted numbers, but fumbles on different types of plays are penalized different amounts based on how often that type of fumble is recovered by the offense:

  • Aborted snap: -2.15
  • Sack: -3.92
  • Rushing play: -4.97
  • Reception behind line of scrimmage: -3.50
  • Reception, 1-4 yards: -4.55
  • Reception, 5-8 yards: -4.80
  • Reception, 9-20 yards: -4.90
  • Reception, 21+ yards: -6.54

These penalties are also multiplied by an additional 12.5% between the 20-yard line and the goal line. In addition, the fumble penalty is added to the positive value of the play prior to the fumble, so a fumble after a 15-yard run is worth a lot more than a fumble after a one-yard run.

Now I get to admit two flaws with the system. First, I don't do this for the length of a pass before it is intercepted, and that's something that should be added, and it will be added the next time I upgrade the system. (This is another leftover from the first two years where I was parsing a different style of play-by-play: it didn't tell you where a ball was intercepted.) Second, the "all fumbles equal" deal doesn't apply to special teams, because I haven't had time yet to separate the muffs from the fumbles on runbacks.

When you say "fumble recovery for a touchdown" I assume you mean the blocked field goal. Blocked field goals don't count in the special teams ratings because they are rare, non-predictive plays. Non-predictive does not mean random: blocked field goals require skill and timing. But one blocked field goal is not an indicator that the team will block another field goal in the future. So the blocked field goal only counts as a negative for San Diego (as a missed field goal) and not a positive for Philadelphia.

It isn't just blocked field goals, however. I don't count touchdowns on interception or fumble returns either. Once again, the length of the return on a turnover is often based on random bounces of the ball, who happens to be standing where when the ball comes out. Not to mention the fact that I don't want to be judging an offense based on its tackling ability or a defense based on whether the ball is recovered by the speedy cornerback or the big fat lineman. The turnover penalty sort of assumes an "average" return, and when I put in the adjustment to take into account the length of an intercepted pass, it will take into account the fact that a three-yard interception is more likely to be returned for a touchdown than a 45-yard interception.

These issues tie directly into today's last question:

Freddie: Falcons win and drop 6 spots to 18th. Am I the only one here that thinks this has to be too low? Obviously, the system needs some fine tuning. A 5-2 team worse than 2-4 team. And one of those loses was without Vick. This system is just not a good predictor of how good a team is.

Aaron: This comment appeared in the DVOA power rankings discussion. When I responded three comments later, I was somewhat flippant. Let me see if I can explain this one better, although I don't think it is going to make the Atlanta fans any happier.

Atlanta's DVOA for the Monday night game was -31.6%. In fact, according to DVOA, it was the worst game Atlanta has played all season.

First of all, remember, the Jets are ranked 28th. So the rating was penalized by opponent adjustments. You don't need DVOA to know that beating the Jets right now does not exactly qualify as "impressive."

Second, let's look at each of the turnovers and the subsequent Atlanta drives:

  • Vinny Testaverde fumbles an aborted snap. One out of four aborted snaps is recovered by the defense, but this was that one out of four. The offense only needed to drive 20 yards for the touchdown.
  • Vinny Testaverde is sacked and fumbles. One out of two fumbles after a sack is recovered by the defense, and this was that one out of two. The Falcons offense managed to go 21 yards but stalled on the five-yard line and kicked an easy field goal.
  • Two drives later, Vinny Testaverde is sacked again and fumbles again. This time Atlanta returns the fumble for a touchdown. But remember, the touchdown return is a non-predictive event that is not included in DVOA.

At this point, the Falcons are leading 17-0 and the offense has really none nothing impressive. The defense has made some big plays that turned into colossally big plays thanks to some luck regarding the bounce of the ball. They have done this, I repeat, against the 28th best team in the league. What have the Falcons done in this 20-minute span to convince us that they are a good team? They look like a mediocre team with a very good pass rush.

And then, for the rest of the game, the Jets outscored the Falcons.

That's two straight weeks where the Falcons have played a mediocre game against one of the worst teams in the league and won thanks to a couple of lucky breaks and bounces. It was just less obvious against the Jets because those breaks and bounces all came early and because most people don't understand that fumble recoveries and defensive touchdowns are mostly random even though the sacks that lead to them are not.

Before these two games, the Falcons were a 3-2 team ranked 10th in DVOA. Frankly, that ranking made sense, and so does this one. Most fans just don't want to accept the role of random chance in determining winners and losers in individual ballgames.

Posted by: Aaron Schatz on 28 Oct 2005

24 comments, Last at 02 Nov 2005, 4:22pm by Andrew (A.B.)


by Pat (not verified) :: Fri, 10/28/2005 - 7:45pm

Yah, I don't know why a Falcons fan would want to hang their hat on that victory. I teased the Falcons fan in my building afterwards and even he said he felt guilty about that win.

Literally, the Jets just handed the Falcons three easy touchdowns (from which they could only capture two), and the Jets very nearly came back.

by Pat (not verified) :: Fri, 10/28/2005 - 7:47pm

Also, while the Eagles coverage units were better, the kickoffs still were poor, and the punt returns and kickoff returns were mediocre as well.

We guessed that the special teams DVOA for that week was probably around ~0%, and that made sense to me. Definitely a far cry from last year for them.

by RowdyRoddyPiper (not verified) :: Fri, 10/28/2005 - 8:15pm

I don't think you're being glib, Matt. Now allow me to jump on this couch and profess my love for Katie Holmes.

If people honestly don't want to read the disclaimers and understand the numbers, I don't see how you're obligated to answer their flames. They might as well complain about the color scheme or fonts on the site. Keep up the good work.

I do take exception with one item. Kevin Jones will not see a miraculous rebound in his numbers because the Run D's he faces will go from 1st to 27th ranked per DVOA. He will rebound because I traded him today. All of those who purchased PFP and drafted K. Jones can thank me by donating to a charity of your choice in remembrance of my fantasy season.

by Tim Kirk (York, UK) (not verified) :: Fri, 10/28/2005 - 8:44pm

Sorry to mention the 'D' word, but I find it strange to see that Denver have played the second best run defences so far this season... This implies that the current two-headed backfield might continue to make serious yards the rest of the season (and remain a conundrum for fantasy players...).

Of course having played the easier pass defencesand not done much they'll need the running (and the play-action that will likely bring) to work to hold off KC & SD for the division, but it looks at least possible from that breakdown.

by Jay B. (not verified) :: Sat, 10/29/2005 - 12:32am

Regarding the first answered question, I think it might be more interesting to look at year-to-year correlations for red-zone DVOA vs. overall DVOA. In other words, are some teams consistently better or worse in the red zone, offensively or defensively, than they are over the rest of the field?

For instance, the Eagles' defense last year was fairly average in overall DVOA, but were fanstastic in the red zone, which was one of the reasons they were among the league leaders in fewest points allowed. Is this something that carries over year to year?

by Titus (not verified) :: Sat, 10/29/2005 - 10:30am

Let me say that I'm extremely happy there's no "funny" hate mail this week. You mean you guys get profanity-laced, poorly written e-mail messages attacking you? Wow! No kiddin'.

It's not funny and it's not surprising. So how about not subjecting us to it?

by Kevin (not verified) :: Sat, 10/29/2005 - 12:09pm

I found them funny. So did many commenters in previous weeks. What, praytell, makes your opinion more important than ours?

by Pat (not verified) :: Sat, 10/29/2005 - 12:56pm

That's another vote from me for the funny hatemail. Just to put Titus in the minority for those who have voiced opinions. :)

by jebmak (not verified) :: Sat, 10/29/2005 - 1:09pm

Seriously, that hate mail was hilarious.
On a completely different note, I hope that the toughest rushing schedule remaining really shuts Atlanta down and that Vick stays healthy. Then the media could focus on important things, like the Manning on Manning Super Bowl!

by Andrew (not verified) :: Sat, 10/29/2005 - 4:26pm

Jay B. #5:

The Eagles have been consistently among the best teams since 2000 in preventing scoring from the red zone, which is why they lead the league since 2000 with the fewest points allowed over that span of time. They are much better by DVOA in the red zone than out of it.

by Andrew (not verified) :: Sat, 10/29/2005 - 4:29pm


Pretty, pretty, please could we have the DPAR and DVOA of Vick, Testaverde, and Bollinger from the Monday Night Fiasco? Did they possibly give Alex Smith a run for his money?

I'm aware that this will require a warning label of "Caution, Simulated Football Player-Like Substance, Handle With Extreme Care".

by Basilicus (not verified) :: Sat, 10/29/2005 - 6:58pm

I don't think that Atlanta comment was terribly flame-like. It was challenging, and perhaps could have been worded more politely at the end, but overall he was asking a question. I was going to write about how the Falcons might be the best team in the NFC, but then I looked at their schedule and realized that since that week 1 victory of Philly, they haven't really had a victory that one could call impressive. (Beating Minnesota 30-10 doesn't count.) The most impressive game they had was losing to New England 31-28. That said, the rest of their schedule isn't killer as long as their defense holds together, and it's not as if any particular NFC team is playing dominantly.

Also, keep the flamey comment of the week. They're not always funny, but Aaron's responses usually are.

In addition, can we please not use the phrase "Manning on Manning" to describe anything, Super Bowls or otherwise? It just conjures up a really bad image.

by SJM (not verified) :: Sat, 10/29/2005 - 9:18pm

How about "Manning-a-Manning"?

The breakdown of first half/second half schedule by offense and defense was very interesting. I still don't buy a Kevin Jones turnaround, since he wasn't good against Cleveland last week, and they're pretty poor against the run. But maybe that's just because I still have him on my team.

by SJM (not verified) :: Sat, 10/29/2005 - 9:20pm

Oops, didn't realize Simmons beat me to it.

by Smeghead (not verified) :: Sun, 10/30/2005 - 1:51pm

Can we call the NYG-Dallas divisonal games "Manning-on-'Boys"?

It's just a pity there are no canine team names; we could designate their games against either brother the Rick Santorum Bowl.

by White Rose Duelist (not verified) :: Mon, 10/31/2005 - 6:03pm

What was the fan who stole the ball from Favre's DPAR?

by Andrew (A.B.) (not verified) :: Mon, 10/31/2005 - 11:51pm

Not sure if I buy the notion that defensive and blocked kick TDs are "nonpredictive". Certainly they're rare and happen so infrequently that you can't make correlations for the same team and the same season. But performance is performance, and if a defense returns a fumble for a TD when the "average" fumble may or may not be recovered at all, that sounds like a substantial value over average.

by Kibbles (not verified) :: Tue, 11/01/2005 - 6:12am

Re #17: I imagine it's a lot like historical plays that wind up getting names with Capital Letters. For example, we have the Music City Miracle. Yes, it was a fantastic football play, but does anyone in their right mind think that it's predictive of future performance? Is Tennessee simply better than the rest of the league at returning miracle multi-lateral kick TDs with no time remaining on the clock? Should their rating have been adjusted upwards because of it, because if the other team ever scored to go up by less than a TD with under 5 seconds remaining they'd still be in it?

Same with the crazy multi-lateral last-second Saints TD from 2003 (where Carney missed the tying extra point). Sure, it was a ridiculous play. Absolutely, the points counted. But should New Orleans see its DVOA rise substantially from that play? Does anyone really think that that play is predictive of future performance?

Another one, actually dealing with blocked kicks. Carolina opened up 2003 blocking a ridiculous number of kicks. In fact, they beat Tampa Bay in a game that was only in overtime because they blocked a FG *and* an XP in week 2 or 3. But for all the kicks they blocked in the first four weeks (and there were a lot of them), how many kicks did they block for the 12 remaining regular season games, or the 4 playoff games? I'm pretty sure not a single one. So if Aaron had ramped up their DVOA after 4 weeks because of the feast of blocks, he would have looked pretty silly for the rest of the season when they all stopped again. Pretty silly indeed.

by Starshatterer (not verified) :: Tue, 11/01/2005 - 8:12am

Kibbles (#18 )--

Carolina partially blocked Vinatieri's second FG attempt in Superbowl 38.

by Nelphonious of Pennefielde (not verified) :: Tue, 11/01/2005 - 1:21pm

Fo total effeciency versus the Jags uncanny knack of screwing up key first downs in the red zone= 9 and 7 again,REGARDLESS OF "ESTIMATED" WINS!

by Andrew (A.B.) (not verified) :: Tue, 11/01/2005 - 3:35pm

Re: #18

Look it at the other way. Are all of those examples [i]completely[/i] random? Surely there's some talent involved (or good coaching, or whatever).

And if they're not 100% random, then they need to be reflected in DVOA somehow (though not necessarily with a normal weight).

by B (not verified) :: Tue, 11/01/2005 - 5:17pm

RE 18/21: Wasn't the Music City Miracle only one lateral?
Anyways, there are definatly coaching/good talent involved in these types of plays, but they are still non-predictive. These types of plays can only be used in very specific situations, and if those situations don't occur in a praticular game, using them to change a team's rating will hurt your predictive powers. If you somehow knew that a Titans game will come down to a last minute kick-return, then you should give them a bonus to thier chances of winning.

by Trogdor (not verified) :: Wed, 11/02/2005 - 11:13am

Actually, it was only one lateral, from whoever picked it up to Wycheck. Then there was a forward pass from Wycheck to Dyson.

by Andrew (A.B.) (not verified) :: Wed, 11/02/2005 - 4:22pm

Re: 22

The problem with that logic is that DVOA already counts plays that only occur in very specific situationn.

Strong production (over average) when the team is down by 36 late in the game, playing backups vs. backups, is given the same weight as a team down by 3 late in the game.

Performance when a team is a hurry-up offense with less than a minute to play in the 2d quarter is weighted the same as an equivalent play with 14 minutes left in the 2d quarter. (For that matter, there may be three times as many plays in the last two minutes of either half as there are in any other two minute span, so you're overweighting these plays even more.)