Looking back at FEI's preseason projections, we find that most teams did about what they were supposed to do -- but not in the Big Ten, where things got screwy.
25 Nov 2005
by Aaron Schatz (and the FO staff)
Time for another look at the Football Outsiders mailbag. We get a lot of e-mail, and there are a lot of comments on the discussion threads, so I apologize if your question doesn't get answered. There simply are too many good questions that require well thought out answers. The best way to get your question answered at this point is to use the contact form. If it is a question not related to the DVOA stats, it is more likely to be answered if you send it to one of the other writers, not me.
Be aware that we reference plenty of our innovative FO stats here, not to mention their unfamiliar terminology, so if you are a recent addition to the readership you might want to read this first.
Flux: I suppose this is the question you'll get the most of from non-FO readers, but how does Steve Smith catch 14 passes for 169 yards against the Bears' pass defense and not make the top five? I didn't see the game, but did Carolina throw him like 25 passes, or what? (I'm in the Bay Area; we get nothing but Raiders and 49ers every week. Yes, it's hell.)
Aaron Schatz: Yep, this question regards this week's Quick Reads over at FOXSports.com, the column which gives the top DPAR ratings for each week. Despite the big numbers, Smith's performance this week comes out exactly at replacement level before applying the opponent adjustment, 2.1 DPAR after applying the opponent adjustment. He was thrown 20 passes: six went incomplete, and another seven went for less than 10 yards without achieving a first down. He also fumbled once.
Brian: How is the Colts' variance so low given their seemingly wildly different games this season? I suppose since the defense was racking up more numbers early and the offense is racking up more numbers lately the total DVOA for each game was still pretty even?
Aaron Schatz: Actually, the Colts are very consistent on offense (30th in variance) and middle of the pack on defense (17th in variance). That early-season "the Colts offense isn't scoring" theme was just overreaction to two games, the 10-3 win over Jacksonville and the 13-6 win over Cleveland. Jacksonville is one of this season's top pass defenses, and the Colts' offense wasn't that bad against Cleveland. They just kept getting stopped in the red zone: Vanderjagt had field goals of 20 and 23 yards, and they ended the game with Manning kneeling on first-and-goal at the Cleveland six-yard line. Since these two games, the Colts have scored at least 28 points every week.
But thanks for the opportunity to break out my favorite little toy for the first time this season -- the week-to-week DVOA graph. I get a perverse joy out of figuring out how to make these things match team colors. Blue here is the Indianapolis offense, black is the defense, and the curved lines represent the trend for each unit. The slight uptick in the trend on offense is really just one mediocre game (San Francisco) and one very good game (Cincinnati) but the trend on defense is fairly clear. Since DVOA represents scoring, that trend shows how the Colts' defense has gradually been giving up more yards and points since taking on the Alex Smith Experience.
Bobman: Seriously, what lawn work do you have in mid-November?
Aaron Schatz: Fall temperatures in New England have been astonishingly mild this year, and the trees in my yard didn't start turning until three weeks ago. The ones in the backyard are particularly annoying because they have lots of little leaves. Scientific ramifications of this can be debated on another website, thanks.
Patrick Leming: I would appreciate an in-depth look at why you were wrong about the Eagles this year. We've seen why you were wrong about the Falcons (at least most of us appreciated that) and why you were right about the Packers. Any detailed look at the Eagles would help brighten up these dark days. They were clearly the team the projections were most wrong on. Do the Eagles have any hope for next year? I'd suspect third-down performance on both sides is a telling indicator. Didn't they defy a trend with their third down defense the past couple of years?
Aaron Schatz: As far as I'm concerned, there was no way to foresee the collapse of the Philadelphia Eagles. I think this team was just hit with a ton of unpredictable problems, all at the same time. It was reasonable to expect one of the other NFC East teams to improve this year, but nobody expected significant improvement from all three. Personality conflicts and contract squabbles like the T.O. situation often completely disappear once the season starts. This time, that didn't happen. There was no indicator that the special teams would collapse. Kicker injuries are not particularly common, so there was no way to predict injuries to both David Akers and Dirk Johnson. Donovan McNabb struggled all year with the injury from the Atlanta game, something we could not predict before the season began. They were 4-2 a few weeks ago, but the Denver blowout was a real turning point, especially since it was T.O.'s last game with the team.
Philadelphia DVOA by Down, 2004-05
As far as down-by-down performance, there's no indicator there. If anything, we would have expected the offense to improve this year. When Mike Tanier and I did two Philadelphia-area book signings, however, we both agreed that this team had one issue that they forgot to address in the off-season.
Mike Tanier: The lack of a "big" or even "medium" sized back is significant. We can see it in the red zone this year. Against the Giants, the Eagles couldn't punch the ball in from the one-yard line. Against the Cowboys they had to settle for a short field goal late in the game. Those are difference making plays.
The Eagles didn't need to go sign Jerome Bettis or anything, but the fact that they didn't grab anyone, even Dorsey Levens, is puzzling.
In terms of the future, I am happy to see this new center Jamaal Jackson playing well. I am glad rookie defensive end Trent Cole is playing well. Reggie Brown has a great opportunity right now. The salary cap situation is great. If McNabb didn't throw that pass against Dallas, we could be 5-5 or 6-4 right now. No reason to lament the future. Just the present.
NF: I was just reviewing your article on special teams VOA, and I was wondering if you have considered adding points scored above or below average by opposing field goal kickers to the indicators you use for why teams have bad years, along with third-down performance and fumble recovery rate.
Also, have you expanded the data for average points scored from a yardage by field goals by including data from more years? Theoretically, because there has not really been a great change in the technique of field goal kicking or teams strategy on it in the last 10 years, you could pool data from every season going back at least that far to find average kick success rates.
Aaron Schatz: You'll see a rating on the special teams page called "VOA w/HIDDEN" which includes field goals against and kickoff distance against.
This year, the teams with the worst luck due to opposing field goal kickers have been Dallas (+11.7 points) and Houston (+11.4 points). The only field goals missed against Dallas have been a 50-yarder and a 60-yarder. Nobody has missed a field goal against Miami, though Phil Dawson missed an extra point.
The luckiest team, by far, has been Kansas City (-8.9 points). Rian Lindell of Buffalo missed 28- and 31-yard field goals against them, Jason Elam missed a 46-yarder in Denver, and Sebastian Janikowski missed a 50-yarder. They've also had opponents abort a field goal and an extra point with bad snaps and they've blocked two field goals for good measure (though that's not really luck).
As far as opposition kickoff distance, the numbers are much less extreme. The one team that stands out is St. Louis. Kickoffs against the Rams have averaged 68.1 gross yards, and no other team is above 67. Even after adjusting for the dome, this has cost the Rams about 6.2 points worth of field position compared to the NFL average.
To your second question, there has actually been a change in league-wide kicking numbers (kickoff, punt, and field goal) twice:
Each time, kicking became harder. Of course, kickers have improved significantly again this year, so who knows, maybe in 2006 they'll make them kick cannonballs or something.
The next question asks about Drive Success Rate, a stat we introduced in Pro Football Prospectus 2005, created by Jim Armstrong and featured on the Drive Stats pages.
Jon Lewallen: I had a question about the definition of drive stats. The site defines Drive Success Rate as the percentage of down series that result in a first down or TD. Does this exclude, then, field goals to win the game? Obviously there's no way to know if it's the "winning field goal" until the clock runs out, but I would think that a team down by one or two kicking a field goal with, say, ten seconds left has made a successful drive.
Jim Armstrong: DSR considers all down series that end in a field goal as an unsuccessful down series, even if it is a "game winning" FG. My reasoning is that most field goals, even game winning ones, occur on 4th down following three downs in that series where the offense was making a legitimate attempt to score a touchdown or, at the very least, gain another first down (which would make for a shorter FG). There may indeed be times when this isn't true, for example when a FG is attempted on first down or is immediately preceded by kneeldown plays to run down the clock. I would guess these are so uncommon that they probably don't have a significant effect on any team's overall DSR.
Of course, DSR doesn't really define whether an entire drive is considered successful, but to me, the successful part of a FG drive consists of those first downs that moved a team into FG range (or within at most nine yards). So a drive that gains three first downs, then attempts a FG would get a DSR of 0.75 (3/4), which is still pretty successful, if not optimal. The drawback of this method is that the offense might not get any credit at all for moving up to nine yards closer for, say, a 39-yard FG attempt vs. a 48-yarder.
Lastly, DSR is really only intended to measure offensive (or defensive) efficiency. Whether a FG attempt is successful or not depends on special teams, particuarly the FG kicker. So in that sense I think it's consistent that DSR rewards the offense for setting up the FG attempt, but not the FG itself, game winning or otherwise.
Dennis Flanagan: Experts are now comparing the 2005 Bears Defense with the 1985 defense, citing the fact that through 11 games they have allowed fewer yards/game and points/game. Using your methodology which is the better defense?
Ned Macey: We not only do not have play-by-play data for 1985, but as far as I know, we do not even have box scores for the 1985 season easily accessible on the Web. So, it is a little difficult to evaluate a defense in these terms. I took a quick look at how each each team's points allowed stacked up based on their opponents played. For the season, the 1985 Bears held opponents to 56.4% of the points they averaged against all other teams. So far, the 2005 Bears have held opponents to 58.7% of their normal point totals. (In case you care, 2000 Ravens were at 56.6%.) So, while the 2005 Bears are allowing fewer per game, they are giving up a slightly higher percentage based on their opponents. Nonetheless, it is very close, and the 1985 Bears had a better offense that could help with field position and limit the number of opposition drives.
|2005 Bears (so far)||11.0||252.1||56.4%||5.0||3.3|
By DVOA standards, the 2005 Bears are truly impressive. If the season ended today, the 2005 Bears would have the best defensive DVOA for the years we have calculated (1998-2005): -38.9%. Using the new second-order opponent adjustments, the 2000 Ravens had -36.7% defensive DVOA, while the 2002 Bucs had -32.4% defensive DVOA.
Of course, we still have over one-third of the season left, and I highly doubt the Bears can maintain this extraordinary level of play.
Aaron Schatz: I actually wrote a bit about the 2005 Bears in today's New York Sun, and I may be writing more about them in Tuesday's power rankings commentary.
Charles Jake: I hate the "line up and try to draw the D offsides" strategy. I was wondering if you could look at all fourth-and-short situations where the O doesn't run an actual play and determine how many are successful (defensive penalty for first down) vs. unsuccessful (delay of game or offensive timeout followed by punt). I know this isn't perfect because there will be situations where someone is penalized or calls timeout when they meant to run a play, but I think it would be cool to get some idea of how often this actually works.
Aaron Schatz: This sounds like a fun research project for the book, but I decided to take a quick look at 2005 play-by-play through Week 11. As you say, it is hard to tell when this strategy is unsuccessful. But we can figure out how often it works: hardly ever. Through 11 weeks of the 2005 season, there have been a grand total of five defensive penalties on fourth down with 1-3 yards to go. One of these penalties is listed as "punt formation," another is pass interference, and a third is an offside that cancels an actual play. That leaves two possibilities, both listed as "run formation" in the play-by-play:
I would guess that the penalty on Haynesworth was a real play, while the penalty on Payne was a "try to draw the other team offsides" strategy, but I can't be sure without asking someone who has tape of these games.
By comparison, there have been 27 offensive penalties on fourth down with 1-3 yards to go, though a handful of those penalties are listed in field goal or punt formations.
Stewart Glickman: I'm wondering how the Chiefs' offensive DVOA looks with Willie Roaf vs. without Willie Roaf. How about when he missed time for the Saints in 2001?
Aaron Schatz: Willie Roaf has missed six games this season: Weeks 2-5 and Weeks 9-11. In 2001 with the Saints, he missed nine games: Weeks 7-8 and Weeks 11-17. The numbers definitely support the idea that Roaf is a great left tackle, but it looks like he has a much bigger effect on pass protection than he does on run blocking.
|2005 Chiefs DVOA||2001 Saints DVOA|
|with Willie Roaf||43.1%||20.5%||3.5%||-7.0%|
|without Willie Roaf||-0.2%||13.5%||-25.3%||-15.4%|
Shannon Russell: Are defensive adjustments calculated 'as-of' the date of the game, or renormalized from games that occur after the two teams have played?
Aaron Schatz: They are renormalized each week during the season. There has been some thought to having some sort of "rolling opponent adjustment" that might take into account the way teams improve or decline during the year, but we haven't had a chance to test something like that yet.
40 comments, Last at 30 Nov 2005, 11:57am by doyle