How much do we tend to know after five weeks? Bill Connelly compares five-week data to full-season data to find out if we should be worried about TCU and Baylor.
14 Nov 2005
by Aaron Schatz (and the FO staff)
Time for another look at the Football Outsiders mailbag. Actually, a week ago Friday was time for another look at the Football Outsiders mailbag. That become last Friday, which became Saturday, which became today. Um, whoops?
The fact is that we've reached a tipping point in the last couple of weeks where the amount of e-mail and questions in discussion threads is so large that I simply cannot answer it all, or even most of it. Things have been steadily backed up in my e-mail and on my to do list since the start of the season. I lost two and a half weeks of work time in September and I've been trying to catch up ever since, and at the same time the readership has been growing each week, and so has the e-mail volume and the need to keep the discussion threads clear of trolls.
Anyway, if you send in an e-mail or leave a question in a thread and it doesn't get answered, I apologize. I'm trying to get to as much as possible. Remember when I asked for interns to help me out? I'm so backed up with things to do that I haven't had the time to choose interns who would be able to help me with the fact that I'm so backed up with things to do.
Anyway, enough apologies. The best way to get your question answered at this point, if it is a question not related to the DVOA stats, is to send it to one of the other writers via the contact form. And while we will answer questions posed in discussion threads, we are much more likely to answer a question asked through the contact form. When those discussions get filled up with comments, they get hard to follow.
Be aware that we reference plenty of our innovative FO stats here, not to mention their unfamiliar terminology, so if you are a recent addition to the readership you might want to read this first.
Before we get to the questions, the beginning of Loser League Part II this week reminded me that I never ended up sharing with everyone the results of the survey we took with Loser League Part I. Some of you may remember that after you picked your loser league team, we asked you to name your favorite NFL team. We wanted to know where the readers were coming from and who they were rooting for. Now, these numbers have probably changed since we asked the question, because this is from before our deal with FOXSports.com. We'll ask again when we do the Third Annual Football Outsiders Awards after the season and see if things have changed.
|Favorite Teams of FO Readers (Start of 2005 Season)|
It's no surprise that the Patriots rank number one among our readers. Our first publicity when we started back in 2003 was from Boston Sports Media Watch, and we've gotten more media coverage in Boston than anywhere else, mostly because I happen to live here. I knew we had a lot of Eagles and Steelers fans reading the site but I didn't realize there were so many Bears fans, and I thought we had more Packers fans reading than this seems to indicate.
Onto the e-mail ...
Mitch Wojcik: Since we're roughly at the mid-point in the season, how do the top teams in 2004 compare to the top teams in 2005, DVOA-wise? In addition, do you see the emergence of any trends in 2005 that differ from 2004?
Aaron Schatz: The top three teams of 2004 (NE, PIT, IND) are all higher than the 2005 Colts. It's also important to remember that the 2004 Eagles were just as high as the top three AFC teams until they stopped playing their starters in Week 16.
As far as emerging trends, do you remember the big rise of offense? Gone. Offense is back to a level halfway between 2002 and 2003. I wrote about this in the New York Times last Sunday. The other thing gone is last year's strange trend where the best defenses were almost all on losing teams. That's not happening so far this year, and it hadn't happened in previous years either. Last year was just a strange case because of the number of teams with great defenses but imploding quarterback situations (Chicago, Washington, Miami, Buffalo).
Ron Anson: Is Red Cashen still refereeing? I like the way he calls "first down".
Michael David Smith: No, Red Cashion is no longer refereeing. He retired in 1997 after a 25-year career that included refereeing Super Bowls XX and XXX. Cashion, a graduate of Texas A&M, was known for the way he enthusiastically drawled "First Down!" Cashion currently serves as chairman emeritus of ANCO, an insurance company.
Todd Schneider: Does the DVOA system reward a team in regard to third down conversions if they never get to third down, i.e., achieve a first down in one or two plays? Denver is bad on third down with more than four yards to go, but their successful drives usually come from the fact that they don't put themselves into third-and-long situations (I give credit to Shanahan/Kubiak for strategizing that way). I would guess that Denver is above the league average in successful drives vs. unsuccessful drives. Therefore, third-and-long is not as big a problem for them as other teams.
Aaron Schatz: Well, when I'm talking about how well a team is doing in a certain situation, you are correct that I do not talk about how OFTEN they are in that situation. And Denver is definitely in third-and-long less than other teams. It doesn't change the fact that it will be a problem when they do get into the situation, but they aren't in it quite as much.
Anyway, there's no reward for converting a third down, per se. The biggest issue here is that the "success points" system has a huge gap on third downs.
That's why third down is so important both in the system and in real life: teams can make up for their failures from the first two downs. But a team like Denver that gets six yards on first down and four on second and never hits third will have a significantly positive value from the first and second downs. So its TOTAL offensive DVOA will be very good, even if its third down numbers are not.
In fact, I'm commonly asked which rates better, a team that gets four yards on every play or a team that goes zero, zero, 12. Both teams will convert every set of downs but actually the more consistent team rates better. That's because there is no such thing as a team that will always convert on third-and-long. You have to take every single down as a separate occurence, and a team that gains 4-5 yards on first downs is putting itself into a much easier situation on second and third down, and doesn't have to depend on having an extreme ability to convert hard third downs. And Denver is this kind of team -- that's partly why the midseason projection system believes they'll be the #1 offense in the second half of the year.
Mark HW: I live in the Twin Cities and shoulder the especially heavy burden of being a Vikings fan. Last Sunday against Carolina, it looked like Fred Smoot was all alone on Steve Smith throughout most of the game. How do the FO metrics take into account poor coaching decisions and schemes? I'm not saying the Vikings are any good, but if you don't make reasonable adjustments, the opposition will take advantage all day long.
Aaron Schatz: We can't. Poor coaching decisions and schemes are just part of the subjective analysis that has to be added to our objective numbers to get a better picture of the game. That's why we write commentaries to go with the numbers. We run straight numbers on our site, but you'll notice that when we write, we always comment on where we think the numbers are missing something.
Sam Mogilensky: Is there any way you could put together some data on whether high-drafted quarterbacks have more successful careers if they sit on the bench for a year or two (or more) rather than starting as rookies? I can already think of one problem with such an analysis, which is that quarterbacks who start as rookies are most likely playing for bad teams, while quarterbacks who sit are probably on good teams. Also, I! don\'t know if DVOA and DPAR go back far enough to get a good sample. But this question has always intrigued me.
Ned Macey: Actually, I dealt with this issue in a year ago and came to some conclusions in this article. The sample size will always be too small to draw adequate statistical conclusions. Further, the success of Palmer, Roethlisberger, and Eli this year shows that different quarterbacks will be able to succeed in different situations. As you see in my article, however, waiting certainly has a higher percentage of successes.
Bill Fish: Your team efficiency rankings appear to be the sum of Offense, Defense and Special Teams DVOA. Is there any type of weighting built into the DVOA ratings? For example, special teams plays account for 15-20 percent of the total plays for a team, yet they have a 33 percent weighting based on the sum of the components. It seems like they are over-weighted.
Also, two-point conversions: special teams plays or offense/defense plays?
Aaron Schatz: To answer the first question, special teams work differently than the rest of DVOA. Unlike offense and defense, which are complicated formulas that try to take into account both total yardage and the ability to gain first downs, special teams just involves straight yards or, in the case of field goals, points. So the special teams ratings are not percentages, they represent points above or below average (field position from punts and kicks is translated into points based on the average number of points a team scores from every yard line on the field). That's explained more here.
But of course I have to turn that into a percentage or else I cannot add it to offense and defense. So I did a regression to figure out what variable X gave you in this equation:
(special teams * X) + unadjusted offense - unadjusted defense = total
with total for every team over multiple years having the closest correlation to winning percentage. This means that special teams DVOA takes into account the fact that special teams are not as important as offense or defense. This is why the special teams ratings are so much closer to zero.
As for two-point conversions, I don't include them at all. Obviously they are very important, but they are too rare to really be predictive. I suppose I could include them in offense/defense just to get a better idea of how well teams defended plays two yards from the goal line. You know, for power rushing or similar stats.
Before we wrap up the mailbag, every so often I run an e-mail when someone needs some information and wants to pose a question to the readership. This week's question is related to pass defense and also might start a fun conversation about football simulation games.
Lee Harris: I design my own cards for the classic "Statis Pro Football" simulation game. The game provides rules on assigning "pass defense" values to each team's secondary. These are distributed to the defensive backs on the team, and affect the completion range of the quarterbacks when they throw to a receiver marked by them.
The problem is, how can I use easily available stats (NFL.com?) to decide which player gets which rating, without having to go through them one by one? Currently, defensive backs are separated into corners and safeties, then ordered by "Pro Bowl?" player, Games Started, and I think that's it. I'm not even sure how ties are broken up.
I've always liked the idea of using Passes Defensed, which is an available stat, but I am aware that the better corners won't be thrown at much, so will tend to have low PD stats. I thought of using PD per game started to avoid that issue, but then a poor corner who just didn't have many PD might end up looking great. So then I thought, could you work in tackles and stuff to estimate how often they're being thrown at, in addition to PD divided by games started or played?
Anyway, that's the kind of thing I would love to discuss with fellow statheads and come up with stuff I can use to modify the game rules. Again, this is purely amateur - the game went out of print in 1992, and so there's just a few enthusiasts who continue to update the rules, make new cards, etc. I have a website devoted to the game here and can be e-mailed at lee.harris4-at-virgin.net.
30 comments, Last at 20 Nov 2005, 6:49pm by Bright Blue Shorts