Week 16 DVOA Ratings
Big wins in Week 16 have moved some teams around at the top of the Football Outsiders DVOA ratings, both in the full-season numbers and in the weighted DVOA ratings that lower the strength of early-season games to get a better idea of how well teams are playing now.
We had six games in Week 16 with a single-game DVOA over 50%. Five of the teams currently in our top six had one of these games. The exception? Kansas City, of course, was the team in our top six that didn't have a big win this week, and the Chiefs end up with just 2.0% DVOA for a win over Atlanta. As a result, the Chiefs have fallen in both of our ratings. Kansas City is now fourth in total DVOA, falling behind Green Bay, and fifth in weighted DVOA, falling behind Tampa Bay, Green Bay, and even Buffalo.
(San Francisco, for those curious, was the team outside our top six that also had a 50% game this week. A surprise since the final score was just 20-12, but the 49ers outgained Arizona on average 7.2 yards per play to 4.4 yards per play.)
Green Bay had the best win of the week with 98.0% DVOA for a dominating victory over a playoff-quality Tennessee team. The Packers moved up to third in total DVOA and fourth in weighted DVOA, passing Kansas City in both metrics.
New Orleans and Tampa Bay maintain our top two spots, each moving up in total DVOA by about 3.5%. Tampa Bay also passes Kansas City in weighted DVOA.
Seattle gets a nice opponent boost for beating the Los Angeles Rams and the Seahawks move up from ninth to sixth in both total and weighted DVOA.
But the hottest team in the NFL may be the Buffalo Bills. Five weeks ago at their bye week, the Bills ranked 12th in the league in both total and weighted DVOA. Since then, the Bills have had five straight games with DVOA over 40%, and they have improved to fifth in full-season DVOA and third in weighted DVOA. While Kansas City maintains a small lead in full-season DVOA, the Bills are now the top team in the AFC by weighted DVOA.
Here's the impressive week-to-week chart for the Bills this year:
The big difference between Buffalo and Kansas City is consistency. You know how I just wrote that the Bills have gone over 40% in five straight games? That streak is more games over 40% than Kansas City has had all season. They have only three. Now, 40% is a bit of an arbitary endpoint, and we could just as easily define a good game as being a game over 30%. If we lower the bar, now Kansas City has six such games this season. But Buffalo has eight. The big difference between the teams is that Buffalo has one big stinker on its resume, the weird Tuesday game where they got keelhauled by Tennessee back in Week 5. That game was even worse than it looks since the Titans had barely practiced due to COVID protocols. The next week, the Bills lost to Kansas City. That's their second-worst game of the year by DVOA. But in recent weeks, Buffalo has certainly been the more impressive of the two teams.
We've talked a lot this year about Kansas City's consistency and that brings up an interesting question: does regular-season consistency make Kansas City more likely to win the Super Bowl despite the mismatch between their win-loss record and their DVOA rating? Curious about this question, I pulled up all the teams who had earned the first or second seed (i.e a bye week) since the league went to six playoff teams in 1990.
| Lowest DVOA Variance by 1/2 Seeds, 1990-2019 | ||||||
| Year | Team | W-L | Total DVOA |
Rank | Variance | Playoff Result |
| 2013 | NE | 12-4 | 17.9% | 5 | 4.7% | Lost CCG |
| 2006 | SD | 14-2 | 28.5% | 1 | 5.4% | Lost DIV |
| 2010 | ATL | 13-3 | 20.4% | 6 | 5.8% | Lost DIV |
| 1990 | LARD | 12-4 | 24.1% | 2 | 5.8% | Lost CCG |
| 2005 | IND | 14-2 | 33.4% | 1 | 6.2% | Lost DIV |
| 2016 | DAL | 13-3 | 24.0% | 2 | 6.3% | Lost DIV |
| 2011 | GB | 15-1 | 26.2% | 2 | 6.8% | Lost DIV |
| 2011 | SF | 13-3 | 18.4% | 6 | 6.8% | Lost CCG |
| 2017 | MIN | 13-3 | 28.7% | 2 | 7.0% | Lost CCG |
| 2018 | LAR | 13-3 | 23.9% | 3 | 7.1% | Lost SB |
Well, that table certainly doesn't suggest that an extremely consistent regular season is a key to winning in the playoffs. Five of the seven teams with the lowest variances got upset in their first playoff game! To find the first team that won the Super Bowl, you have to get past the first 20 teams until you hit... the 2019 Kansas City Chiefs. They have the lowest regular-season variance of any 1 or 2 seed to win the Super Bowl since 1990 at 8.8%.
Does it work the other way around? Are high variance teams more likely to win the Super Bowl? The teams with the highest variances have a some good examples of Super Bowl champions. Four of the 1 or 2 seeds with regular-season variance over 20% went on to win Super Bowls: the 1996 Packers, 1994 49ers, 1993 Cowboys, and 2004 Patriots. However, plenty of Super Bowl champions had just average variance, or sort of low or sort of high. And the lower seeds that went on to win Super Bowls didn't have particularly high variance either. The 2006 Colts are the highest of those teams with 18.8% variance.
It doesn't look like low variance makes Kansas City more likely to win it all, but I doubt that low variance makes Kansas City less likely win it all -- or that high variance makes New Orleans or Tampa Bay more likely to win it all. There really doesn't seem to be a connection either way.
* * * * *
Football Outsiders playoff odds, snap counts, and the FO+ database are now all updated through Week 16. A reminder that all our free stats pages, including DVOA and player position stats, now require registration to view. This is not a paywall! You only need to register (for free) and then log in to the site to view these pages. While you're at it, you can get a seven-day trial of FO+ and check out the FO+ features like a deeper DVOA database, weekly fantasy projections, and picks against the spread.
* * * * *
Here is the Football Outsiders Top 16 through 16 weeks of 2020, measured by our proprietary Defense-adjusted Value Over Average (DVOA) system that breaks down every single play and compares a team's performance to the league average based on situation in order to determine value over average. (Explained further here.)
OFFENSE and DEFENSE DVOA are adjusted for performance indoors and consider all fumbles, kept or lost, as equal value. SPECIAL TEAMS DVOA is adjusted for type of stadium (warm, cold, dome, Denver) and week of season. As always, positive numbers represent more points so DEFENSE is better when it is NEGATIVE.
WEIGHTED DVOA gives recent games more strength than older games to get a better idea of how well teams are playing now.
To save people some time, please use the following format for all complaints:
<team> is clearly ranked <too high/too low> because <reason unrelated to DVOA>. <subjective ranking system> is way better than this. <unrelated team-supporting or -denigrating comment, preferably with poor spelling and/or chat-acceptable spelling>
| RK | TEAM | TOTAL DVOA |
LAST WEEK |
WEI. DVOA |
RANK | W-L | OFF. DVOA |
OFF. RANK |
DEF. DVOA |
DEF. RANK |
S.T. DVOA |
S.T. RANK |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | NO | 32.3% | 1 | 33.8% | 1 | 11-4 | 11.9% | 7 | -17.0% | 3 | 3.5% | 6 |
| 2 | TB | 29.0% | 2 | 29.5% | 2 | 10-5 | 16.0% | 4 | -16.2% | 4 | -3.2% | 26 |
| 3 | GB | 24.1% | 4 | 27.6% | 4 | 12-3 | 26.9% | 1 | 0.4% | 15 | -2.5% | 25 |
| 4 | KC | 22.7% | 3 | 25.2% | 5 | 14-1 | 24.9% | 2 | 2.6% | 19 | 0.4% | 17 |
| 5 | BUF | 20.7% | 7 | 27.7% | 3 | 12-3 | 15.6% | 5 | -1.0% | 13 | 4.0% | 5 |
| 6 | SEA | 20.0% | 9 | 19.7% | 6 | 11-4 | 14.0% | 6 | 1.3% | 16 | 7.3% | 3 |
| 7 | PIT | 18.3% | 5 | 17.0% | 8 | 12-3 | -4.8% | 22 | -22.0% | 1 | 1.1% | 14 |
| 8 | LAR | 17.1% | 6 | 18.4% | 7 | 9-6 | 7.9% | 8 | -15.8% | 5 | -6.6% | 30 |
| 9 | BAL | 16.0% | 10 | 10.3% | 11 | 10-5 | 3.2% | 12 | -5.0% | 9 | 7.8% | 2 |
| 10 | IND | 14.6% | 8 | 15.4% | 9 | 10-5 | 2.2% | 14 | -9.3% | 8 | 3.1% | 7 |
| 11 | MIA | 9.9% | 11 | 12.8% | 10 | 10-5 | 0.7% | 16 | -4.5% | 11 | 4.7% | 4 |
| 12 | SF | 5.5% | 14 | 4.8% | 13 | 6-9 | -2.1% | 18 | -9.8% | 6 | -2.2% | 24 |
| 13 | TEN | 3.5% | 12 | 5.0% | 12 | 10-5 | 19.0% | 3 | 11.1% | 29 | -4.4% | 28 |
| 14 | ARI | 2.2% | 13 | 0.4% | 17 | 8-7 | -2.2% | 20 | -4.5% | 10 | -0.1% | 19 |
| 15 | CHI | -0.1% | 15 | 2.8% | 14 | 8-7 | -11.8% | 25 | -9.4% | 7 | 2.3% | 10 |
| 16 | WAS | -1.8% | 16 | 2.4% | 15 | 6-9 | -20.3% | 30 | -17.8% | 2 | 0.7% | 16 |
For the full table, including variance, schedule strength, and non-adjusted VOA, visit the Football Outsiders DVOA database.
Comments
38 comments, Last at 02 Jan 2021, 2:32pm
#1 by Joseph // Dec 29, 2020 - 6:17pm
So, since NO and TB are the top 2 in variance, one could assume that it has a lot to do with the SNF spanking back about 6 weeks ago. If that game were removed from the database, how would that affect the variance ratings? (Obviously their DVOA rankings would be different as well--but I am only concerned about both teams' high variance--esp. since it is the theme of the article.)
#2 by jheidelberg // Dec 29, 2020 - 6:37pm
I was curious if variance correlated with winning the Super Bowl. The lowest two variance teams and the highest two variance teams have never won a Super Bowl. If the variance numbers stand through week 17 we have a shot of this no longer being true as the Chiefs, Bucs, Saints and why not, lets throw in the Bears, can win the Super Bowl this year.
I compiled these figures but never found the right time to post it. The time is now that an article has been written.
Super Bowl Winners:
Variance rank Super Bowl Winners
Variance 1-11 13
Variance 12-22 22
Variance 23-32 5
Hypotheses:
1. High variance teams tend to not win the Super Bowl.
2. Middle of the road variance teams win the Super Bowl at a higher rate than expected by chance.
I tried running this through a statistically significant calculator and come up with that these two figures are statistically significant at 90%. A few problems are:
1. There have not been 32 teams throughout DVOA history so that the statistically significant calculation is not entirely accurate
2. Statistically significant at 90% is a shaky proof of a theory; if you are writing a masters or doctorate thesis, we would like at least 95%
So I would conclude that we have proof, albeit weak that variance does matter when trying to run the table and win the Super Bowl.
Feel free to throw the challenge flag on me. Upon further review, the hypotheses are unlikely to be currently confirmed, may stand or may be overturned. A few more data points would certainly help. As we move forward in time and when Football outsiders goes back before 1985, there will be more data points
#5 by gomer_rs // Dec 29, 2020 - 7:44pm
Either:
Near 100% injury rate in the NFL so on some level available talent in the game is just a lottery - which is not the interesting variance question.
Or:
Teams that produce widely disparate performances (or not) just because of play/coaching/player talent. This is basically the team wide comparison of Eli Manning and Phillip Rivers.
And, I imagine it comes down like this... a great team wants low variance, but in a short single elimination tournament perhaps a moderate amount of variance is best for the vast majority of merely good teams since there is still a good chance that you will face a better team at some point during the tournament, but the tournament isn't long enough to punish you for too much variance (like March Madness).
#17 by Wifan6562 // Dec 30, 2020 - 11:18am
It’s a good theory, though I think there are two main problems that make it very difficult to validate:
1. Variance is caused by both relevant and irrelevant factors when predicting success.
For example variance can be caused by injuries, motivation, an “Achilles heel” (think inability to stop the run, etc.), tendency to fumble, or many other factors. There are probably elements of variance that matter, but as a whole and as Aaron noted, the data doesn’t support a trend.
2. You may be picking up trends in playoff eligibility. I have no idea if this is supported by the data, but since the hypothesis is related to winning the Super Bowl, I think it’s important to only consider teams who are eligible to do so (like Aaron did).
Ultimately, I suspect that portions of variance matters, but overall it is “noise” in comparison to defense, offense, and special teams performance.
#21 by PTORaven // Dec 30, 2020 - 12:34pm
I have a hunch that (1) lower-seeded and lower-rated teams with high variance do better than those with low variance, and (2) higher-seeded, higher-rated teams with low variance do better (to the extent that variance is predictive). these would tend to cancel out any analysis like what you were trying to do that looked at overall variance versus performance i believe.
i'm picturing a lower-rated team that's a big underdog, something like the '07 Giants and Pats. now i don't know if the Giants were high variance that year or not, but going into a game that big an underdog, i'd feel better about a high variance team being able to pull the upset. conversely, i feel like a low variance heavy-favorite just needs to perform at that narrow average level to get the W, and the only thing they need to do is avoid laying an egg like higher variance teams are more likely to do
#24 by Wifan6562 // Dec 30, 2020 - 2:37pm
Logically I agree. However, the 2007 NYG were 3rd in variance but 15th in DVOA. Their variation started in the postseason. I also checked 2011 and they were 17th.
Long story short, I’m not finding any data to support this even though it 100% makes logical sense to me.
#3 by Q // Dec 29, 2020 - 6:48pm
I dont think 1996 GB was as High Variance as DVOA thinks. They had a 3 Game stretch where their receiving corps was decimated by injury and they had to use TE's as WR's (similar to CLE vs NYJ this last week).
Now yes, in a way that is High Variance if 3/4 of the season is a Juggernaut Offense and 1/4 of the season is vs a blah Offense. However, I think most people when talking Variance they are mainly excluding due to injury.
#4 by theslothook // Dec 29, 2020 - 7:26pm
This is not a reflection on Aaron's choice of topic, but looking at raw variants is probably two course a metric to yield much.
Just looking at raw variance doesn't tell you which teams are inherently highly variable versus having suffered a bunch of injuries at random points in the year and that explaining the variance. You add that to the fact that we're talking about a 16-game sample and I'm not sure there's enough signal there to tell anything one way or the other.
#6 by t.d. // Dec 29, 2020 - 10:26pm
The idea that the top three teams in the NFC have all been better than anybody in the AFC certainly bucks conventional wisdom; it subjectively feels off- a lot was made coming into the week that the Packers had only one win against a team above .500 (it's up to three now with the Bears winning streak), but really, all the top NFC teams have slim resumes compared to the top of the AFC in this regard- but dvoa is pretty good at spotting things that aren't readily apparent, so I'll be curious to see how it plays out. Also, the Chiefs putting up a subpar dvoa while winning every week reminds me of the Colts and Saints in 2009 (great qb doing just enough every week), so maybe it works out for them just fine, just like them being low-variance did last season (fwiw, I bet they weren't in the playoffs)
#9 by DisplacedPackerFan // Dec 30, 2020 - 1:53am
I don't really follow much conventional wisdom, so this is me looking into where that narrative may have come from. I had heard the talk about GB and the only 1 win of course. But I hadn't really heard much about the AFC being tougher or better tested. I'm not arguing against you, I'm just doing some digging and posting to look a little deeper. Now that the rest of the post in written I'll thank you for prompting me to dig into this more and learning some stuff.
So are the top resumes that slim?
It's interesting that by DVOA GB, KC, and BUF all have the same -1.8% past schedule strength. That will end up being BUF, GB, then KC since Miamis is better than Chicago and Chicago is better than the Chargers. KC and GB both played the NFC South so have a decent amount of common opponents for cross conference teams.
KC is 5-0 against teams with .500 or better records, and 9-1 against sub .500 teams.
GB is 3-2 and 9-1
BUF is 4-3 and 8-0
NO is 3-2 and 8-2
TB is 1-4 and 9-1
PIT is 5-1 and 7-2
So I do see at a glance that the top 3 in the AFC have played more and better against teams with winning records than the top 3 NFC. They have played about the same against losing teams. TB, a team that I keep wondering about for DVOA anyway is the clear questionable NFC top team. GB is the only winning team they've beaten, and as I've written using the zlionsfan template there is a curse on Rodgers playing in TB (post 4 here https://www.footballoutsiders.com/dvoa-ratings/2020/week-12-dvoa-ratings).
Since I'm a GB fan living in KC and see both those teams a lot I dug into them first.
If GB beats Chicago in W17 that goes to 4-2. KC will be 5-0. Which is really more "battle tested."
Common opponents by conventional stats GB beat NO 37-30, KC beat them 32-29 so that favors GB. GB beat ATL 30-16, KC beat them 17-14, that favors GB. GB beat Carolina 24-16, KC beat them 33-31; I think that favors GB too. GB lost to TB 38-10, KC beat them 27-24 so that's a huge favor to KC.
Division wise Detroit is 5-10, Denver is 5-10. Minnesota is 6-9 Chargers are 6-9. Chicago is 8-7, Vegas is 7-8. So again similar but favors GB for the majority of the sub .500 opponents by conventional stats. I think I trust DVOA with the schedule strength and the overall ratings for those two with common cross over.
What about the other teams?
Buffalo is more interesting they are listed at a -1.8% DVOA schedule too, but playing Miami in the final game will give them the harder schedule overall. They played the NFC West so that gives them more winning opponents and Miami is a 10 win division rival that speaks well for them. However they are only 4-3 against those teams. They could end up 5-3 or 4-4. GB could end up 4-2 or 3-3. That doesn't feel a lot different, though 1-1 vs 8-8 Chicago would not look as good as 1-1 against an 11-5 Miami. Of course DVOA says that BUF will have had the harder schedule and the results of the games could change the overall ratings. So I do see where the conventional stats wisdom comes from here but I also think DVOA has got it covered.
NO is 3-2 against .500+ plus teams and 8-2 against sub .500 (interestingly sharing a loss to the 7-8 Raiders with the Chiefs). DVOA gives them a 0.9% schedule, but that will drop a bit since Carolina is nothing special. Same number of games as GB and KC, worse results than KC, same results as GB.
TB is 1-4 so I can see the argument there, and GB is the only winning team they have beaten. They have a 2.2% DVOA schedule that will drop some since ATL is nothing special either. As I said earlier in the post, I don't see what DVOA loves so much about this team. I definitely see why conventional wisdom would question how good they really are and could see why KC, BUF, and PIT could all be considered better.
PIT is weird. They have a -6.3% schedule and final game against 10-5 CLE who has a negative DVOA (and negative point differential too so conventional numbers think they have an inflated record) so that schedule is only likely to get worse. They blew out 10-5 Cleveland, beat 10-5 Baltimore twice in close games, beat 10-5 TEN and 10-5 IND in close games, but lost to 6-9 WFT and 4-10 Cincy. Can't really hold the L to 12-3 BUF against them. So they look a lot like NO (Raiders and Eagles losses) but have done better against the winning teams, though NO lost to a 12-3 and 14-1 team, NO also beat the 10-5 teams they faced twice.
So I get it now
After poking like that I see where the thoughts come from but I also see what DVOA sees. The existence of the NFC East is going to make the NFC look like the weaker conference no matter what. But the NFC doesn't have the Jets or Jaguars which is part of why some of those 10-5 teams aren't 8-7 or 7-8 like in the NFC. So without looking at more advanced metrics you'll miss things like that. Just looking at the standings they AFC looks better. More double digit wins, nothing like the NFC East. Both conferences only have 7 teams each with a positive point differential though. But that isn't as obvious a thing. Then you see Detroit is the only sub 100 point differential team in the NFC. The AFC has 3 (add Denver to the Jets and Jags).
It's interesting. But TB is still the only real standout where DVOA and conventional stats don't seem to agree to me. Of course they are the highest variance team by DVOA, so that says something too. It's not even the GB game at this point since weighted and overall are almost the same for them and the GB game is now heavily discounted.
#11 by lauers // Dec 30, 2020 - 7:52am
Nice breakdown all around. After it was apparent the NFC East was a dumpster fire, I started keeping track of standings where wins vs. the NFC East didn't count, a tie counted as a loss (for Cincy), and losses counted double. I eventually added the Jets and Jags to the NFC East. I stopped keeping track a while back but there were a few takeaways:
1) The AFC North was very overrated (conventionally speaking) since they played the NFC East and AFC South (which had both Jax and Houston who is now 2-11 when not playing Jax). I'm glad Pittsburgh came down to earth because they had a very weak schedule when they were undefeated.
2) The NFC West was overrated, since they played the NFC East and AFC East. The Rams opened 4-1 with all wins vs the NFC East and talking heads were saying the NFC West could be the best division ever. The AFC East was two guaranteed wins (vs. Jets and my beloved, terrible Pats), while Buffalo and Miami were tougher outs.
3) The NFC South and North were underrated since they played each other and not the NFC East. Also the NFC South played the AFC West, who were pretty good in their own right. Just about all of these teams received few "free wins" and were better than their records after the first half of the season. The second half brought things more into line with my (very basic) model.
Commentators talk like the Bears are awful and the Cardinals are clearly better (maybe they haven't watched since the Hail Murray). The Bears went 3-0 vs the NFC East + Jax + Hou (the latter 2 during Trubisky's "resurgence"), while the Cardinals went 5-1 vs the NFC East + NYJ + Pats. I think those teams are much closer than the pundits give credit for.
As for TB, they were 1-0 vs NFC East + NYJ/Jax/Hou/Pats. They didn't win many games vs teams with winning records, but in a league full of "free wins", they didn't benefit very much. 9-5 vs the NFC South/North + AFC West + Rams is very solid (though I'm not sure I'd put them above GB/KC/BUF).
#12 by lauers // Dec 30, 2020 - 8:43am
PIT is 6-2 vs the absolute dregs of the NFL (NFC East + Jax/Hou/Cin). They are 6-1 otherwise with a blowout win vs CLE, a blowout loss to BUF and 5 one-score wins!
ATL only has 4 wins, but played a ridiculous schedule and have an overall point differential of -1, which equates to a roughly .500 team. TB beat them twice. If they played in the AFC North, it's possible they have 9+ wins...
#7 by Mr.Morden // Dec 29, 2020 - 11:58pm
It's interesting that NO, KC, and GB are all now bunched together in the odds for Super Bowl winner, with NO having a slight lead over the other two despite the fact that it'll most likely end up as the #2 seed rather than #1 seed. I guess its weighted DVOA is high enough that not getting the bye isn't determinative.
But I wonder to what extent NO's status as favorite is due to the fact that there is still a ~1 in 5 chance of it getting the #1 seed? If you only look at the scenarios in which GB is the #1 seed, does KC (or maybe even GB itself) take the lead as Super Bowl favorite?
#10 by lauers // Dec 30, 2020 - 7:15am
As a statistician you got my juices flowing with a discussion on variance!
One thing we observe frequently is that variance increases with the mean. This is especially true for statistics bounded by zero (which DVOA is not), but that would impact yards per play. A team that averages 8 yards per play will almost certainly have more variance than a team that averages 4 yards per play.
If variance is correlated with DVOA then saying "the teams with the highest variance win SBs" is similar to saying "the teams with the highest DVOA win SBs", which is similar to "the best teams win SBs".
A simple way to adjust for this could be to run a logistic regression for winning the Super Bowl with terms for DVOA and variance and see whether there is any relationship between variance and winning the SB when holding DVOA constant.
#23 by theslothook // Dec 30, 2020 - 1:26pm
"One thing we observe frequently is that variance increases with the mean.
I am a bit confused by this comment. Are you saying variation proportionally increases with the mean? or are you saying as your mean increases, the variance itself increases. Ie - two variables with different units; the one with greater values is just inherently noisier? I am not sure that is literally true and its certainly not implied in most models.
As for the logistic regression, remember that variance is derived from DVOA itself. That sort of implies theres a correlation between DVOA and the variance, which implies you have multicolinearity baked into the regression. So in that sense, the coefficients become hard to interpret
#28 by lauers // Dec 30, 2020 - 11:02pm
I'm not sure how the two scenarios you laid out are different, but I'll try to explain better. If you have two players and one player has a higher average value (yards per touch, TDs per game, receptions per game, etc), that player is also more likely to have a higher variance for that statistic. For counting statistics, variation usually increases with the mean. I have a fantasy database of all stats per game per season (since 2010). There are often strong positive correlations between the player per game average and player per game variance for a given statistic (especially for TDs per game, rushing/receiving yards per game, less so for passing yards per game -- still positive, just weaker). Maybe more intuitively, it's easier to be consistent around a small value than a large value.
If by "not implied in most models" you mean most models fit overall, you're probably right, since most people use ordinary linear regression. If you mean "most models" as in the majority of types of models, it's a closer call. Poisson regression is fairly common and the main assumption there is that the variance is equal to the mean (so as the mean increases, so does the variance). That's a pretty strong assumption, so statisticians prefer the negative binomial distribution in which the variance can be overdispersed, meaning the variance increases with the mean, but can be larger (with a fat tail). The gamma and log-normal distributions are also common and have variances that increase as a function of the mean.
Also, if the mean and variance are collinear, they would also be correlated, which would mean we're on the same page. If game DVOAs for each team are normally distributed variables with their own team-specific means and variances, then the mean and variance would not be correlated/collinear and thus a logistic regression could be appropriate. A simple check would be to look at the correlation coefficient between the two of them. A rough eyeball at the above table makes me think that they aren't correlated, at least among the league as a whole since the most variance is at the very top and very bottom of the league (furthest from the mean). I'd definitely be interested in looking at a larger sample size though.
Sorry for the stats rant. I hope anybody else enjoyed that as much as I did.
#29 by theslothook // Dec 30, 2020 - 11:37pm
I guess I would word it like this. Most models, including poisson, don't assume a correlation between mean and variance. In fact, when doing bayesian priors, they tend to be set up as completely different parameters that get learned from the data. Maybe objectively it doesn't matter and the models themselves capture this dynamic. After all, these moments are all derived from the data.
The reason I think there is collinearity is because whatever latent factors derive DVOA, since variance is taken from DVOA, it will likely leak into it. Sure, that's an assumption but what I likely think is happening.
#30 by alan frankel // Dec 30, 2020 - 11:43pm
It was definitely an interesting rant. I definitely learned something new about how higher means lead to higher variance which made intuitive sense.
I think that even if there is a magic variance figure or range there is likely to be a better reason why a team will win or not win a championship. For example, I don't think a team should look at their offense with bad wide receivers and think, well we need to increase our variance so let's throw one or two more bombs a game ( although the way PI is called these days this actually might be a viable strategy... AJ Bouye sends his regards).
However, what I have seen teams do is realistically determine how many points they are underdogs by, and then try to come up with one or two trick/gimmick plays to make up the difference. Obviously, this is anecdotal and a guess but I'm pretty confident Kevin Stefanski used this theory against Dallas earlier this year when Dak was healthy, as well as the more famous Sean Payton onside kick where Payton felt his team needed to steal possession from Peyton.
So there are creative ways of introducing variance into games. However, I would look at this more like an ace up the sleeve to be used creatively as opposed to a consistent strategy on its own. I also think that it is more likely that a more consistent team pulls out a creative move to steal a game or gets lucky than a high variance team has a good day. In short, I want to increase my odds that luck is a factor in the game and the best way to do that is to keep the game within one possession, and boom-bust teams are more likely to be blowout victims. Also, the NFL season is short and full of weirdness, it may be quite diffficult to determine who is truly a boom or bust team with such a small sample size of games and injuries. Not trying to undermine the conversation just saying this is something to take into account
#31 by jheidelberg // Dec 31, 2020 - 12:49am
https://www.footballoutsiders.com/stats/nfl/team-efficiency/2020
You can get data from 1985-2020.
From all of what you said, this makes the most sense to me, "It's easier to be consistent around a small value than a large value." For example, if a player who plays every game averages 1 reception per game, he would have a 16 catch season. He would be consistent, most of his games would be 0,1, or 2 receptions. If he had a 7 reception game, he would get a large number of 0 reception games to have his mean of 1.
His variance would surely be lower than 2019 Michael Thomas who's mean was 9.3 receptions per game.
This is what you mean isn't it?
However, does your theory apply to DVOA? Why would an average team of 0 DVOA be more likely to be less variant than an excellent team or a poor team? Mediocrity implies to me, sometimes great, sometimes awful, sometimes in the middle. A great team or an awful team I would think would be more consistent.
I picked out some of the best and worst teams of all time:
1991 Washington (best DVOA team ever) : 15th in variance
1985 Chicago 8th in variance
2007 New England 12th in variance
2008 Detroit 19th in variance
2019 Miami 23rd in variance
Then randomly I found the 1996 New York Jets, not historically bad but last in DVOA for 1996, number 1 in variance (lowest variance). As most Jets fans will tell you (thus low variance of opinion) when you think of consistently bad, think J-E-T-S Jets Jets Jets.
Since clearly you have studied statistics at a high level, I would compare DVOA to a student. The valedictorian would have little variance as would those that flunk out of college. I would expect the grade C student to have grades and test scores with higher variance than the valedictorians and those that flunk out.
To get a better understanding of variance, refer to Football Outsiders Vincent Verhei's comment (number 125) last week about variance being a product of DVOA...
I would also be interested in your comments on my table in comment number 2 above. It appears that middle of the road variance correlates to Super Bowls. You can do a better statistically significant analysis than I can. Also we know that correlation does not mean cause and effect. I am wondering if middle of the road DVOA variance also correlates to being awful, but there is no champion crowned each year for being awful.
#14 by big10freak // Dec 30, 2020 - 9:32am
by the defensive improvement as the season has moved along. Certainly Savage, P Smith, Gary have been playing well the last month plus. Barnes being inserted as the regular main inside lineback seems to have been a real positive. Alexander has continued to to amaze. Right now the remaining issue is King's subpar play. But at this point looks like GB is going to take its chances as they apparently don't like the alternatives.
#15 by ammek // Dec 30, 2020 - 10:05am
With all of the national attention on how many points have been scored this season it's interesting that DVOA is seeing fewer elite offenses than usual. In fact there is more bunching towards the average among good offenses than among good defenses for the first time in ages. If the numbers don't change next week, this will be the first time since 2008 that the team ranked fourth on defense has a stronger DVOA than the team ranked fourth on offense. (It's the same team: Tampa Bay!) And the Buccaneers' -16.2% defensive DVOA would be the best for a fourth-ranked defense since 2008.
This is quite a big deal: on average from 2009-19, there were 6.0 offenses with a better DVOA than the fourth-best defense. Perhaps it's getting harder for the best offenses to stand out from the crowd in an environment where even the worst teams are completing 65% of passes and throwing few picks. Conversely, the best defenses might be pulling away from the pack a little.
Or perhaps it's just another quirk of 2020.
At the bottom, there's faint sign of a similar trend. The fourth-worst defense of 2020 has +11.1% DVOA, which would be the third-best since 2008, suggesting that bad defenses are bunching closer to the average. Eight offenses have worse DVOA than that, which is more than the 2009-19 average of 6.5 such offenses, but not wholly exceptional: for example, there were nine offenses worse than the fourth-worst defense in 2009 (and that defense was at a considerably higher +12.9% DVOA).
#25 by jheidelberg // Dec 30, 2020 - 7:04pm
I like your argument in favor of defense and the backup to your points.
Now going forward, the counterpoints. Look at the DVOA statistics for offense and defense when broken down by pass vs run.
The offense calls the plays. The offense will start calling more and more pass plays as analytics take over the game. I believe that this is already occurring. The differential between passing efficiency and running efficiency is astronomical. Simply call more pass plays, and there will be more offense. I believe that the reason that the two minute drill works so well now, is in part because the running game is virtually thrown out the window.
If you have an offense that is better at running than passing, you have a bad offense. The teams listed below are the only teams that have rushing DVOA more efficient than passing DVOA with their overall offensive ranks. Note that all of these teams have negative DVOA on both run and pass. Note also how difficult it is to have negative passing DVOA (9 teams) or positive rushing DVOA (6 teams):
NE 24th
NYG 26th
PHL 29th
WAS 30th
Defensively Miami is the only team with a better passing DVOA than rushing DVOA. As with offense, there are 9 teams with negative pass DVOA, and 6 teams with positive rushing DVOA.
https://www.footballoutsiders.com/stats/nfl/team-offense/2020
https://www.footballoutsiders.com/stats/nfl/team-defense/2020
How much value is there to run defense? If GB matches up with Tampa Bay, will they throw virtually endlessly?
KC and GB are doing you a favor with each running play. There will be few favors in the playoffs unless extremely high winds dictate otherwise. This is true not only for playoff teams. Look at the LAC and HOU pass vs run differential on offense.
#18 by BigBen07 // Dec 30, 2020 - 11:19am
Whether you want TB to refer to tom brady or Tampa Bay, they have been pathetic against good teams and beat up on cellar dwellers. Laughable to take them seriously when you consider how many 10-5 or better teams the Chiefs and Steelers have beaten, and Chiefs beat them all on the road.
#19 by dmstorm22 // Dec 30, 2020 - 11:52am
This Tampa team reminds me a lot of the 2009 Patriots - a team that DVOA seemed to like a lot more than conventional thought that season (particularly Brady, who was I believe the top QB by FOs metrics that year - mostly because of the defenses he faced).
That team, just like Tampa this year, started out 6-2, then lost three of four, before ending the season with a few nice wins against bad teams (exclude the Week 17 quasi-rest loss).
They also were a team with a paper thin resume despite a nice point differntial and the rest, went 2-6 on the road, lost to most of the quality teams they played (losing to both Super Bowl teams), or at the very least eked out wins against Baltimore and Atlanta in teh regular season, and then they got hammered in their first playoff game.
I doubt this team will get beaten as badly in teh playoffs (losing Welker hurt the '09 team) but I too am confused why they grade out so great, other than they're a classic team that dominates weak ones but either plays close or loses to any good team they face.
#22 by ammek // Dec 30, 2020 - 1:06pm
The Bucs defense is much better than it's given credit for. The run defense is really really good, although how much importance you attach to that may vary. The 2009 Patriots, on the other hand, had a just barely average defense. Granted, special teams are the other way round, but I don't think the comparison holds.
If I were to compare 2020 Tampa to a 2009 team, I'd choose Green Bay, which went 11-5 on the back of a #2 ranked defense that smothered bad teams but got lit up by the best QBs it faced. Its offense kept pace in every game but seemed to lack a cutting edge. I don't imagine Tampa will give up 51 points in its first playoff game, especially if that's versus the survivor from the NFC East sinkhole, but I wouldn't be surprised if it got caught up in a shootout at some point. Brady has quite a good record in those, however.
#26 by mrh // Dec 30, 2020 - 9:48pm
The Falcons did a great job defensively against the Chiefs and seemed to give Mahomes fits in reading them. Some of that was due to pressure on him, which may be partly explained by the two OTs not being 100%. But it's possible that Bob Sutton, former KC DC, now on the staff in ATL, had a good read on how to defend and confuse Mahomes.
It's possibly a coincidence, but the only two Chiefs TDs came a) on the drives right after they fell behind and b) when they went to hurry-up on offense.
#32 by ImNewAroundThe… // Dec 31, 2020 - 4:14pm
2020 GBP: 25th, -2.5% (bad)
2019 KCC: 2nd, 4.1%
2018 NEP: 16th .1%
2017 PHE: 16th, .9%
2016 NEP: 8th, 2.3%
2015 DNB: 14th, .7%
2014 NEP: 5th, 5.7%
2013 SES: 5th, 4.7%
2012 BLR: 1st, 9%
2011 NYG: 15th, .3%
2010 GBP: 26th, -2.4% (bad, worse ranking, better rating)
2009 NOS: 28th, -3.4% (bad, both worse)
2008 PTS: 23rd, -1.5% (bad)
2007 NYG: 20th, .9%
2006 INC: 25th, -3.6% (bad, same ranking, worse rating)
2005 PTS: 10th, 1.6%
2004 NEP: 16th, .2%
2003 NEP: 16th, .8%
2002 TBB: 9th, 3.6%
2001 NEP: 6th, 3.1%
2000 BLR: 3rd, 8.4%
1999: SLR: 9th, 2.8%
1998: DNB: 9th, 2.3%
1997 DNB: 4th, 4.3%
1996 GBP: 2nd, 7.4%
1995 DAC: 6th, 4%
1994 SF4: 11th, 1.1%
1993 DAC: 7th, 3.8%
1992 DAC: 8th, 1.9%
1991 WAR: 1st, 8.6%
1990 NYG: 2nd, 6.7%
1989 SF4: 17th, -.3% (bad)
1988 SF4: 7th, 2%
1987 WAR: 18th, -1.4% (bad)
1986 NYG: 11th, 1.2%
1985 CHB: 5th, 3.8%
#33 by theslothook // Dec 31, 2020 - 6:29pm
Has anyone ever quantified which aspects of special teams are more harmful than others.
IE does it hurt more to have a poor kicking unit for field goals or being bad at hunting from a receiving and kickoff point of view?
#35 by ImNewAroundThe… // Jan 01, 2021 - 9:09am
but I imagine there isn't anything concrete as ST is a small(er) portion of the game and trying to break it down even more brings even more noise. There could be 4 teams worse at ST than GB in the playoffs (TB #26, CLE #27, TEN #28 and/or LAR #30) and the highest (SEA, #3) has the 4th best odds in the NFC to win it all.
Maybe Ill put in weighted DVOA though
#34 by DisplacedPackerFan // Dec 31, 2020 - 7:16pm
Well that certainly makes it look like it will be harder as you don't see too many negative DVOA's in that list. Though as we both know what we really want to do is look at special teams DVOA for all the playoff teams and see how they did in general, not just for the super bowl winners. But I'm not doing that either.
I'll dig a little further into the SB winners with sub zero special teams because I'm wondering about the breakdown of those bad special teams. The 2020 Packers are bolstered by Mason Crosby being perfect on FG so far this season so I want to know what the shape of those other teams were. I left off the Hidden and Weather points because as the stats say they are outside team control. That might be a mistake for even this cursory look, but eh.
Negative values bolded just to help with readability since formatting options are limited
Year...Team...FG/XP...Kick O...KO Ret...Punt...Punt Ret
2020 . GNB .... 8.5 ...... -2.0 ..... -4.1 .... -12.2 ... -2.0
2010 . GNB ... -0.1 ...... -7.0 ..... -6.1 ....... 3.5 ... -2.4
2009 . NOR ... -9.2 ...... 1.2 ....... 7.3 ...... -8.4 ... -7.8
2008 . PIT ...... 0.9 ....... 7.8 ... -10.3 ....... 3.2 .... -9.0
2006 . IND ..... 4.8 .... -16.7 ...... 0.8 ...... -9.7 ..... 2.5
1989 . SFO .... 9.4 ....... 3.4 ..... -6.4 .... -11.5 ..... 3.8
1987 . WAS .. -6.9 ...... -0.9 ...... 1.6 ....... 1.1 .... -0.2
With the current rules I would think that Kick Off and Kick Off return are the least relevant of those stats going forward, but were in play for all the previous data. Only the 09 Saints were bad on both punts and punt returns like the 2020 Packers and no one has been as bad at covering punts on that list as the 2020 Packers.
That 2009 Saints teams looks like the one where current rules could really be an issue, bad at FG/XP and everything to do with punting. OK with kickoffs and decent at kick off returns.
There aren't a lot of data points where a team gave up more than 10 points vs expected like the 2020 Packers do when punting.
No team has been negative across the board. I don't see anything like none of them were bad at punt returns or none of them were bad at Field Goals. It appears that any aspect of bad special teams can be overcome well enough to win the Super Bowl. Though as mentioned the only thing we haven't seen is a Super Bowl winner that was bad at ALL aspects of special teams, and the 2020 Packers who were the initial subject of the question fit that category.
Interesting but sadly not hugely insightful unless I go deeper.