Week 8 DVOA Ratings
by Aaron Schatz
You won't find a lot of change in the Football Outsiders DVOA ratings this week. The Pittsburgh Steelers are still No. 1, the Los Angeles Rams are still a shocking No. 2, and the rest of the top eight teams all stayed the same with a couple of them flipping around. The big difference comes from Pittsburgh dropping more than five percentage points after a close victory over Detroit, from 34.7% last week to 29.4% this week. Since they are still at No. 1, this marks only the fourth time in DVOA history that no team had a DVOA over 30% after eight weeks of the season. Perhaps it says something about increasing parity that the other three times were also recent:
- In 2010, Kansas City was 5-2 and No. 1 with 28.9% DVOA.
- In 2011, Green Bay was somehow undefeated (7-0) but also sort of overrated, No. 1 but only at 28.0% DVOA.
- Last year, Philadelphia was No. 1 with just a 4-3 record, at 28.6% DVOA.
Below the top eight there are a couple of teams making nice healthy rises this week. Baltimore (15 to 12) and Dallas (14 to 9) had the strongest single-game performances of the week according to DVOA. Surprisingly, the third-best single-game performance according to DVOA belonged to... Denver? Yes, the Broncos, despite losing 29-19 last night, put up 44.5% DVOA against the Chiefs and move up from 23 to 18 overall. This is not a statement that we've all mis-judged Trevor Siemian. Denver had more yards per play but both teams had low averages: 5.2 for Denver, 4.7 for Kansas City. Denver's offense and special teams are up slightly (though special teams are still dead last) but the rise is really powered by Denver's pass defense, which had a very strong game given how well the Chiefs passing game had been the rest of the season. Kansas City entered the game averaging 7.4 net yards per pass play (including DPIs). Denver held the Chiefs to just 5.9 net yards per pass play, and this was the first game all season where the Chiefs had two turnovers on offense: Alex Smith's fumble on a Shaquil Barrett sack and Tyreek Hill's ridiculous interception on a halfback pass. For crying out loud, you have to teach the non-quarterbacks to throw the ball away if the receiver is covered on the trick play.
Denver also held the Chiefs to 3.2 yards per carry on the ground, but that kind of run defense is nothing new. At least, it's nothing new for the 2017 season.
The Broncos went from 17th to 13th on pass defense, and from fourth to second on total defensive DVOA. In fact, both Denver got a big bump this week that doesn't look quite as big because those teams were already ranked high. Baltimore goes from -12.6% (fifth) to -17.0% (third) by making a bad offense look horrendous, while Denver goes from -13.0% (fourth) to -18.9% (second) by making an excellent offense look pedestrian.
Speaking of that team that the Ravens dismantled on Thursday night, the Miami Dolphins lost so badly that they have now moved behind the Cleveland Browns in DVOA. That's right. Not only are the Browns not in last place, they aren't even in next-to-last place. Miami dropped from -25.5% (28th) to -34.7% (31st) after the 40-0 loss. The Dolphins are by far the worst team in DVOA history to have a winning record after Week 6, more than seven percentage points behind the 2007 Lions at 4-2 or the 1992 Colts at 4-3 or last year's Houston Texans around midseason.
The Dolphins are part of the group that last week I dubbed the Awful Eight, the eight teams that are way behind the rest of the league in DVOA. There's still a big gap between the Awful Eight and 23 of the other 24 teams, although the Tampa Bay Buccaneers are inching down slowly and threatening to make it an Awful Nine instead. After spending the whole season with a horrific defense and an underrated offense, the Bucs went out against Carolina and had a disaster on offense but played much better defense. (Coverage here in Any Given Sunday.) That moves the Bucs from sixth to 11th for offensive DVOA but raises them from dead last to 30th in defensive DVOA.
And in turn, that makes room for the Oakland Raiders to drop to 31st in defensive DVOA and for the New England Patriots to drop back into last place. I apologize for writing so much about the Patriots this year, but they're having a fascinating season. Because of what Denver did to the Chiefs on Monday night, the Patriots also moved up to the No. 1 spot in offensive DVOA. That's right: New England is the best team in the league on one side of the ball and the worst team on the other side of the ball.
Before we get to the history of teams like this, you're probably wondering how the Patriots ended up getting worse on defense after they held the Chargers to just 13 points. Much like Super Bowl LI, this was a game where the Patriots scored more points because they had more plays, even though they were no more efficient on a play-by-play basis. The Chargers gained 6.7 yards per play while the Patriots had only 5.0 yards per play. Take out Melvin Gordon's 87-yard touchdown, and the Chargers gained 5.2 yards per play, which is still higher than the Patriots. But the Patriots had a number of long, drawn-out drives that ended with field goal tries, while the Chargers had two touchdown drives with a grand total of five plays between them. The muffed punt safety cost them another possession, and the 51-yard drive to end the game still counts in DVOA even though the Chargers couldn't get the touchdown to complete the comeback attempt. The Chargers' only turnover on offense was the last interception, which we coded as Hail Mary and treat as a standard incomplete pass. The Patriots get 31.7% defensive DVOA on this game; they won it with special teams and by constantly moving the chains on those long drives.
So, back to the history here. No team has ever finished the year No. 1 on offense and dead last on defense. A few teams have come close. The 1986 Miami Dolphins were No. 1 on offense but were kept out of last place on defense by the worst defense ever measured by DVOA, the 1986 Tampa Bay Buccaneers. In 1987 and 1988, the Dolphins were last in defense but No. 2 on offense. The 2008 Denver Broncos, in Jay Cutler's best season, led the Brady-less NFL in offense but finished 31st on defense. The 2013 Chargers were second in offensive DVOA but had the league's worst defense. And perhaps the weirdest one: the Baltimore Ravens, in their first season, were the complete opposite of everything they have been since. Vinny Testaverde led the No. 1 offense in football but the defense was 29th in a 30-team league.
In fact, once we get past the first couple games of the season, very few teams manage to rank first on offense and last on defense for even just a single week. The last team to do so after September was... the New England Patriots back in 2011. That was a year similar to this one, with a ton of parity and a Patriots team where the dominant offense and some takeaways made up for a porous defense. The Patriots were No. 1 on offense and No. 32 on defense for two weeks, Weeks 15-16, but finished the year third on offense and third from the bottom on defense.
Other than the 2011 and 2017 Patriots, only two other teams since 1989 have been No. 1 on offense and dead last on defense after September. The 2000 Rams did it for a single week, Week 12. And the 2002 Chiefs had this combination in Weeks 4-5, Weeks 8-9, and finally in Week 12. Unfortunately, I don't yet have a week-by-week breakdown with changing opponent adjustments done for 1986-1988. I'm sure the Miami Dolphins did this at some point in those seasons, but I can't say for sure.
By the way, the reverse of this is also extremely uncommon. No team has ever finished the season No. 1 on defense and dead last on offense. Only three teams since 1989 have managed this combination at some point after September. The 1992 Seahawks had this combination in Week 16 but fell to third in defensive DVOA after the final game. The 2004 Miami Dolphins were winless with the league's worst offense but still had the best defensive DVOA after both Week 5 and Week 6. And the 2015 Broncos were last in offense but first in defense after Weeks 5-7 and again after Week 10.
[ad placeholder 3]
The Patriots drop from 13th to 15th after the win against the Chargers. I believe DVOA has the Patriots lower than any other advanced statistical system. Some of that comes from the fact that we're not longer incorporating any past performance or preseason projection. Some of it might come from the play-by-play breakdown that factors out things like opposing field goal kickers. (The Patriots are No. 2 in "hidden" special teams value behind the Rams.) It feels really strange to have the Patriots so low at midseason. But it turns out this is nothing new.
Three other times since the Brady/Belichick dynasty began, the Patriots have been 6-2 with a DVOA lower than what they have right now, and two of these years were very recent:
- In 2003, the Patriots were 13th overall with 7.7% DVOA. The defense was ninth overall but the offense was average. The Patriots were 4-1 in games decided by 8 points or less. Of course, they didn't lose a game the rest of the year, finishing 14-2, fourth in DVOA, and Super Bowl champions.
- In 2013, the Patriots were 13th overall with 7.2% DVOA. They had the best special teams in the league and were sixth on defense but shockingly just 20th on offense. They had gone 4-2 in one-score games. They finished the season 12-4 and fifth in DVOA, losing in the AFC Championship Game to Denver.
- In 2014, the Patriots were 12th overall with 7.0% DVOA. This time the offense was better (10th) but the defense was having problems (22nd). Unlike 2003 and 2013, the 2014 Patriots didn't build a 6-2 record on close wins. They were 2-0 in close games but were only 12th in DVOA because their 41-14 loss to Kansas City in Week 4 dragged them down significantly. This team also finished the season 12-4, fourth in DVOA, and won another Super Bowl championship.
Last week, some readers commented that I was trying to explain away the Patriots' poor performance. No, I don't believe that subjective ranking system is better than this, and I'm not going to use chat-acceptable spelling. But I find this Patriots season to be really fascinating. On one hand, what the Patriots are doing right now is extremely rare. On the other hand, what the Patriots are doing right now is something they've done over and over again. Before this season, Bill Belichick had been head coach of the New England Patriots for 17 years. In 12 of those 17 years, the Patriots had a higher DVOA at the end of the season than they had after Week 8. (The exceptions: 2000, 2002, 2007, 2009, and 2015.)
* * * * *
Once again this season, we have teamed up with EA Sports to bring Football Outsiders-branded player content to Madden 18. This week, we've got TWO sets of content: our weekly content for Madden Mobile and our monthly content for Madden Ultimate Team on consoles.
These players will get special Madden Mobile items branded as "Powerline, powered by Football Outsiders," beginning at 11am Eastern on Friday. The Football Outsiders stars for Week 8 are:
- DE Jadaveon Clowney, HOU (HERO): Sack, FF, 10 hurries according to Sports Info Solutions charting.
- G Jonathan Cooper, DAL: No sacks allowed; Cowboys had 23 carries, 96 yards, 61 percent success rate up the middle.
- WR Robby Anderson, NYJ: Tied for 3rd among WR with 59 DYAR (6-of-6, 104 yards, TD).
- CB Dre Kirkpatrick, CIN: 3 TFL including a sack; only targeted twice with no completions.
- DE Henry Anderson, IND: 5 solo tackles, sack, 2 hurries, blocked kick.
And here are the Football Outsiders October players for Madden Ultimate Team on consoles, which will go live this Saturday morning.
- QB Deshaun Watson, HOU: Led all starting quarterbacks with 42.0% passing DVOA in October. 293 passing yards and 36 rushing yards per game with 16 passing TD and only 5 interceptions.
- HB Aaron Jones, GB: Fourth among running backs with 74 rushing DYAR despite playing only three games in October. 99 yards per game, 6.1 yards per carry. 4 combined touchdowns (2 rushing, 2 receiving).
- TE Cameron Brate, TB: Second among tight ends with 105 receiving DYAR in October. 69.6 yards per game, 13.9 yards per reception, 3 TD.
- LG Rodger Saffold, LARM: Rams RB gained 5.1 yards per carry with 63 percent success rate on runs up the middle in October; Rams tied for third with 3.8 percent adjusted sack rate.
- C Travis Frederick, DAL: Cowboys RB gained 5.3 yards per carry with league-leading 66 percent success rate on runs up the middle in October; Cowboys tied for third with 3.8 percent adjusted sack rate.
- RT Ryan Ramczyk, NO: Saints RB gained 4.7 yards per carry with 62 percent success rate on runs to the right in October; Saints second with 3.4 percent adjusted sack rate.
- DT DeForest Buckner, SF: Tied for fourth in NFL with 11 defeats in October; top 20 for the season with 18 hurries according to Sports Info Solutions charting.
- LOLB Christian Kirksey, CLE: Led all defenders with 52 plays (tackles, assists, PDs) and 13 defeats in October.
- CB Ken Crawley, NO: Fifth in success rate, eighth in yards per pass according to Sports Info Solutions charting.
- SS Jahleel Addae, LACH: 19 run tackles and 7 tackles on failed pass completions in October, both third among NFL safeties.
- K Harrison Butker, KC: 18-of-19 field goals, 27-of-35 touchbacks on kickoffs.
- P Shane Lechler, HOU: 51.4 average gross yards per punt, 9 of 19 punts ending inside the 20.
We also have two special Football Outsiders midseason MVPs, who will be in Madden UItimate Team packs for a limited time this weekend.
- Midseason Offensive MVP: QB Carson Wentz, PHI*
- Midseason Defensive MVP: DE Calais Campbell, JAC
*OK, by Football Outsiders stats it's pretty clear that Tom Brady is the midseason MVP. He's over 200 DYAR ahead of the rest of the quarterbacks and has lifted the league's worst defense to a 6-2 record. However, he's also on the cover of the game and already the best pocket quarterback in Madden Ultimate Team. So think of Carson Wentz as our Midseason Offensive Most Valuable Player Who Isn't Already on the Cover of the Game.
* * * * *
All stats pages should now be updated through Week 8, including snap counts, playoff odds, and the FO Premium DVOA database. This week, we say goodbye to DAVE, our method which combines 2017 performance with our preseason projections. All numbers listed below represent 2017 only. However, just as I did last year, I used a light version of DAVE for the playoff odds simulation. The ratings used for playoff odds consist mostly of WEIGHTED DVOA, with a small amount of preseason forecast still included: five percent for teams that have played eight games, and nine percent for teams that have played seven games.
* * * * *
[ad placeholder 4]
These are the Football Outsiders team efficiency ratings through eight weeks of 2017, measured by our proprietary Defense-adjusted Value Over Average (DVOA) system that breaks down every single play and compares a team's performance to the league average based on situation in order to determine value over average. (Explained further here.)
OFFENSE and DEFENSE DVOA are adjusted to consider all fumbles, kept or lost, as equal value. SPECIAL TEAMS DVOA is adjusted for type of stadium (warm, cold, dome, Denver) and week of season. Opponent adjustments are currently at 80 percent strength; they will increase 10 percent every week through Week 10. As always, positive numbers represent more points so DEFENSE is better when it is NEGATIVE.
WEIGHTED DVOA represents an attempt to figure out how a team is playing right now, as opposed to over the season as a whole, by making recent games more important than earlier games.
To save people some time, please use the following format for all complaints:
<team> is clearly ranked <too high/too low> because <reason unrelated to DVOA>. <subjective ranking system> is way better than this. <unrelated team-supporting or -denigrating comment, preferably with poor spelling and/or chat-acceptable spelling>
- NON-ADJUSTED TOTAL DVOA does not include the adjustments for opponent strength or the adjustments for weather and altitude in special teams, and only penalizes offenses for lost fumbles rather than all fumbles.
- ESTIMATED WINS uses a statistic known as "Forest Index" that emphasizes consistency as well as DVOA in the most important specific situations: red zone defense, first quarter offense, and performance in the second half when the score is close. It then projects a number of wins adjusted to a league-average schedule and a league-average rate of recovering fumbles. Teams that have had their bye week are projected as if they had played one game per week.
- PAST SCHEDULE lists average DVOA of opponents played this season, ranked from hardest schedule (#1, most positive) to easiest schedule (#32, most negative). It is not adjusted for which games are home or road.
- FUTURE SCHEDULE lists average DVOA of opponents still left to play this season, ranked from hardest schedule (#1, most positive) to easiest schedule (#32, most negative). It is not adjusted for which games are home or road.
- VARIANCE measures the statistical variance of the team's weekly DVOA performance. Teams are ranked from most consistent (#1, lowest variance) to least consistent (#32, highest variance).
88 comments, Last at 09 Nov 2017, 6:21pm
#1 by DezBailey // Oct 31, 2017 - 9:26pm
Week 8 BES Rankings went out this morning - http://besreport.com/week-8-bes-rankings-2017/
Are Deshaun Watson's Texans the most spectacular sub-.500 we've seen in recent memory? The BES had them ranked 13th and the Seahawks 10th heading into Week 8. What a game!! And the Chiefs...despite their win over Denver...have slipped a bit in DAVE to No. 5. They've also dropped in the BES from No. 4 to No. 8. Not a good sign headed into that Week 9 clash with the Cowboys.
#88 by DezBailey // Nov 09, 2017 - 6:21pm
My apologies, Ninja. Honestly, DVOA among other metrics inspired me to create my own. So I routinely like to show and compare how the two have teams ranked. Also, I typed DAVE but met Weighted DVOA in regards to the Chiefs. Bottomline, I have a tremendous admiration and respect for FO and met no harm or disrespect with my comment.
#2 by Sophandros // Oct 31, 2017 - 9:31pm
Regarding the Super Bowl Odds:
A special Super Bowl, the "Black and Gold Bowl" between Pittsburgh and New Orleans would be fourth at 5.4%.
Sports talk radio and sports message boards are the killing fields of intellectual discourse.
#10 by dbostedo // Oct 31, 2017 - 11:10pm
True, but with the notable exception of a certain Wiz Khalifa song, the colors are always referred to as "black and gold".
Sometimes the team is referred to that way... as in "Today I'm rooting for the black and gold".
Does New Orleans do the same?
#34 by Sophandros // Nov 01, 2017 - 7:15am
LSU calls their colors "purple and gold", and that's basically the same yellow as Pittsburgh's.
Sports talk radio and sports message boards are the killing fields of intellectual discourse.
#28 by DRohan // Nov 01, 2017 - 4:48am
No, it's not bad. It normalizes the metric so every team's estimate is comparable.
But now that that's been cleared up, I'm surprised that it's the Saints with the highest Estimated Wins. The Rams and Eagles both have higher DVOAs and have had easier schedules.
#53 by rich006 // Nov 01, 2017 - 1:04pm
I thought the same until I read that the Estimated Wins is normalized to an average schedule. It seems the Saints are particularly good at the specific things Estimated Wins looks for. Without having the stats at hand, I'd say they have been quite good at red-zone defense, early-game offense, and they've been very good when the score is close late, although in their recent winning streak the scores have not often been close late.
#8 by Raiderjoe // Oct 31, 2017 - 10:25pm
Raiders at 19 nto too shabby. Ahjead of several teams with winnign record.s Team will continue to rise up the charets. AFC Wrst title not out of question. Thre ga,mes behind Chiefs btu play Ciefs once more so relaly only 2 games behind and have tiebreaker over Chiefs too. Even of Raiders dont' win divison, will at least be wild carf
#9 by Alternator // Oct 31, 2017 - 11:08pm
[New England] is clearly ranked [too low] because [Tom Brady is the Prince of Darkness, and shall unleash the damned upon all opposing defenses]. [Having friends in lower places] is way better than this. [Brady is getting really ****ing tired of people claiming that Bill is the evil one, I'm just saying.]
#11 by theslothook // Oct 31, 2017 - 11:16pm
This colts season is the most boring season I can remember. I've basically stopped watching them regularly and have switched to more interesting contests. That's made me wiser as a football fan, but killed my passion. I know more about the bucs and Vikings and cowboys than I ever have, but at a very real cost I use to bleed blue Sundays with emotion. Now? Not so much
Enjoy the time Pat's fans. The end will come sooner than you expect. Enjoy the emotional highs while they last.
Edit: speaking of which, Mike Zimmer may be my favorite coach in the league. I wish the colts had him. Will can have Wade, I'd rather have Zim.
#16 by Will Allen // Oct 31, 2017 - 11:48pm
Oh, I like Zim plenty, and he's a far superior head coach than Wade, I think. It isn't crazy to think the Vikings will go 5-3 from here, which will put the 4 year record for Zimmer at 37-27, with two division titles, while having two division rivals with clearly better qbs, and a couple bad injury seasons.
#41 by thewhitesnake8 // Nov 01, 2017 - 9:18am
Not to mention a billion eye surgeries...
This Vikings coaching staff has to be top 5 in the league, no?
All the things you mentioned, plus the consistency of the offense through so much QB/OL turnover, plus the shear amount of talent drafted and developed on defense.
Sports Illustrated did a story on Zimmer a few weeks back, interesting read.
#18 by CaffeineMan // Nov 01, 2017 - 12:14am
"Enjoy the time Pat's fans. The end will come sooner than you expect. Enjoy the emotional highs while they last."
Oh, yes, I totally am. I saw the crappy times and I know that when this time is over, it won't come again my lifetime.
#49 by Ben // Nov 01, 2017 - 11:55am
I'm glad it's not just me. I knew that this would be a "rebuilding" year for the Colts, but having Luck out just shows how much he has been covering for an awful coaching staff and roster over the last few years.
It's not the losses that are the issue, it's just that I have no interest in watching poorly played and coached football week after week. Getting beat because the other team has better players than your team is one thing. Getting beat because your team can't get out of it's own way is different. I can't comprehend how Cleveland fans have managed to survive over the years.
At least for Pat's fans, even after Brady retires, as long as the Hoodie is still around, they know they will see well coached football every Sunday. That will go a long way towards keeping fans engaged, I'd think.
#12 by Will Allen // Oct 31, 2017 - 11:21pm
The Eagles' back to back road games at Rams and at Seahawks may loom very large for HFA, which I think will likely be a big deal. The November 26th game is on Sunday night in Seattle, with a 930 pm eastern time kickoff. Seattle's considerable home field advantage will be larger than normal. Would NBC be able to flex the 12/3 Los Angeles game to Sunday night as well? That would suck for the Eagles, though I suppose they may stay out west after playing the Seahawks, which would mitigate the travel effects.
#20 by yuda // Nov 01, 2017 - 12:21am
They’re planning to stay out west. They actually requested getting 2 of their West Coast games back to back so they wouldn’t have to make the trip a full three times. But I have to think they were hoping for Rams and Chargers, not Rams and Seahawks.
I’m not convinced staying out west is better. I guess it depends what facilities they use to practice.
#23 by gomer_rs // Nov 01, 2017 - 1:11am
They ought to fly to Seattle and rent the facilities belonging to the University of Washington. They'd be closer to the CLINK then the Hawks. If they practice outside, Husky Stadium and the CLINK were built as copies of each other, so their kicker may get a feel for the weird winds for the cantilevered rooves on the open end of the stadium.
I remember when they were the Sea-chickens.
#45 by burbman // Nov 01, 2017 - 10:58am
CLINK is aligned in a N-S direction, while Husky Stadium is aligned E-W. Plus Husky Stadium is right on the lake, while CLINK is in the middle of the city. Winds would certainly not be similar. The cantilevered roofs would provide a good feel for how to deal with sound, but is would not matter since the Seattle game is first on the road trip. They would probably want to travel from Seattle to LA to enjoy the better weather.
#31 by Jerry // Nov 01, 2017 - 5:25am
You're off a week and an hour, Will. Eagles at Seahawks is Dec. 3 at 8:30 Eastern, with the Rams the following week. The Currently scheduled Sunday night game on the 10th is Ravens at Steelers, so flexing Eagles-Rams is possible.
#32 by Will Allen // Nov 01, 2017 - 6:05am
Yeah, had my time zones mixed up. The worst West Coast games, for road teams from other time zones, is the late Monday night game in week one. Vikings played in San Francisco a couple years ago with a 10 PM central time kickoff, and played easily their worst game of the year. 830 PM is more manageable, but it still would suck for the Eagles to get flexed into a 2nd consecutive Sunday night on the West Coast.
#33 by JoeyHarringtonsPiano // Nov 01, 2017 - 7:12am
I doubt they’ll flex the latter game. They usually only flex to go from a terrible game to a good game, not an okay game to a better game. And in any case, Fox may insist on holding onto Eagles/Rams as their national game in the 4pm eastern window.
#14 by Hoodie_Sleeves // Oct 31, 2017 - 11:32pm
The fact that the Patriots were able to run significantly more plays than the Chargers is strong evidence that their offense was significantly more efficient at producing first downs than the Chargers was.
The entire second half of that game was garbage time.
#21 by RickD // Nov 01, 2017 - 1:02am
That's the wrong measure for first down percentage.
If team A takes two plays to gain 10 yards and a first down and team B takes three plays to gain 10 yards and a first down, then at the end of both processes, both teams are in exactly the same place.
Another consideration: let's say Team A gets a 1st down on their first play, then another one on their second play, and then stall and punt. Team B has a TD drive where every set of downs requires a 3rd down conversion. Team A will have a first down on 40% of their offensive plays while Team B will have a first down on 33% of their offensive plays. But Team B is clearly playing better.
What you want is something like "first downs achieved per set of downs". At least in conjunction with other stats.
The Chargers/Pats game was weird in that the Chargers had a few drives where they were good at achieving new first downs on first or second down. But their third down conversion rate was considerably lower than the Patriots'. The Chargers converted 3 of 10 third downs while the Pats converted 9 of 19. (Both teams were 1 of 1 on 4th down.) That's not the relative rates one would expect given that the Chargers were getting more yards per play.
Ultimately the Pats had 27 first downs to 16 for the Chargers.
Basically the Chargers had a lot more variance on yards/play than the Pats did.
The interplay between mean and variance has interesting effects. To keep drives moving a coach might want lower mean yards/play if the variance were also lower.
#26 by Perfundle // Nov 01, 2017 - 2:21am
"But Team B is clearly playing better."
This is as simplistic an analysis as saying that the team that won the game was the better team. A team that never converts first downs on first or second down would be extremely fortunate to convert every single third down attempt, and would obviously be due some regression to the mean.
"What you want is something like "first downs achieved per set of downs"."
That sounds like something that penalizes deep passing offenses, and offenses that get gifted great field position. What you really want is percentage of third down plays among all plays (on drives that don't end in a TO). Because while good offenses convert third downs at a high rate, elite offenses don't allow drives to reach third down that often to begin with.
#43 by nat // Nov 01, 2017 - 10:14am
There have been some interesting discussions of developing a stat based on success per set of downs instead of per play - with degree of success being a combination of "did you get the first down?" and "how many extra yards did you get?". (That second bit deals with your objection, I think.)
DVOA is built per play. It's done that way because it gives a lot of data points, lets you do a ton of interesting splits, and adjusts for a lot of distinct situations. I don't think Aaron has ever opined on whether an opponent and situation adjusted Set-Of-Downs stat would be better or worse on VOA's criteria of explaining past success and DVOA's criteria of correlating with future adjusted success. The topic doesn't come up much because a Set-Of-Downs stat would clearly fail on the mission of allowing a feast of situational splits, so FO is never goping to drop its per-play orientation. It's pretty obvious that it would beat VOA on describing past success. Who knows whether it would beat DVOA for predicting the future?
A Set-Of-Downs stat would have fewer data points. But it would not need as many buckets for situations, and would probably have more data per bucket. It's biggest plus is that by and large football is in fact played in sets of downs: Advancing the ball 15 yards in one down or three downs is - in the real world - equally valuable. Either one gets you a first-and-ten fifteen yards further down the field.
It's an open question (I think) whether there exists something along the lines of being better or worse at sets of downs than at individual plays. It obviously happens in the short term, such as in this game. But is it at all predictive?
#52 by ClavisRa // Nov 01, 2017 - 12:33pm
DVOA is based entirely on reasonable sounding, but completely incorrect, assumptions about what is 'successful' of not about a given play. And it only takes into consideration limited factors.
Situational football. The most oft intoned Belichick mantra. Better than any team, the Pats coaches and players adjust to the situations of the game. They play aggressive when behind, and conservative when ahead, especially by two scores.
Ask the Chargers how successful they thought their offense was that game.
#15 by RickD // Oct 31, 2017 - 11:34pm
Well I'll say it. The 31% rating assigned the Patriots after holding the Chargers to 13 points underlines a limitation of the system.
The motivation of judging teams based on play-by-play efficiency presumes that this value will be fairly uniform over all plays. But that was clearly not the case in the Pats-Chargers game. The Chargers got 6 of their 13 points on one play where the Pats' defense blundered. Then their offense did basically nothing for the middle half of the game.
Let's look at things from a drive-by-drive perspective
1st drive: 10 plays, 47 yards, missed FG
2nd drive: 2 plays, 88 yards, TD
3rd drive: 3 plays, 9 yards, punt
4th drive: 4 plays, 32 yards, punt
kneel-down to end 1st half
5th drive: 8 plays, 17 yards, punt
6th drive: 7 plays, 24 yards, punt
7th drive: 3 plays, 63 yards, TD (includes 30 yard DPI not counted as 'play').
8th drive: 6 plays, 17 yards, punt
9th drive: 9 plays, 56 yards, INT (basically ran out of time)
If I were to bin these drives, I'd put the punt drives together, the TD drives together, and the first and last drives together. In five of their nine drives, the Chargers basically did nothing. The two TD drives featured very costly defensive breakdowns that led to massive yardage gains. The first and last drives were in the middle: both gave ample yardage but did not end up in any points allowed.
Seems like the two TD drives are counting more than the five punt drives, simply because they were so explosive. As for the last drive, a lot of the softness of the defense on that drive was clock-oriented. The Chargers' last play was from outside the 20 and they needed a TD. You may think that's "bad defense" but, really, I don't think "score from outside the 20 on demand" is very likely to happen.
Well, those are my thoughts. The Patriots don't play the same defense regardless of game situation so it's a bit simplistic to judge them as if that's what they're doing (or supposed to do). Yes they are flawed, but no, the defense they played on Sunday wasn't so bad that it should represent the expected level of the worst defense in the NFL. It was considerably better than that. Bad defenses don't force five punts in nine drives.
#19 by Craigo // Nov 01, 2017 - 12:15am
This does seem like one of those situations where the median would be a simple and effective solution (the old "Bill Gates walks into the room" problem).
Edit: OR would a harmonic or geometric mean be more appropriate, since the median will likely end up giving us values of 2 or 3 or 4 for both teams in any given game?
#22 by RickD // Nov 01, 2017 - 1:08am
For me the lesson is that we shouldn't any single statistic to be all-encompassing. Statistics is (for me) the art of projecting data from a high-dimensional space in a way that makes the information easier to digest. But if you project the data too much, you may be oversimplifying things. But this approach seems to be very popular in the field of sports statistics. This sounds like I'm making a pointed comment about DVOA, and that's not my intent. I find DVOA to be fantastic. But it's still just a statistic, and one statistic cannot be taken as every sufficient to adequately describe all the possibilities of a complicated system.
#37 by intel_chris // Nov 01, 2017 - 8:25am
I agree that DVOA is fantastic, and does a really good job of doing what it was intended to do. One of the problems with nearly any metric is that it quantitizes the data. That is part of the simplification. The scoreboard is simply a different quantization of the data, albeit the one that is the final judgement of the game in terms of standings.
The tendency to focus on rankings is a further quantization, which is why I am glad Aaron often gives spreads, n% better than the next team, or a bigger gap than between the 2nd and nth team, etc. That extra information puts back some of the details quantization has erased.
#24 by Perfundle // Nov 01, 2017 - 1:41am
"In five of their nine drives, the Chargers basically did nothing.
Bad defenses don't force five punts in nine drives."
Not all drives that end in punts are created equal. Using FO's definition of play success, exactly half of San Diego's plays that ended in punts were successful. Compare that to the Charger's last game against the Broncos, where only 3 of their 27 plays in drives that ended in punts were successful. That's actually doing nothing, whereas the Chargers just happened to have their successful plays in a very unlucky order against New England.
Overall, the Chargers had 28 successes to 24 failures (53.8% success rate), versus the Patriots with 38 successes to 44 failures (46.3%). To put this into perspective, the Patriots have a 47.5% success rate on offense so far, while the worst offense in Cleveland has 36.6%. So for San Diego to have more successes than failures is horrible defense, and worse than any of the Patriots' previous bend-but-don't-break iterations. Even their 2011 defense allowed just a 46.7% success rate, compared to 49.6% this year. And that's just success rate, which doesn't take into account how horrible some of those failures have been.
#39 by Anon Ymous // Nov 01, 2017 - 9:04am
I'll offer a more simplistic analysis. The game is clearly quirky given that DVOA thinks NE played more than 12% worse than they had been for the season up to that point! (edit: expressed as relative percentage, DVOA sees NE's defense as having been 60+% worse than their season average prior to the game.)
No chance that reflects reality.
#44 by Hoodie_Sleeves // Nov 01, 2017 - 10:21am
Right - DVOA sees that game as an almost historically bad defensive effort - in a game where:
1)the defense gave up 13 points, 6 of which were in garbage time.
2)the defense forced 3 turnovers (2 fumbles, and an int)
3)the defense never allowed the opponent into the red zone (both scores were on broken plays).
4)the defense held the opponent to 20 minutes of possession
What's happening here is that the Chargers were so inefficient at converting first downs that the handful of fluky long plays are overwhelming the overall sample of plays - essentially the Chargers didn't get enough first downs to rack up enough negative plays to offset Gordon's run.
#57 by sbond101 // Nov 01, 2017 - 2:21pm
This game is a letter perfect example of the limitations of DVOA's per-play approach, and very badly miss-states the quality of the NE defenses play. The per-drive analysis backs that up, the win-probability chart backs that up, and all subjective analysis backs that up. DVOA says this was a top 10/season bad defense effort, It was definitely at least average, DVOA is wrong because it's looking at data the wrong way for games like that. (That doesn't mean DVOA isn't a fascinating metric).
What I think is more interesting is the larger discussion of previous seasons where BB's Pats have had middling DVOA's and good records at the half way point, and in every single case that middling DVOA has not been predictive of the outcome of the season winning the Superbowl 2/3 times and going a combined 20-4 in the second half of those seasons; There is a ton of evidence to suggest that first half DVOA is predictive of second half success in general, and some to suggest that it is more predictive than first half record. I think some outlier analysis of situations where first half DVOA failed to predict second half DVOA (while attempting to control-out injuries) would be a really interesting exercise. It's really not obvious to me whether the dominate factor in teams which follow this pattern is that they play differently (better/worse) in the second half, or share attributes which DVOA struggles to effectively measure. It might even lead to some better approaches to look at the evolution of teams through the year and some insights as to what, if any, weighting mechanisms for first-half games should be used when looking at DVOA's ability to predict playoff or future season results.
#40 by nat // Nov 01, 2017 - 9:05am
So think of Carson Wentz as our Midseason Offensive Most Valuable Player Who Isn't Already on the Cover of the Game.
It's a good thing you added this caveat. Else we would have thought you had gone totally Brady-hater-nuts on us.
Still, there are five other QBs who best Wentz in both DYAR and DVOA, and another two who beat him in DVOA. Picking Wentz is kind of bandwagon-y.
#67 by bobrulz // Nov 02, 2017 - 2:40am
I would agree, but looking at the specific situations, I'm not sure who I would choose over Wentz.
Alex Smith has been great, but I don't think it's a coincidence that his sudden greatness has directly coincided with the emergence of Kareem Hunt. Obviously Smith has to take some credit, but the Chiefs are a remarkably well-rounded team.
Drew Brees is just being Brees, and he should be in the discussion, but the Saints' record can be most attributed to them finally having a decent defense.
Roethlisberger has clearly had his struggles even though he shows well in DVOA. In a lot of ways, he feels like the weak link of the Steelers this year. At the very least, while he's only rarely hurting the team, the offense doesn't seem to flow around him quite as much.
Keenum has been impressive, but of course, he's Case Keenum....I always thought he was an underrated QB, but it's hard to say that he'll sustain quite THIS level of success.
Deshaun Watson has been very impressive and is a lock for ORotY barring a remarkable Kareem Hunt outburst or sudden drop-off/injury to Watson, but the Texans are 3-4 and a winning record/playoff appearance is essential for MVP.
Matt Ryan is better in DVOA, but does it really feel like he's having an MVP performance?
Wentz is playing on the team with the best record, and beyond the impressive traditional stats and good but not top tier DVOA, he just seems to pass the "eye test". But I think the 7-1 record for the Eagles is the most important factor. At this point, I would still give MVP to Brady, but Wentz would be 2nd (and Brees probably 3rd).
#74 by Richie // Nov 02, 2017 - 1:32pm
Yeah, instinct is to think "wow, Wentz is really playing well and the Eagles are 7-1, so he's probably the MVP".
But thinking a little more and you realize that the Patriots are 6-2 despite having the worst defense in the league, Julian Edelman out and Rob Gronkowski missing playing time. It's boring, but Brady is surely the MVP so far.
#47 by PirateFreedom // Nov 01, 2017 - 11:19am
Has there ever been any attempt to incorporate clock into success rate?
Often we see a team with a lead repeatedly run the ball into a wall of defenders just to keep the clock running. Play calling that would be idiotic in the first quarter but appropriate near the end of the game.
#50 by nat // Nov 01, 2017 - 12:12pm
It's been discussed. But it's impossible to get right. With X:XX left, should you be running out the clock, or preserving time for a potential answering FG, or taking risks to extend your own drive to ice the game? That's not a question you can answer by looking at league averages. It depends entirely on the two teams involved and how they're doing in that game.
DVOA tries to handle this by comparing a play's success to baseline (aka league average) success in that situation with that much time left. This means there is a lot of noise in the baseline data that has nothing to do with the difficulty of the situation. You are comparing each play to a mix of "run out the clock" plays and "go for the first down" plays.
DVOA's approach is better than nothing. But it does mean that as the clock winds down in close games, the "A" in DVOA becomes less and less useful as a baseline. Consequently, late game DVOA can be misleading, occasionally extremely so.
#51 by JMM // Nov 01, 2017 - 12:29pm
There have been some non-footballoutsiders attempts. One notable attempt is Chris Burke's system that is (now at ESPN and the % chance of winning graph in their game cast) based on time remaing, down, distance, yard line and score. The change in chance of victory is roughly an analog to success rate.
Also there is an inherent value assessed to time compared with yardage as most coaches will call a time-out, which is worth some seconds between plays and can possibly be quantified, to save a 5 yard penalty. Except when punting within the punters normal range, I'm not aware of any occasions when a coach has saved a time out (and 30-40 seconds of future game time) in favor of 5 yards. There have to be some situation when it makes sense to save the time out and sacrifice the yards.
#48 by MilkmanDanimal // Nov 01, 2017 - 11:53am
I suspect the Bucs will transform the Awful Eight into the Nasty Nine after a couple more games of Winston failing to deal with that injured shoulder, and he'll be put on IR just to save any more abuse. No specific reason to think that other than it gives announcers an excellent chance to set a new single-season record for saying the word "Harvard".
#63 by ammek // Nov 01, 2017 - 5:09pm
A propos of the Patriots' defense, an observation: the number of passes per game thrown at New England has been almost equally divided among the five categories in FO's chart, ranging from 7.6 (targeting running backs) to 7.2 (#1 wide receiver). That seems unusual.
Aaron mentions that the Patriots have gained more from hidden value in special teams than any team other than the Rams. But look who's there at #3! Why, it's the Miami Dolphins:
Miami's special teams advantage in "elements ... generally out of their control":
2017: +7.8 points (rank: 3rd)
2016: +21.0 (1st)
2015: +18.8 (1st)
2014: +15.3 (2nd)
2013: +11.9 (3rd)
2012: +3.7 (7th)
2011: +11.5 (4th)
That's an average of +14 points a year over seven years. Methinks this isn't completely random:
Oakland's special teams advantage in "elements ... generally out of their control":
2017: +3.0 points (rank: 9th)
2016: +13.2 (2nd)
2015: +6.4 (2nd)
2014: +6.3 (6th)
2013: +13.5 (2nd)
2012: -7.4 (27th)
2011: +9.2 (5th)
2010: +16.2 (1st)
2009: +3.8 (9th)
2008: +17.5 (3rd)
2007: +13.5 (3rd)
The Raiders have six top-three finishes and are on course for a tenth top-10 finish in 11 seasons.
What could be causing these streaks of good fortune?
#70 by panthersnbraves // Nov 02, 2017 - 9:05am
The Panthers/Falcons game is going to have more meaning than I thought. The Panthers want to hold onto the Wildcard and chase down the Saints, but it appears that the Falcons are at a point that a loss pretty much puts their playoff odds in "slim" territory?
#72 by eggwasp // Nov 02, 2017 - 11:11am
Here's an interesting area - as a Raiders fan, and certainly the chatter around the Raiders is that they are playing badly because of the decline in the offense vs last year. That conclusion certainly passes the eyeball test anyway.
BUT DVOA has Oakland basically playing the same on offense this year as last year. Is this strength of schedule, field position, turnovers etc or what is it? Also - amongst all the talk about Carr's regression, he's moved from 19% DVOA to 14% (rank from 7 to 11) - not that much of a difference really. I'd be interested in hearing FO's opinion on what's going on?
#82 by theslothook // Nov 03, 2017 - 5:24am
I want to respond to the thread about dvoas blindspots vis a vi the Pats Chargers. If a team fails on a ton of 3rd and 1s and has to punt, is that good defense? Maybe, but if the game is simulated 1000 times, is the mean result that the pats defense again gets the upper hand in those situations? And why are they getting to such short yardage in the first place? It presumes a defense willingly concedes the yardage but stiffens when it needs to.
I have long been a skeptical that, save for Garbage time, defenses willingly concedes yardage in order to stop them later. The bend but don't break defense feels like a post hoc explanation when a team gives up a ton of yards but fails to score in the red zone. Well, the red zone is the second hardest place to move the ball(besides your own goal line). I've looked at teams with large disparities in red zone dvoa and overall dvoa. It simply does not persist over time.
In short, I agree partially w RickD. Dvoa is a stat, a quantitative parable, not a direct representation of reality. But, I disagree that it's flawed because a defense forces a bunch of punts and garbage time tds. A 3 n out w no yardage conceeded is very different from a 3 n out with 9 yards conceded. Same net result, but the devil is in the details
#86 by sbond101 // Nov 03, 2017 - 10:07am
Your statement above might be true (I don't intend to analyze it), but it doesn't directly bare on this example. The argument here is about looking at results per-drive vs. per play; If we consider a simple 3 drive sample, 2 3 & outs gaining 8 yard between them and a 1 play 80 yard touchdown drive vs. 2 3 & outs and a 2 play 80 yard touchdown drive (2 40 yard plays). which is "worse" defense.
The basic argument here is that when your measuring success per-play you need to add fairly arbitrary "success" bonuses for things like acquiring 1st downs or "staying on schedule" as it used to address the fact that the number of plays an offense gets is not an independent variable. In reality the reward for a "staying on schedule" or "first down" play should flex depending on the relative quality of the offense and defense in question as the number of points likely to be scored dictates the statistical merits of a high-variance result set vs. a low variance result set. Measuring success per-drive gets away from the need to make those inferences (but has other problems like small sample size). The reasonableness of those assumptions is what I think would be interesting to explore in a complex win-correlation analysis looking at DVOA outliers like the 2003 Pats.
The 2017 Patriots over the last 4 games are playing a brand of defense which is extremely high-variance (as opposed to the first 4 games where they just played awful defense); Based on a review of the PFR data their were 40 opponents drives which had a chance of success (i.e. not with only a tiny amount of clock left in the game/half) - I've left the garbage-time data in for this analysis. The results 14 punts and 1 turnover within the first two sets of downs, 6 turnovers & 2 punts on longer drives 7 touchdowns and 1 FG (4 "makeable" field goals blocked/missed). That's getting the opponent off the field quickly 35% of the time, getting a turnover 17.5% of the time, giving up a touchdown 17.5% of the time and a chance for a field goal 12.5% of the time; Put another way 17.5%/35% clear success (turnover/quick punt), 7.5%/17.5% clear failure (short FG try/touchdown), 22.5% long FG try/change in field position. In an average game each team gets 11.5 drives, weighting touchdowns at 7, short FG's at 3, long FG's/field position changes at 1.5 and turnovers at -1.5, that's a defense that surrenders 16.7 PPG against a pretty average sampling of opponents. It gets better if you eliminate the garbage time vs. the Falcons. If the Pats continue to play that way on defense they'll win the vast majority of their games. DVOA "thinks" it's awful defense; DVOA is wrong; and the limitations of the "success" assumptions that are required to build a per-play approach are reason. This is definitely unusual, and how unusual is a really interesting subject for study.
N.B. The average remaining on the 7 3 & outs in the above sample is 4.9 yards, so DVOA is definitely not capturing a lack of repeatability in those short 3 & outs, and I think it's clear the Pats defense is definitely not "stiffing up" in scoring areas. This is a totally different and unrelated phenomena.
#87 by theslothook // Nov 04, 2017 - 2:41am
I guess there are two issues you are bringing up as I understand them.
First that dvoa makes some arbitrary judgements about how to define success. I can't really argue against this - I'm not the creator of the stat. But, I have to think those weights are determined statistically by how well they forecast the future. In that case, they aren't entirely "arbitrary", though I'll concede that they may be flawed.
As to the Patriots - one can interpret the results you've posted in two ways. 1) that they executed a high variance strategy that's paid off. 2) They've been fortunate to force these 3 n outs by stiffing at the right time, but basically otherwise playing with fire and that sort of thing is unsustainable.
In theory - dvoa is meant to suss out 1 from 2. That's why I'm defending it. It suggests that the forced punts have more elements of luck to them than a demonstration of skill. We can agree or disagree on whether it does so correctly, but I don't think it means DVOA is fundamentally ignoring something.