Writers of Pro Football Prospectus 2008

Most Recent FO Features

PrescottDak16.jpg

» The Week In Quotes: August 26, 2016

As actual NFL football returns to our lives, we have observations on good quarterback play in Dallas, bad quarterback play in Denver, the Olympics, baseball, taxes, and mermaids.

18 Nov 2014

Week 11 DVOA Ratings

by Aaron Schatz

Football Outsiders' DVOA ratings take the long view of things, and so one loss won't sink a team in our ratings. Even two losses won't sink a team if that team was good enough before things went sour. Such is the case right now with the Denver Broncos. The Broncos are still No. 1 in DVOA despite losing two of their last three games. But while the Broncos are still on top, the distance between them and the rest of the league has changed dramatically. Three weeks ago, Denver was at 47.6% DVOA, 21 percentage points ahead of No. 2 Baltimore. Now the Broncos are at 31.5%, less than 8 percentage points ahead of the new team at No. 2, Green Bay. The Packers also pulled ahead of the Broncos this week to lead the league in offensive DVOA, although the Broncos are still pulling off the rare feat of being ranked in the top two in both offense and defense.

With the Broncos now fallen to Earth, we're back to where we were earlier in the season, with 2014 as the year where no single team is standing out in any particular way good or bad. How packed together is the league this year? Well, the Broncos are now the worst No. 1 team ever through 11 weeks of the season, narrowly behind the 1989 Cleveland Browns who led the league at 31.6% DVOA after 11 weeks. And on the other side of the league, Tampa Bay is now very close to being the best last-place team in DVOA history, although they are narrowly inched out by the 2001 Lions and the 1997 Saints.

Lowest-Rated Teams Ranked No. 1 in DVOA
as of Week 11, 1989-2014
x Highest-Rated Teams Ranked Last in DVOA
as of Week 11, 1989-2014
Year Team W-L DVOA x Year Team W-L DVOA
2014 DEN 7-3 31.5% x 2001 DET 0-10 -27.2%
1989 CLE1 7-3-1 31.6% x 1997 NO 3-7 -29.0%
2006 SD 8-2 32.2% x 2014 TB 2-8 -29.1%
2011 HOU 7-3 32.6% x 1995 ARI 3-7 -32.2%
2000 TEN 8-2 32.6% x 1994 TB 2-8 -33.8%
2005 IND 10-0 33.7% x 2006 ARI 2-8 -35.9%
1993 PIT 6-3 34.8% x 1990 TB 4-7 -36.7%
1997 SF 9-1 35.4% x 1989 DAL 1-10 -37.8%
2008 NYG 9-1 35.5% x 1992 NE 1-9 -40.2%
2010 PHI 7-3 35.8% x 2003 ARI 3-7 -40.6%

It's really been a strange year overall. We're seeing more blowouts than usual. (I apologize to Mike Tanier for not remembering which of his articles to link to, but he pointed out a few days ago that there have been about two more blowouts every three weeks of games compared to years past.) It seems like half the league is grouped together at 6-4 or 7-3, which has led to a lot of complicated tiebreaker scenarios that I explained over at ESPN Insider today. Check out the FO playoff odds and you will notice that we actually have Kansas City now with a better chance to make the playoffs than Denver, even though Denver still has better odds to win the AFC West and is still our No. 1 Super Bowl favorite. We also end up with Pittsburgh having slightly higher playoff odds than Baltimore even though the Ravens have a slightly higher average of projected wins.

And, as I keep recounting every week, we're dealing with huge differences between the Football Outsiders DVOA ratings and conventional wisdom about which teams are best this season. Nowhere is that more apparent than with the two teams that currently lead each conference, New England and Arizona.

One element of this, of course, is recency bias. Again, as I keep recounting every week, DVOA ratings tend to take a very long-term view of things with a larger sample of games. Even the weighted DVOA ratings count the last eight weeks of games at full-strength or almost full-strength. New England and Arizona don't just have great records; they've also had their best games in recent weeks. Both teams had their two best single-game DVOA ratings of the year in Week 9 and Week 11.

New England moves from No. 8 to No. 5 this week with its big win over Indianapolis. Obviously, it's still being seriously dragged down by that awful Kansas City game back in Week 4. However, you can't pretend that game didn't happen and not pretend the same for other teams. Let's say that we got rid of the worst game of the year from every team in the league. Would New England now rank No. 1 in DVOA? Nope. The Patriots would move up one space, to No. 4, with Baltimore dropping down to the No. 5 spot. If the Patriots get to forget about their loss to Kansas City, then the Dolphins get to forget about their 29-10 loss to Buffalo back in Week 2. And it's only fair that we also let the Packers write off their 44-23 loss to New Orleans in Week 8 as a fluke, although that was a bit more recent.

As for Arizona, the Cardinals have the lowest DVOA rating of any 9-1 team in DVOA history.

Lowest Total DVOA by 9-1 Teams, 1989-2014
Year Team DVOA after
Week 11
Rk Final W-L Final DVOA Playoff Result
2014 ARI 4.4% 15 -- -- --
2012 ATL 4.8% 12 13-3 9.1% Lost NFC Championship
2013 KC 11.3% 9 11-5 17.5% Lost Wild Card
2006 IND 13.3% 9 12-4 16.3% Won Super Bowl
1998 MIN 17.5% 5 15-1 23.1% Lost NFC Championship
1991 BUF 19.3% 5 13-3 19.1% Lost Super Bowl
2012 HOU 20.1% 7 12-4 6.7% Lost Divisional Round
1990 BUF 22.5% 6 13-3 21.2% Lost Super Bowl
2011 SF 23.0% 3 13-3 18.6% Lost NFC Championship
2007 GB 27.1% 4 13-3 19.8% Lost NFC Championship
2009 MIN 27.6% 5 12-4 18.5% Lost NFC Championship

In fact, the Cardinals aren't just the lowest-rated 9-1 team in DVOA history. They're almost the lowest-rated 9-1 or 8-2 team in DVOA history. Only two 8-2 teams ever had lower DVOA ratings than the Cardinals: the 2000 Minnesota Vikings at -8.1% and the 1997 Minnesota Vikings at 3.5%.

Should we make anything of the fact that the three lowest teams on this list are from the last three seasons? Probably not. The Cardinals' situation doesn't feel much like the Falcons of two years ago. That team picked on an easy schedule (ranked 30th through 11 weeks) and had a lot of close wins. The Cardinals, on the other hand, have played an average schedule. They haven't had any really big, dominant wins except maybe the 31-14 victory over St. Louis, but they also aren't winning by a field goal week after week. (I compared the Cardinals on Twitter this week to the 2003 Carolina Panthers, but that team was winning by much closer scores.) Fumble luck was a bit of an issue, but it is less of a deal now after Detroit fumbled twice this week and recovered both of them.

Over to the right, I've got one of our week-to-week graphs showing the single-game DVOA for the Cardinals this season. I've added the scores of each game, and I've colored their four road games black with the six home games white. What you see with this chart is just how consistently close to 0% the Cardinals were until the last three weeks. Again, that seems kind of weird when they were winning every game. Arizona is actually second in the league in variance. Only Oakland has more consistent this year on a week-to-week basis. (For the Raiders, that means "consistently lousy but not prime-time blowout lousy." So nice that they're about to play in prime time, huh?)

I'm not completely confused by the Cardinals. What baffles me is not that their record is better than their DVOA rating, but just how much better it is. I wouldn't be baffled if the Cardinals ranked around eighth or ninth in DVOA. Subjectively, they certainly don't seem to be playing like the best team in the league. But 15th? After diving through the spreadsheets I can't even really pick out elements where I can explain that it has to do with a certain down or situation.

Actually, that's not quite true. There's one thing that stands out to me about a lot of the teams historically that have outperformed their DVOA ratings: a lot of them seem to be dome teams. The lowest-rated 8-2 and 9-1 teams in DVOA history all seem to be Atlanta or Minnesota, and we've underrated a number of different Indianapolis teams as well. It's definitely possible that something is fishy here with the adjustments for playing indoors that I added to the team DVOA formula a couple of years ago. Those adjustments are different for offense and defense and it is possible that this is unfairly hurting dome teams. Unfortunately, I won't have time to truly check that out and tweak the formula if necessary until the offseason. However, if the dome is the issue, it's not a big issue. I tried coding every Arizona game as "warm" instead of "dome" and their total DVOA still only rose by a couple of percentage points.

Tangentially, this ties into something that was discussed thoroughly in the comment thread from last week's DVOA ratings, and that regards the predictive value of DVOA. It's no secret that our premium picks against the spread have now struggled for two straight years. They finally had a good week this week at 9-5, but that doesn't change the fact that they've been terrible all season. That winning week puts us at 67-88-6, which is gruesome. When you combine that with the fact that this year's DVOA ratings disagree so much with conventional wisdom, there seemed to be some suggestion that DVOA is no longer a valid statistic for judging football teams and how well they might perform for the rest of the season.

Is DVOA supposed to be predictive? The answer is "yes, somewhat." DVOA is generally trying to balance two things. First, we want the non-adjusted rating to correlate with wins and losses. Second, we want the adjusted rating to correlate with future performance. Combine these things, and you are trying to measure what I have referred to as "the platonic idea of team quality." I've written about this in the past and I'll write more about it again in the future, I'm sure. But for now, on this specific point, a small mea culpa. The picks we make in the premium section of the site are not based solely on DVOA. They're based on more complicated formulas, and those formulas change a couple times during the season as we get more information on teams. Given the way the the picks have performed the past two years, there's no question that we need to go in and reconfigure those formulas before next season to try to improve things. There are also certainly still elements in the DVOA rating itself that could be changed to improve the accuracy, and we need to spend the time playing with those numbers to try to improve things in the future. If you've followed Football Outsiders for a few years, you know that we do bring out an improved version of our formulas from time to time. The fact is that over the last year or two, I haven't been able to do any work to improve either the main DVOA system or the premium picks formulas because of health problems. I'm hoping to be healthy enough come next spring to really overhaul and improve lots of things around here. I want our readers to feel that they are getting the best I can give them. When readers suggest improvements in the spirit of constructive criticism rather than obnoxious trolling, I truly appreciate it. Unfortunately, I just haven't been able to play with a lot of those ideas. Just be aware that we're not being willfully ignorant of our shortcomings. There are time constraints and personal difficulties that we also must deal with.

* * * * *

Last week's DVOA discussion threads also had some questions about what we mean when we say that an NFL team is "three parts offense, three parts defense, and one part special teams." Actually, that's now four parts offense. To try to explain, I'll copy here a segment from the Pregame Show article in Football Outsiders Almanac 2014.

There are three units on a football team, but they are not of equal importance. Our DVOA ratings provide good evidence for this. For a long time, the saying from Football Outsiders was that the total quality of an NFL team is three parts offense, three parts defense, and one part special teams. Further recent research suggests that offense is even more important than we originally believed. Recent work by Chase Stuart, Neil Paine, and Brian Burke suggests a split between offense and defense of roughly 58-42, without considering special teams. Our research suggests that special teams contributes about 13 percent to total performance; if you measure the remaining 87 percent with a 58-42 ratio, you get roughly 4:3:1. When we compare the range of offense, defense, and special teams DVOA ratings, we get the same results, with the best and worst offenses roughly 130 percent stronger than the best and worst defenses, and roughly four times stronger than the best and worst special teams.

For those who were confused last week: this is already baked into the DVOA ratings. We don't multiply offense by one variable and defense by another or anything along those lines. Total DVOA is simply (offense - defense + special teams). It's the general ranges of those ratings that give us the "4:3:1" concept.

* * * * *

Once again in 2014, we have teamed up with EA Sports to bring Football Outsiders-branded player content to Madden 15 Ultimate Team. Each week, we'll be picking out a handful of players who starred in that week's games. Some of them will be well-known players who stood out in DVOA and DYAR. Others will be under-the-radar players who only stood out with advanced stats. We'll announce the players each Tuesday in the DVOA commentary article, and the players will be available in Madden Ultimate Team packs the following weekend. We will also tweet out images of these players from the @fboutsiders Twitter account on most Fridays. One player each week will only be available for 24 hours from the point these players enter packs on Friday.

The Football Outsiders stars for Week 11 are:

  • RB Jamaal Charles, KC (24-HOUR HERO): Led all Week 11 RB with 83 DYAR (158 rushing yards, 19 receiving yards, 2 TD).
  • SS Morgan Burnett, GB: Led Packers defenders with 8 combined tackles in 53-20 rout of Philadelphia.
  • LT Jermon Bushrod, CHI: Allowed no sacks to Vikings, who entered game third in NFL in sacks.
  • C Bryan Stork, NE: Helped Patriots running game dominate Indianapolis, including 99 yards on 18 carries up the middle with 83 percent Success Rate.
  • K Greg Zuerlein, STL: 5-for-5 on field goals including two of 50+ yards; seven kickoffs resulted in five touchbacks and two returns that didn't make it past the 15.

Strangely enough, this will be the first time Bryan Stork is available in Madden Ultimate Team. He was mistakenly left out of the original base set of players despite being a fourth-round pick.

* * * * *

All stats pages are now updated with Week 11 information -- or will be in the next few minutes -- including FO Premium and playoff odds. You can also read the new weekly playoff odds report on ESPN Insider to get more commentary on the current playoff odds. Snap counts are not yet updated because of an NFL error regarding the Indianapolis-New England numbers, but we'll try to get those fixed as soon as the league does.

* * * * *

These are the Football Outsiders team efficiency ratings through 11 weeks of 2014, measured by our proprietary Defense-adjusted Value Over Average (DVOA) system that breaks down every single play and compares a team's performance to the league average based on situation in order to determine value over average. (Explained further here.)

OFFENSE and DEFENSE DVOA are adjusted for strength of schedule and to consider all fumbles, kept or lost, as equal value. SPECIAL TEAMS DVOA is adjusted for type of stadium (warm, cold, dome, Denver) and week of season. WEIGHTED DVOA represents an attempt to figure out how a team is playing right now, as opposed to over the season as a whole, by making recent games more important than earlier games. As always, positive numbers represent more points so DEFENSE is better when it is NEGATIVE.

To save people some time, please use the following format for all complaints:

<team> is clearly ranked <too high/too low> because <reason unrelated to DVOA>. <subjective ranking system> is way better than this. <unrelated team-supporting or -denigrating comment, preferably with poor spelling and/or chat-acceptable spelling>

TEAM TOTAL
DVOA
LAST
WEEK
WEIGHTED
DVOA
RANK W-L OFFENSE
DVOA
OFF.
RANK
DEFENSE
DVOA
DEF.
RANK
S.T.
DVOA
S.T.
RANK
1 DEN 31.5% 1 30.4% 1 7-3 18.5% 2 -17.2% 2 -4.2% 27
2 GB 23.9% 3 24.9% 2 7-3 22.5% 1 -1.6% 10 -0.2% 16
3 BAL 22.0% 2 23.3% 3 6-4 7.9% 12 -5.7% 7 8.4% 1
4 MIA 20.0% 6 22.1% 4 6-4 8.1% 10 -16.0% 3 -4.1% 26
5 NE 19.6% 8 20.6% 5 8-2 13.8% 4 -0.1% 13 5.8% 4
6 KC 17.1% 10 20.0% 6 7-3 12.7% 7 0.4% 14 4.8% 7
7 SEA 15.1% 4 12.2% 7 6-4 13.1% 6 -5.3% 9 -3.3% 23
8 PHI 9.6% 5 9.8% 8 7-3 -3.4% 18 -5.6% 8 7.4% 2
9 DET 8.9% 9 8.6% 9 7-3 -7.5% 22 -22.3% 1 -5.9% 31
10 PIT 6.9% 11 6.9% 11 7-4 16.9% 3 8.8% 28 -1.2% 20
11 SF 6.8% 16 7.3% 10 6-4 -3.2% 17 -15.5% 4 -5.5% 29
12 BUF 6.8% 13 5.9% 13 5-5 -12.0% 27 -13.2% 5 5.5% 5
13 DAL 5.4% 14 6.0% 12 7-3 11.3% 8 4.4% 21 -1.4% 21
14 IND 5.1% 7 5.3% 14 6-4 6.6% 13 7.3% 26 5.8% 3
15 ARI 4.4% 15 4.6% 15 9-1 -7.8% 23 -12.1% 6 0.1% 15
16 NO 2.6% 12 3.2% 16 4-6 13.6% 5 11.4% 30 0.4% 14
TEAM TOTAL
DVOA
LAST
WEEK
WEIGHTED
DVOA
RANK W-L OFFENSE
DVOA
OFF.
RANK
DEFENSE
DVOA
DEF.
RANK
S.T.
DVOA
S.T.
RANK
17 CIN 0.3% 20 -2.9% 17 6-3-1 -1.0% 15 3.7% 20 4.9% 6
18 SD -3.1% 18 -3.8% 18 6-4 8.3% 9 11.8% 31 0.4% 13
19 CLE -3.6% 17 -4.6% 19 6-4 -2.2% 16 1.0% 16 -0.3% 17
20 ATL -4.1% 19 -6.6% 20 4-6 7.9% 11 15.8% 32 3.7% 8
21 HOU -9.1% 23 -7.9% 21 5-5 -3.9% 19 0.6% 15 -4.6% 28
22 NYJ -11.2% 24 -10.7% 23 2-8 -13.5% 28 -0.1% 12 2.3% 9
23 CHI -11.3% 26 -15.2% 26 4-6 3.1% 14 8.2% 27 -6.2% 32
24 STL -11.7% 28 -9.6% 22 4-6 -9.4% 25 2.9% 18 0.6% 12
25 NYG -13.9% 21 -13.9% 25 3-7 -8.8% 24 4.7% 22 -0.4% 18
26 MIN -15.2% 25 -13.7% 24 4-6 -17.1% 29 -0.4% 11 1.5% 11
27 TEN -16.0% 27 -15.7% 27 2-8 -6.0% 20 9.0% 29 -1.0% 19
28 WAS -17.7% 22 -20.5% 28 3-7 -6.7% 21 5.5% 23 -5.5% 30
29 CAR -19.7% 29 -21.8% 29 3-7-1 -10.9% 26 6.7% 25 -2.1% 22
30 OAK -23.5% 30 -23.4% 30 0-10 -22.3% 31 3.0% 19 1.9% 10
31 JAC -27.6% 31 -24.6% 31 1-9 -22.5% 32 1.3% 17 -3.8% 24
32 TB -29.1% 32 -25.2% 32 2-8 -19.4% 30 5.9% 24 -3.9% 25
  • NON-ADJUSTED TOTAL DVOA does not include the adjustments for opponent strength or the adjustments for weather and altitude in special teams, and only penalizes offenses for lost fumbles rather than all fumbles.
  • ESTIMATED WINS uses a statistic known as "Forest Index" that emphasizes consistency as well as DVOA in the most important specific situations: red zone defense, first quarter offense, and performance in the second half when the score is close. It then projects a number of wins adjusted to a league-average schedule and a league-average rate of recovering fumbles. Teams that have had their bye week are projected as if they had played one game per week.
  • PAST SCHEDULE lists average DVOA of opponents played this season, ranked from hardest schedule (#1, most positive) to easiest schedule (#32, most negative). It is not adjusted for which games are home or road.
  • FUTURE SCHEDULE lists average DVOA of opponents still left to play this season, ranked from hardest schedule (#1, most positive) to easiest schedule (#32, most negative). It is not adjusted for which games are home or road.
  • VARIANCE measures the statistical variance of the team's weekly DVOA performance. Teams are ranked from most consistent (#1, lowest variance) to least consistent (#32, highest variance).



TEAM TOTAL
DVOA
W-L NON-ADJ
TOT VOA
ESTIM.
WINS
RANK PAST
SCHED
RANK FUTURE
SCHED
RANK VAR. RANK
1 DEN 31.5% 7-3 33.1% 8.7 1 1.9% 12 2.9% 10 15.6% 15
2 GB 23.9% 7-3 24.5% 7.0 5 -1.3% 22 -2.2% 19 17.4% 21
3 BAL 22.0% 6-4 29.4% 7.2 3 -5.3% 26 -3.4% 22 10.7% 6
4 MIA 20.0% 6-4 24.1% 7.9 2 1.8% 13 5.9% 6 19.4% 24
5 NE 19.6% 8-2 20.7% 6.8 8 2.0% 11 7.5% 3 15.9% 17
6 KC 17.1% 7-3 13.5% 7.1 4 5.8% 3 -1.2% 17 24.7% 30
7 SEA 15.1% 6-4 13.5% 6.8 6 -1.2% 20 3.4% 8 14.7% 13
8 PHI 9.6% 7-3 8.2% 6.8 7 -5.9% 27 -3.6% 24 14.3% 10
9 DET 8.9% 7-3 13.9% 6.4 9 -0.6% 18 -3.9% 25 8.6% 5
10 PIT 6.9% 7-4 8.9% 5.9 13 -6.4% 28 3.2% 9 15.9% 18
11 SF 6.8% 6-4 3.7% 6.2 10 2.2% 10 -1.6% 18 13.4% 9
12 BUF 6.8% 5-5 3.6% 5.6 16 3.6% 7 6.1% 5 6.9% 3
13 DAL 5.4% 7-3 7.6% 5.8 14 -6.7% 30 -3.1% 21 14.7% 12
14 IND 5.1% 6-4 8.3% 6.2 11 2.3% 9 -11.4% 32 14.5% 11
15 ARI 4.4% 9-1 6.6% 6.2 12 -0.8% 19 6.4% 4 4.5% 2
16 NO 2.6% 4-6 3.6% 5.4 17 -2.6% 23 -5.9% 26 19.9% 25
TEAM TOTAL
DVOA
W-L NON-ADJ
TOT VOA
ESTIM.
WINS
RANK PAST
SCHED
RANK FUTURE
SCHED
RANK VAR. RANK
17 CIN 0.3% 6-3-1 4.3% 5.8 15 0.0% 16 0.6% 15 27.4% 31
18 SD -3.1% 6-4 2.2% 5.2 19 0.9% 14 14.2% 1 17.6% 22
19 CLE -3.6% 6-4 4.6% 5.3 18 -6.7% 29 1.7% 13 15.7% 16
20 ATL -4.1% 4-6 3.8% 5.2 20 -8.5% 32 2.4% 11 16.7% 19
21 HOU -9.1% 5-5 -1.7% 4.0 26 -4.1% 24 -7.3% 28 7.2% 4
22 NYJ -11.2% 2-8 -16.3% 4.6 23 7.7% 1 5.9% 7 12.0% 7
23 CHI -11.3% 4-6 -16.9% 4.7 22 5.1% 5 -3.1% 20 13.0% 8
24 STL -11.7% 4-6 -16.1% 4.2 24 5.2% 4 -6.4% 27 20.3% 26
25 NYG -13.9% 3-7 -21.2% 3.9 27 2.4% 8 -9.7% 30 20.3% 27
26 MIN -15.2% 4-6 -13.9% 4.7 21 -1.2% 21 1.8% 12 16.7% 20
27 TEN -16.0% 2-8 -13.8% 3.7 29 -0.1% 17 -7.8% 29 19.0% 23
28 WAS -17.7% 3-7 -14.6% 3.7 28 -7.6% 31 0.2% 16 23.4% 29
29 CAR -19.7% 3-7-1 -24.2% 4.1 25 4.1% 6 -9.9% 31 15.5% 14
30 OAK -23.5% 0-10 -31.3% 2.3 32 6.1% 2 11.3% 2 3.5% 1
31 JAC -27.6% 1-9 -27.3% 2.5 31 0.7% 15 -3.5% 23 23.3% 28
32 TB -29.1% 2-8 -21.6% 3.4 30 -4.5% 25 0.8% 14 31.4% 32

Posted by: Aaron Schatz on 18 Nov 2014

185 comments, Last at 25 Nov 2014, 3:20am by ronnyo

Comments

1
by Cythammer :: Tue, 11/18/2014 - 8:43pm

Hmm, Miami still ahead of New England, if only by a tiny margin.

Two of the top seven teams are below 50% to make the playoffs, and a third (Baltimore) is only a little above even odds to qualify. Meanwhile the 16th best team in the league is a favorite to be hosting a playoff game.

43
by PaddyPat :: Tue, 11/18/2014 - 10:21pm

Pah! Atlanta will beat them!

47
by Tim F. :: Tue, 11/18/2014 - 10:50pm

"Meanwhile the 16th best team in the league is a favorite to be hosting a playoff game."

This seems unambiguously one of the safest and most accurate of FO "predictions", actually. Not bad data or analysis, just sad commentary for the NFC South.

91
by johonny :: Wed, 11/19/2014 - 12:20pm

Miami plays Den, Balt and NE. That's a brutal schedule to get through.

103
by James-London :: Wed, 11/19/2014 - 1:50pm

Baltimore at home is 'must-win'. Assuming they then hold serve against the Jets (x2) and Minnesota, that's 10 wins, which should be enough for a wild-card, depending on tie-breakers.
Add a win a Foxborough and the division might be up for grabs as Miami would have a sweep over NE. Having said that, I fully expect Miami to lose to the Jets in week 17 and miss the playoffs on the 8th tie-breaker because, well, that's what Miami do...

Phil Simms is a Cretin.

116
by johonny :: Wed, 11/19/2014 - 5:38pm

Note even their easy games are against teams with good defenses. Miami's O doesn't face a light D the rest of the year. Dallas Thomas/RT is going to attacked each week. They really had a tough draw of defenses this season. Tannehill's performance has really been impressive. I'm not a doubter anymore.

2
by Otis Taylor89 :: Tue, 11/18/2014 - 8:45pm

OAK - has played the 2nd most difficult schedule and will be playing the 2nd most difficult schedule - wow.
How many wins would they have if they played ATL schedule? They would probably be in 1st place in the NFC South!

3
by Anon Ymous :: Tue, 11/18/2014 - 8:47pm

"Obviously, it's still being seriously dragged down by that awful Kansas City game back in Week 4. However, you can't pretend that game didn't happen and not pretend the same for other teams."

Certainly not. But this description isn't precisely what is going on with NE. Nearly every team has a bad game or two, but rarely is there a situation where a team opens with a terrible month.... and then suddenly becomes dominant. Every single unit on NE's roster, even the special teams, is performing better than they were in September. I suppose you could point at RB as an exception, due to Ridley's injury and the probability that Gray won't average 200 yards a game from here on out, but that would be on the only one. A very reasonable case could be made that NE's first month no longer holds any predictive value. Or worse, it isn't just irrelevant, it is misleading.

Of course, teams with such a large line of demarcation are rare, so I fully support treating them as you would any other team. If you adjust the formula for NE, then you have to start doing it for everyone else and you ultimately end up with a subjective mess. I just don't see the need for defensiveness and false comparisons. Own the fact that NE is a potential oddity that your system isn't designed to capture and let fans formulate their opinions from there.

I should add that I've been a fan of DVOA since TMQ arrived, so I hold your work in high regard.

5
by Cythammer :: Tue, 11/18/2014 - 8:56pm

What would that reasonable case be? Because I'm not sure how it could be anything other than the usual messing around with arbitrary endpoints. It's really not that rare for a team to look way better if you decide to throw away certain games because they don't fit into your narrative. Fans and commentators do this all the time, in a variety of different ways, like when people will say stuff like a certain 'always' plays them tough, or the players were clearly way more motivated for this game for some reason, etc.
If a team has, say, a superstar player come back and that precipitates a major turnaround, then there's probably a good case to be made the team has genuinely played in quality. Otherwise it just looks like manipulating and massaging the data until it fits the picture you want it to.

7
by Anon Ymous :: Tue, 11/18/2014 - 8:59pm

"Otherwise it just looks like manipulating and massaging the data until it fits the picture you want it to."

This isn't remotely the case. I'd be happy to offer the reasoning (I already did on some level in the first comment) but something tells me you really aren't interested.

10
by Cythammer :: Tue, 11/18/2014 - 9:11pm

So the answer is yes: arbitrarily chosen endpoints. Again, it's very, very common for it to be possible to cut up the data so one particular is 'really' way better or worse than they appear to be. It always boils down to only accounting for the evidence that fits into your narrative, and ignoring what indicates the opposite. Like "let's ignore the first month of the season, because reasons."

If anyone wants to use this week's rankings as an attack on DVOA it would probably make more sense to look at Arizona. The Cardinals have been defying DVOA for weeks. The Patriots?… Well, DVOA actually DOES rank them very highly. They're fifth in the league and are only slightly behind the 2. through 4. teams. That's not some terrible insult. I don't understand why the idea that the Patriots are anything other than unquestionably the best team in the league is supposed to be some evidence DVOA has failed in this case. DVOA is saying they pretty close to the top.

12
by Anon Ymous :: Tue, 11/18/2014 - 9:17pm

"So the answer is yes: arbitrarily chosen endpoints"

Which confirms my suspicion that you really aren't interested in discussing it. I'm still open if you change your mind.

14
by Cythammer :: Tue, 11/18/2014 - 9:19pm

Don't be childish. This is a public discussion board, not a one on one exchange. If you want to post you can post it. If you want to be passive aggressive, then yes, that is also an option.

18
by Anon Ymous :: Tue, 11/18/2014 - 9:27pm

Cyth, you are the one being childish. You disregarded the idea out of hand and when I questioned your desire to really discuss it you openly mocked me.

Again, I'm all for laying out my reasoning, but I'd prefer to do it with someone who is interested in the explanation.

77
by Anonymouse :: Wed, 11/19/2014 - 10:35am

I think Anon Ymous' point is worth probing, but strawman arguments aren't the way to go about it.

136
by usernaim250 :: Thu, 11/20/2014 - 10:13am

Not really. The numbers say what the numbers say. The results on the field are highly arbitrary. There are any number of reasons why the Pats may have underperformed in the first month--and the majority are baked in the probabilistic nature of DVOA. It's the nature of probability that's in question.

You can come up with any number of reasons for a secular shift in New England's results. Be my guest. We watch subjective speculation like this all the time--for instance in Bill Simmons' picks--and it mostly looks laughable on Sundays.

E.g. Gronk wasn't full strength before and now he is. Well, he is often not at full strength and any assessment of the future of a team that depends so much on an oft-injured player has to sit somewhere between the two modal eventualities of full-strength Gronk vs. depeleted or non-playing Gronk. DVOA can't predict whether Gronk will stay healthy any better than taking the average for the season.

E.g. the O-Line has made changes. Well, this gets closer to what might be a secular shift. But opponents may adjust to the new personnel grouping, or injury may intervene, and so forth. Again, the average of what happened is the best prediction of the future. Furthermore, good luck at one position group is likely to be countervailed by bad luck at another group--like the injuries at linebacker.

E.g. (let's pretend) Brady had a bad fight with Giselle and if affected his performance, or Belichik's wife didn't get his preferred brand of toilet paper and it distracted him or whatever, until finally, after the Chiefs game, he was able to wipe as he likes. Once again, the average of what's already happened is the best predictor of what will happen. If Brady spats with his wife once, he is likely to again. If BB's wife screwed up the toilet paper purchase once she might do that (or something else with similar effect) again.

You may be right that the Pats have changed fundamentally this year, but in the long run such predictions will be wrong more than right, or at best will be the result of luck (i.e. if Gronk stays healthy the prediction is right and if not it is bad).

27
by Tim F. :: Tue, 11/18/2014 - 9:44pm

"I don't understand why the idea that the Patriots are anything other than unquestionably the best team in the league is supposed to be some evidence DVOA has failed in this case."

I'd argue that the Patriots are unquestionably not necessarily the best team (but could be by the end of the season) but also are unquestionably one of the best teams, and DVOA is fairly successful at measuring that.

As you indicate, any questions about DVOA currently are probably best served by examining teams not named Patriots (The Patriot argument largely seems to be irritation at the lag in recency adjustment). Arizona, Baltimore. Maybe Denver's continued strength... Maybe Miami... But even at just one or two teams that "feel" wrong, any proposed argument of inaccuracy, evidence to support said argument, and proposed tweak to the metrics probably get pretty iffy.

I say we all pick our favorite of: 1. current DVOA, 2. DVOA - Highest Variance Game, 3. Weighted DVOA (8 wk), 4. Weighted DVOA (5 wk), Weighted DVOA (3 wk) 5. Whatever proposed Adjustment 6. Competing Advanced Analytic Stat 7. Etc. Compare at the end of the season/playoffs.

11
by RickD :: Tue, 11/18/2014 - 9:14pm

"It's really not that rare for a team to look way better if you decide to throw away certain games because they don't fit into your narrative. "

It's more than a narrative. It's a noticeable jump in performance between the Kansas City and Cincinnati game. It's OK to notice that. It's a bit more than cherry-picking the six best games (indeed, they are probably not the six best games).

"If a team has, say, a superstar player come back and that precipitates a major turnaround, then there's probably a good case to be made the team has genuinely played in quality."

Shall we talk about Rob Gronkowski's snap counts? Or perhaps Brandon Browner's? Browner was suspended for the first four weeks of the season.

But really, it's more than just Gronkowski and Browner. The offensive line, which was dreadful, is playing now at a high level.

13
by Cythammer :: Tue, 11/18/2014 - 9:17pm

And they're ranking fifth in DVOA, a very good ranking. Again, it's the Patriots who are supposed to be pointing out the flaw in DVOA? Really? The team that is ranked very highly? But simply because they've looked like number 1. the past few weeks means it's not reasonable? They aren't being put inexplicably low in the rankings. It's Cardinals ranking, if anything, which is very strange and hard to understand.

17
by CaffeineMan :: Tue, 11/18/2014 - 9:25pm

Wait, is this whole thing just a terminology disagreement?

I read Anon Ymous' comments (and RickD's) NOT as saying that DVOA needed to be changed in any way. The way I read it was that, if one wanted to predict what the Patriots were going to do in the future, that person needed to take into account more than DVOA, which is what we all do anyway, right? It wasn't about "breaking DVOA".

20
by Anon Ymous :: Tue, 11/18/2014 - 9:31pm

Precisely. It's been perfectly clear for weeks now that predictions based on September results will be less accurate than one that starts with the Cincy game. And that's fine.

28
by Richie :: Tue, 11/18/2014 - 9:44pm

If the Patriots lose a game, does that mean we should have been predicting that game since the Kansas City loss?

30
by Anon Ymous :: Tue, 11/18/2014 - 9:47pm

I'm not sure what the point of this comment is. Nor do I understand why a team that went from lousy to dominant overnight shouldn't be an interesting discussion topic.

51
by Tim F. :: Tue, 11/18/2014 - 11:11pm

I don't find NE ascendance that interesting a topic, really. The most interesting side topic from that is: what the hell was BB doing rotating the O-line the first 5 weeks, and why was everyone on the line sucking so bad then (not just the rookies)?

57
by RickD :: Wed, 11/19/2014 - 12:13am

Bryan Stork was drafted to take over the center position. And then he got injured. So it took several weeks for him to take over the job. Connolly is better at guard than center, but he was being used as center since Wendell had had so many issues with it last year. Devey was used at guard, and that was a disaster. And for some reason Nate Solder started off the season dreadfully. On top of that, Coach Scarnecchia had retired and Logan Mankins had been traded. That led to a huge morale issue, IMO.

It took the ass-kicking at the hands of the Chiefs to snap them out of their extended self-
pity.

The line has played consistently when they've been able to have a lineup of Solder, Connolly, Stork, Wendell, and Vollmer.

If we ask nicely, we might get Ben Muth to discuss this at some point. His explanation would certainly be much, much better than mine.

63
by Tim F. :: Wed, 11/19/2014 - 12:47am

Meh, you aren't saying anything I don't know. But I think "It took the ass-kicking at the hands of the Chiefs to snap them out of their extended self-pity" is the sort of rationalization that's completely unuseful. That they did finally turn it around doesn't change the fact that they were woefully unprepared and didn't have a clear plan at all for any injuries. Miami wasn't as light-and-day bad without Incognito and Martin last year nor so this year with Pouncey injured, a rookie at RT, and injuries and a rotation inside. Other teams have similar or worse difficulties without comparable disparities in play.

73
by Anon Ymous :: Wed, 11/19/2014 - 9:57am

I can't explain it either. Solder and Vollmer both went from upper tier players for the past few years to looking like they hadn't played a game in their lives... and then suddenly look like themselves again overnight. The middle of the line was a jumbled mess initially, which may explain why the OTs struggled, but why was it such a jumbled mess? Cannon and Connolly both started week one and they've been here for years. Cannon may have started at a new position, but he has played RG before. And if he was going to be so bad, why didn't anyone notice this in practice? And why didn't he play a single snap all preseason if the plan was for him to start there?

Clearly things solidified once Stork got healthy, Connolly went to LG and NE lucked into Wendell being a solid RG. That makes complete sense. But from there, both Connolly and Stork went down for the Buffalo and Jet games without disastrous consequences even though the guys who came in were the same ones who struggled in the first month.

None of it makes any sense to me, but somehow they figured it out and the impact has been dramatic.

94
by FrontRunningPhinsFan :: Wed, 11/19/2014 - 12:35pm

I'm amazed people are bewildered by this when the answer is painfully obvious:

BB realized the team was bad, Brady past his prime, and the Patriots are filming other teams' signals again.

123
by Anon Ymous :: Wed, 11/19/2014 - 7:26pm

Ha! Makes sense, though I wonder why it took them four weeks?

It also should be pointed out that, since defensive calls are radioed in, it would have to be a more modern theft method, like tapping into their line or something.

22
by RickD :: Tue, 11/18/2014 - 9:34pm

I don't know why you placed this comment in response to mine. Nothing you say above has anything to do with what I said. I was merely addressing the point that it is sometimes reasonable to notice that the Patriots are playing better now than they were in the first four weeks of the season.

The Cardinals' DVOA rating isn't hard to understand, their W-L is.

19
by jonnyblazin :: Tue, 11/18/2014 - 9:29pm

"The offensive line, which was dreadful, is playing now at a high level."

Really? In run blocking? Prior to the Colts game, the Pats offensive rushing DVOA has been -26%, -37%, -16%, and -50%.

23
by Anon Ymous :: Tue, 11/18/2014 - 9:34pm

The OL for the first four weeks was positively dreadful. Even if there was a mild reduction in run blocking (which I would contest) it was offset by dramatic improvement in pass pro. Seriously, they went from trying to recreate Cutler's greatest hits to giving Brady 3-4 seconds overnight.

24
by jonnyblazin :: Tue, 11/18/2014 - 9:38pm

Yes passing offense is the only area that the Pats have demonstrated any improvement since the first four games. Rushing offense, rushing defense, and passing defense are not really trending upward.

33
by Anon Ymous :: Tue, 11/18/2014 - 10:02pm

I've noticed in the past that Bill Belichick's defenses seem to go more lax than average when they have a big lead. That is the only explanation I can think of for why DVOA thinks their passing defense hasn't improved. I suppose it could be the lack of sacks the past few weeks, which have definitely gone down in Chandler Jones' absence, but oddly enough *pressure* has been improving to my eyes.

29
by RickD :: Tue, 11/18/2014 - 9:47pm

Yes. Rushing is only one aspect of football. Brady has been getting tons of time to pass, even against elite defenses like that of the Broncos.

And no, I didn't say in run blocking. They've had two or maybe three good rushing games, but the Pats generally game plan to attack a team's weakness. Against Cincinnati and Indy, that meant running the ball. (They had 220 yards rushing against Cincy.) Against Buffalo and the Jets, that meant passing. The Bears were particularly vulnerable to the pass. Denver is strong against both passing and rushing, so the Pats went with their own strength.

15
by jonnyblazin :: Tue, 11/18/2014 - 9:22pm

"Every single unit on NE's roster, even the special teams, is performing better than they were in September."

I'm not sure that's entirely accurate. The defensive passing DVOA for NE's first three games was -17%, -31%, and -6%, followed up by games of 70%, 15%, 20%, 46%, 23%, -10%, and -14%. So they've had two somewhat above average games defending the pass recently, but a whole slew of games before then when they were much worse than they were for the first three weeks.

Looking at the rushing DVOA, they've been poor for most the entire year aside from the Colts game (I don't feel like relaying the DVOA).

They're defensive rushing DVOA has been good the past two weeks, but up and down throughout the season without any obvious trends.

The offensive passing DVOA is the category that really fits your narrative. The three worst passing performances occurred in the first 4 weeks of the season.

25
by Anon Ymous :: Tue, 11/18/2014 - 9:40pm

NE's passing defense is substantially better than it was to start the season. The reason their DVOA was so good to start the year is a combination of turnovers and lousy competition.

In recent weeks they've allowed most of the damage long after the game was out of hand. The only real exception was the Jet game and I'd like to say it was due to the short week, but that *would* be cherry picking.

31
by jonnyblazin :: Tue, 11/18/2014 - 9:47pm

Well if you want to disregard DVOA when it doesn't back up your narrative, ok then. But level of competition and game situation are factored into the efficiency ratings.

34
by Anon Ymous :: Tue, 11/18/2014 - 10:03pm

As I said before, I'm fully versed in DVOA as I've seen every iteration of this site since 2003. It may appear I'm dismissing to fit my narrative, but that is not the case at all.

EDIT to add: If you asked 100 Pats fans whether the secondary was performing better in September than it is now, you wouldn't get a single answer in the affirmative. I'd put serious money on that. Whether you take that as ignorance on my (our) part is up to you. :)

32
by RickD :: Tue, 11/18/2014 - 9:50pm

Yes, the Pats had a great game on pass defense playing the Vikings in Week 2. That would have to be considered an exceptional game: Peterson had just been taken off the field, and Matt Cassel was the QB.

I think it's reasonable to note that Thursday night games are played on a different "playing field" than others, since it's a short week. But that's a factor that would be difficult to untangle, and it affects both teams.

40
by Tim F. :: Tue, 11/18/2014 - 10:13pm

It seems like the argument is boiling down to: should DVOA show Patriots to be #1 and steadily progressing and/or markedly improved post-Kansas vs. should DVOA show Patriots to be a good team, trending upward but NOT BEST.

I don't think Indy told us much. Denver was an impressive win, but they seem to be declining. Everyone seems to be ignoring the division games. Even beating Jets and Bills, those were not the BEST TEAM games. Vikings, Raiders, Bears, even Bengals — not great competition. To say they have been on a tear since Kansas is not really accurate. Depends on how you view Indy and Denver, I guess.

42
by Anon Ymous :: Tue, 11/18/2014 - 10:19pm

It may seem odd since I started this whole thing, but I'm not too upset about where NE ranks right now. I really meant to post this last week, because I think the disparity between their DVOA rating and their true performance level was much larger, but got lazy. I probably wouldn't have posted today if not for Aaron's comments likening NE's turnaround to any cherry picking situation, which I still feel to be an erroneous description.

48
by Tim F. :: Tue, 11/18/2014 - 11:02pm

No, I didn't think that was your take on DVOA, but I do agree with Aaron on cherrypicking. I think we disagree on the Pats.

53
by Anon Ymous :: Tue, 11/18/2014 - 11:45pm

I don't know. I'm no statistician by trade, but I have a hard time believing that when you can draw a direct line between one set of data and another that a causal link wouldn't be one of the first assumptions.

Said another way, lets assume you did several studies on several different (though similar) topics with a measurement taken once a week. All of them end with the anomalies randomly interspersed with no obvious pattern. Then you come across one where the data suddenly changed mid year. All the data on this subject was in one range through February, with a sudden shift in March where it remained for the rest of the year.

Would your first thought be that this was just a random occurrence where random chance happened to create the appearance of order? Or would you postulate that something caused the dramatic change?

I suspect nearly everyone would lean toward the latter. That's all I'm saying.

59
by Tim F. :: Wed, 11/19/2014 - 12:21am

I think you are misinterpreting what I am saying. Yes, the return of Gronk has made the O better, yes the addition of Browner has made the pass D better. Yes, the improvement on the O line is absolutely the most significant change. Despite that, no, I don't think you can say: DVOA shows a first half and second half performance that allows us to discount the first half of the season while not doing the same for other teams because you don't so readily see a comparable change. I don't have premium access so I don't know about individual game DVOAs but Total DVOA plotted through the season don't appear to show the dramatic difference you claim (after the stinker to Miami, NE improves greatly and stays good for two games, then falls again against KC, then rebounds moderately against Cinci as a good team, but stays average/good through Bills, Jets, Chicago... or at least that's what I expect the individual games would show, to an extent. There isn't a black/white, good/bad split that's as dramatic as you claim. Yes, dramatic turnaround, but no, I think it's poor analysis. If I could see a dramatic break in individual game DVOA maybe I would agree but not based on the evidence available to me.

74
by Anon Ymous :: Wed, 11/19/2014 - 10:00am

Well then, I guess I *am* disregarding DVOA for the narrative I prefer. :)

DVOA thinks NE was good against Oak and only average/good against Cincy, Buffalo and Chicago?

155
by Tim F. :: Thu, 11/20/2014 - 2:24pm

I'm not saying I'm certain: I'm just looking at the changes in Total DVOA. I'd defer to a premium member, assuming there is access to individual game DVOAs. But a simple graph or look at the changes in Total DVOA over the season for NE does not reveal this clear improvement in the season that you claim is there.

158
by Anon Ymous :: Thu, 11/20/2014 - 3:17pm

Yeah, this shocks me, to be frank, but now that I know this Aaron's comment makes a lot more sense. According to DVOA, the KC game *is* a random anomaly. I don't view it that way, but that's my prerogative. ;-)

156
by Tim F. :: Thu, 11/20/2014 - 2:24pm

I'm not saying I'm certain: I'm just looking at the changes in Total DVOA. I'd defer to a premium member, assuming there is access to individual game DVOAs. But a simple graph or look at the changes in Total DVOA over the season for NE does not reveal this clear improvement in the season that you claim is there.

68
by intel_chris :: Wed, 11/19/2014 - 3:47am

I don't know. I'm no statistician by trade, but I have a hard time believing that when you can draw a direct line between one set of data and another that a causal link wouldn't be one of the first assumptions.

Actually, no it shouldn't be one of your first assumptions. That's bad science and bad statistics. It is sometimes done, but people who do it are called "data miners" and looked down upon by serious practitioners, because with enough data, one finds all sorts of interesting patterns that look to be statistically significant, but are merely artifacts. Responsible work requires one formulate a hypothesis first and then test it against the facts. That is actually very hard to do, because as humans we are programmed to look for patterns, and thus many of our hypotheses are the result of looking at the data, seeing a possible patterns and then looking for more data that supports are assumption.

So, taking the NE example as a case in point. The argument only works if instead of making a "special pleading" that NE is an exception and that something miraculous happened that transformed an average team into a dominant one. you need to make a general proposal that can be tested across many teams.

Let's say you decide that it is the return of Gronk that sparked the Pats transformation. How do you formulate that into something generally applicable (and thus testable) across a wide swath of teams? Take this as your thesis: Each team has 3 key players whose removal significantly impacts the teams play and without them the teams VOA is severely impacted. For NE, you might postulate those as Brady, Gronkowksi, and some other player. For GB, you might pick Rogers and two other players. For Denver, say Manning, J Thomas, and Von Miller. Now chart the teams performance with and without those players. If the difference is statistically significant, you start to have a case for making that a measurable attribute. If you can relate that to something already measured, for example, the players DVOA or DYAR stats, you really begin to have a case.

The difference between the general formulation and the NE special case, is that I have no idea whether the general formulation works. I suspect it might, but I am only guessing, so it is a true hypothesis that we can measure. Looking at NE, we already have the data, and we are trying to formulate a hypothesis that fits it. If the hypothesis explains only NE, we have no way of testing it, because we based our hypothesis on the NE data to begin with, and thus testing it on that data is circular reasoning.

72
by Anon Ymous :: Wed, 11/19/2014 - 9:47am

I agree that you have to be able to test any hypothesis and that, absent a way to truly measure and compare conclusions are hard to justify.

That said, I disagree with this statement:

"Actually, no it shouldn't be one of your first assumptions. That's bad science and bad statistics. It is sometimes done, but people who do it are called "data miners" and looked down upon by serious practitioners"

"Assumption" was too strong a word, but when the disparity is so blatant (assuming it is, I've now been informed that DVOA doesn't see such a dramatic shift in performance) it is something that must be considered. If this were flips of a coin, I'd agree, but there are too many factors that can contribute to potential improvement that dismissing it as the appearance of order seems like a far worse option. The best way to approach it would be to investigate a potential cause and, absence of that, move on to it being a simple anomaly. At the very least you'd want to determine a cause since that would influence the data sample and how it relates to the entire set.

Of course, that's all assuming DVOA matched my eyes, which it clearly doesn't, so my case isn't as strong as I thought it was. DVOA was far more impressed with NE's first few weeks than I was, making the KC game look more like an anomaly than the continuation of a trend that I consider it. I'm not sure I agree with DVOA on this one, but I know I don't have a convincing argument otherwise.

78
by RickD :: Wed, 11/19/2014 - 10:51am

" Responsible work requires one formulate a hypothesis first and then test it against the facts."

Well, that depends on what we're talking about. This kind of idealized view of hypothesis first, then testing, then conclusion, isn't really exactly what's going on in science. It's correct in terms of when you set up experiments, but a lot of science isn't experimental.

The crux of the issue, as I see it, is whether it's necessary to view a team as one fixed entity over the course of a season or to allow for the possibility of a team playing at different levels at different times. The latter type of modeling is not unusual at all. Indeed, there are well-defined statistical tests for when it's appropriate to refine a model and when it isn't.

Unfortunately for the purposes of this discussion, I'm a mathematician, not a statistician. But I've worked enough with statistics to have a feel for what can be done.

As for your sneering towards "data mining", that's entirely inappropriate. There is an appropriate place for that kind of work: namely in hypothesis formulation. In fields of work where the amount of data is enormous, it's entirely appropriate to sift through it, looking for patterns, as long as one understands the difference between hypothesis generation and hypothesis validation.

89
by Anon Ymous :: Wed, 11/19/2014 - 12:10pm

"Well, that depends on what we're talking about. This kind of idealized view of hypothesis first, then testing, then conclusion, isn't really exactly what's going on in science. It's correct in terms of when you set up experiments, but a lot of science isn't experimental."

Indeed. And even within experimental science, it is perfectly normal to use results to formulate new hypotheses and then test those.

127
by intel_chris :: Thu, 11/20/2014 - 12:57am

Anon Ymous, I believe I owe you an apology. My reply was too strong and strident. There are ways to use data to guide one to hypotheses. Both you and RickD are correct for chastising me on that point.

There are also times where people need to establish some result, and go looking for any data that appears to confirm that result and then use that as their hypothesis, corrupting the process. I'm also not a statistician, simply a computer architect and we see that kind of activity in benchmarking all too frequently, so it is kind of a hot button for me. I'm sorry for implying that what you were doing was of that nature.

The secondary point about teams changing character (and possibly using appropriate statistical tests to measure that) is a good one. Even the simple test of seeing whether the early Patriots games and the recent ones appear to be drawn from the same population could be enlightening. If you do it the DVOA way of rating each play as a distinct event, you probably have enough data to get valid results--although the results might just say each game appears to be drawn from a unique population, but even that result would be an interesting one. That even might be a way to show that certain games are "flukes", e.g. if the majority of a teams games are in a certain population, but one game's plays are outside that population, that game could be judged as unrepresentative (or indicative of something else).

137
by Anon Ymous :: Thu, 11/20/2014 - 10:30am

No worries, Chris. Now that I realize DVOA doesn't see the same dramatic jump that I do, Aaron's statement comes across as far more reasonable. I'm not exactly without blame on this one.

162
by Scott C :: Thu, 11/20/2014 - 5:20pm

Well, every fan of almost every team can claim the same thing.

Believe it or not, it is not rare for performance to change strongly in the middle of the year.

There have been studies on this, and Weighted DVOA addressed it somewhat. However, for future predictive powers the overall picture still works better.

IF (and this is actually, a big IF) the patriots keep up their recent performance, tey will be at the top at the end. But your subjective opinion that they will is no better than any fan of any other team picking on the bits that DVOA doesn't account for.

The chargers just lost 1 starter on defense to IR and got 4 back from injury. I think their defense looks way better in the middle and weaker against the pass. So what?

The predictive power of your subjective "the patriots are way better now" is not likely to be any better than DVOA's "they are very good, but have some weaknesses" view, and is likely to be worse.

167
by Anon Ymous :: Thu, 11/20/2014 - 7:51pm

I've been following football since 1985. Subjective or not, I've never seen a team transform the way NE has over the course of a single week, barring a dramatic injury situation.

If you read through my comments I think you'll find that you are massively misrepresenting my point, hopefully not intentionally.

37
by RickD :: Tue, 11/18/2014 - 10:06pm

Looking at opponents:
Miami -17%
Minnesota -31%
Oakland -6%
KC 70%
Cincy 15%
Buffalo 20%
Jets 46%
Bears 23%
Broncos -10%
Colts -14%

Does it seem weird to you that their pass defense was mediocre against teams with weak passing attacks (esp. the Jets) while it's rebounded against the Broncos and Colts?

The Jets somehow got a 46% rating on these stats: Geno Smith was 20 for 34 for 1 TD and 226 yards. And sacked 3 times. Why is this considered an utter failure by the defense? Because Geno was so bad against everybody else, being merely mediocre against the Pats was viewed as horrible.

Orton was 24 for 38 with 299 yards with 2 TDs and 1 INT. He was also sacked five times. DVOA says that means a 20% rating for the Pats' pass defense. My eyeball says that this shouldn't be viewed as a below-average performance.

My personal opinion is to not to worry about the week-to-week DVOAs.

44
by Anon Ymous :: Tue, 11/18/2014 - 10:29pm

"My eyeball says that this shouldn't be viewed as a below-average performance."

Well, if it is worse than Orton *usually* does, then.... :)

The problem with NE is that they are maddeningly mediocre when the game is in the bag. They are the kings of allowing 300 yards and 2 TDs in a game where the result is never in doubt (or those long 67 yard drives that end of a FG when the game isn't quite "in the bag" but is close). DVOA really has no choice but to consider that a below average performance, but those familiar enough with NE's patterns can see the "surprising" Indy/Denver upgrades coming.

50
by RickD :: Tue, 11/18/2014 - 11:07pm

It was when Orton took over from EJ Manuel. DVOA "thought" Manuel was still the QB.

And the Pats were the only Jets opponent that didn't intercept Geno Smith.

163
by chemical burn :: Thu, 11/20/2014 - 5:34pm

It's weird that they haven't figured out a way to deal with that, at least where QB is concerned. It seems like such an obvious factor in success or failure that the fact that DVOA doesn't account for it well seems... almost stupid? I think this goes back (again) to DVOA being stuck between being descriptive and predictive.

I wonder if they've ever considered slicing it apart more coherently (weighted DVOA is a somewhat incoherent attempt to do this, as is DAVE) where there's a DVOA that's intended to describe what happened and a DVOA that's intended to predict what will happen going forward. Right now, it's DVOA and a bunch of modifiers that you can pick in choose from buffet-style to get a picture of a team's past performance and probably future success, but I wonder if there's a way that does some of the picking and choosing for you... (Like a tasting menu.)

46
by Anon Ymous :: Tue, 11/18/2014 - 10:48pm

Jonny, allow me to take a different tract. You're a Miami fan, right? Do you think NE's defense played well in week one? I certainly don't.

I think the best explanation for NE having a -17% passing defense that day is that they had two first half turnovers (a pick and a fumble) on passing plays, which are obviously highly valuable plays. Miami moved the ball reasonably well, but those two along with another fumble on a rushing play short-circuited all momentum.

In the second half, Miami continued to move the ball, but since NE could go nowhere and the running game was so dominant, they had no need to pass the ball. When they did it was effective, even including three RTP penalties, but they didn't have enough attempts to offset the DVOA NE accumulated with the first half turnovers.

54
by jonnyblazin :: Tue, 11/18/2014 - 11:51pm

Actually I'm a Ravens fan. So you can see why I'm clinging so desperately to DVOA :)

56
by Noah of Arkadia :: Wed, 11/19/2014 - 12:04am

Tannehill was pretty bad that day, it wasn't so much what the defense did. Speaking of sudden performance changes, since he's been pretty good since week 4, the opponent adjustments work in favor of the Patriots (edit: maybe not so much, for the season Tannehill is barely positive in DVOA).

------
Who, me?

4
by MilkmanDanimal :: Tue, 11/18/2014 - 8:49pm

Man, Tampa can't even suck right.

6
by Thok :: Tue, 11/18/2014 - 8:59pm

I mentioned this late last week, but could a Football Outsiders type say something about the 49ers weather adjustment for special teams? In particular, I'd like to know how Levi Stadium is rating for weather compared to Candlestick. I mention this because this is the first year in a long time that the 49ers have had a positive weather adjustment on special teams, and I think you had an issue with this the last time somebody switched stadiums.

(I'm also aware that the 49ers weather adjustment is influenced by an extremely favorable set of road games for special teams. I'd still like somebody to double check.)

8
by oaktoon :: Tue, 11/18/2014 - 9:07pm

Hopefully I have been more constructive critic than obnoxious troll but immense kudos to Aaron for a) admitting there could be a problem afoot and b) making us all understand that his health problems have stood in the way of fixing it... The strength of this site is novelty, decidedly unconventional wisdom, and an awful lot of transparency and reader interaction. Hope you get fully healthy soon, Aaron-- and I am skeptical about the Cardinals too.

As to the rest of you, ignore my green colored glasses for a minute and think about, and react to, if you wish, the assertion I made elsewhere yesterday that Rodgers may just need to fill in the furniture in the room of greatest QB of all time. (Big task, though-- might last 7-8 more years minimum) Or to put it another way, echoing Trent Dilfer, he has been playing the position better these last four years or so than anyone else ever has, including his extraordinary elders in Denver and New England. Brady v Rodgers in 12 days will be interesting, and I truly hope that's only the first of two such encounters this season.

9
by Anon Ymous :: Tue, 11/18/2014 - 9:11pm

Yes, Aaron is great at fostering discussion when DVOA seems at adds with public opinion rather than pretending he has all the answers.

16
by RickD :: Tue, 11/18/2014 - 9:24pm

I think many of us tired of the Brady/Manning arguments a few years ago. I don't speak for everybody, obviously, but further complicating the debate by adding Rodgers to the mix is not very interesting. Rodgers is an excellent QB and is a great candidate for the MVP this season.
It's premature to judge how his career will finish. I don't agree that he's been playing better than Manning or Brady "these last four years". Rodgers hasn't had a season as good as Manning's was last season.

81
by kaesees :: Wed, 11/19/2014 - 11:02am

RickD: when you say that "Rodgers hasn't had a season as good as Manning's was last season", are you forgetting 2011 where he had 2059 DYAR (despite sitting one full game and parts of most others) and 46.6% DVOA (compare Peyton's 2013: 2475 DYAR, 43.2% DVOA)?

85
by dmstorm22 :: Wed, 11/19/2014 - 11:36am

Honestly, Rodgers in 2011 was every bit as good as Rodgers in 2014.

Rodgers will likely win MVP this season unless something strange happens. The only thing stopping that is if Brady plays really well and the Pats run teh table (which would include a h2h win over the Packers).

That 2-MVP club is nice territory to be in for QBs.

BTW, to the earlier guy who said that Rodgers over the past 4-years has been better than any QB ever, I don't even know if he was better than Manning from '03-'06, which included, considering how much worse league-wide passing was even then, two of the three best years by DVOA, as well as a great, controlled year in 2005.

93
by ceo_mr_man :: Wed, 11/19/2014 - 12:25pm

I agree with the sentiment of other posters that there's no call for "greatest ever" discussions right now. There's plenty of time to indulge in those threads in the off season. Surely, Rogers looks unstoppable - with his magically healthy team (for the first time in years), a very good offensive line and two great WRs both in their prime. A great and hard-to-sustain turnover ratio, strong D overall, and good ST unit keeps Mr. Rogers on the Basilica of St. Curly's turf for ages, allowing him to calmly eviscerate all opponents - but that's a lot of ins and outs to all that. I don't find it conducive to very interesting "best ever" discussion.
HOWEVER, I would like to point out that at least in terms of DVOA, Rogers's 2011 was as good or better than Manning's 2013.

164
by Scott C :: Thu, 11/20/2014 - 5:38pm

There are a few other top -5 QBs that would love to have a magically healthy offense surrounding them.

92
by ChrisS :: Wed, 11/19/2014 - 12:20pm

I think Rodgers and Manning are current MVP favorites. However I like Andrew Luck's chances, he leads in YDs, 3rd in DYAR and has faced a fairly tough defensive schedule. Among the Colts remaining opponents there is only one good team, so they could end up with 11 wins and his traditional stats should improve. Qualitatively the rest of the offense is average (TE's) to awful (RB's), the Def sucks and with an average QB they would/will have won far fewer games (TY Hilton is good).

153
by Jeff88 :: Thu, 11/20/2014 - 1:24pm

Luck has two pretty good TE's, and I think it's Trent Richardson being awful that just sucks the life out of that whole position group

26
by Richie :: Tue, 11/18/2014 - 9:43pm

"Brady v Rodgers in 12 days will be interesting"

Are they playing both ways?

110
by bravehoptoad :: Wed, 11/19/2014 - 2:37pm

Rodgers usually plays free safety in a cover 1 while Brady is a run-stuffing strong safety who gets exposed in coverage.

133
by JoeyHarringtonsPiano :: Thu, 11/20/2014 - 8:42am

No, no, no. They're going to duel at midfield, using footballs instead of pistols.

141
by bravehoptoad :: Thu, 11/20/2014 - 11:30am

Gives new meaning to "having a big gun."

69
by big10freak :: Wed, 11/19/2014 - 4:34am

oak

I suspect you are a newer poster and as a fellow Green Bay fan who has observed other GB fans pass thru FO to post gushing praise of either the team or a particular player I have two suggestions.

1. Don't go near the "Best QB of whatever"
2. Freely discuss the team or player in any other manner. Share observations. Share frustrations. Sift through the site's statistical trove and find unique elements tied to Green Bay. Mock Mike McCarthyisms.

But don't gush about Rodgers. Gushing is irritating to the reader, even fellow Green Bay fans. I suspect even Rodgers himself finds it galling

Just let it be.

Thanks.

21
by nat :: Tue, 11/18/2014 - 9:32pm

On the Cardinals: DVOA is built around the net expected value of the next score. So you'd expect it to track better with net points than wins. Arizona is seventh in net points. That's not so far off as their record is.

35
by etcetera99 :: Tue, 11/18/2014 - 10:03pm

I think the problem comes down to over-relying on DVOA for predictive value.

As long as you treat it as one metric/factor among many (quantitative and non-quantitative), and realize its limitations, you'll be OK.

There are non-quantitative factors, which just because they are not measured, do not necessarily have a small/negligible impact. DVOA is a team stat and doesn't measure the change in personnel (due to injuries, rookie improvement due to practice, etc), for example.

One recent example is the Ravens o-line getting healthy at the end of the start of the playoffs for their Super Bowl run a couple seasons ago. DVOA just ignores that for good reason - it's tough, if not impossible to quantify that effect. It's foolhardy, however, to ignore that effect in your own independent analysis.

39
by etcetera99 :: Tue, 11/18/2014 - 10:09pm

And the same holds for Gronk's return.

It seems clear that he's had a large positive effect on the Patriots.

But it doesn't mean that DVOA should be adjusted for that, because you go down the slippery slope of then needing to adjust every team for every injury, and that is hard, if not impossible, to do systematically.

41
by RickD :: Tue, 11/18/2014 - 10:14pm

I personally don't think that DVOA should be adjusted. I just think that people should be aware that teams don't play at the same level every week, and should not think of week-to-week results as somehow being samples of an i.i.d. random variable. Mathematical modeling should be descriptive, not prescriptive.

45
by deus01 :: Tue, 11/18/2014 - 10:36pm

Looking at the variance may be helpful. We obviously wouldn't expect the higher ranked team in DVOA to win everytime.

The Cardinals look like they are the second lowest variance team, which may somewhat explain why their record is better than DVOA would indicate.

105
by bravehoptoad :: Wed, 11/19/2014 - 2:26pm

...and also realize DVOA's strengths. All it has to to is be in the vague realm of other good advanced stats and then you can take advantage of the way you can divide it by situation and matchup.

36
by ClavisRa :: Tue, 11/18/2014 - 10:05pm

With the preseason practice time substantially shortened now, the first few weeks of the season reflect less developed teams. Belichick's philosophy has always been to value depth, develop the whole roster, and build the team to mature in the second half the season. The success of the philosophy is evident by their ridiculous November winning %, their consistency in winning their division and their ability to succeed in the face of injuries to key players. So, more so for the Patriots than most other teams, the early season games are not predictive.

38
by Anon Ymous :: Tue, 11/18/2014 - 10:07pm

While true, the chasm between September and the rest of the year has never been this large. That's why I think it is worthy to mention now and probably had more of a trivial impact in past seasons.

65
by PaddyPat :: Wed, 11/19/2014 - 2:52am

The chasm was huge in 2003, which is the year that "feels" most similar to this one. That year, the Pats got blown out 31 to nothing in Buffalo in week 1 and lost a miserable game in Washington in week 4, and began slowly picking up steam as the year progressed to finish roughly tied with 3 other teams for best DVOA.

101
by Anon Ymous :: Wed, 11/19/2014 - 1:25pm

2003 is an solid comparison, it even has a similar layout of the first four weeks, but IMO the chasm is much larger this year. Yes, they won every game after the loss to Washington but the season was chock full of narrow wins over good and bad teams, alike. The only real dominant performance was the 31-0 shellacking of Buffalo in week 17.

To me, it looks like the first four games of 2003... and then a jump to 2004's level of play. That said, DVOA doesn't see that large a disparity between how NE played against Oakland and how they did against Chicago, so what do I know? :)

90
by coremill :: Wed, 11/19/2014 - 12:14pm

This seems like a BS narrative. They did start relatively slowly in 2003, 2005, 2012, and this year.

But in 2013, they started 4-0, in 2011 5-1, in 2010 6-1, in 2009 6-2, in 2007 8-0, in 2006 6-1, and in 2004 6-0.

I think it's much more likely Belichick tries to win every game, and the timing distribution of the losses is a) random, or b) determined by injuries to key players, rather than some philosophy about "developing the roster" and "building the team to mature in the second half".

96
by PaddyPat :: Wed, 11/19/2014 - 12:56pm

The 2003 comparison to this year is appealing, from a purely sports-writer cheeseball perspective, mind you, nothing scientific or rational, because that year also began with the team suddenly letting go of a well-loved player right as the first game began.

120
by Athelas :: Wed, 11/19/2014 - 6:41pm

This.

98
by Anon Ymous :: Wed, 11/19/2014 - 1:06pm

You wouldn't say that if you lived through the "Targeting September" days! :)

But seriously, it isn't a BS narrative. The data is too consistent and Bill himself has spoken about managing a team to ensure they are at their best in December. That said, I think the earlier post overstates it's effect and I don't think it is all that different from any other well coached team.

140
by coremill :: Thu, 11/20/2014 - 11:28am

"The data is too consistent"

http://xkcd.com/285/

154
by Anon Ymous :: Thu, 11/20/2014 - 2:15pm

Reasonable request, and I'm certain weekly tracking of VOA would support my statement, but unfortunately I'm not a premium member. Certainly winning % in Nov/Dec vs. Sept/Oct does, but we know how misleading that can be.

Truthfully, I would have thought that statement needed supporting about as much "Aaron Rodgers is a good quarterback" which is why I let it stand on its own. As a Pats fan, I can say with certainty that it is true, though it you choose to disregard it because I can't provide absolute proof, I won't be crushed.

165
by coremill :: Thu, 11/20/2014 - 6:32pm

Patriots' record, 2001-2013, by quarter of the season:

Games 1-4: 37-15
Games: 5-8: 35-20
Games 9-12: 42-10
Games 13-16: 44-8.

So they do appear to improve as the season progresses, expect for Games 5-8 when performances declines -- how does that fit the narrative?

166
by Anon Ymous :: Thu, 11/20/2014 - 7:47pm

Well, since I grouped them as Sept/Oct and Nov/Dec, I'd say quite well! ;-)

171
by coremill :: Fri, 11/21/2014 - 12:33pm

But aren't those groupings arbitrary? Using quarters of the season is arbitrary too, but I'm too lazy to calculate week-by-week or rolling four-week averages. But I think the quarters do show that if you graphed the trend line over the course of the season, there's a dip in the middle. I don't know if that dip is meaningful, but I'm not convinced the rise at the end is meaningful either.

172
by Anon Ymous :: Fri, 11/21/2014 - 2:20pm

I don't know. I grouped it like that because, without even running numbers - I felt certainly that's what the data would show. NE tends to start well, but really turn it on once November rolls around.

Frankly, football involves so many different factors most with sample size issues that it is hard to say anything with certainty. The fact that DVOA is at the play level is commendable, and certainly a move in the right direction, but even that one can be tricky at times. I started this with my comment about my subjective opinion of NE's rise in play, but I bet every single team could make a viable case that at least one of their ratings is either misleading or non-predictive.

173
by coremill :: Fri, 11/21/2014 - 3:11pm

I'm not disputing whether NE has generally played better in the second half of the season. There is evidence that that happens. I'm disputing whether that superior performance is the result them "turning it on once November rolls around" or reflects Belichick's philosophy "to build teams that mature in the second half of the season." Those propositions I don't see any evidence for, especially in light of the evidence that their play actually declines as the season progresses before it improves again.. At least, the fact they have played slightly better in the second half does not establish those narratives. Maybe they happened to have had more injuries in the first half of seasons, or easier opponents, or easier travel, or maybe it's just random. Splits happen.

180
by Vandal :: Sat, 11/22/2014 - 4:58pm

Yeah, 538 wrote an article specifically about the patriots in the 2nd half of the season:

http://fivethirtyeight.com/datalab/the-patriots-really-do-own-the-second...

159
by usernaim250 :: Thu, 11/20/2014 - 4:41pm

If you want to see a coach who really won in the last month (and playoffs) at a much greater rate than before, Joe Gibbs is your man.
Overall WP: .621
Regular Season, December and January only: .697

I haven't run any tests but I'm pretty sure that's statistically significant over 238 games.

And he did it in both stints.

1st run WP: .674
1st run Regular season Dec/Jan: .750

Comeback WP: .469
Comeback Dec/Jan: .591

How did he do it? No idea. But it was fun to watch.

49
by cstoos :: Tue, 11/18/2014 - 11:03pm

I am again curious about the KC defensive DVOA. It is vastly different than what I see.

They have held 8/9 opponents to less than their season average scoring, behind only Detroit in that regard. They are #9 in opponent adjusted yards allowed, allowing about 20 yards fewer than average per game.

So they are holding opponents to below average scoring and below average yards, yet they are 14th in defensive DVOA (and in the positive range to boot). Almost every other objective rating has them as a top 3 defense. What causes DVOA to vary by so much with this particular team?

113
by dhulten :: Wed, 11/19/2014 - 4:22pm

One thing I'd look into is how much yardage they're giving up. I haven't looked at the numbers for the rest of the season since I'm a seahawks fan, but last week at least Seattle actually outgained KC by a pretty significant margin. In the end, Seattle put up 372 Total Net Yards to KC's 298. The difference in score was pretty much red zone TD conversion.

Add that to what I would guess is an at least partially fluky "no rushing TDs allowed" streak, and I can see why scoring numbers would paint the KC defense in a more dominant light than DVOA.

121
by chemical burn :: Wed, 11/19/2014 - 6:47pm

They also have been a bend don't break defense, which is something DVOA doesn't love because with a few exceptions it doesn't really seem to exist - most teams that consistently give up yard between the 20's also give up yards in the redzone. Interestingly, in one of Almanacs they looked at this more in-depth and found that only two coaches seemed to field defenses that bucked that trend: Jim Johnson and Bill Belichick. At the time, Johnson was seen as the architect, but there's no reason to rule out Reid as the mastermind.

But I've definitely seen their style of playing defense a lot under Reid: don't give up big plays especially in the passing game, never focus on stopping the run and go for the knockout blows like sacks and heavy pressure at key moments. DVOA never likes this kind of defense because it really doesn't seem to be something that happens frequently, but it fits KC.

On offense, they have no deep ball and no ability to stretch the field. Even if you like KC, their offense is a weird throwback thing with almost all of its action happening within 7 yards of the line of scrimmage.

(Interestingly, DVOA loved Andy Reid's Eagles offenses for the same reason it didn't like it's defenses, a lot of consistent gains between the 20's mixed with knockout blows like long bombs and big runs. That his teams stall out in the redzone was something DVOA never cared about as much as the fans did.)

143
by bravehoptoad :: Thu, 11/20/2014 - 11:33am

It's easy to rationalize a justification for why "bend but don't break" defenses would exist. For a certain kind of defender a short field would make all the difference.

Has FO done any work on the existence or lack thereof of bend-but-don't-break defenses?

144
by tuluse :: Thu, 11/20/2014 - 11:41am

Yes, in one the pro football prospectuses there was an essay on it. I believe in every situation they looked at--with one exception--the bend-but-don't-break defenses collapsed the next year. The exception was Herm Edwards defenses which consistently bended but didn't break.

150
by chemical burn :: Thu, 11/20/2014 - 12:27pm

Hm - I'll have to dig up my Almanac where the article was published - I've been thinking for years that it was Jim Johnson, Belichick, and those Ravens defenses whose DVOA improved significantly in the redzone. And that it didn't matter who was coaching the Ravens. Maybe they just looked at a single year and then published more research about those defenses collapsing the following year? I know it has colored my perception of Belichick's defenses for years, which both DVOA and traditional stats have not been much impressed by, but I felt like I've seen demonstrate the ability to tighten up when it mattered.

There was a specific team though - maybe that 2003 Bears team? - that it was very pointedly arguing was not a "bend but don't break" defense. There was a team whose performance they were addressing directly... It was definitely in one of the early Almanacs... (Also very possible: it mentioned the collapses and I rationalized them away via injuries or the Eagles' disastrous 2005 season.)

157
by tuluse :: Thu, 11/20/2014 - 2:55pm

I'm pretty sure this was when Edwards was still coaching the Jets or had just moved to the Chiefs, so there could have been new developments since then.

160
by chemical burn :: Thu, 11/20/2014 - 4:52pm

Yeah, they have a lot more data since then (as it was maybe a decade ago) - it'd be cool for them to go back and look at it again. They've also tinkered with DVOA a lot in the meantime, so I'm sure that will affect things as well.

Anyway, I think the basic premise that DVOA doesn't like teams that give up consistently small-and-medium games then tighten up is true and also that KC is one of those teams. Look at their final 3 stops versus Seattle. Seattle had more or less been running the ball at will... until the 4th and 1 when KC suddenly sold out to stop the run and stuffed them. Similar thing on the 3rd down run that saw Lynch brought down at the 2. Take away the deep pass and limit Wilson's scrambles between the 20's, then when they get into the red-zone switch the tight man coverage and bring blitzes and clog the gaps.

The weirdest example of this was that game versus the Jets where it seemed like they just could not put them away... but it also never felt like the Jets had any real shot of winning. KC lets teams hang around, which I think DVOA quite rightly doesn't love...

147
by cstoos :: Thu, 11/20/2014 - 12:15pm

I wasn't really commenting on the offense. That unit is efficient, but not mind blowing for sure. Just the defense, and their DVOA decline.

I would agree with the premise that "bend but don't break" defenses probably don't exist, but I wouldn't necessarily put the Chiefs in that category though. Again, teams on the whole are averaging fewer points AND fewer yards per game against the KC defense.

As a comparison: Miami

Defense DVOA
KC: +0.4%
MIA: -16.0%

Vast difference in DVOA, but how do they compare in other metrics?

PPG allowed compared to opponent averages (adjusted for standard 3pt Home Adv.)
KC: -5.82
MIA: -4.08

YPG allowed compared to opponent averages
KC: -20.25
MIA: -30.09

This makes me think that the DVOA formula lends heavily towards things like yards per play, where Miami is clearly better than KC...though I don't have numbers for opponent adjusted yards per play, which could reduce the gap some

Unadjusted Yards per Play Allowed(via Pro Football Reference)
KC: 5.3
MIA: 4.7

142
by coremill :: Thu, 11/20/2014 - 11:30am

KC's defense is also 2nd from bottom in turnovers forced, with only 8. DVOA and points allowed take that into account (at least indirectly, since forcing fewer turnovers leads to more points allowed), but yards allowed does not. Forcing very few turnovers is a good way to underperform your yardage allowed.

148
by chemical burn :: Thu, 11/20/2014 - 12:20pm

Oh wow, I didn't notice that - turnovers are what DVOA loves (and hates) most. Although I'm not sure if I agree with how heavily it weights them. The Greatest Show on Turf doesn't end up even close to one of the best offenses of All-Time, for example, because of their turnovers. If their defense isn't generating turnovers of course DVOA doesn't think too much of it - I wonder what their DVOA would jump to with a league-average amount of turnovers...

149
by cstoos :: Thu, 11/20/2014 - 12:21pm

That is a good point. I assumed forcing fewer turnovers would actually make the defensive DVOA better in that being below the mean for "random events" such as turnovers would imply opponents should have had fewer opportunities for both yardage and points.

151
by chemical burn :: Thu, 11/20/2014 - 12:29pm

It sees fumble recoveries as random, but not forcing fumbles or interceptions. Another weird way in which DVOA gets caught between being descriptive and predictive.

52
by Keith_1 :: Tue, 11/18/2014 - 11:35pm

I have been coming here for 10 years, and I cannot remember a year where more people complained weekly about DVOA ratings. It is becoming, as the commenters loves to say, egregious in nature. Every week of every month of every year, there is a team that -- for better or worse -- is rated above or below, but mostly below, the expectation of some group (read: fans of that team), and nearly every week of every month of every year, the DVOA rating are justified in some way.

The commenters on this site LOVE to purport superiority to all other comment sections. The reality is, outside of MilkmanDanimal and Will Allen (and their obvious self-flagellating fandom of plunder-teams), all the commenters seem to want to do is justify why their team is best. If I wanted that, I could go to ESPN to find random fans with their random metrics, eye-tests, and what-ifs.

At this point, I think I am just better off reading the articles and avoiding the comment section. With that in mind, can we get a box to show/hide the comments, as other major sites have implemented?

55
by Anon Ymous :: Wed, 11/19/2014 - 12:01am

I'm assuming this has to do with my comment, so for the record, I'll say that I have no problem at all with how NE is rated by DVOA. Nor do I have a problem with how it has rated NE all year. I fully understand how DVOA functions and I applaud FO for their consistency and integrity.

My earlier post was not to chide FO or to poke holes in DVOA, it was to point out that NE's data was too organized to dismiss as a simple anomaly. Had Aaron's comments been more along the lines of,

"NE clearly improved after a disastrous effort in KC, and while there is reason to believe that's how they will perform from here on out, DVOA still factors those early games in. If NE keeps up their current pace, they'll earn a higher spot as the year goes on, particularly in the weighted figures."

I would have remained silent :)

61
by Noah of Arkadia :: Wed, 11/19/2014 - 12:23am

Don't apologize, it's not a poster's job to please the readers. Besides, I find anything that brings to the fore the difference between teams as composed by human beings in constant evolution and teams as static mathematical formulas very interesting. In the end, the equation simply has no solution.

------
Who, me?

114
by CaffeineMan :: Wed, 11/19/2014 - 4:37pm

This, a thousand times over.

58
by Noah of Arkadia :: Wed, 11/19/2014 - 12:15am

When you say MilkmanDanimal and Will Allen are the better posters, are you talking about their VOA or their DVOA? Bear in mind Will Allen posts a lot, which might lower his DVOA despite his high VOA.

------
Who, me?

62
by RickD :: Wed, 11/19/2014 - 12:24am

But Will Allen's going to get a high DYAR with a high posting rate. And since he's not a running back, we might actually think of that number as being meaningful.

:)

87
by armchair journe... :: Wed, 11/19/2014 - 12:00pm

DYAR's the wrong side of the ball. Seems like the appropriate metric for him should be Adjusted Success Rate, no?

//AJMQB

60
by RickD :: Wed, 11/19/2014 - 12:23am

There was a comment by Anon Ymous that started all sorts of hand-wringing about people making demands about DVOA. In that comment was the following sentence:


Of course, teams with such a large line of demarcation are rare, so I fully support treating them as you would any other team. If you adjust the formula for NE, then you have to start doing it for everyone else and you ultimately end up with a subjective mess.

I supported the contention that something unusual is going on with the Patriots. I also did not demand that the Patriots be rated best, or that DVOA change, or anything like that.

My interest in today's discussion was to look at the general question of the shift in the level of the Patriots' play. There seems to be a general line of thought that thinks that a team is one entity, and each game is a test of its quality, a quality that doesn't very. But that's not what's actually going on. Every week is a different game, with different circumstances. Players get injured, other players return from injury, some get benched, etc.

Basically, while I appreciate a need to ignore the variation in team play level, it bothers me when people insist that the phenomenon doesn't exist at all. We're not sampling i.i.d. random variables here. It would be more accurate to say that we're tracing a random walk, or rather, each game is an interaction between two teams, each of which is taking a random walk. Modelling the entire complexity of the system is probably hopeless, esp. as the season is only 16 games long. But that doesn't mean we should pretend it doesn't exist.

It is entirely possible that a team can move from a middling level of play to a higher level of play. Indeed, it happens reasonably frequently (see, for example, the Ravens' Super Bowl run two seasons ago).

66
by PaddyPat :: Wed, 11/19/2014 - 2:58am

I honestly think that RaiderJoe wins FO MVP (Most Valuable Poster) by a landslide, and I would put RickD right up there as one of the most consistent and intelligent posters. I also could hardly take greater issue with the disparagement of these message boards. The point is that DVOA has not been as reliable over the past year+, as is clearly evidenced by the expert picks. We here of this community have come to expect a very high standard from Aaron and CO. which they have always delivered. That said, football is a constantly changing sport, from the practice schedules, to the rules and rule enforcement, to the strategic tendencies, etc. It seems perfectly natural that a formula for measuring the game would go through good and bad times and need to adapt to a changing sport. What could possibly be wrong with bringing flaws to the attention of the FO crew?

For my part, I think there may be something to the notion that early season is less predictive since the latest collective bargaining deal. It's also possible that shifts in injury trends have impacted the game. Perhaps the new penalty enforcement is having some odd implications. There seem to be 3-4 topics worthy of exploration to see if there are statistically significant trends that might impact a change in the measure of DVOA and DYAR. I have full confidence that FO will be on it.

106
by bravehoptoad :: Wed, 11/19/2014 - 2:29pm

There's a fellow who just posts as "Pat." He's trailed off a lot recently and these days posts late in the lifespan of a thread, but his posts get my vote for best.

152
by chemical burn :: Thu, 11/20/2014 - 12:31pm

Pat was great and probably the sharpest person to regularly comment, my only complaint about him was that he was totally humorless. I really enjoy reading the comments here because a lot of them are funny or as least fun - this site has never evolved into deadly discussions of math and probability, which is a good thing...

88
by Malene_copenhagen :: Wed, 11/19/2014 - 12:01pm

This is basically THE question for Aaron to look at in the offseason. The hypothesis of a "platonic ideal" of team quality does not look to be as strongly supported by data as it used to.

71
by BDC :: Wed, 11/19/2014 - 6:49am

"At this point, I think I am just better off reading the articles and avoiding the comment section. With that in mind, can we get a box to show/hide the comments, as other major sites have implemented?"

It's interest you mention this. I too have been coming here for 10+ years, and a large part of that is because of the comments. I am not nearly as enamored with DVOA as I was 10 years ago but I almost always find the comment section interesting (even for articles I didn't care for very much or downright hated).

I guess the point is, to each their own?

108
by bravehoptoad :: Wed, 11/19/2014 - 2:32pm

If you can't tell the difference between ESPN comments and FO comments, I don't know what to tell you. But if it's too difficult for you to navigate away from a page before some evil comments strike your eyeballs, I can think of some tips.

131
by mehllageman56 :: Thu, 11/20/2014 - 4:27am

Not sure whether I count as a regular, but I definitely do not argue that my team is the best. Right now I'm rooting against them because of their quarterback, and because I would like to root for Mariota in NFL green. There are plenty other posters who will criticize their favorite teams in an intelligent manner on this site. Hell, this entire argument is over whether a mathematical formula actually works. Where on ESPN would people argue about that?

161
by chemical burn :: Thu, 11/20/2014 - 4:53pm

Wanting your coach and QB to get fired is the Jets equivalent of homerism.

64
by techvet :: Wed, 11/19/2014 - 12:53am

Regarding the health issues, get well soon!

Beyond that, ARI has a helluva schedule coming up. We shall see soon if their record catches up to their DVOA.

67
by PaddyPat :: Wed, 11/19/2014 - 2:59am

or vice versa, no?

70
by big10freak :: Wed, 11/19/2014 - 4:48am

I was curious to see how Green Bay's erratic special teams performance would play out.

Very few folks in Green Bay regard this defense as anything more than ok because of how in some games either the Packers were run over or had qbs go wild.

But one can see incremental improvement as Clinton-Dix continues to improve, Jones stays nailed to the bench, Datone Jones comes back from his ankle injury and Josh Boyd anchors the other end position.

If Shields would stop getting beat deep and play to his contract that would be nice.

82
by dank067 :: Wed, 11/19/2014 - 11:04am

Another defender who has improved this year: Morgan Burnett. Doing a great job in the run game, which is really helpful considering our ILB problem. Clinton-Dix still makes mistakes and misses tackles, but as you point out he definitely has been improving, and he's been good enough lately that he's an every down player now. That allows Micah Hyde to be a full-time nickel back, which also helps the run defense. I still worry about Hyde getting exposed in man-to-man coverage, but if that becomes a significant problem, you have Casey Hayward.

Overall I'm also still hesitant to believe that this defense is any better than just ok. The front 7 is still soft against the run, the pass rush is decent at best, and I truthfully don't trust either safety in coverage against a great passing attack. But it's nice to see some things coming together.

175
by justanothersteve :: Fri, 11/21/2014 - 5:06pm

Morgan isn't playing much better. It's more that he is the deep safety less often so doesn't have to be the last line of defense every play. Jason Wilde said this week Hayward has replaced Hyde in the nickel.

177
by KB :: Fri, 11/21/2014 - 7:38pm

He is a far better safety in the box. It has really helped him having better players on the back end with him. I think he was trying to make up for the deficiencies of MD Jennings last season and having that much on his plate really didn't work out for him.

I fully expected him to get back to his old self and IMO he has even improved. I'm a big fan of GB's secondary, I actually think its one of the better and easily one of the deepest in the NFL.

Hayward would nearly be a full time starter on most teams but isn't even seeing the field 50% of the time even though statistically he has been one of the best slot CB's in the NFL. I mean he was allowing a passer rating under 30 last time I checked with the most snaps in coverage per reception at 18(2 above Revis at 2nd).

178
by KB :: Fri, 11/21/2014 - 7:40pm

Shields is allowing a lower passer rating and comp % than Peterson and the same comp % as Revis(I just used the 2 players right next to him as an example. There have been a couple plays this year where he was beat deep absolutely but he has still performed really well in coverage. I think his only real problem this year has been the missed tackles. I considered him a top 10 CB last year and still do this year.

He started a bit slow and was really looking good before the injury( Look how bad GB did without Shields against Mia in the 2nd half and the Saints). I expect him to pick it back up now that he has been back a couple weeks and perform once again like a top CB. I mean the last few years he has been one of the best in the NFL as far as Comp %, being right around 50% and under. He also is targeted far less than most starting CB's because he is in such tight coverage most of the time.

I for one consider this GB defense better than okay. They're improving each week and with Matthews in the middle going a long way to shoring up the run defense, the pass rush clicking on all cylinders and the amount of playmakers on the back end, I fully expect this defense to be considered a top 10 unit at the very least by the end of the season.

75
by Mugsy :: Wed, 11/19/2014 - 10:00am

The beasts of the east?
Looks like the AFC East & the NFC east are going to be very competitive this winter... dolphins, Pats, Cowboys, and eagles -- all look to be pretty strong now.

76
by Okchief :: Wed, 11/19/2014 - 10:01am

Miami and NE ahead of KC? Kansas City defeated Miami on the road and obviously, KC beat NE but THAT loss is simply a bad loss for NE. I love this site but when the stats miss the "point" it is more than interesting and becomes suspicious. As a long time Chief fan (and Royals), it is easy to disbelieve my eyes. But the Chiefs look like a different kind of team. This site tries to take the bias out but it appears to bake the bias in.

86
by BJR :: Wed, 11/19/2014 - 11:43am

KC is ranked 6th with a weighted DVOA of 20%: you realise that's good, right? So they are a hair's breadth behind a couple other AFC teams, but if all you wanted to be told was that KC is better than Miami and New England because they defeated them, why did you come here? You already knew the results of the games.

Still, if we are talking about one game weighing down a team we have to mention the Chiefs week 1 stomping by the Titans. Since then they have obviously played very well. That has to be the most inexplicable result of the season.

100
by Okchief :: Wed, 11/19/2014 - 1:19pm

I understand and it is all in fun anyway. I realize that the word "bias" sounds hostile but that is not my intent. I am fascinated by the numbers and actually enjoy the different "realities" they display. I admit I'm really just a fan of the Chiefs and like having my fandom stroked.

174
by Tim F. :: Fri, 11/21/2014 - 3:35pm

Still think DVOA had KC wrong after last night?

80
by RickD :: Wed, 11/19/2014 - 10:56am

Well, KC beat the Pats, sure. And the Pats beat the Broncos and the Broncos have beaten the Chiefs.

Team A beat Team B so Team A should be rated ahead of Team B doesn't work for very long.

Should the Titans be ahead of the Chiefs? Without that loss the Chiefs would currently be slotted in as the #1 seed in the AFC.

99
by Okchief :: Wed, 11/19/2014 - 1:17pm

I get it and I do believe KC is getting some props but they totally destroyed NE. They took Denver down to the wire in Denver. I can buy Denver a notch above the Chiefs at this point but not NE. It just seems that it matters too much where you start in expectations. But that's also the fun of it.

129
by dbostedo :: Thu, 11/20/2014 - 1:07am

"It just seems that it matters too much where you start in expectations."

But DVOA doesn't include ANY presumptions about a team. This isn't like an NCAA poll where you bubble up and down based on each week's results; And therefore have a hard time getting high if you started low. In this case, every team starts out even.

They do have the DAVE rankings separate for the first half of the season that factor in the preseason DVOA projections. But the regular DVOA rankings don't use the projections.

109
by bravehoptoad :: Wed, 11/19/2014 - 2:34pm

There used to be a cool site called Beatpaths that rated teams by just this--who beat whom. It was a hoot. Sadly abandoned now.

111
by nat :: Wed, 11/19/2014 - 2:48pm

http://www.beatgraphs.com/graphs.php?league=NFL

It's like an automated sports bar argument. Oh yeah? But Miami beat New England! So there! And your beer looks like piss!

117
by Anonymouse :: Wed, 11/19/2014 - 6:12pm

Can we get DVOA for Rocks/Paper/Scissors? Rock has beat Scissors the last trillion times they've played (true fact, look it up!), but my friend (who is a Paper fan) says this is their year. I think he is full of it...

134
by Okchief :: Thu, 11/20/2014 - 9:21am

That is funny.

135
by Okchief :: Thu, 11/20/2014 - 9:21am

That is funny.

125
by CaffeineMan :: Wed, 11/19/2014 - 11:51pm

"automated sports bar argument" is awesome..

One thing I liked about the site when the guy first put it up was that the (tongue in cheek) subtitle was "The Winning Ways of Winners."

130
by intel_chris :: Thu, 11/20/2014 - 1:07am

Anyone have any idea how to contact the Beatpaths people? I wouldn't mind doing the work to resurrect that site.

139
by Travis :: Thu, 11/20/2014 - 10:40am

The Beatpaths site has moved here, as pointed out in #111 above.

145
by intel_chris :: Thu, 11/20/2014 - 11:42am

Thanks. I guess I need to work on my reading skills. But I'm happy I can bookmark the site. I missed it at least all last year and perhaps longer, and Google search wasn't finding it for me.

176
by BeatGraphs :: Fri, 11/21/2014 - 7:17pm

For the record, I am not the same person who ran the BeatPaths site. Originally I was a visitor there and when we came up with a couple other methods than he showed, but was unable to add them, I began posting them. When he was unable to continue at all, I took over. Feel free to contact me if you wish via the email link on the site.

~May all your sports dreams come true.

115
by Otis Taylor89 :: Wed, 11/19/2014 - 4:48pm

I think the thing that questions the Chiefs is the lack of quality WRs. They do have some very good TE's, but the WR's are going to have to do more than be good blockers for their running game...you would think. If they get behind by 14 or more in a game, especially on the road, they could be in trouble.

79
by nat :: Wed, 11/19/2014 - 10:54am

Aaron says:What baffles me is not that their [Arizona's] record is better than their DVOA rating, but just how much better it is.

There may be nothing to be baffled about.

First, and obviously, you need to look at VOA, not DVOA. I expect you already know that and say DVOA out of habit.

But I think more interestingly, you can divide the VOA v. Record discrepancy into two parts: VOA v. Scoring and Scoring v. Record. You do this by looking at Pythagorean win projections.

For the rest of this, I used the exponent of 2.37 from the Wikipedia article on Pythagorean Expectation for professional football, and projected records for the number of games played so far rather than for the full season.

I calculated the Pythagorean projection for each team using its points for and against so far this season. I plotted that against VOA and dropped in a simple trend line and formula. From that formula I get the difference between what VOA would expect for the Pythagorean projected wins, and what the team's projection is based on actual scoring.

For Arizona, their Pythagorean projected win total is 1.2 higher than we would expect from the VOA. If VOA were broken, this is where it would show up. But having an outlier team be off by a little more than one game is not bad at all.

Next I looked at Pythagorean projections v. actual win-loss records. This is all about scoring a the right times, something that VOA doesn't attempt to measure.

For Arizona, their actual winning percentage is 2.31 wins higher than you would project from their points scored and allowed. This is where Arizona has been surprising. And it has nothing to do with DVOA.

As far as mismatch between VOA and actual points for and against (via the Pythagorean projection), Arizona is not the most surprising team. That is Denver, whose Pythagorean wins are 1.3 below what their VOA would indicate. They are putting up good VOA stats, but not converting those into points allowed or prevented very well.

In conclusion: if you feel the need to investigate why VOA is underrating the Cardinals, you need to look at why it overrates the Broncos, too. But probably they are just edge cases.

83
by nat :: Wed, 11/19/2014 - 11:22am

Here is the data:

Columns: team, record, pythagorean projected record, VOA projected pythagorean record, excess wins (real v. pythag), excess wins (pythag v. VOA), total excess wins

ARI 90.00% 66.94% 55.13% +2.31 +1.18 +3.49
DAL 70.00% 62.08% 56.01% +0.79 +0.61 +1.40
PHI 70.00% 60.22% 56.54% +0.98 +0.37 +1.35
NE. 80.00% 71.74% 67.55% +0.83 +0.42 +1.25
KC. 70.00% 69.28% 61.21% +0.07 +0.81 +0.88
SD. 60.00% 57.47% 51.26% +0.25 +0.62 +0.87
DET 70.00% 60.88% 61.56% +0.91 -0.07 +0.84
SF. 60.00% 49.72% 52.58% +1.03 -0.29 +0.74
PIT 63.60% 55.36% 57.16% +0.91 -0.20 +0.71
CIN 65.00% 50.80% 53.11% +0.92 -0.23 +0.69
CLE 60.00% 56.03% 53.37% +0.40 +0.27 +0.67
CHI 40.00% 32.98% 34.44% +0.70 -0.15 +0.55
STL 40.00% 31.25% 35.14% +0.87 -0.39 +0.48
IND 60.00% 61.81% 56.63% -0.18 +0.52 +0.34
MIN 40.00% 38.64% 37.08% +0.14 +0.16 +0.30
HOU 50.00% 56.81% 47.82% -0.68 +0.90 +0.22
NYG 30.00% 35.65% 30.65% -0.57 +0.50 -0.07
CAR 31.80% 31.23% 28.01% -0.43 +0.35 -0.08
GB. 70.00% 71.25% 70.90% -0.13 +0.04 -0.09
SEA 60.00% 61.07% 61.21% -0.11 -0.01 -0.12
BUF 50.00% 48.83% 52.49% +0.12 -0.37 -0.25
WAS 30.00% 36.86% 36.46% -0.69 +0.04 -0.65
DEN 70.00% 65.39% 78.47% +0.46 -1.31 -0.85
TB. 20.00% 29.71% 30.30% -0.97 -0.06 -1.03
MIA 60.00% 68.33% 70.54% -0.83 -0.22 -1.05
NO. 40.00% 52.08% 52.49% -1.21 -0.04 -1.25
ATL 40.00% 45.92% 52.67% -0.59 -0.67 -1.26
NYJ 20.00% 26.95% 34.96% -0.70 -0.80 -1.50
BAL 60.00% 70.42% 75.21% -1.04 -0.48 -1.52
JAC 10.00% 20.21% 25.28% -1.02 -0.51 -1.53
TEN 20.00% 28.05% 37.17% -0.80 -0.91 -1.71
OAK _0.00% 21.13% 21.75% -2.11 -0.06 -2.17

Arizona looks so surprising because it is an edge case both for scoring better than expected from VOA, and winning more than expected from scoring.

And poor Oakland. It sucks to be you.

95
by ChrisS :: Wed, 11/19/2014 - 12:37pm

Interesting and useful post. What is the formula for VOA Pythag projected wins/record. I don't see a huge resemblance between the formula for pythagorean projected wins and the Pythagorean Theorem. Variables raised to powers seems pretty common in Math.

97
by nat :: Wed, 11/19/2014 - 1:01pm

I'm away from my spreadsheet, so I don't have that formula handy right now.

I agree that the "Pythagorean" name is a stretch. Baseball's original formula used an exponent of 2, which reminded people of A^2 + B^2.

But the plot of record v. pythagorean projection looks pretty good. Wins really are explained well by points scored and points allowed. Who'd a thunk it!

168
by leviramsey :: Thu, 11/20/2014 - 11:43pm

I've long preferred the term Jamesian sine.

If you use the exponent of 2, you have expectation = a^2/(a^2 + b^2).

Remembering what a, b, and c are in the pythagorean theorem (a^2 + b^2 = c^2, a,b: lengths of legs of a right triangle, c: length of hypotenuse), then if we construct a right triangle where one leg's length is points scored and the other leg's length is points allowed, then the length of the hypotenuse is sqrt(a^2 + b^2). If we square the ratio of the first leg to the hypotenuse, we get a^2/(a^2 + b^2); that ratio (before squaring) is the sine of the angle opposite the points scored leg. (Jamesian sine squared is too wordy, so I truncate it a little).

By calling it a sine, it carries with it a slight implication that the exponent can vary, based on particular features of a non-Euclidean geometry.

179
by CaffeineMan :: Sat, 11/22/2014 - 3:04pm

"Jamesian sine"

I hadn't heard that term, but like it...

181
by intel_chris :: Sat, 11/22/2014 - 9:02pm

Ok, after seeing it referred to frequently enough here, I did a Google search on non-Euclidean geometery James[ian] Sine and could not find a "James" to whom to associate it with. So, please, pray tell from whence the term comes....

182
by tuluse :: Sat, 11/22/2014 - 9:27pm

Clearly it's a graph of how enjoyable a person is to be around vs how much they've had to drink (bounded from 0 to 1.5).

183
by Jerry :: Sun, 11/23/2014 - 3:19am

Bill James, who developed and named "Pythagorean record" using runs scored and runs allowed a few decades ago.

184
by intel_chris :: Sun, 11/23/2014 - 5:21pm

Thanks.

104
by nat :: Wed, 11/19/2014 - 1:59pm

Expected Pythagorean Projection = 0.8807*(VOA)+0.4932
Pythagorean Projection = PointsFor^2.37 / (PointsFor^2.37 + PointsAgainst^2.37)
Record = (Wins + 0.5*Ties)/(Games Played)

Each can be multiplied by games played to get Wins.

Take that VOA-based formula with a modicum of salt. It's based on 2014 only, and I can't justify the significant digits anyway. That's what happens when you just use Excel for stuff. :-)

107
by dmstorm22 :: Wed, 11/19/2014 - 2:31pm

Great, interesting work on this.

I have seem some work done recently to update the exponent for the Pythag. Projection for hte NFL away from 2.37, mainly done I believe to account for scoring increases (current system overrates great defenses. Ex, team with a scoring diff. of 350-200 would rank better than 415-250).

I forget the exact new formula, but it uses Log iirc, and it does seemingly do a nice job of leveling scoring increases and adjusting for offense-heavy teams.

169
by leviramsey :: Thu, 11/20/2014 - 11:46pm

I run a regression to find exponents for the NFL regularly over the past decade-plus, and it tends to be in the 2.4 - 2.45 range of late. It's not so much scoring increases, in my experience, but an increase in the number of points scored by losing team that does it.

118
by ChrisS :: Wed, 11/19/2014 - 6:19pm

So for VOA you used Excel to solve the linear regression parameters. Seems like you (not me though, I should be working) could get fancy with some of the more recent years data and look at the components of VOA (off, def, st) and get some pretty equations.

138
by nat :: Thu, 11/20/2014 - 10:35am

I looked at 2013. The Expected Pythagorean Projection formula was

EPyP = 0.7810*VOA + 0.4988

So I'd trust this all to about one, maybe two digits. And since 0% VOA is supposed to represent an evenly played game, I'd probably use something like this, and just accept that it's pretty approximate:

EPyP = 0.83*VOA + 0.5

Or simply put, every 6% VOA (or DVOA for projecting future play) is worth about a 5% boost in your winning percentage over the course of a season. That obviously can't work at the extremes, since your winning percentage cannot exceed 100% or be less than 0%. But the graphs look pretty linear for typical VOA ranges.

I'm satisfied with the principle that VOA and DVOA should be thought of as projecting Pythagorean Wins rather than wins. That clears up some angst about teams like the Cardinals and the Raiders.

102
by Anon Ymous :: Wed, 11/19/2014 - 1:27pm

Thanks for posting this. Good stuff.

146
by JoeyHarringtonsPiano :: Thu, 11/20/2014 - 11:54am

"And poor Oakland. It sucks to be you."

I'm picturing RaiderJoe looking at your chart, shaking his head sadly, and chugging another bottle Sierra Nevada.

84
by Digit :: Wed, 11/19/2014 - 11:33am

One thing I can think of that might affect DVOA this year is the recent change in training camp sessions - I believe they were required to have less hitting and less sessions and as a result, some things that normally would have been practiced pre-season were delayed to the regular season, so the early season games might be -less- predictive- than years past as teams become more and more familiar with their playbooks.

So teams that needed less familiarity with personnel and playbooks and had healthy personnel would be ahead of other teams early on, before the pounding took its toll and depth becomes necessary.

112
by fb29 :: Wed, 11/19/2014 - 3:01pm

Link to Tanier's article that mentions the blowout stats

http://bleacherreport.com/articles/2267832-nfl-week-11-picks-eagles-pack...

and the stats

Year Blowouts per 16 games
2014 : 4.2
2013: 3.3
2012 : 3.5
2011: 3.4

119
by ronnyo :: Wed, 11/19/2014 - 6:40pm

I don't know exactly how DVOA is calculated, but if it's close to what I think it is (linear random-effects model? Bayesian model?), then the following assumptions would apply:

A. Teams (or players) possess some latent, unidimensional construct that reflects their effectiveness on offense/defense/special teams
B. The expected outcome of each play/game is a function of both teams' constructs
C. Observed outcomes are "noisy" observations of expected outcomes
D. Constructs are assumed to be constant, or at least to change very slowly, within the unit of observation (teams for DVOA, players for DYAR)

Is that correct?

If so, then A and D may help explain the differences between DVOA rankings and "conventional wisdom." The problem would be that those assumptions don't leave room for the possibility that teams might "genuinely" improve dramatically in a fairly short period of time.

For the Patriots, for example, suppose their performance in games 1-4 was lousy and their performance in games 6-10 was much better. If their "true" effectiveness is constant (or change very slowly over the season), then games 1-4 would have negative errors, pulling their DVOA estimate down, while the remaining games would pull it up. However, if what actually happened was that they simply got much better after game 4, then you'd have two DVOA estimates, and games 1-4 would have average errors closer to zero, as would games 5-10. The problem is that with the data available, we can't tell the difference, at least without some additional assumptions, like picking arbitrary endpoints, or picking an arbitrary number of DVOAs per team to estimate.

Where prediction is concerned, though, the difference isn't trivial. In the first case, their next game performance against a high-ranking opponent would be expected to be mediocre, but in the second case, we would expect it to be much better.

Another problem is the potential for game-specific errors, i.e. effects such as sloppiness or illness or "high energy" that apply to all plays in a particular game. Similarly, it seems that team x team interaction effects (i.e., if one team happens to match up particularly well against another due to specific personnel, coaches, etc.) can't be estimated within a single season for non-divisional opponents.

Am I off-base here?

124
by Dr. Bill :: Wed, 11/19/2014 - 11:08pm

ronnyo, you're touching the heart of the issue. DVOA's unit of analysis is the team, and the data selection is bounded by seasons. DVOA is useful as a rich, empirical description of team performance over a season, one that goes beyond fan eye-tests. To the degree that teams are (relatively) static constructs, this description of past performance has predictive value. To the degree that teams are (relatively) dynamic constructs, this rich descriptions of past outcomes has low value.

If next week a given number of key players on a team were to be hurt--let's say Denver looses Von Miller, Peyton Manning, and Demaryius Thomas--and the alternatives have lower replacement value, prior description from DVOA has much lower predictive value. It doesn't matter that after 15 games Denver has the highest DVOA in the league--if the replacement players have significantly lower replacement value, then the old description isn't relevant to the new construct.

This is a the weird tension between DVOA and DYAR. When Aaron and others talk about DVOA, they tend to emphasize the correlation of DVOA and future performance, as if the description of an outdated construct was still relevant to future outcomes. But then when they talk about DYAR, they emphasize dynamism and the relative value of starters over replacements:

"When a player is removed from an offense, he is usually not replaced by a player of similar ability. Nearly every starting player in the NFL is a starter because he is better than the alternative."

It can't be both ways--teams can't be static constructs, but dynamically losing Peyton Manning is losing 1,004 yards.

The answer to this isn't to throw out DVOA. DVOA is still adequately descriptive, but it only has predictive value if the team is fundamentally the same construct. If New England is a different construct without OL rotation, with Stork/Wendel/Connolly playing the interior, with Gronkowski playing, etc. then DVOA still describes what it purports to describe: team performance for a season. But it doesn't have a lot of predictive power.

132
by Jerry :: Thu, 11/20/2014 - 5:42am

I have no argument with you, or ronnyo, who's on-base. A couple of things, though:

-As wonderful as one number that reflected the current quality of a team would be, it would be impossible to put together with any sort of accuracy. Evaluating the exact state of a team at any point in time is just too hard.

-There's value to including the entire season in DVOA. In the same way that a team's record reflects good games and bad, with good players and bad, full-season DVOA does the same thing. The Tony Romo Cowboys may be better than the Brandon Weeden Cowboys, but Dallas' 2014 season includes both.

-The stuff about replacement level is just a question of how to evaluate individual players. If it was yards above average, a player who ran once with an above-average result would rank higher than a back with 100 average runs. While the one-rush player will have a higher DVOA, DYAR reflects the usefulness of accumulating many average runs. (Obviously, many above-average runs will lead to a better DYAR.)

185
by ronnyo :: Tue, 11/25/2014 - 3:20am

Yes, I think regardless of predictive ability, DVOA is still the best way to summarize a team's overall performance to date.

Question, though: Is the weighting scheme for Weighted DVOA essentially arbitrary? I.e., it's not based on some empirical investigation of "optimal" weights for predicting performance later in the season, is it?

128
by intel_chris :: Thu, 11/20/2014 - 1:05am

I don't think you are off-base. I think if one does play-by-play metrics the way DVOA does, it is quite likely you have enough data to separate a season into distinct sub-populations. I don't know the name of the precise statistical test one would apply, but I would be extremely surprised if such a test didn't exist.

122
by Bruce Lamon :: Wed, 11/19/2014 - 6:48pm

Re premium picks v. DVOA: I stopped relying on premium picks because simply using 5% DVOA = 1 pt. and 3 pts. for homefield advantage performed better. Small sample size, though.

126
by Jetspete :: Thu, 11/20/2014 - 12:06am

When it comes to predictive value against the spread, DVOA is a victim of its own success. Eight years ago, very few people were using advanced matrices. Now they are main stream, which means Vegas is factoring this enhanced public knowledge into the lines. Football Outsiders used to be the best place to find live dogs….now i think they're the best place to find undervalued favorites.

170
by mitch :: Fri, 11/21/2014 - 9:26am

Looking back weeks from now you'll realize Pats just came-off what will be they're best run of games this season. Many overreact at this now but will realize down the road how silly that was.

It's was the same with Denver, as I posted at the time with Denver being the 6th best team in DVOA history.

And look at Denver now, a few weeks from now Pats may not look so great and the fact they are ranked only 5th after playing they're best run of games is not very promising for this team long term. Pats are not a team to win the SB, they are not that good, they're defense is still the weak link and Brady has not been the same QB in the playoffs for many years now. He's been one of the worst when throwing the ball 20 yards or more down field.