Guest column by Matt Bittle
Who had a better regular season: the 1988 Bengals, who posted a DVOA of 27.5%, or the 2004 Patriots, who had a DVOA of 34.2%? Obviously, the 2004 Patriots, right? Well, not necessarily. If we're asking which team stood out more in that season, the answer is the 1988 Bengals, because they had a superior z-score.
For those unfamiliar, z-scores, also known as standard scores, measure how many standard deviations an item is from the mean. To put it another way, they indicate how widely dispersed a set of values (in this case, team DVOAs in a specific season) are. Applied to football, it essentially reflects the fact different seasons are just that -- different, not identical. Quality varies from season to season; just compare the extreme individual team records of 1976 to those of 1979. Put simply, it's easier to excel when there are more bad teams.
DVOA may be normalized, but a DVOA of 20.0% in one year is not necessarily equal to a DVOA of 20.0% in another year. Take the 2016 Patriots, who led the league with a DVOA of 24.9%. They would have ranked sixth in 2009 with that DVOA. Their z-score, however, based on the standard deviation in 2016, comes in at 1.71. In 2009, that same DVOA would translate to a z-score of 1.13.
Continuing with this example, the 2016 Patriots are tied for 96th since 1985 (the first year that DVOA is available) in DVOA, but 35th by z-score. The hypothetical 2016 Patriots and their 24.9% DVOA transplanted to 2009 would come in at 160th in a ranking of z-scores.
The 1988 Bengals, meanwhile, are 27th all-time in z-score but tied for 65th in DVOA. Their z-score in 2016, by the way, would be 1.88, which would be 18th among all teams! The 1988 Bengals are not remembered as a great team, but that may be mostly because they didn't win the Super Bowl.
So, back to our initial example. The 1988 Bengals may have had a lower DVOA than the 2004 Patriots, but their z-score is higher, 1.78 compared to 1.69. That's not a huge difference, of course, and it certainly doesn't mean Cincinnati was definitely superior; it's just a new way of looking at things.
Best/Worst Teams by DVOA Z-Scores
Here's a list of the top 25 teams by DVOA z-score with rank, as well as actual DVOA and rank. We've also averaged the ranks in both categories for each team.
Best DVOA Z-Scores, 1985-2019
The lists are a little different, but there really is not too much movement here. The 1991 Redskins are first in both categories despite the fact that 1991 had the third-highest standard deviation for any season (more on that later). The 1987 49ers (strikebreaker games not included) and 2007 Patriots each claim second in one of the two categories. The 1995 Cowboys and 2014 Seahawks look a bit better by z-score, as do the 1998 Broncos. Notably, the 1994 Steelers shoot up 20 spots, while the 2003 Chiefs jump 15. This ranking also makes it even wilder that the 1995 49ers won only 11 games.
The 2018 Chiefs and 2000 Titans (who rank 20th in DVOA at 33.3%) offer another great illustration of why z-scores are useful: the teams have very similar DVOAs, but the NFL was more balanced, in a sense, in 2018. 2000 had its share of awful teams, and so the Titans, while slightly higher in DVOA than the Chiefs, are 27 spots lower by z-score.
Going farther down the list a bit, we find some real outliers. To name a few: The 2001 Rams are 87th in DVOA but 44th in z-score, and the 1988 Vikings are 96th and 49th, respectively. The 2005 Colts are 26th in DVOA but 67th in z-score, while the 2005 Broncos come in tied at 34th in DVOA but rank 80th in z-score.
Notably, 2005 saw the 49ers post a DVOA of -55.5%, the worst ever. Well, I'm pleased to report they are also worst by z-score. Consider them the inverse of the 1991 Washington team. Here are the worst DVOA z-scores:
Worst DVOA Z-Scores, 1985-2019
By z-scores, the 2009 Lions fall (jump?) from second to eighth. The 2003 Cardinals and 1987 Falcons come in at second and third in z-score (the 2003 Cardinals are slightly worse if you go out one more decimal place). Those two teams are ninth and sixth, respectively, in a ranking of the worst teams ever by straight DVOA.
The 2009 Rams are 20th in z-score but seventh in terms of actual DVOA. The inclusion of the 2016 Jets also jumps out, as they're just the 22nd-worst team ever by DVOA but come in at 11th by z-score.
Year-by-Year Standard Deviations
I have mentioned 2016 and 2009 several times. Well, there's a good reason I picked those two seasons: no two seasons since 1985 were more extreme than 2009 and 2016. Teams were more "spread out" in 2009 than in any other year. That year, 14 teams posted DVOAs of 10.0% or greater, with six exceeding 20.0%. In 2016, there were just seven teams with a DVOA in the double digits, including two of 20.0% or more. On the flip side, 2009 saw nine teams with DVOAs of -10.0% or worse, including the Rams at -45.1% and the Lions at -51.6% (there really was a tremendous race to the bottom between St. Louis and Detroit in 2008 and 2009). In 2016, although seven teams had a DVOA of -10.0% or worse, the bottom-feeders -- those Jets at -32.4% -- would have ranked just fourth worst in 2009.
2009 and 2016 are both pretty clearly the most extreme years in our sample. The standard deviation for 2016 was 14.6%, the lowest on record; 1988 is in second place at 15.4%. The standard deviation for 2009 was 22.0%, the highest on record; 2005 is in second place at 21.2%.
NFL Seasons by Standard
There's no obvious factor causing 2009 and 2016 to be so extreme; it's just seasonal variance -- again, compare 1976 to 1979. And speaking of those two "pre-DVOA" seasons…
Best/Worst Teams by Estimated DVOA Z-Scores
I've also calculated the standard deviation for all teams post-merger using estimated DVOA for the 15 seasons where DVOA is not available. Using z-scores for these years produces some interesting results. I'm not sure how much of that is due to these DVOA ratings being estimated and not based on actual play-by-play and how much stems from other factors, like how top-heavy the league was for much of the 1970s.
There's a big divergence there; the 1979 Steelers have the best z-score for estimated DVOA (by far, at 2.49) but rank ninth in DVOA. The 1978 Cowboys are second by z-score but 17th by DVOA. The 1976 Steelers, who have the highest DVOA of all teams in this timespan at 42.7%, are only ninth in z-score. (1976 was a weird year -- probably the most imbalanced of all post-merger seasons.)
Here are the top 25 teams from 1970 to 1984 by z-score.
Best Estimated DVOA Z-Scores, 1970-1984
The 1979 Steelers are not generally considered the best of those great Steel Curtain teams -- I wager most fans would pick Pittsburgh's 1975 or 1978 squads -- but by this metric, they're not just the best team from that dynasty, they're one of the best ever. Look how big the gap is between them and the second-ranked 1978 Cowboys!
There are some truly great teams on the list above that may not be considered such. The 1978 Cowboys went 12-4 and lost to the 14-2 Steelers in their Super Bowl, while the 1983 49ers were much better than their 10-6 record would suggest and nearly beat a great Washington team in the conference title game. The 1980 Eagles came up short in the Super Bowl, while the 1979 Chargers and 1973 Rams both had outstanding regular seasons and then lost their first playoff game. (Also, seeing the 1983 Redskins ranked third makes me wonder: how many coaches had two different teams as good as Joe Gibbs' best Washington squads?)
The 1976 Steelers are a fun bunch: they went 10-4 despite outscoring their opponents by 14.6 points per game, 2.4 more than anyone else. Pittsburgh started 1-4, with their losses by a combined 19 points. Then, over the ensuing nine games, the Steelers allowed 28 points, total. Pittsburgh did lose in the AFC Championship Game to what I consider a vastly overrated 13-1 Raiders team (6-0 in one-score games). Thus died the dreams of a threepeat, the closest any team has come to winning a third consecutive Super Bowl.
Best/Worst Offenses/Defenses by Z-Score
Anyway, how about the top units? Which offenses and defenses had the top z-scores? The best offenses by z-score remain the 2007 and 2010 Patriots, in that order, but the third-ranked team is a surprise. Take a guess. Seriously, go ahead.
It's the 1988 Bengals. An offensive DVOA of 30.7% is very, very good, but it's only 16th all time. 1988, however, happened to be one of those years when the league's teams were grouped more closely together, and so the Bengals jump up the list when considering standard deviation.
Best Offensive DVOA Z-Scores, 1985-2019
Those are the top 10 offenses by DVOA z-score. Nine of these teams either made it to the conference title game or won at least 13 games. And then there's the 2002 Chiefs, who led the league in scoring and went 8-8.
If you include estimated DVOA, the best offense in the 50 post-merger seasons is the 1982 Chargers. Although they would rank below both of the aforementioned Patriots teams in DVOA, San Diego tops New England by z-score. (Granted, the Chargers only played nine regular-season games.)
The 2002 Texans remain the worst offense in z-score as they are in DVOA.
The title of best defense in the DVOA era by this method stays with the 1991 Eagles. That team isn't thought of as one of the best ever in most circles, but by DVOA, those Eagles were legendary.
The 2002 Buccaneers are second by z-score, which is not surprising, but the third-ranked team once again will probably make you raise your eyebrows. It's the Bears.
No, not that Bears team. Not that one either.
The 2018 Bears. Khalil Mack and company benefit here big-time, shooting up from the 10th spot. Those Bears may not be thought of as an all-time great unit, but by DVOA, the second-ranked defense in 2018 (Buffalo) was closer to the No. 10 Colts than they were to Chicago.
Best Defensive DVOA Z-Scores, 1985-2019
Again, note the repeat offenders. For what it's worth (not much), one of those top 10 offenses and three of those top 10 defenses won the Super Bowl.
The 2015 Saints and 1986 Buccaneers are the two worst teams by defensive DVOA and also rank as one and two in z-score. The 1997 Bengals jump from 28th to fourth worst going from straight DVOA to z-scores.
With estimated DVOA added, the 2015 Saints aren't the worst defense in the post-merger era, however. Both the 1976 Seahawks and 1981 Colts come in below the Saints, with the Seahawks having a worse (estimated) DVOA and the Colts having a worse z-score.
While Baltimore is a close second in DVOA, no one else comes anywhere near the Colts in z-score. (Fun fact about the 1981 season: Both the Patriots and Colts went 2-14 despite the fact Baltimore was outscored by 14 MORE POINTS PER GAME than New England. Both the Colts' victories came against the Pats, bookending the season.)
If you have any questions or whatnot, please share. I'm happy to give the z-score for any team in this sample upon request.
Matt Bittle is a writer in Delaware. Prior to COVID-19, he was really hoping the 2020 Ravens would follow in the footsteps of the 1988 49ers.