Writers of Pro Football Prospectus 2008

06 Nov 2006

DVOA Remembers

The DVOA rankings are out, and frankly, they look a little funky to the naked eye. Eagles fourth? Colts seventh? Needless to say, our Inbox is humming with people who think that the Football Outsiders have been hitting the sauce and pouring a little onto our computers for good measure.

I know we can never convince the people who write "Youze suck. Yur ratings are stoopid." that our system is really the best one available for evaluating NFL teams. But I can clarify the DVOA system, in a non-technical way, just a little for fence-sitters who may be wondering whether or not we've lost our minds. To do this, let's look at a hypothetical game between Home and Away.

Say Home gets the ball at the start of the game and drives 79-yards to the one-yard line. It's a textbook drive without a wasted play: runs mixed with passes, a few 10-20 yard gains, several 5-7 yard gains. Then, at the one-yard line, the running back fumbles. The Away Team scoops up the ball and runs 99 yards for a touchdown.

Next drive, Home gets the ball on the 20 again, and again drives 79 yards with little resistance. Then at the one-yard line, the left tackle gets called for unnecessary roughness for pushing and shoving after the whistle. Angry, the left tackle stomps the line judge on the toe and calls him a puppy strangler. Home gets backed up to the 31-yard line, and a 48-yard field goal attempt is blocked. Away picks up the ball and races for another touchdown and a 14-0 lead. Let's say that each team scores 10 more points, and Away wins 24-10.

A win is a win is a win, and Away clearly won. Most power rankings would rank Away ahead of Home – maybe not that week, when the two strange touchdowns are fresh in everyone's mind, but a month later, when all that's left is the 24-10 score.

But DVOA says this: Home played better. They may not have won, they may not have "deserved" to win (whatever that means), but they clearly played better. And if Home faced Away again, they would probably win, possibly by a score close to the 24-10 score they might have attained if it weren't for a few fluke plays.

But they fumbled. Bad teams fumble.

But the penalty and the blocked kick cannot be ignored.

But Home couldn't come back when they had to.

14-point comebacks aren't easy. Holding on to a 14-point lead is much easier. Maybe Home quickly cut the lead to 14-10 before halftime, then gave up one long touchdown drive in the second half and a late field goal. The defense may deserve some blame for giving up the touchdown, but it was the only long drive they allowed. The offense may deserve blame for getting shut out in the second half, but they were fighting an uphill battle. Great teams don't execute big comebacks; they roll up leads and sit on them. That's what Home showed they could do in this game. They just didn't quite do it.Bad teams do commit more of some types of penalties, like false starts, than good teams. But some penalties, like roughness fouls, are truly random. And while a blocked kick may be a sign of a lousy kicker, blocked kicks happen less than once per season per team. And again, two penalties and a blocked kick don't usually result in a 10-to-14-point scoreboard swing. Yes, and bad teams fumble more than good teams. But most fumbles don't result in a 14-point swing like this one did. When a team fumbles on the one-yard line, all kinds of things can happen. The quarterback could fall on the ball, and Home could score on the next play. Or a defender could fall on the ball, go three-and-out, and give Home the ball again in great field position. What actually happened was the worst possible scenario for Home, and it's not the type of play that repeats itself. That's because while fumbles aren't random, fumble recoveries are, and long returns absolutely, positively are.

After this Home loss, the fans are ticked off, and the sports radio guys fire them up even more. The coach is stupid. The players can't execute. They've lost their "swagger." The Away quarterback, who was spotted a 14-0 lead and probably handed off 45 times, is now a "winner" and a "game manager." That's human nature. We retrofit our opinions to match the results, and we rationalize away luck or random factors. No sportswriter wants to say that a team got "lucky," because it sounds like whining. Of course, sometimes teams do get lucky, and luck is a much bigger determinant in wins and losses than "swagger." But as the weeks go on, memories of this Home loss fade, and even the Football Outsiders forget just how strange this particular game was.

But DVOA doesn't forget. The system examined every play and ranked it. All of those 10-15 yard gains by Home on the first two drives are still in the system. The fumble is, too, but it's on the books as a big mistake, not a huge, colossal, everything-else-is-meaningless mistake. Same with the botched field goal. DVOA knows that the ability to sustain drives carries over from week to week: we've been checking for years, so we know. DVOA knows that Home's defensive ability to hold Away to 10 points is an ability to carry over from week to week. Away's "ability" to run fumbles back 99 yards or block field goals or force opponents into penalties aren't really abilities. They are flukes that appear one week and then disappear for four months.

So when DVOA ranks the Eagles high or the Colts low, the system is just remembering all sorts of things that we've forgotten. We all remember that the Eagles lost two crazy games, but DVOA can tell you just how crazy they were. We all know that the Colts struggled against the Giants, Jets, and Titans, but DVOA has all of those struggles itemized. The system remembers that the Giants outplayed the Colts and the Eagles outplayed the Giants. It knows that the Colts needed a late touchdown to beat the Jets and the Eagles needed a late touchdown to take the lead against the Bucs, but the Bucs' once-in-a-lifetime desperation play worked while the Jets' desperation lateral play didn't. DVOA isn't some angry fan cherry-picking plays to prove his point or a columnist on a deadline straining to remember the particulars of a 14-13 game he didn't watch. DVOA is a blind system that produces sometimes-surprising results immune to the biases that affect me, you, Aaron, and every other football fan in the world.

That's what DVOA does, and that's one of the reasons why we use it, and why we stand behind rankings that sometimes puzzle even us. If you aren't convinced, we ask you to keep checking back at the site. Dig deeper into the DVOA breakdowns and some of our other stats. E-mail us a specific question, and we'll do our best to answer it. You'll discover that DVOA, DPAR, and our other tools consistently spot trends, strengths, and weaknesses that can't be seen by the naked eye.

If not completely satisfied, drop us an e-mail and say we suck. Our delete button works just fine.

Posted by: Mike Tanier on 06 Nov 2006