Writers of Pro Football Prospectus 2008

Most Recent FO Features


» The Deep Ball Project

Guest columnist John Kinsley breaks down the tape of every deep pass in the NFL in 2017 and comes away with a shocking conclusion: even without Andrew Luck, the Colts had the best long-ball quarterback in the league.

25 Jan 2010

DVOA for Conference Championships

by Aaron Schatz

I realize that I forgot to run the DVOA ratings for the championship games at the bottom of Audibles as promised, so I hereby present the shortest DVOA analysis article ever.

DVOA (with opponent adjustments)
IND 31% 55% 29% 5%
NYJ 5% 18% 10% -2%
NO 37% 3% -19% 15%
MIN -4% -5% -11% -11%
VOAf (no opponent adjustments)
IND 20% 29% 14% 5%
NYJ -11% 15% 24% -2%
NO 13% -3% -2% 15%
MIN -28% -5% 12% -11%

I expected the Colts win to come out with a much higher DVOA than the Saints win -- enough to get rid of whatever weighted DVOA difference remained between them -- so these results were a bit of a surprise. The Colts' win is higher without opponent adjustments, but even there, the difference is almost nil.

Here's a look at both games, trying to figure out why the Colts win didn't score higher and why the Saints win didn't score lower.

COLTS-JETS: It sure felt like the Colts dominated this game in the second half, right? Well, they didn't in the first half, and that keeps the numbers close for the whole game. Remember, the Jets actually had the lead going into halftime. The Colts had 7.1 yards per play to the Jets' 6.5 yards per play, and the Jets were much more efficient on third down. (Without opponent adjustments, the Colts had a negative VOA on third down, -10.8%, while the Jets had an awesome 85.8% VOA.) All in all, this game wasn't quite as impressive as it seemed at the time -- a phrase which has described a lot of Colts wins this year. Then again, all those games have been wins, haven't they?

SAINTS-VIKINGS: Yes, the Vikings had an advantage of more than 200 net yards, but they also had a lot more plays. Minnesota ended up with a smaller advantage in yards per play, 5.8 to New Orleans' 4.7. Yards per play (and DVOA) say more about a team's future performance than total yards. The difference between the Vikings' per-play performance and the Saints' per-play performance gets made up by three things:

  • The Saints were much more efficient in the red zone than on the rest of the field. They ran seven plays in the red zone and three of them were touchdowns. In the third quarter, Pierre Thomas had three straight runs in the red zone and gained 6, 5, and 9 yards against the best run defense in the NFL.
  • The Saints were much better on special teams, for two reasons: Courtney Roby's long kick return to start the second half, and a big day from rookie punter Thomas Morstead. Morstead averaged 52 gross yards and 48.5 net yards per punt.
  • The fumbles

Posted by: Aaron Schatz on 25 Jan 2010

73 comments, Last at 28 Jan 2010, 3:06pm by Dr. I Don't Know


by slim wookie (not verified) :: Mon, 01/25/2010 - 2:51pm

"The fumbles"

Just shoot me.

by TheChadHenneMeme (not verified) :: Tue, 01/26/2010 - 11:41am

Murder-Suicide, deal. I'll pull the trigger and then you..how does this work? Someone call up a Branch Davidi...oh.

by LA Bengal (not verified) :: Mon, 01/25/2010 - 2:56pm

"Then again, all those games have been wins, haven't they?"

Subjectively, I think the Colts looked very impressive on Sunday. When I checked the half-time stats, I thought they were a little deceiving. After all, the Jets really only had two big plays (an 80 yard-pass and a tricky wildcat pass play), while the Colts had been driving at will (albeit stalling and settling for FGs).

Still, watching that game, it was clear to me and my friends that the Colts were in complete control. That's what's impressive. That a team can go into the locker room down in points and outgained in yards, but still be "in control." It seems like Peyton just takes his time figuring out the D and then methodically cutting it apart. It isn't as flashy or overtly dominating as, say, the Patriots of the past. But as you pointed out, it has led to a lot of wins this season...

by Quality Control (not verified) :: Mon, 01/25/2010 - 3:18pm

What's strange with Manning is how palpable the sense of wheels turning can be. It almost makes one forget that other quarterbacks, coordinators, and head coaches are making adjustments and thus improving their teams' play in the second half as well. But it doesn't always, maybe only because the Colts have won every game in which they decided to compete.

by dk240t :: Mon, 01/25/2010 - 4:45pm

Ask any Texans fan, they know exactly what you mean.

Texans up by 17 at the half? Sounds like the Colts are the favorite.

by Dave Bernreuther :: Mon, 01/25/2010 - 8:24pm

It was definitely stressful because of the big plays, but once they held them to the FG after the turnover, I came off the ledge, and the Jets fans around me started to quiet down. When the Jets knelt to end the first half, I got more confident. At the half, I told them that the Jets needed points on the opening drive or the Colts were going to win. Once they gave the Colts the ball with a chance to get a lead, that was it. It wasn't OVER, since a turnover could've easily changed things, but suddenly it was going to take something extreme, such as that turnover, to give the Jets a chance.

by Paul R :: Tue, 01/26/2010 - 12:07am

That's been my impression of Manning & Co. lately.
The offense huddles around pictures while the defense is on the field, talking about different strategies. Have they tried this? Have they considered that?
Then, about halfway through the game, Peyton says to himself, "Oh, yeah! We forgot to score some touchdowns! Shoot! I knew I was forgetting something."
Then they go, touchdown, touchdown, touchdown, game over.

by Sandals (not verified) :: Tue, 01/26/2010 - 12:56am

This is why teams try to play keepaway from Manning. The more time he spends on the field, the more time he has to figure out the defense. Limiting possessions sounds nice, but you trade possessions anyway. The 4th quarter comebacks are your evidence - Manning doesn't lead comebacks because he magically improves under pressure, he leads the comebacks because he's figured out the defense by the 4th quarter.

I'm also starting to think Larry Coyer is doing the same thing on the other side of the ball - the Colts D has been much more effective at the ends of games than the beginnings.

by tuluse :: Tue, 01/26/2010 - 4:54am

Is this the essence of clutch? The ability to figure out what the opposition is doing to you, and make corrections.

Maybe this is why Elway had all those 4th quarter comebacks. People like to blame it on Reeves's conservative play calling, but maybe Elway was using that time to figure out what the defense was doing.

You can see this on defense too, where a defender sees something the offense is doing. Maybe a signal leads to a certain route, and he just pockets it away waiting for the right opportunity. Then late in the game, bam, jump the route and save the day.

Maybe clutch is the ability to file all that information away, and turn it into play on the field.

by the K (not verified) :: Tue, 01/26/2010 - 4:09am

I thought exactly this myself. My buddy was at work and I called him at halftime,I told him the score and all the major stats, and I told him I'd still bet him 20 bucks the Colts would win.

by commissionerleaf :: Mon, 01/25/2010 - 3:11pm

Can't believe that all those 3-and-outs didn't cause New Orleans to drop in DVOA more. What an awful performance by Brees and the NO receivers (especially in overtime, when their mistakes should have cost NO the ball and, probably, the game). Both teams deserve demerits for their performance in the NFC.

The Colts did the Colts thing: Come out flat, sort of fool around in the first half, then score some points (however many you think you need) and play defense.

by alexbond :: Mon, 01/25/2010 - 3:51pm

Agreed, the 4th quarter was awful for the Saints offense. All those turnovers in the second half, and the Saints just go 3 and out over and over unless they already have stunning field position because of Roby's return or the Harvin fumble. I kept on thinking "this is the drive the Saints put it away" and they go 3 and out. The Saints shouldn't have needed OT to win.

by ammek :: Mon, 01/25/2010 - 4:58pm

I'm more surprised that the Vikings' defensive VOA comes out positive (ie, worse than average). Perhaps the first quarter was even more inept than I remember.

On a related note, the difference in yards-per-play might have been less than it seemed, but time of possession certainly played a part in the game. Usually you hear the announcers saying how a defense gets tired and needs to get off the field, but here it was the Vikings' offense. It ran, what, 82 plays on 13 drives; I think only three plays gained more than 20 yards; and because of all the fumbles, it was rarely ahead, even though it kept moving the chains. I was exhausted watching. Bevell, Favre & co. really showed a lot of patience and concentration to keep at it — until the last two minutes of regulation.

by lester bangs (not verified) :: Mon, 01/25/2010 - 3:12pm

I don't know, when the Colts (well, Manning) absolutely shred and undress the league's best pass defense (by far), and they do it with Garcon and Collie catching most of the passes, I think it's pretty interesting. And I say that having no stake in the Colts whatsoever.

by Aaron Schatz :: Mon, 01/25/2010 - 3:30pm

The flip side is that their defense allowed a lot of pass success to a team which had a terrible passing game during the regular season. So the opponent adjustments end up going both ways.

by Bobman :: Mon, 01/25/2010 - 3:45pm

Yeah, as humans (at least I'll assume) I think our brains tell us that one big pass play was a bit of a fluke and the other was a trick and somehow they "count less" in our minds, especially with the Colts stoning the expected run game and shutting the Jets out for the final 35 minutes or so. But those two pass plays for 132 yards and a lot of 3rd and long conversions did occur, after all....

The Colts O I did expect bigger D-adjustments for, but then I look at the 3rd down conversion stats, which were pedestrian. (Of course who needs 3rd downs when you throw three straight passes to Collie for 80 yards and a TD?)

by Bobman :: Mon, 01/25/2010 - 3:55pm

Argh, I overlooked the media hype that rested its argument on the Jets having the #1 running game, which is different from have the 11th ranked DVOA rushing game. That will make the Colts D performance less impressive to DVOA than to somebody thinking "the Jets are #1, the Jets are #1...."

by The Ninjalectual :: Mon, 01/25/2010 - 8:11pm

but the fact that they were facing 3rd and long in the first place is a sign the offense wasn't playing well...

"Just look at that pumpkin."
-John Madden, looking at the moon.

by Anonymous Jones :: Mon, 01/25/2010 - 3:15pm

Amazing and counter-intuitive results (at least based on projections prior to kickoff). Really good defense by both sides in the NFC game and really good offense by both sides in the AFC game.

by Bobman :: Mon, 01/25/2010 - 3:40pm

I expected better D-adjusted numbers for the Colts as well--maybe it was all the Dominant D hype that had me suckered in.

And on the other hand, the Colts D, while doing a nice job on the Jets run game and forcing a lot of 3rd and longs, DID allow a good amount of 3rd down conversions ('Chez really impressed me--from his TD pass to Keller last week through this whole game). And those two big plays.... One was all on the rookie CB looking into the backfield with no safety help over the top (let's see if he makes THAT mistake ever again!), and the other was a bit of a trick with the first pass out of the tiger cat all year--DVOA doesn't know about that and probably doesn't give a shit. It was a play, 11-on-11, counts the same as any other offensive snap, and the Jets got 52 yards out of it.

by Kal :: Mon, 01/25/2010 - 3:41pm

I had thought that the formula for total dvoa was 3/7ths off, 3/7th def and 1/7th ST - but in the above you simply add the three together. Is that correct? If it isn't, that right there is a big reason why the Saints are better than the Colts; a 15% ST DVOA eclipses almost everything else the saints do.

by Arkaein :: Mon, 01/25/2010 - 3:44pm

The proportions go into calculating the percentages; that's why ST DVOA numbers are always so small compared to offense and defense.

I think this is done to give the correct perceptions of the relative value of ST, and allow the easy addition to find total DVOA.

by Bobman :: Mon, 01/25/2010 - 3:51pm

That's fine and makes sense, but then for the Saints to generate a +15 for ST (when +30 is a phenomenal ooffensive day) I'd expect a couple run backs, maybe a blocked punt on D, etc.

You may be right, but then the 15 really stands out as huge.

by alexbond :: Mon, 01/25/2010 - 3:55pm

Especially considering the Bush fumble/muff.

by huston720 :: Mon, 01/25/2010 - 4:49pm

Remember that the DVOA is the percentage better than a similar special teams play, so it's not just that you get a lot of credit for returns only. The Saints got a lot of value on punts and kickoffs compared to the average in the comparable situation. Meaning that got numerous touchbacks on kickoffs which is above average, and also had an excellent net punting day compared to average. Meaning they were consistently getting a field position advantage on all types of special teams plays rather than just having an advantage on one return.

by Arkaein :: Mon, 01/25/2010 - 5:12pm

Good point, I thought that a huge "hidden point" within the game was that the Saints' kickoffs completely neutralized Harvin, which is a big deal considering how good of a return guy he is.

I'm a bit surprised that the Saints had the same ST VOA as DVOA, considering how good the Vikings ST were throughout the season.

by Eddo :: Mon, 01/25/2010 - 5:20pm

I'm not sure, but I don't think any special teams DVOA is adjusted based on your opponent's overall special teams DVOA. I think it's actually adjusted in-game, based on things like where you catch kickoffs.

by mm (not verified) :: Mon, 01/25/2010 - 10:27pm

One thing that helps out the Saints special teams was the bizarre decision by a Vikings player to return a punt from the 1 yard line.

by mediator12 :: Mon, 01/25/2010 - 4:20pm

The Jets offense was respectable because they made some plays, not just the big plays. The big plays were both blown coverages, something INDY did very little of all year. Missing Powers hurt there a lot as Lacey has made more of those errors than Powers. The Wildcat play was just a real poor read by Hayden. He guessed and lost badly.

Now, Sanchez played real well until Indy started blitzing late again. The four man Rush was slowed by an excellent OL and Protection scheme, and it was not until they played some different fronts and pressures that he was average. I will say his first third down completion to Cotchery was probably not a catch, but that certainly helped his confidence early instead of going 3 and out.

Kudos to the Jets for an excellent gameplan and execution early. Unfortunately, INDY's defense stopped hurting themselves and Peyton figured them out when they went to the 3 WR sets. Jets were just not able to matchup any longer with the same sort of reliability they had when INDY started in Twin TE foramtions.

That was a great game that shows the matchups are much more important than the stats. I love this site and all the commentary. You all do a magnificient job of making football fun to discuss on the internet again. Thanks for all you do.

by dk240t :: Mon, 01/25/2010 - 4:47pm

I thought the Colts win was impressive, and the Saints-Vikes game was 2 teams playing really badly.

Normally, I like to give a lot of credit to DVOA and FO, but on this one, I'm going to side with my gut.

Still, anything can happen at the Super Bowl...these are still 2 very good teams.

by huston720 :: Mon, 01/25/2010 - 4:57pm

Well the point is to use both your gut and the stats, especially when dealing with a single game. Remember every incompletion is the same to DVOA since it can't tell whether it was a drop by a receiver, a bad throw to a wide open receiver, or a pass defended with excellent coverage. But over the course of a season as DVOA accumulates it can start to give more context to incompletions, and the like, thanks to opponent adjustments. I.e. an incompletion to a number one wideout against the Jets is different than against another team because the Jets defend number ones so well. But over the course of a single game it is going to be hard for DVOA to tell whether one team was playing poorly because of the other team, or simply having a bad game.

by Bobman :: Mon, 01/25/2010 - 6:26pm

At the risk of invoking a not very good movie, this reminds me of Bruce Willis's repeated question in The Last Boy Scout: Head or Gut? he was talking about slugging someone, but in this case, it's both head and gut. (and a few characters in that movie deserved a duplex hit as well)

by mm (not verified) :: Mon, 01/25/2010 - 10:47pm

Well, the Saints, according to VOA, was average in offense and defense, but both numbers are boosted by opponent adjustments. They were awesome on special teams.

According to VOA, the Vikings were around average on offense, even with the turnovers. However, they were poor on defense (because of red zone performance and limited potential turnovers, I suppose) and awful on special teams.

So the only really great unit in the game (without opponent adjustments) was the Saints' special teams.

Meanwhile both the Colts and Jets had good offenses, poor defenses, and average special teams before opponent adjustments.

I would say we're more likely to remember a great offensive game as a reflection of good play and a great special teams game as luck.

by ugarte (not verified) :: Mon, 01/25/2010 - 5:18pm

I like this chart. It makes me feel much better about the Saints chances in the Super Bowl than I felt just a few minutes ago.

Do you think we can get Addai some of that hand-grease AP was using?

by Bobman :: Mon, 01/25/2010 - 6:23pm

Speaking of hand grease.... you know those super-tackified gloves? Why the hell don't they make forearm sleeves of the same stuff? I imagine it would be a HUGE boost and if the gloves are legal, why the hell not the other body parts that touch the ball.

Oh, and Addai will fumble, but you have to hit him with a wrecking ball just as Manning is sticking it in. Good gravy, that was some play by the Jets and some blown protection by the Colts OL. Crediting him (or Manning) with the fumble is like blaming somebody for getting in the way of a lightning bolt. I guess you can't quite give the OL fumble credit.... but they owe HIM a big steak dinner now.

by Dan :: Mon, 01/25/2010 - 5:39pm

I'm surprised that the Saints' ST DVOA is so good, given that Bush fumbled. Did that get counted as a muff, which is less costly because it's assumed that the receiving team usually gets it back? If so, I think that's a problem: muffs that are due to hits from the kicking team probably have recovery rates that are more similar to fumbles than to muffs where the returner drops the ball untouched. Is there any way to distinguish them in the play-by-play?

by jmaron :: Mon, 01/25/2010 - 8:31pm

The Minn-NO game is exactly the type of game that makes me question DVOA in terms of rating the quality of a teams play. I think to's and red zone performance have far more to do with luck than does the DVOA ratings.

Before all of you guys start jumping down my throat to explain why I'm wrong and how DVOA measures things so much better and how my observations are useless - why don't you ask these folks the same question:

Vince Verhei: Peterson score to tie the game at 28. You know, you take away the turnovers, and this has been a complete Vikings asskicking -- they're up 429 to 227 in yards, 28 to 12 in first downs.

Vince Verhei: SEVEN three-and-outs! Two second-half touchdown drives that totaled 44 yards! I figured the Vikings defense would be the reason they lost, but they just about dominated one of the best offenses in football.

Tom Gower: Really, though, the Vikings have themselves to blame. They couldn't hang on to the ball, turning it over thrice when they could've gotten key scores, and lost.

Vince is suggesting the Vikings defence had a dominant performance. DVOA says no. I think Vince is right.

Tom Gower seems to be suggesting the Vikings are to blame for the turnovers more than NO should be getting credit. I think that is accurate.

by NickS (not verified) :: Tue, 01/26/2010 - 12:07pm

"Tom Gower seems to be suggesting the Vikings are to blame for the turnovers more than NO should be getting credit. I think that is accurate."

How is that accurate? Only 1 of the 4 turnovers wasn't forced (the Favre/Peterson botched handoff). 2 of the 3 fumbles the Saints recovered and the 2 fumbles Peterson didn't lose were all punched out.

by Eddo :: Tue, 01/26/2010 - 12:57pm

Taking it a step farther, it sure seems that forcing fumbles (and other turnovers) is definitely a repeatable skill for a defense.

Now, it may not be as repeatable as an offense's propensity for committing turnovers, but it's there.

by NickS (not verified) :: Tue, 01/26/2010 - 1:04pm

Let's look at turnovers for the year. The Saints were #2 in the NFL in takeaways while Minnesota was #3 in the NFL in fewest giveaways. One is great at taking the ball, the other is great and holding on to it. So Minnesota was very good at NOT committing turnovers yet they commit 5.

That points to the Saints' ability to force turnovers overcame Minnesota's ability to prevent them.

by jmaron :: Tue, 01/26/2010 - 1:14pm

Certainly nothing that happened during the season regarding fumbles portended the fumbles in that game.

The Vikings fumbled 19 times during the year and NO "forced" 18. On the other hand the Vikings "forced" 35 and NO fumbled 25 times on offence.

So if anything one would have expected NO to fumble more often in that game if fumble rates are predictive.

Look, 6 fumbles by Minn in a game is simply bad luck when your established norm is about 1. Recovering 3 opponent fumbles and 2 of your own inside the 20's is flat out good fortune for NO.

by mrh :: Tue, 01/26/2010 - 4:47pm

I got interested in all those fumbles "forced" by MIN.

9 were on sacks (26%)
7 were on completed passes (20%, with an asterisk on one, which was on a "Stanford band play" at the end of the 2nd GB game)
6 were on runs (17%)
4 were on aborted snaps (11%)
4 were on kick returns, incl 1 muff (11%)
5 were on punt returns, incl 4 muffs (14%)

I have no idea how the %s compare to league averages but it's logical to assume the Vikings, who led the league in sacks also finished high in fumble-sacks, which is probably a skill. Aborted snaps are probably luck/non-predictive events although Brees had one - maybe the Williams wall is distracting. Causing fumbles on runs/receptions may be a skill or it may be luck, probably a combination. That brings up returns.

9 total on returns, the average number of kr + pr fumbles is 4.5. All the regular season muffs were at home in the Metrodome. Bush' muff on Sun was the first one vs. MIN on the road this year.

I seem to recall the dome was known as a hard place to catch fly balls in baseball, I wonder if it also hard to catch punts (the muffed kick seems to have been to an "up-man", not sure if it was a squib or a pop-up kick)? Any FO data on the muff-rate in the Metrodome vs. elsewhere, domes vs. outdoors?

by bravehoptoad :: Tue, 01/26/2010 - 5:19pm

Look, 6 fumbles by Minn in a game is simply bad luck when your established norm is about 1.

Any evidence to support that? Is a high number of fumbles in a game non-predictive? Or are you just talking?

by Kevin from Philly :: Tue, 01/26/2010 - 5:54pm

You're all wrong, Minny's fumbling is out of character yet completely predictable. Right now, there's a collection of purple and gold dolls in the French Quarter, each with a dozen pins in the hands. The one with a "4" on the back has a few more pins in the legs. Next week, they all get dressed in blue and white.

by bravehoptoad :: Tue, 01/26/2010 - 1:15pm

No, I won't fight with you. The things New Orleans did right seem flukey and hard to repeat. I guess there's an absolute answer to those questions: are turnovers and red zone performance repeatable and predictable or not? Are they more or less, compared to their effect on the game, than other indicators?

My gut tells me they're flukey, but then one reason I like this site is that it contradicts my gut so much.

by jmaron :: Mon, 01/25/2010 - 8:41pm

One last thought about the NO-Minn and DVOA.

According to Advanced Football Stats when the Vikings had a 1st and 10 with a minute or so to go at the NO 33 - the Vikings win probability was 88%.

At that point the Vikings had an 88% chance of winning a game they were negative 3 in turnovers. But VOA says they were the inferior team. That just makes no sense what so ever to me.

by Arkaein :: Tue, 01/26/2010 - 1:36pm

Win Probabilities are based almost solely on game situation (score, possession, down and distance) compared to similar historical context.

VOA is based in specific performance on a play-by-play basis up to that point, but ignores the current game context.

So roughly speaking, VOA says that the Vikings didn't play all that well in the game compared to their opponent (and is a good indicator that they would lose an overtime game), but Win Probability says regardless of how they got there, they had the advantage at that point.

Another way to look at it: if the Vikings had lost instead of recovered one more fumble, yet played just as well afterward, they probably would have been trailing and had a much lower Win Probability, but the same VOA.

by Imbroglio21 :: Mon, 01/25/2010 - 10:41pm

The guys on FOX last night mentionned how Adrian Peterson was good during the first half of the season and then how his production came down from November 22 and on. Is it possible to have Peterson's DVOA covering the first 9 games of the season and then the one for the final 9 games? Thanks in advance,

Chicoutimi QC Canada

by Paul R :: Tue, 01/26/2010 - 12:03am

To me, the VOAf chart more accurately reflects my impressions of the games I watched rather than DVOA.
Perhaps that's as it should be. The potseason is a season of its own.

by Eddo :: Tue, 01/26/2010 - 12:18am

That's exactly how it should be; VOA/VOAf reflects how the teams played that day, ignoring strength of opponent. If the Colts played against a high school team, they'd look dominant, and their VOA would be extremely high.

However, once you factored in opponent strength (the D in DVOA), the Colts' DVOA wouldn't look particularly impressive, considering they should dominate a high school team.

So what VOA says is that the Colts, for example, had a good day offensively. DVOA says that that good day is even more impressive than it looked on the surface, because the Colts were facing an elite defense.

by Mathias (not verified) :: Tue, 01/26/2010 - 10:09am

There is def. some flaws in FOA system, when VOA thinks NO outplayed Vikings. I agree that fumbling the football isn't totally the random, but come on, would you seriously expect the Vikings do fumble the football more than 1-2 times in average against the Saints?

by huston720 :: Tue, 01/26/2010 - 12:21pm

What does the average number of times a team would be expected to fumble have to do with how a team plays an expected game? The fact is that in this game on this day the Vikings did fumble 5 times. Also the whole point of the game is to score points and prevent the other team from scoring points, turning the ball over affects both of these goals negatively, which is what VOA is pointing out. And it doesn't really matter who caused the fumble, VOA is simply measuring who played better or worse. Maybe it would help to think that VOA is saying the Viking played worse than the Saints rather than the Saints played better.

by C (not verified) :: Tue, 01/26/2010 - 10:52am

Win Loss Records had...
Colts #1 in the AFC
Saints #1 in the AFC

Power rankings had
Indy usually as #1 or #2
The Saints a top 5ish team

DVOA had Indy #8, and NO #6

The DVOA's #8 team is favored by now 6 points to beat the #6 team in the SB on a neutral field.

by bravehoptoad :: Tue, 01/26/2010 - 1:20pm

On the other hand, they nailed San Diego, who everyone else in the world had ranked in the top 5.

by C (not verified) :: Tue, 01/26/2010 - 1:37pm

Really, they nailed it? So you think the 13-3 Chargers really were the 11th best team in the 32 team NFL? If the #11 Chargers played the #10 Jets 10 times they'd win at least 6 of them or maybe SD had a bad game and their kicker had a bad game, there was a fluke INT etc.? I'd say if San Diego plays the Jets in San Diego they win win 8 out of 10 times, maybe 7 but no less...

The top 5 teams by DVOA didn't even MAKE it to the conference championships. The #3 and #4 teams were blown out in round 1...

You have the #8 team playing the #6 team in the super bowl. Win Loss records have the Saints #1 and Colts at #1 in their conferences, and power rankings would have them right up there or a little back. DVOA has both of them outside of the top 5, and both of those teams beat teams that blew out teams DVOA liked a lot more...

Minnesota was ranked #7 (4th best in the NFC), beat the #2 rated Packers twice, and beat the snot out of the #5 rated Cowboys. They were probably the best team in the NFC this year.

You could say that DVOA is descriptive and not predictive, but that's what W'L records are, and that's what traditional power rankings are anyway. Most fans & Experts trust what they SEE and focus on the RESULT and not the PROCESS.

by Eddo :: Tue, 01/26/2010 - 1:50pm

"Most fans & Experts trust what they SEE and focus on the RESULT and not the PROCESS."

Fair enough. But why do you have such a problem with this site focusing on the process? Do we really need another set of power rankings just like all the others?

by C (not verified) :: Tue, 01/26/2010 - 7:47pm

I don't have a problem with the outsiders rankings, I like them as they see the game differently. Sometimes I agree with their "strange" or unconventional calls, and sometimes I disagree... I'll tell them why I disagree too.

I've been posting here for years and I vote with my feet. With that being said, I don't take the DVOA as gospel like some long time posters and I'm still not 100% sure how it all works behind that black box of theirs.

by bravehoptoad :: Tue, 01/26/2010 - 5:30pm

Sure, there's 3700 sites focusing on what people SEE, and they tend to have a herd mentality. If that's what you want to read, I can give you plenty of links. It's weird to search out a quantitative site and complain that it's quantitative.

They have SD ranked #6 weighted, which seems about right.

by Martial (not verified) :: Tue, 01/26/2010 - 8:14pm

Humbly disagreeing. Experts worthy of being called experts focus on process, not on results. Further, process is observable and experts should be able to "see" it. In my field (humanitarian assistance), I do not tolerate bad process and will call people out for it even if they have good results. Because they won't have good results the next time they try to pull that sh*t and there are very large costs to failure.

The press, by contrast, doesn't understand what I do and lauds my field's supposed successes and complains about its supposed failures regardless of whether the process was good or bad. But humanitarian assistance isn't binary or a game. We don't "win" or "lose". A newspaper article about Haiti today won't tell you whether more lives were saved by a particular intervention or if more would have been saved by another because the only tool of measurement journalists have is counting the curent number of dead (sometimes actually called the "score") and they love something big and flashy. But I, in the middle of an ongoing crisis, can tell you what will have a better or worse overall life saving capability if we take a 12 or 24 month time frame.

Process matters a whole lot.

by C (not verified) :: Wed, 01/27/2010 - 10:15am

Agreed, and I focus ON THE PROCESS.

That's why I never liked Byron leftwhich despite mixed success
That's why I never liked Jason Campbell despite mixed success
That's why I liked Eli Manning despite mixed success
That's why I was super bullish on Aaron Rodgers and still am.

Just looking at pages of stats won't tell the story. Were balls dropped, did they play in extraordinary weather, did the defense just make super plays? Was the team up 21 points and not under normal circumstances...

Most people just look at the results, and I agree 100% to look at the process. When I was a kid playing goalie for my soccer team I gave up 5 goals one game ( which was not very good), but the coach and one of the parents told me I played an amazing game. The other team was clearly better than ours, and the ball was on our side of the field the entire game. It was basically me defending them shooting for an entire game. They told me if we didn't have such a good goal keeper we probably would have lost 10 or 12 to 0 instead of 5 to 0. Same concept with your work, and same concept with football.

by mrh :: Tue, 01/26/2010 - 11:54am

While the Vikings out-gained the Saints by a lot of yards, at least some of that yardage advantage was not meaningful.

First, the TD drives. The Vikings scored TDs on drives that gained 290 yards. The Saints scored their TDs on drives totalling 184 yards because they twice converted excellent field position into TDs, once after the Roby KO return to the MIN37 and once after recovering a fumble on the MIN7. The Vikings only scored once on a relatively short field (their last TD, starting at the MIN43) and blew their other short field chance at the NO10. But the NO offense did all it could with their two short-field chances - if the Vikings defense was really "dominating" (Vince's word) the Saints, they would have held on those possessions and left the Saints' with one chip shot FG and one long try; the extra 100+ yards the Vikings gained may not tell us anything about how much better they were playing than the Saints.

Second, MIN had four drives where they gained substantial yardage but did not come away with points immediately. One of those drives, early in the 4th period, gained 60+ yards before ending in Berrian's fumble. The Vikings defense held - thanks in part to Brees blowing a snap - and forced a punt from the NO14, resulting in the Vikings getting the ball at the MIN43. That drive, although it did not result in points, helped change the field position and set up the Vikings final TD, so it had some value. On three other drives, which gained about 150 yards, the Vikings did not even accrue any field position.
-After the Saints' 2nd TD, the Vikings started at their 20, drove in Saints' territory, and punted. The defense held NO w/o a 1st down, but the punt moved the Vikings back to the MIN24. Despite out-gaining the Saints by 20+ yards on that exchange of possessions, the Vikings only netted 4 yards of field position.
- Early in the 3rd period, the Vikings drove from the MIN10 over 50 yards before Favre threw his 1st pick (the play where he got hit below the knees). Again the Saints couldn't get a 1st down but the punt left the Vikings on the MIN17. Over 40+ yards of offense and a 15-yard penalty, a 30+ yard offensive advantage for the Vikings, and the net on the exchange was 7 yards. (Two plays later the Harvin fumble gave the Saints the ball at the MIN7).
- And on the Vikings' final drive, they gained over 40 yards before turning it over.

The Vikings offense generated almost 100 yards more than the Saints on those three possessions that was rendered essentially meaningless because of turnovers (their own poor play or the Saints' good play, take your pick) and excellent punting,which was the heart of Aaron's comments.

Roughly 200 yards of extra yardage gained that turned into no extra points or field position advantage. Turnovers and punts are part of the game too.

by whodat :: Tue, 01/26/2010 - 3:24pm

Good post, mrh.

Repeating a point I made on another thread, if the def coord decides his team can't win by optimizing (=minimizing) for yardage outside the red zone, instead choosing a different strategy, in this case making plays / beating an old man senseless, then why is it valid to judge the quality of the defensive effort by yardage allowed?

See: sending Harper / Sharper on blitzes from deep in backfield.

See: Williams going cover 2 briefly in second half, until Smith convinced him Favre had been pulped into an immobile state, leading him to revert.

See: Bill Belichick's def strategy to "let" Thurman Thomas rush for more than 100 yards in SB XXV. If you thought the Giants' def failed b/c Thomas rushed for something like 135 yards, then you were wrong.

See: Ewing's goaltending in 1982 NCAA bball championship.

See: Ali's rope-a-dope.

These are examples of jujitsu strategies, where the enemy's energy is used against him.

by Dr. I Don't Know (not verified) :: Tue, 01/26/2010 - 5:50pm

What I find remarkable and unpersuasive about the DVOA/VOA totals is not merely that the Saints are rated higher but that the disparity is so large: 40%+ in a game that most observers would have described as either a close contest or a "Vikings asskicking." Instead, DVOA seems to conclude that the game was actually closer to a Saints blowout -- the Vikings only managed to keep it close, I guess, by gaining a lot of "meaningless" yards and first downs, and holding the Saints to a bunch of "meaningless" three-and-outs.

Seriously, these findings bear no resemblance to the game I watched, which I wouldn't call an "asskicking," but in which the Vikings had a clear advantage on both sides of the ball. Is there value in DVOA so consistently offering up conclusions that contrast to one's own gut? Yes, I think so, and that's why I value this site. Looking at these numbers I'm now a little suspicious of my idea that the Vikings basically dominated the game -- it was in some ways a lot closer than traditional stats would show.

But really it's just too much to claim that a team that doubles its opponents yardage and first downs (and loses in OT on a phantom penalty) was in fact SIGNIFICANTLY OUTPLAYED by its opponents. Yet this is what the bald DVOA numbers assert, without qualification from Aaron or many other FO-ites on here.

Finally, since Aaron half-heartedly justifies these numbers on the basis that DVOA says more about future performance than total yards... what the hell are we supposed to make about 6 "forced" fumbles? Does that have any predictive value? Especially when NOT ONE of them was a no-change sack-strip play. Sure, the Saints should get credit for making several very shrewd ball-pokes, but I think you're arguing into the wind if you think SIX middle-of-the-field strips/bobbles say more about future performance than a 2-1 advantage in ball movement.

by huston720 :: Thu, 01/28/2010 - 2:10pm

Yes the disparity is a large one, but you have to look at the breakdown of the stats. If you do you can see that over half of the disparity comes from special teams where the Saints simply killed Minnesota. The special teams play is largely the reson why the conventional stat favor the Vikings os much because they consistently had a lot of yardage to gain. And if not for the fumbles and turnovers they largely would have made up much of this advantage and won. if that had been the case conventional wisdom would be that they dominated the Saints, while DVOA would probably say it was a closer game. Instead they put up a lot of yardage because of poor starting position and then fumbled away the ends of those drives.

To me the most counter-intuitive number is the Viking defensive VOA. I'm not sure why it is saying they played such a poor game with all those three and outs, and the short fields the Saints had. I'm guessing they are being significantly penalized for allowing touchdowns in the redzone rather than forcing field goals, and that when opponent adjustments are added in it reduces the penalties since the Saints are good in the redzone.

by mm (not verified) :: Wed, 01/27/2010 - 4:23pm

Even if you ignore the fumbles (and the Saints had some role in some of them), there was still a 2-0 advantage in interceptions. That's a +2 advantage in turnovers right there which makes up for a lot of yards.

And again, people ignore special teams unless there was a big turnover (like the Bush muff) or TD return. But, aside from the Bush muff, the Saints dominated special teams, which is one reason they didn't need nearly as many yards as the Vikings (turnovers being the other).

by jmaron :: Thu, 01/28/2010 - 12:41am

The Saints only had one legitimate int. The first was perhaps the key play of the game. The int occurred on a play that McCray hit Favre low. The proper call (as the NFL now admits was missed) would have given the Vikings 1st and 10 on the Saints 19 with 2:06 left in the 3rd quarter with the score tied 21-21.

Also the only reason that NO comes out with such a huge DVOA advantage on special teams is because Minnesota matched Bush's muffed punt return with one of their own. But it wasn't remotely the same. NO had zero chance to recover the Minnesota muff where Bush's error was much worse. Reynaud merely muffed the ball slightly with lots of room to recover - Bush fumbled a ball when he failed to call a fair catch that NO had very little chance of recovery.

Kickoffs and punts amounted to about 80 yards for NO - which is very good, but I think Bush's fumble inside his own 10 evens that up on special teams.

I don't see any way that the special teams play makes up for 250 yards or so of offence.

by huston720 :: Thu, 01/28/2010 - 2:09pm

You are right that the first interception should have been wiped away by a roughing the passer penalty. (Though that had nothing to do with the interception) However, that drive was kept going by a very marginal roughing the passer penalty earlier, which in my opion as someone without a rooting interest should not have been called. Essentially the call and the no-call evened out. Or if you prefer, sometimes you get a call and sometimes you don't.

As for the muffs, how would you distinguish between good and bad muffs with DVOA? I know the Minn muff didn't seem like a big deal, but anytime you put the ball on the ground anything can happen. There is no guarantee that Reynaud doesn't boot the recovery of the muff to NO.

You are right that special teams alone don't make up for the 250 yard difference, but they account for a lot of it, while the turnovers account for the rest. Like I said above special teams makes up over half of the difference with the rest most likely due to the turnovers and red zone performances.

by bravehoptoad :: Tue, 01/26/2010 - 7:55pm

Yeah, I was just about to say the same thing. 26% of that disparity comes from special teams, which normally contributes nowhere near that. If you ignore special teams it's only a 15% difference, which isn't bad considering the turnover ratio. A lot of people -- myself included -- tend to think of special teams play as lucky, so these stats bear out that the Saints victory looked pretty lucky.

by Dan :: Tue, 01/26/2010 - 7:58pm

Thomas Morstead might have been the Saints' player of the game. 12 of Minnesota's 13 possessions came off of his foot, 5 on kickoffs and 7 on punts, and their average starting field position on those 12 possessions was their own 22.5 yard line. His 5 kickoffs included 3 touchbacks and left them starting, on average, at their own 21.4. His 7 punts averaged 49.1 net yards and left them starting inside their own 20 three times.

by Jeff Fogle :: Thu, 01/28/2010 - 1:13am

Was playing around with some numbers tonight at the NFL.com site. Saw something interesting that reminded me of discussions here. Figured I'd post it.

I had mentioned in a prior post something about emphasizing production when teams were within two scores of each other...something that might help focus on when the game was on the line versus when there was less of a sense of urgency about things at a particular moment. Looked over the "situational stats" on a QB's page, and saw "4th quarter within 7" or something like that. Probably too precise, and it doesn't give you a large sample size. But, the data lined up with some of the debate about teams this year.

P. Manning 135.8
Brees 108.2
Favre 107.7
Roethlisberer 98.2
Flacco 77.1
McNabb 76.1
Brady 52.4

I think many felt that Indy/NO/Minny weren't given their due in DVOA during the regular season. Very strong QB numbers in "clutch" spots may have been an influence there (whether you take the "getting a little lucky in close games" theory, or the "winners know how to produce when the game is on the line" theory). A sample size like that would be overwhelmed by the full season in DVOA, but would loom large in the minds of people watching the games.

The bottom three listed are from teams that many believed were overrated by DVOA this year. Baltimore #1? With a second year QB who still hasn't learned how to produce on command when it matters? (note: Flacco was at 75.8 for the year on the road, while the top four on the above list were all at 96.2 or better on the road). McNabb is often derided for not being trustworthy when a game is on the line. Brady usually is money, but was dealing with injuries and other issues that had many Patriots fans thinking the mojo was gone sometime in the middle of the season.

Anyway...kind of the heart of the debate...should performance when close games are on the line be considered as something heavily influenced by luck (hence, Balt, Philly/NE would be better in the future and justify their DVOA rankings). Or, is there an ability (or inability) to perform when it matters most that means clutch situations should be isolated and carry more weight.

Can see either side...though I remain skeptical that a team with a second-year QB who isn't producing on the road or late in close games (which was largely road spots this year) would grade out as the best in the league. They'd REALLY have to be great at everything else to compensate for that relative weakness amongst elite QB's.

Noticed how the data lined up with many discussions from this season. Figured I'd post them...

by Jerry :: Thu, 01/28/2010 - 3:51am

The classic response is "Play well enough early, and you can watch your backup hand off in the fourth quarter." The missing datum here is how often each QB was in close games - if Brady was bad in just a couple, his low rating would be less important than it is for someone whose team was regularly battling down the stretch.

by Jeff Fogle :: Thu, 01/28/2010 - 1:48pm

A lot of missing data really....strength of schedule/home-road/weather. And, ALL of them represent small sample sizes relative to the full season even if you even them all up. Roethlisberger had the most passes with 103...out of his 506 for the full season. Not suggesting by any means it's a stat for rating the guys in terms of true quality at the expense of everything else. Just pointing out how those differences line up with some of the debates in the DVOA threads this year. Manning was seen as money in the bank in a close game during those weeks his team didn't win big. The guys at the bottom of the list weren't thought of in nearly the same light THIS year (and in the past for McNabb or your typical second-year QB whoever he is).

So...people/readers/viewers who place a lot of weight on what happens when their emotions are highest in a close game would be out of synch with an overall rating system that only has that particular element at about a fifth or sixth of the full rating.

Went back and looked up what the numbers were...but I'm not going to do that every time somebody says "What about...." Not hard to find nfl.com or other sites...answer your own questions...then post the data for everyone.

Pass Attempts 4th quarter within 7 points:
Roethlisberger 103
Brees 89
Favre 77
Flacco 74
Brady 69
Manning 68
McNabb 48

All small sample sizes...with 74 being the median, and four of the seven guys being within six passes of the median. I do think the stat reflects well on Big Ben with over 100 passes and the 98.2 rating. Brady and Manning sure showed extremes given the almost identical number of throws.

Decided to look up past years for them to see the magnitude of the outliers. Both were in the extreme, though Manning does have a better run over his last 7 years.

4th Quarter Within 7 passer ratings (Brady missed '08 in this stat)
Manning 2009: 135.8 (68 passes)
Brady 2007: 112.0 (84 passes)
Manning 2005: 103.5 (44 passes)
Manning 2007: 98.8 (67 passes)
Manning 2004: 98.8 (103 passes)
Manning 2002: 95.2 (166 passes)
Manning 2008: 94.2 (79 passes)
Brady 2002: 89.1 (124 passes)
Manning 2007: 87.1 (67 passes)
Brady 2005: 86.9 (89 passes)
Brady 2003: 86.3 (118 passes)
Brady 2004: 82.8 (51 passes)
Brady 2006: 77.9 (145 passes)
Brady 2009: 52.4 (69 passes)

Median over his last 7 years in this stat:
Manning 98.6 (79 attempts)
Brady 86.3 (89 attempts)

7 years allows us to pick a median easily, and adds up to well over 600 passes for each guy. That's more than a full "regular season" in this high stress category (Brady only topped 600 attempts once in a full season, Manning's never done it) I've got the totals at 680 attempts for Brady, and 628 for Manning. That volume helps allay some of the missing data concerns. There could still be a "style" impact with Manning maybe emphasizing shorter passes while Brady emphasizes more stuff downfield or something.

Manning's 135.8 this year was off the charts great for him, but he's still great normally. Brady's 52.4 was off the charts horrible for him (consistent with the thinking that he was playing hurt or something else was wrong behind the scenes), though he's normally not the same rating-wise as Manning in this stat.

Anyway, a potential topic for discussion amongst the authors this summer. Should "high stress" situations count more in the ratings? Do quarterbacks show meaningful skew in terms of how they perform in high stress situations in such a way that it trumps other issues (ie, a Joe Flacco led team shouldn't rank ahead of a Peyton Manning team without some EXTREME extenuating circumstances because of how they'll perform when all the marbles are at stake---a playoff game has the intensity for 60 minutes that a "4th quarter within 7" does in the regular season in other words...and the goal of each team is to win a championship)?

by Dr. I Don't Know (not verified) :: Thu, 01/28/2010 - 3:06pm

Interesting analysis, Jeff. Score one more point for Peyton in the Rational Manning-Brady Argument Page.

And yes, DVOA's assertion that Baltimore, Philly and New England were the two best teams in football was and is ridiculous. To be sure, mainstream analysts probably underrated those teams, but I'd say a safe formula for assessing real team/player/etc strength is Conventional Wisdom (*.75) + DVOA (*.25). And by Conventional Wisdom I mean something like Vegas oddsmakers, not Trent Dilfer or Howie Long or whatever, or there would have to be a negative coefficient in there.