Opposition Matters

by Bill Barnwell

These are heady times at Football Outsiders. During this year's playoffs, we introduced a free preview of the premium stat database [1] that will be out in full force sometime late this summer. One of the reasons that the split database is pretty exciting is that it will have 11 years of data, increasing our data points for DVOA and basis for analyses by more than 20 percent compared to a year ago. We're welcoming friends of varying levels of depravity: the Houston Oilers, Sean Salisbury, Bernie Kosar, Willie "Big Play" Clay... All fun, new additions to the FO family.

(Yes, 1996 is done. We'll have the numbers and commentary up on the site sometime this summer, once the book is finished. For now, I'll just say that Green Bay dominates the league and these numbers will finally kill the myth that DVOA doesn't like Barry Sanders. -- Aaron)

The other advantage with having 11 years of data to analyze is that splitting that data up and then looking at the cumulative results provides more meaningful numbers. With quarterback numbers, for example, two more seasons' worth of games provides some muscle to some otherwise-flimsy splits.

In the new DVOA era (1996-2006), there have been 5,807 instances of a quarterback throwing 10 passes in a game -- this definition is used as opposed to starts because there are times where a starter throws three passes and gets knocked out of a game, or throws eight passes because he's Kyle Orton and can't be trusted with a remote control and a bottle opener, let alone an offense. There have been 5,536 starts by quarterbacks over that time frame, which would leave around 271 times, give or take a few, where a quarterback threw ten passes and left the game in the place of a backup who did the same thing.

These quarterbacks have thrown for nearly 1.2 million yards. They've posted ten perfect games, according to the official NFL passer rating: Peyton Manning thrice, Kurt Warner twice, Ben Roethlisberger, Chad Pennington, Trent Green, Doug Flutie, and Kerry Collins. There have also been 13 devoid of any value whatsoever: stand up Kent Graham, Scott Mitchell, Trent Dilfer, Anthony Wright, Dave Brown, Jeff Garcia, Eli Manning, Tony Graziani, Joey Harrington, Randy Fasani, Ryan Leaf, Rex Grossman (Week 17 against the Packers), and Tim Hasselbeck. Let's look at them judged against two variables, to see how a quarterback's opposition affects his play.

Opposition Wins

How do quarterbacks perform when they play against excellent teams? On one hand, it would follow that they throw more because their team is more likely to be behind; on another, that their team would want to run the ball to keep the other team's offense off of the field. If you're Goro, you can take on two more hypotheses, neither of which would be particularly supported by FO: Since the opposing team is obviously a winner, they must be causing turnovers, which would seemingly indicate more interceptions; on the other hand, quarterbacks would be gunning up to play a team they know to be good, and you've gotta knock down the champ, more bombs get thrown, leading to higher yardage totals and more touchdowns. Only the best ex-Redskins quarterbacks can get away with using all of these in the same game, let alone the same drive, as he does. Oh well; one person's deft is another's daft, right?

Let's take a look at the numbers and see if any of those are true.

The first column (games played) is the count of all performances playing against teams with that range of wins, while the other ones are all averages.

What the data shows is a steady degradation in every category. Completion percentage, yards, yards per attempt, touchdowns, and QB rating all go down, while interceptions go up. Is it enough to make a Voros McCracken-style hypothesis that a quarterback has little to no effect on his passing performance? Definitely not. The correlation coefficient [2] between quarterback rating and the opposing team's wins on the season is -.18, not particularly strong. It does provide some evidence, though, that a quarterback's performance in a given game may have a lot more to do with the defense he's playing than anything about him in particular.

Opposition Pass Defense DVOA

Of course, team wins also aren't a perfect measure of how good a team is at affecting the play of the opposition quarterback. To focus on that, let's look at how quarterbacks fare against different sets of pass defenses.

Note that some of the averages might be different due to rounding.

Interesting -- the same trend that we saw in the wins grouping rears its head again, but even more virulently. Against an excellent defense, as opposed to one below replacement-level, quarterbacks lose a full yard per attempt, half a touchdown turns into an interception, 40 yards disappear, and five percent fewer passes get completed. Those are some pretty dramatic effects. QB Rating shows a .28 correlation with pass defense DVOA, which is actually a negative correlation (since a negative DVOA represents a better pass defense, while a high QB rating represents a better passing offense), but not as strong as it might seem looking from that chart.

Let's think about fantasy football for a minute, too. Should you be picking your quarterback solely based upon the defense he's playing? On average, playing a quarterback against a replacement-level pass defense versus an upper-echelon pass defense would earn you five fantasy points. Of course, you can use the Football Outsiders' breakdowns of performance against receiver types and common sense to figure out whether a certain matchup is better or worse than the average, but five free points are nice. The correlation between fantasy points and defensive DVOA is -.25, so it's not strong enough to be a quick-and-dirty rule, but it's something to keep in mind those weeks that you feel uncomfortable starting Aaron Rodgers in Denver next season.

The research raises a pretty interesting, but difficult question: how much credit does a quarterback deserve for a good performance? Alternately, what about the defense? How much credit should they get? Is the question unanswerable?

Fun With Splits

You'll see some of these splits show up in the Pro Football Prospectus 2007 player comments, but here are a few tidbits to tide you over:

• Brett Favre, he of the maximum 176 starts, is a great example of how performance changes by opponent. In his 23 games against teams with a replacement-level or worse pass defense, his QB rating is 98.8; he has a 3:1 TD to Interception ratio and completes 64% of his passes. Against the top pass defenses? 35 games, 71.8 QB rating, a 1:1 TD to Interception ratio, and he's completing 56% of his passes. He loses nearly 27 points in QB rating in the process.
• The regulars who lose the most under the same switch? Drew Bledsoe and Peyton Manning, who lose 35.61 and 35.09 points of quarterback rating. It's amazing that some Boston writer didn't do this research first to point out that Peyton can't win in the playoffs because he doesn't show up against the great teams and beats up on the crappy ones, A-Rod style. Oh well. Too late now! Bledsoe has a particularly ugly 58.6 QB rating in 34 games.
• Poor Shane Matthews. He got 25 games in this eleven-year period and 11 of them were against pass defenses who were better than -13.3%. Five of the aforementioned Graziani's nine career performances came against the same type of defense; there's a reason he's in the Arena League now.
• Finally, want a consistent quarterback you can rely on? The guys with the lowest variances across the different splits were Trent Green, Jon Kitna, Gus Frerotte, and Jake Delhomme. No fun, you say? The guys with the wildest split differentials were Bledsoe, Drew Brees, Dan Marino, Ben Roethlisberger, and -- sigh -- Eli Manning. Give me a guy banging his head into the wall any day.