Bill Connelly takes a look at the early-season S&P+ outliers and whether his eyeballs agree or disagree with the stats. Spoiler: Alabama is underrated. Alabama! Underrated!
16 Nov 2007
by Sean McCormick
(Note: Too Deep Zone is off this week.)
This Sunday in Philadelphia, John Beck will get his first NFL career start for the Miami Dolphins. That makes the Dolphins the third team in the AFC East to sit a healthy starting quarterback in favor of a younger backup this year. Chad Pennington, who once came off the bench to take the league by storm, has now returned to the bench so that Kellen Clemens can take his turn at trying to make Jets fans forget about Joe Namath. Trent Edwards played well enough in relief of J.P. Losman to convince the Bills' coaching staff to name him the starter for the season, before promptly getting hurt himself. And of course, sitting high above them all are the New England Patriots, who set the gold standard for quarterback switches when they rummaged around under the bench back in 2001 and pulled out a future Hall of Fame quarterback in Tom Brady.
There are generally two arguments put forward when a quarterback switch is made, a short-term argument and a long-term argument. The short-term view is that the team isn't getting enough production from the position and that plugging in a new guy might generate some offense and spark the team. The long-term angle is that even if the new guy doesn't play well, he is gaining valuable experience that will translate into wins the following season. There is some evidence that tentatively supports those views. In an article for the New York Times, Greg Bishop noted that since 2003, there have been 11 instances where a starting quarterback was replaced for performance reasons rather than due to injury, and that two teams, Arizona and Cleveland, chose to make a switch twice. Before the switch, those teams had a .395 winning percentage; after the switch, their winning percentage went up to .437. In the following season, those teams improved dramatically, posting an average winning percentage of .581.
But wins are not necessarily a good indicator of performance, as can be seen with two of the quarterbacks Bishop singled out as responsible for their team's turnarounds, Jason Campbell and Vince Young. Yes, both Washington and Tennessee are better, but a quick look at their DVOA shows that each team's resurgence has been built on defensive improvement. Campbell and Young have been the beneficiaries of their team's improvement, not the cause. Indeed, the struggles of both Campbell and Young in their second season as starters should raise questions about the validity of the assumption that letting a quarterback take his lumps by plugging him in midway through a season will make him a better player the following year.
At Football Outsiders, we use two tools to isolate how much responsibility any individual player has for his team's performance, DPAR and DVOA. DPAR is a cumulative statistic, showing how many points the player is generating beyond what a replacement-level player would be expected to generate in the same position, while DVOA measures how effective a player is on each play. By using these tools to examine 17 cases between 2002 and 2006 where a coach either benched his starter for non-injury reasons or left the backup in after the starter healed enough to play, we can get a strong idea as to whether or not teams that go to the bench end up getting better quarterback play, and whether the development of the new quarterback really leads to improvement in the following season.
While each situation is different, there are some clear trends. The first is that in most of the cases, the starter really is playing badly enough to force the coaching staff to consider making a change. In 13 of the 17 cases, the starter posted a negative DVOA for the season, and in nine of them the starter's DPAR was near or below replacement level. Although there are exceptions (most of them involving Mark Brunell, for some reason), the starters are giving the coaching staff no choice but to consider alternatives. When the starter is posting a positive DVOA but is pulled anyway because the team as a whole is struggling, the replacement quarterback invariably performs worse.
Unless your name is Joe Gibbs, you don't go sitting younger quarterbacks so older quarterbacks can play (and even Gibbs eventually followed the trend to go young when he elevated Jason Campbell in the middle of last season). In a majority of the situations, the quarterback who comes off the bench is someone who the organization has made a significant investment in, often in the form of a first- or second-round pick, and is a player who the fans and local media identify as the future of the franchise. Having that kind of player around clearly creates pressure to play him as soon as the starter begins to struggle. Those players may have talent, but they haven't earned the starting job, they are being given it, and often before they are ready. In the case of top-five selections like Joey Harrington, Eli Manning and Alex Smith, they are often being thrust into starting roles on teams with poor personnel or dysfunctional coaching staffs. Unsurprisingly, they tend to struggle. In contrast, players like David Garrard and Tony Romo, who were never looked at as future stars when they were drafted and who had to earn their shot through strong performances in practice and the preseason, have generally performed better when thrust into the starting role.
We can break down the variation in passing performance provided by a quarterback switch into a few simple categories to get a feel for how often the move is successful. If we consider a net change in DVOA of 20.0% or more to be a major improvement or decline, and a net change smaller than that to be a minor improvement or decline, then the results break down like this:
|QB Replacements, 2002 to 2006|
|Year||Team||First QB||DPAR||DVOA||Replacement QB||DPAR||DVOA||Net Improvement|
|2006||DAL||D. Bledsoe||â€“7.3||â€“22.9%||T. Romo||51.3||20.1%||43.0%||2006||CAR||K.Collins||â€“8.4||â€“34.7%||V.Young||7.7||â€“8.1%||26.6%|
|Year||Team||First QB||DPAR||DVOA||Replacement QB||DPAR||DVOA||Net Improvement|
It's a fairly even distribution of results, but in this case, an even distribution does not make a strong argument for the effectiveness of making a quarterback switch in the middle of a season. Remember, teams that are getting good or even above-average play almost never make a change, and it is notable that only two teams that went to the bullpen transformed themselves from poor passing teams into good ones: the Jets in 2002 when they replaced Testaverde with Pennington, and Dallas in 2006 when they subbed in Tony Romo for Drew Bledsoe. Otherwise, about the best a team could do was take a terrible passing attack and transform it into an average one, as Tennessee did last year when they opted to play Vince Young full-time.
If you have a good defense and a good running game, getting average play from your quarterback can make a real difference, but on a team that is struggling in multiple aspects of the game, that kind of marginal improvement is unlikely to translate into many more wins. And roughly half the teams got worse as a result of a change, some of them significantly so. Nine teams ended up with both starters posting a negative DVOA for the season, which, while not flattering for the quarterbacks involved, does suggest that their performances were as much the result of poor play by the rest of the offense as the cause of it.
But if changing quarterbacks is unlikely to immediately transform a team's passing game, what about the argument that it aids in the development of the younger player so that he is more prepared to be effective in the following season? It makes sense intuitively, but we can test it by comparing each quarterback's numbers to those they generate in the following season:
|QB Replacements, The Year After|
As it turns out, the quarterbacks who performed worst tended to show the biggest improvements, which suggests strongly that they were thrown out on the field before they were ready. Even with tremendous improvement in his sophomore season, Eli Manning was no better than an average quarterback, while Alex Smith went from being catastrophically bad to merely below-average. It seems likely that these two would have done no worse had they sat out their rookie seasons.
Meanwhile, many of the quarterbacks who were effective regressed the following season, although it should be noted that both Chad Pennington and Chris Simms had their seasons profoundly impacted by injuries. The only quarterback who went from a below-average player to solidly above-average the following season is Jay Cutler; otherwise, players stayed largely the same or improved to a point of competence, not excellence.
The two players who stick out the most are again Tony Romo and David Garrard. Romo was the only quarterback to be able to build on an already strong first season, while Garrard was the only player able to make the jump from average play to elite play. Not only did Romo and Garrard have to fight to get onto the playing field in the first place, they had to show enough to keep their teams from drafting their replacements. But on the whole, the evidence showing that quarterbacks are better off being thrown into the fire midseason rather than sitting and getting a full training camp worth of reps the following year is inconclusive at best.
So what about this year's quarterback switches? We don't know how any of the new players will play next year as starters, or even if they will be starters, but we can see how the changes have effected their teams in 2007:
|QB Replacements, 2007|
|Team||First QB||DPAR||DVOA||Replacement QB||DPAR||DVOA||Net Improvement|
Derek Anderson is poised to join Chad Pennington and Tony Romo as the third backup in the last six years to take a terrible passing offense and transform it into a great one. Cleveland spent first-round picks on Joe Thomas, Braylon Edwards and Kellen Winslow, but they weren't getting a return on those investments because of Charlie Frye's limitations. With the strong-armed Anderson, all those pieces are coming together to create a dynamic offense. But Cleveland also spent two first-round picks to acquire quarterback Brady Quinn. Will Anderson even have a chance to build on his successful season, or will the economics of football force Brady Quinn into the lineup?
The other five cases are more in line with what we would expect from a quarterback change, as teams waffle between bad options or throw in the future of the franchise to placate fans angry over a lost season. Daunte Culpepper was not good when he subbed for an injured Josh McCown, but he wasn't the worst quarterback in the league, either, which counts as progress in Oakland. Chicago had no choice but to sit Rex Grossman down, and while Brian Griese has been a bit better, he hasn't been nearly good enough to compensate for an aging offensive line and a rapidly regressing defense, or even to keep Grossman safely on the bench. Buffalo looks set to flip-flop between Losman and Edwards throughout the season; the two quarterbacks provide a radical contrast in styles, but early returns suggest that the results they generate will be roughly equivalent.
Quarterback play is not the primary culprit for Jets' 1-8 start, but Chad Pennington was getting worse as the season went along, and the quarterback of the future was sitting there on the bench, in clear view of the fan base and the New York media. Clemens looked promising in his second start, a 23-20 loss to Washington, but his success merely highlighted the fact that the team's biggest trouble spots are on the defensive side of the ball. Clemens is the one player most likely to be starting for his team next year, and strong play from him will undoubtedly encourage optimism from the traditionally morose Jets fans, but strong starters have generally regressed the following season.
The most curious quarterback switch of the year is taking place in Atlanta, where Joey Harrington, who has played reasonably well, is going to be benched for Byron Leftwich, who was signed midway through the season and who was absolutely terrible in his brief time on the field. It's possible that Bobby Petrino simply doesn't know what he's doing, and it's also possible that he knows exactly what he's doing and doesn't want to have Harrington be just good enough to cost the team a shot at Brian Brohm.
For all the flurry of quarterback switches that have been made, it's possible that the biggest story of the season is the one that has not been made. The Tennessee Titans currently have the best defense in the league, and the twelfth-best rushing attack to go along with it. They also have a quarterback in Vince Young who is struggling mightily to move the ball through the air. Kerry Collins has posted a 23.3% DVOA in the games he has played in, while Young is sputtering along with a â€“24.0% DVOA. In short, this is one of the rare situations where a quarterback switch promises to have a major impact. The Titans have a substantial investment in Young, who is already the face of the franchise and figures to be for many years to come. But if Jeff Fisher is serious about giving this year's team the best chance to compete against the likes of New England, Indianapolis and Pittsburgh come playoff time, he needs to seriously consider going to the bullpen. It's one of the few times when the move might pay real dividends.
(Ed. Note: Just to add an idea here... I think Tennessee is a team where the two-QB platoon they tried in Arizona earlier this year might actually work. Collins is a much better downfield passer than Young, while Young's running ability makes him more dangerous the closer the Titans get to the goal line. What if the Titans played Collins on any play between their own goal line and the other 30- or 40-yard line, and then Young once the drive hit the last 30-40 yards of the field? -- Aaron)
41 comments, Last at 19 Nov 2007, 2:11pm by johonny