Writers of Pro Football Prospectus 2008

Most Recent FO Features


» Futures: Josh Rosen

UCLA's quarterback clearly has the talent to succeed as an NFL starter. The question is whether or not he can avoid enough mistakes to become a superstar.

20 Jan 2004

The Two Carolinas

by Aaron Schatz

At the start of last week's AFC Championship preview, with Peyton Manning coming off two ridiculously good games, I asked the question, "How long is momentum?"  Which was the better indicator of how the Colts would play against New England: the first two playoff games, or the entire season?

It turns out that the answer was "the entire season" and in particular "every game Peyton Manning had ever played against Bill Belichick-coached teams."

Now that we start to look at the Super Bowl matchup, we are faced with the same question.  What's the best indicator of the quality of a football team?  Their performance over the entire season, or in recent games?  And if the answer is "recent games," how many games count?

All during the regular season, I wrote that the Carolina Panthers were overrated, that they were winning games they should not have won thanks to fluke plays.  In my final team efficiency ratings based on our Defense-adjusted Value Over Average (DVOA) statistic, they were ranked #18.  (For newcomers, DVOA, which analyzes every single play during the season and compares it to a league average based on situation and opponent, is explained here.)  Based on DVOA, and taking their performance in close games and the red zone into account, their record should have been 7-9.  According to points scored and allowed, their record should have been 9-7.

So if I were to write my usual playoff preview article analyzing all of the numbers from the regular season, there wouldn't even be a question who was favored.  Even though they too do not come out as the top team in football according to my breakdowns, the Patriots' statistical performance was far superior to that of the Panthers.

Until the playoffs.

You know those funky DVOA graphs I have been running throughout the playoffs, showing the ups and downs of each team's season?  Here is the graph of the total DVOA performance of the Carolina Panthers -- offense, defense, and special teams -- on a game-by-game basis, including the three playoff games.

How good have the Carolina Panthers been during the playoffs?  According to the DVOA system, Carolina's last three games represent the best three-game stretch of football played by any team in the NFL this entire year, including both the playoffs and the regular season.  Part of that is the adjustment for opponent -- beating three playoff teams is more impressive than clobbering Arizona, San Diego, and Detroit.  Here are the top three-game stretches ranked by the average of the team's total DVOA in each of the three games:

Team Weeks Avg. DVOA Results
CAR 18-20 75.8% beat DAL 29-10, beat STL 29-23 (2OT), beat PHI 14-3 
TEN 7-10 75.0% beat CAR 37-17, beat JAC 30-17, bye, beat MIA 31-7
STL 6-8 73.9% beat ATL 36-0, beat GNB 34-24, beat PIT 33-21
KAN 7-10 73.4% beat OAK 17-10, beat BUF 38-5, bye, beat CLE 41-20
GNB 16-18 72.0% beat OAK 41-7, beat DEN 31-3, beat SEA 33-27 (OT)
KAN 8-11 69.1% beat BUF 38-5, bye, beat CLE 41-20, lost to CIN 24-19
SEA 1-3 64.6% beat NOR 27-10, beat ARI 38-0, beat STL 24-23
TAM 3-6 64.6% beat ATL 31-10, bye, lost to IND 38-35 (OT), beat WAS 35-13
IND 2-4 64.5% beat TEN 33-7, beat JAC 23-13, beat NOR 55-21
KAN 6-8 64.5% beat GNB 40-34, beat OAK 17-10, beat BUF 38-5

Want to measure the streak in terms of four games instead of five?  Toss on Carolina's final week victory over the New York Giants, and only the Rams (Weeks 4-8) and Titans (Weeks 6-10) have played a better four-game stretch.

Does Carolina's performance invalidate all the work we've done at Football Outsiders?  I don't think so.  Look closely at those first few wins, and the Panthers definitely look like a team getting all the breaks and winning despite inferior play.

In Week 1, the Panthers were outgained in net yards by Jacksonville, 331 to 242, and turned the ball over twice to Jacksonville's once.  But they got lucky when they fumbled four times and Jacksonville couldn't grab a single one.  Jacksonville showed a total inability to run out the clock -- there's that clock management Mike Smith wrote about last week.  After Carolina finally pulled ahead, Jacksonville came back, got into position to kick the winning field goal -- and Carolina blocked it.  That's a fluke play, not because it doesn't represent great play by Carolina but because it is so rare and doesn't correlate to how a team performs the rest of the season.

In Week 2, the Panthers were outgained in net yards by Tampa Bay, 397 to 258, and once again turned the ball over twice to Tampa Bay's once.  This was the game with the three blocked kicks -- two field goals and an extra point that would have won the game instead of sending it to overtime.  I know, you're saying that if Carolina blocked four kicks in two weeks, how can it be a fluke?  Well, how many more did they block the rest of the year?  Zero.

Week 4, they beat Atlanta, but it doesn't score that high on DVOA because Atlanta was one of the league's worst teams this year.  Week 5 they narrowly beat New Orleans, in a game where both teams had DVOA ratings very close to 0%, or average.  At that point, Carolina was 4-0 and riding high, and according to our stats they were one of the league's worst teams, with two mediocre performances and two games where they were outplayed and won anyway.

Looking back, that was the end of the amazing Carolina luck.  Over the next three weeks they beat Indianapolis and New Orleans, playing their two best games of the season until the playoffs, and got smoked by Tennessee for their first loss of the season.  The second time they faced Tampa, they played much better than the first time and beat them without any flukes.  Over the second half of the year their luck actually turned on them a bit.  They lost to Dallas in a game where they played better than some of their wins early in the year.  They outplayed Philadelphia and lost when John Kasay missed three field goals and an extra point -- essentially, a reverse of the first Tampa Bay game.

So coming into the playoffs, we had a team that had played a very easy schedule, that was 11-5 but probably should have been 9-7 or 8-8.  It wasn't that they were a bad team, just that they were mediocre.  What has happened in the playoffs?  They have been the best team in the NFL.

"Well," you say, "The Panthers have always been this good. The problem is that there is something wrong with the way you compute your statistics. They clearly need a lot more work."  But our newfangled statistics aren't the only numbers that show Carolina has played far better over the past five weeks than they did earlier in the season.  Those oldfangled statistics show it too.

Over their first nine wins, the Panthers won only twice by more than three points and only once by more than ten points.  The last five Carolina wins have all been by more than three points with three of them by more than ten points.

In their first playoff game, the Panthers outgained the Cowboys in net yards, 380 to 204, and grabbed two Dallas turnovers without once turning the ball over themselves.

In their second playoff game, the Panthers outgained the Rams in net yards, 485 to 380, and grabbed three St. Louis turnovers while only turning the ball over once themselves.  The game should have never even smelled overtime, but this time it was the Panthers who could not run out the clock.

In their third playoff game, the Panthers were outgained by the Eagles in net yards, 289 to 256, which isn't too bad when you can intercept the ball four times and never turn it over yourself.

Now, if you want to say that a team that gets outgained 728 yards to 500 yards in its first two games -- while turning the ball over twice as often as its opponents -- should be expected not only to reach the playoffs but to outgain its opponents 1121 yards to 873 yards once it gets there -- turning the ball over only once while its opponents turn it over nine times -- well, you're seeing something that I'm not seeing.  Not to mention you need to move to Vegas and spend a lot of time in sports books.  Looking back, I think that when the DVOA system thought the Panthers were overrated at the start of the year, it was pretty much on target.  And when the DVOA system says that the Panthers have been dominant in the playoffs, well, I think it is pretty much on target again.

So that brings us to the next question.  Exactly what has changed?  From what I can tell, the Panthers have not changed their personnel at all.  These are the same players, simply playing much better.  Russell from Football Junkie emailed me to say that perhaps they were pressing too hard when they were trying to clinch the division, but once they got over that hump they had nothing to lose.  A team that plays like it has nothing to lose is a dangerous team.

What do my numbers say?  Well, I've taken all the different splits I used in writing the playoff preview articles, and looked at how Carolina performed over the past five games compared to the first 14.  I say the past five games because to create a little more statistical significance, and the current Panther streak of quality games really begins not in the playoffs but in Week 16 with a 20-14 win over Detroit where the Panthers dominated until the Lions scored on a fumble return and a punt return in the fourth quarter.

Let's start with the defense, where the change is very easy to isolate in a single word: interceptions.  Over the first 14 weeks of the season, the Panthers had only 11 interceptions.  That was tied for 24th in the league.  Since then, the Panthers have had 13 interceptions in five games.  Yes, they had four against Jesse Palmer, essentially a rookie, and three against Marc Bulger, who led the NFL in getting picked off.  But last week, I wrote that Philadelphia had turned the ball over less than almost any other team in the NFL -- and the Panthers picked off McNabb and Detmer four times anyway.

In general, the Carolina defensive DVOA has gone from +4.1% over the first 14 games of the season to -44.9% over the past five.  (Remember, numbers above zero represent scoring points so defenses are better if they are more negative.)  Take all the interceptions out, and the Carolina defense DVOA improves much less, from +18.6% to +4.4%, and the improvement is even smaller if you compare the entire regular season to just the postseason.

Pull out the interceptions, and there are still some very clear places where Carolina has improved, just fewer of them.  For example, the Panthers have improved a little bit on first down, but dramatically on third down:

Weeks 1-15
Weeks 16-20
Avg. Yd/Play
Weeks 1-15
Avg. Yd/Play
Weeks 16-20
1st Down, not counting interceptions 20.1% 1.3% 5.2 3.8
2nd Down, not counting interceptions 13.8% 7.4% 4.9 4.6
3rd Down, not counting interceptions 44.1% -6.3% 5.3 (7.7 to go) 2.8 (7.4 to go)

I will note that this improvement on third down is mostly on third-and-long, where Carolina's defense was awful for most of the season.  The DVOA on third-and-short hasn't changed at all, but then again Carolina's defense was pretty good in those situations already.

Another impressive change comes in the red zone.  On most of the field, interceptions are the only difference between Carolina's recent streak and their defense the rest of the season.  But in the red zone, Carolina's defense has gone from -4.4% DVOA in Weeks 1-15 to -45.5% DVOA in Weeks 16-20.

How about the offense?  When you compare the past five weeks to the first 15, Carolina's offense has improved on every down and every distance except second-and-short.  Carolina's offense has improved on every zone of the field except between the 40-yard lines.

Two splits do stand out for the offense.  First, Carolina's offensive DVOA in the first half of games has gone from -23.0% to 18.1%, an incredible jump.  That's basically the equivalent of exchanging Baltimore's offense for Minnesota's.  But you might be surprised to know that Carolina's offense, in total, has not improved at all in the second half of games.

The other important split?  Despite improvement on the part of the running game over the last five weeks, despite all the attention on Stephen Davis and DeShaun Foster, Carolina still registers as league average when it comes to rushing.  Passing, however, bar the doors.  Carolina's passing offense has gone from -5.7% DVOA over the first 15 weeks to 26.4% DVOA over the past five weeks.  Delhomme!  You the homme!

So offense is better, defense is better.  What about special teams?  Actually, not better.  Most of the fade has come from Carolina's most dependable player.  Even without counting the two blocked punts against the Giants, Todd Sauerbrun's punts (and the Carolina return coverage) have gone from being worth .25 points of field position each in Weeks 1-15 to -.25 points of field position each in Weeks 16-20.  Carolina has also weakened, slightly, on kickoffs and kickoff returns.

A week from now, I'm going to sit down and write my Super Bowl preview.  And I don't know if the regular season numbers that I've used to analyze all previous games make sense anymore.  Which of the two Carolinas is coming to play next week?  The one from the playoffs?  Some combination?  At least we know it won't be the Hurricanes.

Someday, hopefully, statistical study will tell us how many games it takes for us to feel confident that a team has truly established a new level of performance.  Until then, we have to guess.  If the last three weeks are just part of the standard ups and downs of the regular season Carolina Panthers, just like the losses to Atlanta and Dallas, then we are probably headed for a New England rout.  But if Carolina's playoff performance truly represents advancement to a higher level of skill, then we have one heck of a good game on our hands.  I'm guessing it is the latter.

Posted by: Aaron Schatz on 20 Jan 2004