Derek Carr: Lies, Damned Lies and Statistics

by Scott Kacsmar

When a sportswriter incurs the wrath of a fan base with his work, he probably said something they find highly disagreeable. One thing sportswriters usually want to avoid is poking the bear for fun, because that's when the bears get really angry, and the mama bear is most protective when you target her young cub.

Last week, I poked the bear on Twitter when I said Oakland quarterback Derek Carr was overrated. I was working on the second part of our look at passing plus-minus for ESPN Insider, and I already knew Carr would cause some controversy. One of his fans already left a misguided comment in the discussion thread for our ESPN article on the top 10 quarterbacks in this metric, which included Teddy Bridgewater at No. 3. You basically cannot talk about Bridgewater without someone bringing up Carr and vice versa, for obvious 2014 draft-related reasons. Between that comment and hearing on my TV that Carr was the best young quarterback in the league, I guess I felt the need to say "stop the presses" with the late-night tweet that he was overrated, which I quickly summarized in our look at the bottom passers in adjusted completion rate. Carr's C%+ went from minus-1.3 percent (29th) as a rookie to minus-1.2 percent (31st) in 2015, and this even adjusts for things like dropped passes and throwaways.

Oakland fans came out in full force to back their guy, because that's what a fan base starved for success will do when you criticize the player who might be the best quarterback their team has drafted since Ken Stabler in 1968. Oakland is tired of the retreads, reclamation projects, and draft busts from the last three decades. Carr is the 53-touchdown-throwin', possibly-eyeliner-wearing, better-than-his-brother-ever-was, baby-faced stud. How dare you criticize, Scott. How dare you.

One usually inactive Twitter user even flooded my mentions with links to articles filled with puffery about Carr. In an attempt to cure the insomnia I had while at a sleep study, I actually read some of them and started seeing the same arguments, which I found to be highly disagreeable. In another recent article, Carr ranked 16th in Mike Sando's 2016 quarterback tier rankings at ESPN, voted on by 42 league insiders. That article did little to praise him this year, and there is also little to distinguish him from Tyrod Taylor, the 26th-ranked quarterback.

The more I looked at it, the more I started to see what was happening here. Carr's first two seasons are an excellent case study for how fans view fantasy football value compared to real football value, and the perceived divide between statistics and film study.

Fantasy vs. Real Life

Can you name an NFL player who was widely considered good, but had below-average stats and played on losing teams? Have fun with that one in the comments, because as I told Cian Fahey earlier this summer, it's almost impossible to find one. Success will almost always show up, either on the stat sheet or in the team's record. Haloti Ngata has some pretty soft numbers even for a defensive tackle, but he has anchored some of the best defenses in the NFL since 2006, and has a ring. Brandon Marshall has yet to even play for a playoff team, but he is carving out a nice Hall of Fame resume given his stats achieved with so many different teams and quarterbacks.

In the case of Carr, Oakland is clearly not winning yet, hence the 10-22 (.313) record since 2014. As for his stats, we have to continuously adjust what "good" means as the game changes. As Chase Stuart recently pointed out, the league's 2016 average touchdown-to-interception ratio is likely to eclipse Joe Montana's career average of 1.96, which was the highest in NFL history when he retired after the 1994 season. Now Montana's number would be below average! This is why we do opponent and era adjustments. Given how many future Hall of Fame quarterbacks and other accomplished players there are in the NFL right now, Carr's numbers must be compared to a higher modern standard.

Back in the day, a second-year Pro Bowl quarterback with 32 touchdown passes would have been on his way to superstar status. The first three quarterbacks to do that were Dan Marino (1984), Kurt Warner (1999), and Daunte Culpepper (2000). Marino and Warner were both league MVPs who got to the Super Bowl, while Culpepper was in the NFC Championship Game with Minnesota. The only other second-year player to even throw at least 28 touchdowns was Jeff Garcia (2000), who like Warner, got a late start to his NFL career and took advantage of a soft NFC West schedule.

Last season, Blake Bortles (35) and Carr (32) joined that list, but they had nowhere near the overall success of Marino, Warner, and Culpepper. Carr's Pro Bowl berth was one of the highlights of his resume held aloft by his supporters. Fine, but just remember he was one of 11 quarterbacks to get a Pro Bowl selection. That includes injury and Super Bowl replacements, and not the snubbed Kirk Cousins, who had a better year than most of the replacements.

The Pro Bowl does not have the same meaning it used to, but neither do the touchdown passes these days. Here's another list that Bortles and Carr headline: when you look at the lowest seasons in ESPN's QBR for quarterbacks with at least 30 touchdown passes since 2006, Bortles (46.4) and Carr (49.2) are the only two players under 50.0, which is the benchmark for average. The next lowest seasons belong to 2010 Eli Manning (56.4) and 2013 Andy Dalton (56.8).

While 32 to 35 touchdown passes used to indicate strong play at the position, Bortles and Carr showed otherwise last year, and it is reflected in their teams' losing records. Among the 74 seasons in NFL history where a quarterback had at least 32 touchdown passes, 66 of them (89.2 percent) resulted in at least a .500 record. From 1920 to 2014, there were three seasons in NFL history where a quarterback threw at least 32 touchdowns and had a losing record as a starter. Vinny Testaverde did it for the 1996 Ravens, FO's favorite 4-12 team given how out of whack they were with the defensive approach that was to come in Baltimore. Drew Brees did it twice in defenseless New Orleans (2012 and 2014).

Last season, a whopping five quarterbacks did this, including Carr (7-9), Bortles (5-11), Eli Manning (6-10), Matthew Stafford (7-9) and Brees (7-8) again. In most of these cases, a lack of rushing touchdowns helped inflate the passing total. Jacksonville only scored five rushing touchdowns, and Bortles even had two of those. Since 2014, Oakland's 11 rushing touchdowns are the second-fewest in the league, and Jacksonville's 14 are the third-fewest. (San Diego has a league-low 10 rushing scores.)

While Carr's red zone passing has actually been exceptional through two years, his offense does not score many points overall and he gets a high share of the few touchdowns they do produce. This is great for fantasy football, but in real football, you want an offense that can score a lot regardless of how the ball is actually getting into the end zone. Oakland went from 31st (1.24) in points per drive in 2014 to 20th (1.89) last year -- an improvement, but still below average.

A lack of context is why Carr's money stat -- ranking second in touchdown passes (53) through two seasons in NFL history behind only Dan Marino (68) -- bugs me. The stat is true, but how meaningful is it when Carr is only the eighth quarterback to start 32 games in his first two years, and he has the second-most pass attempts (1,172)? Carr threw one more touchdown in his first two years than Peyton Manning on 64 more attempts, and one more touchdown than Russell Wilson on 372 more attempts. Carr's touchdown percentage (4.5 percent) is identical to what Andy Dalton had through two seasons.

If it was not clear enough already, the 2014 and 2015 seasons featured the most touchdown passes in NFL history and the two highest touchdown passing rates since the 1970 merger. I'll give credit to Carr for earning a Week 1 rookie starting job and having the durability to go 32 consecutive starts, but 14 NFL teams have thrown 53 touchdown passes since 2014, with four more sitting on 51. When Manning threw 52 touchdowns in 1998-99, only six teams had more. If you throw a touchdown pass on 4.5 percent of your passes in 2014-15 like Carr, that doesn't make you good. That just means you're average.

So when people say Carr's stats are good, it really is a simplified fantasy football outlook of focusing on volume. He starts every game, he throws the ball a lot, and he does get a high percentage of his team's touchdowns. But the lack of more efficient play overall is not helping Oakland win more games, and Carr's second-half slump in 2015 would have disappointed even fantasy owners. After starting the season with four 300-yard passing games and four games with three-plus touchdown passes though Week 9, Carr hit that yardage only twice and that touchdown benchmark just once in his final eight games. While year-to-year defensive correlation makes predicting schedule strength trickly, Carr may even stumble again in 2016's second half given that the Raiders will play Denver twice, Kansas City, Carolina, and Houston after Week 8.

When you get past the touchdown total that had decreased real football value last year, it is hard to see where this idea that Carr has arrived as a franchise quarterback comes from. He is not as rough around the edges as Bortles, but they are similarly inconsistent players through two seasons.

Stats vs. Film

Just as the standards of quarterback play have risen in the NFL, the standard in using football stats for analysis has really improved. I feel like I am preaching to the choir on this part, so we will jump right to the conclusion. An analyst should use both game film and statistics. It does not have to be exclusively one or the other, and if it is, you probably are missing some valuable information and doing your readers a disservice.

The part that really irks me is the perception that these two methods are so far apart. They may be if you are bad at using them, but a lot of our data that gets presented on the site or featured in the Football Outsiders Almanac comes from carefully watching the games and rewinding plays several times to get things right. The game-charting project, with its roots going back to 2005, makes it absolutely essential to watch every play to get accurate charting data. When Cian Fahey breaks down a quarterback in Film Room, you can bet he watched every dropback multiple times. When we do our annual studies on adjusted interceptions and pass pressure, we had sets of eyes on every play. When I wrote about Amari Cooper's catch radius, I watched and charted every catch he made. Is there always a level of subjectivity to many of our charting metrics? Yes, but that is unavoidable. The fact is we are looking at the full picture and trying to offer unique stats to help with the analysis.

Too often I see articles that do a nice job of breaking down a few plays for a player, but when a quarterback has more than 700 plays in a season, what can you really learn from a few plays? You can make those plays say whatever you want if you find the right ones. Stats count every play. It's like watching a teaser compared to the full movie. I am not trying to pick on any individual writer, because I know how time-consuming it can be to chart a full season for a player. Then, if you want to make it meaningful enough to read, you need at least one other player or one other season to compare it to, and hopefully more. But the payoff can be huge when you combine stats and film, even if some want to continue distancing them from each other.

So a few hours after my Carr tweet, Matt Bowen had a piece at ESPN Insider that ranked the young quarterbacks on upside. He had Carr at No. 1, which of course had Oakland fans sending me another link. I like Bowen, an ex-player with the experience to break down film and the writing skills to simplify it for his readers to understand. Since his focus was on upside, this was more of a forward-thinking piece than a raw ranking of what's happened the last two seasons.

Here is an excerpt of what Bowen had to say about Carr:

You want to see Carr sling it? Go check out some of the throws he made against the Pittsburgh Steelers. The skinny post, the seam to split the safeties. Big time. Or turn on the tape against the Green Bay Packers and watch Carr drop an absolute dime on the deep corner route to Amari Cooper. Wow. That's legit stuff.

I think I know which three throws Bowen is talking about, but are these three any more important than the other 570 passes Carr threw in 2015? We can always go back to find great plays from Josh McCown's last three seasons, especially in 2013 with the Bears when he was the best quarterback under pressure in the last six seasons. Of course, in the last two years McCown has helped Tampa Bay and Cleveland earn very high draft picks, an unexpected value to be sure, but not surprising when you look at his full body of work. The bad plays still count.

Furthermore, Bowen ranked Teddy Bridgewater fifth, giving him the highest floor, but not much room for upside with his conservative nature. That may be fair, but I think Bowen too easily dismisses the edge Bridgewater holds over Carr (and Bortles) in QBR in 2014 and 2015. I am not saying he should have pimped the stat more because the article was on ESPN, but QBR is valuing every play with context (except, admittedly, strength of schedule). Bowen mentions Carr's struggles under pressure and against better competition (AFC West defenses), but Bridgewater should get credit for playing some of his best ball against teams like Denver and Arizona while facing the most pressure in the league last year, while Carr was one of the least pressured quarterbacks. Getting rid of the ball really seems to be Carr's most significant statistical achievement, something that likely became a goal after watching his brother David get pummeled in Houston.

In fact, here is a very interesting split from last season.

• Bridgewater (14th) ranked higher in DVOA with pressure than Carr (17th).
• Bridgewater (12th) ranked higher in DVOA without pressure than Carr (17th).
• Bridgewater (22nd) ranked lower in passing DVOA than Carr (13th).

The inclusion of scrambles in the pressure splits (but not in passing DVOA as of now) is not enough to explain a nice example of Simpson's paradox in sports stats. Jason Lisk had a good one years ago at Pro Football Reference about the rushing averages for Jim Brown and Jim Taylor. This particular split comes back to Bridgewater being pressured on a league-high 36.0 percent of his plays compared to 20.5 percent for Carr (sixth-lowest). If these quarterbacks were pressured at a similar rate, Bridgewater would likely have the better overall DVOA too.

But pressure rates and sack percentages take a backseat to several metrics we have found that annually highlight strong quarterback play and lead to success. Stats where the cream generally rises to the top: DVOA, QBR, yards per attempt, passing plus-minus. Even more unit-driven results such as yards per drive often reflect favorably on the top quarterbacks, assuming they started the full season.

We can excuse Carr's rookie year for obvious reasons, and should note the improvement last year. But in a 32-team league, Carr and the Raiders failed to make the top 16 in any of these categories. That makes it hard to believe the full picture shows a good quarterback. Just remember: the stats won't change until the film does first. The full film.

Conclusion

Rarely should you lay out the "Jump to Conclusions" mat and set your mark for a quarterback's career after his second season. Things can change in a hurry. I have found that a four-year rule works well, but two years are usually not enough time. You might have been sold by Dan Marino and Peyton Manning that early, but Terry Bradshaw and Troy Aikman needed more time to transition from disappointing No. 1 picks into future Hall of Famers. Of course, the latter pair had the advantage of playing on dynastic teams with many great players, but you would have been dead wrong to project them to be poor players moving forward after Year 2.

Too often in today's social media world, which keeps an unforgiving history of your old takes, we get forced into choosing a side. But any good analyst should know that in light of new information, you have the right to change sides. If Derek Carr is legitimately good in 2016, then the critics are going to quiet down. I will quiet down. However, that does not make his 2014-15 performance any better. That would be all in the past, when he was a different level of player.

Good players rarely need much justification, because the results tend to speak for themselves. The results have not been there yet for Carr, but on the bright side, he is still young and has time to deliver. And if you can make Oakland relevant again after 13 non-winning seasons, then you are probably good.