Herbert, AFC's Young QBs: Better Than You Think

June 01, 2022, 9:54 am ET

Los Angeles Chargers QB Justin Herbert

Photo: USA Today Sports Images

NFL Offseason - The NFL's American Football Conference is loaded with talented young quarterbacks, from the Chargers' Justin Herbert to Baltimore's Lamar Jackson, New England's Mac Jones, Cincinnati's Joe Burrow, and others. But according to our numbers, each of those young quarterbacks is better than you think.

Today we are going to discuss adjusted interceptions. Unlike the NFL's raw interception totals, these numbers account for plays when a defender drops a pass that he should have caught, or when a wide receiver makes a big play to turn what should have been a turnover into an incompletion instead. On the other hand, sometimes quarterbacks are charged with interceptions that aren't really their fault—passes that bounce off a receiver's hands and straight to a defender—or interceptions that don't matter, like Hail Mary passes.

How it Works

After each season, we go back and account for these discrepancies and tabulate each quarterback's adjusted interceptions. Here's the process:

We start with each player's actual interception total. Matthew Stafford of the Rams and Jacksonville's Trevor Lawrence tied for the league lead in this category with 17 apiece in 2021. (Yes, one of these teams won the Super Bowl and the other had the worst record in the league. Life is unfair.)
We then add plays where the quarterback threw a ball that could have or should have been intercepted but was not, either because the defender outright dropped the ball (which we have been tracking in game charting since 2007), or he had it knocked out of his hands by an offensive receiver (a "defensed interception," which we have been tracking since 2012). These are listed as "Drop/Def INT" in the table at the end of this page. Ben Roethlisberger led the league in this category for the second year in a row. We'll get back to him.
Next, we subtract those interceptions that were tipped by receivers into the hands of defenders. In 2017, we dubbed these Matt Ryan specials because the Falcons quarterback threw five of them. Further research revealed that Eli Manning also threw five in 2010, and now Patrick Mahomes has tied that mark with five last season. We also subtract passes that are tipped by receivers but then dropped by defenders to make sure they are not double-counted. There were only eight of these in 2021; no player threw more than one.
We subtract Hail Mary interceptions, as well as interceptions thrown in desperation on fourth down in the final two minutes of a game. We're flexible on these definitions, but there weren't many "close-call" examples to debate last year. There were 12 interceptions on Hail Mary attempts last year; Mac Jones, Taylor Heinicke, and Teddy Bridgewater had two apiece.
We subtract dropped interceptions that occur in Hail Mary situations, since those plays wouldn't count as adjusted interceptions even if they had been caught—or, we would, but there weren't any in 2021.

When we started running these numbers, we had to get the data from our own in-house volunteer game charters. For the last several seasons, we have had access to data from Sports Info Solutions. Determining whether or not a defender should be charged with a dropped interception will always be subjective on some plays, but you can rest assured that all the obvious calls have been counted here.

Washington's Taylor Heinicke and Pittsburgh's Ben Roethlisberger were tied for the NFL lead with 25 adjusted interceptions in 2021. Neither will be starting this fall after Roethlisberger's retirement and Washington's acquisition of Carson Wentz. Among those who will be starting, Josh Allen, Matthew Stafford, and Ryan Tannehill were tied with 23 each, followed by Trevor Lawrence with 22 and then Matt Ryan and Jimmy Garoppolo with 19 apiece.

Heinicke's adjusted interception rate of 5.1% was highest among qualified passers in 2021. Tom Brady eked out a very narrow win over Aaron Rodgers for lowest adjusted interception rate, 1.67% to 1.70%.

Justin Herbert: The NFL's Most Unlucky Quarterback

It's always interesting to see who threw the most adjusted interceptions, or who had the highest or lowest rates, but the difference between adjusted interceptions and actual interceptions is usually more informative, because it tells us which quarterbacks suffered the worst luck or enjoyed the best. Historically, NFL quarterbacks each season have thrown about 30% more adjusted interceptions than actual interceptions. In 2021 that rate climbed to about 40%. This looks like random noise—there just happened to be a lot of tip-drill picks and interceptions on Hail Marys last season. For consistency's sake, we will use the more typical 30% rate to calculate how many interceptions a given quarterback "should" have thrown based on his total of adjusted interceptions.

Take Justin Herbert, or example. Calculating adjusted interceptions for the Chargers quarterback is a pretty straightforward process: take the 15 interceptions he actually threw, add two more that were dropped by defenders, then take those two back because two of his picks were accurate throws that were tipped by receivers, leaving us with 15 adjusted interceptions. Based on that 30% rate, we would expect a quarterback with 15 adjusted interceptions to throw 11.5 actual interceptions. Herbert threw 3.5 more interceptions than expected, making him the most unlucky quarterback of 2021. (Mike Tanier talked about this a bit in Walkthrough last week.) He didn't throw very many passes to the other team, but he had the misfortune to face a bunch of sticky-fingered defenders who caught nearly every pass they got their hands on.

The top of the leaderboard is dominated by young passers from the AFC. Lamar Jackson (+3.0) is in second place; his backup Tyler Huntley (+0.2) finished in 10th place. Mac Jones (+2.2) is in fourth, Joe Burrow (+0.9) is in seventh, and Trevor Lawrence (+0.1), Tua Tagovailoa (+0.0), and Baker Mayfield (-0.1) just missed the top 10.

Not all of last year's unlucky quarterbacks were young and from the AFC. Tom Brady (+2.8), a very old quarterback from the NFC, finished in third place between Jackson and Jones. Chicago's two quarterbacks, Justin Fields (+1.5) and Andy Dalton (+1.3), finished right next to each other in fifth and sixth place.

Ben Roethlisberger: The NFL's Luckiest Quarterback

To be honest, the extremely lucky nature of Roethlisberger's 2021 season was the biggest numerical story to fall out of the data, but it seems less relevant to point out since he has played his last game. It also seemed like a dick move to kick a guy on his way out the door. But now that we're here, let's break down his season.

Roethlisberger only threw 10 actual interceptions in 2021. Seventeen quarterbacks threw more than that, including Sam Darnold, who only played in 12 games. However, Roethlisberger threw sixteen other passes that should have been intercepted but were dropped by defenders. (A 17th pass was also dropped by a defender but only after it was first dropped by a Steelers receiver. Whole lotta great football out there in Pittsburgh, folks.) That's the most dropped interceptions a quarterback has benefitted from in a single season since 2007, breaking Mark Sanchez's record of 15 set in 2010. Roethlisberger threw 9.2 fewer interceptions than expected, the biggest gap in our records, breaking the mark of -8.5 that was also set by Sanchez in 2010.

Luckiest QBs, Single Seasons, 2007-2021
Year	Name	Team	INT	HM/ End Q4	Drop/ Def INT	Tip INT	Tip AND Drop INT	Drop INT also HM/ End Q4	Adj INT	Exp. INT	Diff.
2021	B.Roethlisberger	PIT	10	0	16	0	1	0	25	19.2	-9.2
2010	M.Sanchez	NYJ	13	0	15	0	NA	NA	28	21.5	-8.5
2009	T.Romo	DAL	9	0	12	0	NA	NA	21	16.2	-7.2
2016	B.Roethlisberger	PIT	13	0	12	0	0	NA	25	19.2	-6.2
2009	D.Garrard	JAX	10	0	11	0	NA	NA	21	16.2	-6.2
2007	B.Favre	GB	15	1	13	0	NA	NA	27	20.8	-5.8
2018	D.Carr	OAK	10	0	10	0	0	NA	20	15.4	-5.4
2020	B.Roethlisberger	PIT	10	0	10	0	0	0	20	15.4	-5.4
2019	C.Wentz	PHI	7	1	10	0	0	0	16	12.3	-5.3
2012	A.Luck	IND	18	1	14	1	0	NA	30	23.1	-5.1
2020	N.Foles	CHI	8	0	9	0	0	0	17	13.1	-5.1
2014	D.Stanton	ARI	5	0	8	0	0	NA	13	10.0	-5.0

Now here's where things get weird: you would expect a lot of fluctuation in these numbers, since a quarterback has no control over whether defenders drop his passes or not, but Roethlisberger was also the luckiest quarterback in the NFL in 2020 and 2016. That's three times in his last five qualifying seasons (he only played two games in 2019) that Roethlisberger has been the NFL's luckiest passer, and he has four other seasons in the top 10. Things haven't always been so rosy for Roethlisberger—he ranked seventh or higher in bad luck in 2008, 2013, and 2015—but as a starter since 2007, he has thrown 21.5 fewer interceptions than expected, based on his adjusted interceptions. We're just going to guess that's a lot of dropped interceptions by defenders in Cincinnati and Cleveland.

Ryan Tannehill and Other Lucky Quarterbacks

The luckiest quarterback in 2021 who will be starting again in 2022 was Tennessee's Ryan Tannehill (-3.7). Tannehill only threw 14 interceptions, but he saw 10 other passes dropped by defenders, with one pick that should have been caught by his receiver. Among returning starters, Tannehill is followed by Kirk Cousins (-3.0) and—really!—Aaron Rodgers (-2.9). Though Rodgers had one of the lowest adjusted interception rates in the league, he wasn't quite as good as his total of four interceptions would suggest—defenders dropped five of his passes.

Taylor Heinicke (-4.2) and Teddy Bridgewater (-3.0) finished second and fourth in luck in 2021. Their teams saw this and figured they would not be so fortunate a second time around, which is partly why they won't be starting in 2022.

Adjusted Interceptions, NFL QBs, 2021

2021 NFL Quarterbacks, Adjusted Interceptions
Name	Team	INT	HM/ End Q4	Drop/ Def INT	Tip INT	Tip AND Drop INT	Drop INT also HM/ End Q4	Adj INT	Pass Att (no DPI/ spikes)	INT Rate	Adj Rate	Exp. INT	Diff.
B.Roethlisberger	PIT	10	0	16	0	1	0	25	603	1.7%	4.1%	19.2	-9.2
T.Heinicke	WAS	15	2	12	0	0	0	25	491	3.1%	5.1%	19.2	-4.2
R.Tannehill	TEN	14	0	10	1	0	0	23	530	2.6%	4.3%	17.7	-3.7
J.Allen	BUF	15	0	9	0	1	0	23	646	2.3%	3.6%	17.7	-2.7
M.Stafford	LAR	17	0	9	2	1	0	23	601	2.8%	3.8%	17.7	-0.7
T.Lawrence	JAX	17	1	8	2	0	0	22	599	2.8%	3.7%	16.9	0.1
J.Garoppolo	SF	12	0	7	0	0	0	19	438	2.7%	4.3%	14.6	-2.6
M.Ryan	ATL	12	0	8	0	1	0	19	559	2.1%	3.4%	14.6	-2.6
Z.Wilson	NYJ	11	0	9	2	0	0	18	382	2.9%	4.7%	13.8	-2.8
P.Mahomes	KC	13	0	10	5	0	0	18	658	2.0%	2.7%	13.8	-0.8
D.Carr	LV	14	0	5	1	0	0	18	623	2.2%	2.9%	13.8	0.2
B.Mayfield	CLE	13	1	6	0	1	0	17	416	3.1%	4.1%	13.1	-0.1
J.Burrow	CIN	14	0	5	2	0	0	17	518	2.7%	3.3%	13.1	0.9
D.Mills	HOU	10	0	7	1	0	0	16	392	2.6%	4.1%	12.3	-2.3
D.Prescott	DAL	10	0	7	1	0	0	16	596	1.7%	2.7%	12.3	-2.3
S.Darnold	CAR	13	0	3	0	0	0	16	406	3.2%	3.9%	12.3	0.7
J.Herbert	LAC	15	0	2	2	0	0	15	671	2.2%	2.2%	11.5	3.5
J.Goff	DET	8	0	6	0	0	0	14	489	1.6%	2.9%	10.8	-2.8
K.Murray	ARI	10	0	5	1	0	0	14	481	2.1%	2.9%	10.8	-0.8
M.Jones	NE	13	2	4	1	0	0	14	520	2.5%	2.7%	10.8	2.2
T.Bridgewater	DEN	7	2	8	0	0	0	13	426	1.6%	3.1%	10.0	-3.0
K.Cousins	MIN	7	0	7	0	1	0	13	556	1.3%	2.3%	10.0	-3.0
T.Tagovailoa	MIA	10	0	4	1	0	0	13	387	2.6%	3.4%	10.0	0.0
L.Jackson	BAL	13	0	1	1	0	0	13	378	3.4%	3.4%	10.0	3.0
C.Wentz	IND	7	0	6	0	1	0	12	516	1.4%	2.3%	9.2	-2.2
J.Hurts	PHI	9	0	5	1	1	0	12	430	2.1%	2.8%	9.2	-0.2
T.Brady	TB	12	1	3	2	0	0	12	717	1.7%	1.7%	9.2	2.8
R.Wilson	SEA	6	0	6	1	0	0	11	400	1.5%	2.8%	8.5	-2.5
J.Fields	CHI	10	1	2	0	0	0	11	268	3.7%	4.1%	8.5	1.5
D.Jones	NYG	7	0	3	0	0	0	10	360	1.9%	2.8%	7.7	-0.7
A.Dalton	CHI	9	0	2	1	0	0	10	235	3.8%	4.3%	7.7	1.3
A.Rodgers	GB	4	0	5	0	0	0	9	528	0.8%	1.7%	6.9	-2.9
J.Brissett	MIA	4	0	4	1	0	0	7	224	1.8%	3.1%	5.4	-1.4
T.Huntley	BAL	4	0	1	0	0	0	5	185	2.2%	2.7%	3.8	0.2

Comments

24 comments, Last at 03 Jun 2022, 5:18pm

#1 by Aaron Brooks G… // Jun 01, 2022 - 10:48am

That looks like a misstatement. Two drops that were picked?

#8 by Vincent Verhei // Jun 01, 2022 - 12:53pm

Sorry, the latter part there should have read "tipped by receivers" instead of dropped by defenders. That's fixed now.

#2 by BigRichie // Jun 01, 2022 - 11:53am

I'm actually unimpressed that Big Ben has been consistently very lucky.

If this were luck, the year-on correlation should be zero. So what is the correlation year-on-year for QBs? Have you run it?

Bill Walsh noted that receivers don't tip balls that are placed where they're supposed to be placed. Those they catch. I'm thinking you folks are most likely luckifying some actual skills here.

Until you show me that correlation coefficient, I see very little value here.

#3 by HitchikersPie // Jun 01, 2022 - 12:20pm

If this were truly random you’d sometimes expect someone to be the league leader two years in a row. Though I would love to see a year on year N+1 correlation if there is a tendency to throw/not throw picks, it seems entirely in line with the article for this to have happened to Roethlisberger.

#4 by Aaron Brooks G… // Jun 01, 2022 - 12:31pm

There's only like a 1/1000 chance of seeing a serial leader if it's truly random.

Frankly, over 15 cycles, you wouldn't expect to find the same leader twice, although it wouldn't be outlandish if you did.

#5 by theslothook // Jun 01, 2022 - 12:34pm

Technically speaking, the process being described is called a random walk. Ie - a qb's interception rate is going to depend on his own true level + some random noise as opposed to it being a purely random noise event. Its not like the interception rate is completely random for every QB.

As such, to address the point above. If the process is a random walk, then the equation looks like this.

Yt = Big Ben Int Rate at time t

et = random noise at time t

Yt = Yt-1 + et

Now, if you run a regression of this model, you will get biased results and it will show up as correlation in time even though it really isn't. The way to test the correlation is to take the first difference of the series and see if that is purely white noise. Ie

Yt - Yt-1 = et

But even this test is probably biased. Int rates depend on time and context which if you leave out of the estimation is going to cause bias again and it will look like its not white noise when it probably is white noise once you remove those factors.

#6 by Aaron Brooks G… // Jun 01, 2022 - 12:41pm

Int rates depend on time and context which if you leave out of the estimation is going to cause bias again and it will look like its not white noise when it probably is white noise once you remove those factors.

If INT rates depend on time and context, then the noise you are seeing isn't white. It's probably not even noise.

As to the latter comment, once you remove all factors, then yes, what remains is white noise. The argument was that factors were still present after the initial removal of bias.

I'm curious whether ADoT correlates to interception rate, but there may not be enough data for that to fall out yet.

#7 by theslothook // Jun 01, 2022 - 12:44pm

I think the point being discussed was whether big Ben was unlucky or if its intrinsic. Ie, do dropped ints correlate with time.

By just looking at the true int rate and ignoring those other factors, you could attribute that correlation to dropped ints vs those other factors. Basically the omitted variable bias.

As to your second question, I would think unquestionably the answer is yes, though the estimation would be a lot harder than people would think. Deep passes don't just happen in some fixed quantity distribution. They are selected based on game circumstances, the qb you have, and all other things.

#9 by Aaron Brooks G… // Jun 01, 2022 - 12:59pm

That's it's Roethlisberger, and in the last two years where it corresponds to his collapse in ADoT and ALEX at least raises the question whether short throws appear luckier because they may be harder to intercept. There's at least some logical basis, because a sudden short throw may give the DB less time to react to the throw, because they don't know the route.

There are a lot of captains checkdown on that lucky list. Although there's also Favre and Romo, who may just be lucky. There's also Luck, who is by default.

#11 by tjb // Jun 01, 2022 - 1:29pm

Also, short throws are just different in general

1) the defender is more likely to be a LB or even DL.

and

2) the defender in coverage, regardless of position, is probably more focused on making a tackle than attacking the ball

#10 by Pat // Jun 01, 2022 - 1:24pm

Lucky's just a term. Really, you're just saying "Roethlisberger's thrown a lot of balls which have been dropped by defenders." Casually you'd look at those and say "man, that guy's getting away with one" if that happens (so to a casual fan you call it luck), but if the QB's throw isn't easy to catch in that situation, it's not really luck. There's no a priori reason to assume there's no correlation.

#14 by Aaron Brooks G… // Jun 01, 2022 - 1:55pm

Considering DBs are WRs who can't catch, I'm curious what the appropriate expected catch rate is for DBs.

#18 by Hoodie_Sleeves // Jun 02, 2022 - 10:49am

We've had this discussion before - luck is just a word, but it's the wrong one.

"Unexplained" and "lucky" don't mean the same thing, and conflating the two just makes the discussion less clear and less useful.

Luck is an excuse to stop looking and stop trying to understand.

#19 by theslothook // Jun 02, 2022 - 11:24am

Just to be clear, my point above was to say you cant prove it isn't luck simply by running a time series regression, finding a statistically significant result, and calling it a day.

If its a random walk, which is what the equation declaring its all luck would show, then a correlation will show up that's completely spurious.

#12 by jimbohead // Jun 01, 2022 - 1:33pm

I think they've shown correlation coefficients on this stuff before, but I for one would love to see a retrospective on X number of years of this stat, comparing year-over-year correlation of Total Ints v. Adjusted Ints.

I also wonder, in our bold new age of machine vision and analytics, if a pass-by-pass analysis of ball placement (which might include 3d position relative to the WR when it arrives, velocity, relative position of defenders, etc) could predict WR drops and WR drops leading to INTs.

One of these projects is much easier than the other.

#17 by Hoodie_Sleeves // Jun 02, 2022 - 10:35am

I'm with you here - this is almost certainly not luck/randomness.

Eli Manning is given as an example of an unlucky QB - I watched a good amount of Giants football during that time period, and Eli had a bunch of really tall WRs and had a habit of throwing the ball high over the middle on crossing routes. It led to a lot of big completions, but it also led to a ton of balls being tipped directly at the deep safety, or being too high altogether for the receiver and just being thrown directly at the safety.

This seems like "pitchers have no control over batted balls" - interesting, but ultimately very incomplete.

#13 by KnotMe // Jun 01, 2022 - 1:35pm

It might make sense to do luck/unluck as a rate per att rather than an absolute value. J.Brissett (-1.4) in only 224 passes stands out at the lucky end and Lamar Jackson was even more unlucky as he was second, but in only 378 passes. (he would probably pass Herbert for worst luck).

I suppose what they are going for is, assuming it actually is luck, the young QB may improve slightly with better luck next year.

#15 by Raiderfan // Jun 01, 2022 - 8:54pm

“Mac Jones (+2.2) is in fourth, ”

Also “Tom Brady (+2.8), a very old quarterback from the NFC, finished in fourth place between Jackson and Jones.”

#16 by Vincent Verhei // Jun 01, 2022 - 9:58pm

The text was wrong, the table was correct. Brady finished in third.

#20 by Seattle-Brian // Jun 03, 2022 - 11:27am

I'm surprised to see Wentz grading out so well on adjusted interception rate, tied for 4th in the league.

#22 by Vincent Verhei // Jun 03, 2022 - 3:56pm

Wentz has been pretty good about avoiding interceptions for most of his career. 2020 was a major exception in that regard. But he was 27th in completion percentage last season, and that kind of erratic performance led to slumps where he couldn't get very much going. But even then he wasn't often turning the ball over left and right. He only had two games with multiple interceptions.

#23 by theslothook // Jun 03, 2022 - 4:00pm

Yeah I think Wentz reputation is heavily colored by the fact that his turnovers are especially gag worthy and inexplicable in how and when they happen.

#24 by Vincent Verhei // Jun 03, 2022 - 5:18pm

This is also true. He didn't have a lot of turnovers, but it seemed like all of them were especially bad, as far as turnovers go.

#21 by nat // Jun 03, 2022 - 1:02pm

Way back in https://www.footballoutsiders.com/dvoa-analysis/2022/cowboys-lead-way-too-early-2022-dvoa-projections Aaron (sorta) predicted a smaller than typical second-year improvement for Jones. His DVOA was already high for a rookie, probably benefitted from some lucky breaks, and should be expected to regress - or at least to not improve as much as second year QBs typically do.

This article is encouraging if you hope for better. It eliminates interception luck from last season’s picture, which is the biggest piece of luck-noise in QB DVOA. So maybe Jones’ DVOA mostly reflected actual skill, and we should expect a normal-sized second year improvement.

It’s something to watch for in the coming season.