Writers of Pro Football Prospectus 2008

13 Jun 2017

Quarterbacks and Salary, 2016

Guest column by Kevin Kolbe

This is the third year in a series of columns analyzing how much various quarterbacks around the league are worth relative to each other. For those of you who don't remember the previous articles, you can read the original 2015 article here. The goal of this annual series is to develop a formula to accurately gauge a quarterback's worth. This will be done by comparing each quarterback's yearly cap hit to statistical performance in TANY/A (Total Adjusted Net Yards/Attempt), and then by adjusting that base amount according to his importance to the offense (judged by plays per game). The final result is deemed a player's PAYD (Pay by Average Yards and Demand).

A couple of administrative notes for past readers: this is the first version to use annual cap hits as the standard for comparison (as opposed to average value), and also the first to calculate time on field to one-tenth of a game, resulting in a more accurate measurement of plays per game.

Having refined the process a bit, I also added more data to evaluate, so this year's article will look at all qualifying quarterback seasons (those who played at least eight full games) since 2012 to see what can be determined. However, let's start with the results from the 2016 season:


Quarterbacks and PAYD, 2016 (8+ games played)
Player Games Plays/G TANY/A Base Value Rookie-Adjusted
Base Value
PAYD Cap Hit PAYD-Cap
M.Ryan 15.7 39.6 8.55 $24.2
$24.2 $23.8 $0.4
T.Brady 11.6 40.9 8.19 $22.3
$23.8 $13.8 $10.0
D.Brees 16.0 45.2 6.97 $15.7
$20.1 $17.3 $2.8
A.Rodgers 15.7 45.4 6.93 $15.4
$19.9 $19.3 $0.6
D.Prescott* 15.3 35.4 8.11 $21.8 $23.7 $18.7 $0.5 $18.2
K.Cousins 16.0 41.4 7.03 $16.0
$17.5 $20.0 $(2.5)
D.Carr* 14.9 41.3 6.68 $14.1 $15.9 $17.3 $1.5 $15.8
A.Luck 14.9 43.6 6.54 $13.3
$16.0 $18.4 $(2.4)
B.Roethlisberger 13.3 40.8 6.74 $14.4
$15.3 $24.0 $(8.7)
M.Stafford 16.0 41.8 6.46 $12.9
$14.3 $22.5 $(8.2)
M.Mariota* 14.4 37.2 6.68 $14.1 $15.9 $14.0 $5.5 $8.5
R.Wilson 15.3 43.2 6.03 $10.6
$12.5 $18.5 $(6.0)
A.Dalton 16.0 40.6 6.16 $11.3
$11.9 $13.1 $(1.2)
T.Siemian* 13.5 40.4 5.67 $8.6 $10.5 $10.9 $0.5 $10.4
T.Taylor 14.7 39.0 6.13 $11.1
$10.8 $6.9 $3.9
Player Games Plays/G TANY/A Base Value Rookie-Adjusted
Base Value
PAYD Cap Hit PAYD-Cap
S.Bradford 14.9 41.0 5.92 $10.0
$10.7 $7.0 $3.7
A.Smith 14.2 39.9 5.97 $10.2
$10.4 $17.8 $(7.4)
P.Rivers 15.9 39.5 5.96 $10.2
$10.1 $16.5 $(6.4)
J.Winston* 15.9 41.2 5.44 $7.4 $9.2 $9.9 $5.8 $4.1
B.Bortles* 16.0 44.9 5.16 $5.9 $7.7 $9.8 $5.6 $4.2
C.Palmer 14.7 44.3 5.48 $7.6
$9.4 $18.4 $(9.0)
C.Kaepernick 10.8 40.4 5.69 $8.7
$9.1 $16.8 $(7.7)
J.Flacco 15.8 45.9 5.21 $6.1
$8.0 $22.6 $(14.6)
E.Manning 16.0 40.0 5.50 $7.7
$7.9 $24.2 $(16.3)
C.Newton 14.7 43.3 5.30 $6.6
$7.8 $19.5 $(11.7)
R.Tannehill 12.9 35.4 5.69 $8.7
$6.9 $11.6 $(4.7)
C.Wentz* 16.0 42.9 4.57 $2.7 $4.5 $5.3 $4.9 $0.4
C.Keenum 9.1 40.1 4.67 $3.2
$3.3 $3.6 $(0.3)
R.Fitzpatrick 11.8 38.6 4.46 $2.1
$2.0 $12.0 $(10.0)
B.Osweiler 13.9 40.9 4.18 $0.6
$0.6 $12.0 $(11.4)
* Base value adjusted for players still on their rookie contracts.

As you probably would have guessed, Matt Ryan led all quarterbacks last year with 8.55 adjusted net yards/play. Unsurprisingly, he ended up with the highest PAYD value. Tom Brady was a very close second, being only slightly less efficient. Drew Brees (not Dak Prescott) was a somewhat distant third, being far less efficient than Ryan or Brady, but with the overwhelming volume that we generally expect with his role in the Saints offense. Brock Osweiler finished dead last, with Ryan Fitzpatrick and Case Keenum not far away.

Let's talk about Drew Brees for a moment, because his numbers have been of particular interest to me. When I first presented my proposal for the PAYD formula, I was asked by both commenters and editors two questions: First, why I took plays per game into account, and second, why I chose the modifier that I did. The answer to the first was pretty easy, as I genuinely feel that plays per game is a good indicator of how much trust a coach places in his quarterback. In 2013, for example, Sam Bradford was averaging 42.3 plays per game until he got injured. Kellen Clemens finished out the year for the Rams, and, despite spending less time with a lead, only averaged 31.4 plays per game. This clearly spoke to Clemens not being asked to do as much. Of course, other things affect plays per game as well. Over the past five years, the season in which Aaron Rodgers was entrusted with the fewest plays per game was 2014, despite the fact that it was his best season during that span. Likewise, Matt Schaub's 47.4 plays per game in 2013 is the second-highest such mark since 2012, even though that was one of the worst seasons we've seen. That's because Rodgers spent a lot of time handing the ball off to run out the clock, while Schaub was desperately trying to get back into games.

So now the second question: Why did I put this particular value on play demand? PAYD modifies base value by twice the percentage difference of plays per game (in 2016, using Matt Ryan's 39.6 plays per game as the standard). For example, if a quarterback with a base value of $10 million averaged 43.6 plays per game (10 percent more than Ryan's mark), his PAYD would be $12 million per year (20 percent more than his base value). I chose this method in order to account for quarterbacks who played badly and were forced to pass the ball more to catch up. For example, in 2013, Matt Schaub had 21.5 percent more plays per game than Nick Foles (the most efficient quarterback that year). However, because Schaub's base value was so low, that huge percentage increase did almost nothing for him. Unfortunately, another type of quarterback presents another issue: Drew Brees (who I promised to talk about two paragraphs ago).

Over the past five years, Drew Brees has had the most plays per game with 45.0, and the highest PAYD value in the league, averaging $20.6 million per year. This seems odd: after all, even in Brees' best season, he was still only the fourth most efficient quarterback in the NFL. That can't be good. However, I looked at his worst season in that span and discovered it actually wasn't that much lower. In fact, looking at all five seasons, Brees is actually the fifth most efficient quarterback in the league. The most efficient has been Prescott, whom we can probably disregard due to a small sample size of one season. (Kirk Cousins, at No. 4, was similarly brushed off.) The second- and third-ranked quarterbacks have been Rodgers and Brady, each within two-tenths of a yard per play. Additionally, while the top nine quarterbacks in per-play efficiency each cracked the top 10 list for PAYD, only two of the top four quarterbacks in plays per game could make the same boast. Andrew Luck was the only player to make the top 10 in PAYD without being one of the 10 most efficient, but he benefitted from both a high number of plays per game and the "potential" modifier (see last year's article for a description of this). Feel free to let me know what you think in the comments, but for now I think I am going to leave that modifier as is.

For those interested, here is full PAYD and value data for all qualifying quarterbacks in the past five years:


Quarterback PAYD Value, 2012-2016
Name 2012 2013 2014 2015 2016 Average
D.Brees $22.3 $19.3 $15.7 $25.7 $20.1 $20.6
T.Brady $23.6 $9.0 $14.4 $26.0 $23.8 $19.4
K.Cousins*


$20.0 $17.5 $18.8
D.Prescott



$18.7 $18.7
A.Rodgers $19.9 $18.3 $20.4 $14.1 $19.9 $18.5
M.Ryan $19.8 $9.0 $15.4 $16.3 $24.2 $16.9
B.Roethlisberger $15.6 $9.5 $18.4 $24.5 $15.3 $16.7
P.Manning $20.9 $27.1 $16.9 $1.2
$16.5
A.Luck* $15.8 $12.9 $20.8
$16.0 $16.4
R.Wilson* $12.2 $10.5 $13.8 $24.4 $12.5 $14.7
M.Mariota


$14.8 $14.0 $14.4
M.Stafford $17.9 $10.5 $10.9 $17.2 $14.3 $14.2
J.Freeman* $13.6



$13.6
C.Palmer $14.7 $5.9
$23.8 $9.4 $13.5
C.Newton* $18.0 $9.7 $9.2 $22.3 $7.8 $13.4
T.Romo $17.2 $10.7 $12.2

$13.4
R.Griffin* $17.7 $8.9


$13.3
A.Dalton* $12.0 $12.2 $8.1 $21.5 $11.9 $13.1
Name 2012 2013 2014 2015 2016 Average
J.Winston


$16.3 $9.9 $13.1
T.Taylor


$15.2 $10.8 $13.0
D.Carr

$5.3 $15.8 $17.3 $12.8
P.Rivers $6.7 $15.2 $11.0 $19.1 $10.1 $12.4
N.Foles*
$22.6
$2.2
$12.4
B.Hoyer

$7.8 $14.7
$11.3
T.Siemian



$10.9 $10.9
J.Locker* $10.6



$10.6
A.Smith $10.4 $9.0 $9.1 $14.0 $10.4 $10.6
M.Vick $10.4



$10.4
E.Manning $12.8 $0.7 $12.8 $16.9 $7.9 $10.2
S.Bradford* $11.4

$7.8 $10.7 $10.0
B.Bortles

$2.8 $16.7 $9.8 $9.8
J.Cutler $7.1 $8.8 $7.8 $14.4
$9.5
R.Tannehill* $8.8 $6.4 $11.9 $13.3 $6.9 $9.5
B.Gabbert* $6.8

$11.1
$9.0
J.Flacco $12.4 $1.2 $10.7 $10.7 $8.0 $8.6
R.Fitzpatrick $8.2 $6.6 $9.2 $16.9 $2.0 $8.6
Name 2012 2013 2014 2015 2016 Average
B.Weeden* $8.5



$8.5
K.Orton

$8.4

$8.4
T.Bridgewater

$8.2 $8.2
$8.2
C.Kaepernick
$9.5 $9.4 $4.1 $9.1 $8.0
M.Schaub $12.2 $2.4


$7.3
C.Ponder* $7.2



$7.2
M.Sanchez* $2.7
$9.7

$6.2
C.Wentz*



$5.3 $5.3
G.Smith
$2.0 $5.4

$3.7
E.Manuel
$3.7


$3.7
C.Henne
$3.5


$3.5
M.Cassel $3.3



$3.3
C.Keenum



$3.3 $3.3
M.Glennon*
$3.2


$3.2
K.Clemens
$1.3


$1.3
J.McCown

$0.8

$0.8
B.Osweiler



$0.6 $0.6
* Player was on rookie contract for at least one season.


Difference Between PAYD and Cap Hit, 2012-2016
Name 2012 Dif 2013 Dif 2014 Dif 2015 Dif 2016 Dif Total Dif
R.Wilson* $11.7 $9.8 $13.0 $17.3 $(6.0) $45.8
D.Carr*

$4.3 $14.6 $15.8 $34.7
A.Luck* $11.8 $7.9 $14.8
$(2.4) $32.1
A.Dalton* $10.8 $10.8 $(1.0) $11.9 $(1.2) $31.3
T.Brady $9.8 $(4.8) $(0.4) $12.0 $10.0 $26.6
R.Tannehill* $6.5 $3.5 $8.4 $8.4 $(4.7) $22.1
N.Foles*
$21.9
$(1.8)
$20.1
C.Palmer $10.7 $1.9
$16.4 $(9.0) $20.0
R.Fitzpatrick $5.8 $4.2 $5.8 $13.6 $(10.0) $19.4
M.Mariota*


$10.4 $8.5 $18.9
D.Prescott*



$18.2 $18.2
T.Taylor


$14.3 $3.9 $18.2
R.Griffin* $13.9 $4.1


$18.0
A.Rodgers $11.4 $6.6 $2.8 $(4.2) $0.6 $17.2
K.Cousins*


$19.3 $(2.5) $16.8
C.Newton* $13.0 $3.7 $2.2 $9.3 $(11.7) $16.5
B.Hoyer

$6.5 $9.5
$16.0
J.Winston*


$11.7 $4.1 $15.8
Name 2012 Dif 2013 Dif 2014 Dif 2015 Dif 2016 Dif Total Dif
B.Bortles*

$(1.0) $12.0 $4.2 $15.2
T.Bridgewater*

$7.0 $6.6
$13.6
B.Gabbert* $4.1

$9.3
$13.4
J.Freeman* $12.3



$12.3
T.Siemian*



$10.4 $10.4
D.Brees $4.9 $1.9 $(2.7) $1.9 $2.8 $8.8
J.Locker* $7.7



$7.7
B.Weeden* $7.0



$7.0
G.Smith
$1.1 $4.3

$5.4
C.Ponder* $4.9



$4.9
T.Romo $5.4 $(1.1) $0.4

$4.7
M.Ryan $10.2 $(0.6) $(2.1) $(3.2) $0.4 $4.7
K.Orton

$4.4

$4.4
M.Glennon*
$2.6


$2.6
E.Manuel*
$2.1


$2.1
B.Roethlisberger $6.7 $(4.1) $(0.5) $7.3 $(8.7) $0.7
K.Clemens
$0.7


$0.7
C.Wentz*



$0.4 $0.4
Name 2012 Dif 2013 Dif 2014 Dif 2015 Dif 2016 Dif Total Dif
C.Keenum



$(0.3) $(0.3)
M.Cassel $(0.4)



$(0.4)
C.Henne
$(1.2)


$(1.2)
S.Bradford* $(1.2) - - $(5.2) $3.7 $(2.7)
M.Sanchez* $(10.2)
$7.4

$(2.8)
A.Smith $1.1 $0.5 $4.5 $(1.6) $(7.4) $(2.9)
M.Vick $(3.5)



$(3.5)
P.Manning $3.4 $9.6 $(0.6) $(16.3)
$(3.9)
J.McCown

$(4.0)

$(4.0)
C.Kaepernick*
$8.1 $5.6 $(11.2) $(7.7) $(5.2)
M.Schaub $1.4 $(8.4)


$(7.0)
B.Osweiler



$(11.4) $(11.4)
J.Cutler $(2.5) $(1.6) $(10.7) $(2.1)
$(16.9)
M.Stafford $0.1 $(7.3) $(4.9) $(0.5) $(8.2) $(20.8)
P.Rivers $(8.6) $1.4 $(5.7) $(2.1) $(6.4) $(21.4)
J.Flacco $5.6 $(5.6) $(4.1) $(3.9) $(14.6) $(22.6)
E.Manning $(8.1) $(20.2) $(7.6) $2.4 $(16.3) $(49.8)
* Player was on rookie contract for at least one season.

Looking back at the last five years, here are some fun facts:

The best single-season value was Nick Foles, who overachieved his pay by $21.9 million in 2013. If we throw out players on rookie deals, though, the best value becomes 2012 Aaron Rodgers, whose PAYD exceeded his cap hit by $11.4 million. The worst single-season value was Eli Manning, coming in a full $20.2 million below his cap hit in 2013.

Over the last five years, the best total value was Russell Wilson. This shouldn't really surprise anyone. Given how absurdly low his cap hits have been, and considering that young players get a bonus to their PAYD value, he had a huge advantage for four of the five years in this study. Andrew Luck missed 2015 and had a higher average cap hit, so despite a slightly higher PAYD, he comes out behind Wilson on value. Among non-rookie deals, the even less surprising winner is Tom Brady, whose combined PAYD from 2012-2016 was $26.6 million higher than his total cap hits. Carson Palmer came in next at $20.0 million above, and Ryan Fitzpatrick third at $19.4 million. Despite only having two qualifying seasons, Tyrod Taylor managed a strong fourth place with $18.2 million.

On the other end of the spectrum, the worst PAYD deficit came from Eli Manning, and it wasn't close. You may remember that he had the worst single-season value in the last five years in 2013, but he also managed the second-worst in that span in 2016. Man, having rings sure pays off. Over the past 5 years, Eli's PAYD has been worth $49.8 million less than his cap hits. A distant second place goes to Joe Flacco at $22.6 million below, with Philip Rivers and Matthew Stafford being less than $2.0 million behind. Stafford, by the way, was one of the league leaders in plays per game at 44.5. Fourth from the bottom, and worthy of mention, is Jay Cutler at minus-$16.9 million. Had he qualified in 2016, he likely would have moved into second place.

Next year, in addition to adding in additional seasons (I eventually plan on going back to 2004), I plan on weighting cap percentage more heavily to aid in comparing across seasons. After all, Peyton Manning's 27.1 million PAYD value may already seem impressive, but in 2013, that was 22 percent of the cap, which would have been unheard of. If you have other suggestions, feel free to comment below.

Guest columnist Kevin Kolbe is originally from Houston, Texas, and has been in the Air Force since December of 2001. If you are interested in writing a guest column, something that takes a new angle on the NFL, please email us your idea at info-at-footballoutsiders.com.

Posted by: Guest on 13 Jun 2017

34 comments, Last at 20 Jun 2017, 10:32am by nat

Comments

1
by Aaron Brooks Go... :: Tue, 06/13/2017 - 4:46pm

Why do you track games played, if you don't seem to use it for anything?

I get that the metric is based on a per-game basis, but cap hit is per-season. Shouldn't games played show up somewhere in the calculation. Roethlisberger and Brady seem skewed without it.

2
by kckolbe :: Tue, 06/13/2017 - 6:30pm

Total plays per season is divided by games played to calculate plays per game. The metric is based on season performance.

8
by Aaron Brooks Go... :: Wed, 06/14/2017 - 9:15am

The metric is based on per-game performance, multiplied out over a 16 game season. Except a notable list of QBs did not play a full season, so their per-season effectiveness is over-represented.

Brady played 475 plays in 2016. Matt Ryan played 606. Their per-play effectiveness was roughly the same, but the PAYD suggests they were equivalent on a cap-value basis.

Brady has an excuse, but consider Roethlisberger. The criticism again him is that he's a great QB for 12-13 games, but it's a 16-game season. Shouldn't his tendency towards injury be factored into his value on a per-season metric?

Basically, this is presented as DYAR, when it's really DVOA.

9
by kckolbe :: Wed, 06/14/2017 - 12:46pm

Now I see what you mean, and it's actually a really good point.

I mentioned in comment #7 that durability was something I eventually wanted to tackle, and I am working on it, but I haven't quite figured out how. The easy approach would be to simply factor games played into the formula, but that would have quite a few unintended effects.

For one, QBs that get benched in the lead so the backup can take the kneel-downs would be penalized, but I actually want that, as kneeling out the clock hurts both TANY/A and PPG, so this would be a nice offset to that.

Secondly, what about QBs that get benched partway through a season? We saw it this last year with Ryan Fitzpatrick, and the reason why was pretty clear. However, it happened to Fitzpatrick in 2014 as well, and performance certainly wasn't the issue there. If anything, it seemed to be a desire to evaluate a different QB before his contract was up. I'd hate to see a QB be penalized there.

Lastly, time lost to injury isn't always straightforward. In 2012, Alex Smith suffered a concussion that caused him to miss somewhere between the rest of one game and just under half a season. In 2015, Ryan Mallett got his bell rung, causing him to miss between one play and the rest of the game. In both cases, they were benched for injury, but stayed on the bench due to backups playing well. How much time should be counted?

Please believe I am completely on board for factoring in time off the field, as it absolutely does matter, but I don't feel I have a fair enough method to implement it yet.

22
by Aaron Brooks Go... :: Thu, 06/15/2017 - 9:16am

Doesn't TANY/A provide the efficiency metric you're looking for?

I get the garbage time problem, but that affects DVOA, too. It would seem less harmful to just manually prune the handful of running plays that affects.

23
by kckolbe :: Thu, 06/15/2017 - 10:44am

The problem was more the last two paragraphs of my comment, accounting for players sitting out for non injury reasons

3
by eagle97a :: Tue, 06/13/2017 - 8:09pm

I assume that the data you have only goes back to 2004? I had faint hope you can go back to 1993 to enclose all the salary cap years. Very nice study! Many thanks!

4
by kckolbe :: Tue, 06/13/2017 - 8:23pm

The current plan is to go all the way back to 2004, as that marked a significant jump in league-wide pass efficiency, not to mention three very noteworthy QBs. At the moment, all I've compiled is from 2012-2016. Glad you liked it, though!

5
by serutan :: Tue, 06/13/2017 - 9:09pm

Posted before I read original article. Ooops.
______
Was wr

6
by Dan :: Wed, 06/14/2017 - 3:23am

As I mentioned in the comments to one of the earlier articles, I think the formula puts too much weight on having a cheap salary and too little weight on performance. There should be a wider spread in the dollar value of PAYD between the top QBs and the bottom QBs.

Matt Ryan production at Matt Ryan's salary is more useful for a team than Sam Bradford production at Sam Bradford's salary or Carson Wentz production at Carson Wentz's salary. Eli Manning has a 39-41 record over the past 5 years, which is much more in keeping with his middling performance numbers than his bottom-of-the-barrel PAYD-Cap.

To be fair, there are some data points which point to the importance of cap hit (like Drew Brees). But I bet that, if you looked at things systematically, you'd find that PAYD is more important than cap hit at predicting which QBs have more success.

7
by kckolbe :: Wed, 06/14/2017 - 5:56am

Hi there, and I do remember your comments in my first article. You weren't the only one to question just how much a true franchise QB is worth. Please remember, though, that I am still not making that determination. I only compare QB pay to QB pay, and am not comparing it to other positions. I think the PAYD formula places a great weight on performance. The PAYD-Cap table is just a reference, not part of the formula.

The way the formula works, it is possible to have a VERY wide gap in PAYD. In 2013, for example, Peyton Manning had a PAYD of $27.1 million, while Eli Manning was at $0.7 million. The league has never had that great a difference in actual cap hits. Now, there are things that could be done to boost the pay of top QBs, such as using a non-linear progression, but that would also cause a "down year" to be more punishing.

So, your next point requires some clarification. For 2016, I agree. Bear in mind that Wentz's PAYD is affected by the rookie adjustment. He wasn't started this year for his ability, but so that he could develop. The 0.34 bonus to TANY/A he receives raises his PAYD significantly. Were it not for that, his PAYD would have been $3.2 million, over 2 million less.

Eli Manning's TANY/A over the past 5 years is 5.82. The TANY/A of all qualifying QBs (which includes backups and developing rookies) over the same period is 5.89. I would argue that his PAYD is pretty accurate for that kind of performance. However, I will point out that once his 2004-2011 numbers are factored in, he becomes a MUCH better bargain due to stronger performance and lower cap hits. Additionally, PAYD currently has no account for durability, something I do believe should matter. A factor like that would be of significant value to him, and that is something I intend to experiment with down the road. Remember that this formula is being added to over the years. So while he is getting hit pretty hard right now, it's only going to look better for him.

10
by Hoodie_Sleeves :: Wed, 06/14/2017 - 3:39pm

Eli is a bad example for this - but his point is absolutely valid. Eli's issue isn't that he's being paid like an elite quarterback - his issue is that he isn't an elite quarterback anymore. The Giants aren't bad because they're overpaying him, they're bad because he's no longer any good.The fact that they're paying too much is just icing on the cake.

At QB, performance is way more important than cost - for example - Tyrod Taylor's margin is significantly higher than Matt Ryan's, but there's no way in hell Atlanta would make that swap (and Buffalo would make it in a heartbeat).

13
by RickD :: Wed, 06/14/2017 - 7:11pm

Trying to figure out when Eli was considered elite...

Certainly he played like an elite QB in his Super Bowl runs, and at least the 2nd MVP was well-deserved. But for the majority of his career he hasn't been considered even a Pro Bowl-level QB. At his best he's been a top-10 QB, but never really a top-5 QB, and he's been middle-tier as often as he's been top-10.

14
by Dan :: Wed, 06/14/2017 - 8:27pm

I agree that PAYD measures performance. But it's translated into dollar amounts, and the reason for that is so that it can be compared with cap hit. And in order for that to be a meaningful comparison, it is very important how it is scaled.

The range of PAYD is a bit wider than the range of QB salaries, but I think that it should be significantly wider. First: The top QBs are underpaid, as evidenced by the fact that they win a lot. Second: The top single season PAYD performances happen when a great QB has an unusually good year even for him, so they should be higher than the top cap hits because teams decide how much to pay QBs based on their expected performance not based on this sort of unexpectedly good performance.

I agree that Eli Manning has been getting paid like the top QBs do even though he's only an averageish QB rather than a top QB. But I think that he's been getting paid about what he's worth (maybe slightly too much), it's just that the top QBs are drastically underpaid relative to what they're worth.

16
by sbond101 :: Wed, 06/14/2017 - 10:32pm

I think how to scale PAYD is a fascinatingly complicated question; One which in the end comes back to inter-connected performance in football. That is to say, to what extent are the top tier QB's in football top tier QB's because of the situation in which they play. By extension, on teams like Green Bay, the Pats, is it the QB whose under-paid or other players on the team. It's a question that can't be readily answered with the statistical tools at our disposal, but I would submit into evidence the last 5 years of the Drew Brees Saints as a counter-example to the idea that teams with great QB's win a ton, and therefore great QB's are under paid (7-9 4 out of 5 years). I do think the scaling near the bottom is badly wrong, as there are 10 or so teams that don't seem to be able to get value out of any QB (I maintain that even if the Browns had drafted Aaron Rogers, they would have found a way to lose 10+ games a year, and make him look like a journeymen).

Eli Manning to me is the quintessential example of a 75th percentile QB, there have always been 5-10 guys better than him in the league at any given time. The fact that the Giants have missed the playoffs 4 of the last 5 years, and have had an otherwise fairly average cap performance (at least as far as I have followed it), suggests to me that Manning is paid at least full value, and likely somewhat overpaid, but at this level the analysis is really course.

17
by kckolbe :: Wed, 06/14/2017 - 10:49pm

It really is fascinating. I meant to respond to it with comment #15, but posted it at the bottom in my haste.

The QB vs system debate is a fine one, but also one I don't try to solve. There are plenty of analysts on this site and others far more qualified to judge the "eye tests." I'm not good at it. I like numbers.

What do you mean about the scaling near the bottom? I use a linear system, so it's the same proportion of pay all the way through for base value, though plays per game are less rewarded the less efficient a QB is. I do see what you mean about certain teams having poor success, and I do believe it has at least as much to do with the team as with the QB. As I said, though, I'm not trying to differentiate.

18
by sbond101 :: Wed, 06/14/2017 - 11:19pm

Putting aside the issue of team vs. QB in the PAYD analysis; The comment about scaling near the bottom concerns the concept of value over a replacement-level rookie. I'm not in a position to look at the numbers, but if the average rookie season for a QB drafted after pick 20 over the last 10 years had an TANY/A of 3.5, the PAYD value of TANY/A from 0 to 3.5 should scale to a middling rookie contract, as a player delivering a TANY/A of 3.5 delivered middle-draft replaceable performance. In reality a TANY/A value below 1.5 might be undrafted free agent replacement level. Having key points like this as reference should help shape the function properly at the low end (it's my impression that the slope of PAYD/TANY should have a curve somewhere at the low end, which I think would help identify the myriad of overpaid below-replacement level performances).

The reason why this thought is proximal to the discussion of team vs. QB performance, is that at the low end I think you likely see powerful "system" effects. These effects make allocating PAYD to the QB vs. the rest of the offense, as well as establishing "replacement" level deeply problematic. On the other end, there are those that would argue that 2008-level Matt Cassel represents replacement-level QB play in the Pat's system. I would expect that the PAYD value of that season significantly exceeded a mid-first round rookie deal.

As I said, scaling the function with large un-accounted for factors like system and replacement is a huge challenge, and whatever methods you use will be vulnerable to criticism, I just think linearly scaling something that is intuitively not linear is obscuring some insights that might be made from this data.

19
by kckolbe :: Thu, 06/15/2017 - 8:54am

So bear in mind that a QB has to play 8 full games to qualify for my analysis. Because of this, you will almost never see a TANY/A at 3.5. The lowest we've seen in the past 5 years is Mark Sanchez in 2012, with 3.61. At the time, he was an established starter on a big new contract. Any unproven who QB who played that badly would get benched, or not get the chance until too late in the season. Bryce Petty would have had a sub 3 TANY/A last year, but he didn't get 8 full games. To make this more painful for Jets fans, in one game Ryan Fitzpatrick had a TANY/A of NEGATIVE 1.28. I would say that veteran backup replacement level is around 5.0 adjusted net yards/play. That's a "from the hip" assessment, though. Again, many players will do worse, but not for long enough to qualify.

25
by Dan :: Thu, 06/15/2017 - 11:38pm

I would submit into evidence the last 5 years of the Drew Brees Saints as a counter-example to the idea that teams with great QB's win a ton, and therefore great QB's are under paid (7-9 4 out of 5 years)

Brees's 39-40 record over the last 5 years is some evidence against the idea, but there is a still a very strong trend where, on the whole, the top QBs (Peyton, Brady, Brees, and Rodgers) win more than their share of games even after they're on big contracts (and Brees's 2006 contract was up there). There is an even stronger trend where the top QBs consistently have good offenses - the Saints have struggled over the past 5 years because of their bad defense. Spending big bucks on a QB does limit what you can spend on your defense, but I bet you'd find that the Saints defense has been worse than what you'd predict based on the resources that they've spent on it.

Brady has been blessed with an amazing coach/franchise/system, and some of the past greats were too (like Montana & Young with Walsh), but I don't think that's especially true of guys like Peyton Manning (who succeeded with a few different coaches) and Rodgers.

26
by sbond101 :: Fri, 06/16/2017 - 8:26am

It's definitely fair to say that Manning & Rodgers have been underpaid through their careers (Brady might be in that category as well, but between BB, his constant contract restructuring, the Defense/Moss/Gronk I think it's fair to put him to the side in this analysis as he's always been hard to evaluate from a cap POV). That argument often gets unjustifiably extended to QB's the next tier down of QB's (Brees & Big Ben (HOFers but not among the GOAT), sometimes even Rivers, Romo, and Wilson), who generally don't win anything without very good teams around them. The issue is that Rodgers, Big Ben, Brees, & Rivers have all had cap hits +/-10% the last three years, those players clearly aren't all +/- 10% in terms of their skills. To make your argument successfully, you really need to restrict what you call the "top QB's" tightly.

28
by kckolbe :: Fri, 06/16/2017 - 10:48am

This was actually the premise of my first article. A large percentage of franchise QBs were being paid as though they were all among the elite, despite some clearly not being close. I disagree with your assessment of Brees, as he has been among the best these past 5 years, but in general, I fully agree.

31
by The Ninjalectual :: Mon, 06/19/2017 - 12:38pm

If the Browns drafted rookie Aaron Rogers, you mean, then they would have ruined him. Right? Because if the Browns somehow acquired the veteran 2017 Rogers, I think they'd contend for the playoffs instantly.

32
by sbond101 :: Mon, 06/19/2017 - 1:58pm

Ruined him is basically what I meant (through poor development, and very likely injuries resulting from poor game-planning).

11
by RickD :: Wed, 06/14/2017 - 7:01pm

Does anybody else suspect Matt Ryan might witness a major regression this season, esp. with Kyle Shanahan having left town?

12
by kckolbe :: Wed, 06/14/2017 - 7:06pm

With the amazing season he had this year, compared to his average performance, a regression is pretty much a certainty regardless of circumstances.

20
by Aaron Brooks Go... :: Thu, 06/15/2017 - 9:12am

The second-half of the Super Bowl suggests he might actually be dead.

On the other hand, a dead QB would have run the clock out more effectively.

27
by nat :: Fri, 06/16/2017 - 9:55am

It's fun to say that.

But it's factually dead wrong. Ryan had a decent passer rating in the second half, and used up a lot more time by gaining first downs (or scoring a TD) via passes than he added back with incomplete passes.

29
by Aaron Brooks Go... :: Fri, 06/16/2017 - 1:46pm

The incomplete passes weren't really his problem. He took some disastrous sacks (and sack-fumbles) at inopportune times. These were plays that were below replacement-level, and beyond the pale for an MVP.

It was like he had a spread to cover.

30
by nat :: Mon, 06/19/2017 - 10:29am

Sacks and turnovers are always - in isolation - below replacement level. Saying so is like saying a QB had a perfect passer rating on all his TD passes: true, but silly. Replacement level as a useful concept really only applies to whole sets of plays, such as seasons, playoff seasons, games, and to a lesser extent quarters.

Getting sacked in the fourth quarter of a Super Bowl is hardly "beyond the pale" for MVP QBs. Since 2001, the list includes Warner, Gannon, Brady, P. Manning, Newton, and Ryan. Since that's the complete list of league MVP QBs who played in the Super Bowl since 2001, I'd say fourth quarter sacks by MVP QBs are pretty normal.

If you meant for a Super Bowl MVP, then that's selection bias. Still, Eli Manning and Tom Brady have done it. If the Patriots had failed on their final two-point conversion, then Ryan would have done it, too.

33
by dank067 :: Mon, 06/19/2017 - 3:57pm

If we're talking about mistakes that are "beyond the pale," the Super Bowl MVP himself threw a couple of passes straight to defenders on the game-tying drive.

More to the original point, looking forward to watching Ryan (really the whole Falcons offense) this coming season. Though there will be some expected regression that comes just from having been the #1 offense, virtually all of their offensive personnel are returning intact but with a new coordinator. Even if they run a similar overall scheme, should be an interesting look at the importance of playcalling and coaching.

34
by nat :: Tue, 06/20/2017 - 10:32am

Yup.

Ryan and the Falcons offense should expect a regression towards the mean for the usual statistical reasons. But other indications, including their quality of play in the Super Bowl (yes, including that second half), point toward a strong season next year.

Atlanta fans should be optimistic. They stand a decent chance of going deep into the playoffs, and might win it all.

15
by kckolbe :: Wed, 06/14/2017 - 10:02pm

" The top single season PAYD performances happen when a great QB has an unusually good year even for him, so they should be higher than the top cap hits because teams decide how much to pay QBs based on their expected performance not based on this sort of unexpectedly good performance."

That is a fine point, and would also help alleviate the pain of the occasional down year. Perhaps I should add "bonuses" for exceeding certain thresholds? Currently, it is much easier to have a PAYD-destroying year than a PAYD-spiking one, and that should be addressed.

The question then would be where to draw those lines? What is the "mark" for a passable starter? There have been 148 qualifying seasons in the past 5 years. The mean TANY/A was 5.89. The median was 5.86. Here were some of the benchmarks.

52.7% of them managed a 5.8 or higher.
48% managed a 5.9+
41.9% managed 6.0+
39.9% for 6.1
37.8% for 6.2
32.4% for 6.3
29.7% for 6.4
...and so on, in small changes, until
5.5% for 7.5
And then NO ONE from 7.60 to 8.10
4.1% for 8.11 or higher

There's definitely something there to where anything over 7.6 is a rarity, but with so small a sample size I am reluctant to create "targets."

21
by Aaron Brooks Go... :: Thu, 06/15/2017 - 9:15am

I think it's a mistake to calibrate the scale off the TANY/A leader. That implicitly rejects the concept of outliers, even though your top performance will often be an outlier.

Yes, these guys are all elite QBs relative to the general population, but comparative performance is fractal in nature -- all comparisons, no matter the granularity, tend to look the same. I think you still see outliers even in intra-starter comparisons.

24
by kckolbe :: Thu, 06/15/2017 - 10:51am

You're right. A more accurate way would be to calibrate the scale off of league performance. In next year's article, I should have over 200 qualifying seasons. That should be enough to arbitrarily set a standard from.