Writers of Pro Football Prospectus 2008

27 Jan 2011

Rodgers' NFL Passer Rating is Ridiculous

Not ridiculous as in "it is ridiculously good," although it is. More like ridiculous as in "it is ridiculous that we are judging modern-day quarterbacks using a passer rating from 1970."

Posted by: Mike Tanier on 27 Jan 2011

70 comments, Last at 10 Feb 2011, 3:43pm by Bright Blue Shorts

Comments

1
by Chirality the Broilermaster (not verified) :: Thu, 01/27/2011 - 4:52pm

What would things look like if we used the current (or former) year's "averages" for completion percentage, touchdown percentage, etc? That's a better, though still obviously imperfect, statistic, no?

13
by Red (not verified) :: Thu, 01/27/2011 - 10:56pm

Over at P-F-R, they have Passer Rating Index, which is normalized to league average. A score of 100 is average, and 15 points equal one standard deviation. Here are the career leaders (min 1500 attempts):

Otto Graham 127
Steve Young 126
Joe Montana 123
Sammy Baugh 122
Roger Staubach 121
Len Dawson 120
Sid Luckman 120
Tom Brady 118
Peyton Manning 118
Philip Rivers 118
Aaron Rodgers 118
Kurt Warner 117

So you're right, it's still a flawed stat, but it looks much better when adjusted for era.

30
by Aaron Brooks' Good Twin (not verified) :: Fri, 01/28/2011 - 12:43pm

But that only lets you compare against players within their own era.

The argument about best all-time does involve elements of how the QB position has objectively become more functional over time. Part of that is rule changes, but part of it is just 90 years' worth of experience of figuring out the position, and offenses starting to maximize the abilities of QBs.

If you only sort positions by comparison to their own era, than all of a sudden John Kuhn becomes favorably compared to Jim Brown. Both were FBs, right? But FBs were a *lot* better in the 1960s.

35
by Danish Denver-Fan :: Fri, 01/28/2011 - 5:19pm

I think that's the point of it all. I'd always approach a comparison of, say Unitas and Manning by measuring how much better was Unitas than his contemporaries vs. how much better Manning was/is than his contemporaries. I think it's the right way to think about it, and it's the only way I know how.

Comparing "raw" passer ratings from '73 and '03 makes very little sense - I don't know what a comparison like that would tell me.

Your Kuhn-Brown example has nothing to do with this. If you wanted to compare those two, you'd have to compare Kuhn to other ball-carriers not FB's - I doubt that Brown spend much of his time blocking for anyone. By the way; I'm absolutely certain that when we talk about Jim Brown the HOF ballcarrier, we are comparing him to other ballcarriers, ie RB's.

44
by RickD :: Sat, 01/29/2011 - 12:00am

I don't see why Unitas is a better passer than Manning just because Brady played at the same time as Manning.
QBs are asked to do a lot more today than they were in the 1960s. Everything is different - players are bigger, stronger and faster; offenses and defenses are more complex; the season is longer; there are more teams, etc.
If you put Otto Graham's bare statistics next to Peyton Manning's, the latter's statistical advantage would be obvious. It's really not close. Except for yards/attempt, Manning has a huge advantage in every stat ordinarily considered to be important for a QB.

60
by Jeff (not verified) :: Sun, 01/30/2011 - 2:46pm

Yes, but offensive lineman can hold more than they used to, defensive lineman can't held slap, quarterbacks are protected from late hits more than they use to be, defensive backs can't mug receivers down the field, etc. All of these things are to the advantage of the modern QB. Which is why using raw stats to compare across eras is a mistake. Adjusting for how well the QB did in his era, while not perfect, is much better than just using raw data.

2
by tally :: Thu, 01/27/2011 - 4:53pm

I don't think it really helps because passer rating was flawed from the get-go. Its rating system is closer to fantasy football scoring than DVOA. Pass completions for negative yards are positive plays. Even era adjustments won't solve those issues.

5
by Arkaein :: Thu, 01/27/2011 - 5:59pm

Passer rating could be improved a fair amount pretty easily by decreasing the importance of TDs, increasing the importance of INTs, making yards per attempt more important than completions percentage, and by incorporating sacks as incomplete passes that lose yards.

Other improvements could involve incorporating percentage of passing attempts that gain first downs.

Of course, do all this and you probably end up with something like PFR's ANY/A (Adjusted Net Yards per Attempt): much better than the current passer rating, but still not great for comparing QBs from different eras.

9
by Dan :: Thu, 01/27/2011 - 6:56pm

The top 10 quarterbacks by ANY/A (min. 1000 attempts) are:

1. Philip Rivers 7.28
2. Tony Romo 7.15
3. Aaron Rodgers 7.04
4. Peyton Manning 7.04
5. Steve Young 6.85
6. Tom Brady 6.82
7. Matt Schaub 6.72
8. Kurt Warner 6.71
9. Joe Montana 6.60
10. Dan Marino 6.55

This only goes back to 1969, since we didn't have sack data before then. If you use Adjusted Yards per Attempt, Otto Graham is the only pre-1969 QB who joins the elite, and Rodgers moves into #1:

1. Aaron Rodgers 8.08
2. Philip Rivers 8.05
3. Otto Graham 7.99
4. Steve Young 7.94
5. Tony Romo 7.84
6. Ben Roethlisberger 7.68
7. Kurt Warner 7.55
8. Tom Brady 7.50
9. Peyton Manning 7.48
10. Matt Schaub 7.44

Just about any reasonable way to statistically rate quarterbacks will strongly favor recent QBs like Rodgers & Rivers unless it includes an era adjustment.

52
by Arkaein :: Sat, 01/29/2011 - 2:12pm

Era adjustments would be good. However, it might be even better to introduce defensive adjustments, which would have the side effect of improving comparisons across eras.

For example, if we start with ANY/A as the base for our better passer rating, we could also calculate ANY/A for defenses based on all passes against that defense. Then we could calculate DANY/A (defense-adjusted adjusted yards per attempt) by subtracting the defense's ANY/A for the season from the QB's ANY/A for a game. Full season DANY/A would be calculated using a weighted average of since game DANY/As (weighted by number of adjusted passing attempts in each game).

We could get even fancier by calculating second order effects, but it would still be a pretty good formula just with first order comparisons, since most defenses over the course of a season will face a broad range of QBs that will most average out.

This rating would be positive for QBs that are above average, and negative for QBs below average. By removing the baseline YPA through cancellation this rating would also be better for comparing QBs across eras, although there is still a good chance that certain eras with have greater variance than others.

If we wanted to make it look more like YPA (for familiarity's sake) for comparing QBs within an era we could just add the average YPA for that year back into the formula.

3
by theslothook :: Thu, 01/27/2011 - 5:36pm

Well said

4
by Vicious Chicken Of Bristol (not verified) :: Thu, 01/27/2011 - 5:43pm

"As quarterbacks age, the law of averages starts to temper their statistics, which is why Brady, Manning, Roethlisberger and Brees have slipped below the newcomers. Brady and Manning will probably wind up like Young and Montana, hovering at the top of the list forever"

And we can presume that Ben's will wither away? Does the fact that he has had 5 seasons with passer ratings above his current career average (vs only 2 below) not imply that he would also stay high on the list?

6
by Chappy (not verified) :: Thu, 01/27/2011 - 6:24pm

Well, a statistician would quibble with this statement. The law of large numbers is what is at play here. The law of averages is sometimes called the gamblers fallacy because the gambler thinks that deviations will be 'evened out' in the future. I submit that Mr. Tanier has no idea whether Mr. Rodgers is due for a decline (or any other player for that matter). He only can assume that a players 'true' perfomance will be reached as they accumulate more seasons. Alternatively, maybe passer rating declines with age, but, in that case, this has nothing to do with the law of large numbers.

11
by Whatev :: Thu, 01/27/2011 - 8:29pm

You're looking at a leaderboard. It's only natural that biases in measurement would tend to be positive.

18
by Vicious Chicken Of Bristol (not verified) :: Fri, 01/28/2011 - 9:28am

I understand what you are saying. My issue is that Tanier makes a statement that cannot be supporting by anything. It is just his personal bias.

Ben's career PR is 92.5. 5 out of his 7 seasons produced passer ratings higher than that, 2 lower.

I am not discounting it but one of the low seasons was the infamous "bounce my head off a buick/appendectomy/multiple concussion" season. The other was just a substandard season for him passing wise.

So my argument is, if over 70% of the time (or 80% if you ignore 2006, which I dont) Ben ends the season with a passer rating of 97 or higher....what reason is there to think that his career passer rating of 92 will decrease?

On the flip side, Brady is the opposite. He has a current career passer rating of 95.2, 6 of his 9 seasons (I dont include his rookie year, or his injury year) as a starter resulted in PR's lower than that.

In Ben's case the statistical outliers bring his overall rating down, in Brady's case they bring it up.

Yes it is a small sample size to determine a definate trend but Tanier (in his statement) seems to be doing just that, only backwards.

31
by Mike Tanier :: Fri, 01/28/2011 - 1:17pm

True, the Law of Large Numbers is what I should have said, not "Law of Averages"

Roethlisberger's career passing rating will probably decline because he is likely to play past his prime. I was not going for the point that Roethlisberger or Rodgers were going to slip in 2011 or 2012. I meant that eventually, they will have a few "old quarterback" seasons on their resumes.

32
by Vicious Chicken Of Bristol (not verified) :: Fri, 01/28/2011 - 2:16pm

And you assume that Brady and Manning wont? The statement still doesnt hold water.

Brady has mentioned wanting to play until 40 or so hasnt he? Are we to assume that he is so good that not even father time can touch him?

42
by Intropy :: Fri, 01/28/2011 - 10:09pm

No, he's not saying that because the article isn't about comparing Ben, Brady, and Roethlisberger. It's about comparing across eras. The only part that could be construed that way is "As quarterbacks age, the law of averages starts to temper their statistics, which is why Brady, Manning, Roethlisberger and Brees have slipped below the newcomers. Brady and Manning will probably wind up like Young and Montana, hovering at the top of the list forever, but always wedged among a bunch of young guns coming off two or three hot seasons." But I think it's uncharitable to take that view because it's just trying to make a point about there always being "young guns" on the list, not trying to discriminate between {Manning, Brady} and {Roethlisberger, Brees}.

46
by RickD :: Sat, 01/29/2011 - 12:31am

John Elway had a 16-year career. I'll let you take the first 8 years while I take the last 8.

I win.

You are making assumptions about the play of older QBs that are not necessarily supported in all cases.

51
by Anonymous1 (not verified) :: Sat, 01/29/2011 - 1:05pm

So you're saying QBs never see a decline in play? They'll play just as good in their 15th year as their 4th?

54
by RickD :: Sat, 01/29/2011 - 6:20pm

No, I'm not saying "QBs never see a decline in play". I was rejecting the opposite argument, that "QBs always see a decline in play".

Taken any courses in logic?

61
by An Onimous (not verified) :: Mon, 01/31/2011 - 2:57am

I've taken enough logic courses to know that the argument you're rejecting is a straw man. Mike Tanier never said "QBs always see a decline in play". Taken any courses in reading comprehension? Words like "probably" and "likely" are pretty strong indicators that the author is not making any absolute claims.

With that said, there is an incredibly high likelihood that a QB's last season will be below average. Why? Just look at Brett Favre- if a QB has an above-average season, it's unlikely that it will be his final season. He'll think he still has something left and come back for another go-round. Is it a guarantee? No, there are always exceptions to the rule (John Elway is the most notable, as he remains the only player to earn SBMVP in his final NFL game). Still, if we assume that every single future season has an equal chance of being above average as it does of being below average (which seems like a solid assumption to me, given the definition of "average"), except a QB's last season has a greater chance of being below average (given the phenomenon I mentioned where QBs coming off of above average seasons are likely to continue playing), then the logical assumption should be that every single established, veteran QB in the league is more likely to post a below-average QB rating over his remaining seasons than an above-average QB rating over his remaining seasons.

45
by RickD :: Sat, 01/29/2011 - 12:21am

There is little reason to think that Brady (or Manning, or indeed any elite QB) will slip because of "the law of averages". Brady's best two seasons have been among his last three (recall he didn't play in 2008). Favre's had two of his best seasons at the ages of 38 and 40.
This kind of abuse of statistical language is really annoying. The "law of averages" (or, more properly, the law of large numbers) is based on a presupposition of what the average is supposed to be. Tanier uses it to mean that when a QB outperforms the rest of the league, it should be viewed as a statistical anomaly, and not as a sign of superior play.
This is a curious way to view the situation.

47
by Jerry :: Sat, 01/29/2011 - 1:05am

No, Mike clarifies his point in #31: These guys will all get to hang around until they start putting up bad seasons, which will drag down their career numbers.

7
by QQ (not verified) :: Thu, 01/27/2011 - 6:45pm

"How the heck does he outrank Brady, Manning, and Brees?

As you know, quarterbacks usually play poorly as rookies, improve until they peak for a few seasons, then (if they are very good) hang around for a few seasons as their performance and statistics decline. A quarterback’s lifetime passer rating accounts for all three phases of his career: rise, peak, fall.

Now, look at Rodgers’ career. It’s all peak"

Brady is a poor example in this argument. Brady did not sit 3 years like Rodgers did but did indeed sit for 1 year. Additionally, even if you count their careers from simply Year 4 onwards, Rodgers comes out ahead.

Here are the Passer Ratings for Brady and Rodgers for years 4-6 in the league:

Brady 85.9 Rodgers 93.8
92.6 103.2
92.3 101.2

8
by JoeHova :: Thu, 01/27/2011 - 6:47pm

Any "average" stat will invariably have a few current players on the leaders list who won't end up among the leaders when their career is over because of their (yet to occur) decline phase. I don't see why that would be considered a problem.

10
by DenverCheeze (not verified) :: Thu, 01/27/2011 - 7:09pm

Stats are stats...and like in marketing class can be used to make a case for whatever you want. Can't we rate their career rating based on the average passer rating of all the starting QBs in that year (removing the bottom 10% just because some 3rd stringer played badly in his golden opportunity may be useful). That way we can see who was the best in a given era...and rank them accordingly somehow based on that?

12
by Ezra Johnson :: Thu, 01/27/2011 - 8:47pm

I don't really agree with the notion that you have to discount Rodgers (or anyone else) because he didn't play his first 3 years. He did what he did.

But all this and other similar arguments really lead to are that *any* individual statistic (including DVOA) in football is so highly context-dependent as to be almost entirely unreliable. Why should the passer rating be singled out? Take the James Starks piece from the other day. If DVOA were truly a reliable measure of player performance, the argument would be that if you had replaced Starks on the roster with Jamaal Charles and dropped him into exacly the same situation, the results would be substantially different. While anyone would agree that Jamaal Charles is by far a better RB than James Starks, no one in his right mind would say it because DVOA told him so.

14
by Overrated (not verified) :: Fri, 01/28/2011 - 12:05am

Personally, I disagree with the idea that Aaron Rodgers isn't a better quarterback than Roger Staubach. Staubach might be more impressive, but better? I doubt it.

Normalizing to eras puts things in a context, but it overlooks the evolution and progression of the game. Jim Brown was amazing. Would he be amazing today? I dunno, he's probably too slow. Players today are more specialized, and with that specialization they've become, frankly, better at what they do.

But I'm just not a traditionalist at all, and while I don't really have a problem romanticizing the innovators and amazing performers of yesteryear, I'm not at all sold that they were empirically, physically better than today's players.

15
by AnonymousOverrated (not verified) :: Fri, 01/28/2011 - 2:25am

Good to know.

/never listening to anything that guy says again.

16
by Red (not verified) :: Fri, 01/28/2011 - 2:52am

I'm glad someone brought this up. Passing statistics may be biased in favor of modern players, but I would agree that today's quarterbacks really are better than the guys from yesteryear. Here's why:

1) Echoing you said, modern players are physically superior (bigger, faster, more agile, etc), and advancements in coaching and tactics have allowed players to utilize their unique abilities better than they did in the past.

2) Depth of competition. Modern QB's play in a 32 team league in which any team can beat any other team. On the contrary, guys like Unitas played in a 12 team league which was usually watered down by a few doormats. Being a top 5 QB in 2010 is a lot more impressive than being a top 5 QB in 1950.

3) The modern passing game comprises a much higher % of a team's offense than it did back in the day, so I would argue that QB's now are more important to their team's success than at any point in history. Assuming this is true, shouldn't they get more credit/blame as well? Sid Luckman's 8.42 YPA looks impressive until you see that he only attempted 1744 passes in his career, with only one season over 300. I'm almost certain his efficiency would've dropped if he had to throw 500+ passes every year.

17
by Jerry :: Fri, 01/28/2011 - 8:35am

Over the years the game has changed significantly, and the business around the game has changed significantly. Put a modern passer in Otto Graham's NFL, where offensive linemen couldn't extend their arms and defensive backs could push receivers all over the field, and his numbers won't look nearly as good as they do now.

My guess is that great players are born with enough talent that they would be great in any era. Sammy Baugh and Sid Luckman would be stars now; Tom Brady and Peyton Manning would have been stars then. The large salaries and accompanying ability to focus on football twelve months a year mean that the general level of football today is higher than it used to be, but we're talking about the guys who significantly exceed that level in any era.

22
by Vicious Chicken Of Bristol (not verified) :: Fri, 01/28/2011 - 10:48am

I somewhat disagree. Brady's success is in a timing based offense in which he faces very little pressure.

If he played "back in the day" when QB's were thrown around like ragdolls, WR's got mugged on every play, and they actually called holding on the OL...Brady would not be nearly the player he is in today's game.

I dont think many players span generations like you say. Certain skill sets are perfect for certain styles of play. Brady's skill set is ideal today, but he would probably have been killed in the 70's.

28
by Vicious Chicken Of Bristol (not verified) :: Fri, 01/28/2011 - 11:53am

Didnt mean to single out Brady, Manning (and many of the QB's today) would likely die as well.

19
by Vicious Chicken Of Bristol (not verified) :: Fri, 01/28/2011 - 9:36am

While what you say is true, I think the real reason passers today get better stats is because the NFL wants them to. The rules consistantly change to benefit the passing game.

You get penalized and fined for even looking at a QB in a menacing way.

You cant touch receivers before the ball gets there, you cant hit them too hard after.

Offensive holding is by and large ignored (as it happens on every frickin play).

So the NFL has methodically cripples a defense's ability to stop a passing game, because that is what they think fans want to see (which is largely true in most cases).

43
by Intropy :: Fri, 01/28/2011 - 10:11pm

"You get penalized and fined for even looking at a QB in a menacing way."

I think officially looking at a QB "in a menacing way" is legal, but "with malice in your heart" is a personal foul.

50
by jebmak :: Sat, 01/29/2011 - 8:42am

+1

20
by Dingle-Doodah (not verified) :: Fri, 01/28/2011 - 10:19am

Here's a thought: What's the point? Why a PR at all?

I you allow that a MLB pitcher and NFL QB are equally as valuable to their respective games, and that MLB is moving from (more like, being forced from) its archaic method of judging pitchers by W and L, and without that, pitchers will be judged by ERA, WHIP and K (until more complex stats like xFIP become mainstream), what's the point?

Judge a QB by Y, Y/A, TD and INT, independently. It works for baseball, so it's not like fans wouldn't be able to discern the best QB's without a silly stat combining all those qualities into a pretty package.

Hey, PR (pitcher rating)! = (ERA (to some degree)x(WHIP(to some degree)x(K(to some degree)x(BB(to some degree))/(some degree).

Pretty ridiculous. If it's broke, trash it.

21
by Vicious Chicken Of Bristol (not verified) :: Fri, 01/28/2011 - 10:41am

First, it isnt really "broke". It is fine for what it is, it compares players evenly (in any given short time span) and it is not subjective.

Which brings me to your baseball analogy. There is no real comparison because almost all of a pitcher's stats ARE subjective.

ERA is determined by whether or not someone thinks a fielder SHOULD have made a play on a ball (earned vs unearned run).

Balls and strikes are judged by umps that replays show are dead wrong about 50% of the time, and are vastly different from person to person.

WHIP can, same as above, be controlled by whether someone thinks a fielder SHOULD have made a play on a ball (error vs hit).

The only thing that isnt really subjective in terms of pitchers stats is Wins and home runs. Not saying those are the best things to judge, just that it is the nature of the game.

Your suggestion of Y, YPA, TD, INT is fine for people who have a clear understanding of the relative importance of each number, but for those who dont...there is passer rating.

And like I said, it may not be perfect...but it isnt subjective. It places an arbitrary cap, but at least there is a finite range.

That is why the NFL will never adopt something like DVOA...not only would the average fan have no idea what it means, but any secret formula that assigns certain levels of success to any given play is inherrently subjective.

23
by Dingle-Doodah (not verified) :: Fri, 01/28/2011 - 10:51am

What I did there is relate how I look back at a particular year to judge the best MLB pitcher. If we consider sortable stat columns, I think 'we' don't just open ESPN, for example, and accept that Hernandez was 2010's best pitcher because ESPN's default sort is 'ERA' and he's at the top of the list. We accept that he had the best ERA and then sort for WHIP, IP, K, et el. because we know that ERA doesn't tell the whole story.

Same could be applied to football. Fans aren't idiots that need all those stats combined into some pretty package.

I don't at all agree with you that pitcher stats are all subjective while QB are not. If we accept that INT are random events, how are they any different than BB, except in quantity? Y/A are subjective because it's somewhat dependent on who's catching the football. TD are subjective because if you're Vick, for example, your team has the damnedest time scoring in the red zone.

25
by Vicious Chicken Of Bristol (not verified) :: Fri, 01/28/2011 - 11:40am

"I don't at all agree with you that pitcher stats are all subjective while QB are not. If we accept that INT are random events, how are they any different than BB, except in quantity? Y/A are subjective because it's somewhat dependent on who's catching the football."

That isnt the point I was making at all. Randomness has nothing to do with subjectivity. Subjective, random, and dependant are all getting mixed up in your statement above.

Like I said, an error vs a hit, a strike vs a ball, are all based on one person's opinion. One ump could rule on a play differently than another one. It doesnt matter what actually happened, it only matters what the ump thinks happened. That is subjective.

In football, an INT is an INT. A TD is a TD. etc. Sure a QB's stats are dependant on the team around him, and INT's can be random events, but they are not (blown instant replays aside) subjective.

58
by matt w (not verified) :: Sun, 01/30/2011 - 9:05am

I think it's confusing the issue to say that balls and strikes are subjective, because the umpire calls them, but wins and losses are objective. Wins and losses are also affected by the umpires' calls. They're both equally subjective or equally objective. (Actually there's a subjective element in W-L; if the starting pitcher leaves after less than five innings with a lead that's never given up, then I believe the win is awarded to one of the relievers at the scorer's discretion. But that's a minor aspect.)

Anyway, even if W-L were objective in some way that other stats weren't, it'd still be a bad stat to judge pitchers by. Height/weight and pitch speed are objective too (of course at least some of those aren't measured very accurately), but what would be the point of judging pitchers by those stats alone?

26
by GlennW :: Fri, 01/28/2011 - 11:48am

"We accept that he had the best ERA and then sort for WHIP, IP, K, et el. because we know that ERA doesn't tell the whole story."

The problem with this argument is that ERA as a rate statistic tells you most of what you need to know about a starting pitcher within the team context (that context being the quality of team defense, home ballpark etc., factors which still aren't nearly as significant as the team-dependent factors present in football). Yes, there are some luck factors around the "earned" run judgment and distribution of earned runs allowed (i.e. where ERA is a bit out of whack relative to WHIP over a single season), but for the most part ballpark-adjusted ERA is a damned good measure of starting pitcher performance, even for a single season (and much better over multiple seasons). But there is no analogue to ERA as a measure of passing in a single conventional statistic, hence the "need" for a formulated passer rating. (I agree that this "need" is subjective-- yes, we can look at several conventional statistics and gain a pretty good idea of a QB's overall performance-- but having a single well-formulated statistic such as DVOA is still not a bad thing.)

24
by Andy Watkins (not verified) :: Fri, 01/28/2011 - 11:24am

I tend to agree, but for this reason specifically: I don't think a single omnibus statistic can be especially meaningful.

Let's say that we find the perfect set of factors on which to judge a quarterback. Average net yards per attempt, touchdown percentage, hair, whatever. I believe that the next step--where we weight them and apply some mathematical operation to combine them--can get tricky.

Take the discussion upthread where people are saying "well, PR would be better if it placed less weight on TDs and more on INTs and..." That's all well and good. That new weighting is what applies to the average NFL team today, and the old weighting probably applied better to the average NFL team of 1970.

But neither will an Average 2011 Weighting apply in twenty years nor will it apply today equally well to every NFL team. Different teams require different sets of quarterback attributes. No team would swap a quarterback with a higher completion percentage for one with a lower completion percentage, ceteris paribus, but ceteris is rarely paribus. There are NFL teams that--perhaps because of their offensive scheme, perhaps because of their other attributes, in particular the strength of their defense and special teams--need wholly different abilities from their quarterbacks. Get a strong enough defense, and your need for a high TD percentage declines. (Give an offensive coordinator a strong running back, and your opportunities for a high TD percentage decline.) There's also the question of variance: a quarterback whose ANY/A is created from an eighty yard bomb every ten attempts, the others of which are all completions for one yard, has a great passer rating according to any weighting but isn't very useful.

The real question is: does attempting to create an omnibus stat, with all the assumptions that go into weighting, obscure more insights than it creates? I tend to think that the shear number of assumptions involved do obscure too much, but I don't have nearly enough data to be certain. (Nor can we, as even the venerable Mr. Tanier has to evaluate PR by comparing it to the gut assumption "no way Aaron Rogers is better than Roger Staubach! No way!")

27
by Vicious Chicken Of Bristol (not verified) :: Fri, 01/28/2011 - 11:49am

Excellent post. Which brings me back to a statement that has gotten me ridiculed many times on this site...no stat compares to the eyeball test.

33
by Turin :: Fri, 01/28/2011 - 3:08pm

"There's also the question of variance: a quarterback whose ANY/A is created from an eighty yard bomb every ten attempts, the others of which are all completions for one yard, has a great passer rating according to any weighting but isn't very useful."

I'm pretty sure every team in the league would love a quarterback who throws a touchdown approximately once per 10 attempts, since even the best QB's ever aren't much above 1 per 20 attempts. :)

29
by erniecohen :: Fri, 01/28/2011 - 11:55am

The biggest statistical injustice of NFL history is that if Randle-El had, over his career, passed for 5 more yards but 2 fewer touchdowns and one fewer completion, he would have a perfect passer rating.

40
by Jerry :: Fri, 01/28/2011 - 7:31pm

Which is why it should be referred to as "maximum" passer rating, not "perfect."

34
by Kal :: Fri, 01/28/2011 - 5:04pm

I still think the biggest flaw in passer rating as a measure of a quarterback is that it ignores:

running yards
sack losses (both as an incompletion and as an actual lost yard)
fumbles

And basically anything that isn't recorded as a pass. When you start counting how often a QB fumbles or gets sacked, people like Ben go significantly down. When you start counting first down gains from runs, people like Rodgers (and Young for that matter) go up.

36
by Danish Denver-Fan :: Fri, 01/28/2011 - 5:28pm

Sure - but thats why it's called a *passer* rating. Sacks should still be included of course.

Now a qb-rating should account for all of the above.

39
by Kal :: Fri, 01/28/2011 - 6:52pm

I'd agree with you if QBs didn't do that much other than pass. But that's just not true. Having things like a RB rated on their rushing yards and YPC makes sense, as does their total overall yards from scrimmage. Does it matter that we don't factor in the halfback pass? it's less accurate but probably doesn't matter.

But sacks are a huge deal to a QB's success. Fumbles are too. Running yardage can be depending on the QB. It's not a 'passing' rating, it's a passer rating - meaning rating the passer.

Just seems myopic. Especially given that the passer rating is already a fairly meh metric.

56
by Vicious Chicken Of Bristol (not verified) :: Sat, 01/29/2011 - 11:20pm

"When you start counting how often a QB fumbles or gets sacked, people like Ben go significantly down. When you start counting first down gains from runs, people like Rodgers (and Young for that matter) go up."

You do realize the Rogers gets sacked a lot. 81 sacks and 14 fumbles the last two years. Almost the exact same as Ben (82,14).

63
by Spielman :: Mon, 01/31/2011 - 7:56am

Yes, but that's compared to 1016 pass attempts for Rodgers, and 895 for Roethlisberger. Ben really does get sacked more than Aaron. Then again, he gets sacked more than just about anyone.

Of course, that also points out how very little Roethlisber fumbles, considering how much he's sacked. There seems to be a pretty strong correlation between how often QBs fumble, and how often they are sacked, for obvious reasons. Ben gets sacked at a very, very high rate, but doesn't fumble much, considering. Again, the reasons for that are kind of obvious, but it's interesting to see how they show up in the stats.

65
by Vicious Chicken Of Bristol (not verified) :: Mon, 01/31/2011 - 5:59pm

True, but I'd say that for every time Ben gets sacked from holding the ball too long he probably avoids 2-3 that are on the OL just missing blocks. That cant be figured into a passer rating either.

Passer rating isnt perfect, but there is no single stat that can account for everything...current site included.

64
by Joel (not verified) :: Mon, 01/31/2011 - 5:11pm

Very true that running and sack yards are ignored and should be included...but shouldn't when they happen be referenced as well? Which is why DVOA is so nice in that it includes context to the analysis. Ben's sacks are above average normally, but much higher on 3rd downs where he extends plays longer than other QBs trying to "make a play".

Most would agree that his penchant for that outweighs the higher number of sacks that he takes during those situations. Not that sacks aren't bad, but some sacks are less bad than others.

Ultimately, passer rating has many other flaws. Is Completion % a good picture? Well a little bit, but if its dink and dunk passing then its worth less than longer passing. First downs really aren't weighted in this metric...success isn't really measured...passer rating merely takes a series of somewhat important rate stats and tries to conglomerate them to be greater than the whole.

Which is does...its just that it's Batting average waiting to be overtaken by OBP or WOBA.

37
by Jim Glass (not verified) :: Fri, 01/28/2011 - 5:56pm

Adjusting passer rating for era does not adjust for the bias and absurdity of the rating itself.

It is so biased towards completion pct that you get a rating of 79.2 for completing a pass that *loses* yards in *any* amount -- and remember, it was created so that the average passser was rated 50!! That's how bad it was in 1970. Before 1978, that's what you are ajusting by era. (Since the passing rules changes of 1978 completion pcts have inflated so that in 2010 only 25% of starting QBs could have increased their ratings by throwing more passes that *lost* yards. Only.)

Of course, in all eras this bias towards completion % hugely penalizes the long-throwers (Unitas, Namath, etc.) and boosts the short throwers (Starr, Dawson, etc.) "Adjusting for era" leaves that bias fully in place.

And it's why in the NFL Hall of Fame's official all-time top 20 passer rating list *all* the QBs played within the last 16 years.

Think about that. If MLB and its Hall of Fame published an official all-time top 20 batter rating list that included only players from the last 16 years, putting the baseball equivalent of Duante Culpepper at #9 all time, but leaving off everyone from Ty Cobb to Babe Ruth, Ted Williams, George Brett, etc, those responsible for it would be ridiculed into early retirement and enshrined in a hall of shame.

But football fans take it in stride. Different sports have different fan cultures.

38
by GlennW :: Fri, 01/28/2011 - 6:21pm

> But football fans take it in stride. Different sports have different fan cultures.

Yep. How often do you hear football fans (and not just casual ones), announcers and even sportswriters talking about passer rating, especially in a historical context? Rarely. I think there's an unconscious understanding that the rating is convoluted and flawed, and thus has been rejected from mainstream discussion.

Fortunately in baseball batting average now gets the same treatment-- it could be argued that this was a statistic that was paramount 20-30 years ago but was likewise fundamentally biased towards the old-timers, not to mention overrated in general. I don't recall much mainstream outrage towards BA though (just from the SABR culture which only gradually made its point), which just goes to show that the culture around baseball statistics isn't perfect either.

49
by Packer Pete (not verified) :: Sat, 01/29/2011 - 6:55am

Starr YPA 7.8. Unitas YPA 7.8. Dawson YPA 7.7. Namath YPA 7.4. Who are the "long" throwers and who are the "short" throwers?

53
by Temo :: Sat, 01/29/2011 - 2:19pm

Right, so they have similar YPA but one gets a higher rating because he completed more passes? One does it by consistently completing passes, the other does it with big chunks of yardage.

The argument is that completion % is overrated in the formula, and you haven't made a counter-argument against it.

62
by An Onimous (not verified) :: Mon, 01/31/2011 - 3:15am

Namath 14.7 yards per completion
Unitas 14.2 yards per completion
Starr 13.7 yards per completion
Dawson 13.4 yards per completion

The fact that Namath averaged a lower YPA than Dawson despite averaging almost a yard and a half more per completion perfectly illustrates the difference between the "long ball" and "short ball" guys.

Of course, that's all pretty era-specific, too. For comparison purposes, Peyton Manning averages 11.7 yards per completion, while Tom Brady averages 11.6. Ben Roethlisberger and Tony Romo are probably the closest things we have to a "long ball" guy today, and their career ypc average is only 12.7 and 12.6 respectively- they get dwarfed by the old-school passers. Which is why Roethlisberger and Romo have higher passer ratings than guys like Unitas and Namath- they're averaging more completions for fewer yards, which is rewarded by a passer rating formula that double-counts completion percentage (counting it once directly as its own category, and then counting it a second time in YPA, which is essentially a derivative stat based on yards per completion times completion percentage).

Personally, I don't even see the need for a passer rating in the first place. Yards per attempt is really the only quick off-the-cuff statistic needed to compare QBs across offenses, eras, or anything else for that matter. It's remained surprisingly static through all the years and all the offensive changes to the game. Plus, it has the benefit of being incredibly easy to calculate and understand.

41
by Bright Blue Shorts :: Fri, 01/28/2011 - 7:55pm

Coincidentally I happened to be reading Roger Staubach's Wikipedia entry yesterday ... apparently when he retired in 1979-80ish he was the all-time leader in NFL passer rating at 83.6 ... although I think that partly depends on whether you include Otto Graham's AAFC years ...

48
by Mike Elseroad (not verified) :: Sat, 01/29/2011 - 3:55am

That was a nice article. Dr Z, Sports Illustrated's legendary NFL writer Paul Zimmerman, has been ranting and raving against the NFL's passer rating formula for years.

Mike just put a nice spin on it while explaining it's flaws in a nice coherant way.

55
by ppabich :: Sat, 01/29/2011 - 8:39pm

Wow, i'm getting really tired of comparing quarterbacks of the same or different eras. I think we have all figured out that QB rating is a terrible way to measure greatness. All QB rating does is give us a number on the efficiency of a quarterback. Systems and QBs themselves have been better in recent times rather than the past. Does that mean that Rodgers is better than Unitas? No, but I you can safely say Rodgers has been a more efficient QB.

Even in baseball numbers don't translate between eras (ballparks, mound height, quality of pitches/hitters). Football is even worse when it comes to comparing players. What if Manning had Billichek? What if Brady had great receivers his whole career? Rodgers is blessed with great receivers and a great offensive scheme, he was also able to sit behind a legend for 3 years, learn the game and start when he was reaching his prime. Manning started as a rookie and has an outlier passer rating from that year. So of course passer rating over a given career is a pointless stat, but it does give good insight to who was good in a given year.

The who is the greatest player in NFL history argument is so incredibly subjective that it is pointless. The only way we can gauge how great a football player is, is to factor is impact on the game. Unitas, was the first truly great pass first QBs. Manning/Marino put up stellar numbers. Brady/Montana won a lot of super bowls. All of them are important in different ways, it is purely subjective to decide what factors/players are more important and therefore better.

57
by Jim Glass (not verified) :: Sun, 01/30/2011 - 4:32am

"The passer rating did a fine job selecting the best quarterback in the NFL in the context of the 1970s..."

No, it was even *worse* then than it is now. The biggest of several problems with it is it so over-values completions that passes that *lose* yards (in any amount!) get a rating of 79.2. Yet back in 1970 it was designed to set the average rating at 50! In 1971 the league average rating was 59.3 -- a 79 rating was Pro Bowl QB quality. All but four QBs in the league could have boosted their ratings (most by *a lot*) by dumping off a lot of passes for losses, the more the better!

This is just like a baseball rating system that rewards batters for striking out, or rewards pitchers for giving up hits -- would that be good for the 1970s?

But with the big rise average completion pct since then, only about a quarter of today's QBs would increase their ratings by intentionally throwing passes for losses. (Only.)

This huge over-valuation of completion %, combined with the steady rise in completion rates with rule changes, is the reason why the NFL Hall of Fame official list of Top 20 passers says they all played within the last 16 years, Brian (not Bob) Griese is rated so far ahead of John Unitas, and the latest hot QB of the moment is always the #1 of all time. At this moment, Rodgers. And yes, back in 1970 it gave a huge bonus to Bob Griese at the cost of John Unitas and the other long-throwers.

As for the other problems with the passing rating formula, Pete Palmer noted in the 1980s that it reduces to...

"yards per attempt with a bonus of 20 yards for each completion, an additional 80 yards for each touchdown, and a 100-yard penalty for each interception. So two completions for ten yards each are worth the same as one completion for forty yards and one incompletion. A ninety-yard pass play from goal line to the opponent ten is worth the same as a ten yard TD pass..."

Does that formula make sense for *any* era?

59
by Raiderjoe :: Sun, 01/30/2011 - 9:18am

passer rsating stat very f;awed. really nkot good to use at all

66
by Anonymous Beef (not verified) :: Tue, 02/01/2011 - 12:17pm

My beef with Tanier is it appears he summarily dismisses the possibility that Rodgers is better than Manning, Brady, Unitas.....

Honestly, Rodgers just might be better than all of these guys. The NFL is always very late to acknowledge facts. You almost have to win a SB before anyone notices you. Rodgers is clearly the best QB this year and perhaps last as well. Yet the league, fans and media won't see it that way until he wins a SB. This Packers team has an unusually high number of superstars and most are just starting. The Steelers have 3 super stars, the Packers have at least 7.

We could be witnessing the start of an incredible 5 year run by Rodgers that will crush anything Manning ever did and approach Montana and Brady.

Brady plays in a Texas Tech gimmick offense. Nobody wants to admit this, but it is true. 2007 was the first time ever an NFL team took >50% of snaps out of shotgun, this was also the best offense in NFL history, statistically.

It is clear to anyone with eyes Rodgers has incredible physical tools in the passing game, only approached by Marino....instant release and cannon.

Tanier will be writing a column someday arguing how Rodgers is the best QB in NFL history.

67
by Jerry :: Tue, 02/01/2011 - 7:44pm

And maybe Colt McCoy will be better than Aaron Rodgers. We'll see.

You definitely want to wait a decade before you compare Rodgers' career to guys who were great for ten years or more. It's possible that Rodgers will be considered the greatest ever at that point. It's also possible, if less likely, that 2010 will be considered his one great year. It's certainly too early to draw any conclusions about Rodgers' career.

68
by Michael LaRocca (not verified) :: Wed, 02/02/2011 - 3:02am

Aaron Rodgers is like Brett Favre without the interceptions. And the photos. I hope he's having fun out there.

69
by Anonymous Beef (not verified) :: Mon, 02/07/2011 - 12:57pm

Now that Rodgers has his SB....the NFL can now discuss his greatness without apologizing.

I have eyes. Rodgers has more physical talent than Brady by a mile.

It appears Rodgers mental preparation, execution are closing in on Tom Brady now.

Rodgers will play in at least 2 more Super Bowls, I have a hunch that number will be 5 more.

Ted Thompson is an utter genius at finding super star talent..

70
by Bright Blue Shorts :: Thu, 02/10/2011 - 3:43pm

I don't disagree with many of your comments but to suggest that Ted Thompson is an utter genius for drafting Rodgers entirely forgets the situation.

SF have their choice of 2QBs with the #1 pick ... both are rated about the same ... they choose Alex Smith.

Rodgers freefalls down the draft, team after team not picking him; because few of them have obvious QB needs.

Packers have ageing Brett Favre who has already begun to retire/unretire.

Is it that genius to use your first round draft pick on a QB who was considered worthy of the #1 overall pick?

I don't think you can even say Thompson was genius in deciding when to jettison Favre. Three seasons on the bench was enough to get Rodgers up to speed. Favre was in his 17th-ish season.