Writers of Pro Football Prospectus 2008

Most Recent FO Features

hutsonmason.jpg

» SDA: Welcome Back, Folks

The college football season returns in a big way with South Carolina-Texas A&M, Wisconsin-LSU, and Clemson-Georgia headlining opening weekend. Seventh Day Adventure is back with advanced stats, analysis, and F/+ predictions.

02 Sep 2008

What's a Perfect Game?

Guest column by Pete Palmer

The NFL passer rating has many faults, including the fact that nobody understands it. Other major problems include the 80-yard bonus for touchdowns, the 100-yard penalty for interceptions, and the 20-yard bonus for completions. A passer who completes two passes in two attempts for 0 yards gets a total of 40 yards credit, the same as another passer who completed one pass in two attempts for 20 yards. A 90-yard pass from the goal line to the opponents' 10 is worth 110 yards (90 plus the 20-yard bonus for the completion), the same as a 10-yard touchdown pass (10 plus 20, plus 80 for the score).

Of course, they didn't plan it that way, but that's the way it turned out.

The History of Rating Passers

From 1932 to 1937, passers were ranked simply by total yards passing; from 1938 to 1940, completion percentage was used.

In 1941, qualifying passers were ranked in four categories: completions; completion percentage; total yards; touchdown passes; interceptions; and interception percentage. Each passer's ranks were totaled, and the player with the lowest score won. This method was used until 1948. In 1949, the number of interceptions was dropped from the equation.

From 1950 to 1959, the NFL went simply with yards per pass.

In 1960 and 1961, they used a system similar to that used in 1941-49, but using six categories: total completions; total yards; total touchdowns; completion percentage; interception percentage; and average yards per attempt. In 1962, they used four categories, dropping total completions and total yards. That system lasted until 1972, when total touchdowns were dropped in favor of touchdown percentage.

The number of pass attempts needed to qualify for the passing title also changed through the years. In 1950 it took 100 attempts to qualify. This was raised to 10 attempts per game in 1960 and 14 attempts per game in the 1980s.

1973: The NFL Passer Rating Is Born

The current method, which debuted in 1973, was developed and tested over a three-year period by a committee headed by Don Smith of the Hall of Fame, and including Seymour Siwoff of Elias, Don Weiss of the NFL, Jan Van Duser, Curt Mosher, Tom Grimes, Bill McGrane and Jack Horrigan.

The rationale behind the new system was to calculate a rating that was independent of other passing stats and could be used to get a figure for non-qualifiers. They decided to give a value of 1.0 for average performance and 2.0 for record performance. These two points were then used to find the value for 0.0. Any value below zero would be rated at zero, so there were no negative ratings.

In addition, an absolute maximum rate of 2.375 in each category was established. Why 2.375? That's what a passer would score for interception percentage if he never threw a pick. Since that was the maximum score for interception percentage, they applied the same cap to all categories across the board.


Values of Passing Statistics Used in NFL Passer Rating
Passing Statistic Rate Needed to Score...
0.0 1.0 2.0 2.375
Completion percentage 30.0 50.0 70.0 77.5
Average gain 3.0 7.0 11.0 12.5
Touchdown percentage 0.0 5.0 10.0 11.875
Interception percentage 9.5 5.5 1.5 0.0

Since this was established in the days before computers were generally available, the league published a booklet with tables that contained the values for each item. To get the final rating, you would add up the four components and check the fifth table, which simply divided the number by six and multiplied it by 100, converting it to a percentage. Since average performance would give you a 1.0 rating in each category, the average overall rating was 66.7. The current average has increased to about 80.0 today, due to the refinement of the passing game.

A player who hit the 2.375 cap in all four categories would have a rating of 158.3, the highest score possible. Here is the complete list of the NFL's perfect games, using the current system. Rate1 is the normal NFL method, while Rate2 will be discussed later:


Perfect Games in NFL History
Year Passer Team Week Att. Comp. Yards TD INT Rate1 Rate2 Opp. Score Date
1967 John Unitas BAL 9 20 17 370 4 0 158.3 216.7 ATL 49-7 12-Nov
2007 Tom Brady NE 7 25 21 354 6 0 158.3 211.1 MIA 49-28 21-Oct
2003 Peyton Manning IND 4 25 20 314 6 0 158.3 201.1 NO 55-21 28-Sep
1995 Chris Chandler HOU 4 26 23 352 4 0 158.3 183.5 CIN 38-28 24-Sep
2007 Donovan McNabb PHI 3 26 21 381 4 0 158.3 181.7 DET 56-21 23-Sep
1999 Kurt Warner SLR 4 21 17 310 3 0 158.3 178.7 CIN 38-10 3-Oct
1974 Ken Anderson CIN 8 21 17 297 3 0 158.3 176.1 BAL 24-14 3-Nov
1980 Vince Evans CHI 14 22 18 316 3 0 158.3 175.6 GB 61-7 7-Dec
2000 Peyton Manning IND 8 20 16 268 3 0 158.3 174.6 NE 30-23 22-Oct
1963 Y.A. Tittle NYG 9 20 16 261 3 0 158.3 173.1 PHI 42-14 10-Nov
2007 Ben Roethlisberger PIT 16 20 16 261 3 0 158.3 173.1 SL 41-24 20-Dec
2000 Doug Flutie BUF 17 25 20 366 3 0 158.3 169.8 SEA 42-23 23-Dec
2002 Peyton Manning IND 10 23 18 319 3 0 158.3 168.6 PHI 35-13 10-Nov
1995 Jeff Blake CIN 8 22 18 275 3 0 158.3 167.8 PIT 27-9 19-Oct
1986 Ken O'Brien NYJ 9 32 26 431 4 0 158.3 167.6 SEA 38-7 2-Nov
2000 Kurt Warner SLR 5 30 24 390 4 0 158.3 167.4 SD 57-31 1-Oct
1994 Dave Krieg DET 13 25 20 351 3 0 158.3 167.3 BUF 35-21 24-Nov
2002 Kerry Collins NYG 16 29 23 366 4 0 158.3 166.7 IND 44-27 22-Dec
2003 Trent Green KC 15 25 20 341 3 0 158.3 165.6 DET 45-17 14-Dec
1994 Craig Erickson TB 2 24 19 313 3 0 158.3 164.1 IND 24-10 11-Sep

Problems With The System

The biggest problem with this system was mentioned earlier: Almost nobody understands it, and those who do still find it awkward to use.

Another problem with this system is that the limits for each component make it so stats above or below certain values have no effect. The limits are shown in the table above. These work fairly well for season stats, although Sid Luckman got cheated out of the all-time rating record because in 1943 his touchdown percentage was 13.9%, but he only got credit for 11.875%. His rating should have been 114.2, not 107.6, and it would have been his record that Peyton Manning broke in 2004, not Steve Young's. More importantly, the rate caps make this a poor method of measuring individual games.

With a little algebra, you can convert the rating formula into yards, with bonuses and penalties for various events. Rate A below is the adjusted yard value, while Rate B shows the conversion to a percentage.

Rate A = (Yards + Comp x 20 + TD x 80 - Int x 100) / Att
Rate B = 100/24 x Rate A + 50/24

Tom Brady scored a perfect 158.3 "true" rating for the Miami game last year, but his rating would have been 211.1 if you eliminate the arbitrary maximums. There have been 20 games with a 158.3 NFL rating in league history. If we remove rate caps, that figure has been matched or exceeded 124 times, but only Johnny Unitas in 1967 vs. Atlanta had a better figure than Brady. Atlanta was an expansion team in their second year. Under the no-limits system, Brady had three 150-plus games in 2007, tying him with Steve Young in 1993. Brady's career mark of six such game ties him at second with Young and Kurt Warner, behind Peyton Manning's seven.

The table below shows the top 20 passing games from 1950 to date, using the no-limits system. A minimum of 20 attempts was required. Again, Rate1 is the normal NFL method, while Rate2 eliminates the limits. Only five of these passers scored a maximum 158.3 rating using the NFL method:


Best Games in NFL History, Removing Limits From Current System
Year Passer Team Week Att. Comp. Yards TD INT Rate1 Rate2 Opp. Score Date
1967 John Unitas BAL 9 20 17 370 4 0 158.3 216.7 ATL 49-7 12-Nov
2007 Tom Brady NE 7 25 21 354 6 0 158.3 211.1 MIA 49-28 21-Oct
2003 Peyton Manning IND 4 25 20 314 6 0 158.3 201.1 NO 55-21 28-Sep
1981 Lynn Dickey GB 15 21 19 218 5 0 149.5 200.1 NO 35-7 13-Dec
1966 Don Meredith DAL 4 26 19 394 5 0 154.7 190.2 PHI 56-7 9-Oct
2004 Drew Brees SD 8 25 22 281 5 0 153.1 188.9 OAK 42-14 31-Oct
1999 Kurt Warner SLR 5 23 20 323 5 1 140.2 187.4 SF 42-20 10-Oct
1993 Steve Young SF 16 23 17 354 4 0 155.3 185.8 DET 55-17 19-Dec
1986 Dave Krieg SEA 15 21 15 305 4 0 153.3 185.6 SD 34-24 14-Dec
2001 Kurt Warner SLR 12 23 17 342 4 0 155.3 183.6 ATL 35-6 2-Dec
1995 Chris Chandler HOU 4 26 23 352 4 0 158.3 183.5 CIN 38-28 24-Sep
1965 John Brodie SF 1 20 14 269 4 0 152.1 183.1 CHI 52-24 19-Sep
1966 Don Meredith DAL 1 24 14 358 5 0 142.4 182.3 NYG 52-7 18-Sep
2003 Gus Frerotte MIN 4 21 16 267 4 0 157.2 182.0 SF 35-7 28-Sep
2007 Donovan McNabb PHI 3 26 21 381 4 0 158.3 181.7 DET 56-21 23-Sep
2004 Daunte Culpepper MIN 1 23 17 242 5 0 147.1 180.0 DAL 35-17 12-Sep
1992 Wade Wilson ATL 15 26 19 324 5 0 154.5 179.0 TB 35-7 13-Dec
1984 Mark Malone PIT 13 22 18 253 4 0 154.2 178.8 SD 52-24 25-Nov
1952 Norm Van Brocklin LA 8 20 11 308 4 0 139.6 178.8 CHI 40-24 16-Nov
1992 Jim Kelly BUF 4 20 15 308 3 0 156.3 178.8 NE 41-7 27-Sep

A New System

I would propose a similar system to the no-limits formula; however, the bonuses and penalties would be changed. The completion bonus would be eliminated, the touchdown bonus would be changed to 10 yards and the interception penalty reduced to 40 yards. These are consistent with the actual values of the plays. If a passer goes 90 yards downfield in 10 pass attempts, it really does not matter if he had four completions or six, or even one. The average number of points scored from a first-and-goal on the 1-yard line is about 6, not 7, so the last yard is the hardest to gain and there should be a bonus for crossing that threshold. It normally takes about 12 yards to put one point on the scoreboard, but here it is only 1, so a bonus of 10 is about right. A turnover on the spot is worth about 50 yards. However, the passer gets no credit for the length of the intercepted pass and the return is usually short, often zero.

Also, there would no arbitrary limits to any of these values.

Since the improved method emphasizes yards per pass, while the old method stresses completion percentage, there is not much crossover between the two lists above and the top 20 using the new method. Only four of those on the 158.3 list show up again, and only 10 from the no-limits list. Rate3 is the new method, expressed in adjusted yards per attempt. Johnny U still leads, but Brady drops down to No. 12. Joe Namath comes in second, thanks to his extremely high average gain, which even overcomes his interception. The players shown below have a much broader historical range than the previous lists, with almost half going back before 1980, even though there were fewer teams and fewer games. Passing then was less concerned with the short possession game.


Best Games in NFL History, By Adjusted Yards Per Attempt
Year Passer Team Week Att. Comp. Yards TD INT Rate1 Rate2 Rate3 Opp. Score Date
1967 John Unitas BAL 9 20 17 370 4 0 158.3 216.7 20.50 ATL 49-7 12-Nov
1972 Joe Namath NYJ 2 28 15 496 6 1 123.5 177.1 18.43 BAL 44-34 24-Sep
1952 Norm Van Brocklin LA 8 20 11 308 4 0 139.6 178.8 17.40 CHI 40-24 16-Nov
1993 Steve Young SF 16 23 17 354 4 0 155.3 185.8 17.13 DET 55-17 19-Dec
1966 Don Meredith DAL 4 26 19 394 5 0 154.7 190.2 17.08 PHI 56-7 9-Oct
1963 John Unitas BAL 13 22 17 344 3 0 158.1 177.1 17.00 MIN 41-10 8-Dec
1966 Don Meredith DAL 1 24 14 358 5 0 142.4 182.3 17.00 NYG 52-7 18-Sep
1992 Jim Kelly BUF 4 20 15 308 3 0 156.3 178.8 16.90 NE 41-7 27-Sep
1990 Steve DeBerg KC 13 21 15 331 2 0 145.4 159 16.71 NE 37-7 2-Dec
2003 Steve McNair TEN 6 27 18 421 3 0 146.8 159.7 16.70 HOU 38-17 12-Oct
2001 Kurt Warner SLR 12 23 17 342 4 0 155.3 183.6 16.61 ATL 35-6 2-Dec
2007 Tom Brady NE 7 25 21 354 6 0 158.3 211.1 16.56 MIA 49-28 21-Oct
1986 Dave Krieg SEA 15 21 15 305 4 0 153.3 185.6 16.43 SD 34-24 14-Dec
1991 Mark Rypien WAS 11 31 16 442 6 0 136.8 169.1 16.19 ATL 56-17 10-Nov
1999 Kurt Warner SLR 4 21 17 310 3 0 158.3 178.7 16.19 CIN 38-10 3-Oct
2007 Donovan McNabb PHI 3 26 21 381 4 0 158.3 181.7 16.19 DET 56-21 23-Sep
1968 Daryle Lamonica OAK 2 24 15 344 4 0 145.8 169.4 16.00 MIA 47-21 21-Sep
1983 Lynn Dickey GB 2 20 14 290 3 0 152.1 170.8 16.00 PIT 21-25 11-Sep
1974 Dan Fouts SD 8 21 12 333 4 1 121.5 159.4 15.86 SD 36-35 3-Nov
1978 Brian Sipe CLE 12 22 15 309 4 0 150.6 178 15.86 BAL 45-24 19-Nov

Pete Palmer is one of the original co-authors of The Hidden Game of Baseball and The Hidden Game of Football, and currently serves as one of the co-editors of the ESPN Pro Football Encyclopedia.

Posted by: Guest on 02 Sep 2008

63 comments, Last at 06 Oct 2008, 9:03am by Albert

Comments

1
by David Mazzotta (not verified) :: Tue, 09/02/2008 - 10:42am

A shame we can't get a DYAR on that Unitas game.

2
by Opponent Adjustments Made Real (not verified) :: Tue, 09/02/2008 - 10:51am

1: It wouldn't be that fantastic. His YAR might be off the charts, but the terrible opposition would make his DYAR more mundane.

3
by Joseph (not verified) :: Tue, 09/02/2008 - 11:01am

Is it just because I am used to high 50's/low 60's comp. % that I see method 3 as slightly flawed? (check the #2 & #3 games on that last table). I mean, WOW--33 yds. per completion??!!! That's unreal. (I do recognize that the passing game was different a few decades ago) However, my guess is that DVOA/DYAR would say something a little different. I do agree with #1 and this article in that the Unitas game definitely has to be the best game by a QB ever (although adjusting for defense might drag it down).

4
by Lance (not verified) :: Tue, 09/02/2008 - 11:04am

Re "If we remove rate caps, that figure has been matched or exceeded 124 times, but only Johnny Unitas in 1967 vs. Atlanta had a better figure than Brady. Atlanta was an expansion team in their second year." Why do we get a qualifier for Unitas' game? Does Brady really need more apologetics?

5
by ptfe (not verified) :: Tue, 09/02/2008 - 11:05am

FO guys: think you could post the DYAR (or DPAR, or both) for the games we do have data for? I'd be interested to see how this system compares.

6
by Rich Conley (not verified) :: Tue, 09/02/2008 - 11:05am

Removing the completion bonus breaks the system IMO. I bet a couple of those games at the top suffer from some really long broken plays, as opposed to just great QB play.

7
by Rich Conley (not verified) :: Tue, 09/02/2008 - 11:21am

"If a passer goes 90 yards downfield in 10 pass attempts, it really does not matter if he had four completions or six, or even one."

Doesn't DVOA disagree with that?

8
by Ken (not verified) :: Tue, 09/02/2008 - 11:47am

4. Surely it's just giving context, to show that Unitas achieved his game against a team that could be expected to play at a higher level than Miami last year?

9
by billsfan (not verified) :: Tue, 09/02/2008 - 11:51am

4:
Also bear in mind that Brady's performance was against a Miami team that went 1-15. Almost like you need an historically bad opponent to put up gaudy passing numbers--just look at the rest of the opponents on that table!

10
by Yaguar (not verified) :: Tue, 09/02/2008 - 11:57am

I think it is a mistake to remove completion percentage from a quarterback rating system.

Let's say that we're in the 2nd quarter of a game, and each team has had three drives. I'm not going to tell you the score, but I'll tell you the stat-lines for each QB. Say one quarterback goes 3/8 for 81 yards with a long pass of 72 yards and a TD. The other is 11/13 for 101 yards and a TD with a long pass of 21 yards.

Would you have any doubt that the latter quarterback is winning the game, probably by a score of something like 17-7? One of Aaron's favorite points is that running backs who offer consistent gains are more valuable than running backs who break long runs but get stuffed a lot.

The equivalent is true for quarterbacks. Now, I wholeheartedly believe that the current QB rating system is too completion-percentage-heavy. But let's not say for an instant that completion percentage is unnecessary in valuing a QB performance.

11
by MJK (not verified) :: Tue, 09/02/2008 - 12:00pm

For the record, the Miami game last year, when Tom Brady put up those numbers...was my wedding day. Since I'm a Patriots fan living on the west coast, it was rather nice of Tom and company to give me something entertaining and happy to watch that morning. But strangely enough, I don't really remember anything about that game...

12
by Yaguar (not verified) :: Tue, 09/02/2008 - 12:09pm

11: I remember Tom Brady coming back into the game with a 42-14 lead and throwing a deep ball to Moss, drawing a DPI.

/me ducks into a flameproof bunker.

13
by dryheat (not verified) :: Tue, 09/02/2008 - 12:29pm

I think the fact that Vince Evans, Jeff Blake, and Craig Erickson?!? have recorded perfect games is reason enough for the NFL to come up with a new rating stat, stat.

14
by steelberger1 (not verified) :: Tue, 09/02/2008 - 12:31pm

Not sure where you got your list of perfect games, but you didnt get them all. You say there have been 20, but I believe that there have been 41 (if wikipedia is correct, that is).

http://en.wikipedia.org/wiki/List_of_NFL_quarterbacks_who_have_posted_a_...

I know Roethlisberger has 3 perfect games. He got 2 last season (only QB to ever get two in one season) and one in 2005.

In his game against the Ravens last season, his uncapped rating was 228.4 (which is higher than any of the others you have listed).

15
by JJcruiser (not verified) :: Tue, 09/02/2008 - 12:42pm

Why use the old formula at all?

You said that one of the biggest knocks on the current is that almost "nobody understands it."

Your new system may be more accurate, but I still don't understand it. You're making it too complicated. Why use "yards" as an equalizer at all?

I'm not a math guy, but what's wrong with assigning a 100 point scale for each of the major statistics, and then correlating them? Something like:

0 TDs = 0
1 TD = 25
2 TD = 50
3 TD = 75
4 TD = 100
5 TD = 125
6 TD = 150

Completion percentage = completion percentage.

Yards per pass x 10 = percentage

Interceptions
0 = 100
1 = 75
2 = 50
3 = 25
4 = 0
5 = -25
6 = -50

ETC.?

I'm not saying that's perfect, but at least it would be understandable.

16
by Patrick (not verified) :: Tue, 09/02/2008 - 12:49pm

@14:

he's using a minimum of 20 pass attempts to qualify.

odd choice, the NFL requires 14 attempts/game to qualify in general. I know a lot of QBs through NFL history averaged less than 20 attempts/game,

and in general, if you are one of the high scores on this scale, you won't need to be throwing much.

17
by pawnking (not verified) :: Tue, 09/02/2008 - 12:51pm

So what you are saying is using improved and advanced statistical evaluation methods you have determined that Johnny Unitas was good?

18
by steelberger1 (not verified) :: Tue, 09/02/2008 - 12:55pm

16: Why pick an arbitrary number to qualify, when it isnt the official NFL number?

So, in Ben's case...they were blowing Baltimore out and he was pulled midway through the 3rd quarter (he was 13/16 at that point).

So, because they didnt attempt to run up the score, his game doesnt count?

19
by Bobman (not verified) :: Tue, 09/02/2008 - 1:08pm

Look at Namath's game in the third table--15/28 with one pick would get him laughed at in today's game. But 496 yards and 6 TDs against the defending SB champs... wow. Yeah, I'd like to see a D-adjusted rating for these games--that one might just be off the charts. I say that as an old Baltimore Colt fan/Namath and Jets disliker.

Regarding perfect games, I think playoff games should count and count double! After all, your opponent it not likely to be an expansion team or go 1-15 (cough).

/Colts bias

#17 Excellent point.

20
by Kelley Ritchey (not verified) :: Tue, 09/02/2008 - 1:18pm

Interesting and detailed article. I agree with the weaknesses of the NFL passer rating.

That said, I disagree with the conclusions of the new rating system. Brady's performance in week 7 was a good one, but not the top rated one overall last year (even by New England), in my opinion. Some of the rating adjustments seem quite arbitrary (40 points for any interception?, etc.)

Would anyone like to guess an alternative Brady game from last season?

21
by mattfwood (not verified) :: Tue, 09/02/2008 - 1:22pm

I know that adding rate stats and counting stats is always problematic, but rather than JJ's resort to counting I've always thought that putting this on a 100 point scale might make it easier to comprehend -- even if true outlier games would bust the 0-100 paradigm I'm shooting for.

What's wrong with the below formulation? (I'm sure that all of you deep thinkers and real stat experts can tell me what's wrong with it too, but I offer it up anyway.)

(Completion Percentage x 0.25) +
(yards per attempt x 0.02) +
(Touchdown Percentage x 2) - (Interception Percentage x 2)
x 100

Using that formula for Brady's game last year would yield a score of 97.32. Unitas's game would be a 98.25. In other words, darn near perfect, but after all they did throw 3 or 4 incompletions each. (Meaning, to me, that it's pretty nonsensical to say either had a "perfect game" and that a 20 for 20, or 25 for 25, would be no more perfect.)

The fact that I would not cap TD scores or INT penalties means that truly horrendous or video-game like performances would bust the 100 point scale. But the day that a guy goes 10 for 10 with ten 100-yard TD passes, I'll be glad to hand him his 425 on my scale.

22
by Temo (not verified) :: Tue, 09/02/2008 - 1:29pm

20. That Buffalo game had me wondering if I was actually watching an NFL game... it looked like Brady could do whatever he wanted that game. I didn't see the Redskin game, but that score was outlandish as well.

23
by Temo (not verified) :: Tue, 09/02/2008 - 1:36pm

Looking back at the stats though (in my name), I do think that Miami game was his best one. He had the most yards/attempt (by a lot), the most TDs, and the least incompletions.

24
by JJcruiser (not verified) :: Tue, 09/02/2008 - 1:37pm

21: That's a fine system. Maybe better than my idea though more complex. My system wasn't the point. The point is that this Palmer guy's "new" system isn't easy to understand, and his main criticism of the old system was that no one could understand it.

As my old professor used to say: "That just don't make good nonsense."

25
by NewsToTom (not verified) :: Tue, 09/02/2008 - 1:39pm

As I see it, there are three main distinctions between the NFL Passer Rating formula and AdjYPA:
1. Which is more important, completion percentage or yards per completion? In AdjYPA, these are deemed equally important. In Passer Rating, completion percentage is three (3) times as important.
2. What is the proper weighting of touchdowns and interceptions? In the Passer Rating formula, an interception is worth 25% more than a touchdown. In AdjYPA, the interception is worth four (4) times as much as a touchdown.
3. How should we relate touchdowns and interceptions to completion percentage and yards per completion? Passer Ratings treats them as categories of equal weight, while AdjYPA uses yardage equivalencies.

Note that since I first heard of AdjYPA, it's been my preferred QB statistic of choice. I'd ignored Passer Rating in favor of YPA since before FO existed and before I'd heard of Hidden Game of Football, let alone acquired the copy I've been meaning to read for 6 months, and AdjYPA is an improvement to raw YPA. Its handling of the relative importance of TDs and INTs, and how to compare those to Comp% and YPC strikes me as far superior to the way they're handled in Passer Rating.

I'm not quite sure, though, that equal rating of Comp% and YPA is the ideal solution. It far better rewards actual success than Passer Rating, which is good, but downs are valuable in football-not quite like outs in baseball, but the closest equivalent thereto we can get a discrete handle on from PBP data. I don't have time to run numbers right now, but one thing I'd be interested in seeing is how single-game and season-long numbers change with a relative importance rating in between that of Passer Rating and AdjYPA, but leaning to the AdjYPA side.

26
by Kelley Ritchey (not verified) :: Tue, 09/02/2008 - 1:43pm

Temo, very good. I know the FO community is a good one. I would propose the Buffalo game in Week 11 as Brady's best.

My assessment is more detailed (based on play-by-play). Obviously, one of the criteria of a rating method is simplicity. However, we would argue the game of football is not that simple.

In other words, the risks and rewards of a pass are different based on field position, game situation etc. To be great, a rating method should assess this dimension.

27
by Temo (not verified) :: Tue, 09/02/2008 - 1:49pm

26. On a personal viewpoint, I would agree that the Buffalo game was just simply amazing to watch. I watched both Miami and Buffalo, and the latter just blew my mind moreso than Miami. Perhaps that was because of the ridiculous way that Moss kept scoring in that Miami game compared to how efficient Brady was in Buffalo. But you can't deny that the paper stats look better for the Miami game.

As for the "better rating system", your description pretty much describes what DVOA is supposed to be :)

28
by Arkaein (not verified) :: Tue, 09/02/2008 - 1:58pm

Wouldn't it make sense to create an improved passer rating by finding the weighted formula that best correlates with VOA (not DVOA, since passer ratings have never taken opponent abilities into account)?

The formula would look something like this:

rating = x1 * YPA + x2 * PCT + x3 * TDs + x4 * INTs

Then some basic numerical analysis coule be done to find the x's that maximize the correlation with VOA.

29
by Kelley Ritchey (not verified) :: Tue, 09/02/2008 - 2:00pm

Temo, you are right. My rating method's objective and DVOA's(and other FO stats) are likely very similar.

I expect there are some differences in execution, but a broad level of agreement, too.

30
by SamoanRob (not verified) :: Tue, 09/02/2008 - 2:01pm

Why ignore sacks, fumbles and rushing yards?

31
by Scott P. (not verified) :: Tue, 09/02/2008 - 2:03pm

The point is that this Palmer guy’s “new” system isn’t easy to understand, and his main criticism of the old system was that no one could understand it.

That wasn't his main criticism. He only mentioned it in passing.

32
by mattfwood (not verified) :: Tue, 09/02/2008 - 2:11pm

Scott P. -- is that a pun? If so, well done. If not, look back at the first line of the article.

Unrelated note: why is that whenever I see people discussing YAR and DYAR on this site, I constantly have running through my mind the refrain "Oceania is at war with Eurasia; Oceania has always been at war with Eurasia."

p.s. looks like only the anti-spam word missed re-education classes, as I was just asked to enter dpar there instead of dyar.

33
by Temo (not verified) :: Tue, 09/02/2008 - 2:19pm

32. Hey, not our fault; Aaron destroyed all DPAR stat tables.

34
by mattfwood (not verified) :: Tue, 09/02/2008 - 2:22pm

33 -- I wasn't blaming the proles; just Big Brother.

35
by Joe A (not verified) :: Tue, 09/02/2008 - 2:25pm

Love the guest column from Pete Palmer. (I paid a lot of money for used copies of the "Hidden Game" books; I think they're the best books ever written about baseball and football stats.)

36
by Temo (not verified) :: Tue, 09/02/2008 - 2:26pm

34. Damn you, Schatz!

And to rub it in, my anti-spam word is Schatz.

37
by Kelley Ritchey (not verified) :: Tue, 09/02/2008 - 2:42pm

Samoan Rob, I would agree that sacks should be in there. Rushing and fumbles may not be as relevant, in my opinion.

The problem with creating stats tied to the QB is the performance is tied to others on the team- having a strong offensive line provides more time to find a receiver, great receivers may catch difficult passes (making the QB look better than otherwise). We're kind of saying "other things being equal".

38
by NewsToTom (not verified) :: Tue, 09/02/2008 - 3:11pm

Re #28
Are you confident enough that VOA is "right" that we can blithely assume it's the correct baseline and we should measure other stats based on how well they match up to DVOA? I really like VOA, but not enough that I'd go that far. For disaggregation purposes of your solver, I'd also suggest using YPC instead of YPA, which is just (YPC * Comp%).

39
by Temo (not verified) :: Tue, 09/02/2008 - 3:35pm

VOA is in itself a rate-based measure of performance. Using it to determine passer rating doesn't make much sense.

40
by David Lewin (not verified) :: Tue, 09/02/2008 - 4:27pm

#28

We at FO have done just that in the past. I ran some regressions to determine the best linear weight approximations of VOA and PAR. I believe I posted them previously, probably in my Ken Anderson article, but I can look up again if people want. As some commenters have noted Palmer's hypothesis that completion percentage does not matter is incorrect. At a given YPA a higher completion percentage leads to a higher VOA and to greater team success. Consistency matters.

41
by JJcruiser (not verified) :: Tue, 09/02/2008 - 4:32pm

31: Maybe you are reading this differently than me, but this is what he said:

"The biggest problem with this system was mentioned earlier: Almost nobody understands it, and those who do still find it awkward to use."

I don't think that's mentioned "in passing."

42
by Bright Blue Shorts (not verified) :: Tue, 09/02/2008 - 4:43pm

"If a passer goes 90 yards downfield in 10 pass attempts, it really does not matter if he had four completions or six, or even one."

Having watched the 2006 Raiders try to recreate the era of 7-step drops and the vertical passing game, I'd have to say even RaiderJoe would agree that the game doesn't often allow QBs to pass deep successfully.

There's a reason why the aged riverboat gambling Jets QB is career leaders in every statistical category INCLUDING interceptions.

All in all, I'm not sure I can buy into this disposal of the completion percentage thing especially when YPA is influenced by the receiver's ability to gain YAC (or the defender's inability to tackle).

In the end, the perfect QB (RoboQB?) would pick the open receiver and complete the pass. That receiver might be downfield in a busted coverage or he might be a checkdown fullback exiting the backfield. When no open receiver existed he would throw the ball away or run with it. He would only be able to take what the defense gives him. Even his open receivers might break tackles to improve their YAC and the QB's YPA.

PS It should also be noted that the new system still doesn't get around one of the criticisms of the old system which is that the numbers don't mean much ...

43
by BadgerT1000 (not verified) :: Tue, 09/02/2008 - 4:53pm

Lynn Dickey in 1983 was as good as one could be given the context. Dickey's knees were shot so the guy fell over if a defensive player breathed on him. The Packers defense was poor so GB was always in catch up or keep it close mode. The running game was Gerry Ellis who was mostly just a show between Lynn dropping back to pass.

The Packers did have James Lofton in his prime, Paul Coffman in his prime and a qb who got them the ball.

Dickey threw for 4458 yards with a YPA of 9.21! 32 Touchdowns. The negative was 29 interceptions which given the circumstances is somewhat understandable.

GB played in five overtime games that season, losing three. The highlight of the season was beating the Redskins 48-47 on Monday Night Football. Lynn Dickey hit 22 of 30 passes for 387 yards and three scores.

Dude was done two years later. But when he was standing upright Lynn Dickey could throw with anyone.

44
by whatnow (not verified) :: Tue, 09/02/2008 - 5:19pm

When I was getting into football in 1990 I wrote to the league asking them to explain passer rating to me and they were good enough to send me a manual. It made two things very clear:

1) this is a passer rating system, NOT a QB rating system

2) the system is meant to compare passer performance over at least a few games, preferably the entire season. It is NOT meaningful for single games or drives.

Please read point (2) as many times as you need to get that single-game passer ratings are meaningless and not the original intention of the system at all (although, of course, widely cited on TV and other media). In particular, TD and INT percentages of zero in individual games heavily effect the rating, but don't necessarily mean a whole lot.

The main "weakness" of the passer rating system is that many people simply use it the wrong way.

45
by Scott C. (not verified) :: Tue, 09/02/2008 - 8:39pm

I don't like throwing out completion % entirely ...

As mentioned many times: on offense CONSISTENCY is critical.

8 plays each 8 yards is MUCH better than 4 plays for 0 yards and 4 plays for 16 randomly dispersed.
The former, will never punt, while the latter can go 3 and out.

(the opposite is true on defense, big plays and stuffs are more important than average gain).

So, yards per pass needs to be tempered by completion %, or some other thing that accounts for variance.

20 yards/pass with a 5 yard distribution width is a LOT more valuable than 20 yards/pass with a 15 yard distribution width.

46
by stan (not verified) :: Tue, 09/02/2008 - 9:19pm

Why no inclusion of the perfect games in the playoffs?

My biggest beef with all these systems is that they reward a QB for taking a sack and penalize him for throwing the ball away. That's nuts.

47
by stan (not verified) :: Tue, 09/02/2008 - 9:23pm

I'm too tired to crunch them. What would 22-26 for 377, 5 TDs and 0 ints work out to in each of the different calculations?

48
by stravinsky (not verified) :: Wed, 09/03/2008 - 12:18am

I seem to recall reading that the passer rating system was designed specifically to reward QB's who have a high completion percentage, move the ball and score points. Guys who heave-ho all game (like Namath) and have low completion pct and high yardage and guys who dink-and-dunk for high completion pct and low yards were kind of like the 2 end points and the system was designed to reward the guys in the middle.

49
by David Hickey (not verified) :: Wed, 09/03/2008 - 12:28am

That 1983 Dickey season was incredible. I think he started off with a 31-27-331-5-1 game against Houston. Also was like 15-10-267-3-1 against Tampa as well.

Normally, one wouldn't be surprised to see Dan Fouts with one of the best rated games ever, but what did shock me was that it happened in 1974, his 2nd year. Fouts pretty much sucked his first few seasons for a variety of reasons, but he still managed to have an all-time game in 1974.

50
by Bryan L (not verified) :: Wed, 09/03/2008 - 12:32am

It's amazing how Dan Fouts in that 1974 game beat San Diego while playing for San Diego...

51
by Mark Finzel (not verified) :: Wed, 09/03/2008 - 3:49am

What Pete Palmer is suggesting is a slight modification of George Ignatin and Allen Barra's Passing Effectiveness Rating (PER), which they brought up in a book called "Football By The Numbers" in the mid 1980s. They didn't include the 10 yds for the TDs at all, and used a figure of 50 yds for the INTs. Team ratings included sacks, and THAT was a powerful and simple tool. I wish there was a way to separate line sacks from QB sacks, it would be perfect to rate individuals then, and simple to figure.

52
by mattfwood (not verified) :: Wed, 09/03/2008 - 11:26am

47 - Under the current NFL system, your stat line (22 of 26 for 377 yards, 5 TDs, and no picks) is a perfect game of 158.3. (And what kid doesn't cherish the memory of the first time he or she scored 22 of 26 on a test and ran home to report "I got a perfect 158.3 on my paper!!"?)

Under the unlimited system ("Rate2" in Palmer's table, your stat line would rate a 197.12.

I'm not quite good enough at this to understand how Palmer gets his "Rate3" adjusted YPA. (See? I told you I couldn't really do math.) I'm missing some minor adjustment, and the numbers I get don't match up to what he reports in his last table. i'm off by like a half of a yard or a yard, somehow. Anybody out there who's better at explaining Palmer's method than he is, and better at doing it than I am?

Meanwhile, nder my admittedly half-baked system proposed above at 21, your hypothetical stat line would yield an 88.62.

53
by NewsToTom (not verified) :: Wed, 09/03/2008 - 11:55am

Re #52
(Yards + 10*TD - 40*Int)/Att. Pretty simple, and the calculations look right to me. I'm pretty sure P-F-R uses 45 yards as the interception penalty, so that may be throwing you off.

54
by mattfwood (not verified) :: Wed, 09/03/2008 - 3:51pm

53 - No, what was throwing me off was his explanation that the completion bonus is eliminated under his system. I was no longer multiplying the number of completions by 20 but I was still adding in the raw # of completions because that's how I read "bonus would be eliminated."

In that case, Stan's stat line produces a 16.42 adjusted YPA.

55
by mattfwood (not verified) :: Wed, 09/03/2008 - 3:54pm

53 - no, my error was that when he said that the completion bonus would be eliminated, I didn't realize that completions dropped out altogether. All I did was to stop multiplying completions by 20, but I was still adding in the raw # of completions. Pretty dumb, but it was a thinking error rather than a math error.

Anyway, in that case Stan's stat line yields a 16.42 in Rate3 adjusted YPA.

56
by NewsToTom (not verified) :: Wed, 09/03/2008 - 5:01pm

Re #54/55
Well, completion percentage is still there, it's just meshed in combined with yards per completion (I'll be shocked if this turns out legible):
Yards Completion
---------- x ---------- =
Completion Attempt

Yards
----------
Attempt

57
by Peter Koski (not verified) :: Wed, 09/03/2008 - 6:45pm

I thought Steve Young's SB XXIX performance would have rated higher, but the 6 TDs don't make up for 325 yards on 36 attempts. Too bad.

West Coast offense discrimination!!

58
by Zac (not verified) :: Wed, 09/03/2008 - 10:14pm

Re: 28. That there explains the real problem with the Passer Rating system. Palmer doesn't mention it because his systems suffer from the same problem: It's arbitrary. It's like OPS (On-Base Plus Slugging) in baseball (which wikipedia tells me was first popularized by Palmer's book). You can't add On-Base Percentage and Slugging Percentage together. They don't mean anything.
Gross Production Average takes this into account in baseball, and your suggestion would do the same thing to Passer Rating.

59
by Kellerman (not verified) :: Thu, 09/04/2008 - 4:49pm

I couldn't help thinking that Palmer's method values the "Mad Bomber" types over every other and there I see Lamonica actually appearing! To me, this is one of the most nonsensical articles ever to appear at FO. Why pick 20 attempts per game? Why assume that completion percentage is worthless? Why the unsupported assertion that boom-and-bust is just as good as consistency? Why replace a system that yields an arbitrary number with ANOTHER system that yields an arbitrary number? (At least the old system attempted to fudge the numbers so that 100 was recognizably excellent. What is theoretical excellence in Palmer's method?) Finally, why take a system never designed to rate GAMES and talk ONLY about games, rather than seasons? (with apologies to Luckman)

If this type of article is what passes for expertise elsewhere in the Blogosphere, then no thanks! Did FO put this here just as an example of others' work? I hope it's not because they're impressed with the work and somehow endorse the conclusions by inclusion here. I also hope it's not just so they can say "Hey, recognizable name Pete Palmer posted here so we must be big-time!" Sorry for the diatribe, but I hope that the comings and goings at FO this year don't portend the replacement of guys like Michael David Smith with guys like Pete Palmer.

60
by Miles (not verified) :: Fri, 09/05/2008 - 3:47pm

The average number of points scored from a first-and-goal on the 1-yard line is about 6, not 7, so the last yard is the hardest to gain and there should be a bonus for crossing that threshold.
Interesting that:
In Hidden Game of Football, you used a linear trend line to determine the value -- the last yard was not more valuable than the 2nd to last one.
in this article it looks like the biggest point differential was around the 5 yard line in 2002. Why not get a bonus for that threshold too?
how does a touchdown get 7 points? (I know how one gets 6, the value used in the book)

61
by Scott C. (not verified) :: Sat, 09/06/2008 - 12:28pm

#56:

No, it does not factor in completion percentage. Its a per attempt stat, but as I stated before, the true value of a QB's game MUST account for competion percentage.

This adjusted YPA is totally useless in evaluating a QB's performance. USELESS.
The old QB rating system, equally arbitrary, is better.

Its pretty simple why. Consistency matters. A LOT.

A QB with a line of : 30-30 4TD 0 INT, 300 yards had an amazing day.

But this scheme would rate a guy who went 15-30, 4TD, 0 INT, 305 yards as having a better day.

The line "If a passer goes 90 yards downfield in 10 pass attempts, it really does not matter if he had four completions or six, or even one.”

Misses the point ENTIRELY. Yes, one pass for 90 yards is as good as 6 passes at 15 each (roughly). However, 6 passes, with 5 incompletions and 1 90 yard pass is a LOT less valuable than either of the two above, so while fixing one problem, a worse one was introduced.

Although a completion bonus is a bad idea (completing a 0 yard pass is the same value as an incomplete pass), there must be more than just the # of attempt denominator to balance things out.

Of course, that is EXACTLY what (D)VOA does. (D)VOA rewards consistency and chain-moving and does not reward 10 yard completions on third and 15, and penalizes for failed third down conversions or other bad plays. Its not arbitrary, and does what is needed. Its just not simple enough for people to understand.

62
by nelphonious of pennfielde (not verified) :: Wed, 09/10/2008 - 2:17pm

yards per passing attempt as the core QB rating stat reflects completions for zero or minus yards, interceptions, and completion %. ..start there and come up with the unit value for (TD - int.)It will track games on List three. Right on Scott #56.

63
by Albert (not verified) :: Mon, 10/06/2008 - 9:03am

According to the equations above, if the maximum quarterback rating is 158.3 is the maximum for Rate B (in the the equation above), then the maximum for Rate A is: (158.3 - 50/24) * 24/100 = 37.492. If we furthermore assume that each of the 4 statistics got the maximum, then we divide by 4 to get 9.37 yards per attempt for each of the 4 categories. However that is inconsistent with the table right above, where you need an average gain of 12.5 yards to get the maximum. It is also inconsistent with the completion percentage, since at 20 yards per completion, you wouldn't need to complete even half of your passes.

What am I missing?