Receiving Plus/Minus, Part I
by Bill Barnwell
One of my pet projects at Football Outsiders since my very first article for the site nearly three years ago has been to point out the importance of a receiver's catch rate. Mostly, that's come through focused analyses of former Dolphins teammates Chris Chambers and Wes Welker, who represented, at different times, the furthest outliers on either side of league average.
Of course, there's something inherently unfair in those calculations; you only need to look at Chambers and Welker to see that they're dramatically different players. Chambers is a lanky, lithe downfield receiver that uses his frame and elite jumping ability to make catches that few other receivers in football could. Welker is an undersized waterbug of a receiver, using his low center of gravity and excellent agility to get open against bigger players on underneath routes.
It's pretty obvious, even without our research, that a pass thrown 20 yards downfield is less likely to be caught than one five yards away. It's also obvious that not all passes are created equal; in all likelihood, Chambers will see more passes in a year that are absolutely uncatchable than Welker will.
In doing our research for a potential essay on the topic in this year's book, I came across a third factor: Distance away from the sticks. As mentioned several weeks ago, passes right at the first down marker are significantly more likely to be incomplete than passes within several yards of said marker on either side.
A potential statistical quirk chalked up to overzealous official scorers? Perhaps, but then it hit me that we had the solution to eliminating many of the problems above: The Game Charting Project data.
Using the four years of Game Charting data compiled by our volunteers, it occurred to me that we could strip out as many of the "uncatchable" passes as possible and separate yards in the air from yards after the catch, while factoring the down and distance in as well. The methodology is simple enough: Remove as many "uncatchable" passes from the database as possible, create a baseline expected catch rate for each pass, and then measure every remaining pass against that baseline. It's from there that it gets tricky.
For those of you unfamiliar with the Game Charting Project, our charters mark every incomplete pass with one of several different excuses. Some of them are obviously the fault of the wide receiver, most blatantly "Dropped" or "Receiver Tripped". Others have more to do with the quarterback or the pass rush: "Thrown Away" or "Batted Down At Line" clearly have very little to do with the wide receiver.
To create a database of "catchable" passes, I removed those pass plays that the receiver clearly had no shot of catching, like the plays above. I kept in potentially difficult ones like "Overthrown" or "Thrown Behind" since those sort of plays can very well be chalked up to the speed or timing of a receiver's break. I also kept in passes defensed, but removed pass interference penalties.
That left just over 60,000 pass plays from 2005 through 2008 to calculate our baselines with. Since I'm not a math whiz, I wanted a quick and dirty way to figure out the baseline while accounting for both the distance of the pass as well as the location relative to the down-and-distance. My admittedly oversimplified methodology was to create a baseline for the raw distance of the pass, and then create a second baseline for the down-and-distance, with the final baseline being the average of the two.
I also adjusted for the location of the pass by separating the baselines into "up the middle" and "not up the middle" (since we don't have specific location data for each pass to define how far to the left or the right a pass is). Passes up the middle are, on average, 2.1 percent more likely to be caught than those to either side of the field.
As an example, let's use a pass attempt up the middle of 12 yards on 2nd-and-15. The raw catch rate for a pass of 12 yards up the middle, after removing all those uncatchable passes, is 58.3 percent. Since 12 yards is three shy of a first down, our down-and-distance calculation was for all passes thrown on second down three yards short of a first down. Those passes are caught 74.4 percent of the time. The average of those two figures is our final baseline for that catch: 66.4 percent. (In reality, there are only four passes in the database to those exact specifications; three of them were caught.) Most of the baselines don't have a split of 16 percent or so between the two of them; the average difference between the raw yards baseline and the down-and-distance baseline is 5.6 percent.
For those pass distances or situations that didn't have a large enough sample, I calculated the expected catch rate by expanding the measurement to factor in the catch rates of neighboring distances; there's not too many passes in the NFL that are 56 yards past the line of scrimmage, for example, but it's safe to compare that distance to passes that are, say, 50-60 yards downfield if it creates a larger, more reliable sample.
Table 1 reveals the baselines at six different pass distances for the same situation, first-and-10. A pass that was thrown "-2" yards in the air is one thrown two yards backwards; we separate these out from laterals.
| Table 1: Expected Catch Rate Baselines, First-and-10 | |||||||
| Middle | Wide | ||||||
| Distance in Air |
Distance to First Down |
Raw Distance Baseline |
Relative Distance Baseline |
Average Baseline |
Raw Distance Baseline |
Relative Distance Baseline |
Average Baseline |
| -2 | 12 | 0.84 | 0.83 | 0.84 | 0.86 | 0.88 | 0.87 |
| 5 | -5 | 0.74 | 0.78 | 0.76 | 0.71 | 0.75 | 0.73 |
| 8 | -2 | 0.65 | 0.71 | 0.68 | 0.65 | 0.70 | 0.67 |
| 10 | 0 | 0.65 | 0.55 | 0.62 | 0.58 | 0.53 | 0.55 |
| 12 | 2 | 0.61 | 0.54 | 0.57 | 0.57 | 0.59 | 0.58 |
| 15 | 5 | 0.57 | 0.59 | 0.58 | 0.48 | 0.48 | 0.48 |
With those baselines, we then established a point value for each play, similar to the plus/minus system employed by John Dewan in his analysis of fielding in baseball. Let's take the example of a five-yard pass to the sidelines on first-and-10. As you can see above, the baseline for such a play is .73; since you can only make zero catches or one catch on a play, those are the only values we can use relative to the baseline. A catch in this scenario is worth .27 points (1 catch - .73 expected catches), while an incomplete pass is worth -.73 points (0 catches - .73 expected catches).
After making Excel angry with the million or so queries required to produce a value for each catch, we finally had a distance-adjusted catch rate. For the purposes of giving the metric a simple, easy to remember name, we'll call it Receiver Plus/Minus.
The results? Well, at first they sort of told us what we already knew.
| Table 2: Best and Worst Receiver +/-, 2005-2008 | ||||||
| Year | Player | Team | Targets | Catches | Catch Rate |
Rec. +/- |
|---|---|---|---|---|---|---|
| 2007 | 87-R.Wayne | IND | 142 | 104 | 73.2% | 19.23 |
| 2008 | 80-A.Johnson | HOU | 158 | 115 | 72.8% | 15.13 |
| 2006 | 88-M.Harrison | IND | 143 | 95 | 66.4% | 12.76 |
| 2007 | 83-W.Welker | NE | 139 | 112 | 80.6% | 12.55 |
| 2007 | 84-B.Engram | SEA | 128 | 94 | 73.4% | 12.45 |
| 2006 | 87-M.Furrey | DET | 132 | 98 | 74.2% | 12.04 |
| 2005 | 84-T.Houshmandzadeh | CIN | 97 | 75 | 77.3% | 12.04 |
| 2005 | 87-R.Wayne | IND | 116 | 83 | 71.6% | 11.59 |
| 2008 | 15-S.Breaston | ARI | 107 | 77 | 72.0% | 11.31 |
| 2006 | 87-R.Wayne | IND | 131 | 86 | 65.6% | 11.08 |
| Year | Player | Team | Targets | Catches | Catch Rate |
Rec. +/- |
| 2008 | 82-D.Bowe | KC | 155 | 86 | 55.5% | -9.75 |
| 2006 | 85-G.Jennings | GB | 96 | 45 | 46.9% | -9.82 |
| 2008 | 81-T.Owens | DAL | 135 | 69 | 51.1% | -10.60 |
| 2008 | 11-Ro.Williams | DAL | 81 | 36 | 44.4% | -10.87 |
| 2008 | 86-M.Booker | CHI | 44 | 14 | 31.8% | -11.33 |
| 2008 | 87-A.Lelie | OAK | 41 | 11 | 26.8% | -11.47 |
| 2006 | 84-J.Galloway | TB | 129 | 62 | 48.1% | -11.89 |
| 2005 | 17-P.Burress | NYG | 144 | 70 | 48.6% | -15.03 |
| 2008 | 17-B.Edwards | CLE | 128 | 55 | 43.0% | -19.75 |
| 2006 | 84-C.Chambers | MIA | 135 | 59 | 43.7% | -21.11 |
The great raw catch rates of guys like Welker and T.J. Housmandzadeh still translate well after adjusting for distance, while the incompetence of Braylon Edwards and Chambers in their respective valley years is held up for all to see.
Of course, it takes volume to accrue large receiving plus/minus figures, one way or another. What if we divide the plus/minus by the number of targets?
| Table 3: Best and Worst Receiver +/- Per Target, 2005-2008 (Min. 50 Targets) |
|||||||
| Year | Player | Team | Targets | Catches | Catch Rate |
Rec. +/- |
Rec. +/- per Target |
|---|---|---|---|---|---|---|---|
| 2008 | 19-I.Hilliard | TB | 54 | 47 | 87.0% | 9.76 | 0.18 |
| 2005 | 82-J.Witten | DAL | 75 | 61 | 81.3% | 10.87 | 0.14 |
| 2008 | 82-J.Reed | BUF | 71 | 56 | 78.9% | 9.91 | 0.14 |
| 2007 | 87-R.Wayne | IND | 142 | 104 | 73.2% | 19.23 | 0.14 |
| 2007 | 83-H.Miller | PIT | 60 | 47 | 78.3% | 7.90 | 0.13 |
| 2005 | 82-D.Jackson | SEA | 54 | 38 | 70.4% | 6.91 | 0.13 |
| 2007 | 80-A.Johnson | HOU | 82 | 60 | 73.2% | 10.21 | 0.12 |
| 2005 | 84-T.Houshmandzadeh | CIN | 97 | 75 | 77.3% | 12.04 | 0.12 |
| 2007 | 11-A.Gonzalez | IND | 50 | 37 | 74.0% | 6.20 | 0.12 |
| 2006 | 19-D.Henderson | NO | 50 | 32 | 64.0% | 6.00 | 0.12 |
| Year | Player | Team | Targets | Catches | Catch Rate |
Rec. +/- |
Rec. +/- per Target |
| 2006 | 80-B.Scaife | TEN | 51 | 29 | 56.9% | -5.54 | -0.11 |
| 2007 | 83-D.Bennett | STL | 70 | 33 | 47.1% | -8.11 | -0.12 |
| 2008 | 82-L.Smith | PHI | 63 | 37 | 58.7% | -7.35 | -0.12 |
| 2007 | 83-K.Colbert | CAR | 64 | 32 | 50.0% | -7.73 | -0.12 |
| 2005 | 86-M.Booker | MIA | 72 | 32 | 44.4% | -8.74 | -0.12 |
| 2008 | 11-Ro.Williams | DAL | 81 | 36 | 44.4% | -10.87 | -0.13 |
| 2006 | 27-L.Johnson | KC | 61 | 41 | 67.2% | -8.20 | -0.13 |
| 2008 | 17-B.Edwards | CLE | 128 | 55 | 43.0% | -19.75 | -0.15 |
| 2006 | 84-C.Chambers | MIA | 135 | 59 | 43.7% | -21.11 | -0.16 |
| 2006 | 81-R.Davis | CHI | 52 | 22 | 42.3% | -8.90 | -0.17 |
There's the sort of list we were looking for! The top ten mixes in a variety of players; possession receivers playing over their heads (Ike Hilliard and Josh Reed a year ago), a deep threat that had a better-than-average year (Devery Henderson), and your garden-variety elite receivers (Reggie Wayne, Andre Johnson). The bottom ten features a similar mix, and even throws in Franchise Player for good measure.
That seems like a better fit for what we're trying to measure, but what does it mean? Is it a consistent statistic from year-to-year, or subject to random vagaries and whims? We'll hit those questions in Part II of this analysis, coming later this week.
Comments
41 comments, Last at 08 Aug 2009, 10:06pm
#1 by TimK // Aug 04, 2009 - 1:40pm
Interesting, looking forward to part II to see what it really means. Year-on-year consistency in this could make a very useful stat.
(As a Denver fan I'm also interested that Brandon Marshall is not showing up in the bottom few for last season, and not surprised to that Ashlie Lelie does...)
#2 by Karl Cuba // Aug 04, 2009 - 1:54pm
Would it be possible to arrive at a similar metric for quarterbacks? It would highlight schematic differences between teams.
The difference in average +/- for receivers who have played for different teams or in different systems would also be interesting.
#3 by Jacob Stevens (not verified) // Aug 04, 2009 - 2:00pm
This is really great, Bill. At first glance, it looks like even still the quality of the passer has some kind of effect. That might be giving these great WRs too little credit, though. Is Reggie Wayne underrated?
I can't believe you've only been writing at FO for three years. Feels like so much longer.
#7 by billsfan // Aug 04, 2009 - 2:21pm
I don't think it's possible for Reggie Wayne to be underrated.
(I also like the Eagles)
#26 by El Nino Meon (not verified) // Aug 04, 2009 - 10:03pm
Well, maybe. But there isn't a control for the fact that Reggie Wayne is being thrown to by Peyton Manning. And it isn't exactly unreasonable to think that a 20-yard Peyton pass is a bit more catchable than average.
#32 by John (not verified) // Aug 05, 2009 - 2:56pm
And it isn't exactly unreasonable to think that a 20-yard Peyton pass is a bit more catchable than average.
Yeah, I think Peyton reserved his truly unreasonable passes for Marvin Harrison, who made quite a career out of catching them anyway.
Reggie seems to get a fair number of "stick my hands out and there's the ball" passes.
#33 by billsfan // Aug 05, 2009 - 4:45pm
Obviously we all agree that it's difficult to separate the performance of a QB from his receivers, but developing a "stick out my hands and there's the ball" rapport with any QB, even Peyton Manning, still takes a very talented receiver.
re: Marvin, my all time favorite is still the one on the right side of the endzone, on his tiptoes, inch(es) from the sideline, falling out of bounds to catch a ball that's easily 3 ft. out already. They had to have practiced that one!
(I also like the Eagles)
#34 by John (not verified) // Aug 05, 2009 - 7:27pm
For me, the play that stays with me is when Peyton threw it behind him in the endzone; Marvin sticks out his hand, knocks it up in the air, then turns around and grabs it.
I would seriously consider giving a kidney to have a few more years of a healthy 18 to 88 combo. Watching those two in action was simply amazing.
#4 by Mountain Time … // Aug 04, 2009 - 2:13pm
Franchise Player? What am I missing?
----------------------------------
"Just look at that pumpkin."
-John Madden, looking at the moon.
#8 by Brendan Scolari (not verified) // Aug 04, 2009 - 2:25pm
L.J. Smith
#29 by Kevin from Philly // Aug 05, 2009 - 2:34pm
The only way L.J. is a franchise player, is if he buys a bunch of McDonalds. Not surprised to see an Eagle in the "worst of" list, but I really expected FredEx or Pinky.
#35 by Keith (not verified) // Aug 06, 2009 - 8:41am
You are not from around here, are you? It is a joke about L.J. Smith, and it has nothing to do with his skill.
#5 by Thomas_beardown // Aug 04, 2009 - 2:15pm
I like where you are going with this, but wouldn't it have been better to use percentage of a first down instead of yards away from the markers?
Going back to you example, I bet it's a lot easier to complete a pass 3 yards short of a first down on 2nd and 5 than it is on 2nd and 15. However, if you used percentage, 12 yards on 2nd and 15 gives you 80% of a first down, which would be a 4 yard pass on 2nd and 5.
#6 by AnonymousA (not verified) // Aug 04, 2009 - 2:19pm
This is really interesting -- but is it useful? The goal is to measure how hard what a receiver does is, not how useful. For example, a guy who catches a 30 yard out on 4th and 29 is a hero -- unless it's the last play of the game, he went out at the 12 yard line, and his team lost by 5. Of course the defense gave him that -- they had 12 yards to tackle him and win the game!
Part of this may get washed out with the massive sample size you have. It might be interesting, however, to separate plays that increase and decrease win expectancy.
I also wonder if a little bit o' multiple regression could eliminate the discrete 3D table of baselines and turn them into a happy little curve (this could also be done on the raw data, though a 60,000 point multiple regression may not be super-fast. This would, however, allow the addition of the good/bad play component, above, as well as dealing with low sample size for specific distances). From there, a little bit of analysis could yield some fascinating approximations, e.g. (numbers here obviously wrong, since I don't have the data) "every time you double a pass' distance, you make it 3 times harder on the receiver".
Eager to see where this goes.
#9 by billsfan // Aug 04, 2009 - 2:26pm
As a hockey fan, I was a bit disappointed to see +/- as something other than an integer. I also think two decimal places is a bit excessive, at least for how I learned Significant Figures.
Based on the distribution of players and values, this might ultimately be just as effective presented as a whole number.
(I also like the Eagles)
#13 by White Rose Duelist // Aug 04, 2009 - 2:58pm
As a hockey fan, I would like +/- (in hockey and elsewhere) renamed. I like my stats giving you some idea what they mean, and to the novice "plus-minus" only tells you that it can be either above or below zero.
No, I don't have a good idea what it should be called instead.
And, failing that, I want to hear something like "Tiger Woods is two strokes back, with a plus-minus of -3."
#10 by Brendan Scolari (not verified) // Aug 04, 2009 - 2:30pm
As #6 said, is there really any usefulness to this kind of statistic?
Yes, Andre Johnson and Reggie Wayne are on there, but Randy Moss, Terrell Owens, and Larry Fitzgerald combine for 0 of the 10 best receiving seasons in the last 4 years?
I understand that your measuring efficiency, but if your not showing the best players as having the best seasons I really don't see the point of the statistic. It'd be like if there was a new stat for measuring hitting ability in baseball and Albert Pujols, Manny Ramirez, and Alex Rodriguez don't appear on the top 10 list for the last 4 years. The stat just isn't revealing much of the truth.
#12 by shake n bake // Aug 04, 2009 - 2:56pm
It isn't an overall WR ranking. It doesn't include YAC which is a big part of some WRs game and is certainly important.
To adjust your analogy it's like Pujols, Manny and A-rod not being in the top handful in a measure of a certain aspect of hitting like contact% or swings% on pitches outside the zone.
It's possible to be a good or even great player without being in the top handful at every aspect of your job.
#14 by Keith (not verified) // Aug 04, 2009 - 3:08pm
What snb said, really. This method measures catch rate and nothing else.
#30 by Danish Denver-Fan // Aug 05, 2009 - 2:38pm
I guess you could call it "advanced catchrate" or just "hands-rating"...
#11 by Tom Gower // Aug 04, 2009 - 2:40pm
Interesting stuff, Bill.
The one thing I always worry about with things like these is usage rate-that an unspectacular player looks bad because he's a relatively good option (Scaife in 2006 as VY's security blanket) or a good player having a good year looks better than a spectacular player more involved in the offense (compare Devery Henderson 2006 v Randy Moss 2007). Obviously, these concerns are nowhere near unique to WR +/-, and I discount DVOA/DYAR in terms of rating individual WRs and RBs for similar reasons.
#15 by Bowl Game Anomaly // Aug 04, 2009 - 3:21pm
People,
This is not a measure of the best WRs. This is a measure of catching ability only. It's essentially an adjusted catch %. It does not measure WR performance as well as DYAR or DVOA but it measures a specific aspect of WR performance very well- that of catching the passes thrown to you.
Essentially, the point of all this is to see that catch % is inadequate when comparing WRs, but this stat allows us to compare "hands" for WRs accross different skill sets. It's interesting, but it doesn't represent any grand innovation.
(edit: Keith beat me.)
#16 by starzero // Aug 04, 2009 - 3:31pm
so basically wayne and house were amazing in 2007 and 2005, respectively?
#17 by Keith (not verified) // Aug 04, 2009 - 3:37pm
Yes.
#27 by Yaguar // Aug 04, 2009 - 10:58pm
Additionally, consider that Wayne did that with Devin Aromashadu and the like playing across from him.
#18 by Independent George // Aug 04, 2009 - 3:52pm
The thing about the 'per target' stat that really sticks out for me is that, aside from one clear outlier (Hilliard), #s 2-10 are essentially tied.
#19 by Dave B (not verified) // Aug 04, 2009 - 3:56pm
I think this article is useful to see how much the length of the pass effects the AVERAGE catch rate. However I don't think applying it to individual WR's is truly meaningful b/c there are too many other variables that get left out that are very important.
For applying it to individual WR's I think these problems exist.
1. Obviously the amount of coverage and who is covering the WR makes a big difference.
2. I'm not sure leaving in passes marked as overthrown, underthrown, and even passes defensed works when applying it to individual WR's as it is unclear who is it fault between the WR and QB in those cases. At minimum it might be better to weight those cases less.
#20 by Keith (not verified) // Aug 04, 2009 - 4:05pm
I think the effort was to determine a "catchable ball" as defined as one that was where it needed, as put there by the quarterback and caught by the receiver. By removing every "uncatchable ball," it helps to figure out that GIVEN A PERFECT THROW, did the receiver actually catch it?
At least, that is my understanding.
#21 by Jimmy // Aug 04, 2009 - 5:28pm
80-B.Scaife TEN 51 29 56.9% -5.54 -0.11
I knew he was a dirty under-achiever.
2006 81-R.Davis CHI 52 22 42.3% -8.90 -0.17
Doesn't suprise me in the slightest. What is this guy still doing on the Bears roster?
#22 by John (not verified) // Aug 04, 2009 - 6:42pm
I don't know how Indy got so lucky as to have Bill Polian drafting receivers for Peyton Manning to throw to, but thank you Jebus.
#31 by Kevin from Philly // Aug 05, 2009 - 2:38pm
Don't forget to thank Andy Reid, who passed over Reggie Wayne (and Ocho Dope-o) in favor of Freddie Mitchell.
#23 by Scott_ (not verified) // Aug 04, 2009 - 7:37pm
I would absolutely love to see a video of all the passes thrown to Chris Chambers in 2006. I remember arguing with that & Welker after that season. Something like 30% of the passes thrown to Chambers were uncatchable.
#24 by Rich Arpin (not verified) // Aug 04, 2009 - 9:34pm
I wonder how the routes a receiver usually runs play into this? For example, more vertical routes for moss means that he attacks a DB in their back pedal or stride whereas welker runs crossing routes more and it's like he runs away from DB's or towards depending on the coverage of Man or Zone.
#25 by mm (not verified) // Aug 04, 2009 - 9:36pm
I think this is probably still capturing a good deal of the QB's affect. If we're looking at all passes thrown within a few yards of the receiver (which seems to be what we're doing), we'd expect Peyton Manning to have a much higher percentage of those passes right at the receiver and a lower percentage 2 yards away.
Could you try and grab receivers who played with different QBs and see how much this rate is consistently better/worse with certain QBs?
That said, this is still interesting stuff. Devery Henderson, especially early in his career, is one of those receivers who makes a lot of highlight-type catches where he'll come down with the ball when its thrown in a hard to catch place (behind him/inbetween 2 defenders, etc.) but will frequently manage to drop the easy to catch passes.
#28 by njjetfan12 // Aug 05, 2009 - 1:44am
very interesting stuff here, I'm also looking forward to part 2
#36 by CornerBlitz (not verified) // Aug 07, 2009 - 5:37am
1) This is an interesting stat, and highlights (to me) the value of all-around receivers in conventional schemes. Not the chuck-and-duck method used by the Saints and Cardinals, but time-worn offensive philosophies like Indy's and the many ubiquitous West Coast varieties.
2) Fitz and Moss get a lot of looks ridiculously far downfield, which is less and less catchable.
3) I too will miss 18 to 88, and I'm not even a Colts fan. Truly legendary and we were blessed to see it.
4) What's with blaming HCs for bad draft picks? I'd like to see some documentation that Andy Reid wanted Freddie Mitchell over Wayne.
5)Back to the subject at hand: I find it especially interesting that this is really an effective indictment of QBs as well. Two Indy receivers among the best...go figure. Two Miami receivers among the worst...no wonder they're still looking for Marino.
6) Biggest WTF: Larry Johnson? How does a running back figure in the bottom 10, when mostly they catch flares, screens, and quick outs (read: gimmes). That's just shameful.
#38 by Vincent Verhei // Aug 08, 2009 - 12:24am
On LJ: These numbers are adjusted for distance. So LJ is compared only to other receivers running flares, screens, quick-outs, etc.
#39 by CornerBlitz (not verified) // Aug 08, 2009 - 4:37am
Ah...then a fine indictment of KC's QB's as well. Thanks Vince.
#37 by armchair journ… // Aug 07, 2009 - 6:55pm
DJ on the best hands list.... no one else finds that surprising?
_______________________________
armchair journeyman quarterback
#40 by Bowl Game Anomaly // Aug 08, 2009 - 9:19pm
Well according to the chart he caught 70% of his attempts that season.
#41 by Thomas_beardown // Aug 08, 2009 - 10:06pm
I think in this case it's not that he has good hands, but that he was getting wide open and Hasselbeck was delivering very accurate passes.