Writers of Pro Football Prospectus 2008

Most Recent FO Features

HarvinPer09.jpg

» Impact of the NFL's Kickoff Rule Change

After three NFL seasons of kicking off from the 35-yard line, what has been the impact on touchbacks, returns, field position, scoring and injuries? Also, is this rule responsible for a record number of big comebacks?

04 Aug 2009

Receiving Plus/Minus, Part I

by Bill Barnwell

One of my pet projects at Football Outsiders since my very first article for the site nearly three years ago has been to point out the importance of a receiver's catch rate. Mostly, that's come through focused analyses of former Dolphins teammates Chris Chambers and Wes Welker, who represented, at different times, the furthest outliers on either side of league average.

Of course, there's something inherently unfair in those calculations; you only need to look at Chambers and Welker to see that they're dramatically different players. Chambers is a lanky, lithe downfield receiver that uses his frame and elite jumping ability to make catches that few other receivers in football could. Welker is an undersized waterbug of a receiver, using his low center of gravity and excellent agility to get open against bigger players on underneath routes.

It's pretty obvious, even without our research, that a pass thrown 20 yards downfield is less likely to be caught than one five yards away. It's also obvious that not all passes are created equal; in all likelihood, Chambers will see more passes in a year that are absolutely uncatchable than Welker will.

In doing our research for a potential essay on the topic in this year's book, I came across a third factor: Distance away from the sticks. As mentioned several weeks ago, passes right at the first down marker are significantly more likely to be incomplete than passes within several yards of said marker on either side.

A potential statistical quirk chalked up to overzealous official scorers? Perhaps, but then it hit me that we had the solution to eliminating many of the problems above: The Game Charting Project data.

Using the four years of Game Charting data compiled by our volunteers, it occurred to me that we could strip out as many of the "uncatchable" passes as possible and separate yards in the air from yards after the catch, while factoring the down and distance in as well. The methodology is simple enough: Remove as many "uncatchable" passes from the database as possible, create a baseline expected catch rate for each pass, and then measure every remaining pass against that baseline. It's from there that it gets tricky.

For those of you unfamiliar with the Game Charting Project, our charters mark every incomplete pass with one of several different excuses. Some of them are obviously the fault of the wide receiver, most blatantly "Dropped" or "Receiver Tripped". Others have more to do with the quarterback or the pass rush: "Thrown Away" or "Batted Down At Line" clearly have very little to do with the wide receiver.

To create a database of "catchable" passes, I removed those pass plays that the receiver clearly had no shot of catching, like the plays above. I kept in potentially difficult ones like "Overthrown" or "Thrown Behind" since those sort of plays can very well be chalked up to the speed or timing of a receiver's break. I also kept in passes defensed, but removed pass interference penalties.

That left just over 60,000 pass plays from 2005 through 2008 to calculate our baselines with. Since I'm not a math whiz, I wanted a quick and dirty way to figure out the baseline while accounting for both the distance of the pass as well as the location relative to the down-and-distance. My admittedly oversimplified methodology was to create a baseline for the raw distance of the pass, and then create a second baseline for the down-and-distance, with the final baseline being the average of the two.

I also adjusted for the location of the pass by separating the baselines into "up the middle" and "not up the middle" (since we don't have specific location data for each pass to define how far to the left or the right a pass is). Passes up the middle are, on average, 2.1 percent more likely to be caught than those to either side of the field.

As an example, let's use a pass attempt up the middle of 12 yards on 2nd-and-15. The raw catch rate for a pass of 12 yards up the middle, after removing all those uncatchable passes, is 58.3 percent. Since 12 yards is three shy of a first down, our down-and-distance calculation was for all passes thrown on second down three yards short of a first down. Those passes are caught 74.4 percent of the time. The average of those two figures is our final baseline for that catch: 66.4 percent. (In reality, there are only four passes in the database to those exact specifications; three of them were caught.) Most of the baselines don't have a split of 16 percent or so between the two of them; the average difference between the raw yards baseline and the down-and-distance baseline is 5.6 percent.

For those pass distances or situations that didn't have a large enough sample, I calculated the expected catch rate by expanding the measurement to factor in the catch rates of neighboring distances; there's not too many passes in the NFL that are 56 yards past the line of scrimmage, for example, but it's safe to compare that distance to passes that are, say, 50-60 yards downfield if it creates a larger, more reliable sample.

Table 1 reveals the baselines at six different pass distances for the same situation, first-and-10. A pass that was thrown "-2" yards in the air is one thrown two yards backwards; we separate these out from laterals.

Table 1: Expected Catch Rate Baselines, First-and-10
    Middle Wide
Distance
in Air
Distance
to First Down
Raw Distance
Baseline
Relative Distance
Baseline
Average
Baseline
Raw Distance
Baseline
Relative Distance
Baseline
Average
Baseline
-2 12 0.84 0.83 0.84 0.86 0.88 0.87
5 -5 0.74 0.78 0.76 0.71 0.75 0.73
8 -2 0.65 0.71 0.68 0.65 0.70 0.67
10 0 0.65 0.55 0.62 0.58 0.53 0.55
12 2 0.61 0.54 0.57 0.57 0.59 0.58
15 5 0.57 0.59 0.58 0.48 0.48 0.48

With those baselines, we then established a point value for each play, similar to the plus/minus system employed by John Dewan in his analysis of fielding in baseball. Let's take the example of a five-yard pass to the sidelines on first-and-10. As you can see above, the baseline for such a play is .73; since you can only make zero catches or one catch on a play, those are the only values we can use relative to the baseline. A catch in this scenario is worth .27 points (1 catch - .73 expected catches), while an incomplete pass is worth -.73 points (0 catches - .73 expected catches).

After making Excel angry with the million or so queries required to produce a value for each catch, we finally had a distance-adjusted catch rate. For the purposes of giving the metric a simple, easy to remember name, we'll call it Receiver Plus/Minus.

The results? Well, at first they sort of told us what we already knew.

Table 2: Best and Worst Receiver +/-, 2005-2008
Year Player Team Targets Catches Catch
Rate
Rec.
+/-
2007 87-R.Wayne IND 142 104 73.2% 19.23
2008 80-A.Johnson HOU 158 115 72.8% 15.13
2006 88-M.Harrison IND 143 95 66.4% 12.76
2007 83-W.Welker NE 139 112 80.6% 12.55
2007 84-B.Engram SEA 128 94 73.4% 12.45
2006 87-M.Furrey DET 132 98 74.2% 12.04
2005 84-T.Houshmandzadeh CIN 97 75 77.3% 12.04
2005 87-R.Wayne IND 116 83 71.6% 11.59
2008 15-S.Breaston ARI 107 77 72.0% 11.31
2006 87-R.Wayne IND 131 86 65.6% 11.08
 
Year Player Team Targets Catches Catch
Rate
Rec.
+/-
2008 82-D.Bowe KC 155 86 55.5% -9.75
2006 85-G.Jennings GB 96 45 46.9% -9.82
2008 81-T.Owens DAL 135 69 51.1% -10.60
2008 11-Ro.Williams DAL 81 36 44.4% -10.87
2008 86-M.Booker CHI 44 14 31.8% -11.33
2008 87-A.Lelie OAK 41 11 26.8% -11.47
2006 84-J.Galloway TB 129 62 48.1% -11.89
2005 17-P.Burress NYG 144 70 48.6% -15.03
2008 17-B.Edwards CLE 128 55 43.0% -19.75
2006 84-C.Chambers MIA 135 59 43.7% -21.11

The great raw catch rates of guys like Welker and T.J. Housmandzadeh still translate well after adjusting for distance, while the incompetence of Braylon Edwards and Chambers in their respective valley years is held up for all to see.

Of course, it takes volume to accrue large receiving plus/minus figures, one way or another. What if we divide the plus/minus by the number of targets?

Table 3: Best and Worst Receiver +/- Per Target, 2005-2008
(Min. 50 Targets)
Year Player Team Targets Catches Catch
Rate
Rec.
+/-
Rec. +/-
per Target
2008 19-I.Hilliard TB 54 47 87.0% 9.76 0.18
2005 82-J.Witten DAL 75 61 81.3% 10.87 0.14
2008 82-J.Reed BUF 71 56 78.9% 9.91 0.14
2007 87-R.Wayne IND 142 104 73.2% 19.23 0.14
2007 83-H.Miller PIT 60 47 78.3% 7.90 0.13
2005 82-D.Jackson SEA 54 38 70.4% 6.91 0.13
2007 80-A.Johnson HOU 82 60 73.2% 10.21 0.12
2005 84-T.Houshmandzadeh CIN 97 75 77.3% 12.04 0.12
2007 11-A.Gonzalez IND 50 37 74.0% 6.20 0.12
2006 19-D.Henderson NO 50 32 64.0% 6.00 0.12
 
Year Player Team Targets Catches Catch
Rate
Rec.
+/-
Rec. +/-
per Target
2006 80-B.Scaife TEN 51 29 56.9% -5.54 -0.11
2007 83-D.Bennett STL 70 33 47.1% -8.11 -0.12
2008 82-L.Smith PHI 63 37 58.7% -7.35 -0.12
2007 83-K.Colbert CAR 64 32 50.0% -7.73 -0.12
2005 86-M.Booker MIA 72 32 44.4% -8.74 -0.12
2008 11-Ro.Williams DAL 81 36 44.4% -10.87 -0.13
2006 27-L.Johnson KC 61 41 67.2% -8.20 -0.13
2008 17-B.Edwards CLE 128 55 43.0% -19.75 -0.15
2006 84-C.Chambers MIA 135 59 43.7% -21.11 -0.16
2006 81-R.Davis CHI 52 22 42.3% -8.90 -0.17

There's the sort of list we were looking for! The top ten mixes in a variety of players; possession receivers playing over their heads (Ike Hilliard and Josh Reed a year ago), a deep threat that had a better-than-average year (Devery Henderson), and your garden-variety elite receivers (Reggie Wayne, Andre Johnson). The bottom ten features a similar mix, and even throws in Franchise Player for good measure.

That seems like a better fit for what we're trying to measure, but what does it mean? Is it a consistent statistic from year-to-year, or subject to random vagaries and whims? We'll hit those questions in Part II of this analysis, coming later this week.

Posted by: Bill Barnwell on 04 Aug 2009

41 comments, Last at 08 Aug 2009, 10:06pm by tuluse

Comments

1
by TimK :: Tue, 08/04/2009 - 1:40pm

Interesting, looking forward to part II to see what it really means. Year-on-year consistency in this could make a very useful stat.

(As a Denver fan I'm also interested that Brandon Marshall is not showing up in the bottom few for last season, and not surprised to that Ashlie Lelie does...)

2
by Karl Cuba :: Tue, 08/04/2009 - 1:54pm

Would it be possible to arrive at a similar metric for quarterbacks? It would highlight schematic differences between teams.

The difference in average +/- for receivers who have played for different teams or in different systems would also be interesting.

3
by Jacob Stevens (not verified) :: Tue, 08/04/2009 - 2:00pm

This is really great, Bill. At first glance, it looks like even still the quality of the passer has some kind of effect. That might be giving these great WRs too little credit, though. Is Reggie Wayne underrated?

I can't believe you've only been writing at FO for three years. Feels like so much longer.

7
by billsfan :: Tue, 08/04/2009 - 2:21pm

I don't think it's possible for Reggie Wayne to be underrated.

(I also like the Eagles)

26
by El Nino Meon (not verified) :: Tue, 08/04/2009 - 10:03pm

Well, maybe. But there isn't a control for the fact that Reggie Wayne is being thrown to by Peyton Manning. And it isn't exactly unreasonable to think that a 20-yard Peyton pass is a bit more catchable than average.

32
by John (not verified) :: Wed, 08/05/2009 - 2:56pm

And it isn't exactly unreasonable to think that a 20-yard Peyton pass is a bit more catchable than average.

Yeah, I think Peyton reserved his truly unreasonable passes for Marvin Harrison, who made quite a career out of catching them anyway.

Reggie seems to get a fair number of "stick my hands out and there's the ball" passes.

33
by billsfan :: Wed, 08/05/2009 - 4:45pm

Obviously we all agree that it's difficult to separate the performance of a QB from his receivers, but developing a "stick out my hands and there's the ball" rapport with any QB, even Peyton Manning, still takes a very talented receiver.

re: Marvin, my all time favorite is still the one on the right side of the endzone, on his tiptoes, inch(es) from the sideline, falling out of bounds to catch a ball that's easily 3 ft. out already. They had to have practiced that one!

(I also like the Eagles)

34
by John (not verified) :: Wed, 08/05/2009 - 7:27pm

For me, the play that stays with me is when Peyton threw it behind him in the endzone; Marvin sticks out his hand, knocks it up in the air, then turns around and grabs it.

I would seriously consider giving a kidney to have a few more years of a healthy 18 to 88 combo. Watching those two in action was simply amazing.

4
by The Ninjalectual :: Tue, 08/04/2009 - 2:13pm

Franchise Player? What am I missing?

----------------------------------
"Just look at that pumpkin."
-John Madden, looking at the moon.

8
by Brendan Scolari (not verified) :: Tue, 08/04/2009 - 2:25pm

L.J. Smith

29
by Kevin from Philly :: Wed, 08/05/2009 - 2:34pm

The only way L.J. is a franchise player, is if he buys a bunch of McDonalds. Not surprised to see an Eagle in the "worst of" list, but I really expected FredEx or Pinky.

35
by Keith (not verified) :: Thu, 08/06/2009 - 8:41am

You are not from around here, are you? It is a joke about L.J. Smith, and it has nothing to do with his skill.

5
by tuluse :: Tue, 08/04/2009 - 2:15pm

I like where you are going with this, but wouldn't it have been better to use percentage of a first down instead of yards away from the markers?

Going back to you example, I bet it's a lot easier to complete a pass 3 yards short of a first down on 2nd and 5 than it is on 2nd and 15. However, if you used percentage, 12 yards on 2nd and 15 gives you 80% of a first down, which would be a 4 yard pass on 2nd and 5.

6
by AnonymousA (not verified) :: Tue, 08/04/2009 - 2:19pm

This is really interesting -- but is it useful? The goal is to measure how hard what a receiver does is, not how useful. For example, a guy who catches a 30 yard out on 4th and 29 is a hero -- unless it's the last play of the game, he went out at the 12 yard line, and his team lost by 5. Of course the defense gave him that -- they had 12 yards to tackle him and win the game!

Part of this may get washed out with the massive sample size you have. It might be interesting, however, to separate plays that increase and decrease win expectancy.

I also wonder if a little bit o' multiple regression could eliminate the discrete 3D table of baselines and turn them into a happy little curve (this could also be done on the raw data, though a 60,000 point multiple regression may not be super-fast. This would, however, allow the addition of the good/bad play component, above, as well as dealing with low sample size for specific distances). From there, a little bit of analysis could yield some fascinating approximations, e.g. (numbers here obviously wrong, since I don't have the data) "every time you double a pass' distance, you make it 3 times harder on the receiver".

Eager to see where this goes.

9
by billsfan :: Tue, 08/04/2009 - 2:26pm

As a hockey fan, I was a bit disappointed to see +/- as something other than an integer. I also think two decimal places is a bit excessive, at least for how I learned Significant Figures.

Based on the distribution of players and values, this might ultimately be just as effective presented as a whole number.

(I also like the Eagles)

13
by White Rose Duelist :: Tue, 08/04/2009 - 2:58pm

As a hockey fan, I would like +/- (in hockey and elsewhere) renamed. I like my stats giving you some idea what they mean, and to the novice "plus-minus" only tells you that it can be either above or below zero.

No, I don't have a good idea what it should be called instead.

And, failing that, I want to hear something like "Tiger Woods is two strokes back, with a plus-minus of -3."

10
by Brendan Scolari (not verified) :: Tue, 08/04/2009 - 2:30pm

As #6 said, is there really any usefulness to this kind of statistic?

Yes, Andre Johnson and Reggie Wayne are on there, but Randy Moss, Terrell Owens, and Larry Fitzgerald combine for 0 of the 10 best receiving seasons in the last 4 years?

I understand that your measuring efficiency, but if your not showing the best players as having the best seasons I really don't see the point of the statistic. It'd be like if there was a new stat for measuring hitting ability in baseball and Albert Pujols, Manny Ramirez, and Alex Rodriguez don't appear on the top 10 list for the last 4 years. The stat just isn't revealing much of the truth.

12
by shake n bake :: Tue, 08/04/2009 - 2:56pm

It isn't an overall WR ranking. It doesn't include YAC which is a big part of some WRs game and is certainly important.

To adjust your analogy it's like Pujols, Manny and A-rod not being in the top handful in a measure of a certain aspect of hitting like contact% or swings% on pitches outside the zone.

It's possible to be a good or even great player without being in the top handful at every aspect of your job.

14
by Keith (not verified) :: Tue, 08/04/2009 - 3:08pm

What snb said, really. This method measures catch rate and nothing else.

30
by Danish Denver-Fan :: Wed, 08/05/2009 - 2:38pm

I guess you could call it "advanced catchrate" or just "hands-rating"...

11
by Tom Gower :: Tue, 08/04/2009 - 2:40pm

Interesting stuff, Bill.

The one thing I always worry about with things like these is usage rate-that an unspectacular player looks bad because he's a relatively good option (Scaife in 2006 as VY's security blanket) or a good player having a good year looks better than a spectacular player more involved in the offense (compare Devery Henderson 2006 v Randy Moss 2007). Obviously, these concerns are nowhere near unique to WR +/-, and I discount DVOA/DYAR in terms of rating individual WRs and RBs for similar reasons.

15
by Bowl Game Anomaly :: Tue, 08/04/2009 - 3:21pm

People,

This is not a measure of the best WRs. This is a measure of catching ability only. It's essentially an adjusted catch %. It does not measure WR performance as well as DYAR or DVOA but it measures a specific aspect of WR performance very well- that of catching the passes thrown to you.

Essentially, the point of all this is to see that catch % is inadequate when comparing WRs, but this stat allows us to compare "hands" for WRs accross different skill sets. It's interesting, but it doesn't represent any grand innovation.

(edit: Keith beat me.)

16
by starzero :: Tue, 08/04/2009 - 3:31pm

so basically wayne and house were amazing in 2007 and 2005, respectively?

17
by Keith (not verified) :: Tue, 08/04/2009 - 3:37pm

Yes.

27
by Yaguar :: Tue, 08/04/2009 - 10:58pm

Additionally, consider that Wayne did that with Devin Aromashadu and the like playing across from him.

18
by Independent George :: Tue, 08/04/2009 - 3:52pm

The thing about the 'per target' stat that really sticks out for me is that, aside from one clear outlier (Hilliard), #s 2-10 are essentially tied.

19
by Dave B (not verified) :: Tue, 08/04/2009 - 3:56pm

I think this article is useful to see how much the length of the pass effects the AVERAGE catch rate. However I don't think applying it to individual WR's is truly meaningful b/c there are too many other variables that get left out that are very important.

For applying it to individual WR's I think these problems exist.

1. Obviously the amount of coverage and who is covering the WR makes a big difference.

2. I'm not sure leaving in passes marked as overthrown, underthrown, and even passes defensed works when applying it to individual WR's as it is unclear who is it fault between the WR and QB in those cases. At minimum it might be better to weight those cases less.

20
by Keith (not verified) :: Tue, 08/04/2009 - 4:05pm

I think the effort was to determine a "catchable ball" as defined as one that was where it needed, as put there by the quarterback and caught by the receiver. By removing every "uncatchable ball," it helps to figure out that GIVEN A PERFECT THROW, did the receiver actually catch it?

At least, that is my understanding.

21
by Jimmy :: Tue, 08/04/2009 - 5:28pm

80-B.Scaife TEN 51 29 56.9% -5.54 -0.11

I knew he was a dirty under-achiever.

2006 81-R.Davis CHI 52 22 42.3% -8.90 -0.17

Doesn't suprise me in the slightest. What is this guy still doing on the Bears roster?

22
by John (not verified) :: Tue, 08/04/2009 - 6:42pm

I don't know how Indy got so lucky as to have Bill Polian drafting receivers for Peyton Manning to throw to, but thank you Jebus.

31
by Kevin from Philly :: Wed, 08/05/2009 - 2:38pm

Don't forget to thank Andy Reid, who passed over Reggie Wayne (and Ocho Dope-o) in favor of Freddie Mitchell.

23
by Scott_ (not verified) :: Tue, 08/04/2009 - 7:37pm

I would absolutely love to see a video of all the passes thrown to Chris Chambers in 2006. I remember arguing with that & Welker after that season. Something like 30% of the passes thrown to Chambers were uncatchable.

24
by Rich Arpin (not verified) :: Tue, 08/04/2009 - 9:34pm

I wonder how the routes a receiver usually runs play into this? For example, more vertical routes for moss means that he attacks a DB in their back pedal or stride whereas welker runs crossing routes more and it's like he runs away from DB's or towards depending on the coverage of Man or Zone.

25
by mm (not verified) :: Tue, 08/04/2009 - 9:36pm

I think this is probably still capturing a good deal of the QB's affect. If we're looking at all passes thrown within a few yards of the receiver (which seems to be what we're doing), we'd expect Peyton Manning to have a much higher percentage of those passes right at the receiver and a lower percentage 2 yards away.

Could you try and grab receivers who played with different QBs and see how much this rate is consistently better/worse with certain QBs?

That said, this is still interesting stuff. Devery Henderson, especially early in his career, is one of those receivers who makes a lot of highlight-type catches where he'll come down with the ball when its thrown in a hard to catch place (behind him/inbetween 2 defenders, etc.) but will frequently manage to drop the easy to catch passes.

28
by njjetfan12 :: Wed, 08/05/2009 - 1:44am

very interesting stuff here, I'm also looking forward to part 2

36
by CornerBlitz (not verified) :: Fri, 08/07/2009 - 5:37am

1) This is an interesting stat, and highlights (to me) the value of all-around receivers in conventional schemes. Not the chuck-and-duck method used by the Saints and Cardinals, but time-worn offensive philosophies like Indy's and the many ubiquitous West Coast varieties.

2) Fitz and Moss get a lot of looks ridiculously far downfield, which is less and less catchable.

3) I too will miss 18 to 88, and I'm not even a Colts fan. Truly legendary and we were blessed to see it.

4) What's with blaming HCs for bad draft picks? I'd like to see some documentation that Andy Reid wanted Freddie Mitchell over Wayne.

5)Back to the subject at hand: I find it especially interesting that this is really an effective indictment of QBs as well. Two Indy receivers among the best...go figure. Two Miami receivers among the worst...no wonder they're still looking for Marino.

6) Biggest WTF: Larry Johnson? How does a running back figure in the bottom 10, when mostly they catch flares, screens, and quick outs (read: gimmes). That's just shameful.

38
by Vince Verhei :: Sat, 08/08/2009 - 12:24am

On LJ: These numbers are adjusted for distance. So LJ is compared only to other receivers running flares, screens, quick-outs, etc.

39
by CornerBlitz (not verified) :: Sat, 08/08/2009 - 4:37am

Ah...then a fine indictment of KC's QB's as well. Thanks Vince.

37
by armchair journe... :: Fri, 08/07/2009 - 6:55pm

DJ on the best hands list.... no one else finds that surprising?
_______________________________
armchair journeyman quarterback

40
by Bowl Game Anomaly :: Sat, 08/08/2009 - 9:19pm

Well according to the chart he caught 70% of his attempts that season.

41
by tuluse :: Sat, 08/08/2009 - 10:06pm

I think in this case it's not that he has good hands, but that he was getting wide open and Hasselbeck was delivering very accurate passes.