Writers of Pro Football Prospectus 2008

Most Recent FO Features

LockettTyl15.jpg

» Futures: Kansas State WR Tyler Lockett

The Wildcats receiver isn't the best athlete you'll ever see, but Matt Waldman says he could be an effective pro with small improvements in his technique.

07 Aug 2009

Receiving Plus/Minus, Part II

by Bill Barnwell

In Part 1 of our look at a Receiver Plus/Minus metric, I presented a new methodology for analyzing a receiver's success at catching the ball, leaving off at the point of analyzing the year-to-year consistency of the +/- metric.

No matter how you slice it, the year-to-year consistency for the stat isn't very good. The correlation for year-to-year performance amongst all players is 0.13; if we limit it to players who have at least 49 targets in each season, or even players who fit that criteria and remain on the same team, it doesn't help whatsoever.

Applying the same methodology to quarterbacks as opposed to receivers, we found a year-to-year correlation of 0.25 for quarterback with 100 or more attempts in each year; not much, but twice as much as what we saw for wide receivers.

That got us looking at the numbers on a yearly basis by each team, and that's where we found something very interesting. When we look at receivers who were targeted 50 times or more and analyze them strictly in the context of how they did versus the other receivers on their team, we found that plus/minus was markedly similar across the roster, regardless of receiver type.

Keeping in mind that the range of values we saw from the full player pool went from a plus/minus per pass of -0.17 to 0.18 (an absolute difference of .35), the average distance between the maximum and minimum receiver plus/minus for teams with three or more receivers receiving 50 targets or more was 0.1. In other words, most teams saw their values clumped together. That would indicate -- to me at least -- that the primary variables in determining catch rate are the quarterback and/or the offensive scheme, not the wide receiver's hands.

That came out several different ways. Let's look at several examples from last season to see how plus/minus looked across teams:


Table 1: Dolphins +/-, 2008
Year Player Team Targets Catches Catch
Rate
Rec. +/- Rec. +/-
Per Target
2008 19-T.Ginn MIA 89 56 62.9% 2.83 0.03
2008 80-A.Fasano MIA 50 34 68.0% 1.72 0.03
2008 83-G.Camarillo MIA 79 55 69.6% 4.10 0.05
2008 15-D.Bess MIA 75 54 72.0% 2.99 0.04

The Dolphins are a great example of plus/minus being consistent across player types. Despite the difference of nearly ten percent in raw catch rate between Davone Bess and Ted Ginn -- one that expands to 12 percent when we throw back in the "uncatchable" passes we stripped out -- after you adjust for the distance downfield and the situations in which those passes were thrown, the resulting plus/minus is remarkably similar.

The team with the largest difference a year ago was the Dallas Cowboys; a look into their plus/minus reveals a different sort of trend.

Table 2: Cowboys +/-, 2008
Year Player Team Targets Catches Catch
Rate
Rec. +/- Rec. +/-
Per Target
2008 82-J.Witten DAL 118 81 68.6% 3.49 0.03
2008 11-R.Williams DAL 81 36 44.4% -10.87 -0.13
2008 81-T.Owens DAL 135 69 51.1% -10.60 -0.08
2008 84-P.Crayton DAL 65 39 60.0% -2.55 -0.04
2008 24-M.Barber DAL 59 52 88.1% 4.98 0.08

A pretty clear split is evident here; the Cowboys had better-than-average catch rates throwing to their halfback and their tight end, but did not experience the same reliability when hooking up with wide receivers. That effect wasn't as pronounced in 2007, but there were still traces of it; Barber led the team with a 0.07 +/- per target, while Crayton and Witten were in a dead heat at 0.03 and 0.02, respectively, and Owens was almost literally right at zero, having accrued a total of 0.22 +/- over 140 targets.

Even when a team's receivers aren't all positive or negative, though, they seem to be relatively close-knit. Take the 2008 Giants:


Table 3: Giants +/-, 2008
Year Player Team Targets Catches Catch
Rate
Rec. +/- Rec. +/-
Per Target
2008 34-D.Ward NYG 54 41 75.9% -2.16 -0.04
2008 81-A.Toomer NYG 85 48 56.5% -3.08 -0.04
2008 12-S.Smith NYG 79 57 72.2% 4.92 0.06
2008 87-D.Hixon NYG 71 43 60.6% 1.81 0.03
2008 17-P.Burress NYG 66 35 53.0% -3.61 -0.05
2008 89-K.Boss NYG 53 32 60.4% -2.64 -0.05

Unlike the previous two teams, we see a mix of positive and negative numbers on either side of zero; however, the numbers are all relatively close to zero, and the absolute difference between our minimum and maximum is only 0.11. That's been remarkably consistent for the Giants over the past few years, with every one of their 50-plus-target receivers besides Plaxico Burress in 2005 posting a +/- per catch between -0.06 and 0.01.

Using this metric, we can also get a good glimpse at the consistent effectiveness of a pass offense that's not normally considered to be among the league's elite. Take Jacksonville, which was third in pass DVOA in 2007, and 12th a year ago:


Table 4: Jaguars +/-, 2007-2008
Year Player Team Targets Catches Catch
Rate
Rec. +/- Rec. +/-
Per Target
2007 11-Re.Williams JAC 55 38 69.1% 4.26 0.08
2007 19-E.Wilford JAC 71 45 63.4% 2.73 0.04
2007 86-D.Northcutt JAC 70 44 62.9% 1.39 0.02
2007 89-M.Lewis JAC 54 37 68.5% 1.74 0.03
2007 32-M.Jones-Drew JAC 51 40 78.4% 1.66 0.03
 
2008 11-Re.Williams JAC 59 38 64.4% 0.69 0.01
2008 86-D.Northcutt JAC 66 44 66.7% 3.01 0.05
2008 89-M.Lewis JAC 64 41 64.1% -0.76 -0.01
2008 18-M.Jones JAC 102 65 63.7% 4.15 0.04
2008 32-M.Jones-Drew JAC 73 62 84.9% 3.57 0.05

That sort of success despite a shift in roles and some turnover is a testament to the abilities of David Garrard, as well the scheme drawn up by oft-maligned (by us) offensive coordinator Dirk Koetter. It also bodes well for Jacksonville's performance in 2009.

What about players who move to different teams, though? How do their performances change? It's hard to say because of the small sample; there were only 27 instances of a player catching 50 targets in consecutive seasons for different teams from 2005 through 2008. In general -- although the sample is too small to say definitively -- the players seemed to mirror the performance of their new brethren.

To check, we removed each player's stats from his old team, then measured which team had the higher +/- per target in the year before the player moved: the player's old team, or the team he would be joining the following season. In 18 of the 27 instances, it was the player's new team. The next year, once the player was on the new team, he saw his catch rate rise in 12 of those 18 instances. (Five of the nine instances of a player moving to a team with a worse catch rate saw the player's own catch rate decline.)

With that in mind, let's look at the two players who started this whole debate: Chris Chambers and Wes Welker. How has the shift in teams -- and roles -- affected their plus/minus ratings?


Table 5: Chris Chambers +/-, 2005-2008
Year Player Team Targets Catches Catch
Rate
Rec. +/- Rec. +/-
Per Target
2005 84-C.Chambers MIA 149 77 51.7% -6.84 -0.05
2006 84-C.Chambers MIA 135 59 43.7% -21.11 -0.16
2007 89-C.Chambers 2TM 123 66 53.7% -5.53 -0.04
2008 89-C.Chambers SD 58 33 56.9% -0.98 -0.02

Chambers' 2007 figures were helped some by the in-season move to San Diego:


Table 6: Chris Chambers +/-, 2007
Year Player Team Targets Catches Catch
Rate
Rec. +/- Rec. +/-
Per Target
2007 84-C.Chambers MIA 63 31 49.2% -5.87 -0.09
2007 89-C.Chambers SD 60 35 58.3% 0.35 0.01

What we see is a player who had one really awful year (driven by an expansion in his role and usage pattern unsupported by his abilities) and three years where he was just slightly below what might be expected of him, considering how deep the throws to him normally are. My criticisms of Chris Chambers have been somewhat exaggerated, but it's fair to say that I've been excessively harsh on him through underestimating the impact of being the target on so many deep passes.

One interesting note about San Diego; since Chambers arrived, only he (2008) and LaDainian Tomlinson have put up plus/minus figures below zero. In fact, Tomlinson's put up a +/- far below the rest of his teammates in each of the four seasons we have data for. I'd be interested in hearing why you guys think that might be.

As for Welker?


Table 7: Wes Welker +/-, 2005-2008
Year Player Team Targets Catches Catch
Rate
Rec. +/- Rec. +/-
Per Target
2005 83-W.Welker MIA 45 27 60.0% -0.91 -0.02
2006 83-W.Welker MIA 93 67 72.0% 4.27 0.05
2007 83-W.Welker NE 139 112 80.6% 12.55 0.09
2008 83-W.Welker NE 142 112 78.9% 9.23 0.06

Plus/minus reveals a player who was already better than the players surrounding him in Miami. In New England, he is performing at right about the level we'd expect after considering the quality of offense around him and the role in which he is used. The Patriots were correct to identify him as a player who was performing at a superior level to the players surrounding him, and to think that he would benefit from playing in a superior offensive scheme with a far superior quarterback. Note the difference in his +/- with Matt Cassel as his quarterback as opposed to Tom Brady; it's not that cut-and-dry, but it's pretty obvious that Brady has a definite impact.

The final player I'll provide a table for is one who has played for four different offenses in four different years, seen his context change dramatically in each of the four schemes, and had varying success as a result. I present the new poster boy for receiver plus/minus: Donte' Stallworth.


Table 8: Donte' Stallworth +/-, 2005-2008
Year Player Team Targets Catches Catch
Rate
Rec. +/- Rec. +/-
Per Target
2005 83-D.Stallworth NO 110 62 56.4% -3.58 -0.03
2006 18-D.Stallworth PHI 76 38 50.0% -4.20 -0.06
2007 18-D.Stallworth NE 70 46 65.7% 2.32 0.03
2008 18-D.Stallworth CLE 40 17 42.5% -8.25 -0.21

That 2008 is in a smaller sample, but you can see how +/- shifts depending upon the quality of the offense he's in and (to a lesser extent) his health.

So, after breaking down the data, developing a new metric, and then realizing its inadequacies, I feel that it's safe to posit the following:

  • Catch rate has more to do with team- and quarterback-related variables than it does with the "hands" of a wide receiver. Although there are total fluke years like Chambers' 2006 or Ike Hilliard's 2008, the impact of a team's offensive scheme and the quality of the quarterback in that scheme does have a legitimate impact on catch rate.
  • Acquiring a player because he had a good catch rate or +/- in a particular scheme does not mean that he's likely to perform as well in your scheme. You can call this the "D.J. Hackett Effect," in which we saw the former DVOA Darling go from being a player we considered to be a sleeper free agent in 2007 to a street free agent this year.
  • If a player has a +/- far better or worse than the players around him, though, you should expect some of that ability (or lack thereof) to stick in a new spot. Welker was superior to his fellow receivers in both 2005 and 2006; anyone would have gotten better playing for New England, but Welker was a sound player to target as undervalued in his scheme.
  • We need more data. Having three years of players swapping teams isn't enough; we'll know more with three more years of data and Game Charting. So check back in 2012.

Posted by: Bill Barnwell on 07 Aug 2009

46 comments, Last at 15 Aug 2009, 3:03pm by Ken

Comments

1
by Ed (not verified) :: Fri, 08/07/2009 - 12:53pm

Excellent work

Now you can start compiling data on the relative catch rates as related the the QB who is throwing the ball.

:-)

2
by El Miriodor :: Fri, 08/07/2009 - 12:56pm

Interesting and excellent research with a lot of numbers and context Bill. I'm very curious if in the future this could help explain any of the fantasy football shenanigans under Gruden's Tampa regime. Specifically the conventional wisdom goes that the X or flanker in Gruden's offense is tailor made for big fantasy numbers. It would seem that Joey Galloway, Michael Clayton (when he was the flanker before Galloway arrived) and Antonio Bryant last year are considered elite talent when in that system, but this year will be interesting to note how those players do outside it, if indeed receiver catch performance is intricately tied to QB and system.

3
by R O (not verified) :: Fri, 08/07/2009 - 1:01pm

LT's catch rate is almost entirely on Philip Rivers.

While Rivers is pretty accurate mid and long range, if you've watched a lot of the Chargers the last couple of years his short range accuracy (the kind that go to running backs) has been horrendous.

To Rivers' credit he has noticed this on film and is working on it this year. I just hope he is able to improve.

One final note, if you are going to point out Sproles having a better catch rate than LT, I would probably attribute that to Rivers being more effective on designed RB pass plays than on the checkdowns and improvisations that tend to go to LT.

I would be very interested to see a breakdown on catch rate according to range for various QBs around the league. You might see some interesting trends like Rivers short range woes or confirm or debunk myths about QB's like Chad Pennington not being able to get it done down field (I think it was you guys who pointed out he was better than a certain decisionally challenged QB with a reputation for the long ball).

36
by Big Johnson (not verified) :: Sat, 08/08/2009 - 1:23pm

No its not.

LT caught 100 passes one season when they were awful and forever got dubbed as a great pass catcher. Flutie was the qb half that season and brees was the other half. You don't think that LT gets screens lined up for him much like sproles does? He is overrated due to his past accomplishments. One of the greatest runningbacks of all time. Top 5 material. But his time is up. The downside of a runningbacks career usually starts with a decline in the passing game. LT is welcomed as the newest member of that club

4
by AnonymousA (not verified) :: Fri, 08/07/2009 - 1:07pm

Another question: what's the selection bias here? What type of receivers tend to be thrown deep passes? This might explain the baseline, though not the fact that receiver seems to have very little effect on their own catch rate.

5
by Danish Denver-Fan :: Fri, 08/07/2009 - 1:13pm

I absolutely love it when you guys show us some of the metrics-under-development. This gives us a great look at how you guys comes up with it. And also showing us the metrics that are only semi-succesfull.

6
by Keith (not verified) :: Fri, 08/07/2009 - 1:18pm

Interesting stuff. So, given any quarterback, all wide receivers will play similar, unless they are just completely better than everybody else on the team, such as Welker.

Given an amazing quarterback, all wide receivers will play out of their mind, regardless of their perceived skill at catching the ball.

That settles the argument about Manning/Brady -- both had equal impact on their receivers, as far as receptions. The only difference between them is the footwork, route-running, and separation. Fun!

9
by tuluse :: Fri, 08/07/2009 - 1:48pm

This also means getting Cutler was very very good for the Bears, and we don't need to go get him a stud receiver as many in the media would have you believe.

11
by Keith (not verified) :: Fri, 08/07/2009 - 1:58pm

Unless his receivers are down in the "actually suck" category outside of their catching abilities. Though, I feel Hester will be just fine with Cutler, even if he does go out for long balls 90%+.

15
by cjfarls :: Fri, 08/07/2009 - 3:05pm

The problem is, Brandon Marshall had a LOUSY catch rate with Cutler. So Cutler may not really "improve" the Bears receivers in that respect.

Eddie Royal had an extremely good catch rate though... on longer passes than Welker (see FOA)

So I guess the question then is, what did the rest of the Denver receivers do, and what were the team in aggregate plus-minus numbers?

Scheme is also important, so simply adding Cutler may not help if the scheme remains the same.

7
by Dean :: Fri, 08/07/2009 - 1:26pm

I'd be especially curious to see the effect of both QBs changing teams and of teams changing coaches (or, more specifically, systems). What happens to the collective +/- of a group of WRs when some of the component parts change?

23
by Bowl Game Anomaly :: Fri, 08/07/2009 - 6:00pm

Not too many starting QBs change teams (and are the primary starter on both teams). You're looking at a tiny sample. I don't think we'll be able to draw conclusions until the charting project has been in place for quite a while.

Changing offensive systems represents a different problem, because there remains the issue of unfamiliarity. You would expect that a new offensive system isn't going to be as smooth as one which has been in place for a few years.

39
by Dean :: Wed, 08/12/2009 - 10:59am

I suspect you're right about the sample size among the QBs. I'd still like to see the data, though.

With the systems, there's a much greater sample size. And we might see a teams performance improve in year 2 - we might see different offensive systems yielding greater results. We might see specific head coaches and offensive coordinators having having specific performance improvements (or in the case of Norv Turner, the opposite).

8
by Lou :: Fri, 08/07/2009 - 1:28pm

so does this mean catch percentage will be made less important when calculating wr DYAR? I've always thought dividing the blame for an incomplete evenly between qb and wr was absurd, and that the qb is to blame for the majority of incompletes.

10
by lex-town Nathan (not verified) :: Fri, 08/07/2009 - 1:50pm

It will be really interesting to track this formula this coming season for the the Bears and the Broncos. Comparing 08-09 wr +/- for each team. My guess is that Marshall/Royal won't be nearly so successful this coming year and that there will be a noticeable improvement in the bears receiving corps.

12
by R O (not verified) :: Fri, 08/07/2009 - 2:10pm

"My guess is that Marshall/Royal won't be nearly so successful this coming year and that there will be a noticeable improvement in the bears receiving corps."

That is an interesting point. My personal view is that the Bears pedestrian system is going to make Cutler look worse than he did in Denver. The media will blame the receivers, but this story would seem to confirm (at least in part) that the system will be more to blame.

I can't wait to see what actually happens.

19
by Jacob Stevens (not verified) :: Fri, 08/07/2009 - 5:30pm

What's so pedestrian about the Bears' system? Did it hold the 90s triplets Cowboys back? His brother's demonstrated a lot of effectiveness with it. I think Ron Turner is one of the more underappreciated coordinators in football. Mike Tanier's breakdown of the usage of their FB gave me some new respect for it, as well.

22
by R O (not verified) :: Fri, 08/07/2009 - 5:47pm

That is a good point I suppose. But Norv mixes things up with downfield passing well which gives the O the ability to score a lot of points and make comebacks when necessary.

I don't see that with Chicago. They seem content to chug away win or lose. But maybe that was because of the QB, WRs or O-line. Is it the chicken or the egg?

But I also don't expect much out of Orlando Pace. I think he's just about done and won't be much of an upgrade, if any, over what they had last year. I'd be shocked if he actually played all year. I think Cutler is in for a rude awakening after getting spoiled by Ryan Clady last year.

26
by Jacob Stevens (not verified) :: Fri, 08/07/2009 - 6:32pm

I agree with you about Pace. Extra protection will be called for, and Jason McKay and Greg Olsen may not have the same kind of receiving numbers.

But the Bears take their share of downfield shots. I dunno what has kept it from being real dangerous. Obviously in 2006 it was working most of the time.

27
by tuluse :: Fri, 08/07/2009 - 6:55pm

I find it hard to believe you watch much of the Bears if: 1) you don't think Ron Turner takes shots down the field, and 2) you don't think just about any player would be an upgrade on John St Clair.

28
by R O (not verified) :: Fri, 08/07/2009 - 8:05pm

Well, you are correct sir. I didn't watch much Bears football last year.

But I did watch a fair bit of Broncos last year (mostly Chargers though). And I must say the play of the O-line may not be much of a hindrance now that I think about it. Because Jay Cutler throws exceptionally well on the run. He might throw better on the run than he does from the pocket.

So Bears fans had better hope Turner finds a way to move Cutler around in the pocket because that is when he's at his best. But be warned, if the O struggles early in the game or if the D can't get off the field then Cutler gets frustrated VERY quickly and just starts chucking away with half the balls going to the other team.

Bears fans better be ready for a roller coaster ride.

29
by Ambientdonkey :: Fri, 08/07/2009 - 8:16pm

I assure you, Bears fans will be prepared for inconsistent play by the QB.

45
by Shanahans (not verified) :: Thu, 08/13/2009 - 3:22am

As a former resident of both Denver and Elmhurst, IL, I can assure you that Mr Cutler will do nothing to slow down the Chicago Bears fan rollercoaster experience. Cutler will give you great days, true; but there will be repeated moments of such obstinance that you will wonder if the Cubbies' curse somehow wore off on the Bears.

32
by Dan :: Fri, 08/07/2009 - 9:27pm

FOA has a table of DVOA when hurried... let's see... Cutler had a 0.9% DVOA when hurried last year, good for 9th in the league (min 20 attempts). Not bad - worse than his overall DVOA, but better than the Bears passing DVOA.

13
by Scott C :: Fri, 08/07/2009 - 2:22pm

"One interesting note about San Diego; since Chambers arrived, only he (2008) and LaDainian Tomlinson have put up plus/minus figures below zero. In fact, Tomlinson's put up a +/- far below the rest of his teammates in each of the four seasons we have data for. I'd be interested in hearing why you guys think that might be."
Rivers is consistently overthrowing or underthrowing routes to LT in the backfield. He also regularly has his poorest accuracy in the dump-off to LT, not always "uncatchable" but often, a difficult catch.

Rivers himself said so a couple weeks ago:

"The running back route, by far, was the route I missed more than any others,” he said Monday after the Chargers' first training camp workout. “I'm working on being more efficient with our running backs.”

http://www3.signonsandiego.com/stories/2009/jul/27/28chargers192924-char...

So, look for a lot of thrown ahead or behind to LT and other running backs behind the LOS.

37
by Big Johnson (not verified) :: Sat, 08/08/2009 - 1:49pm

Rivers said that to satisfy the insatiable hunger the fans have in making LT the cornerstone of that franchise. He didn't throw behind sproles last year. They should just give it up. LT, dont ruin the chargers chances this year by expecting urself to play at an elite level. Rivers Gates and Jackson are the stars.

44
by WTF? (not verified) :: Wed, 08/12/2009 - 8:14pm

Rivers said it because its true. He didn't throw behind Sproles as much because most of Sproles passes were designed screens instead of check downs or dump offs made under pressure.

LT, dont ruin the chargers chances this year by expecting urself to play at an elite level

Are you honestly trying to tell one of the all time great players to not play his best? Because anyone who has watched LT knows he'll give 100% on every play, and most of the time that 100% is better than whatever the defenders 100% is. And how exactly could he "ruin" the Chargers chances? He's not the coach, he doesn't call the plays.

My only guess at your motivation for writing such inane things is that you're a Raiders fan who is praying LT won't run all over your team AGAIN this year.

14
by ammek :: Fri, 08/07/2009 - 2:47pm

Interesting stuff.

Not sure that the Giants' numbers are so "close-knit"; there's a difference of 0.11 between the best and worst, which is much more significant than, say, the Jags' 2007. And they're really divided into two groups: Smith/Hixon and the rest. What's the median team spread for the four years in your study? Which teams have been (most) consistently plus and most consistently minus?

To help identify the cause of WR plus/minus you could ask if there is any correlation between, say, 'minus' teams and uncatchable passes — something that could indicate a "QB effect"? And how about offensive coordinators that change teams or become head coaches?

Lots of questions — you have my curiosity well pricked!

16
by Rich Conley (not verified) :: Fri, 08/07/2009 - 3:07pm

"Note the difference in his +/- with Matt Cassel as his quarterback as opposed to Tom Brady; it's not that cut-and-dry, but it's pretty obvious that Brady has a definite impact."

I'm not sure I completely agree. I would think the difference is probably much more easily explained by the extremely pared down offense the patriots ran the first 8 weeks of the season, and not by differences in the QB.

When the offense isn't throwing down field at all, its expected that the short passing game will receive more attention.

17
by Pete (not verified) :: Fri, 08/07/2009 - 4:34pm

"That would indicate -- to me at least -- that the primary variables in determining catch rate are the quarterback and/or the offensive scheme, not the wide receiver's hands."

Or one could say that the scouting dept keeps picking the same kind of WR maybe? Just a thought.

18
by Dr. Mooch :: Fri, 08/07/2009 - 4:45pm

So, interesting. A lot of the pessimism reflected in the Buffalo Bills' projection appeared (at least from the accompanying text) to be derived from the belief that TO will have little effect on the passing game, not pulling much coverage from Evans. This belief was based, in no small part, on TO's poor catch rate.

24
by Bowl Game Anomaly :: Fri, 08/07/2009 - 6:10pm

TO is 35. Players do change in ability over the course of their careers. Most of the WRs from TO's draft class (and there were a lot of good ones) are retired or have severely reduced roles. TO is also coming off 3 years with the same team, same QB, and for the last couple of years, same coach and offense. It seems unlikely it was his role that changed, and more likely than he was the one who changed.

25
by Dr. Mooch :: Fri, 08/07/2009 - 6:22pm

So the research suggests that catch rate is more due to the system or quarterback factors than to the specific player, except in cases where it agrees with what you already think?

38
by Bowl Game Anomaly :: Sat, 08/08/2009 - 9:36pm

Um... TO was bad last year. His catch rate was 50% (a bit lower than his career average), but his DVOA was -7.4, 56th in the league (i.e. much worse). That's way lower than his usual performance, as he usually had a quite high DVOA even with a less-than-elite catch rate. He's declining. I don't understand how anyone could disagree with that.

20
by Chris Chambers (not verified) :: Fri, 08/07/2009 - 5:32pm

Thanks a lot Bill. Your apology means a lot to me.

21
by Jacob Stevens (not verified) :: Fri, 08/07/2009 - 5:38pm

I would go so far as to say this might be the most significant piece of research Football Outsiders has done since the Curse of 370. I really think it's on equal ground. I think it's tremendous stuff.

I'm left wondering if this speaks moderately poorly of my beloved Matt Hasselbeck. The DJ Hackett effect notwithstanding, he's had to suffer through a lot of mediocre WR corps, with droppsy endemics, injury endemics, etc. Engram 2007 & Jackson 2005 showed up in these pieces, as standout years for Seattle receivers. They were really nice years.

I still think Hasselbeck's been held back by his receivers, more than the other way around. We just haven't had great receivers. But I also can't see a scenario where all the Reche Caldwells of the corps are swept out and Hasselbeck gets a standout hot read guy and a deep threat, and turns into Tom Brady 2007. I'm also not thinking about the impact of the system, but all in all, it makes me think, Hasselbeck's been as good as he's been, and isn't being dramatically held back by lack of receivers.

30
by mm (not verified) :: Fri, 08/07/2009 - 8:48pm

Does part of this relate to that phenomenon talked about in the Basketball 'extra point' you linked to the other day?

A QB targets his favorite receiver a lot; the more that receiver is targeted, the more the defense focuses on him and his catch rate drops; meanwhile, less popular receivers find the defense pays less attention to them and sees their catch rates rise until the whole team is at a similar level.

I do think QB and system have a lot to do with it, but the above might have an affect. Awfully hard to measure, though. You might try and see how 'surprise' receivers have their catch rates change during the season. For instance, in Marques Colston's rookie season, you might compare his numbers and his fellow Saints in the first 6 games and the last 6 games; you'd expect his catch rate to drop as defenses focused on him...did his fellow receivers see theirs rise?

31
by Anonymous- (not verified) :: Fri, 08/07/2009 - 8:52pm

This is pretty amazing stuff, and could have huge implications on figuring KUBIAK numbers. For instance, according to the numbers we might say that Chad Pennington was a +.02 QB last year, while Tony Romo was a -.06 (on passes to WR). We might also postulate that Steven Smith is +.06 (or whatever) better than the average WR would be in New York. We take the QB adjustment, the WR adjustment, and an estimate of how far down the field most of the players catch attempts are (or even the percentages listed in the WR summaries in the book), and we've got estimated completion percentage.

33
by Marver :: Fri, 08/07/2009 - 10:20pm

Another thing to remember with respect to Philip Rivers: he's been known to purposely throw the ball at his runningback's feet on 'blown-up' plays. A few sniffed out screens where Rivers alertly -- albeit, maybe not value-smart -- throws an uncatchable pass in the vicinity of LT that are registered as 'targets' could significantly harm data. I'd suggest, if you have the time, trying to find an instance of this on tape, going back and counting all targets to LT attributed in said game, and making sure that 'target' isn't being registered. If it is being registered, it's something to look into pertaining to lower-than-expected +/-.

34
by Vincent Verhei :: Sat, 08/08/2009 - 12:20am

Hopefully, those passes would be marked as "thrown away" by our charters, and removed from this study -- see part one for more info.

35
by Noah of Arkadia :: Sat, 08/08/2009 - 9:52am

Peace at last on the Chris Chambers front! Hooray!

Congrats on an excellent piece of research!

40
by Andrew R (not verified) :: Wed, 08/12/2009 - 11:12am

Your work is an interesting start but it would really help to have more of a math guy's perspective on what a .11 between the best and worth on a given team means in the context of this sort of data. It feels like a tiny number but my gut is that in this data it's a big difference. Think about batting average in baseball, the difference between a .300 hitter and a .350 hitter is .05 or half of one tenth. Intuitively it's a small number, on the baseball field it's a huge difference. Also by selecting the guys who were targeted at least 50 times, you're not just selecting the guys who have enough of a sample size to determine anything, you're also selecting the best receivers on the team. The guys the coaches and QB thought were worth throwing to frequently.

Also, think about the fact that they are often on the field together and their individual differences in talent have an effect on how the unit performs. It's wholly possible than their individual rates won't perfectly reflect their differences in talent because the defense is keying in on the best receivers. You still end up targeting your best receiver the most often because he's the most talented but it may end up creating opportunities for the other guys on the field. Remember the FO article the estimated the numbers of Welker and Moss based on prior history of the NE offense and how is was amazingly wrong because no one predicted the NE offense with Moss & Welker would be historically great? One of the lessons of this sort of analysis over the years is that the interplay of players has huge impacts on individual numbers.

42
by Bowl Game Anomaly :: Wed, 08/12/2009 - 3:59pm

I'm not a baseball guy, but I'll take a shot:

The difference between the best and worst rec +/- per target over the 4 years of the charting project is .18- -.17=.35. The difference between the best average in MLB and the worst might be .375-.200=.175. The typical range per team in football is .1 +/- per target. As you say, a range of 50 points of batting average in baseball is rather large (although probably smaller than the average difference between the best and worst guys on a team). So, let's compare those numbers. For football, .1/.35=28.6%. For baseball, .050/.175=28.6%. That's the exact same number. so it seems that in football a range of .1 +/- per target within a team is as meaningful as a range of 50 points of batting average in baseball.

However,

Here's where it gets interesting. The range in rec. +/- per target on a given team is about equivalent to 50 points of batting average in baseball, and that is relevant in evaluating performance. But it is not relevant in evaluating talent. Because rec. +/- per target only has a year-to-year correlation coefficient of .11, which is tiny, it's clear that the stat does not measure a player's intrinsic talent. (For comparison, batting average has a year-to-year correlation coefficient of .39, from what I was able to find.) .11 squared is .012, which means that the relationship between between a receiver's +/- per target from one year to the next only explains a minuscule 1.2% of the variance, leaving 98.8% of the variance to be explained by other factors. In other words, a receiver's +/- per target in one year gives us only a small hint about what his +/- per target will be in the next year, and it tells us virtually nothing about why his performance will be what it will, meaning it tells us almost nothing about his "hands" independent of how he was used and who his teammates were.

As to the 50 target cutoff, it's not that big of a deal in terms of selecting for talent. You don't have to be very good to get 50 targets. Ike Hilliard, Hank Basket, Jerheme Urban, Jason Avant, Jordy Nelson, Koren Robinson, Deion Branch, Rashied Davis, Mark Bradley, Robert Royal, Dominic Rhodes, Warrick Dunn, Tim Hightower, Kevin Smith and Steve Slaton all had at least 50 targets last season (and Brandon Lloyd, Marty Booker, and Vernon Davis had 49). That list includes a bunch of 3rd and 4th WR options, two of the worst receiving TEs in the league, and some old backup RBs and rookies. Most of these players had poor receiving DVOA (including Slaton).

41
by Dan Tomlinson (not verified) :: Wed, 08/12/2009 - 2:05pm

I suspect this study really shows one thing - that football is a game of dynamic adjustments on both sides of the ball. If receiver A is way better than receiver B, two things are going to happen. First, the offense is going to force-feed throws to receiver A. This is going to mean that receiver A gets the ball thrown to him even when he's not really open, while receiver B only gets the ball when he's wide open. Second, defenses are going to shade coverage toward receiver A and away from receiver B. Both adjustments will tend to drive down the completion % to receiver A and raise the completion % to receiver B.

If the above is true, then this stat is going to create as much confusion as it sheds light. Also, teams with an abnormally large spread between receivers indicates that somebody on that roster is underrated/overrated. In that context, I would be worried in Buffalo with TO coming to town.

43
by Eddo :: Wed, 08/12/2009 - 5:46pm

In the same comment, you suggest that:

1. If a receiver ("A") is much better than his teammates, his numbers will look worse, because his QB will try to force him the ball and defenses will be keying on him.

2. Because of point #1, you'd be worried if you were Buffalo, because TO is coming to town.

Doesn't #2 fly in the face of #1? If anything, TO is a guy that defenses key on, and also, due to his whining, may get forced the ball more often. Your first point would seem to imply that TO is underrated according to this metric, and that Buffalo got a pretty good deal.

46
by Ken (not verified) :: Sat, 08/15/2009 - 3:03pm

I feel like this exactly what would be expected. All receivers should have pretty similar +/- ratings. It just means the quarterback is approaching equilibrium in his distribution of passes. If Jason Witten has a significantly higher catch rate than TO, it means that he should be targeted more and Owens less. It is just game theory, similar to much talked about situations in the pass/run selection on 3rd or 4th and short.

That said, as players switch teams, it is a useful thing for fantasy football.