Writers of Pro Football Prospectus 2008

19 Jul 2010

Scaled Plus-Minus

by Bill Barnwell

In the article that introduces our plus-minus statistic for receiving in Football Outsiders Almanac 2010, I mention that I'll be publishing a scaled plus-minus figure on the website. This is that article.

For those who haven't read the book yet, here's a quick explanation. The plus-minus figure we introduced in these articles is included in the player comment tables for each wide receiver and tight end in the FOA player comments. We didn't have room in the running back tables, but the top ten and bottom ten running backs are listed in the appendix.

The figures provided are relative to that player's target total, not to a league-wide scale. That makes sense in the context of looking at an individual player, but it makes comparing different players with dramatically disparate usage patterns tricky. A guy with a plus-minus of -5.0 catches on 130 targets didn't have a great year, but he did far better than a player who accrued the same -5.0 plus-minus on 55 targets. Had that second player received the same 130 targets and continued to catch passes at a similarly poor level, he would have ended up with a plus-minus of -11.8 catches.

With that in mind, I wanted to offer plus-minus as a scaled metric, commenting on notable performances. I'll be taking all the plus-minus figures from the past few seasons and scaling them to a rate of 150 targets.

For players with a small sample of passes, this naturally produces an absurdly high figure. I sincerely doubt that Dan Kreider -- who caught none of the four "catchable" passes thrown to him in 2008 -- would produce a plus-minus of -120.9 if he was actually thrown 150 passes. (This would either be a terrible offense, or a season's worth of third-and-longs for the Carolina Panthers.) Therefore, I'll limit the discussion to players who only had 30 or more relevant targets in the given season.

Using raw plus-minus, last year's leader was unsurprisingly a player who gets a large number of targets: Indianapolis' Reggie Wayne, who had a plus-minus of 16.9 on 134 targets. (Remember, this target total differs from the one on his player page because we eliminate a group of uncatchable targets.) Because Wayne only gets a slight bump from our scaling, though, Wayne's scaled plus-minus is 18.9; still superb, but only good for sixth in the league in 2009.

So who led the league in scaled plus-minus? One of the league's most surprising performers in 2009:

Year Player Pos Team Targets Recepts +/- +/- per
150 Targets
2009 Robert Meachem WR NO 61 45 10.6 26.0
2009 Jacoby Jones WR HOU 35 27 6.0 25.5
2009 Benjamin Watson TE NE 39 29 5.4 20.9
2009 Kevin Walter WR HOU 67 53 9.0 20.2
2009 Greg Camarillo WR MIA 68 50 8.9 19.6
2009 Reggie Wayne WR IND 134 100 16.9 18.9
2009 Heath Miller TE PIT 91 76 11.4 18.8
2009 Jason Avant WR PHI 55 41 6.7 18.2
2009 Jermichael Finley TE GB 71 55 8.0 16.9
2009 Rock Cartwright RB WAS 30 27 3.4 16.8

There are several major 2009 surprises in that group, notably league leader Robert Meachem. A 70 percent catch rate is extremely likely to regress towards the mean, even for a possession receiver, but 30 percent of the passes thrown to Meachem last year were bombs. Given his target package, his expected catch rate was something like 56 percent; even Drew Brees can't make up that difference.

Behind him is one of this year's Top 25 Prospects, Jacoby Jones, and two spots behind him is fellow Texans wideout Kevin Walter. Naturally, they get a bump through playing across from elite wide receiver Andre Johnson, whose plus-minus per 150 targets (4.1) isn't particularly impressive. Interestingly enough, while you might expect that bump to be consistent, this method of analyzing catch rates doesn't always suggest that to be the case. In 2006, it was markedly similar; Johnson had a scaled plus-minus of 2.6 while Eric Moulds was among the league leaders at 17.0. A year later, Johnson was at 17.8, twice the figure of anyone else on the team. A year ago, all four qualifying targets -- Owen Daniels, Walter, Johnson, and Steve Slaton -- had scaled plus-minus figures between 10.7 and 14.2. That's three totally different statistical signatures in three years.

What happened to players with excellent scaled plus-minus figures from a year ago, though? Were they able to maintain their impressive rates? The evidence is inconclusive. Of the top ten players from a year ago, two (Ike Hilliard and David Martin) did not catch a single pass, and two more (Thomas Jones and Jason Hill) did not get 30 targets. The other six were a mixed bag; league leader Malcom Floyd (a whopping scaled plus-minus of 35.1) was down to 10.7 last season. Steve Breaston maintained his rate, but players like Josh Reed (21.3 to -5.7) and Devery Henderson (17.9 to 4.9) exhibited serious dropoffs, and even those blessed with the magic of Favre declined: Visanthe Shiancoe went from 17.3 to 8.6, while Chester Taylor's figure dropped from 16.4 to -2.6. The year-to-year correlation for receivers with 30+ targets in consecutive seasons is .22, which isn't very strong at all.

Floyd's figure is the highest over the four-year stretch. There are some other interesting players in the top ten that we'll get to in a later article.

On the flip side, the eternal battle of stone hands between Ashley Lelie and Darrius Heyward-Bey that we discuss in DHB's player comment really comes into focus with these scaled figures. In this analysis, Lelie finishes with the worst figure of the past four seasons, at a gruesome -43.9; Heyward-Bey is just behind, at -42.7. The rest of the top ten for 2009:

Year Player Pos Team Targets Recepts +/- +/- per
150 Targets
2009 Darrius Heyward-Bey WR OAK 37 9 -10.5 -42.7
2009 Michael Clayton WR TB 45 16 -11.2 -37.4
2009 Deon Butler WR SEA 40 15 -8.5 -32.0
2009 Jerramy Stevens TE TB 31 15 -6.3 -30.4
2009 Chansi Stuckey WR 2TM 64 30 -11.8 -27.6
2009 Louis Murphy WR OAK 89 34 -15.6 -26.2
2009 Bryant Johnson WR DET 83 35 -12.8 -23.2
2009 Mohamed Massaquoi WR CLE 87 34 -12.2 -21.0
2009 Roy Williams WR DAL 80 38 -10.3 -19.4
2009 Randy McMichael TE STL 59 34 -7.2 -18.2

The only player in that group who was part of a passing game you wouldn't be embarrassed to show your family is Roy Williams. That's no surprise on all counts. I subjectively thought Louis Murphy played reasonably well for a player in the Raiders' offense last year, but he had a pretty gruesome plus-minus figure.

Most of the players in the bottom ten from a year ago dropped off of league radar. Only two -- Braylon Edwards and Adrian Peterson -- had 30 or more targets in 2009. The 11th-ranked player in 2008 was Benjamin Watson, who you'll note was in the top ten in 2009. In 2007, only three of the top ten made it to 30 targets; that suggests some impending attrition for borderline veterans like Jerramy Stevens, Bryant Johnson, and Randy McMichael.

Of course, there's still much more work to be done. Much like Dean Oliver did for his sport in Basketball On Paper, we need to analyze and chart the relationship between efficiency and usage. Next, though, I'll be tackling another problem: how well a player caught passes compared to the rest of his teammates.

Posted by: Bill Barnwell on 19 Jul 2010

19 comments, Last at 22 Jul 2010, 9:19pm by guy

Comments

1
by Dean :: Mon, 07/19/2010 - 2:04pm

"how well a player caught passes compared to the rest of his teammates"

Been waiting for years for this! Seems like years, anyway.

I'd also be interested to see year over year for situations where the QB changes.

2
by spenczar :: Mon, 07/19/2010 - 2:14pm

Cool. Seems like a really useful stat. It seems like the "possession" receivers still have a slight edge over the "speedy burner" receivers, though. Maybe there's some way to fix that by making the calculation of the +/- baseline a little more sophisticated? I'm not sure I believe that the total distance of the pass and the distance to the first down marker should be given equal weight. Weighting the total distance of the pass a little to be, say, 2/3 of the final baseline might help.

Also, is there any way to measure defensive ability using this metric? Maybe a "total +/- of opponents while player is on the field" or a "total +/- of receivers the defender was responsible for." The first one is no doubt calculable but of dubious value, and the second probably suffers from either small sample size or imprecise data (or both!) but something in that direction could be useful.

3
by dryheat :: Mon, 07/19/2010 - 2:19pm

I'm going to go ahead and say Ben Watson is a huge beneficiary of the scaling. I didn't think the metric where his hands would be held in such high regard had been conceived yet. I didn't think that metric's creator would have been conceived yet. I didn't think that creator's father had been...well, you seen where I'm going with this.

For once, I hope I don't understand what is going on here. I've watched every game he's played in the NFL. A fifty-cent wristwatch has more reliable hands.

4
by Rich Conley (not verified) :: Mon, 07/19/2010 - 3:59pm

And yet Watson has had a way above average catch rate his whole career.

5
by dryheat :: Mon, 07/19/2010 - 4:02pm

According to whom or what? In 30+ years of watching the Patriots, I have never seen a WR/TE with such poor hands. Brady clearly lost faith in him last year and used him as a last resort.

6
by Vince Verhei :: Mon, 07/19/2010 - 5:07pm

Watson led all tight ends in receiving DVOA last season, and he was seventh in 2007. He was pretty horrible in 2008. His +/- figures are similar.

7
by Shattenjager :: Mon, 07/19/2010 - 5:10pm

For every season in which he has at least 25 passes, here are Watson's number of passes, catch rate, and rank in catch rate among TEs with at least 25 passes, all according to FO*:

2009: 41 passes, 71% catch rate (rank t-11)
2008: 47 passes, 47% catch rate (rank 42)
2007: 49 passes, 73% catch rate (rank t-8)
2006: 91 passes, 54% catch rate (rank 36)
2005: 54 passes, 54% catch rate (rank 41)

*They are not ranked by catch rate on the list, so I had to count them--doing that is certainly not error-free, so it would not shock me if I'm a place or two off somewhere.

8
by Thok :: Mon, 07/19/2010 - 8:39pm

Wouldn't number of standard deviations above/below average WR make more sense than scaled plus-minus? Or is that too hard to implement? That captures the regression to the mean effect that scaling ignores and allows you to give a value to outliers like Kreider that isn't ridiculous. (Granted, that value has fairly little meaning, but still...)

9
by Joseph :: Mon, 07/19/2010 - 11:20pm

The reason that Devery Henderson dropped off is because he took more of a possession receiver role, and Meachem took over the big play role. I think a FO writer commented that Meachem led all receivers in DVOA this year. Consider that Henderson was #1 in DVOA in 2006 & 2008, and was 12th & 17th in DYAR in those 2 years, also.
"Given his target package, his expected catch rate was something like 56 percent; even Drew Brees can't make up that difference."--Um, yes he can, and he does. See the stats above. Don't forget that he completed over 70% of his passes, setting a record in that regard.

14
by Bill Barnwell :: Tue, 07/20/2010 - 4:02pm

Drew Brees completed 70 percent of his total passes, but he doesn't complete 70 percent of his passes regardless of where they were thrown. Meachem runs deeper routes, which yields an expected completion percentage rate that's below Brees's OVERALL completion percentage, even if it's higher than the league average expected rate of 56 percent.

10
by Bobman :: Tue, 07/20/2010 - 12:12am

Bill,

I think this is pretty cool stuff. Will we now be seeing this as a stat for WRs on the FO website?

Keep up the good work.

15
by Bill Barnwell :: Tue, 07/20/2010 - 4:03pm

You'll see it in each year's book. It won't be done on a week-by-week basis because it uses charting data, which is usually a week or two behind.

11
by Jimmy :: Tue, 07/20/2010 - 10:27am

Is 30 targets a reasonable place to cut off the data? Two targets a game doesn't seem all that many and may fall foul of (now familiar) sample size issues. I don't have the data and so have no idea what the tables would look like if you moved the cut off to say 64 (or four targets a game). It would limit the number of players you can look at but might provide more valid conclusions.

12
by Big Johnson :: Tue, 07/20/2010 - 12:48pm

or why not add an extra tier. the reason for 30 targets is extremely small sample size and will undoubtedly lead the league in dvoa and in worse dvoa. But why not have a medium tier of 2-5 targets and call it the limited role. You could have the small sample size tier of 0-2 targets, a limited #2 role 2-5 target tier, and the star tier of 5+. This would make sense because it would compare apples to apples instead of apples to lemurs.

13
by Bobman :: Tue, 07/20/2010 - 2:54pm

I like that idea, but I think you meant apples to orangutans... scientifically the primate that is closest in DNA, color, and spelling to an orange.

17
by Big Johnson :: Tue, 07/20/2010 - 5:30pm

haha i definately did. good catch. biologically speaking, orangutan meat is very popish, similar in consistency to an orange pulp or a shrimp. This would support your DNA assertion.

16
by Bill Barnwell :: Tue, 07/20/2010 - 4:07pm

It would just make for different lists. The data would still be the same; plus-minus is calculated for all players, even those that had fewer than 30 targets. The only difference is reporting.

18
by nat :: Wed, 07/21/2010 - 6:58am

I suspect that cutting off at anything much higher than 30 would mean that many teams would not have a TE on the list. Not a big deal, but it would be annoying if you were trying to compare TEs. It helps that they show the sample size, so we can draw our own conclusions about how much to trust each player's rating.

19
by guy (not verified) :: Thu, 07/22/2010 - 9:19pm

The Raider WRs will score low because of the extremely innaccurate passing of Jamarcus Russell. It was not NFL caliber.