Playmaker Score 2014
Guest Column by Nathan Forster
Long-time followers of Playmaker Score may notice a different name now connected to Football Outsiders' projection system for college wide receivers. I'm Nathan Forster, the creator of SackSEER, which is a similar model used to identify likely successes and failures on defensive edge rushers. Vince Verhei, creator of Playmaker Score, has retired from updating the system, and I'm talking over to rebuild Playmaker so it has the same dual-output as SackSEER, with both a "Playmaker Projection" and a "Playmaker Rating."
Playmaker Projection projects the wide receiver prospect's average NFL regular season receiving yards per season through the first five years of the player's career. Playmaker Projection incorporates the projected draft round per NFL Draft Scout as an additional factor in order to create a list of prospects that balances conventional wisdom with the factors identified by Playmaker. Playmaker Rating, on the other hand, uses the statistical trends identified by Playmaker only. Playmaker Rating is expressed in terms of the percentage of historical prospects that the player beats in terms of these statistical trends. So, for example, a Playmaker Rating of 75 percent would mean that the prospect is stronger than 75 percent of the prospects in Playmaker's database based on the trends identified by Playmaker.
Many of the factors that make-up New Playmaker Score will be familiar to long-time followers, but others are new. There will be a much longer article in Football Outsiders Almanac 2014 explaining the reasoning behind these changes and why they will make Playmaker Score more accurate going forward, but for now let's just go over the basics. The new Playmaker Score consists of the following elements:
- The wide receiver prospect's best or "peak" season for receiving yards per team attempt (i.e. a wide receiver with 1,000 receiving yards whose team passed 400 times would score a "2.50");
- The wide receiver prospect's peak season for receiving touchdowns per team attempt;
- The difference between the prospect's peak season for receiving touchdowns per team and the prospect's most recent season for receiving touchdowns per team attempt (this factor is simply "0" for a player whose peak season was his most recent season");
- The wide receiver's vertical jump from pre-draft workouts;
- A binary variable that rewards players who enter the draft as underclassmen and punishes those who exhaust their college eligibility;
- The wide receiver's college career yards per reception; and
- The wide receiver's rushing attempts per game during their peak season for receiving yards per team attempt.
Some other things that we looked at that did not improve the model are:
- Forty-yard dash time;
- Height and weight;
- More holistic measures of the prospect's college career, as opposed to just the prospect's "peak" season. It turns out that the prospect's peak season (with an adjustment for players who regressed from the peak season in their final year) is the best way to measure wide receiver prospects;
- Yards per team reception and touchdowns per team reception;
- A number of measures to adjust for the quality of quarterbacks; and
- Measurements for strength of conference.
The previous version of Playmaker had an r-squared of 0.25, meaning that 25 percent of the variation for wide receiver success was accounted for by the model. New Playmaker Rating increases that r-squared to 0.345. Playmaker Projection increases the r-squared to 0.434.
Here is a list of the top 20 wide receivers from 1996-2011 by Playmaker Rating (we will add receivers to the database from the 2012 and 2013 drafts once we are three years removed from those drafts):
|Top 20 Wide Receivers by Playmaker Rating, 1996-2011|
|Player||Year||Rnd||Pick||College|| Actual NFL Yards/Season
(through Y5 only)
|Demaryius Thomas||2010||1||22||Georgia Tech||862.5|
|Terry Glenn||1996||1||7||Ohio St.||893.0|
|Charles Rogers||2003||1||2||Michigan St.||88.0|
|Calvin Johnson||2007||1||2||Georgia Tech||1174.4|
|Golden Tate||2010||2||60||Notre Dame||509.7|
|Dez Bryant||2010||1||24||Oklahoma St.||957.8|
|Player||Year||Rnd||Pick||College|| Actual NFL Yards/Season
(through Y5 only)
|Hakeem Nicks||2009||1||29||North Carolina||924.4|
|Andre' Davis||2002||2||47||Virginia Tech||323.0|
|David Boston||1999||1||8||Ohio St.||923.8|
|Santonio Holmes||2006||1||25||Ohio St.||916.2|
|Koren Robinson||2001||1||9||North Carolina St.||702.8|
|Chris Henry||2005||3||83||West Virginia||365.2|
There are a couple of lessons from this chart. One is that Playmaker Rating, despite being completely ignorant about draft position, rarely breaks too strongly from conventional wisdom. Except for two third-round picks and three second-round picks, the list is composed completely of players who were taken in the first-round of the NFL Draft. Indeed, the most highly rated fourth-rounder is Justin McCareins, Playmaker's 37th highest ranked player.
Second, Playmaker is only okay at identifying great players. The list above includes future Hall of Famers like Randy Moss, Calvin Johnson, Marvin Harrison, and Larry Fitzgerald, but is nonetheless missing greats such as A.J. Green, Torry Holt, and Andre Johnson, whose Playmaker Ratings were respectable but not nearly as strong as their draft position and talent level warranted. There are also several clunkers in this list, most notably David Terrell and Charles Rogers. You can also probably add Stephen Hill to that list too-he had a top ten Playmaker Rating and already looks like a bust with the New York Jets.
The next chart shows the first-round picks with the worst Playmaker Ratings from 1996-2011:
|Worst First-Round Wide Receivers by Playmaker Rating, 1996-2011|
|Player||Year||Rnd||Pick||College||Playmaker Rating|| Actual NFL Yards/Season
(through Y5 only)
|Bryant Johnson||2003||1||17||Penn St.||37.8%||535.0|
|R. Jay Soward||2000||1||29||USC||43.7%||30.8|
|Michael Jenkins||2004||1||29||Ohio St.||64.1%||474.4|
Reggie Wayne and Santana Moss are two of the most obvious misses by the new Playmaker Score, but it's not a coincidence that they both played for the University of Miami at the same time. The 2001 Hurricanes were loaded with NFL talent. They are the only team in the data set with two first-round wide receivers in the same draft, and moreover, the team also included tight end Jeremy Shockey and a very young Andre Johnson. Moss and Wayne each stole receptions from one another, and so it's unsurprising that they are considerably underrated by Playmaker. This "double prospect" effect is something we're looking to study more in the future.
[ad placeholder 3]
One thing that Playmaker's best and worst lists teach us is that Playmaker Score is not, and can never be, the be-all-end-all of wide receiver analysis. The numbers may be a good starting point in setting a baseline as to the strength of a wide receiver prospect, but the evaluator should not turn off his common sense.
This year, Playmaker agrees with conventional wisdom that this year's Draft is an exceptionally strong class for finding wide receiver talent. Indeed, Playmaker likes all of the receivers that are likely to be selected in the first round. However, according to Playmaker, the real strength of this wide receiver class is not the players on the top, but rather, its unprecedented depth. This year, there may be as many as 15 wide receivers drafted that have Playmaker Ratings of 80 percent or more. No other year has had more than eight such players.
Here's a look at Playmaker Score for five projected first-round picks, plus a player who stands out as a potential bust and one who stands out as a potential sleeper.
Brandin Cooks, Oregon State
Playmaker Projection: 638.0 Yards
Playmaker Score: 95.8%
The hype around Brandin Cooks has been growing over the past couple of weeks, and Playmaker believes that it is well-deserved. (How appropriate of our good friend Robert Mays to writea nice big feature on Cooks at Grantland today.) Cooks recorded 1,730 receiving yards his junior year, which is still a strong number even though the Oregon State Beavers dropped back to pass 625 times. Moreover, Cooks had 32 rushing attempts last year, more than any other prospect in his class, suggesting that his receiving numbers would have been even higher had he not proven to be so versatile.
The knock on Cooks is his small-ish frame, but he is heavier than both DeSean Jackson and Steve Smith were when they were drafted. A greater concern is his reported inability to get off the line cleanly, as he reportedly has trouble with physical defenders and NFL cornerbacks will be much more physical than those native to the pass-happy Pac-12.
(Ed. Note: a number of people have asked how the top of this year's class compares to the All-Time Playmaker Top 20 listed above. Cooks is the only player who would make the top 20; his Playmaker Score is narrowly ahead of Antonio Bryant (95.7%) and Chris Henry (95.5%).
Odell Beckham Jr., LSU
Playmaker Projection: 608.4 Yards
Playmaker Score: 93.6%
If there was ever a wide receiver who could be considered a sure first-round pick and still fly under the radar, it would be Odell Beckham Jr. Few are talking about Beckham because he is typically ranked behind Sammy Watkins, Mike Evans, and Marqise Lee, but he is a legitimate prospect in his own right.
Beckham had 1,152 receiving yards and eight touchdowns as a junior, which does not seem impressive until you consider that LSU passed the ball 326 times (and one of those pass attempts was Beckham's). As a result, Beckham actually outgunned his more highly regarded draftmates in yards per team attempt and touchdowns per team attempt last season (with the exception of Mike Evans, whom he tied in touchdowns per team attempt). Beckham also had a strong Combine performance and had a high average yards per catch.
None of the non-Playmaker factors seem to militate against drafting Beckham either. Beckham's domination of his passing game is all the more impressive considering that he shared the field with fellow pro prospect Jarvis Landry, who likely vultured a number of Beckham's catches and touchdowns. LSU's quarterback, Zach Mettenberger, is possibly an NFL talent, but he's certainly not a Peyton Manning-level quarterback who could make a talent-poor wide receiver look as good as Beckham did in 2013. A team in need of a wide receiver at the end of the first-round can draft Beckham with confidence.
Mike Evans, Texas A&M
Playmaker Projection: 558.4 Yards
Playmaker Score: 86.8%
Mike Evans may have an unusual combination of size and speed, but according to the factors that Playmaker Score cares about, Evans is a pretty typical first-round wide receiver prospect. Which is to say that his numbers were good, but not in any universe that would invite Calvin Johnson or Randy Moss comparisons.
What is more interesting about Evans are the potential factors that might cause Playmaker to overrate or underrate him. Evans might arguably be overrated because he had the privilege of playing for Johnny Manziel, who completed nearly 70 percent of his passes last season and may be the most highly drafted quarterback on Thursday. However, punishing wide receivers who had highly efficient quarterbacks does not seem to improve the model. Evans might arguably be underrated because, while Playmaker adjusts for underclassmen, Evans is really busting out early: Evans is a redshirt sophomore, rather than the more typical underclassman, who is a junior. Accordingly, the argument might be that Evans should get an extra bump in addition to the typical underclassman bump because he developed so quickly, and thus, would have likely been even more dominant had he stayed in school for one more year. The limited amount of data, however, does not exactly support that conclusion. Larry Fitzgerald, who entered the draft as a redshirt sophomore, has been truly excellent, but Charles Rogers and Mike Williams, two players who were effectively redshirt sophomores even though they may have technically been juniors when drafted, became humongous busts.
Sammy Watkins, Clemson
Playmaker Projection: 543.7 Yards
Playmaker Score: 83.8%
For someone familiar with Playmaker Score, Sammy Watkins might be the most inscrutable prospect in this draft. Has Playmaker identified trends that expose Watkins as being slightly overrated? Or is Playmaker giving evaluators an opportunity to overthink a prospect whose tape verifies that he is the best wide receiver prospect since A.J. Green and Julio Jones -- who, coincidentally, Playmaker also underestimated a little bit?
[ad placeholder 4]
Watkins' 2013 numbers look good in the aggregate -- 1,464 yards and 12 touchdowns -- but are somewhat less impressive when you discover that Clemson put the ball in the air nearly 500 times. On the one hand, surely some of Watkins' potential catches went to Martavis Bryant, who broke out with 828 receiving yards as a junior and is a potential second-round pick in this year's draft. On the other hand, Bryant's 22.2 yards per catch put Watkins' 14.1 yards per catch to shame, suggesting that maybe it wasn't the style of offense at Clemson that gave Watkins numbers more appropriate to a possession receiver. An examination of Watkins' quarterback situation is similarly unavailing. Tajh Boyd has looked absolutely awful in pre-draft workouts and is probably, at best, a 50-50 shot to be drafted. However, given that most college quarterbacks do not even smell the end of the seventh round, it's perhaps being too charitable to Watkins to give him a bump just because his quarterback might not get drafted.
One caveat to add to Watkins' projection is that Playmaker Score uses his junior season but arguably should use his freshman season instead. Watkins was less prolific catching the ball as a freshman, but he had a many more rushing attempts. Thus, Watkins would have a stronger projection if his freshman year was used (600.2, 88.9%) but his junior year is used because Playmaker is designed to use a player's topreceiving season, not his top offensive season overall.
A team that is confident about what it sees on tape from Watkins should not hesitate to draft him. His Playmaker Scores are pretty good. They just happen to fall short of his hype, and a little short of the Playmaker Scores of other top prospects in his class.
Marqise Lee, USC
Playmaker Projection: 525.6 Yards
Playmaker Score: 90.8%
The most noteworthy factor for Marqise Lee is the difference between his production in his "peak" season and his production last year. As a sophomore, Lee recorded 1,721 receiving yards with 14 touchdowns and still managed to squeeze in 13 rushing attempts. That guy would undoubtedly be the strongest prospect in this class from a Playmaker perspective. However, that's not what teams are looking at here, as Lee crashed to the Earth in a big way in 2013, recording only 791 receiving yards with four touchdowns.
On balance, Playmaker likes Lee and only places him behind Evans and Watkins because his draft stock has slipped a bit in the eyes of draftniks recently (NFL Draft Scout projects him as a "1-2" rather than a "1"). However, scouts have been pretty good at sorting out receivers who turn in poor performances in their final seasons. Dez Bryant and Torry Holt were first-round picks, but lesser talents who experienced production drops, such as Lee Mays and Eugene Baker, were allowed to drop to the later rounds. So the fact that Lee is still in relatively high regards with scouts could suggest that he may be closer to the guy who had 1,721 receiving yards than the one who had only 791.
Potential Bust Alert: Robert Herron, Wyoming
Projected Round: 3-4
Playmaker Projection: 171.2 Yards
Playmaker Score: 24.0%
Picking a potential bust this year was difficult, given that Playmaker is fairly high on all of the wide receivers slated to go in the first two rounds. So the award goes to Robert Herron, who scouts think should come off the board in the middle of the draft, but who Playmaker thinks should go at the end of the draft, if at all. Herron stayed in school for a full four years, which is typically a sign of a lesser talent at the wide receiver position. This could be forgivable potentially, given that Wyoming is a smaller school, but Herron also failed to put up impressive numbers against inferior competition. Herron failed to crack 1,000 receiving yards during his time at Wyoming, even during his senior year, when the Cowboys passed 473 times. Herron's yards per catch is also fairly low (13.4) despite only rarely being used in the running game.
Potential Sleeper Alert: Austin Franklin, New Mexico State
Projected Round: 7
Playmaker Projection: 245.1 Yards
Playmaker Score: 86.6%
Franklin weighs only 189 pounds and recorded only a 4.56 forty-yard dash. Add that to the less than stellar competition that Franklin faced playing for New Mexico State, and you have a recipe for indifference from NFL scouts. However, Franklin had a nice career statistically. During his best season, he had over three yards for every team attempt and had a peak touchdown per attempt season that was nearly as good as Sammy Watkins'. Moreover, Franklin was also used as a running threat, averaging nearly a rushing attempt per game in his peak season, and still managed a solid 15.2 yards per catch. Seventh- round picks are nearly throwaways anyway, and although Franklin will not likely amount too much due to his low draft position, a team could make much worse bets at the end of the draft.
What follows is a list of all of the FBS wide receivers who were invited to the Combine and their Playmaker Projections and Ratings, ordered by Playmaker Projection:
|Playmaker Score for 2014 NFL Draft Prospects|
|Brandin Cooks||Oregon St.||1||638.0||95.8%|
|Odell Beckham Jr.||LSU||1||608.4||93.6%|
|Mike Evans||Texas A&M||1||558.4||86.8%|
|Allen Robinson||Penn St.||1-2||496.0||88.2%|
|Kelvin Benjamin||Florida St.||2||465.7||88.0%|
|Davante Adams||Fresno St.||2-3||437.0||89.4%|
|Donte Moncrief||Ole Miss||1-2||436.0||80.7%|
|Bruce Ellington||South Carolina||2-3||371.2||82.4%|
|Austin Franklin||New Mexico St.||7-FA||245.1||86.6%|
|Willie Snead||Ball St.||UDFA||220.1||83.8%|
|Josh Stewart||Oklahoma St.||7-FA||125.2||58.8%|
|Corey Brown||Ohio St.||UDFA||107.5||46.6%|
|Mike Campanaro||Wake Forest||5||98.2||20.7%|
|Bennie Fowler||Michigan St.||UDFA||14.0||16.7%|
|T.J. Jones||Notre Dame||5-6||2.1||3.1%|
|Isaiah Burse||Fresno St.||7-FA||0.0||3.5%|
24 comments, Last at 28 Mar 2017, 3:44am
#1 by CBPodge // May 07, 2014 - 4:48pm
Interesting stuff. I presume that the players used to develop and test the model are all players drafted from 2011 and before? So therefore would it be possible to just get a list of the projections for players drafted in 2012 & 2013 to see how accurate playmaker has been in the years that weren't used to develop the model? Obviously the conclusions drawn from that would be subject to some interpretation, but it'd be interesting to see whether playmaker was high on guys like Keenan Allen.
#4 by Dr. Mooch // May 07, 2014 - 5:38pm
The Playmaker Rating ranking is presumably based on a statistic similar to previous scores. I'd like to know what, if any changes, have been made to the formula and to see the scores compared to previous years.
Without that, Playmaker is pretty useless, for reasons covered in last year's article.
#16 by Dr. Mooch // May 08, 2014 - 1:33pm
Last year's article gave frequencies for various score ranges for the 2005-2009 data set. For those data, the magic Playmaker Scores of >400 (for a 50-50 chance of success) and <200 (for virtually zero chance of success) fell at roughly the 87th and the 40th percentiles, respectively. Clearly the system incorporates different variables this year, but until I get more information to work with I guess I"m going to interpret Playmaker Rating with reference to those to milestones.
#7 by herewegobrowni… // May 07, 2014 - 6:43pm
David Terrell 2001 1 8 Michigan 320.4
Wow, looking at that NFL yardage, Tom Brady really did make him look better than he was on the college level.
Looks like Braylon Edwards correctly really didn't project as high as he was drafted, either.
Nobody could've predicted Carruth's off-the-field issues, either (even though there were some even before he was drafted.)
#8 by meblackstone // May 07, 2014 - 10:00pm
Sammy Watkins YPC are going to be low because he was heavily targeted with screen passes to supplement the run game.
Martavis Bryant wasn't because he isn't as elusive. Martavis is solid, but he's not as versatile, and takes longer to get open.
#10 by MC2 // May 07, 2014 - 11:57pm
Moss and Wayne each stole receptions from one another, and so it's unsurprising that they are considerably underrated by Playmaker. This "double prospect" effect is something we're looking to study more in the future.
I would imagine the opposite effect happens with a team like Georgia Tech. Since they rarely throw the ball, they usually don't put much effort into recruiting a good #2 WR, and consequently, on those rare occasions when they do throw the ball, they almost always throw it to the same guy, so their #1 WR will tend to account for an unusually high percentage of their total passing offense.
Of course, when it's a guy like Megatron, and to a lesser extent Demaryius Thomas, it probably doesn't matter that much, since those guys likely would have been very productive in almost any system. But I think it almost certainly helps to explain why Playmaker Score appears to have significantly overrated a guy like Stephen Hill.
#12 by Pottsville Mar… // May 08, 2014 - 9:43am
This was my thought exactly. Georgia Tech often only has one receiver on the field.
Offensive scheme has an effect on four of the seven criteria, actually. The three "per attempt" metrics obviously will depend on how many receivers a team regularly employs, and the rushing attempts metric will also depend heavily on scheme (and won't be able to count the fly sweep shovel passes, essentially rushing attempts, that have become so common). Looking at the top scores of all time, even aside from the Georgia Tech guys, almost all of them played in a pro style offense with at most 2 WRs on the field.
#13 by Aaron Schatz // May 08, 2014 - 12:58pm
The tough part here is that there's no obvious factor that would have indicated that Stephen Hill was not going to follow in the footsteps of Thomas. Any argument that Hill was overrated would have also been an argument that Thomas was overrated.
Also, don't overlook the possibility that Hill would have been much more successful in a different environment rather than the offense-swallowing circus that is the New York Jets.
(Georgia Tech had a more standard offense when Megatron was there, so this does not apply to him.)
#19 by MC2 // May 09, 2014 - 6:13am
Yeah, I don't know how to correct for it, but I do think that the inability to account for the wide variety of different offenses found in the college game is a significant flaw of the Playmaker system.
As for the Jets and their bad QB play making Hill look worse than he really is, that's certainly possible, but it's also quite possible that Peyton Manning is making Thomas look a lot better than he would look on a team with an average QB.
#21 by mehllageman56 // May 13, 2014 - 2:21pm
I think the problem with Stephen Hill is that he gets injured early in the season and does nothing after that. His stats tail off after the first couple of games each year, and the Jets put him on injured reserve late last season when Kerley, Winslow and Holmes came back from their injuries. I'm not sure if you can blame Hill's problems on Geno Smith, since Smith was the 6th most accurate deep passer last year according to PFF. One would think fly patterns would be Hill's bread and butter. Hill was probably the walking dead well before they put him on IR, they just didn't have anybody to replace him at all.
#20 by coboney // May 12, 2014 - 9:48pm
I'm only surprised you didn't hold a ritualistic burning of it when Millen was fired to celebrate the freedom of your Lions from him.
Perhaps when the curse of Millen has worn off and the Lions make the playoffs?
#18 by armchair journ… // May 08, 2014 - 11:31pm
I think its a bit unfair (to the projection system) to call Charles Rogers a humongous bust. He broke his collarbone in successive seasons, and never recovered form. Call it an injury-bust, and give Playmaker a pass on that one... I don't think it was built to predict weak bones (or drug issues).
#22 by Duff Soviet Union // May 23, 2014 - 8:07pm
I'm very late to this, but this is good stuff. Have you looked at the following things?
1) Age. This has to be taken into account. It should be probably combined with the underclassmen variable. For example, Kelvin Benjamin is a redshirt soph, meaning he gets a bonus in your system, but he's 23. He's older than Tavon Austin who was a 4 year senior a year ago.
2) Speed score. I know you looked at 40 yard dash time and weight, but have you tried combining them? I reckon that could be illustrative.
3) Have you tried looking at other combine data like broad jump, short shuttle and 3 cone drill?