Writers of Pro Football Prospectus 2008

19 Apr 2010

The Most Tepid Defense of Scouting You'll Ever Read

The always worthwhile Chris Brown examines the relatively recent phenomenon of the draft scout and the frustration that comes from having limited statistical tools to project college players to the NFL.

Posted by: Sean McCormick on 19 Apr 2010

22 comments, Last at 20 Apr 2010, 4:55pm by AlanSP


by Trust Doesn't Rust (not verified) :: Mon, 04/19/2010 - 11:06am

This is a pretty weak article. It creates a strawman about football scouts all being these guys who evaluate players based on half-assed subjective observations, and spew the cliches of the football media. In reality, the majority of scouts are focused on technique and physical projectability, and the best probably have developed local connections that give them valuable intangible information about a player.

Take baseball scouts. I mean, what magic statistical formula does anybody have to evaluate baseball draft picks or prospects when you have a zillion pitchers each year with live arms, and a zillion hitters whose overmatched college and high school opponents could never get them out? The main things you have to evaluate are technique, projectability, and subjective intangibles that might suggest a player being able to make the slow climb to the majors. Perhaps you might also look for secondary skills like plate discipline or defensive versatility, but in the end these skills also are most meaningful in terms of how they project to the majors. You can't evaluate, say, a college kid who takes a bunch of walks, and think that's a foolproof indicator that he'll take a bunch of walks in the pros where pitchers are able to hit the corners or throw offspeed pitches that go in and then out of the strikezone.

by TruFalcons (not verified) :: Mon, 04/19/2010 - 5:22pm

Not a Billy Beane fan apparently.

by Dean :: Mon, 04/19/2010 - 11:11am

Sounds like "this is the best we have, and it's better than nothing."

by Marko :: Mon, 04/19/2010 - 12:44pm

"It’s fair to say that, nowadays, no team thinks they are the only ones who 'know about' some player, even if the player went to a small school, as Rice did."

The implication that the 49ers thought that no one knew about Jerry Rice is just idiotic. I remember reading an article in The Sporting News in 1984 about Mississippi Valley State's prolific offense featuring Rice and QB Willie "Satellite" Totten. So Rice wasn't exactly a secret.

And I remember when the 49ers traded up to get him, the reaction wasn't "who's Jerry Rice"? The reaction was that Bill Walsh had made another genius move. No one would have thought that if they didn't know who Jerry Rice was.

by langsty :: Mon, 04/19/2010 - 12:57pm

This is an astonishingly bad article by a writer I ordinarily respect. It's so lazy that I almost have to wonder if it's a troll.

It's just... he plucks all these assertions out of thin air and doesn't bother to illustrate any of them or explain how he knows that, say, scouts are 'Typically not former coaches, they are instead self-styled “talent gurus.”'

I have to wonder how much attention he actually pays to sabermetrics, because most of its leading minds don't display anywhere near this kind of arrogance about the work they do. Bill James and Tom Tango (for example) often go out of their way to stress the importance of scouting and an integrated approach to player evaluation; the stats-vs-scouts narrative popularized by Moneyball is outdated and nonsensical.

That's just the tip of the iceberg. This is bad writing and bad analysis.

by Jeff Fogle :: Mon, 04/19/2010 - 6:03pm

Have to agree completely with langsty. I was thinking this through the long reading ordeal...it was a little over 3,000 words about something tepid! What's the point of that. Felt like a high school essay where somebody HAD to write 3,000 words or else, but they were allowed to use a thesaurus as often as they wanted. Wandered and meandered until it got to 3,000 words.

I think I said "that's not a fair statement" about 50 times through the piece. Inexplicable that the NY Times would run it under their banner. Unless the editors just like words like tepid, paean, sundry, idiosyncratic, efficacy, ubiquitous, verbiage, dictum, tropes. That's in just the headline and the first six paragraphs. Out of almost 30.

Scouts on TV were unleashing "angry rhetoric" or "wildly misdirected anger" at Graham Harrell and Chase Daniel for running up their stats in college? Does anyone have you tube clips of that?

The ultimate conclusion seems to be that there's some sort of collected wisdom of the banal and vapid...

"scouts’ exhortations are just feeble attempts at explanations of something that cannot be put into words, namely their pre-verbal impressions of how good a football player is. This is not as uncommon as it sounds, and, strange mix of subjectivity and theater as it is, it reconciles the otherwise inexplicably efficient N.F.L. draft."

That's not really an explanation. It's the lack of an explanation for something that only seems "inexplicably efficient," when you start with the premise that scouts are banal and vapid. (And, we can put "exhortations, feeble, pre-verbal" on the literary posturing list).


by Raiderjoe :: Mon, 04/19/2010 - 9:13pm

Scouts on TV were unleashing "angry rhetoric" or "wildly misdirected anger" at Graham Harrell and Chase Daniel for running up their stats in college? Does anyone have you tube clips of that?

Yes, that bad part of article . alsmost want to stop reading at that point.
dont remember wanythting like that. Most people with brain knew harrell and C Daniel crap NFL prospeocts. Short colege Qbs with pop gun arms, only trained monkeys thought those 2 guys would be goods in NFl. Dont remeber any tv scouts (menaning M K Jr, t McAhay types)00 whinign about Harrell and dabiel and cursing TT coach and Misoorui coach for creating uniqe systems that make Qbs have wild stats

by Raiderjoe :: Mon, 04/19/2010 - 9:14pm

yeysh! html in other posut work,. Sierra Nevada Tropedo workin mirachles

by billsfan :: Tue, 04/20/2010 - 10:06am

Inexplicable that the NY Times would run it under their banner.

The Fifth Down Blog has previously had content by the guy responsible for Cold Hard Football Facts, which is all of the above, but without the thesaurus. Or the high-school education.

Unless the editors just like words like tepid, paean, sundry, idiosyncratic, efficacy, ubiquitous, verbiage, dictum, tropes

It's the New York Freakin' Times... of course they just like those words!

(I also like the Eagles)

by JIPanick :: Mon, 04/19/2010 - 12:58pm

"Note that arm strength is frequently misunderstood. More arm strength is not always better — being able to throw it 80 yards compared with 65 is meaningless. Instead the question is, Can you throw the ball 25-30 yards, on a line, from one hash to the other on a deep comeback route? If you can, the arm strength box is checked."

Doesn't increased arm strength also mean increased velocity, and therefore less time for a defender to react? I always figured how fast it got there was more important than how far it went.

by Illmatic74 :: Mon, 04/19/2010 - 3:05pm

Exactly more people misunderstand that than what Chris Brown is talking about.

by Nathan :: Mon, 04/19/2010 - 3:29pm

Does unless you're Leftwich and your release is so slow it cancels out how strong your arm is.

by Joseph :: Mon, 04/19/2010 - 1:09pm

Based on this article, the GM/personnel person who doesn't at least CONSULT things like Lewin's forecast, Speed score, and the recent SackSEER article is just not smart. None of these systems are an end-all, but should at least be used as a tie-breaker.

by Noah Arkadia :: Mon, 04/19/2010 - 2:22pm

Every day life must be a heck of a challenge for the author. How do you quantify a good lettuce? How to decide which girl to date?

by Theo :: Tue, 04/20/2010 - 9:15am

Easy, depends on how much you'd like to toss her lettuce.

by Raiderjoe :: Mon, 04/19/2010 - 5:19pm

Why NYT insist on shwoing NFl as N.F.L.? If wrote for 5th down blog, would tell NYT editor to shoev period marks where sun dont shine. NFL=NFl, not N.F.L.

by Dean :: Tue, 04/20/2010 - 9:31am

Because the N.Y.T. is stodgy, pretentious, and loves to point out in subtle ways how much better they are than all of us in the great nonkulturny unwashed masses.

by San Marcos Landlord (not verified) :: Mon, 04/19/2010 - 5:58pm

I thought it was exactly as advertised -- a tepid defense of scouting. The connection with the "wisdom of crowds" school of thought made a lot of sense.

by FMTEMike (not verified) :: Tue, 04/20/2010 - 3:10am

the arm strength condundrum is, to many teams, a distraction, and the key points are quickness of reading, reacting, and releasing (the 3 Rs)and accuracy. Some teams may still be impressed by 55 yard throws from the knees, but others just want to see enough zip to get the throws there

which leads to the key about scouting/evaluation. The point isnt whether scouts themselves coach, but how well they can translate what they see into what their team's coach wants/needs for his system. Take Richard Seymour and Dwight Freeney as examples: the Pats were criticized heavily for drafting Seymour (ahead of David Terrell) because he had few sacks in college...the Colts found Freeney available because he was 'too small' to play RDE in the NFL. But the Pats knew Seymour was a perfect fit for their 3 front, and the Colts knew Freeney's speed and 'arm strength': left arm to hold off blockers, made him a fir for Tony Dungy's Tampa 2.

In a sense, Dungy's use of the Tampa 2, a system designed for speed, allowed his teams to draft players whose specific attributes skills were undervalued at the time, and whose major lack (size) was overvalued, at least in his context. Kind of like on-base pct in Moneyball...It's not necessarily the best way to build a defense, but it is a cost-effective one, when youre spending big bucks on Manning, Wayne etc...

Just as good coaching centers on not asking players to do what they cant do, and taking advantage of what they can, good scouting centers on asking whether they can do what your system wants them to do.

by bubqr :: Tue, 04/20/2010 - 8:07am

Awful article for many reasons stated above. I'd like to add one : I'm pretty sure teams love the combine less for their drills than for the possibilities of interviewing so many prospects at the same time, and also without the whole world knowign that you're bringing them in for a visit.

by tuluse :: Tue, 04/20/2010 - 4:43pm

I think they also like to see players put up good times because it means they took training seriously and worked hard to advance themselves.

It's also a good way to see if small school prospects have raw athleticism that it looked like they had on film

by AlanSP :: Tue, 04/20/2010 - 4:55pm

I agree with others that the article's narrative is overly simplistic. It contends that numbers can be misleading, so we're left with nothing but vague impressions, which still aren't all that bad.

The problem is that it's a rather large logical leap from saying something along the lines of "college passing yards don't tell you how good a QB will be in the NFL" to saying that numbers in general don't tell you anything. Maybe you just need to find different metrics to look at.

This could involve looking at traditional stats in new ways (as in LCF), combining college stats with combine numbers (as in SackSeer), or it could mean using "unofficial" measures and recording them yourself. There's nothing stopping a scout from watching an offensive linemen and recording how often they win their blocking assignment on running plays (as I'm sure many scouts do), or from doing the type of game charting that FO does (as, again, I'm sure many NFL teams do).

The important thing is trying to find ways to improve your prediction accuracy and looking to see what actually works. One thing that I've always found frustrating is that previous scouting evaluations are rarely analyzed in any serious way (e.g. asking if scouts tend to overvalue certain attributes or undervalue others). Instead, they simply repeat the same mistakes over and over again. A seemingly endless supply of small, incredibly productive running backs hasn't stopped scouts from looking at smaller RBs and immediately concluding that they're 3rd down backs, and a mountain of data hasn't stopped them from looking at inaccurate college QBs and thinking that they'll be able to fix that with a little coaching.