Writers of Pro Football Prospectus 2008

19 Mar 2008

Nothing to Choose From Between Ryan and Brohm

Remember Brian Brohm? This time last year, he was almost universally projected to be the top quarterback prospect available in the 2008 draft. Right now, Boston College signal caller Matt Ryan seems to be the flavor-of-the-month. Which guy should go first? The guys over at pro-football-reference blog have looked at the historical performance of quarterbacks taken in the first fifty picks, and their research suggests that unless one of them establishes himself as worth the first pick overall, it probably doesn't matter.

Posted by: Sean McCormick on 19 Mar 2008

21 comments, Last at 20 Mar 2008, 5:59pm by sonofbrocklanders


by Pat (not verified) :: Wed, 03/19/2008 - 5:17am

I don't actually think that this kind of analysis really works (as I mention in the comments there). What you're actually interested in is how a player's performance compares to some idealized draft position, where most teams would agree he "should" be drafted in an "average" draft.

The problem is that all you get are actual draft positions, and there's way too much slop in those for reasons completely unrelated to a player's talent.

So basically I think all of the draft positions you get for players are probably uncertain by probably +/- 10 picks or so.

That is, imagine the draft being run again, except switch all of the order of the teams around (clearly, the order of the teams has nothing to do with the ability of the player). Where would the player be drafted this time?

That number may seem excessive, but I really don't think it is. In 2006, switch Cleveland from drafting #2 to drafting #15, and I'd bet they grab Quinn at #15.

by Theo, Holland (not verified) :: Wed, 03/19/2008 - 8:26am

I read about some QB prediction system somewhere. Don't remember if it was here, or NFL.com or something.
Anyway, it said that the completion% and number of games played, where a good indicator for a college QB in the Pros (pros = NFL, dunno if it was any indication for the Arena League or the CFL)

by drobviousso (not verified) :: Wed, 03/19/2008 - 8:40am

Theo, it was here.

by Mr Shush (not verified) :: Wed, 03/19/2008 - 9:19am

Pat, that doesn't strike me as at all excessive, certainly for quarterbacks. I suspect the Browns would have taken Quinn as high as #6, had they been picking there.

Further, I would suggest that the "pick range variance" increases as the draft goes on. I doubt there is ever a non-QB drafted first overall who would ever have gone lower than fifth based on team needs and scouting variance, or a non-QB drafted lower than fifth who would ever have gone first.

And, as implied in the above paragraph, I believe that pick variance is almost certainly higher for QBs than for any other position, at least in the first two rounds. Only one quarterback takes the field for a team in a game, barring injury or wierdness. At any other position, a player can see substantial playing time even if he's not a starter. As a result, team needs at quarterback can almost be thought of as partially discrete - teams either need a starting quarterback or they don't - where at other positions they are continuous. So for example, in a year in which five quarterbacks had legitimate first round talent, but only three teams saw themselves as needing to acquire a quarterback to start in the near future, there is a real possibility that two of those players would still be available in the third round. I do not think the same could be said of any other position.

As for PFR's research, it hangs on the assumption that evaluating high end quarterback prospects is difficult. I submit that a combination of the wisdom of crowds and Lewin's system would draft vastly better than the badly managed NFL franchises whose poor talent evaluation creates a falsely low impression of the value of the right to choose.

by Joey Jo-Jo Junior Shabbadu (not verified) :: Wed, 03/19/2008 - 9:24am

Speaking of which, the Lewin projection system has to love Brohm. 33 GS, 65.8%.

Actually, while looking up those numbers, I came across this... (click name)

by Mr Shush (not verified) :: Wed, 03/19/2008 - 11:25am

Obviously that's a nice completion percentage, but 33 starts is a little on the low side. I imagine his projection will be good but not great.

by MJK (not verified) :: Wed, 03/19/2008 - 11:42am

the badly managed NFL franchises whose poor talent evaluation creates a falsely low impression of the value of the right to choose

Mr Shush...

That's an awfully wordy way of saying "The Raiders"... :-)


I completely agree with your and Mr Shush's points about reading too much into draft position. This is one of the primary reasons why I think tying a rookie's salary to draft position, or at least tying it too strongly to a specific draft position (i.e. the #1 pick gets X dollars, the #2 pick gets Y < X dollars) is daft--a player ends up getting paid less or more because a team that happened to be picking high in the draft had a specific positional need they wanted to fill.

by Andy (not verified) :: Wed, 03/19/2008 - 11:49am

#3: Theo make a joke.

by beedubyuh (not verified) :: Wed, 03/19/2008 - 12:41pm

I hope KC takes Ryan in the first round. That will mean an extra OT prospect may be around when the Bears pick comes up. Hell, the more QB's that go in the first half of round one the better.

by drobviousso (not verified) :: Wed, 03/19/2008 - 12:50pm

#8: In my defense, my reply was pre-coffee.

by David Lewin (not verified) :: Wed, 03/19/2008 - 2:13pm

I'll be weighing in on this with an article sometime in the next month. I prefer Brohm obviously, I just don't think Ryan has ever played at that high of level, whereas Brohm was superb for three years. That said, Ryan's game started total probably understates his true experience as he threw a ton of passes this year in an NFL style offense.

by Kevin11 (not verified) :: Wed, 03/19/2008 - 3:45pm

I keep seeing Brohm going as high as #17 and as low as mid-second. Someone is going to get a real bargain. On the other hand, I do not see Ryan as a top-five pick.

While I don't agree with the reasoning behind it, I do agree there is very little seperating Ryan and Brohm. I give Brohm the slightest of edges.

by Pat (not verified) :: Wed, 03/19/2008 - 4:30pm

As for PFR’s research, it hangs on the assumption that evaluating high end quarterback prospects is difficult.

It's still a little early to say how Lewin's projection system is doing ever since it was first published. The first big "bust" that Lewin's system is projecting is Russell, and obviously, he hasn't played yet.

Of Cutler, Leinart, and Young, Leinart was projected to be the best prospect, and right now it looks like Cutler is.

by Yaguar (not verified) :: Wed, 03/19/2008 - 5:15pm

Lewin suggested that Cutler was likely to outplay his projection, because he noted that great NFL quarterbacks with mediocre collegiate completion percentages often came from overmatched schools that had far, far worse passing statistics before and after the quarterback's tenure. Vanderbilt totally fits the bill there.

Lewin was spot on about Cutler, so we'll give him credit for that. We'll also give him credit on Rivers. I believe he was suspicious of Alex Smith's low games started, so we'll give him credit for that one. Jason Campbell, though, is definitely not living up to Lewin's high expectations, even though he's better than people think he is. He called three good quarterbacks in 2006, and he's right about at least Cutler, but he might be wrong about the other two.

I think the research has merit, but I'm not sure how much of the regression's accuracy has to do with small sample size and the kind of conveniently "expansion team" and Division I-AA adjustments to deal with outliers. With such a small data set, we can't be sure that Lewin's system isn't just something that matches past data well.

I think he's going to be very right about Russell. And I think the general feeling on this board that there's no good reason to pick Ryan so much earlier than Brohm is also going to be very right.

by admin :: Wed, 03/19/2008 - 5:44pm

Please, please, please do not paste in URLs that are wider than the page. That's why we give you a place to link a URL when you post. It messes with the look of the site in people's browsers. Thanks.

by Mr Shush (not verified) :: Wed, 03/19/2008 - 8:32pm

I'd go so far as to say that in projecting the later picks it deals with it's downright shaky. But I believe the weaker findings, that players who the scouts see as elite and who have a lot of starts really are elite, and that low college completion percentages are a truly damning indicator, are almost certainly right. That's still a pretty powerful way of tweaking the odds in your favour towards the top of the draft.

by JeffD (not verified) :: Wed, 03/19/2008 - 8:58pm

This year reminds me of 2005 with Smith and Rodgers.

I think that the value for rodgers was good while not for Smith and the same if it was opposite.

One will go to high and one to low. I would rather be the low one since the expectations are lower and the team better. Giving the QB more time to develope wth less pressure. Recently the NFL has shown that a QB with time to develope with out getting his confidence crushed produces a better result.

by The McNabb Bowl Game Anomaly (aka SJM) (not verified) :: Wed, 03/19/2008 - 11:10pm

Re: 13

It's not really true that Russell was projected as a bust, only that he is projected to disappoint for a #1 overall pick. Lewin projects him to be decent during his peak.

There has not been a QB drafted in the 1st round since Lewin's original article who projects to be a total bust. Remember, Russell's completion % is actually well higher than Brady Quinn's.

by Tom D (not verified) :: Thu, 03/20/2008 - 2:42am

Re 18:

But Russell had far fewer starts, I'm pretty sure Quinn had the higher projection.

by pharmboy (not verified) :: Thu, 03/20/2008 - 10:09am

I am not a big Vince Yong fan, but I think he has proven to be an above average QB given the success has had from a win-loss standpoint, even if his QB style is unconvenetional to say the least. As for Leinert, he fell into the black hole that is Arizona, where success (even with good wrs) is more difficult to achieve than other places. If Leinert was picked by Denver and Cutler by Arizona, I guarantee that Leinert would be considered the best QB prospect in the group.

by sonofbrocklanders (not verified) :: Thu, 03/20/2008 - 5:59pm

When vince young gets his DVOA above 0, I will consider him above average.

I'm not going to criticize or praise the Lewin system until it has a larger sample size of projections to look analyze for success or failure.