Writers of Pro Football Prospectus 2008

05 Sep 2009

Packers to Release Brian Brohm

Remember when Brian Brohm was expected to be the number one overall pick in the 2008 draft? Now he goes down as the Lewin Career Forecast's biggest flop. Hopefully, his failure -- and the other new QB data points since Lewin first published his work a couple years ago -- might help us figure out ways to improve our projections (and ways for NFL teams to avoid throwing away second-round picks on bad quarterbacks).

Posted by: Aaron Schatz on 05 Sep 2009

47 comments, Last at 07 Sep 2009, 9:19pm by BroncosGuy


by fyo :: Sat, 09/05/2009 - 5:23pm

The kid is 23, completed 63% of his passes this preseason with 1 interception. That's a marked improvement from preseason last year -- the only other pro experience the kid has.

I'm willing to bet he'll get more chances to make it in the NFL.

by Raiderjoe :: Sat, 09/05/2009 - 5:52pm

Not good. Em,barrassing to get beat out of job by 7th round pick Matt Flynn. Then next year ((which is this year) Broohm dont even make team. Pathetic Qb he is. Maybe can make something of carerr if go to Ufl or CFl.

by Sean McCormick :: Sat, 09/05/2009 - 6:24pm

Brohm may actually be a case where more widespread acceptance of the LCF pushed up his value a bit and got him into the second round despite his physical skills not being up to snuff. That said, the theory certainly took some hits with the quick success of Joe Flacco and (especially) Matt Ryan, and it may take another one if Mark Sanchez plays well this year. Nothing is dogma, and if guys with limited career starts continue to outperform their projections (or if guys like Brohm continue to face plant), it will obviously need to be re-visited.

by BroncosGuy (not verified) :: Sat, 09/05/2009 - 7:45pm

"Widespread acceptance of the LCF"? In the NFL?

Do you have any evidence to support this?

by Mr Shush :: Sun, 09/06/2009 - 7:57am

He doesn't mean, I don't think, widespread acceptance of the LCF per se by NFL front offices, so much as NFL front offices coming to conclusions similar to Lewin's - viz. that accuracy was being under-rated as a crucial attribute of NFL quarterback prospects, and (possibly) that scouts were more likely to be right about guys on whom they had more film. Lewin acknowledged from the start that if this happened his forecasts would become less useful.

by Independent George :: Sun, 09/06/2009 - 7:53pm

So by observing the event, one changes the outcome?

by BroncosGuy (not verified) :: Sun, 09/06/2009 - 10:05pm

Ah, you're not going all Heisinger on us, are you?

by tally :: Mon, 09/07/2009 - 1:50am

I'm not certain, but I think you meant Heisenberg.

by xtimmygx :: Mon, 09/07/2009 - 3:10pm

Well played sir

by BroncosGuy (not verified) :: Mon, 09/07/2009 - 9:19pm

Well, yes, getting Heisenberg's name right would have been a much more effective acknowledgment of the allusion. Maybe I subconsciously combined Heisenberg with Schrodinger? More likely, I am slowly devolving into Raiderjoe. Also, I'm drunk.

by BroncosGuy (not verified) :: Sun, 09/06/2009 - 10:45pm

Well, Shush, I appreciate you telling us what Sean means, but I still have to go by what Sean says: "widespread acceptance of the LCF pushed up his value a bit and got him into the second round". That's not terribly ambiguous. He is clearly suggesting that front offices have accepted the LCF enough to impact Brohm's draft position. That might be true; I'm merely asking him to back up the claim. (to be fair, please read Sean's entire sentence; he does not outright declare this acceptance as fact, but it is a very strong suggestion)

At any rate, the idea of using LCF to determine where to draft somebody is silly, as Lewin uses draft position as an independent variable in his forecast. You would have to believe "hey, if we trade up for this guy instead of letting him fall to us, he'll actually have a better career because of it!"

I'd still like to hear from Sean. Is there any actual evidence that acceptance of the LCF changed where Brohm was drafted?

by Sean McCormick :: Mon, 09/07/2009 - 1:56pm

No, I don't want to suggest that. However, the idea of using completion percentage and career starts as a way to sort upper tier college prospects was very much in the water in the months and weeks before last year's draft, and it's certainly a possibility that it swayed teams a bit in regards to one player or another. A guy like Brohm, whose physical skills weren't top notch, but who had an excellent college resume, might have gotten the benefit of the doubt in 2008, while he might not have in, say, 2005. Certainly, in the second round when the financial ramification of missing on a player is so much less, a team might have been more inclined to pull the trigger on Brohm based on a very positive projection. It's also possible that nothing of the sort happened. The basic point was that to the extent that the LCF was an effective tool, it was effective so long as teams did not incorporate it into their scouting evaluations.

by BroncosGuy (not verified) :: Mon, 09/07/2009 - 9:13pm

Thanks, Sean, for the clarification. I suspected that, linguistically, you might have leaned a bit in front of your skis on that. Still, we know at least some NFL personnel read PFP, so who knows? Sillier things have happened. I still find your evidence of a "stats effect" on Brohm pretty flimsy (I can claim a lot of things are "in the water"; you shouldn't believe it without proof. You shouldn't even suspect it without a hint of proof). As you acknowledge, its possible nothing of the sort happened. Which is why you should not have suggested it.

Above, I said that draft position is an independent variable in the LCF. This is not correct, and I apologize for the misstatement. But draft position does matter in LCF, so my point, though demonstrated a bit dramatically, is still valid, if only directionally. Sorry for the bad information.

by Illmatic74 :: Sat, 09/05/2009 - 6:46pm

Well Kelvin Kolb and Matt Leinart aren't helping out the Lewin Career Forecaster either

by Mr Shush :: Sun, 09/06/2009 - 8:02am

Kolb and Leinart are sitting on the bench behind veteran pro-bowlers, both of whom could still conceivably end up in the Hall of Fame. Either one could perfectly well still turn out to be a stand-out player.

Leinart is also a bit of a weird case, because if he does fail (and I don't think he will) the suspicion will always be that the issue was a lack of commitment to football, not a lack of ability. No statistical system can be expected to have a variable for "would rather be hanging out with groupies in a hot tub".

by Illmatic74 :: Sat, 09/05/2009 - 6:49pm

The problem I have with mainstream sports coverage is a lot of times people try to be experts on everything. That is a major reason why ATH is unwatchable. You guys on Football Outsiders have found your niche of charting NFL and college football games stick to that and do whatever you can to be better at doing that. Leave the prediction on who will be succesful NFL Quarterbacks to the Mike Mayock and Mel Kiper's of the world stay in your lane.

by Basilicus (not verified) :: Sat, 09/05/2009 - 7:12pm

I've never felt that the logic behind the LCF made as much sense, was as statistically sound, or was proven very well in the real world. Certainly not as much as most things on this site.

That said, it hardly makes sense that Football Outsiders should stop trying to come up with new theories. Just because this one isn't wholly sound doesn't mean they should stop trying new things. Trying new things is what made this site exist in the first place.

by Lance :: Sat, 09/05/2009 - 7:20pm

Is Mel Kiper actually all that great at predicting NFL talent? This isn't an attack-- I don't have figured either way to know. But my feeling is that he doesn't offer anything different from dozens of other pundits in print, radio, and TV.

by tuluse :: Sat, 09/05/2009 - 7:41pm

He's not especially good, but he can talk about every player in the draft. To be just as average for every single player, is just outstanding.

So mediocre DVOA, high DYAR.

by t.d. :: Sat, 09/05/2009 - 7:52pm

good analysis

by Marko :: Sat, 09/05/2009 - 11:08pm

Mel Kiper has no more credentials to do what he's doing than Bill Tobin's neighbor, who is a postman.

by loneweasel (not verified) :: Sat, 09/05/2009 - 7:51pm

and nothing more, no matter who made the first "discovery".

The sooner we understand that, the better off we'll be.

by Q (not verified) :: Sat, 09/05/2009 - 8:30pm

Sanchez to me really would not be a sign of a flaw with the system. The system gives credit for qbs with large numbers of starts which is almost impossible at a place like USC which produces Top 10 qb after Top 10 QB. John Elway as a recruit would not have been able to unseat Leinart before he graduated

by MC2 :: Sun, 09/06/2009 - 1:05am

I agree with your point, but I don't see how it's not a flaw of the system. Specifically, the system is flawed because it doesn't take into account the strength of the program. Starting 35 games for USC is obviously way more impressive than starting 35 games for Duke, but the system treats them as the same, so that's a flaw of the system.

by MJK :: Sat, 09/05/2009 - 8:36pm

The thing about the LCF is that it's not a true statistically-based prediction. All it says really is that we trust scouts to be able to do their job, and oh by the way, a good completion % helps too. What LCF really says is:

1) If scouts have a lot of data to work with, and
2) If, even after looking at that data, they think a guy will be good, he probably will be.
3) Looking only at the narrow pool of guys selected by 1 and 2, we expect the ones that were more accurate in college to be better in the pros.

So really, LCF is just a way of taking the massive amounts of data that scouts work through and sorting it based on their recommendations into something the common man can easily understand. This means that LCF will only ever perform as well as the scouts do. It's not really an innovative statistical tool, it's just common sense that is contingent on the assumption that scouts do their job well.

But obvioulsy scouts make mistakes. So therefore, LCF will, too.

I wouldn't say that LCF is broken if a bunch of QB's it liked don't work out. I would say that it's broken if more QB's that it doesn't like (i.e. QB's picked in late rounds, or QB's picked in the 1st two rounds with few starts or lousy completion %'s) start doing well than guys it likes. But I"m not sure that's the case. In the past 3 years, have more QB's that LCF didn't like succeeded than QB's that it liked?

by Bowl Game Anomaly :: Sat, 09/05/2009 - 9:26pm

Let's see:

Didn't like Russell, Ryan, Flacco, Sanchez, Stafford (maybe), Freeman.

Liked: Beck (maybe), Kolb, Henne, Brohm.

I don't remember if it liked Drew Stanton. I wouldn't include Pat White. I'm not confident if it liked or disliked Stafford and Beck.

I would say the jury is out on Kolb and Henne, but they certainly haven't proven themselves, so the LCF is not identifying successes very well. For projected busts, Ryan has definitely succeeded, and Flacco has been decent as well, so some players have been able to overcome the LCF projection.

by tuluse :: Sat, 09/05/2009 - 9:33pm

Wasn't Russell's projection that he was going to be OK, but not great?

by Bowl Game Anomaly :: Sun, 09/06/2009 - 12:21am

Yeah, more of a "disappointment for a #1 overall pick" than an outright bust.

by tuluse :: Sat, 09/05/2009 - 9:32pm

LCF doesn't dislike picks after the 2nd round, it just doesn't rate them period.

by BucNasty :: Sun, 09/06/2009 - 8:24am

On your last point, I think the opposite is true. The system seems best suited to identifying the guys who are supposed to be the safe picks and tagging the guys who shoot up the draft board due to a rocket arm and a possible fluke season as high risk. It's a lot more damning if the supposed "safe" picks fail, but then it also says a lot about NFL scouts if they can't accurately gauge a 3 year starter.

Conversely, because the system shouldn't be read as saying that the guys with low scores will certainly be bad, just that it's far more likely, you should expect that some of those guys really will live up to their potential. This is particularly true if the reason for their poor forecast was a low number of starts. In that case, the system isn't saying that he's bad, it's saying that we don't know enough to know either way so think twice before you stake your job on it.

Regarding late round quarterbacks, my philosophy would be to target guys with high completion percentages that my scouts like who have a low number of starts. I'd rather gamble on potential than take someone that I know is going to be bad.

by Mr Shush :: Sun, 09/06/2009 - 9:40am

I totally agree. If the scouts really love a guy with a ton of starts and a good completion percentage, they're right. He's Rivers or Palmer or Peyton. Trade the farm to get him. Two entire drafts if you have to. It doesn't matter. If a guy's completion percentage sucks, he's almost certainly no good. Don't touch him in the first round, don't touch him in the seventh. If the scouts love him for a position other than wide receiver or wacky-wacky wildcat gimmick man, ignore them. If a guy has a low start count, but your scouts love him and he has a good completion percentage, they might be right (Ryan) but they might very well not (Smith, Carr, Russell). Bear in mind also that LCF does a much, much better job with picks in the top 16 than with picks 17-64.

There needs to be a disclaimer, which probably could be statistically framed but to the best of my knowledge never has been, to the effect that guys with low attempts-per-game numbers should be downgraded. Call it the Jason Campbell rule. There also needs to be a disclaimer, which almost certainly could never be statistically framed, to the effect that guys who show signs of being normal young men, who prefer twincestuous threesomes to watching game film, are unlikely to develop into Peyton Manning, even if they have the talent to do so. Call it the Matt Leinart rule.

Finally, on the subject of quarterbacks drafted too late for Lewin's research to apply, I heartily recommend a visit to this article from the PFR blog a couple of years ago. The conclusions could be summarised as: draft guys from BCS schools who ran pro-style offenses, with prototypical NFL height, but not necessarily prototypical NFL bulk. Combine this with what one might term "peripheral Lewin conclusions/speculation" - low (sic) start/attempt counts, good completion percentage - and the message appears to be that the good QBs who fall in the draft are the guys who the scouts miss because they didn't post gaudy numbers in gimmicky systems, didn't get enough playing time to produce lots of tape, and don't have laser rocket arms, but who are accurate, good decision makers and used to playing in pro-style systems. It may well be no coincidence that the Bradys, Bulgers and Schaubs of this world have missed considerably more games over their careers than the Mannings and Favres: lack of bulk may translate to both indifferent arm strength (over-rated fault leading to over-late drafting) and increased injury risk.

by Sean McCormick :: Sun, 09/06/2009 - 10:54am

Excellent post. I think the way to approach the LCF is to look at it as risk management. If you have a guy with 45 starts and a 64% completion percentage up at the top of the board, don't fret about taking him, because he's probably going to be very good. If you have a guy with terrific physical skills but who hasn't produced at the college level, you should be very concerned about investing a top pick. If he's someone like Sanchez, who produced at a high level in a pro system but had limited starts, you need to recognize that he's a higher risk selection than someone like Philip Rivers. (And in reality, Sanchez is a safer projection than others simply because he runs a pro offense rather than the spread and has received NFL coaching his entire career- his 65% completion percentage looks better than someone who was coming out of, say, an Urban Meyer offense.)

You can use LCF to project guys who seem to be very safe- the Mannings, Rivers and Roethlisbergers of the world. You can use it to take out guys who are clearly not up to the task- Boller, Leaf, Harrington, Vick, etc. For guys who fall into the middle, you need to take a long look at the circumstances. Matt Ryan's receivers really were that bad, for instance.

(Incidentally, I would still not be ready to write of Leinart or even Vince Young, as they've both showed signs of being ready to step in this year or next year. Kevin Kolb, in contrast, has looked awful for two consecutive preseasons.)

by Flounder :: Sat, 09/05/2009 - 10:25pm

He'll probably wind up on the GB practice squad. If they trade or drop a fullback (they've kept 3!) I guess might wind up back on the regular roster, although that's probably doubtful.

by jebmak :: Sun, 09/06/2009 - 6:16am


by Noah Arkadia :: Sun, 09/06/2009 - 10:27am

Guys are freaking out and taking shots at FO as if this were Black Tuesday. LCF is a forecast, not a law. As far as forecasts go, if it works more than 50% of the time, you're in business. LCF is in business. Can it get better? Surely. But, with Vinny Testaverde, Kerry Collins and a few more, plus others like Vince Young, Ken O'Brien, and Tony Eason in the opposite camp in mind, let's give this some time, shall we?

by MilkmanDanimal :: Sun, 09/06/2009 - 10:39am

I recall yelling at the TV during the 2008 draft for the Bucs to draft Brohm. From all accounts, he was a smart, accurate QB, and he seemed a really good fit for Gruden's scheme. Complex schemes, lots of movement, quick throws . . . I figured he'd be perfect, right?

Of course, they drafted Dexter Jackson instead, who was cut a few weeks ago for being utterly worthless in every way conceivable. Sigh.

by packerpalooza (not verified) :: Sun, 09/06/2009 - 10:56am

While Brohm has some work to do it's important to acknowledge that Matt Flynn REALLY worked to improve his arm strength and other skills. Flynn took the same route as Aaron Rodgers who after his first camp looked like a flop in the making. Mediocre to poor arm, poor footwork compounded by the usual rookie adjustment to NFL speed of the game.

Flynn adjusted. Brohm has not. THAT is what concerned Green Bay. That BB had all offseason to DO SOMETHING and didn't seem that much different than last camp.

by tabsports :: Sun, 09/06/2009 - 2:16pm

The Lewin forecast still looks sharp for first-round choices. I think he needs to drop the forecast for second-round QBs, though. In general, second-round QBs have been gawd awful in recent seasons.

Beck, Brohm, Stanton, Henne, Kolb, etc... None of them have done anything, and Henne is the only one with a future IMO.

by tabsports :: Sun, 09/06/2009 - 2:20pm

I'm still going to consider the LCF perhaps the most reliable forecast to look at for first-rounders (besides doing to work in the film room and study room). Ryan was on the fence anyways last year, so he could be seen as the exception to the rule, just like Cutler was a few years ago. For Cutler to do what he did at Vandy deserved high praise, anyway.

by Bowl Game Anomaly :: Sun, 09/06/2009 - 10:59pm

Lewin himself definitely granted Cutler an exemption due to the fact that he was clearly better than previous Vandy QBs and so his completion % needs to be taken in context. I think Lewin did not give Ryan enough credit for his high number of attempts per game, which may offset his less than ideal number of starts.

by Mr Shush :: Mon, 09/07/2009 - 5:38am

Second rounders have been putrid for a lot longer than that. In the past 20 years, only two QBs drafted between picks 33 and 64 have even been long time NFL starters, never mind actually good. One of them is Jake Plummer. The other? Oh, darn, his name escapes me, ach, it's on the tip of my tongue . . . Oh, well, never mind. I think he signed for some NFC team somewhere up north this off-season - Detroit, or Chicago, or somewhere. Someone will remind me, I'm sure.

by Bowl Game Anomaly :: Mon, 09/07/2009 - 10:30am

Good job using multiple endpoints to skew the data. If you had just said 2nd round instead of "picks 33-64," you'd have to include Drew Brees. If you'd said 25 years instead of 20, you'd have to include Randall Cunningham and Boomer Esiason. You also flat out missed Kordell Stewart and Tony Banks, depending on your definition of
"long time NFL starters." Charlie Batch was also a starter for a while, although I can see why you left him off. (If you stretch it out another 3 years you could also catch Neil Lomax.)

Now, having said all that, the 2nd round still hasn't produced a whole lot of QB talent in the last 30 years. It's just not quite as bad as you implied.

by Mr Shush :: Mon, 09/07/2009 - 12:06pm

Brees was a deliberate fudge, of course - but then again if I was really picking a cut-off to make the case it would be #34, to exclude Favre as well. Banks has fewer career starts than David Carr. Stewart has 8 more. I think reasonable people could differ on the matter of whether either was a "long time NFL starter"; I don't think anyone would say that either was what you were hoping for when you spent a second round pick on a quarterback. On the timeframe issue, I could equally well have said a decade, and ruled out Plummer, or "since the start of free agency", and ruled out Favre. I also think it's worth noting that even Plummer and Brees stank it up in their first three (Brees) and four (Plummer) seasons in the league. Even if these guys are going to contribute, the evidence seems to be that it won't be early. I also don't think it's a coincidence that Brees and Favre were taken right at the top of the second round, and Plummer and Banks (the third and fourth best passers in the group) were both taken in the first half of the round. We don't know which, if any, of the Henne-Kolb-Clemmens-Jackson etc. group will ultimately pan out (though my money would be on Henne). It would be completely unreasonable for us to expect any of them to be effective starters as of now. The more worrying point is that there is prima facie no very compelling reason to expect any member of a random sample of say five quarterbacks drafted in the second round to work out.

by Bowl Game Anomaly :: Mon, 09/07/2009 - 5:49pm

Fair enough (except the Brees omission, that was unjustified). I think we agree that the 2nd round is hardly a good place to find successful QBs.

Here's a paraphrase of a quote from Lewin on why he uses the top 2 rounds: 1. QBs from the top 2 rounds get to play, while later drafted QBs often never get a shot. 2. QBs from the top 2 rounds are certified by scouts as having the necessary talent to play in the NFL.

While I partially agree with his first point (3rd round QBs also usually get a long look before they are cut), I think the second point is arbitrary. What QBs are drafted in the 3rd round who don't have the talent to play? Hell, what QBs are drafted in the 6th-7th rounds that don't have the talent to play? If you don't have the arm strength to complete passes, or are unable to get the ball out quick enough, or are shorter than 6', you will be an UDFA almost every time. Now, what Lewin really meant is that the later rounds often feature QBs from gimmicky offenses who have outlandish or misleading stats, and that statistical analysis won't be able to deal with those guys. But with the pervasive use of shotgun/spread/other non-pro style offenses, that now applies to the 2nd round and even the 1st. Just look at Vince Young, Alex Smith, or Rex Grossman.

I think there are compelling arguments for dropping the 2nd round from the LCF. I would also make the following tweaks: Adjust the projection for number of attempts per game (see Matt Ryan vs Jason Campbell), and include a bonus for pro-style offense or penalty for non-pro-style offense.

EDIT: As it happens, my suggestion's are very similar to this post:

by PaulH (not verified) :: Mon, 09/07/2009 - 3:26am

But in many ways did the scouts not effectively nail it on Brohm? After the 2006 season, a lot of people thought he would be the #1 pick, but his stock slid a lot his senior year. And even on draft day itself, most of the "experts" thought he was still a first round draft pick.

Ultimately, though, he fell to very late in the second round, and a lot of teams that REALLY needed a quarterback -- like the Vikings -- passed him up. And even the team that did take him (the Packers) I imagine saw him as more of a long-term project because of Aaron Rodgers on the roster. Moreover, they even took out a bit of an insurance policy on Brohm later in the draft by taking Matt Flynn out of LSU.

My point is that I'm not necessarily sure the scouts missed it on Brohm. Most people thought he'd go much higher than he did, and you know that his slide -- because he didn't have any character issues or anything of the sort -- had to be the result of some poor scouting evaluations. At the very least, a lot of scouts had some serious concerns with him or they wouldn't have let a possible high-end NFL quarterback drop to the low second round.

by tuluse :: Mon, 09/07/2009 - 4:18am

Getting cut your 2nd year after being a 2nd round pick means the scouts did not nail it.

I remain convinced to this day that the Packers drafted him to keep the Bears from getting him. I don't know if the Bears wanted him, but I think the Packers think they did.

by Dan :: Mon, 09/07/2009 - 6:24am

Then why did they take him at 56, when they also had pick 60?

The Bears couldn't have wanted him that badly if they'd already passed on him with their 2nd rounder (in favor of some running back).