QBASE 2022: Can Pickett Prosper in Pittsburgh?

Steelers QB Kenny Pickett
Steelers QB Kenny Pickett
Photo: USA Today Sports Images

NFL Offseason - Guest column by Alex Olbrecht and Jeremy Rosen

Projections for the 2022 rookie quarterback class were lukewarm at best going into the draft, and NFL teams largely agreed. With the exception of Kenny Pickett, who was drafted by the Pittsburgh Steelers in the first round, all the quarterbacks predicted to go highly fell into or past the third round.

Because the 2022 draft had not yet occurred when we published QBASE 2.0 at ESPN.com and Football Outsiders, we used Scouts Inc.'s projected draft position as an predictor variable instead of actual draft position. However, the model is designed to use actual draft position, so after the draft, we always update our projections accordingly. We would ordinarily release our updated projections next spring, but for this class, the differences between projected and actual draft position were quite large. In addition, we need to drop Sam Howell and Carson Strong from our projections because neither of them was drafted in the first three rounds, a requirement for QBASE 2.0. Therefore, we are releasing the updates in this column.

First of all, Kenny Pickett's projection improves very slightly since, at 21st overall, he was drafted one spot ahead of his Scouts Inc. projection (22nd overall). But this change is not significant, and we still find that he has about a 50/50 chance of being an adequate NFL starter.

However, there are significant changes in our projections for Desmond Ridder (projected by Scouts Inc. to be drafted 36th overall, actually drafted 74th overall), Matt Corral (projected 34th, drafted 94th), and Malik Willis (projected 21st, drafted 86th) because they were all selected well below their projected draft positions. As such, their projected bust rates increase significantly, and their chances of being an adequate, upper tier, or elite starter decrease significantly.

Postdraft QBASE Projections, 2022 Draft Class
Quarterback School Team TDYAR/A Bust Adequate Upper Elite
Kenny Pickett PITT PIT 0.04 48.5% 26.2% 16.8% 8.4%
Desmond Ridder CIN ATL -0.67 73.3% 17.2% 7.1% 2.4%
Matt Corral MISS CAR -0.69 73.9% 16.9% 6.9% 2.3%
Malik Willis LIB TEN -1.06 83.3% 11.8% 3.9% 1.0%
Bust: < 0.00 TDYAR/A; Adequate Starter: 0.00-0.75; Upper Tier: 0.75-1.50; Elite: > 1.50

However, we present Willis' projection with a significant deal of caution. Ultimately, how much to trust his projection comes down to why he fell so far. If his fall was due even in part to NFL teams using internal models that resemble QBASE 2.0 (which always preferred Pickett over him), then actual draft position will have become correlated with the rest of the model. Therefore, including it means we are effectively double-counting it, which in turn gives Willis an overly harsh penalty. So, for example, while it is safe to say that QBASE 2.0 does not love Willis' chances of being a good NFL quarterback, we would not be shocked if he ends up being one.

In conclusion, QBASE 2.0 projects that this year's quarterback class is now even more likely to descend into obscurity (like the 2007 and 2013 classes) than it was before the draft. However, even with these perhaps overly pessimistic projections, there is still a 13.5% chance that at least one of these quarterbacks will be elite, and a 78.1% chance that at least one will be an adequate starter or better. In addition, NFL teams can consider unquantifiable information that statistical models cannot. For example, the Steelers have been able to watch Pickett for four years, as they and the University of Pittsburgh practice in the same facility. Therefore, if Pickett is successful, it will illustrate that drafting quarterbacks is still very much an art, even as statistical models continue to become a more instrumental part of the process.

Jeremy Rosen is a doctoral student of economics at Georgetown University. Alexandre Olbrecht is a professor of economics at Ramapo College of New Jersey and the Executive Director of the Eastern Economic Association. The views in this column are expressly our own and do not represent the views of Georgetown University, Ramapo College, the State of New Jersey, or the Eastern Economic Association.

Comments

14 comments, Last at 05 Aug 2022, 6:18pm

1 Fascinating

So, the players didn’t change at all, yet you estimate their bust rate is much higher now.  Interpreting your model, do you think it Is that because they fell so low they will not be afforded the chance(s) a first round pick would get?  Or that the consensus of the other teams reflects information that your model doesn’t?

does your model take into account the “market” in any given year, e.g. five qbs were taken the previous year, several known quantity qbs became available this year, and several recently drafted qbs hit big, therefore demand is less than normal, and qbs will be drafted lower than normal?

2 The league controls the means of production.

In reply to by Raiderfan

AKA leashes. If they pass on these guys twice, chances are they won't give them as many chances as if they were their first selection. And the opportunities to give a third rounder instead of a first, a second chance aren't as enticing. 

How much more info do the teams know that we don't? Medical professionals are pretty found all over, nor is that info not leaked. Personality/intelligent tests are dubious (Woonderlic). So probably less than you think. Tape (and the numbers gathered from it) is still freely avaiable.  

10 I think it's less of a they…

I think it's less of a they have top secret info thing and more of a wisdom of crowds thing; if all 32 teams think someone isn't fit to be picked in the first two rounds there's probably a good reason why(of course not all the time).  It also tells us that theres not some huge disparity between what the scouts see and what the numbers say that results in the scouts thinking he's a top ten pick despite the stats.

3 I think demand certainly…

In reply to by Raiderfan

I think demand certainly plays into it, though QB appears to be one of those things where if the perceived value is there, you take the guy regardless. 

5 The problem is that much of…

In reply to by Raiderfan

The problem is that much of the writeup (and two of the key variables) are based on college performance and the third is a predictive variable based on NFL ownership that can be thought of as Relative Opportunity. Or as a poster said -- leash. We think of guys as busts or failure to realize intrinsic value -- it's tempting to think of QBase as representing QB quality. But it's really not. It's basically the inversion of draft slot performance. In draft slot value, getting a guy in the first round who is identically as good as a guy in the third round is bad -- that's wasted margin. It's hard for a 3rd rounder to bust, because they aren't expected to have any value.) But QBase loves that 1st rounder because he's a 1st rounder.

This is both clearly insane and likely represents actual observed behavior, because deciphering sunk cost from wisdom of the crowds is hard.

Part of the difficulty is the history of QBase.

Lewin Career Forecast (LCF) was just the college part. It was actually making a judgment of which college QBs were best, in a vacuum, divorced from their actual draft slot. (Explicitly, anyway -- the regression was based on QBs taken in the first two rounds, so it's always been something of a hidden variable)
https://www.footballoutsiders.com/stat-analysis/2006/college-quarterbacks-through-prism-statistics

This led to things like this:
"According to this projection system, Philip Rivers will emerge as one of the top quarterbacks in the league over the next couple of years." <-- good!
"Bowling Green's Omar Jacobs a solid starter" <-- bad!

LCF v2 expanded into the 3rd round and added some variables, but was basically LCF in concept. It's a quality rating.
https://www.footballoutsiders.com/stat-analysis/2011/introducing-lewin-career-forecast-v20

QBase 1.0 simplified the college stats a touch, and most importantly, added draft slot.
https://www.footballoutsiders.com/stat-analysis/2015/introducing-qbase

At this point, it goes from a performance comparison to a mixed-model blending past performance and likely future opportunity.

QBase 2.0 basically changes the wallpaper, but works along similar grounds.
https://www.footballoutsiders.com/stat-analysis/2021/introducing-qbase-v20

What I'm unclear about is the weighting of the variables. It's possible that if you took me and my giant black hole of college performance and drafted me #1, I'd be likely to become a better NFL QB than Tom Brady.

LCF would not have thought so -- but then it wouldn't have scored Brady at all. But the QBases might have liked me more.

6 Your points are well-said. …

Your points are well-said. 

I think one of the big problems with LCF into QBASE is mentioned in the article - (some) teams pay attention and have heavily changed their evaluations accordingly.  This has led to other problems, the biggest of which is that historical data is probably poor to include with recent data because the effectiveness of one is priced into the other.

I am becoming convinced that the draft time career projection projects become obsolete when published.

11 As a Steeler's fan, I…

As a Steeler's fan, I disagree with this assessment. The Steelers do NOT run a QB friendly system and the OL, even with 2 free agent signings and expected improvement of last year's rookies, projects to be bottom 5 in the NFL.  That's not a good environment for any QB, let alone a rookie.  On the plus side, Steeler's skill position players are pretty good - DJ, Claypool, Freiermuth, Najee + rookies Pickens and Austin are an above average assortment of weapons.  

I expect the Steelers offense to struggle as much, if not more than last year.  They were very good late in games when behind in 2021 - not sure we can count on that this year no matter the QB.  

13 Fair enough. I'll defer to…

Fair enough. I'll defer to anyone who follows the team more than I do. I was viewing it kinda like Mac Jones in NE. (A bit reversed as the Pats have a good OL and  weak weapons). Both have a decent defense and a stable coaching staff (more so in the Steelers case). While not totally ideal, I feel it is pretty decent situation for a rookie where he can kinda grow to his potential. YMMV however. 

4 Still have to applaud TN

And how they handled the QB position. Pre draft they had their entrenched starter but did no unnecessary (guaranteed) money towards a backup. For example, did the Texans REALLY have to spend more than the min to get Kyle Allen AND Jeff Driskel? What's the difference between then and the Titans Logan Woodside? I couldn't tell you. Add in them being more expensive and I'm sure you couldn't give me a definite answer either.

I wasn't a Willis truther but understood the appeal. Getting consensus QB1 as QB3 with 2 years of Tannehill is a fantastic balance of now and later in the draft. Fantastic process. 

8 QBASE

"If his fall was due even in part to NFL teams using internal models that resemble QBASE 2.0"

I am not part of an NFL scouting staff, and I have no sources within an NFL scouting staff.  Still, I can promise you that no team approached drafting quarterbacks with an internal model that resembles QBASE 2.0. 

For an organization that has for many years self-servingly championed its mostly useless quarterback projections, this might represent a new low (or new high, if we are grading on degree of arrogance).  Are you really suggesting your crappy model influenced the draft?

 

 

9 Gee

In reply to by BroncosGuyAgain

Someone woke up on the wrong side of the bed 

12   Interesting that you're…

In reply to by BroncosGuyAgain

 

Interesting that you're willing to make promises about things you admit you know nothing about.