Stat Analysis
Advanced analytics on player and team performance

QBASE 2019

by Aaron Schatz

There are no sure things in the NFL draft, but quarterback may be the riskiest position of all. No defense revolves around a single player the way every offense revolves around the quarterback. Trust your offense to the wrong young quarterback, and your team isn't going to climb back into playoff contention. Recent history is littered with teams that drafted a quarterback high, only to find out that player couldn't play at the NFL level.

The lessons of history can at least help us figure out how much of a risk each quarterback prospect will be. That's the point of Football Outsiders' Quarterback-Adjusted-Stats-and-Experience (QBASE) projection system. It looks at college performance, experience, and expected draft position (to incorporate scouting information that college stats will miss). To allow some time for development, QBASE projects a quarterback's efficiency (passing only) in Years 3 to 5 of his career according to Football Outsiders' Defense-adjusted Yards Above Replacement (DYAR) metric. 50,000 simulations produce a range of potential outcomes for each prospect, with players drafted later generally having a larger range of possibilities.

QBASE favors quarterbacks expected to go high in the draft who also have a relatively long resume of college success according to the stats. Those stats include completion percentage, adjusted yards per attempt (adjusted for touchdowns and interceptions), and team passing efficiency (measured with Bill Connelly's passing S&P+ stats). These numbers are adjusted both for the quality of the defenses that a prospect had to face as well as the quality of his offensive teammates. QBASE is meant to only be used on players chosen in the top 100 picks; after that, the judgment of scouts becomes even more important, and statistics become even less predictive.

However, there's a bit of a problem with using our QBASE system to analyze this year's prospects. As noted, one of the strongest elements in QBASE is the quarterback's resume: how many years has he started in college. Over the past 20 years, most of the quarterbacks taken at the top of the draft who then developed into NFL stars were quarterbacks who started at least three years in college, and often four.

But the top two quarterback prospects of the 2019 draft, Kyler Murray and Dwayne Haskins, both have a measly one year of starting experience in college. It's not just rare for a quarterback with only one year of starting experience to become a successful starter in the NFL. It's rare for a quarterback with only one year of starting experience to get drafted in the first three rounds, period.

Here's the list of quarterbacks with just one year of starting experience to be drafted in the top 100 picks since 1997: Mark Sanchez, Brock Osweiler, Mitchell Trubisky, and Cam Newton if you don't count the year he spent starting in junior college. That's it. Four men.

Expand the list of quarterback prospects to those with just two years of starting experience, and this group of passers still has a lousy track record. The best of the two-year starters are Joe Flacco, Michael Vick, and Alex Smith. The jury's still out on players such as Carson Wentz and Deshaun Watson. Aaron Rodgers could count as either a two-year or three-year starter since, like Newton, he had a year starting in junior college. But for the most part, highly drafted quarterbacks with only two years of college starting experience have underperformed their draft position. These are some of the biggest busts in quarterback history: JaMarcus Russell, Ryan Leaf, Akili Smith, Blake Bortles, and so forth.

Obviously, this year's quarterback prospects are not necessarily to blame for their lack of starting experience. Murray had to sit behind Baker Mayfield, a Heisman winner who had one of the five highest projections in QBASE history. And the experience issue is exacerbated by the number of quarterbacks in this year's draft who transferred midway through their college careers. Murray transferred from Texas A&M to Oklahoma. Will Grier went from Florida to West Virginia. Ryan Finley began at Boise State before starting three years at North Carolina State. Jarrett Stidham left Baylor and finished his college career at Auburn. Whether it was because of injury, job-sharing, or losing the competition to be a starter, all of these quarterbacks missed out on getting the early playing time that QBASE feels is a prerequisite for top prospects.

It seems like more quarterbacks are coming out with less starting experience in recent years, but that's not really true. There are still plenty of top prospects who started three or four years in college: Mayfield, Patrick Mahomes, Lamar Jackson, or from this year, Drew Lock. Will more recent top prospects with less college experience do better than the prospects of a decade ago? We'll need to see more out of them before we can answer that question. Until we do, QBASE will remain a little bit skeptical of quarterbacks with less starting experience.

Despite the lack of experience, Murray and Haskins (in that order) still have the highest QBASE projections in this year's class. But they don't match the top QBASE projections of recent years. This is the first draft since 2014 where no quarterback has a QBASE projection over 600. Here are projections for Haskins, Murray, and five other quarterbacks currently in the top 100 of the Scouts Inc. rankings for the 2019 draft.

QBASE PROJECTIONS

Dwayne Haskins, Ohio State (Scouts Inc. rank: 8)

Mean Projection in Years 3-5: 527 DYAR
Bust (< 500 DYAR) 51%
Adequate Starter (500-1499 DYAR) 28%
Upper Tier (1500-2500 DYAR) 16%
Elite (>2500 DYAR) 5%

Kyler Murray, Oklahoma (9)

Mean Projection in Years 3-5: 595 DYAR
Bust (< 500 DYAR) 47%
Adequate Starter (500-1499 DYAR) 28%
Upper Tier (1500-2500 DYAR) 18%
Elite (>2500 DYAR) 7%

There's another reason why the projections aren't higher for Haskins and Murray: both of them played alongside a lot of draft-worthy teammates who may have boosted their games. We add up the draft value of receivers, tight ends, and offensive linemen based on projected draft position for the next two drafts. Since 1997, the only top-100 drafted quarterbacks with a stronger collection of teammates based on this formula were Johnny Manziel (2014), Matt Leinart (2006), and Danny Wuerffel (1997).

Haskins will probably have two receivers go on Day 2 this year (Parris Campbell and Terry McLaurin) plus two offensive linemen on Day 3 (Michael Jordan and Isaiah Prince). He also had blocking from one of the top projected offensive linemen of the 2020 draft, Thayer Munford. Meanwhile, Oklahoma's offense may get three first-rounders this year, with Murray joined by tackle Cody Ford and receiver Marquise Brown. And wide receiver Ceedee Lamb and tight end Grant Calcaterra should go in the first two rounds of the 2020 draft.

Despite the lack of experience, and the small penalty for playing in good situations, both Haskins and Murray had outstanding production, which puts them among the top QBASE projections ever for players with less than three years of starting experience. Haskins ranked fourth in FBS in all three of the stats that QBASE uses to measure production. Murray was even better; he broke Baker Mayfield's record with 13.0 adjusted passing yards per attempt. Here's a look at the top QBASE projections since 1997 for players with less than three years of college starting experience.

Top QBASE Projections for 1-Year and 2-Year Starters, 1997-2019
Alex Smith 2005 798
Cam Newton 2011 698
Kyler Murray 2019 595
JaMarcus Russell 2007 570
Sam Bradford 2010 545
Vince Young 2006 536
Dwayne Haskins 2019 527
Tim Couch 1999 474
Mitchell Trubisky 2017 456
Johnny Manziel 2014 456

One other issue that's important for discussing Murray, although not Haskins: QBASE only gives a projection for passing performance in the NFL, not a projection for rushing value. That's going to be a big additional part of what Murray brings to an NFL offense.

Drew Lock, Missouri (17)

Mean Projection in Years 3-5: 271 DYAR
Bust (< 500 DYAR) 59%
Adequate Starter (500-1499 DYAR) 28%
Upper Tier (1500-2500 DYAR) 11%
Elite (>2500 DYAR) 3%

Drew Lock is the only four-year starter among this year's top prospects, but QBASE does not give him credit for being a four-year starter. The system doesn't want to give extra credit to quarterbacks who had bad years as starters, so there's a minimum floor of completion rate to have a season counted: 55 percent. Lock fell below this baseline in both his freshman (49.0 percent) and sophomore (54.6 percent) seasons.

Lock's senior stats in the three statistical categories measured by QBASE are almost exactly the average for all quarterback prospects from the last 20 years. Each one -- AYPA, completion rate, and team passing S&P+ -- is within 0.1 standard deviations of the mean.

Lock's profile in QBASE looks a lot like another four-year starter from over a decade ago who disappointed at the pro level: Brady Quinn. Quinn also fell short of the 55 percent completion rate baseline in his first two seasons, had senior stats that were very close to the average for all prospects, and was projected as a late first-round pick. Quinn's QBASE was a little lower, with a projection of 137 DYAR over years 3 to 5.

Ryan Finley, North Carolina State (54)

Mean Projection in Years 3-5: 398 DYAR
Bust (< 500 DYAR) 52%
Adequate Starter (500-1499 DYAR) 27%
Upper Tier (1500-2500 DYAR) 15%
Elite (>2500 DYAR) 6%

Finley's odd college career impacts his standing as a prospect in a couple of ways. Finley spent three years at Boise State, with two medical redshirts in his first three seasons. He also graduated in just three years, which made him a graduate transfer who didn't have to sit out a season before playing for North Carolina State. The NCAA granted Finley a rare sixth year of eligibility, which meant he could start three seasons for the Wolfpack.

QBASE looks at Finley and likes that three years of starting experience. But QBASE is not accounting for the fact that Finley is significantly older than this year's other quarterback prospects. He's older than even the usual prospects with three years of starting experience. At the age of 24, he theoretically has less room for growth before he hits his athletic peak, especially when compared to someone like Kyler Murray who won't turn 22 until August.

Statistically, Finley's best attribute is his completion rate, a robust 67.4 percent last season. He is docked a bit for having the second-worst schedule strength among these seven prospects and for playing with a lot of draftable talent, including receivers Kelvin Harmon and Jakobi Myers as well as center Garrett Bradbury.

Jarrett Stidham, Auburn (59)

Mean Projection in Years 3-5: -45 DYAR
Bust (< 500 DYAR) 67%
Adequate Starter (500-1499 DYAR) 25%
Upper Tier (1500-2500 DYAR) 7%
Elite (>2500 DYAR) 1%

Stidham struggled a bit in 2018. His completion rate dropped from 66.5 percent to 60.7 percent, his yards per attempt dropped from 8.5 to 7.6, and Auburn's ranking in passing S&P+ dropped from 23rd to 34th. Those declines, combined with only two years of starting experience, result in a poor QBASE projection for Stidham.

Daniel Jones, Duke (76)

Mean Projection in Years 3-5: 263 DYAR
Bust (< 500 DYAR) 56%
Adequate Starter (500-1499 DYAR) 25%
Upper Tier (1500-2500 DYAR) 14%
Elite (>2500 DYAR) 5%

Jones had the lowest numbers in all three of the statistical performance categories included in QBASE: adjusted yards per pass attempt (6.9), completion rate (60.5 percent), and team passing S&P+ (102.2, which ranked just 62nd in FBS last season).

However, the attempt to separate a player from his context plays a big role in scouting Jones. Duke University is not exactly known as a football factory. Jones is the only quarterback among these top prospects who had no teammates who are also expected to be drafted. And Jones played the hardest schedule among these seven quarterback prospects, with the average opponent ranking 52nd in pass defense S&P+. Combine these facts with three years as a starter, and QBASE sees Jones as a moderate prospect and good value in the third round.

Will Grier, West Virginia (87)

Mean Projection in Years 3-5: -151 DYAR
Bust (< 500 DYAR) 71%
Adequate Starter (500-1499 DYAR) 21%
Upper Tier (1500-2500 DYAR) 7%
Elite (>2500 DYAR) 1%

Grier presents by far the greatest discrepancy between QBASE and the quarterback projection system that Josh Hermsmeyer debuted on FiveThirtyEight last month. Hermsmeyer's CPOE system came out with Grier as the second-best prospect in this draft class, behind Murray. QBASE has Grier as the worst of these seven top prospects. What gives?

What you're seeing are the effects of two projection systems built very differently. Hermsmeyer's CPOE system is based solely on the on-field production and statistics. It's built on a smaller sample of quarterbacks, going back to just 2012. (QBASE is built on quarterbacks going back to 1997.) But because it only includes recent quarterbacks, it can base its projections on more advanced, intricate game-charting statistics that we don't have for older quarterbacks, primarily average depth of target. Grier had excellent stats last year. His 10.7 adjusted passing yards per attempt ranked third in FBS behind Murray and Tua Tagovailoa. He completed 67.0 percent of his passes. And he had a high average depth of target, which helps him score well in the CPOE system.

Grier comes out poorly in QBASE because of the elements that aren't included in the CPOE system. Grier has only two years of starting experience. QBASE is also factoring in scouting consensus to try to account for the things we can't measure with statistics, and consensus has Grier as a third-round selection, not a first-rounder. Finally, Grier also played the easiest schedule of the seven prospects listed here, with his average opponent ranking 71st in pass defense S&P+.

This article originally appeared on ESPN+.

Comments

21 comments, Last at 05 Apr 2019, 3:49am

1 Re: QBASE 2019

Good read. About 1 year starters - assuming for the sake of argument that 1) at least one and maybe both of Murray and Haskins turn out to be above average and 2) this starts happening with something resembling regularity, how long would be be before QBASE would start to lose its skepticism?

12 Re: QBASE 2019

The other question that I have is why is QBase skeptical. It could be that a lot of players who can't start until they're a junior or senior means that they weren't able to beat out the competition as freshman or sophmores and that proving that you're a freshman starting calibre player is meaningful, but if a player was behind a legit NFL talent with a terrific QBase, that aspect of their reason for starting fewer seasons seems less pertinent. Then there's the case of the young guys who leave the draft early after a terrific by QBase predictive metrics season. Do we think that more time playing in college would have helped them learn more and that learning time in the NFL is inferior? Or is it just that the sample of QB's who have one good season only to never duplicate that feat are too numerous? If that's the case, that's the best reason to potentially downgrade a prospect.

The first two reasons seem weaker given some context and the sample size isn't particularly that large as pointed out by the article.

18 Re: QBASE 2019

Experience was one of the two factors David Lewin looked at in the first iteration of these projections, and it helped to explain a lot of busts. And (most) teams are likely aware of this research, so experience is probably more of a consideration in ranking quarterbacks.

It's always possible that one player or another will significantly exceed his projection. It's more likely, though, that an inexperienced quarterback will drafted high by a team that comes up with some of the same explanations that you did, and is eventually added to the list of guys who show why it's a bad idea.

19 Re: QBASE 2019

I always suspected that the main work being done by starts in the LCF was just sample size. More starts means the completion percentage is less likely to be a fluke. More tape to watch makes it more likely the scouts who think this guy is good have got it right.

20 Re: QBASE 2019

You may be right. Regardless, as the list of names in the article reminds us, the correlation is still there.

2 Re: QBASE 2019

Great article. I enjoy following QB prospects and trying to guess how they'll pan out in the NFL. IMO this year's QB class is especially weak, with no one close to last year's crop. Murray is a major talent, I like him but IMO he's not worth the first pick and the Cards will be making a huge mistake if they use the #1 on him. I'd trade down and try to salvage Rosen by surrounding him with more talent instead. Haskins is another seriously talented guy who's going to end up going far too high. I see Murray and Haskins as having similar potential but I just don't see either of them being able to come in and start right away.

I like Lock as a 2nd or 3rd and I especially agree re: Daniel Jones who could turn out to be a real value pick a la Kirk Cousins a few years back. Some QB-challenged team like Miami would do well to use a pick on him. Ditto Grier, although IMO he'll need to be "coached up" before he's ready for the NFL game. He'd be a good choice to sit behind an established starter for a bit.

5 Re: QBASE 2019

According to Qbase, any team would be better off taking a chance on Josh Rosen than any of this year's prospects; his projected dear was over 600. Darnold didn't do well in last year's projection, with a little over 300. I'm thinking he's going to mess up the Qbase must start more than 2 years' philosophy, along with Deshaun Watson. When prospects couldn't come out early, the number of game started measured whether the prospect was good enough to start. Now, not as much.

6 Re: QBASE 2019

That's not really how it works though - we learned some things about Rosen last year - and none of them are good. Teams don't generally try to jettison first round picks after a year unless there's really something wrong with them.

First round draft pick QBs that end up good are usually obviously better than their competition once they hit camp/preseason/etc. The fact that Rosen couldn't do that is a sign that his uncertainty mostly lies in a downward position.

8 Re: QBASE 2019

That's not what happened with Goff, and two years later he played in a Super Bowl. The other two examples I can think of are Steve Young and Vinny Testaverde. Apparently being drafted by the equivalent of the 1980s Buccaneers means that there is still hope for a first round quarterback with a bad rookie year. Given the mess Arizona was last year, I would argue there's hope for Rosen. He certainly had bad coaching last year, as did Darnold. By the way, Rosen still played better than Josh Allen. He had a higher completion percentage than Allen last year. His passer rating was lower by a point.

I'm sure Belichick is using the same argument you make in trade talks with Arizona right now. You are a Patriots fan after all. So I'm just giving you the arguments to make in favor of the trade when it happens. As a Jets fan, I'm certain it will, and that it will work out very well for the Pats.

9 Re: QBASE 2019

As a Bills fan, seeing Rosen go to the Pats is also my worst nightmare. I’m certain Belichick would get the most out of his intelligence and terrorize the AFC East for another decade if he got his hands on him. Hopefully Brady continues to be petulant and insist on running out of town any heir apparent, as he presumptively did with Jimmy G

15 Re: QBASE 2019

I'm curious about Rosen's destination should the Cards draft Murray as expected. I can't see him lasting beyond a 2019 mid-second as I think both the Dolphins and Packers would probably make that trade. The Packers could have him take over for Rodgers in 2-3 years (likely the sooner as he hasn't been the most durable QB recently) if they think Rodgers isn't worth the price anymore. It would probably tee him off because he already saw this from the other side with Favre, but he's more aware than Favre was that it's a business. The Packers might even be able to recoup a similar draft pick from some team that could really use a new QB. (Paging John Elway) The Dolphins just because they've needed a good QB since Marino retired. I could also see almost every team (not Buffalo or obviously AZ) earlier in the second round also trading their #2 pick. So the question is whether some team (Oakland, LAC, Packers or Pats) trade their late #1?

16 Re: QBASE 2019

Per DVOA, Rosen was literally twice as bad as Allen (and he was catastrophically bad -1000 DYAR).

But that's not even the real problem - the real problem is that he was a top 10 pick and his team has already decided that he's not worth keeping. Yes, the Cardinals are a bit of a mess - but that's a really bad sign. The list of players high 1st round picks who get run out of town in their second year is pretty much all awful players. Not just bad - but Ryan Leif franchise-crushing-bad

Goff wasn't great his rookie year - but I don't remember the team going all fire-sale on him his second year.

17 Re: QBASE 2019

Sure, Rosen had lower DYAR than Goff, but that's just because he played more. On a per-play basis, Goff was actually much worse (-74.8% DVOA vs -53.0% for Rosen).

As for the idea of the Cardinals giving up on Rosen after one year, I'm still not convinced that's going to happen, but even if it does, I think that speaks more to the team's lack of foresight than the shortcomings of Rosen. After all, why would they hire Kingsbury (a guy with a losing record over 5 years as a college head coach) unless they thought he had what it takes to "fix" Rosen? They couldn't have hired Kingsbury with the intention of drafting Murray, because at the time Murray was still planning to play baseball. He didn't announce he was giving up on baseball until over a month after the Cardinals hired Kingsbury. So, the Cardinals must have thought (and presumably still think) that Kingsbury can "fix" Rosen. Otherwise, why hire him in the first place?

7 Re: QBASE 2019

For what it’s worth, PFFs grading is also kind of lukewarm about this years crop but loves Murray, saying he played at a similar level to mayfield and they are similar sized and types prospects, so the one year or data should translate. Haskins they like but not love, seeing him as a similar prospect to Rosen without the larger sample size. Lock and Jones are interesting 2nd and 3rd round projects with lots of potential, but would be overvalued in the first round and my opinion would greatly benefit going to teams with aging vets at QB (New Orleans, NE, LAC, Pittsburgh) than being the guy day one for a team like Miami.

What’s fascinating is they are also super high on Grier, so it’ll be interesting how QBASE stacks up with these other modeling systems when the projections are so drastically different. I haven’t seen him play at all, so I’m interested to see what the naked eye view is on him and if there are any other theories why there’s such a disparity—-does strength of schedule really matter that much? What if he goes late 1st or early 2nd, does that change the projection?

3 Re: QBASE 2019

Remind me, the QBASE projection is the per year DYAR for yrs 3-5? It’s not cumulative DYAR over yes 3-5?

4 Re: QBASE 2019

No, it is cumulative. Note that each player is given a chance of hitting 2,500 DYAR or more. Only one QB has ever hit that mark in a single season (Tom Brady, 2007). If we were projecting each player's chance to hit 2,500-plus DYAR in a single season, let alone average that over a three-year stretch, everyone's odds would be zero.

If 500-some DYAR over a three-year stretch doesn't sound very impressive ... well, it's not. Twenty-four QBs have more than that over the last three seasons. But it's better than what Cam Newton, Joe Flacco, Nick Foles, Josh Allen, or Josh Rosen have done since 2016, and those guys will all be starting this fall.

13 Re: QBASE 2019

Thanks.

Looks like Trubisky is gonna beat his QBASE projection. 500 DYAR (2018) in yr2 vs his 500 yr 3-5 projection.

14 Re: QBASE 2019

The DYAR distributions look fairly asymmetrical (Haskins' 51%tile is 500, mean is 527), would there be a significant difference looking at the median as opposed to the mean?

21 Re: QBASE 2019

I honestly wonder what QBASE would say about Trevor Lawrence would he be eligible.