QBASE 2017

QBASE 2017
QBASE 2017
Photo: USA Today Sports Images

by Aaron Schatz

There are no sure things in the NFL draft, but quarterback may be the riskiest position of all. No defense revolves around a single player the way every offense revolves around the quarterback. Trust your offense to the wrong young quarterback, and your team isn't going to climb back into playoff contention. Recent history is littered with teams that drafted a quarterback high, only to find out that player couldn't play at the NFL level.

The lessons of history can at least help us figure out how much of a risk each quarterback prospect will be. That's the point of Football Outsiders' Quarterback-Adjusted-Stats-and-Experience (QBASE) projection system. It looks at college performance, experience, and expected draft position (to incorporate scouting information that college stats will miss). To allow some time for development, QBASE projects a quarterback's efficiency (passing only) in Years 3 to 5 of his career according to Football Outsiders' Defense-adjusted Yards Above Replacement (DYAR) metric. 50,000 simulations produce a range of potential outcomes for each prospect, with players drafted later generally having a larger range of possibilities.

QBASE favors quarterbacks expected to go high in the draft who also have a relatively long resume of college success according to the stats. Those stats include completion percentage, yards per attempt, and team passing efficiency. These numbers are adjusted both for the quality of the defenses that a prospect had to face as well as the quality of his offensive teammates.

There has been a lot of debate about whether any quarterbacks will go in the top ten picks this year, and QBASE suggests that none of this year's prospects really deserve to go that high. None of these players have a projection to match the best quarterbacks drafted in recent years. The highest-rated among them come with big questions about experience or about whether their college schemes affect our ability to translate college performance to the NFL.

One reason for this year's mediocre projections is that the top prospects of the 2017 draft did not lead the top passing offenses of the 2016 college season. College production isn't the only indicator of whether a quarterback translates to the NFL level, but usually the top-drafted quarterbacks have played for top-ten passing offenses according to Football Outsiders' Passing S&P+ ratings.

In 2014, for example, Oregon (Marcus Mariota) ranked fourth in Passing S&P+ while Florida State (Jameis Winston) was seventh. The year 2011 is another strong example, as Wisconsin (Russell Wilson) was No. 1, Baylor (Robert Griffin) was No. 4, and Stanford (Andrew Luck) was No. 10. Cam Newton and Sam Bradford also led offenses that ranked No. 1 in Passing S&P+.

However, only three of the top-10 passing offenses in 2016 were led by quarterbacks who are eligible for the 2017 NFL draft.*

Top NCAA Pass Offenses, 2016*
Team S&P+ Rank Quarterback Class Likely Draft
Oklahoma 1 Baker Mayfield rJR 2018
Penn State 2 Trace McSorley SO 2018/2019
USC 3 Sam Darnold rFR 2019/2020
W. Kentucky 4 Mike White rJR 2018
Washington 5 Jake Browning SO 2018/2019
Clemson 6 Deshaun Watson JR 2017
Toledo 7 Logan Woodside rJR 2018
W. Michigan 8 Zach Terrell SR 2017
Tennessee 9 Joshua Dobbs SR 2017
Florida State 12 Deondre Francois rFR 2019/2020
* This table leaves out Georgia Tech (10) and Navy (11)
as their passing ratings are the product of option offenses
where pass plays tend to be successful because they are rare.

Zach Terrell isn't considered a serious NFL prospect, which leaves just Deshaun Watson and Joshua Dobbs in the top 10. North Carolina (Mitchell Trubisky) ranked No. 16 and Pittsburgh (Nathan Peterman) ranked No. 19, while this year's other top quarterback prospects all ranked below the top 20.

Every year, there are some draftniks who insist that next year is the really good year for quarterbacks, if teams are just willing to wait. Again and again, we hear this refrain only to see these young quarterbacks suffer in the eyes of the scouting community after another year of nitpicking. But this year, the "wait until next year" mantra for quarterbacks may be accurate.

Of course, it's hard to wait at the most important position in football. And there's still a good chance that at least a couple of this year's prospects will turn into reasonable NFL starters. Here are QBASE projections for eight quarterbacks who Scouts Inc. has rated among the top 120 players of the 2017 NFL draft.


Mitchell Trubisky, North Carolina (Scouts Inc.: No. 24)

Mean Projection in Years 3-5: 435 DYAR
Bust (< 500 DYAR) 49.3%
Adequate Starter (500-1499 DYAR) 30.8%
Upper Tier (1500-2500 DYAR) 15.6%
Elite (>2500 DYAR) 4.4%

Trubisky may have had the most impressive single season of any of these prospects, with the highest completion rate (68.2 percent) and only six interceptions compared to 30 touchdown passes. But the risk with Trubisky is primarily related to one phrase in that sentence: "single season." In the past 25 years, only three quarterbacks have been chosen in the top 100 selections after only one year as a college starter: Brock Osweiler, Mark Sanchez, and Matt Blundin. If you're asking "Who's Matt Blundin?" that's somewhat the point: he was a 1992 second-round Kansas City pick out of Virginia who never started a game in the NFL.

This is not a good group of quarterbacks to be associated with, although Cam Newton is sort of attached with an asterisk. (He started only one year at Auburn, but also a year at junior college.) You won't find many successful one-year college starters chosen below pick 100, either: Matt Flynn is probably the best among them. On the other hand, do we really learn anything from a group with such a small sample size? It's likely more accurate to compare Trubisky to quarterbacks with either one or two years as a college starter. Since 1998, that group has over 30 players chosen in the top 100 picks instead of just two. With that in mind, we changed QBASE this year to treat players with one year of college experience the same as players with two.

[ad placeholder 3]

Unfortunately, the track record of top-100 quarterbacks with only two college years started isn't particularly good either. The best among them are Newton, Joe Flacco, Michael Vick, and Alex Smith. Ryan Tannehill and Jameis Winston may climb into those ranks. Aaron Rodgers could count as either a two-year or three-year starter, since like Newton, he also had a year starting in junior college. But for the most part, highly-drafted quarterbacks with only two years of college starting experience have underperformed their draft position. These are some of the biggest busts in quarterback history: JaMarcus Russell, Ryan Leaf, Akili Smith, David Carr, Sanchez, and so forth.

Of course, there was a two-year college starter selected in the later rounds of the draft who turned out to be pretty successful in the NFL: Tom Brady. But Brady is the historical outlier of all outliers, not a historical precedent you base a top-five draft pick on.

All college quarterbacks are a risk in the NFL. Does Trubisky's lack of college experience somehow negate all what scouts see on film? No, but we're trying to determine here which quarterbacks are high risk and which quarterbacks are really high risk. JaMarcus Russell, Ryan Leaf, and the other busts of the past also had glowing scouting reports. There's no question that, based on historical patterns, Trubisky's lack of starting experience makes him a particularly high-risk selection.

Deshaun Watson, Clemson (28)

Mean Projection in Years 3-5: 261 DYAR
Bust (< 500 DYAR) 56.5%
Adequate Starter (500-1499 DYAR) 28.3%
Upper Tier (1500-2500 DYAR) 11.8%
Elite (>2500 DYAR) 3.4%

There's an argument that Watson is guaranteed to bring strong leadership on the field, forged by the mental pressure of playing in high-profile playoff games. The statistical counterpart to this is that Clemson played the hardest schedule of opposing pass defenses in all of college football last season. That's part of the reason Watson's Tigers were highest among these eight teams in Passing S&P+, which is adjusted for opponent strength.

On the other hand, Watson was helped by better teammates than most of these other top prospects. Mike Williams stands out the most, although Artavis Scott and tight end Jordan Leggett are also expected to be drafted this year. There are a number of underwhelming NFL quarterbacks who built their college production on a highly-drafted receiver who was the better NFL player, including Johnny Manziel (with Mike Evans), JaMarcus Russell (with Dwayne Bowe), and Jimmy Clausen (with Golden Tate). Then again, our Playmaker Score system thinks Williams may be a bit overrated by scouts, so the QBASE system may be overestimating his effect on Watson's performance.

Patrick Mahomes, Texas Tech (47)

Mean Projection in Years 3-5: 656 DYAR
Bust (< 500 DYAR) 45.7%
Adequate Starter (500-1499 DYAR) 27.0%
Upper Tier (1500-2500 DYAR) 17.9%
Elite (>2500 DYAR) 9.4%

[ad placeholder 4]

Mahomes has the highest QBASE projection in the class of 2017. He led all college quarterbacks last year in Expected Points Added on passing plays. He's only 21 years old, so he's got room to grow, but he also was a three-year starter at Texas Tech, so he has experience. And he put up huge numbers despite the fact that none of his receivers or offensive linemen are likely to be drafted either this year or in 2018.

Nonetheless, Mahomes' projection sets off alarm bells -- or more specifically, very loud Air Raid sirens. Every single prospect from an Air Raid offense has underperformed his QBASE projection in the NFL, although obviously you can't shut the book on Jared Goff after just one season. Playing in the Air Raid doesn't mean Mahomes' skills can't translate to the NFL, but it seems clear at this point that the scheme juices passing stats in a way that leads to QBASE overestimating these prospects. Any team that wants to draft Mahomes probably needs to discount his college performance and be very sure that his skills fit its offensive scheme.

DeShone Kizer, Notre Dame (50)

Mean Projection in Years 3-5: -30 DYAR
Bust (< 500 DYAR) 65.5%
Adequate Starter (500-1499 DYAR) 21.4%
Upper Tier (1500-2500 DYAR) 9.8%
Elite (>2500 DYAR) 3.4%

Kizer was the only one of these eight prospects to complete less than 60 percent of his passes last season, even though only Nathan Peterman faced an easier schedule of pass defenses. Notre Dame ranked No. 54 in Passing S&P+, nearly 20 spots below the seven other teams listed here. Kizer also benefited from the presence of next year's top offensive tackle prospect, Mike McGlinchey, and he has only two years of starting experience. There are better quarterback gambles to make in 2017.

Joshua Dobbs, Tennessee (66)

Mean Projection in Years 3-5: 419 DYAR
Bust (< 500 DYAR) 53.8%
Adequate Starter (500-1499 DYAR) 26.1%
Upper Tier (1500-2500 DYAR) 14.3%
Elite (>2500 DYAR) 5.8%

Dobbs seems to be this year's "late riser," getting talked up more and more as we get closer to the draft. Normally, these late-rising quarterbacks are players who dazzle in workouts but had unimpressive college production. That's not the case with Dobbs. As noted above, Dobbs and Deshaun Watson are the only 2017 draft prospects who had their teams in the top ten of our college passing ratings, and the Volunteers faced the No. 2 toughest schedule of opposing pass defenses, trailing only Clemson. Dobbs' stats were much better in his senior year than in previous seasons, but that usually suggests quarterback growth rather than a one-year wonder. Dobbs is this year's best chance at finding a starting-quality quarterback with a mid-round pick.

Nathan Peterman, Pittsburgh (74)

Mean Projection in Years 3-5: -245 DYAR
Bust (< 500 DYAR) 71.6%
Adequate Starter (500-1499 DYAR) 21.3%
Upper Tier (1500-2500 DYAR) 7.0%
Elite (>2500 DYAR) 0.2%

Dobbs won the Tennessee job over Nathan Peterman, who then transferred and got two years as a starter at Pitt. Last year, he led these eight prospects with 10.1 adjusted passing yards per attempt (passing yards with a 20-yard bonus for touchdowns and a 45-yard penalty for interceptions). However, he did this against the easiest schedule of any of these eight prospects, ranked 59th in FBS, and with the benefit of an offensive line that's likely to get two linemen drafted in the first four rounds (guard Dorian Johnson and tackle Adam Bisnowaty). Combine these facts with a lower expected draft position, and QBASE is very pessimistic about his chances for NFL success.

Davis Webb, California (107)

Mean Projection in Years 3-5: 286 DYAR
Bust (< 500 DYAR) 56.9%
Adequate Starter (500-1499 DYAR) 25.1%
Upper Tier (1500-2500 DYAR) 13.6%
Elite (>2500 DYAR) 4.5%

The Air Raid siren that went off for Patrick Mahomes goes off even louder for his former teammate Davis Webb, since Webb's numbers after transferring to Cal were nowhere near as impressive as the numbers Mahomes put up at Texas Tech. Webb had just 7.3 adjusted passing yards per attempt last season, the lowest of these eight prospects. His numbers weren't really any better in his two years as the starter at Texas Tech in 2013 and 2014. QBASE does like the fact that he has three years of starting experience and that Cal played the No. 9 schedule of opposing pass defenses last season.

Brad Kaaya, Miami (112)

Mean Projection in Years 3-5: 6 DYAR
Bust (< 500 DYAR) 60.9%
Adequate Starter (500-1499 DYAR) 24.3%
Upper Tier (1500-2500 DYAR) 11.5%
Elite (>2500 DYAR) 3.3%

Kaaya is the winner of this year's Greg McElroy Memorial Trophy for top-school quarterback who is probably the product of his environment. Tight end David Njoku may be a first-round pick, and it's likely that four of Kaaya's five starting offensive linemen will be drafted either this year or next. Despite all this talent, Miami was just 35th in Passing S&P+ in 2016. Overall, QBASE agrees with conventional wisdom that Kaaya is a later-round pick with an NFL future as a backup.

Here is what QBASE had to say about quarterback prospects in past seasons.

Portions of this article originally appeared on ESPN Insider.


11 comments, Last at 28 Apr 2017, 12:44pm

#1 by nath // Apr 26, 2017 - 7:29pm

Did you evaluate Jerod Evans?

Points: 0

#3 by Aaron Schatz // Apr 26, 2017 - 10:04pm

QBASE is meant for players chosen in the top 100 picks. So I did players who have a reasonable chance to be chosen in the top 100 picks, based on the rankings I've seen around the Interwebs.

Points: 0

#2 by serutan // Apr 26, 2017 - 8:38pm

Just out of curiosity, are you going to have an "Asterisk - 6 years later" to
accompany the 2012 draft retrospective?

It is nice to see QBASE confirm that this is likely an underwhelming QB class.
Of course, more than one team will still reach further than it should for one.
Was wr

Points: 0

#4 by Chip // Apr 26, 2017 - 11:08pm

The distributions seem too optimistic - and not bc it's a bad draft class. Even the projection distributions last year were similar albeit more bearish.

This is basically telling us that there are 5 starters who have a ~50% chance of busting or a 50% probability of being an adequate starter to elite. Adequate starter based on actual yr 3-5 DYAR include V.Young, S.Bradford, J.Campbell (600-ish DYAR), Cutler (~800), C.Netwon (~900), McNabb (1100), Eli Manning ~1200). Click through to 2016 for the full list.

I can't imagine that there are 2-3 guys in this class who will have a career similar to Bradford, Cutler, Manning, Newton, McNabb, level.

Points: 0

#8 by Hoodie_Sleeves // Apr 27, 2017 - 3:15pm

You're probably right.

QBASE works on a bunch of assumptions - one of which is that "top 100 picks" which is a proxy for "scouts think this guy could work", and that largely relies on the draft pool being relatively consistent on a year to year basis.

In a good QB year, a bunch of these guys would fall outside that top 100 I think.

I don't think 2-3 guys having journeyman starter type careers is all that off, personally.

Points: 0

#9 by Sixknots // Apr 27, 2017 - 4:15pm

I agree that "2-3 guys having journeyman starter type careers" is probable. It's that some will probably be drafted in the top 10-12 that is crazy.

Points: 0

#5 by Bucs_Rule // Apr 27, 2017 - 8:04am

Does QBASE factor in offensive teammates who are expected to be drafted high?

Also, Cleveland management is reportedly very high on Trubisky, what do you think their analytics department does so differently?

Points: 0

#6 by mehllageman56 // Apr 27, 2017 - 12:03pm

Perhaps that's why there is so much dissension in their front office, according to rumors. Also, Qbase is rating Trubisky the second best prospect this year, so it's not so much that there are better guys to draft then him, but that everyone in this draft will probably bust.

Points: 0

#11 by Aaron Schatz // Apr 28, 2017 - 12:44pm

1) Yes. See above where I note WR/TE/OL and their likely draft positions.

2) I always figured this was a smokescreen because it made no logical sense.

Points: 0

#7 by Nahoj // Apr 27, 2017 - 2:23pm

I wound up watching a lot of Joshua Dobbs (married to a UT fan) in college, and while his game looks pretty solid (looks to pass first generally, but a definite threat to run if you don't watch out for it), I think his biggest strengths aren't physical.

Dude is smart and has a serious work ethic, both attributes that should serve well in the NFL.

As the link below notes, he's still pursuing his last 6 hours toward graduation with a major in aerospace engineering while preparing for the draft.


Points: 0

#10 by mehllageman56 // Apr 27, 2017 - 10:39pm

So if he graduates before mini-camp he passes the Parcells test? It looks like he does.

Points: 0

Save 10%
& Support Aaron
Support Football Outsiders' independent media and Aaron Schatz. Use promo code SCHATZ to save 10% on any FO+ membership and give half the cost of your membership to tip Aaron.