by Aaron Schatz
(Ed. Note: This year, we're presenting two different models for projecting the quarterback class of 2018. After you read about how these quarterbacks do in our QBASE model, check out our guest column by Jeremy Rosen and Alexandre Olbrecht that projects ANYA rather than DYAR and finds functional mobility to be the most important variable for projecting quarterback success in the NFL. Our hope for next year is to work on combining the two models, adding a variable for functional mobility into the QBASE model. But for now, we hope you enjoy debating two different models that come out with two different favored No. 1 quarterback prospects for 2018.)
There are no sure things in the NFL draft, but quarterback may be the riskiest position of all. No defense revolves around a single player the way every offense revolves around the quarterback. Trust your offense to the wrong young quarterback, and your team isn't going to climb back into playoff contention. Recent history is littered with teams that drafted a quarterback high, only to find out that player couldn't play at the NFL level.
The lessons of history can at least help us figure out how much of a risk each quarterback prospect will be. That's the point of Football Outsiders' Quarterback-Adjusted-Stats-and-Experience (QBASE) projection system. It looks at college performance, experience, and expected draft position (to incorporate scouting information that college stats will miss. To allow some time for development, QBASE projects a quarterback's efficiency (passing only) in Years 3 to 5 of his career according to Football Outsiders' Defense-adjusted Yards Above Replacement (DYAR) metric. 50,000 simulations produce a range of potential outcomes for each prospect, with players drafted later generally having a larger range of possibilities.
QBASE favors quarterbacks expected to go high in the draft who also have a relatively long resume of college success according to the stats. Those stats include completion percentage, yards per attempt, and team passing efficiency. These numbers are adjusted both for the quality of the defenses that a prospect had to face as well as the quality of his offensive teammates. QBASE is meant to only be used on players chosen in the top 100 picks; after that, the judgment of scouts becomes even more important, and statistics become even less predictive.
It's important not to overestimate the importance of a small difference in the QBASE projections. Most of this year's top quarterbacks are grouped together in the middle of the scale. It's a bit of a surprise that Sam Darnold came out with a lower QBASE rating than some of the other top prospects -- Josh Rosen, for example. But if I were the analytics director for an NFL franchise, I would feel no need to disagree with a scouting director who placed Darnold ahead of Rosen.
However, there is one top prospect who QBASE clearly likes ahead of all the others, and one top prospect whose statistical profile is a gigantic red flag. We explain both projections below, along with the QBASE projections for six other top quarterbacks based on Scouts Inc. rankings for the 2018 draft.
Sam Darnold, USC (Scouts Inc.: No. 3)
|Mean Projection in Years 3-5:||412 DYAR|
|Bust (< 500 DYAR)||51.9%|
|Adequate Starter (500-1499 DYAR)||29.0%|
|Upper Tier (1500-2500 DYAR)||15.1%|
|Elite (>2500 DYAR)||4.1%|
Based on Football Outsiders' Passing S&P+ stats, the USC Trojans had college football's No. 3 passing offense when Darnold was a redshirt freshman in 2016. That dropped to tenth in his sophomore year, but tenth out of 130 FBS teams is still pretty good. So why is Darnold's QBASE lower than that of other prospects expected to be chosen after him in this year's draft?
The problem is that Darnold has only two years of experience as a starting quarterback in college football, and the track record of top-100 quarterbacks with only two college years started is downright lousy. The best among them are Cam Newton, Joe Flacco, Michael Vick, and Alex Smith. Aaron Rodgers could count as either a two-year or three-year starter, since like Newton, he also had a year starting in junior college. But for the most part, highly-drafted quarterbacks with only two years of college starting experience have underperformed their draft position. These are some of the biggest busts in quarterback history: Jamarcus Russell, Ryan Leaf, Akili Smith, Blake Bortles, Mark Sanchez, and so forth.
Of course, there was a two-year college starter selected in the later rounds of the draft who turned out to be pretty successful in the NFL: Tom Brady. But Brady is the historical outlier of all outliers, not a historical precedent you base a top-five draft pick on.
To overcome this penalty in QBASE, a prospect needs to have dominating statistics. Cam Newton did. Alex Smith did. Sam Darnold did not. His stats don't particularly stand out when compared to Josh Rosen, Lamar Jackson, or Mason Rudolph.
All college quarterbacks are a risk in the NFL. Does Darnold's lack of college experience somehow negate what scouts see on film? No, but we're trying to determine here which quarterbacks are high risk and which quarterbacks are really high risk.
In Darnold's defense, the track record of two-year starters may look a lot better with a couple more years of data. Carson Wentz was a two-year starter, as were Deshaun Watson and Jameis Winston. Also, Darnold faced the most difficult 2017 schedule of any of this year's top quarterback prospects, and his raw statistics are a little more impressive in that light. But that element is already factored into Darnold's QBASE rating.
Josh Allen, Wyoming (7)
|Mean Projection in Years 3-5:||-83 DYAR|
|Bust (< 500 DYAR)||62.7%|
|Adequate Starter (500-1499 DYAR)||20.7%|
|Upper Tier (1500-2500 DYAR)||11.5%|
|Elite (>2500 DYAR)||5.2%|
Like Darnold, Josh Allen was only a two-year starter in college. But Allen's statistics are horrifying compared to Darnold's.
Allen has an obscenely powerful arm, but it's a howitzer without a targeting system. He completed just 56.3 percent of his passes last year. Here is the list of quarterbacks chosen in the top 100 picks since 2005 despite having a completion rate below 58 percent in their final college season: Andrew Walter (2005), Jake Locker (2011), Christian Hackenberg (2016), Connor Cook (2016), and C. J. Beathard (2017). That's it.
Last year, Wyoming finished 119th in passing S&P+ out of 130 teams in FBS. That will be the lowest rank ever for a quarterback chosen in the top 100 picks of the NFL draft. Yes, I know, Allen wasn't playing with a bunch of NFL-bound talent around him. He also wasn't facing a lot of NFL-bound talent on defense. The average opponent faced by Wyoming ranked just 83.5 in pass defense S&P+. Allen's performance against top opponents was brutal. He threw two picks with no touchdowns against Iowa, with just 4.35 yards per attempt. He completed just 9-of-24 passes with 64 yards and a pick against Oregon. He completed 44 percent of passes with two picks and only 131 yards against Boise State.
Since 1997, there have been 27 different quarterbacks chosen in the top 100 with QBASE ratings below zero. The best of these quarterbacks was either Josh McCown or Brian Griese. It's a terrible group of quarterback busts. Negative-QBASE passers chosen in the first round include Mark Sanchez, Josh Freeman, Kyle Boller, Rex Grossman, J.P. Losman, and Patrick Ramsey.
Allen supporters talk about how his 2016 season was much better than his 2017 season, and it was. In 2016, Wyoming finished 52nd in passing S&P+. Allen still couldn't complete more than 56.0 percent of his passes. If we pretend Allen's 2017 season never happened, then Allen has a QBASE of 161, still the worst of this year's top prospects.
Josh Rosen, UCLA (8)
|Mean Projection in Years 3-5:||623 DYAR|
|Bust (< 500 DYAR)||46.9%|
|Adequate Starter (500-1499 DYAR)||24.8%|
|Upper Tier (1500-2500 DYAR)||18.0%|
|Elite (>2500 DYAR)||10.4%|
Josh Rosen is a very typical first-round quarterback prospect. That's not meant to be a negative statement. The average QBASE for first-round picks since 1997 is 620; Rosen is at 623. The average first-round quarterback has 2.7 years of starting experience; Rosen has three. Rosen's APYA (adjusted passing yards per attempt) of 8.5 and completion rate of 62.6 percent are a little below average for a first-round quarterback prospect, but not by much. (The averages are 9.0 and 64.0, respectively.)
Rosen played a slightly harder than average schedule, but also gets docked by QBASE because he had a lot of high-rated talent around him at UCLA. Offensive tackle Kolton Miller and tight end Caleb Wilson are likely to be first-round picks (Miller this year, Wilson in 2019). Center Scott Quessenberry and wide receiver Jordan Lasley are also likely to be drafted this year.
Baker Mayfield, Oklahoma (13)
|Mean Projection in Years 3-5:||1,480 DYAR|
|Bust (< 500 DYAR)||29.3%|
|Adequate Starter (500-1499 DYAR)||21.8%|
|Upper Tier (1500-2500 DYAR)||20.7%|
|Elite (>2500 DYAR)||28.2%|
Baker Mayfield's stats at Oklahoma were completely ridiculous, so it's no surprise that a stat-based projection system such as QBASE loves him with the heat of a thousand suns.
One of the stats included in QBASE is adjusted passing yards per attempt. This metric looks at yards per attempt, then adds 20 yards for each touchdown and subtracts 45 yards for each interception. Baker Mayfield is the only quarterback in FBS history to top 12.0 APYA in a season, and he's done it twice: 12.3 APYA in 2016, and then a record 12.9 APYA in 2017. Mayfield also completed more than 70 percent of his passes as a senior, and Oklahoma's offense was No. 1 in Football Outsiders' Passing S&P+ rating.
[ad placeholder 3]
On top of this, the No. 1 predictor of NFL success for top quarterback draft picks is still how many years of starting experience they had in college. Mayfield is the only one of this year's top prospects with four years of starting experience. (Luke Falk, likely to be drafted later, started for three-and-a-half seasons.)
Like Rosen, QBASE docks Mayfield a bit because he played with high-quality teammates. Both offensive tackle Orlando Brown and tight end Mark Andrews are likely to be drafted in the first three rounds this year, and wide receiver Marquise Brown and guard Ben Powers are likely to go in the first three rounds of the 2019 draft. However, Mayfield does not get docked for playing an easy Big 12 schedule. Because of the Big 12 championship game, the Sooners had to play twice against TCU, which at No. 15 in S&P+ was the Big 12's best pass defense. Mayfield also had to face Ohio State (No. 12) and Georgia (No. 6), so the average pass defense he faced was ranked 52.8, the same as Rosen and better than any of this year's prospects other than Darnold.
Put all these elements together, and Mayfield has the fourth-highest QBASE projection of any prospect since 1997. The only players projected higher were Philip Rivers, Carson Palmer, and Donovan McNabb. Here are the top 10 prospects since 1997 based on the current formula:
|Top QBASE Projections, 1997-2017|
None of this is a guarantee that Mayfield is going to be a star in the NFL. Just below the QBASE top ten are prospects who didn't pan out, including John Beck, Matt Leinart, and Christian Ponder. But based on his college performance, Mayfield has the best odds of any quarterback in this class of being an elite starter in the NFL, and the lowest odds of being a bust.
Lamar Jackson, Louisville (38)
|Mean Projection in Years 3-5:||656 DYAR|
|Bust (< 500 DYAR)||46.2%|
|Adequate Starter (500-1499 DYAR)||26.9%|
|Upper Tier (1500-2500 DYAR)||18.5%|
|Elite (>2500 DYAR)||8.5%|
[ad placeholder 4]
QBASE is only projecting a quarterback's passing performance, so it doesn't care about Jackson's phenomenal running abilities. But QBASE also clearly thinks that talk of Jackson switching positions is kind of silly. He's just as good a quarterback prospect as any of this year's other young passers other than Mayfield.
Jackson has one stat that lags far behind his others: a completion rate of just 59.1 percent in 2017 and just 57.0 percent over his three-year college career. But his AYPA (8.7) and Louisville's passing S&P+ (11th, right below USC) peg him as a late first-round pick. And unlike other prospects who led top passing offenses last year, Jackson didn't have a lot of great talent around him. Any Louisville receivers and linemen who are drafted (perhaps offensive tackle Gerron Christian this year and wide receiver Jaylen Smith next year) are likely to be Day 3 picks.
Mason Rudolph, Oklahoma State (79)
|Mean Projection in Years 3-5:||343 DYAR|
|Bust (< 500 DYAR)||54.2%|
|Adequate Starter (500-1499 DYAR)||24.8%|
|Upper Tier (1500-2500 DYAR)||15.5%|
|Elite (>2500 DYAR)||5.6%|
Rudolph isn't quite at the Baker Mayfield level, but his raw stats last season were outstanding. His 10.7 AYPA far surpasses the rest of these prospects, thanks in large part to 37 touchdowns with only nine interceptions. A completion rate of 65.0 percent is also second among these 2017 prospects. Rudolph's QBASE is lower for two reasons. First, QBASE is factoring in that scouting consensus has Rudolph as a third-round selection, not a first-rounder. Second, unlike Mayfield, Rudolph is getting penalized for putting up his big stats against an easier Big 12 schedule. Mayfield had five games against pass defenses ranked in the S&P+ top 20, while Rudolph had only two (No. 15 TCU and No. 16 Texas).
Kyle Lauletta, Richmond (89)
|Mean Projection in Years 3-5:||273 DYAR|
|Bust (< 500 DYAR)||56.0%|
|Adequate Starter (500-1499 DYAR)||24.7%|
|Upper Tier (1500-2500 DYAR)||14.0%|
|Elite (>2500 DYAR)||5.4%|
Projecting FCS-level quarterbacks is extremely difficult. For Lauletta's projection, we approximated that the pass defenses of the Colonial Athletic Association are roughly equivalent to those in the lowest FBS conference, the Sun Belt. That would give the average defense faced by Lauletta a rank of 90.8, the worst of this year's top prospects.
Lauletta's 8.7 AYPA in his senior year is similar to what Carson Wentz (8.7) and Joe Flacco (8.6) had in their final FCS years, but is far behind the 10.0 APYA put up by Jimmy Garoppolo.
Luke Falk, Washington State (104)
|Mean Projection in Years 3-5:||277 DYAR|
|Bust (< 500 DYAR)||54.5%|
|Adequate Starter (500-1499 DYAR)||23.6%|
|Upper Tier (1500-2500 DYAR)||14.4%|
|Elite (>2500 DYAR)||7.6%|
Falk is right in the cusp of qualifying for QBASE, depending on where he is selected on draft day. The basic idea of only using QBASE on the top 100 picks is that if you drop to the third day of the draft, scouts have probably done a good job of determining that your great college stats are a sign of a system quarterback rather than a top NFL talent. But Falk didn't even have great college stats. His 6.8 APYA last year was even lower than Allen's 6.9, thanks to short throws (just 10.1 yards per completion) and 13 interceptions.
(This article originally appeared on ESPN Insider.)
30 comments, Last at 12 Apr 2018, 4:28pm
#1 by OldFox // Mar 29, 2018 - 1:44pm
The Cleveland media folks keep saying that the Browns' choice at #1 comes down to Sam Darnold or Josh Allen (!!!). Baker Mayfield supposedly is not even in the mix. Hopefully those media folks don't know what they're talking about. The Browns do have an analytics guy, Paul DePodesta, so maybe he can talk some sense into the team's decision-makers.
#4 by Led // Mar 29, 2018 - 3:59pm
I was going to say that clearing 1200 on QBASE is as much of a guaranty of non-bust status as any metric we have, but then I double checked Leftwich's numbers. He was not as solid as I had remembered, and I was surprised to see that he never started a full season. If he were a top 5 pick, he'd be at least a borderline bust. Still it's hard to imagine Mayfield totally tanking. I'm actually sold enough on him that I'd pick him #1 if I were the Browns.
#6 by jtr // Mar 29, 2018 - 5:16pm
General Browns stupidity is the main reason I think Allen has the potential to be a top-5 pick. The other possibility is that the Bills trade up, but not far enough to score Mayfield, Darnold, or Rosen, and take Allen just because they really really hate scrambling quarterbacks for some reason.
#17 by mehllageman56 // Mar 30, 2018 - 3:17pm
Honestly, we have no real idea where Allen is being graded by specific teams. According to Adam Schefter, one team has Lamar Jackson as their 2nd rated quarterback. A lot of the mock drafts are taking into account the Jets drafting Hackenberg, and assuming they love Allen. Mel Kiper is also pushing Josh Allen hard. I would love to see the Browns draft Mayfield or Jackson and see Kiper have a meltdown comparable to what he did when the Colts took Trev Alberts over Trent Dilfer. It'd be even better when the Browns guy does better than everybody else. Todd McShay seems like a much better guy to pay attention than the coiffed one.
#28 by Megamanic // Apr 04, 2018 - 5:29am
QBase Rate Jackson second so that's a rational grading.
Can I guess Chargers? They've had a private meeting with him. Also the last two times they drafted a QB in the first two rounds they went with a Lewin/QBase darling (Brees, Rivers).
The time before that, less so. I'm guessing the "Ryan Leaf Experience" drove them to try to figure out what caused Leaf to Flame-out while Manning flourished and might have led them to a statistical approach similar to Lewin which caused them to identify Brees as the better prospect than Dog-killer, trade down, get Brees and Tomlinson. Then 3 years later they do the same, trade down, get Rivers plus a slew of draft picks. Rivers is clearly better than Eli, it's not his fault he's been mostly saddled with mediocrity.
#9 by MarkV // Mar 30, 2018 - 1:37am
Wow, leftwich was much worse than I remembered him. Whats funny is if QBase had counted years 1-2, he has almost 1000 career dyar. Not great, but not awful, especially considering how he only started 3 years-(ish).
#10 by Theo // Mar 30, 2018 - 6:59am
Browns fans are mostly talking about Darnold.
Mayfield gets some love but hes short mobile and cocky, which opens some johnny football wounds that have not yet healed completely.
The top of the draft is going t9 be very very interesting.
#20 by OldFox // Mar 30, 2018 - 5:35pm
Yeah, I think Darnold is the most likely #1 pick, and I think he'll be an okay pro, but all of the indicators point toward Mayfield being really special, so I hope the Browns choose him instead. The Cleveland media folks don't seem to think much of Mayfield's chances, though, and I don't know whether that's because they have some inside information or because they're guessing like the rest of us.
The Manziel comparison doesn't concern me. Sure, Mayfield is cocky and Manziel was cocky, but that was never the issue with Manziel. Manziel didn't flop because he was cocky, he flopped because he was a drunk.
#14 by Karl Cuba // Mar 30, 2018 - 11:55am
I spent most of last spring wondering what all of the pundits weren’t seeing in Mitch* Trubisky, they all had him right at the end of the first when I was convinced he’d go in the top three (I said I’d have been happy for the niners to take him second overall on this site).
This year I just can’t work out why people don’t rate Rudolph much, much higher. You can point to his scheme but it’s the same as you’ve seen for Goff, Mahomes, Trubisky and Mayfield.
His release, accuracy, arm strength, timing and anticipation are about as good as I’ve seen from a college qb and while he’s slow, he does a fantastic job of keeping his eyes downfield before using that arm talent to get the ball in the hands of his playmakers.
I must be crazy because I think he’s the best qb in this class. Maybe there’s a bunch of off-field stuff like he’s a meth addict, walks around ‘balls out’ 24/7 or hits on his coaches' wives.
*This was before he demanded to be a full Mitchell.
#16 by mehllageman56 // Mar 30, 2018 - 3:09pm
I agree with you that Rudolph should be rated higher, but he has a weaker arm than the top five, with the possible exception of Jackson. Lamar Jackson posted a 49 mph at the combine while Rudolph posted a 52 mph, but Jackson barely uses his legs when throwing, while Rudolph has good mechanics for the most part. Unlike DeShaun Watson, Rudolph's lack of arm strength showed up in some games; check out footage from the 2016 Bedlam game against Oklahoma. Baker Mayfield had no problem with the weather, but Rudolph's ball was wobbly. When Rudolph isn't in bad weather conditions, his deep ball is the best in this class: accurate and a thing of beauty. It wouldn't surprise me if the Saints drafted him early; he'd fit in well there.
As far as off-field stuff for Rudolph, there isn't any. I actually like most of the top 8 guys in this draft, with the exception of Josh Allen.
#18 by mehllageman56 // Mar 30, 2018 - 3:24pm
Another thing I noticed; the percentages for each quarterback to be elite do not line up with their chances to bust. Josh Allen has a higher chance to be elite than Darnold or Lauletta; Luke Falk also has a much higher chance to be elite than Darnold.
#21 by Sifter // Mar 31, 2018 - 4:05am
OK boffins, who are the teams you think could/should take these 8 notables?
I see there as being 3 groups:
1) teams with older QBs: Steelers, Chargers, Giants, Patriots, Saints
2) teams with short-term/journeyman QBs: Jets, Cardinals, Bills, Browns, Broncos
3) teams that probably need an upgrade to win more (think Chiefs moving on from A.Smith in last years draft): Ravens, Bengals, Dolphins
Of group 1), the old QB teams, only Eli at the Giants is playing poorly. The other 4 feature heavily at the top of the QB DVOA charts. Maybe only the Giants draft a QB in that group.
Of group 2), most/all should probably be drafting a QB. Browns did trade a 3rd for Tyrod, and I think they could survive without a new QB, but it's hard to pass up the opportunity to pick 'your guy' at #1 (or #4...). Broncos might not take one either as Case Keenum got a decent (but short) deal. Probably depends who is available when they pick.
Of group 3), these are wildcards. Probably depends on who's available at their draft spots. Hard to guess whether these teams are ready to move on yet, similar to group 1) I suppose.
I'd say Giants, Jet, Bills and Cards are definites to draft a QB. Of the top 6 QBs, you'd think 4 of them would go to those teams. The other 9 teams I've listed don't need to spend a draft pick, but may, depending on if they feel they are getting good value at their draft spot. Although the Browns seem almost certain to take a QB, for that very reason of value I suppose. If these 5 teams draft a QB high, that would leave one more of the top 6 prospects to be scooped up early by one of the other 8 teams mentioned. Who will blink first? Then Lauletta and Falk to be drafted in the 3rd. Lauletta and the Patriots seems to be a much projected pairing. We'll see. Much speculating to be had!
#25 by OldFox // Apr 01, 2018 - 12:06pm
I agree with both of the last two comments. Yeah, I think that Browns fans would go berserk if the Browns failed to select a QB with the #1 pick, and yeah, I also think that at this point, Browns fans expect that any decision the Browns make will be the wrong one.
#26 by TecmoBoso // Apr 02, 2018 - 2:58pm
--I was not a Mayfield fan, but QBASE is making me reevaluate.
--I'm most excited about Jackson's career of these QBs. The limitations as a passer are obvious, but he's such an electric runner and seeing how NFL coaches appear to be better suited (more open minded?) to QBs with his skill set in today's NFL than compared to 15 years ago... I'm cautiously optimistic. I really hope he ends up somewhere with a strong, open minded, innovative coaching staff.
--Yes, Darnold was only a two year starter, but sat behind Cody Kessler as a frosh (redshirted) who was good college QB and has started in the NFL—this isn't like Trubisky not starting over Marquise Williams. Darnold then starts his next two years, aged 19 and 20. He's also 2+ years younger than Mayfield, which in baseball and basketball we'd view as a major advantage from a prospect perspective, but I realize QBs are different.
#30 by Aaron Schatz // Apr 12, 2018 - 4:28pm
Just a late comment here... I agree with you about age being an issue. I've tried a couple times to get age into QBASE in a way that is statistically significant but can never get it to work so it's not in there. But I agree, that's an advantage to a quarterback who comes out with only two years experience. I'd rather have a 21-year-old with two years experience than a 23-year-old with two years experience.