Expected Failed Completions
by Scott Kacsmar
One of the founding principles of Football Outsiders was that not every X-yard gain is of equal value. An 8-yard gain on first-and-10 is more valuable than an 8-yard gain on second-and-23. Those factors such as down, distance, and field position matter a great deal. Through game charting, we can take things a step farther with the passing game and break down those 8-yard gains, focusing on where the pass was thrown and what happened after the catch. Now that the game charting data is complete for the 2015 season, we have a decade of data on passes going back to 2006. This is the basis for stats such as receiving plus-minus, YAC+, and ALEX.
All of those stats need refined game charting due to our exclusion of certain incompletions, such as passes intentionally thrown away or batted down at the line. However, one related stat we can always quickly provide from the play-by-play is failed completions. These are any complete passes that fail to gain 45 percent of needed yards on first down, 60 percent on second down, or 100 percent on third/fourth down. You can see this year's study here.
As I mentioned in that article, we can look at the data associated with calculating YAC+ to create an expected failed completion (EFC) stat. Based on the down, distance, and field position, each throw has an expected amount of YAC. By adding together actual air yards and expected YAC, we get an Expected Gain for each pass, which is then used with the 45/60/100 baselines to determine if the completion was likely to be a success or failure. By comparing to the actual results, we can see which quarterbacks and receivers fell short or exceeded the expectations when it came to failed completions (FC).
Clearly, a high rate of successful plays will lead to success in DVOA and on offense in general. In the grand scheme of things, FCs do not top the list of the worst things a quarterback can do, but they are usually not good plays for an offense, especially on later downs. Since EFC rate is based on an expected YAC value, it should theoretically reflect more on the quarterback's skill than actual FC rate, which is more influenced by what the receiver did with the ball in his hands. For the period of 2006-2015, we had 102 quarterbacks with at least 300 pass attempts. Here are the top and bottom dozen from that group in EFC and FC rates. Data is only for the regular season, and the total number of completions is slightly higher than the official NFL total since Football Outsiders includes backward (lateral) passes.
|Expected Failed Completion Rate: Lowest & Highest Since 2006 (Min. 300 Passes)|
|1||Andrew Luck||1946||6.4%||91||Robert Griffin||994||14.2%|
|2||Peyton Manning||4785||6.5%||92||Zach Mettenberger||318||14.4%|
|3||Sage Rosenfels||419||7.4%||93||Charlie Whitehurst||327||14.6%|
|4||Tom Brady||4926||8.1%||94||JaMarcus Russell||628||15.0%|
|5||Tyler Thigpen||475||8.7%||95||Christian Ponder||974||15.0%|
|6||Tony Romo||4088||8.7%||96||Brady Quinn||499||15.2%|
|7||Byron Leftwich||406||8.8%||97||Trent Edwards||841||16.0%|
|8||Matt Moore||712||8.8%||98||Brandon Weeden||880||16.1%|
|9||Ben Roethlisberger||4546||8.8%||99||Damon Huard||604||17.2%|
|10||Drew Brees||5869||8.9%||100||Brad Johnson||480||17.3%|
|11||Aaron Rodgers||3731||9.2%||101||Jimmy Clausen||414||18.0%|
|12||Jake Delhomme||1294||9.2%||102||Blaine Gabbert||993||19.8%|
|Actual Failed Completion Rate: Lowest & Highest Since 2006 (Min. 300 Passes)|
|1||Sage Rosenfels||419||15.8%||91||Charlie Frye||466||30.1%|
|2||Tim Tebow||315||19.1%||92||Bruce Gradkowski||636||30.1%|
|3||Tom Brady||4926||19.3%||93||Tyrod Taylor||395||30.3%|
|4||Jameis Winston||504||19.9%||94||Kyle Boller||490||30.7%|
|5||Peyton Manning||4785||20.0%||95||Zach Mettenberger||318||30.8%|
|6||Derek Anderson||1419||20.5%||96||Charlie Whitehurst||327||31.1%|
|7||Trent Green||380||20.5%||97||Brad Johnson||480||31.4%|
|8||Tyler Thigpen||475||20.7%||98||Nick Foles||1126||31.7%|
|9||Andrew Luck||1946||20.8%||99||David Carr||571||31.8%|
|10||Ben Roethlisberger||4546||21.1%||100||Brady Quinn||499||33.0%|
|11||Cam Newton||2254||21.2%||101||Blaine Gabbert||993||33.6%|
|12||Austin Davis||349||21.2%||102||Jimmy Clausen||414||36.5%|
Rank them how you wish, but anyone following the NFL closely in this era will tell you that Peyton Manning, Tom Brady, Ben Roethlisberger, Drew Brees, and Aaron Rodgers have been the gold standard of quarterback play. All five rank in the top 11 for EFC rate, but not all five get there in actual FC rate. You also see some surprising names creep to the top: lesser quarterbacks who loved to sling it deep, such as Sage Rosenfels and Tim Tebow. The bottom of each table is filled with quarterbacks we have come to know as some of the worst of this era, including that dreadful 2007 draft with JaMarcus Russell, Brady Quinn, and Trent Edwards. Tyrod Taylor may be the only hope here, because otherwise the Browns are well represented and no one can really outdo the ineptitude of Jimmy Clausen and Blaine Gabbert. While we pick on Alex Smith for ALEX, it is actually Gabbert with three of the eight worst seasons in EFC rate.
Most of the quarterbacks that do poorly in EFC% do not remain starters for very long as you can see. Nick Foles is the only player in the bottom dozen of either table to surpass 1,000 attempts in this time period, and he was benched for Case Keenum last season. The five bottom quarterbacks in EFC% with at least 1,000 attempts are Foles, Alex Smith (13.4 percent), Derek Carr (13.0 percent), Jason Campbell (13.0 percent) and Josh McCown (12.6 percent).
What may stand out most here are the numbers themselves. Even the bottom-ranked quarterback in EFC rate (Gabbert) is still under 20 percent, while only four out of 102 passers have an actual FC rate under 20 percent. This speaks to the large variance you get in the passing game, and the different styles of offense out there. A 4-yard pass can be to a wide-open receiver or to someone wearing a defender as a coat, but you still never know when the receiver is going to duck out of bounds or when he'll break three tackles to turn a minor gain into a big play. Coverage and tackling are probably better than we give credit for, but some offenses are also better at getting receivers open, and some have the better talent to do explosive things after the catch. Quantifying the openness of receivers is still not something we can do with our charting, but maybe that is a dream that can be realized with some of the player tracking data that the league is starting to collect. For now, we have to settle for a decade of expected YAC data.
|Full NFL Failed Completions by Down, 2006-2015|
Where the expected and actual numbers really differ is on first and second down. They are about three times closer on the later downs when getting to the sticks is crucial. Since most first-down passes are of the first-and-10 variety, and since most completions gain more than 4 yards, the EFC rate is very low at 1.6 percent. Yet the actual observed rate of failure is 19.4 percent, and this is consistent for the 10 seasons, ranging from as low as 17.3 percent (2012) to as high as 21.0 percent (2015).
Of course, the actual performances of NFL offenses do not strongly follow what we have deemed through research to be a successful play that keeps drives on track. Many offenses today are content with the little screen pass on first-and-10 that may only gain a yard or two, giving that wide receiver an easy fantasy point in PPR leagues, but also a failed completion. If the alternative is to plow ahead with the ground game for a yard or 2, then maybe the failed completion is not so bad here, but offenses are not as concerned with the 45/60 guideline as we are. On first down, the average 2015 starter threw short of the 45-percent benchmark 41.8 percent of the time. On second down, it climbed to 45.4 percent. Fortunately, there is more of a league-wide effort to get the 100 percent on third and fourth down, but 40.3 percent of the passes were still short of the sticks on third down. That gets cut in half on fourth down, but we know very well some quarterbacks just cannot help themselves from checking down well short of the sticks time and time again in those key situations.
2015 Expected Failed Completions
Let's look at the EFC and FC results for the 36 qualified quarterbacks with at least 200 passes in 2015. Again, the FC data will differ from February's results due to the exclusion of certain passes through game charting. The table also includes bail-out completions (BOC), which are the rare times when a short pass was not expected to be a success, but became one due almost entirely to the YAC effort. There were 301 such completions in 2015, or 2.6 percent of all completions. Philip Rivers led the league with 19 BOCs. With numbers that low, the percentages were left out for space, but for those curious: Ryan Fitzpatrick (4.5 percent) had the highest BOC rate, and Andrew Luck (0.6 percent) had the lowest. Somehow that makes a lot of sense.
Here is how to read this table. Out of Aaron Rodgers' 347 completions, 94 of them were failed, but only 23 of those were expected to fail based on where he threw the ball and the average YAC expected in those situations. That surplus of 71 FCs was the highest total in 2015, and no quarterback had a larger difference in percentage points between his actual FC rate and his EFC rate than Rodgers at 20.5 percent -- the seventh-highest difference in any season since 2006.
|2015: Failed Completions vs. Expected Failed Completions (Min. 200 Passes)|
If someone asked which wide receiving corps was the most disappointing in 2015, a lot of people would say the Packers, especially after Jordy Nelson's preseason injury. We know about Davante Adams' historical ineffectiveness and the way James Jones could not separate in an offense that devolved into backyard football, hard counts to draw out free plays, and eventually the Hail Mary miracles. Now if you asked which wide receiver corps was the best in 2015, the most frequent answer might have been Arizona's cast of Larry Fitzgerald, John Brown, and Michael Floyd. It just so happens that Carson Palmer had the smallest difference between his FC rates at 7.3 percent, the fourth-lowest season in the last decade. (The lowest is actually Tom Brady's 2007 MVP season at 5.5 percent with that totally revamped receiving corps of Randy Moss, Wes Welker, and Donté Stallworth.)
[ad placeholder 3]
Now you might be ready to conclude that the difference in FC rates can mostly be explained by quality of receivers, but things rarely work out this neatly. For instance, Ben Roethlisberger may have been at his vertical best in 2015 with arguably his strongest cast yet, but he had his largest difference in FC rates (15.9 percent) in the last decade. Sure, his EFC rate was the lowest it has ever been at 6.6 percent, but he still had his share of FCs. Roethlisberger without question played his worst football in an injury-plagued 2006 season when he had a career-high EFC of 12.1 percent, yet his FC rate was never lower than it was that year, hence the third-lowest difference (7.1 percent) out of 337 qualified seasons.
The caliber of quarterback with a difference of more than 20 percentage points in FCs is usually nowhere close to Rodgers. We are talking about the likes of backup gunslinger Drew Stanton (2014 Cardinals); rookie Bruce Gradkowski (2006 Buccaneers); extreme dink-and-dunk David Carr (2006 Texans); without-a-paddle Matt Cassel (2011 Chiefs); the forgettable Charlie Frye (2006 Browns); and Harvard man Ryan Fitzpatrick (2009 Bills) as the only seasons ahead of Rodgers' 2015.
There are other stats we can look at to put these FC numbers into better context. The next table has three different measures. Short% is the percentage of a quarterback's completions that were thrown short of what was needed for a successful play, meaning he was going to rely on YAC to make the play gain enough yards. If a quarterback always threw the ball at least 5 yards on first-and-10, then he would never have to worry about a failed completion unless the receiver fumbled or had negative YAC. The second stat is Air Need%, which is the quarterback's average air yards divided by need yards. If it is greater than 100 percent, then the quarterback threw the ball to or beyond the first-down marker. The last stat is YAC+, which is the average YAC compared to what an average receiver would have gained in similar field position given that down-and-distance situation. Negative YAC+ means below-average performance. Like with our ALEX tables, the quarterbacks are highlighted by how much they deviate from the average, with shades of green implying good and red implying bad.
The contrast in some players really jumps out at you here. Some of the top YAC beneficiaries include quarterbacks we have come to know from YAC-based offenses, such as Brady, Alex Smith, and Teddy Bridgewater. Smith is not always this high, and last year happened with Jamaal Charles largely shelved. Brady's YAC+ was actually 0.71 thru Week 10, so the compounded losses of players such as Dion Lewis and Julian Edelman did not hurt his season's average. We also saw Philip Rivers and Matthew Stafford get into the dink-and-dunk style quite a bit in 2015, but they found more success with it than the putrid Blaine Gabbert.
But you can see how someone like Andy Dalton still did a very good job of attacking the benchmarks for a successful play and throwing to the sticks. His receivers just happened to be good after the catch as well. Arizona fared even better here, with no quarterback throwing more FC-avoidant passes than Palmer. Consider that Roethlisberger's receivers ranked third from the bottom there, which has to be one of the more surprising results of the season given the way Antonio Brown and Martavis Bryant played.
In the case of Rodgers, he simply was not very good at getting the ball down the field enough for more successful plays, while his receivers were more average than terrible in the YAC+ department. This is also supported when breaking things down by all passes thrown by down. Rodgers threw short of a successful play on 54.2 percent of his first-down passes, the third-worst mark in 2015 and nearly double that of Palmer (27.5 percent). On second down, Rodgers was 25th, and while he climbed to third on third down, we know converting those passes into completions was a real struggle all season. Like most quarterbacks with a high FC rate, Rodgers did his share to earn it. Rodgers should rebound in 2016 given his track record, but last season was a tough one for him.
[ad placeholder 4]
Getting to the bottom of the table, we are reminded that Jameis Winston had a strong rookie year in terms of avoiding FCs, but his vertical spraying of the ball to large targets will leave a lot to be desired in the YAC category. That offense needs to work in some more efficient throws in 2016. Nick Foles put a bow on the St. Louis Rams era with the highest FC rate on record (since 1989), but the Rams were lousy after the catch too. Jared Goff has his work cut out for him in Los Angeles.
Brian Hoyer, Ryan Mallett, and Tyrod Taylor rank near the bottom in YAC+, which can partially be blamed on the lack of YAC from Sammy Watkins and DeAndre Hopkins. We know Watkins' deep targets make it hard for him after the catch, but Hopkins' lack of YAC has been a consistent problem in an otherwise excellent start to his career. Mallett's accuracy can certainly be questioned when it comes to YAC, but his YAC+ was -1.06 in Baltimore compared to -1.98 in Houston, and -1.95 to Hopkins, specifically. Of course, that Baltimore number would still easily be the lowest in the league last year, so Mallett stands out for the wrong reasons here. We will see how Houston's offense changes after adding Brock Osweiler, Lamar Miller, Will Fuller, and Braxton Miller.
2015 Receivers: Expected Failed Completions
We will close with a brief look at FCs for wide receivers and tight ends in 2015. The receivers will get their own YAC+ study in due time, so this is more or less just looking at which players (minimum 30 receptions) had the smallest and largest differences in their FC rates.
|Largest FC% Difference (2015 WR/TE)|
|Smallest FC% Difference (2015 WR/TE)|
|108||Ted Ginn Jr.||CAR||44||3||1||6.8%||2.3%||4.5%|
The Packers added Jared Cook this offseason, so that actually gives them four of the top nine players here, which is really not a good thing. Eddie Royal had the highest EFC rate of any receiver in 2015, but the inclusion of Bryant is probably the biggest surprise in the top half. His stunning 88-yard touchdown against Arizona was a YAC beauty, but Bryant still does his best work down the field. He won't be doing any work in 2016 after a season-long suspension for repeated drug test failures.
Carolina was the opposite of Green Bay last year, with three players in the bottom table. Add Kelvin Benjamin to the mix again this year and Cam Newton and the Panthers should rank very low in FC rate and EFC rate again. While the 2015 rookie wideout class had nothing on 2014's legendary class, we see some interesting names here with Tyler Lockett, Dorial Green-Beckham, and Devin Funchess in position to get better. Surprising DVOA runner-up Rishard Matthews joined the Titans and should be a good fit with Marcus Mariota this season.
The reason we leave running backs out here is that they gobble up a lot of FCs each season. However, they also are more likely to turn sure-FC plays into successes. No player had more of those BOCs in 2015 than Atlanta's Devonta Freeman (seven). Danny Woodhead, Theo Riddick, and Bilal Powell had six each, while Darren Sproles, Charles Sims, and Mark Ingram were the only other players with at least five. The most by any wide receiver was four, done by two rookies: the aforementioned Lockett and Washington's Jamison Crowder. Marcedes Lewis led all tight ends with three for Jacksonville.
(This year's game charting data combines data from ESPN Stats & Information and Sports Info Solutions. Thanks to both organizations for their hard work in 2015.)