Expected Failed Completions

Expected Failed Completions
Expected Failed Completions
Photo: USA Today Sports Images

by Scott Kacsmar

One of the founding principles of Football Outsiders was that not every X-yard gain is of equal value. An 8-yard gain on first-and-10 is more valuable than an 8-yard gain on second-and-23. Those factors such as down, distance, and field position matter a great deal. Through game charting, we can take things a step farther with the passing game and break down those 8-yard gains, focusing on where the pass was thrown and what happened after the catch. Now that the game charting data is complete for the 2015 season, we have a decade of data on passes going back to 2006. This is the basis for stats such as receiving plus-minus, YAC+, and ALEX.

All of those stats need refined game charting due to our exclusion of certain incompletions, such as passes intentionally thrown away or batted down at the line. However, one related stat we can always quickly provide from the play-by-play is failed completions. These are any complete passes that fail to gain 45 percent of needed yards on first down, 60 percent on second down, or 100 percent on third/fourth down. You can see this year's study here.

As I mentioned in that article, we can look at the data associated with calculating YAC+ to create an expected failed completion (EFC) stat. Based on the down, distance, and field position, each throw has an expected amount of YAC. By adding together actual air yards and expected YAC, we get an Expected Gain for each pass, which is then used with the 45/60/100 baselines to determine if the completion was likely to be a success or failure. By comparing to the actual results, we can see which quarterbacks and receivers fell short or exceeded the expectations when it came to failed completions (FC).

Clearly, a high rate of successful plays will lead to success in DVOA and on offense in general. In the grand scheme of things, FCs do not top the list of the worst things a quarterback can do, but they are usually not good plays for an offense, especially on later downs. Since EFC rate is based on an expected YAC value, it should theoretically reflect more on the quarterback's skill than actual FC rate, which is more influenced by what the receiver did with the ball in his hands. For the period of 2006-2015, we had 102 quarterbacks with at least 300 pass attempts. Here are the top and bottom dozen from that group in EFC and FC rates. Data is only for the regular season, and the total number of completions is slightly higher than the official NFL total since Football Outsiders includes backward (lateral) passes.

Expected Failed Completion Rate: Lowest & Highest Since 2006 (Min. 300 Passes)
Rk Player Passes EFC% Rk Player Passes EFC%
1 Andrew Luck 1946 6.4% 91 Robert Griffin 994 14.2%
2 Peyton Manning 4785 6.5% 92 Zach Mettenberger 318 14.4%
3 Sage Rosenfels 419 7.4% 93 Charlie Whitehurst 327 14.6%
4 Tom Brady 4926 8.1% 94 JaMarcus Russell 628 15.0%
5 Tyler Thigpen 475 8.7% 95 Christian Ponder 974 15.0%
6 Tony Romo 4088 8.7% 96 Brady Quinn 499 15.2%
7 Byron Leftwich 406 8.8% 97 Trent Edwards 841 16.0%
8 Matt Moore 712 8.8% 98 Brandon Weeden 880 16.1%
9 Ben Roethlisberger 4546 8.8% 99 Damon Huard 604 17.2%
10 Drew Brees 5869 8.9% 100 Brad Johnson 480 17.3%
11 Aaron Rodgers 3731 9.2% 101 Jimmy Clausen 414 18.0%
12 Jake Delhomme 1294 9.2% 102 Blaine Gabbert 993 19.8%
Actual Failed Completion Rate: Lowest & Highest Since 2006 (Min. 300 Passes)
Rk Player Passes FC% Rk Player Passes FC%
1 Sage Rosenfels 419 15.8% 91 Charlie Frye 466 30.1%
2 Tim Tebow 315 19.1% 92 Bruce Gradkowski 636 30.1%
3 Tom Brady 4926 19.3% 93 Tyrod Taylor 395 30.3%
4 Jameis Winston 504 19.9% 94 Kyle Boller 490 30.7%
5 Peyton Manning 4785 20.0% 95 Zach Mettenberger 318 30.8%
6 Derek Anderson 1419 20.5% 96 Charlie Whitehurst 327 31.1%
7 Trent Green 380 20.5% 97 Brad Johnson 480 31.4%
8 Tyler Thigpen 475 20.7% 98 Nick Foles 1126 31.7%
9 Andrew Luck 1946 20.8% 99 David Carr 571 31.8%
10 Ben Roethlisberger 4546 21.1% 100 Brady Quinn 499 33.0%
11 Cam Newton 2254 21.2% 101 Blaine Gabbert 993 33.6%
12 Austin Davis 349 21.2% 102 Jimmy Clausen 414 36.5%

Rank them how you wish, but anyone following the NFL closely in this era will tell you that Peyton Manning, Tom Brady, Ben Roethlisberger, Drew Brees, and Aaron Rodgers have been the gold standard of quarterback play. All five rank in the top 11 for EFC rate, but not all five get there in actual FC rate. You also see some surprising names creep to the top: lesser quarterbacks who loved to sling it deep, such as Sage Rosenfels and Tim Tebow. The bottom of each table is filled with quarterbacks we have come to know as some of the worst of this era, including that dreadful 2007 draft with JaMarcus Russell, Brady Quinn, and Trent Edwards. Tyrod Taylor may be the only hope here, because otherwise the Browns are well represented and no one can really outdo the ineptitude of Jimmy Clausen and Blaine Gabbert. While we pick on Alex Smith for ALEX, it is actually Gabbert with three of the eight worst seasons in EFC rate.

Most of the quarterbacks that do poorly in EFC% do not remain starters for very long as you can see. Nick Foles is the only player in the bottom dozen of either table to surpass 1,000 attempts in this time period, and he was benched for Case Keenum last season. The five bottom quarterbacks in EFC% with at least 1,000 attempts are Foles, Alex Smith (13.4 percent), Derek Carr (13.0 percent), Jason Campbell (13.0 percent) and Josh McCown (12.6 percent).

What may stand out most here are the numbers themselves. Even the bottom-ranked quarterback in EFC rate (Gabbert) is still under 20 percent, while only four out of 102 passers have an actual FC rate under 20 percent. This speaks to the large variance you get in the passing game, and the different styles of offense out there. A 4-yard pass can be to a wide-open receiver or to someone wearing a defender as a coat, but you still never know when the receiver is going to duck out of bounds or when he'll break three tackles to turn a minor gain into a big play. Coverage and tackling are probably better than we give credit for, but some offenses are also better at getting receivers open, and some have the better talent to do explosive things after the catch. Quantifying the openness of receivers is still not something we can do with our charting, but maybe that is a dream that can be realized with some of the player tracking data that the league is starting to collect. For now, we have to settle for a decade of expected YAC data.

Full NFL Failed Completions by Down, 2006-2015
Down EFC% FC% Diff.
1st 1.6% 19.4% 17.8%
2nd 9.0% 24.8% 15.8%
3rd 26.7% 31.3% 4.6%
4th 8.8% 14.8% 6.0%

Where the expected and actual numbers really differ is on first and second down. They are about three times closer on the later downs when getting to the sticks is crucial. Since most first-down passes are of the first-and-10 variety, and since most completions gain more than 4 yards, the EFC rate is very low at 1.6 percent. Yet the actual observed rate of failure is 19.4 percent, and this is consistent for the 10 seasons, ranging from as low as 17.3 percent (2012) to as high as 21.0 percent (2015).

Of course, the actual performances of NFL offenses do not strongly follow what we have deemed through research to be a successful play that keeps drives on track. Many offenses today are content with the little screen pass on first-and-10 that may only gain a yard or two, giving that wide receiver an easy fantasy point in PPR leagues, but also a failed completion. If the alternative is to plow ahead with the ground game for a yard or 2, then maybe the failed completion is not so bad here, but offenses are not as concerned with the 45/60 guideline as we are. On first down, the average 2015 starter threw short of the 45-percent benchmark 41.8 percent of the time. On second down, it climbed to 45.4 percent. Fortunately, there is more of a league-wide effort to get the 100 percent on third and fourth down, but 40.3 percent of the passes were still short of the sticks on third down. That gets cut in half on fourth down, but we know very well some quarterbacks just cannot help themselves from checking down well short of the sticks time and time again in those key situations.

2015 Expected Failed Completions

Let's look at the EFC and FC results for the 36 qualified quarterbacks with at least 200 passes in 2015. Again, the FC data will differ from February's results due to the exclusion of certain passes through game charting. The table also includes bail-out completions (BOC), which are the rare times when a short pass was not expected to be a success, but became one due almost entirely to the YAC effort. There were 301 such completions in 2015, or 2.6 percent of all completions. Philip Rivers led the league with 19 BOCs. With numbers that low, the percentages were left out for space, but for those curious: Ryan Fitzpatrick (4.5 percent) had the highest BOC rate, and Andrew Luck (0.6 percent) had the lowest. Somehow that makes a lot of sense.

Here is how to read this table. Out of Aaron Rodgers' 347 completions, 94 of them were failed, but only 23 of those were expected to fail based on where he threw the ball and the average YAC expected in those situations. That surplus of 71 FCs was the highest total in 2015, and no quarterback had a larger difference in percentage points between his actual FC rate and his EFC rate than Rodgers at 20.5 percent -- the seventh-highest difference in any season since 2006.

2015: Failed Completions vs. Expected Failed Completions (Min. 200 Passes)
Player Team Passes Comp. FC EFC BOC FC% Rk EFC% Rk DIFF Rk
Aaron Rodgers GB 523 347 94 23 5 27.1% 23 6.6% 2 20.5% 1
Teddy Bridgewater MIN 395 292 96 37 9 32.9% 34 12.7% 28 20.2% 2
Nick Foles STL 311 190 80 43 5 42.1% 36 22.6% 36 19.5% 3
Ryan Tannehill MIA 540 364 106 37 6 29.1% 28 10.2% 12 19.0% 4
Peyton Manning DEN 314 198 59 22 3 29.8% 30 11.1% 17 18.7% 5
Joe Flacco BAL 383 266 77 31 8 28.9% 27 11.7% 22 17.3% 6
Tyrod Taylor BUF 364 242 74 34 7 30.6% 33 14.0% 30 16.5% 7
Ryan Mallett 2TM 229 136 38 16 3 27.9% 25 11.8% 23 16.2% 8
Ben Roethlisberger PIT 449 320 72 21 7 22.5% 12 6.6% 1 15.9% 9
Kirk Cousins WAS 517 379 102 42 9 26.9% 22 11.1% 16 15.8% 10
Sam Bradford PHI 492 346 102 49 13 29.5% 29 14.2% 31 15.3% 11
Alex Smith KC 439 308 88 41 7 28.6% 26 13.3% 29 15.3% 12
Eli Manning NYG 584 387 99 40 8 25.6% 19 10.3% 14 15.2% 13
Drew Brees NO 598 429 114 49 11 26.6% 21 11.4% 19 15.2% 14
Colin Kaepernick SF 229 144 43 22 5 29.9% 31 15.3% 33 14.6% 15
Josh McCown CLE 271 186 56 29 8 30.1% 32 15.6% 34 14.5% 16
Andrew Luck IND 269 162 36 13 1 22.2% 10 8.0% 4 14.2% 17
Tom Brady NE 589 402 98 43 11 24.4% 16 10.7% 15 13.7% 18
Player Team Passes Comp. FC EFC BOC FC% Rk EFC% Rk DIFF Rk
Brock Osweiler DEN 256 170 37 14 3 21.8% 8 8.2% 6 13.5% 19
Blaine Gabbert SF 269 178 64 40 6 36.0% 35 22.5% 35 13.5% 20
Ryan Fitzpatrick NYJ 532 335 84 39 15 25.1% 18 11.6% 21 13.4% 21
Derek Carr OAK 539 350 91 44 9 26.0% 20 12.6% 27 13.4% 22
Matt Ryan ATL 575 407 91 39 12 22.4% 11 9.6% 9 12.8% 23
Russell Wilson SEA 446 329 73 31 10 22.2% 9 9.4% 8 12.8% 24
Matt Hasselbeck IND 232 157 38 18 5 24.2% 15 11.5% 20 12.7% 25
Johnny Manziel CLE 212 129 35 19 3 27.1% 24 14.7% 32 12.4% 26
Jay Cutler CHI 462 311 75 37 10 24.1% 14 11.9% 24 12.2% 27
Philip Rivers SD 613 439 108 55 19 24.6% 17 12.5% 25 12.1% 28
Andy Dalton CIN 360 256 54 25 6 21.1% 6 9.8% 10 11.3% 29
Blake Bortles JAC 555 355 69 29 10 19.4% 3 8.2% 5 11.3% 30
Matthew Stafford DET 557 398 92 50 11 23.1% 13 12.6% 26 10.6% 31
Marcus Mariota TEN 346 230 50 26 6 21.7% 7 11.3% 18 10.4% 32
Jameis Winston TB 504 312 62 31 7 19.9% 4 9.9% 11 9.9% 33
Brian Hoyer HOU 349 224 45 23 3 20.1% 5 10.3% 13 9.8% 34
Cam Newton CAR 460 296 47 22 9 15.9% 1 7.4% 3 8.4% 35
Carson Palmer ARI 495 342 55 30 12 16.1% 2 8.8% 7 7.3% 36

If someone asked which wide receiving corps was the most disappointing in 2015, a lot of people would say the Packers, especially after Jordy Nelson's preseason injury. We know about Davante Adams' historical ineffectiveness and the way James Jones could not separate in an offense that devolved into backyard football, hard counts to draw out free plays, and eventually the Hail Mary miracles. Now if you asked which wide receiver corps was the best in 2015, the most frequent answer might have been Arizona's cast of Larry Fitzgerald, John Brown, and Michael Floyd. It just so happens that Carson Palmer had the smallest difference between his FC rates at 7.3 percent, the fourth-lowest season in the last decade. (The lowest is actually Tom Brady's 2007 MVP season at 5.5 percent with that totally revamped receiving corps of Randy Moss, Wes Welker, and Donté Stallworth.)

[ad placeholder 3]

Now you might be ready to conclude that the difference in FC rates can mostly be explained by quality of receivers, but things rarely work out this neatly. For instance, Ben Roethlisberger may have been at his vertical best in 2015 with arguably his strongest cast yet, but he had his largest difference in FC rates (15.9 percent) in the last decade. Sure, his EFC rate was the lowest it has ever been at 6.6 percent, but he still had his share of FCs. Roethlisberger without question played his worst football in an injury-plagued 2006 season when he had a career-high EFC of 12.1 percent, yet his FC rate was never lower than it was that year, hence the third-lowest difference (7.1 percent) out of 337 qualified seasons.

The caliber of quarterback with a difference of more than 20 percentage points in FCs is usually nowhere close to Rodgers. We are talking about the likes of backup gunslinger Drew Stanton (2014 Cardinals); rookie Bruce Gradkowski (2006 Buccaneers); extreme dink-and-dunk David Carr (2006 Texans); without-a-paddle Matt Cassel (2011 Chiefs); the forgettable Charlie Frye (2006 Browns); and Harvard man Ryan Fitzpatrick (2009 Bills) as the only seasons ahead of Rodgers' 2015.

There are other stats we can look at to put these FC numbers into better context. The next table has three different measures. Short% is the percentage of a quarterback's completions that were thrown short of what was needed for a successful play, meaning he was going to rely on YAC to make the play gain enough yards. If a quarterback always threw the ball at least 5 yards on first-and-10, then he would never have to worry about a failed completion unless the receiver fumbled or had negative YAC. The second stat is Air Need%, which is the quarterback's average air yards divided by need yards. If it is greater than 100 percent, then the quarterback threw the ball to or beyond the first-down marker. The last stat is YAC+, which is the average YAC compared to what an average receiver would have gained in similar field position given that down-and-distance situation. Negative YAC+ means below-average performance. Like with our ALEX tables, the quarterbacks are highlighted by how much they deviate from the average, with shades of green implying good and red implying bad.

The contrast in some players really jumps out at you here. Some of the top YAC beneficiaries include quarterbacks we have come to know from YAC-based offenses, such as Brady, Alex Smith, and Teddy Bridgewater. Smith is not always this high, and last year happened with Jamaal Charles largely shelved. Brady's YAC+ was actually 0.71 thru Week 10, so the compounded losses of players such as Dion Lewis and Julian Edelman did not hurt his season's average. We also saw Philip Rivers and Matthew Stafford get into the dink-and-dunk style quite a bit in 2015, but they found more success with it than the putrid Blaine Gabbert.

But you can see how someone like Andy Dalton still did a very good job of attacking the benchmarks for a successful play and throwing to the sticks. His receivers just happened to be good after the catch as well. Arizona fared even better here, with no quarterback throwing more FC-avoidant passes than Palmer. Consider that Roethlisberger's receivers ranked third from the bottom there, which has to be one of the more surprising results of the season given the way Antonio Brown and Martavis Bryant played.

In the case of Rodgers, he simply was not very good at getting the ball down the field enough for more successful plays, while his receivers were more average than terrible in the YAC+ department. This is also supported when breaking things down by all passes thrown by down. Rodgers threw short of a successful play on 54.2 percent of his first-down passes, the third-worst mark in 2015 and nearly double that of Palmer (27.5 percent). On second down, Rodgers was 25th, and while he climbed to third on third down, we know converting those passes into completions was a real struggle all season. Like most quarterbacks with a high FC rate, Rodgers did his share to earn it. Rodgers should rebound in 2016 given his track record, but last season was a tough one for him.

[ad placeholder 4]

Getting to the bottom of the table, we are reminded that Jameis Winston had a strong rookie year in terms of avoiding FCs, but his vertical spraying of the ball to large targets will leave a lot to be desired in the YAC category. That offense needs to work in some more efficient throws in 2016. Nick Foles put a bow on the St. Louis Rams era with the highest FC rate on record (since 1989), but the Rams were lousy after the catch too. Jared Goff has his work cut out for him in Los Angeles.

Brian Hoyer, Ryan Mallett, and Tyrod Taylor rank near the bottom in YAC+, which can partially be blamed on the lack of YAC from Sammy Watkins and DeAndre Hopkins. We know Watkins' deep targets make it hard for him after the catch, but Hopkins' lack of YAC has been a consistent problem in an otherwise excellent start to his career. Mallett's accuracy can certainly be questioned when it comes to YAC, but his YAC+ was -1.06 in Baltimore compared to -1.98 in Houston, and -1.95 to Hopkins, specifically. Of course, that Baltimore number would still easily be the lowest in the league last year, so Mallett stands out for the wrong reasons here. We will see how Houston's offense changes after adding Brock Osweiler, Lamar Miller, Will Fuller, and Braxton Miller.

2015 Receivers: Expected Failed Completions

We will close with a brief look at FCs for wide receivers and tight ends in 2015. The receivers will get their own YAC+ study in due time, so this is more or less just looking at which players (minimum 30 receptions) had the smallest and largest differences in their FC rates.

Largest FC% Difference (2015 WR/TE)
Rk Player Team Catches FC EFC FC% EFC% Diff.
1 Mychal Rivera OAK 32 13 4 40.6% 12.5% 28.1%
2 Richard Rodgers GB 58 20 4 34.5% 6.9% 27.6%
3 Randall Cobb GB 79 24 3 30.4% 3.8% 26.6%
4 Tavon Austin STL 53 23 9 43.4% 17.0% 26.4%
5 Jarvis Landry MIA 111 35 8 31.5% 7.2% 24.3%
6 Eddie Royal CHI 37 19 10 51.4% 27.0% 24.3%
7 Stefon Diggs MIN 52 14 2 26.9% 3.8% 23.1%
8 Jared Cook STL 39 13 4 33.3% 10.3% 23.1%
9 Davante Adams GB 50 12 1 24.0% 2.0% 22.0%
10 Maxx Williams BAL 32 9 2 28.1% 6.3% 21.9%
11 Martavis Bryant PIT 50 11 1 22.0% 2.0% 20.0%
12 Chris Hogan BUF 36 12 5 33.3% 13.9% 19.4%
Smallest FC% Difference (2015 WR/TE)
Rk Player Team Catches FC EFC FC% EFC% Diff.
108 Ted Ginn Jr. CAR 44 3 1 6.8% 2.3% 4.5%
109 T.Y. Hilton IND 69 8 5 11.6% 7.2% 4.3%
110 Mike Evans TB 74 4 1 5.4% 1.4% 4.1%
111 Tyler Lockett SEA 51 6 4 11.8% 7.8% 3.9%
112 Corey Brown CAR 31 3 2 9.7% 6.5% 3.2%
113 Donte Moncrief IND 64 10 8 15.6% 12.5% 3.1%
114 Torrey Smith SF 33 4 3 12.1% 9.1% 3.0%
115 Vincent Jackson TB 33 2 1 6.1% 3.0% 3.0%
116 Alshon Jeffery CHI 54 6 5 11.1% 9.3% 1.9%
117 Devin Funchess CAR 31 2 2 6.5% 6.5% 0.0%
118 Rishard Matthews MIA 43 2 2 4.7% 4.7% 0.0%
119 Dorial Green-Beckham TEN 32 0 0 0.0% 0.0% 0.0%

The Packers added Jared Cook this offseason, so that actually gives them four of the top nine players here, which is really not a good thing. Eddie Royal had the highest EFC rate of any receiver in 2015, but the inclusion of Bryant is probably the biggest surprise in the top half. His stunning 88-yard touchdown against Arizona was a YAC beauty, but Bryant still does his best work down the field. He won't be doing any work in 2016 after a season-long suspension for repeated drug test failures.

Carolina was the opposite of Green Bay last year, with three players in the bottom table. Add Kelvin Benjamin to the mix again this year and Cam Newton and the Panthers should rank very low in FC rate and EFC rate again. While the 2015 rookie wideout class had nothing on 2014's legendary class, we see some interesting names here with Tyler Lockett, Dorial Green-Beckham, and Devin Funchess in position to get better. Surprising DVOA runner-up Rishard Matthews joined the Titans and should be a good fit with Marcus Mariota this season.

The reason we leave running backs out here is that they gobble up a lot of FCs each season. However, they also are more likely to turn sure-FC plays into successes. No player had more of those BOCs in 2015 than Atlanta's Devonta Freeman (seven). Danny Woodhead, Theo Riddick, and Bilal Powell had six each, while Darren Sproles, Charles Sims, and Mark Ingram were the only other players with at least five. The most by any wide receiver was four, done by two rookies: the aforementioned Lockett and Washington's Jamison Crowder. Marcedes Lewis led all tight ends with three for Jacksonville.

(This year's game charting data combines data from ESPN Stats & Information and Sports Info Solutions. Thanks to both organizations for their hard work in 2015.)


24 comments, Last at 02 Jul 2016, 12:52am

#1 by TheOneEye // May 31, 2016 - 9:07pm

How large is the separation between mean and median expected YAC? This feels like a stat where median may be more representative than mean, especially if there's a sizable difference.

Points: 0

#2 by wrbrooks // May 31, 2016 - 10:46pm

I was just going to log in and make this comment. The only way I can understand the big gap between expected and actual FC% is if the expected YAC is a mean that is heavily influenced by a few short passes that turn into long TDs.

Working with the median YAC would give a better estimate of how many yards a receiver is likely to pick up after a typical reception.

Points: 0

#5 by Scott Kacsmar // Jun 01, 2016 - 12:21am

Maybe Aaron can answer this, but I'm pretty sure it's not a hard average derived from the data that's used. For instance, a 16-yard throw on 2nd-and-24 has happened twice in the last decade. Only one was completed, and that play had no YAC. The expected YAC was still 4.3 yards, so there is some kind of curve applied to this.

Points: 0

#8 by TheOneEye // Jun 01, 2016 - 9:15am

Even so, "expected YAC" can be designed to meet one of a couple different interpretations. I assume the "expected YAC" here is attempting to represent the "mean YAC" for that situation, though with some manner of fit over all situations and not based strictly off the mean produced in those specific situations in the past (as you allude to).

In the expected failed completion study (at the very least), the mean may not be the best representation of the distribution. Ideally, if your model included a full distribution, you could say that for an 10-yard thrown on 2nd-and-20, 42% of the time the receiver is expected to gain the 2 yards of YAC necessary to produce a "successful" completion. Actually making this determination requires a much more complex model. However, the pass/fail nature may mean that the distribution is better described by its median than its mean, and a model based on the median shouldn't be more computationally difficult than the one based on the mean.

In general, I tend to think that the "mean expectation" is better for summed statistics (total yards, etc.) while the "median expectation" is better for pass/fail scenarios (failed completions, 3rd Down conversion %, etc.).

Points: 0

#15 by wrbrooks // Jun 01, 2016 - 9:18pm

Indeed. I assume YAC+ is a regression model. If so, it is using the model structure to account for the fact that the data don't fall into a bunch of neat bins like 16 yard passes on 2nd-and-26. But most regression models are models of the expected value (mean).

If YAC+ were to be implemented using quantile regression, it could spit out the median YAC for any down-and-distance-by-air-yards combination. Or maybe you've implemented YAC+ as a kernel smoother? It's pretty easy to get a weighted median as opposed to a weighed mean.

None of this should detract from the point that EFC% is a really cool statistic. Nice work, guys!

Points: 0

#10 by nottom // Jun 01, 2016 - 10:24am

I have a hard time understanding this as well. Expected FC% suggests some sort of average FC% for a given throw, but the data seems to show that basically all QBs produce more failed completions than expected. This certainly implies that the expectations haven't been calibrated very well and the expected FC%s should be higher.

Points: 0

#20 by Dan // Jun 03, 2016 - 3:43am

Here's a better way to do it, if you have the data. Suppose that the QB completes the pass 8 yards downfield on 3rd & 10. Then, based on your data, you can estimate (let's say) that the receiver will get enough YAC for the first down 59% of the time in this situation. So you credit the quarterback with 0.59 successful completions and 0.41 failed completions (regardless of whether or not the receiver managed to pick up the necessary 2 yards after the catch in this case).

Points: 0

#3 by Scott C // May 31, 2016 - 11:55pm

I apologize for not reading the whole thing before posting, but only got this far before thinking : "no".

> Since EFC rate is based on an expected YAC value, it should theoretically reflect more on the quarterback's skill than actual FC rate, which is more influenced by what the receiver did with the ball in his hands.

If a QB has a known YAC machine of a WR, he may be more likely to throw an expected FC but be more likely to succeed. Its not the QB's poor skill to do this, so.... no the stat can't be used in isolation to judge a QB's skill. Its quite different to throw a quick slant to Keenan Allan short of the sticks on 3rd down than it is to throw a dump-off to an average WR. Furthermore some plays are specifically designed to give a WR that is good after the catch space to run. Judging the play of the qb by expected FC seems silly without considering the target and play.

Points: 0

#4 by Scott C // Jun 01, 2016 - 12:18am

OK, I finished the article.

Some of this is covered / discussed later.

There is a lot of context and nuance in these numbers on both the QB and WR side.

I wonder what Rivers' splits for the first and second half were. His WR and OL both completely collapsed mid year in a stark drop in performance for the whole offense. Charting the late season charger's backup WRs has them barely getting open meanwhile the OL is in shambles with pressure up the middle as Trevor Robinson whiffs block after block.

The first half had ok blocking and much better WR play. How that affected where the ball was thrown relative to the sticks would be interesting to know.

Points: 0

#6 by Scott Kacsmar // Jun 01, 2016 - 12:46am

Many of the receivers known as YAC machines earn that reputation by getting a lot of short passes that provide plenty of YAC room. Don't you think Sammy Watkins, especially with the way he was used at Clemson, could pile up more YAC if he was used that way instead of being such a deep threat in Buffalo? He's certainly athletic enough.

Keenan Allen has 0.1 YAC+ on 3rd-down throws short of the sticks since 2013, ranked 45th out of the 80 players with 20+ targets. He converted 9-of-29 plays for a first down. Antonio Gates would have been a nice choice. He's the only player to convert more than 50% of those throws into a first down (12-of-23 since 2013). His YAC+ is 3.6. It probably helps that he has the shortest ALEX of all 85 players, needing an average of 3.7 yards to convert whereas Allen needed 6.6.

I keep finding that stats that downplay what happens after the ball leaves the QB's hand tend to reflect better on the caliber of passer than those that don't. We'll look at this more with +/- next.

There's not a huge difference in the order between EFC% and FC%, but I think the EFC% lists tend to look better. Here's the leaders with a minimum of 1,000 attempts.

Lowest EFC% since 2006
1. Andrew Luck, 6.4%
2. Peyton Manning, 6.5%
3. Tom Brady, 8.1%
4. Tony Romo, 8.7%
5. Ben Roethlisberger, 8.8%
6. Drew Brees, 8.9%
7. Aaron Rodgers, 9.2%
8. Jake Delhomme, 9.2%
9. Jon Kitna, 9.3%
10. Matt Ryan, 9.4%

Lowest FC% since 2006
1. Tom Brady, 19.3%
2. Peyton Manning, 20.0%
3. Derek Anderson, 20.5%
4. Andrew Luck, 20.8%
5. Ben Roethlisberger, 21.1%
6. Cam Newton, 21.2%
7. Tony Romo, 21.8%
8. Philip Rivers, 21.8%
9. Matthew Stafford, 21.9%
10. Aaron Rodgers, 21.9%

Points: 0

#12 by Scott C // Jun 01, 2016 - 5:08pm

> I keep finding that stats that downplay what happens after the ball leaves the QB's hand tend to reflect better on the caliber of passer than those that don't. We'll look at this more with +/- next.

I agree with that in general. I'm not convinced that for this stat in particular that the difference between FC and EFC strongly indicates QB quality. That isn't the same thing as saying that EFC is not an improvement over FC. It is different, and tells us something 'more pure' about most qbs in most situations. How that relates to overal qb quality is rather fuzzy, since scheme and surrounding talent can affect the difference between FC and EFC quite a bit.

As for the two lists above, I'm not sure which one is better. One swings and misses on Derek Anderson, the other on Jon Kitna. Other than that its a who's who of mostly multi-year pro bowl QBs in both cases.

For most of these players, the year-to-year variation would be very interesting to help understand the stat. Every player here that has had a long career probably has had some big variations in scheme or supporting cast. Brady 2006 vs 2007, Rivers and Roethlisberger have both had several OC's calling plays. Manning had a variety of WR / OL quality. Romo has been all over the place too. Are there any trends when these good QBs have stronger running games, blocking, or reciever talent? Do they shift to throw more or less aggressively or is it something that is more consistent over a career?

Points: 0

#16 by Scott Kacsmar // Jun 02, 2016 - 1:22am

I might do a separate post before the season on the year-to-year correlation for various QB stats.

Points: 0

#7 by fyo // Jun 01, 2016 - 7:54am

The calendar reads 2016 and we get a JPEG of a table.

You'd think a quality site like Football Outsiders could manage an HTML table with colored cells.

It really is time for FO to update their CMS. Considering the technical nature of FO's readership, I would assume I'm not the only one who could easily (and has) created CMSes with easy support for advanced table layouts, including colored cells -- and, while we're at it, js sorting on all table headings.

Points: 0

#9 by Raiderjoe // Jun 01, 2016 - 10:24am

Would the clored cells let us do thigns with it or is it just to look nice? if latte, not sure of big deal. iof former, then yes tghat would be good.

Points: 0

#17 by fyo // Jun 02, 2016 - 4:13am

Right now, FO is using a JPEG of a table every time they want to use colored cells. I'd honestly rather have no colored cells if an image is the only way they can achieve it, but that's just me...

Points: 0

#18 by Scott Kacsmar // Jun 02, 2016 - 8:38am

Not every time. Just when I don't feel like pasting in "bgcolor=#code" 300+ times like this one would have needed. Unless someone can suggest a program that will produce perfect html code...

Points: 0

#19 by fyo // Jun 02, 2016 - 10:05am

Perfect html for the types of simple tables that you use here?

LibreOffice Calc does just fine, although there are probably countless examples.

Calc produces very clean code, arguably cleaner than what pops up in the source of the tables in this article. The significant differences are: Calc doesn't include tbody, uses b instead of strong, adds align for each cell and height for the first cell of each row, uses vertical whitespace (tabs).

Use of tbody is optional for the types of tables you use, but trivial to add if you really want it. The other differences could be handled with a few trivial regexp'es (e.g. "%s/^\t*//g" for tab-removal in vim, "%s/align=\"[a-z]*\"//g" for the aligns and similar for heights, and "%s/b>/strong>/g" for b to strong).

The source in the article contains some random spaces here and there that would not be replicated, of course ;). E.g. first td is on the same line as the tr begin tag, but with a space between them. And the tr begin tags have a space after "tr", for some reason. Most lines are indented one space, but a few (seemingly random) are not.

Points: 0

#13 by Scott C // Jun 01, 2016 - 5:13pm

What else:

restricted layout that does not adjust to screen width.

Slow clunky loading that is impossible to read until the page is mostly finished (its green until the white shows up)

Several other little bits. It still looks/feels like 2002.

Points: 0

#21 by OSS117 // Jun 03, 2016 - 8:41am

Guessing AirNeed% is more of an indicator of who is more/less likely to heave a 50 yard bomb attempt on 3rd and 1? Seems another way of looking at it might be length of pass relative to success%. How many passes traveled the yards necessary for success divided by total pass attempts, to see who is relying on their receivers and who is relying on their arm. A 1 yard pass on 3rd and 1 is treated the same as that 50 yard bomb.

Points: 0

#22 by Vincent Verhei // Jun 03, 2016 - 6:37pm

Seems another way of looking at it might be length of pass relative to success%. How many passes traveled the yards necessary for success divided by total pass attempts, to see who is relying on their receivers and who is relying on their arm.

That would just be the opposite of Short%, which is already in the table.

Points: 0

#24 by David180 // Jul 02, 2016 - 12:52am

As you pointed out, Tim Tebow was throwing some of the longest passes in the NFL with a lower completion percentage. This is why he ranked higher than Tom Brady in your stats.
In 2010, rookie Tebow threw 15.95 yards per completion with a 50% completion percentage and ranked 1st for TDs per completion and 5th for TDs per attempt. If Lance Ball had caught the TD pass that he dropped, Tebow would have also ranked 1st for TDs per attempt.
http://www.youtube.com/watch?v=KvxK_Xkoz8M Rookie Highlights
In their first 16 games,
Tim Tebow scored more TDs than Tom Brady in 109 fewer pass attempts.
Brady threw 18 TDs in 483 attempts with 13 ints and won the super bowl.
Tebow threw 19 TDs in 374 attempts with 9 ints
Tebow added 15 rushing scores and 937 yards rushing.

Tebow had a 47.3% completion percentage throwing 15.3 yards per completion.
Brady had a 63.9% completion percentage throwing 10.6 yards per completion.
Tebow scored every 19.6 attempts and Brady scored every 26.8 attempts.
Brady has scored every 18.2 attempts in his career.
It looks like Tebow's numbers have also held up against Brady's 2006-2015 seasons.
Throwing 31% longer passes works better than dinking and dunking for a completion
The difference between 60% and 48% is 4 completions in 30 attempts.
I think Tebow could throw 4 more short passes out of 30, but try teaching the other
QBs to average over 15 yards per completion.
You can't teach that. It indicates courage in the pocket to let your deep routes develop and arm strength with deep ball accuracy. I'd show you a clip of Tebow's short passes but he rarely threw them. He just ran for 5.4 yards per carry.
Brady Quinn once said that the coaches in Denver were considering putting in another QB to convert on 3rd down because Tebow's 3rd down conversion percentage was low. Then they discovered that Tebow was getting so many 1st downs on 1st and 2nd down that it wasn't really an issue. Tebow was averaging 15.3 yards per completion and 5.4 yards per carry. He also ran for 47 1st downs in his first 16.
Tebow holds an NFL record for throwing long passes
In his 15th start he broke the NFL record for the highest average gain in a playoff
game which has stood for 32 years. He threw 15.05 yards per attempt and 31.6 yards
per catch.
Highest Average Gain, Game (20 attempts)
15.05 Tim Tebow, AFC-FR: Denver vs. Pittsburgh, 2011 (21-316)
14.71 Terry Bradshaw, SB: Pittsburgh vs. Los Angeles, 1979 (21-309)
14.50 Peyton Manning, AFC-FR: Indianapolis vs. Denver, 2003 (26-377)

Points: 0

Save 10%
& Support Scott
Support Football Outsiders' independent media and . Use promo code WRITERS to save 10% on any FO+ membership and give half the cost of your membership to tip Scott.