Failed Completions 2013
by Scott Kacsmar
We have probably all been guilty at least once of judging a quarterback's performance by his incompletions and not acknowledging that some of the completions may actually have been bad plays too.
A completion that loses yards, a 5-yard gain on third-and-15, or the worst: a 2-yard pass on fourth-and-10. None of these plays are going to benefit the offense, but they help the quarterback's completion percentage and the receiver's reception total. Advanced stats are not quite to the level where we can credit the quarterback who throws a long incompletion on third-and-17 more than we credit the dump pass on third-and-13 -- hey, at least that guy tried to move the chains -- but tracking failed completions is a start.
What are failed completions? They are any complete passes that fail to gain enough yardage to count as a successful play based on these guidelines: 45 percent of needed yards on first down, 60 percent on second down, or 100 percent on third/fourth down.
I became very interested in failed completions after first reading about them on Football Outsiders a few years back. I kept track of them for years (without knowing any fancy name) when doing research on third downs, but it was good to see methods that could be used for every down-and-distance situation. The only criticism I think one could throw at this stat is that occasionally a failed completion on third down sets up a significantly shorter field goal or puts a team into field-goal range on what otherwise would have been a punt. It also helps when you can turn an awful third-and-long into a fourth-and-short that's manageable to go for, but overall I think those situations do not represent the vast majority of failed completions.
In preparing our totals for 2013 failed completions, I used only regular-season data. The total number of completions is slightly higher than the official NFL total since Football Outsiders includes backward (lateral) passes as pass plays and completions.
First, here is the rate of failed completions for every team along with where that team finished in pass offense DVOA.
| Offense - 2013 Failed Completions | |||||||
| Rk | Team | Completions | Failed | Pct. | Failed Yards | PASS DVOA | DVOA RK |
| 1 | SD | 378 | 70 | 18.5% | 236 | 52.5% | 2 |
| 2 | DET | 371 | 70 | 18.9% | 268 | 9.9% | 16 |
| 3 | DEN | 461 | 88 | 19.1% | 323 | 60.3% | 1 |
| 4 | SF | 244 | 50 | 20.5% | 250 | 31.8% | 4 |
| 5 | MIA | 357 | 77 | 21.6% | 278 | 4.0% | 20 |
| 6 | PHI | 310 | 67 | 21.6% | 256 | 30.0% | 5 |
| 7 | NE | 380 | 84 | 22.1% | 327 | 28.2% | 6 |
| 8 | DAL | 376 | 84 | 22.3% | 288 | 15.9% | 10 |
| 9 | SEA | 269 | 61 | 22.7% | 266 | 27.4% | 8 |
| 10 | NYG | 325 | 75 | 23.1% | 336 | -17.9% | 29 |
| 11 | GB | 366 | 85 | 23.2% | 338 | 12.2% | 13 |
| 12 | ARI | 365 | 85 | 23.3% | 339 | 10.8% | 15 |
| 13 | IND | 354 | 83 | 23.4% | 260 | 8.3% | 17 |
| 14 | PIT | 378 | 89 | 23.5% | 309 | 23.5% | 9 |
| 15 | CIN | 364 | 86 | 23.6% | 264 | 13.5% | 12 |
| 16 | CHI | 375 | 89 | 23.7% | 322 | 28.0% | 7 |
| Rk | Team | Completions | Failed | Pct. | Failed Yards | PASS DVOA | DVOA RK |
| 17 | NO | 447 | 107 | 23.9% | 357 | 35.9% | 3 |
| 18 | NYJ | 266 | 65 | 24.4% | 243 | -15.9% | 28 |
| 19 | STL | 301 | 74 | 24.6% | 291 | 0.1% | 21 |
| 20 | WAS | 358 | 90 | 25.1% | 379 | -13.9% | 26 |
| 21 | OAK | 298 | 76 | 25.5% | 276 | -20.3% | 31 |
| 22 | CAR | 292 | 75 | 25.7% | 288 | 11.5% | 14 |
| 23 | KC | 333 | 90 | 27.0% | 314 | 6.7% | 18 |
| 24 | ATL | 445 | 122 | 27.4% | 432 | 14.7% | 11 |
| 25 | TEN | 328 | 90 | 27.4% | 294 | 5.6% | 19 |
| 26 | BUF | 300 | 83 | 27.7% | 312 | -14.0% | 27 |
| 27 | HOU | 372 | 106 | 28.5% | 404 | -19.7% | 30 |
| 28 | MIN | 326 | 93 | 28.5% | 335 | -8.3% | 23 |
| 29 | TB | 292 | 84 | 28.8% | 311 | -1.7% | 22 |
| 30 | JAC | 349 | 101 | 28.9% | 398 | -24.2% | 32 |
| 31 | CLE | 380 | 110 | 28.9% | 417 | -9.2% | 25 |
| 32 | BAL | 363 | 112 | 30.9% | 349 | -9.0% | 24 |
| AVG | 347.6 | 85.0 | 24.5% | 314.4 | - | - | |
On average, 24.5 percent of completions in the NFL are not successful plays, averaging 3.7 yards per reception. As you probably expected, many of the elite passing games had the lowest rates of failed completions while the worst passing games had some of the highest. The correlation coefficient between failed completion rate and pass DVOA was -0.70, because DVOA is built on the concept of successful plays. Now some plays are bigger failures than others, which is where the fractional points in the DVOA "success point" system come into play, but it was no surprise to see strong correlation.
(DVOA, for example, does give credit for partial success for plays such as 13-yard completions on third-and-15.)
The Falcons led the league with 432 yards off of failed completions. That includes a minus-8-yard completion by defensive back Shann Schillinger on a blocked punt against the Jets. The completion actually saved a little bit of field position compared to a normal block, but this was not a typical play by any means.
No team had a larger differential in where they ranked in these two stats than the Giants, who were 10th at failed completion rate, but only 29th in DVOA. Of course, Eli Manning led the league in a special kind of failed completion: 27 passes completed to a player wearing the other team's jersey.
Next is a look at the 30 quarterbacks with at least 150 completions and their rate of failed completions. I also included the quarterback's DYAR and DVOA on these failed completions.
| Quarterbacks: 2013 Failed Completions (Min. 150 Completions) | ||||||||
| Rk | Quarterback | Completions | Failed | Pct. | DYAR | Rk | DVOA | Rk |
| 1 | 17-P.Rivers | 378 | 70 | 18.5% | -197 | 19 | -59.6% | 16 |
| 2 | 18-P.Manning | 450 | 84 | 18.7% | -222 | 23 | -58.3% | 11 |
| 3 | 9-M.Stafford | 371 | 70 | 18.9% | -201 | 21 | -64.5% | 19 |
| 4 | 12-J.McCown | 150 | 29 | 19.3% | -114 | 3 | -85.9% | 30 |
| 5 | 7-C.Kaepernick | 243 | 50 | 20.6% | -46 | 1 | -31.5% | 1 |
| 6 | 17-R.Tannehill | 355 | 76 | 21.4% | -195 | 17 | -61.4% | 18 |
| 7 | 12-T.Brady | 380 | 84 | 22.1% | -196 | 18 | -55.0% | 6 |
| 8 | 9-T.Romo | 343 | 77 | 22.4% | -188 | 16 | -54.2% | 5 |
| 9 | 10-E.Manning | 317 | 72 | 22.7% | -163 | 11 | -65.6% | 21 |
| 10 | 12-A.Rodgers | 193 | 44 | 22.8% | -54 | 2 | -38.3% | 2 |
| 11 | 9-N.Foles | 203 | 47 | 23.2% | -158 | 10 | -71.9% | 25 |
| 12 | 3-R.Wilson | 259 | 60 | 23.2% | -131 | 4 | -57.4% | 8 |
| 13 | 10-R.Griffin | 276 | 64 | 23.2% | -153 | 8 | -58.6% | 13 |
| 14 | 3-C.Palmer | 364 | 85 | 23.4% | -199 | 20 | -55.2% | 7 |
| 15 | 7-B.Roethlisberger | 376 | 89 | 23.7% | -285 | 28 | -68.4% | 24 |
| Rk | Quarterback | Completions | Failed | Pct. | DYAR | Rk | DVOA | Rk |
| 16 | 14-A.Dalton | 363 | 86 | 23.7% | -326 | 29 | -76.4% | 29 |
| 17 | 12-A.Luck | 347 | 83 | 23.9% | -254 | 24 | -64.7% | 20 |
| 18 | 9-D.Brees | 447 | 107 | 23.9% | -271 | 27 | -57.6% | 10 |
| 19 | 7-G.Smith | 247 | 60 | 24.3% | -183 | 15 | -73.7% | 26 |
| 20 | 1-C.Newton | 292 | 75 | 25.7% | -182 | 14 | -58.4% | 12 |
| 21 | 11-A.Smith | 308 | 80 | 26.0% | -264 | 25 | -75.1% | 27 |
| 22 | 6-J.Cutler | 225 | 60 | 26.7% | -136 | 6 | -58.9% | 14 |
| 23 | 3-E.Manuel | 180 | 49 | 27.2% | -134 | 5 | -66.7% | 22 |
| 24 | 2-M.Ryan | 439 | 120 | 27.3% | -267 | 26 | -52.6% | 3 |
| 25 | 4-R.Fitzpatrick | 217 | 61 | 28.1% | -221 | 22 | -75.1% | 28 |
| 26 | 7-C.Henne | 305 | 87 | 28.5% | -179 | 12 | -53.7% | 4 |
| 27 | 8-M.Schaub | 219 | 63 | 28.8% | -158 | 9 | -59.8% | 17 |
| 28 | 8-M.Glennon | 248 | 74 | 29.8% | -181 | 13 | -59.1% | 15 |
| 29 | 17-J.Campbell | 180 | 54 | 30.0% | -137 | 7 | -57.5% | 9 |
| 30 | 5-J.Flacco | 362 | 111 | 30.7% | -336 | 30 | -67.4% | 23 |
In dark corners of the internet, there rages an irrational Joe Flacco vs. Matt Ryan debate. In 2013, they led the league in failed completions with Flacco having the worst rate of anyone. Jason Campbell was the only other quarterback to hit 30 percent for failed completions, living up to his reputation of checking down when he shouldn't.
Most of the best quarterbacks in the league have a top 10 rate, but Drew Brees (18th) is one big exception. Given the Saints had 171 receptions by running backs, maybe that's not a surprise.
Josh McCown's completely unexpected season comes with another oddity. He was one of only four quarterbacks to be under 20 percent for failed completions, yet he had the worst DVOA (-85.9%) of anyone on those plays. Expecting to see McCown with a high rate of his failed completions coming on third/fourth down, I was surprised to see he still ranked 11th:
| Percentage of Failed Completions on 3rd/4th Down | |||||||||
| Rk | Quarterback | Failed | 3D | Rate | Rk | Quarterback | Failed | 3D | Rate |
| 1 | 9-T.Romo | 77 | 17 | 22.1% | 16 | 7-B.Roethlisberger | 89 | 31 | 34.8% |
| 2 | 18-P.Manning | 84 | 19 | 22.6% | 17 | 8-M.Schaub | 63 | 22 | 34.9% |
| 3 | 4-R.Fitzpatrick | 61 | 16 | 26.2% | 18 | 3-C.Palmer | 85 | 31 | 36.5% |
| 4 | 2-M.Ryan | 120 | 32 | 26.7% | 19 | 3-R.Wilson | 60 | 22 | 36.7% |
| 5 | 5-J.Flacco | 111 | 32 | 28.8% | 20 | 12-T.Brady | 84 | 31 | 36.9% |
| 6 | 12-A.Luck | 83 | 24 | 28.9% | 21 | 17-J.Campbell | 54 | 20 | 37.0% |
| 7 | 1-C.Newton | 75 | 22 | 29.3% | 22 | 10-R.Griffin | 64 | 24 | 37.5% |
| 8 | 12-A.Rodgers | 44 | 13 | 29.5% | 23 | 11-A.Smith | 80 | 30 | 37.5% |
| 9 | 9-N.Foles | 47 | 14 | 29.8% | 24 | 7-G.Smith | 60 | 23 | 38.3% |
| 10 | 9-D.Brees | 107 | 32 | 29.9% | 25 | 8-M.Glennon | 74 | 29 | 39.2% |
| 11 | 12-J.McCown | 29 | 9 | 31.0% | 26 | 9-M.Stafford | 70 | 29 | 41.4% |
| 12 | 14-A.Dalton | 86 | 27 | 31.4% | 27 | 7-C.Henne | 87 | 38 | 43.7% |
| 13 | 17-P.Rivers | 70 | 23 | 32.9% | 28 | 17-R.Tannehill | 76 | 34 | 44.7% |
| 14 | 6-J.Cutler | 60 | 20 | 33.3% | 29 | 10-E.Manning | 72 | 34 | 47.2% |
| 15 | 3-E.Manuel | 49 | 17 | 34.7% | 30 | 7-C.Kaepernick | 50 | 25 | 50.0% |
Someone has to be on the receiving end of these plays. Finally, I looked at the failed completions for everyone with at least 30 receptions. The following table excludes running backs since they dominate these lists with all the short passes they catch. For those curious, Ben Tate had a league-high 62.9 failed completion rate.
| Most Failed Completions | Lowest Failed Completion Rate (WR/TE) | Highest Failed Completion Rate (WR/TE) | ||||||||||
| Receiver | Team | Failed | Receiver | Team | Rec. | Failed | Pct. | Receiver | Team | Rec. | Failed | Pct. |
| 11-J.Edelman | NE | 28 | 81-C.Johnson | DET | 84 | 3 | 3.6% | 18-A.Sanders | JAC | 51 | 23 | 45.1% |
| 88-P.Garcon | WAS | 27 | 12-M.Colston | NO | 75 | 6 | 8.0% | 84-C.Patterson | MIN | 45 | 17 | 37.8% |
| 13-K.Wright | TEN | 25 | 16-L.Moore | NO | 37 | 3 | 8.1% | 89-J.Carlson | MIN | 32 | 11 | 34.4% |
| 84-A.Brown | PIT | 24 | 89-J.Cotchery | PIT | 46 | 4 | 8.7% | 15-D.Bess | CLE | 42 | 14 | 33.3% |
| 18-A.Sanders | JAC | 23 | 15-M.Floyd | ARI | 65 | 6 | 9.2% | 80-E.Bennett | CHI | 32 | 10 | 31.3% |
| 84-J.Cameron | CLE | 20 | 85-K.Thompkins | NE | 32 | 3 | 9.4% | 11-T.Austin | STL | 40 | 12 | 30.0% |
| 80-A.Johnson | HOU | 19 | 10-D.Hopkins | HOU | 52 | 5 | 9.6% | 82-K.Rudolph | MIN | 30 | 9 | 30.0% |
| 11-L.Fitzgerald | ARI | 18 | 88-D.Bryant | DAL | 93 | 9 | 9.7% | 18-G.Little | CLE | 41 | 12 | 29.3% |
| 13-T.Hilton | IND | 18 | 82-M.Jones | CIN | 51 | 5 | 9.8% | 22-D.McCluster | KC | 53 | 15 | 28.3% |
| 83-M.Bennett | CHI | 18 | 13-K.Allen | SD | 71 | 7 | 9.9% | 85-T.Eifert | CIN | 39 | 11 | 28.2% |
It is fitting that the consensus-best receiver in the game, Calvin Johnson, had the lowest failed completion rate. That helps explain some of the Detroit numbers from above. Elsewhere in the NFC North, the Vikings have three of the seven worst failed completion rates, which probably says more about the unholy trinity of quarterback play (Christian Ponder, Josh Freeman and Matt Cassel) than the receivers.
This is a good spot to highlight the greatest failed completion of the 21st century. In Week 9, the Steelers faced a third-and-30 against the Patriots. Ben Roethlisberger completed a screen pass to Le'Veon Bell and after a missed tackle, Bell gained 29 yards on the final play of the first quarter. Technically, that's a failed completion. The Steelers did go for it on fourth-and-1 at the New England 42 and Bell converted on the ground, which never would have happened without the big gain on the screen.
That play is the longest completion since 1999 that came on third or fourth down but failed to convert for a first down.
In closing, let's look at how defenses fared at forcing failed completions. An alarming stat from 2013 would be the Carolina Panthers allowing a 66.6 completion rate. We know their overall defense was great, but the secondary was suspect. However, no defense was better at forcing failed completions than the Panthers:
| Defenses - 2013 Failed Completions | ||||
| Rk | Team | Completions | Failed | Pct. |
| 1 | CAR | 377 | 128 | 34.0% |
| 2 | DET | 339 | 97 | 28.6% |
| 3 | SEA | 310 | 88 | 28.4% |
| 4 | NO | 304 | 83 | 27.3% |
| 5 | CIN | 363 | 98 | 27.0% |
| 6 | BAL | 314 | 84 | 26.8% |
| 7 | KC | 336 | 89 | 26.5% |
| 8 | TB | 352 | 92 | 26.1% |
| 9 | OAK | 375 | 98 | 26.1% |
| 10 | SF | 348 | 90 | 25.9% |
| 11 | DEN | 357 | 92 | 25.8% |
| 12 | IND | 325 | 82 | 25.2% |
| 13 | NYJ | 345 | 87 | 25.2% |
| 14 | NYG | 361 | 90 | 24.9% |
| 15 | JAC | 357 | 89 | 24.9% |
| 16 | PIT | 329 | 82 | 24.9% |
| Rk | Team | Completions | Failed | Pct. |
| 17 | BUF | 310 | 77 | 24.8% |
| 18 | TEN | 344 | 83 | 24.1% |
| 19 | SD | 364 | 87 | 23.9% |
| 20 | MIN | 421 | 100 | 23.8% |
| 21 | CLE | 363 | 86 | 23.7% |
| 22 | DAL | 405 | 95 | 23.5% |
| 23 | HOU | 287 | 67 | 23.3% |
| 24 | ARI | 367 | 85 | 23.2% |
| 25 | MIA | 344 | 79 | 23.0% |
| 26 | STL | 356 | 79 | 22.2% |
| 27 | GB | 332 | 72 | 21.7% |
| 28 | CHI | 315 | 66 | 21.0% |
| 29 | ATL | 341 | 71 | 20.8% |
| 30 | WAS | 338 | 68 | 20.1% |
| 31 | NE | 336 | 64 | 19.0% |
| 32 | PHI | 408 | 73 | 17.9% |
Comments
56 comments, Last at 02 Apr 2014, 3:31pm
#1 by JoeyHarringtonsPiano // Mar 27, 2014 - 12:18pm
great stuff.
"An alarming stat from 2013 would be the Carolina Panthers allowing a 66.6 completion rate. We know their overall defense was great, but the secondary was suspect. However, no defense was better at forcing failed completions than the Panthers:"
This what you like to see from advanced stats: confirming what our eyes are telling us (In this case, that the Panthers front 7 made up for a weak secondary by forcing the QB to get rid of the ball early and settle for checkdowns).
#2 by stanbrown // Mar 27, 2014 - 1:13pm
I don't think there is anything significant to be learned from the identity of the receiver on a failed completion. Unless he didn't run his route properly, it's not his fault that the QB had to throw it to him. Perhaps, he doesn't make people miss on WR screens or quick under routes, but I'm not sure this stat is going to be much help. Blown up screens are usually the fault of a missed block, not the receiver.
#36 by dcaslin // Mar 28, 2014 - 12:17am
Isn't a common problem where a receiver cuts his route 1 yard short for an easy completion that fails to convert on 3rd and 10? While the QB could arguably share some blame there, that's mostly on the receiver right? (Admittedly this is only one of many possible causes)
#3 by nat // Mar 27, 2014 - 1:19pm
It seems like you're looking at the wrong ratio here. We don't really need to look at the percentage of completions that were failures. We need the percentage of pass plays that are failed completions.
For example, your chart shows Rivers and P. Manning as being the best at avoiding failed completions. But of your top five, it's actually Stafford and Kaepernick who are least likely to have a pass end in a failed completion (then Rivers, McCown, and Manning). And who knows about the rest of the list?
You're trying to get to a statement like "Joe Schmo-Thrower has an usually high tendency to complete passes that nonetheless are failures". But the ratio that you use can't get you there, can it? Joe might be perfectly good at choosing the right target to throw to, fine at hitting underneath routes in stride, but inaccurate on longer passes.
This is kind of like judging baseball players on the percentage of their hits that go for extra bases. It would be much more useful to study the percentage of their plate appearances that result in extra base hits.
#22 by Eddo // Mar 27, 2014 - 5:19pm
I disagree. This is interesting to note as a supplement to completion percentage. Sure, QB X might have a surprisingly high completion percentage, but that's because he also has a high failed completion percentage.
What you're describing is just the inverse of success rate.
#30 by nat // Mar 27, 2014 - 7:19pm
You've missed my point. I suggest he use the "failed completion percentage". What he's using is something different. It's percent of all pass plays that are failed completions divided by the completion percentage, or more simply, failures per completion.
It's analogous to Yards per Attempt vs. Yards per Completion. In the bad old days, people focused on Y/C, and worshipped mad bomber QBs. Now we know better.
#4 by nat // Mar 27, 2014 - 2:02pm
Sorry all for the formatting of the following list...
2103 Percentage of Pass Plays Resulting in Failed Completions
QB Pct. Rank
9-M.Stafford 10.64% 1
7-C.Kaepernick 10.96% 2
17-R.Tannehill 11.82% 3
17-P.Rivers 12.17% 4
10-E.Manning 12.20% 5
12-J.McCown 12.29% 6
7-G.Smith 12.37% 7
18-P.Manning 12.37% 8
12-T.Brady 12.63% 9
10-R.Griffin 12.93% 10
3-R.Wilson 13.25% 11
9-T.Romo 13.46% 12
9-N.Foles 13.54% 13
12-A.Luck 13.70% 14
3-C.Palmer 13.84% 15
14-A.Dalton 13.98% 16
12-A.Rodgers 14.19% 17
7-B.Roethlisberger 14.33% 18
1-C.Newton 14.62% 19
3-E.Manuel 14.63% 20
11-A.Smith 14.68% 21
9-D.Brees 15.57% 22
7-C.Henne 16.08% 23
8-M.Glennon 16.12% 24
6-J.Cutler 16.17% 25
17-J.Campbell 16.27% 26
4-R.Fitzpatrick 16.53% 27
8-M.Schaub 16.58% 28
5-J.Flacco 16.79% 29
2-M.Ryan 17.27% 30
Scott's chart misleads us for Geno Smith, Peyton Manning, Aaron Rodgers, and Matt Ryan. The rest are with 4 in rank.
#5 by Scott Kacsmar // Mar 27, 2014 - 2:10pm
If you use every pass, aren't you just piling on more failure (incompletions) and getting away from the original focus? All passes would penalize QBs who are very good at completing passes. Peyton, Rodgers and Ryan were all top 5 in completion percentage. Can't have a FC if you don't complete the pass. A 2-yard pass on 2nd-and-8 still beats having an incompletion.
#10 by nat // Mar 27, 2014 - 2:45pm
A more complete picture would show all the various rates: Successful completion, failed completion, incomplete, interception, sack, and a few odd cases involving fumbles and busted plays. We could look at all the different ways QBs fail on pass plays. But the numbers would all be rates per pass play.
I do like the idea of looking at failed completions. But the relevant question is each QB's propensity for throwing them. Using a percentage relative to completions just clouds that picture.
For example, you honestly thought that Flacco was more likely to throw a failed completion than Ryan and used Flacco's picture to headline the article. While it's close, it turns out that Ryan is the one more likely to throw one. It's just that Flacco is worse at other things (completing passes in general). Your choice of ratio led you to emphasize the wrong thing about each QB, and ultimately to the wrong photo. Oops.
Ditto for Peyton Manning. His season was good yet not all that unusual for avoiding failed completions. But your choice of ratio led you to that false conclusion.
#40 by Beavis // Mar 28, 2014 - 11:32am
You're just giving people credit for not completing passes. Not completing a pass doesn't make them a better QB in my book.
QB A, attempts 400 passes, completes 200, 50 of which are failures
QB B, attempts 400 passes, completes 300, 75 of which are failures
According to the article both A and B have failure rates of 25%
You are suggesting that the correct numbers are 12.5% for A, and 18.8% for B, so A is the better QB by your measure. But, I think everyone would agree that B is the better QB.
I think the correct number to use is actually (incompletions + failed completions)/ attempted passes
For QB A that gives a value of 62.5%, and for QB B 43.8%
#41 by nat // Mar 28, 2014 - 1:13pm
Well, clearly QB B is more likely to throw a failed completion on any given play. That's indisputable.
That doesn't mean he's worse, and I never said so. It means he is more likely to do this one thing, which was the topic of the article.
QB B is also much more likely to throw a successful completion, with 75 more for the same number of attempts.
The complete picture with this data would be that QB B is more likely than QB A to throw failed completions (+25 out of 400 attempts), much more likely to throw successful completions (+75), and much less likely to throw incomplete passes (-100).
Your preferred number is 1 - success rate. That's a perfectly fine number, although it doesn't make a statement about failed completions themselves, which was the point of the article.
Scott's approach is a bit of a mess, and a step back from DVOA or DYAR. Sometimes it's fun to ask "How much could we figure out by just focusing on failed completions?" I just think he took it a step too far by excluding the total attempts from the ratio.
#47 by Scott Kacsmar // Mar 31, 2014 - 12:56pm
It's not my approach. I did the article the way it's been done in the past on this site and I see no problem with that. There's a level of skill involved in completing a pass, so let's harness that skill. If I'm looking at which QBs complete the most passes that are not successful plays, then why would I care about the passes they don't complete? Again, you're raining down more failure to cover up the original area of focus. We have other big-picture stats for that. Failed completions were never meant as a be-all and end-all stat, and I do think it matters a lot more on 3rd/4th down.
#48 by nat // Mar 31, 2014 - 1:34pm
If I'm looking at which QBs complete the most passes that are not successful plays, then why would I care about the passes they don't complete?
Why would you look at the successful passes they complete either? If "most failed completions" was what you wanted, you wouldn't use a ratio at all.
If you wanted to know which QB throws unsuccessful completions at the highest rate, then you would use a ratio of "failed completions" to all pass plays, not just to completions.
It's a classic blunder. It's like confusing Yards/Completion for Yards/Attempt. It's like confusing an offensive line that has a bad "quick sacks as percentage of all sacks" ratio with one that gives up "quick sacks" at a high rate. (Hint: you need to look at quick sacks per pass play to get the right answer).
If you're going to look at rate stats, it is critical to know the implications of the denominator you choose, and to pick the right one.
#49 by Scott Kacsmar // Mar 31, 2014 - 1:52pm
"If you wanted to know which QB throws unsuccessful completions at the highest rate, then you would use a ratio of "failed completions" to all pass plays, not just to completions."
That's what you think should be done. I disagree. If I'm looking at failed completions, I'm still looking at passes that were completed, so that's the ratio I care about.
Situational stats - http://www.cbssports.com/nfl/players/player/splits/2013/420095
See the "1ST %" stat? That's basically what I'm calculating, though each down has its own success rate and it's not always 100%. See how Matt Ryan has a 100% 1ST % on 4th down (3/3)? His actual conversion rate is 3/9 (33.3%), but 1ST % is looking at which percentage of completions gained a first down.
That's all we're doing here -- looking at what percentage of completions failed to gain enough yards to be considered successful plays.
#50 by nat // Mar 31, 2014 - 4:39pm
That's all fine. But you can't then say "Flacco having the worst rate of anyone...". Because he definitely does NOT throw failed incomplete passes at the highest rate. (Although his rate is not good!)
This is not just a terminology problem, or a matter of taste. You are doing the equivalent of looking at Joe Schmo's yards per completion and then saying "Joe Schmo gains passing yards at the worst rate." Completions is a bad choice of denominator if you want to reach conclusions like that. You need to use something like attempts.
#51 by LionInAZ // Apr 01, 2014 - 8:49pm
Any idiot could work out the stat that you think is so much more important with just 10 minutes worth of arithmetic based on the the data presented here and make their own conclusions. The failed completion data shown here is value-added data, and the results presented here are interesting in their own right. I can't understand why you insist on bashing the authors for not doing your preferred analysis. You could have easily worked up your own numbers and presented them for discussion, but instead you chose to criticize the article.
For what it's worth, I agreed that looking at failed completions plus incompletions was interesting, but your attitude is so obnoxious that I can't defend it as a positive contribution.
#52 by nat // Apr 02, 2014 - 9:56am
You could have easily worked up your own numbers and presented them for discussion, but instead you chose to criticize the article.
See comment #4 at the top of this very thread. Did you really want more of my "own numbers" than that?
Notice the follow on discussion. In the back and forth, we cover the problem with using failed completion rate as meaning simply "better QB", and the problem with using a "per completion" ratio was meaning "rate that it happens". There are examples, worked out math, appeals to authority (not always the best thing, but Scott is allowed if he wants to), and clarifying analogies.
Of course we can criticize what looks to be a flawed stats article. This is FO.
So please don't be so quick to post rude comments. Join the discussion instead, if it's a discussion that you want. Actually look at the numbers that are posted or post your own, if it's numbers that you need.
#6 by mehllageman56 // Mar 27, 2014 - 2:19pm
Pretty interesting grist for the mill. My takeaways: Tannehill ranks really high up on both lists, which fits what I see from him. Give him an offensive line, and he'd be really efficient. That EJ Manuel and Geno Smith aren't all the way on the bottom of the lists gives some hope for their futures. I'm also suprised that there were no Jets receivers on the failed completions list, but they probably didn't catch enough passes to qualify.
#7 by Karl Cuba // Mar 27, 2014 - 2:22pm
So Kaepernick is really good at avoiding failed completions, which makes sense for a guy who is a much better deep passer than he is at the short stuff.
But he stinks at it on third down. I wonder why, other than a sample size anomaly. Weird.
#16 by Perfundle // Mar 27, 2014 - 4:06pm
For a team that was pretty good at offense, the Niners had a surprisingly high percentage of third-down plays. They were in 8th place, behind the Ravens, Jets, Buccaneers, Raiders, Jaguars, Titans and Browns, pretty much all awful offenses. Because of that, Kaepernick had a very high percentage of his throws that occurred on third or fourth down, at 30.3%; for comparison, Romo was at 24.1% and P. Manning at 24.6%. Kaepernick still throws a high percentage of failed completions on third down, but at least he's now tied for 19th place at 31.3%, instead of dead-last. This is the full list:
P.Manning 19.0%
P.Rivers 22.5%
A.Rodgers 24.1%
R.Fitzpatrick 24.6%
J.McCown 25.0%
T.Romo 25.8%
M.Stafford 27.1%
N.Foles 27.5%
A.Dalton 27.6%
R.Wilson 27.8%
R.Griffin 28.6%
G.Smith 28.8%
D.Brees 29.1%
M.Ryan 29.4%
C.Newton 30.1%
J.Flacco 30.2%
T.Brady 31.0%
J.Cutler 31.3%
C.Kaepernick 31.3%
A.Luck 32.0%
B.Roethlisberger 32.3%
C.Palmer 32.6%
R.Tannehill 34.3%
E.Manuel 35.4%
E.Manning 36.2%
A.Smith 37.5%
M.Schaub 38.6%
M.Glennon 39.2%
J.Campbell 40.8%
C.Henne 42.2%
#18 by Deelron // Mar 27, 2014 - 4:19pm
I don't find it all that surprising that they had so many 3rd down plays, it jives pretty well with having running a ton of the time relatively ineffectively (14th DVOA, terrible Run Blocking stats) with an efficient (4th DVOA) passing attack.
#28 by Karl Cuba // Mar 27, 2014 - 6:43pm
OK, apparently the niners had the 6th longest yards to go on 3rd down. Isn't context nice?
Though it should be pointed out that teams know the niners love to run and so are probably stacking the box and making the pass game easier.
#8 by Perfundle // Mar 27, 2014 - 2:25pm
The QB DYAR on failed completions isn't very interesting, since it's basically an inverse ordering of the number of passes each QB threw. What does QB DVOA on non-failed completions look like? I would like to see if there are any surprise names on the bottom of that list.
#9 by Scott Kacsmar // Mar 27, 2014 - 2:36pm
"What does QB DVOA on non-failed completions look like?"
Here it is (min. 100 non-failed completions)
Rk QB DVOA Non-failed Comp.
1 9-N.Foles 224.7% 156
2 3-R.Wilson 197.1% 199
3 7-C.Kaepernick 196.3% 193
4 7-C.Keenum 190.2% 100
5 7-G.Smith 189.7% 187
6 10-K.Clemens 189.3% 104
7 16-M.Cassel 187.6% 108
8 1-C.Newton 184.0% 217
9 4-R.Fitzpatrick 183.4% 156
10 6-J.Cutler 182.9% 165
11 14-A.Dalton 182.8% 277
12 18-P.Manning 180.2% 366
13 12-A.Rodgers 179.3% 149
14 3-C.Palmer 178.5% 279
15 12-J.McCown 177.1% 121
16 9-D.Brees 175.8% 340
17 10-R.Griffin 174.4% 212
18 9-M.Stafford 173.2% 301
19 5-J.Flacco 173.2% 251
20 8-M.Glennon 172.5% 174
21 17-P.Rivers 172.2% 308
22 7-B.Roethlisberger 169.8% 287
23 17-J.Campbell 169.7% 126
24 12-T.Brady 166.4% 296
25 2-T.Pryor 166.2% 117
26 3-E.Manuel 166.1% 131
27 10-E.Manning 165.3% 245
28 9-T.Romo 163.6% 266
29 11-A.Smith 161.5% 228
30 8-M.Schaub 160.9% 156
31 7-C.Henne 160.6% 218
32 2-M.Ryan 160.6% 319
33 12-A.Luck 158.1% 264
34 17-R.Tannehill 155.1% 279
35 8-S.Bradford 152.5% 123
36 7-C.Ponder 150.2% 111
#31 by Perfundle // Mar 27, 2014 - 7:33pm
True enough, but the outliers are of interest. Rodgers and Wilson, for instance, have the same YPC, and same percentage of failed completions, but Wilson has a noticeably better DVOA on "successful" completions, which suggests that his failed completions are shorter and his successful completions are longer.
#45 by eggwasp // Mar 31, 2014 - 5:35am
This is the problem with the FO arbitrary decision to impose a binary system (success v failure) on a continuum of data (yards gained on a play). It makes for a cute number, but move the boundary between success & failure and the stat would change. I've never understood why they do this. Sure, it means something for 3rd down (and even then, this is altered by game situation, whether in FG range (& who would kick it) etc.
Is gaining 4 vs 5 yards on first down really so different in predicting prolonging the drive? Why is the boundary here, not at 3 yards, or 6? Is there a public explanation of this?
#46 by nat // Mar 31, 2014 - 9:47am
From the "Our New Stats Explained" link (which they really should rename)
DVOA breaks down every single play of the NFL season, assigning each play a value based on both total yards and yards towards a first down, based on work done by Pete Palmer, Bob Carroll, and John Thorn in their seminal book, The Hidden Game of Football. On first down, a play is considered a success if it gains 45 percent of needed yards; on second down, a play needs to gain 60 percent of needed yards; on third or fourth down, only gaining a new first down is considered success.
That's less an explanation as an appeal to authority. I believe the idea is to call a play a success if it increases the likelihood of getting a first down on that set of downs.
But it doesn't matter very much, because everything is compared to an average for most of FO's work.
Every single play run in the NFL gets a “success value” based on this system, and then that number gets compared to the average success values of plays in similar situations for all players.
It's only in articles like this one that the somewhat arbitrary cutoff between success and failure becomes an issue. Scott seems to be trying to see how much useful information you can get from the raw "complete but not a success" numbers for passing. Your concern is valid for this article, but not for FO in general.
#12 by wwhitman // Mar 27, 2014 - 2:59pm
Interesting that Joe Flacco leads in all failed completions, but he had the fifth lowest percentage of failed 3rd and 4th down passes. Seems to fit with my observations this year: he made a lot of conversions in 3rd and 13 situations...after two bad plays put them in 3rd and 13. The short passing game for the Ravens stunk with Rice hurt, Pierce hurt, Pitta hurt, Dickson bad, and Anquan Boldin on another team, but there were a lot of successful deep slants to Smith and Jones when the offense was in a hole.
#19 by silm // Mar 27, 2014 - 4:28pm
I had the same thought whit.
This analysis is great but it seems to penalize the QBs who had an unusually bad run game. the 2013 ravens were i think the 7th worst run game in NFL history. That's alot of 2nd and 12's, and 3rd and 11's.
For instance, let's say it is 2nd and 12 after another run gets stuffed in the backfield. Now you need 7 yards just to not count as a failed completion. But getting 7 yards suddenly becomes tough when everyone KNOWS you can't run (especially after a failed run play).
I would be curious to see their run game DVOA ranking next to the team's/QB's failed completion rate
#15 by JoeyHarringtonsPiano // Mar 27, 2014 - 4:01pm
His team might have made the playoffs if he actually checked down a little more in the last 7 games instead of taking so many unnecessary risks. (Admittedly, I'm being a bitter, armchair, Monday morning quarterback).
#23 by Q // Mar 27, 2014 - 6:11pm
I dislike the name "Failed Completion." We all know that each yard a team obtains increases the odds of it being the next team to score and reduces the odds of the other team scoring. A 5 yard gain on 3rd and 15 is obviously not ideal but has lowered the other team's expected points in comparison to if the pass was incomplete or only gained 1,2,3, or 4 yards.
A "failed" completion should only be a completion that does not reduce the other team's expected points:
1)A negative gain
2)A 0 yard gain
3)A gain on the last play of a half that does not result in points
#26 by Perfundle // Mar 27, 2014 - 6:29pm
"A 5 yard gain on 3rd and 15 is obviously not ideal but has lowered the other team's expected points in comparison to if the pass was incomplete"
Not always. If the completed pass takes 20 more seconds off the clock than the incompletion, then the incompletion might be preferable.
"A "failed" completion should only be a completion that does not reduce the other team's expected points."
By that standard, most of the failed completions discussed here match your criteria: a gain on 3rd-and-15 that does not pick up a first down usually decreases a team's chance of scoring relative to the 3rd-down chances, and it becomes more and more likely as the down distance goes down. To take an example, a team with a 3rd-and-15 at midfield should expect 0.54 points. A team with a 4th-and-1 at the opponent's 36 should only expect 0.32 points, whereas a team with a 1st-and-10 at the opponent's 35 should expect 2.96 points.
#32 by Q // Mar 27, 2014 - 7:46pm
1)You are right that in end of half situations the incompletion is preferred.
2)The comparison shouldn't be between gaining 5 on 3rd and 15 vs gaining the 1st down. The comparison should be between gaining the 5 vs an incompletion or gain of 1-4. A QB should naturally be trying to gain as many yards as yards as realistically possible on that play.
With field position being so important. Having your opponent start at their own 22 vs their own 27 does hold some value.
#35 by Vincent Verhei // Mar 27, 2014 - 11:03pm
Here is yet another way to look at the data. Matt Ryan had 120 failed completions, with 209 incomplete passes (including interceptions). So, 36.5% of his failed passes (not including sacks/DPI/etc.) were completed, the highest rate among starters:
PLAYER FC INC FC% 2-M.Ryan 120 209 36.5% 9-D.Brees 106 202 34.4% 7-C.Ponder 42 87 32.6% 6-J.Cutler 60 125 32.4% 4-R.Fitzpatrick 61 130 31.9% 12-A.Rodgers 44 96 31.4% 8-M.Schaub 63 138 31.3% 7-C.Henne 87 195 30.9% 16-M.Cassel 45 101 30.8% 5-J.Flacco 111 250 30.7% 7-B.Roethlisberger 89 203 30.5% 8-M.Glennon 73 167 30.4% 17-P.Rivers 71 165 30.1% 1-C.Newton 75 178 29.6% 9-N.Foles 47 112 29.6% 11-A.Smith 80 196 29.0% 18-P.Manning 84 206 29.0% 3-C.Palmer 84 207 28.9% 3-R.Wilson 59 146 28.8% 9-T.Romo 77 191 28.7% 17-J.Campbell 54 135 28.6% 3-E.Manuel 49 124 28.3% 14-A.Dalton 86 221 28.0% 12-J.McCown 29 75 27.9% 12-A.Luck 83 227 26.8% 10-R.Griffin 64 178 26.4% 8-S.Bradford 36 103 25.9% 12-T.Brady 84 243 25.7% 2-T.Pryor 39 113 25.7% 17-R.Tannehill 76 229 24.9% 10-E.Manning 72 229 23.9% 7-G.Smith 60 195 23.5% 7-C.Kaepernick 50 173 22.4% 9-M.Stafford 70 257 21.4%
I'm not sure if Ryan's position is a good or bad thing, so I'll leave the analysis out of it, except to say there are a lot of good quarterbacks on either end of the list.
#53 by nat // Apr 02, 2014 - 10:06am
I suspect (without evidence) that the sweet spot is near the middle of this list, if it is anywhere. If your failures are split between too short completions and incompletions of all lengths, perhaps you are more balanced and attacking all parts of the field.
#37 by dcaslin // Mar 28, 2014 - 12:27am
Is the receiver failed completion list intentionally removing RB's or are there just not as many of them? I'm really surprised that Ray Rice isn't on that list... Or perhaps three-headed-mediocrity-beast of Rice/Pierce/Dallas Clark ended up spreading the failure around? (Similar question for Ryan and ATL; surprised he doesn't have a receiver on the bad list either)
#44 by JamesPM2412 // Mar 29, 2014 - 4:01pm
I took it only as a supplement to a qbs overall completion percentage. Just to see whos comp% might be inflated by settling for "failed completions" vs a qb who consistanly trys the more difficult pass to move the sticks but his overall comp% suffers because of it. So you're right its not a very complete picture but I dont think it was intended to be.
#55 by panthersnbraves // Apr 02, 2014 - 11:04am
Great stuff. I have to wrap my head around it a little bit. The problem I have is trying to figure out a way to give partial credit.
On a busted screen, throwing an incompletion at the receivers feet is a better over-all play than throwing a catchable ball. On third-and 12, if all of your downfield receivers are covered, throwing a 7 yard pass, and hoping the receiver makes someone miss is a play that at least has a chance. Panicking and throwing a 2-yard swing with a defender right there, rather than at least looking downfield for another moment or two is probably bad.
Given these things, I'll take away that the Panthers D was pretty good, and that Cam was more or less middle of the pack. I'm just not sure that I would want to try and parse things really finely.
#56 by nat // Apr 02, 2014 - 3:31pm
FWIW, here are similar stats per attempt (not per completion) for defenses in 2013, sorted by the topic of the article, failed completions:
Team Att [Fail Pct Rk] [Suc. Pct Rk] [Inc. Pct Rk]
CAR 563 128 22.74% 1 249 44.23% 12 186 33.04% 30
OAK 549 98 17.85% 2 277 50.46% 28 174 31.69% 31
DET 572 97 16.96% 3 242 42.31% 4 233 40.73% 11
TB. 547 92 16.82% 4 260 47.53% 20 195 35.65% 23
SEA 524 88 16.79% 5 222 42.37% 5 214 40.84% 10
NO. 507 83 16.37% 6 221 43.59% 9 203 40.04% 15
JAC 551 89 16.15% 7 268 48.64% 23 194 35.21% 24
CIN 614 98 15.96% 8 265 43.16% 6 251 40.88% 9
SD. 548 87 15.88% 9 277 50.55% 29 184 33.58% 29
MIN 648 100 15.43% 10 321 49.54% 25 227 35.03% 25
SF. 585 90 15.38% 11 258 44.10% 11 237 40.51% 14
IND 536 82 15.30% 12 243 45.34% 15 211 39.37% 18
DAL 623 95 15.25% 13 310 49.76% 26 218 34.99% 26
TEN 545 83 15.23% 14 261 47.89% 21 201 36.88% 22
BAL 552 84 15.22% 15 230 41.67% 2 238 43.12% 3
STL 521 79 15.16% 16 277 53.17% 32 165 31.67% 32
KC. 592 89 15.03% 17 247 41.72% 3 256 43.24% 2
DEN 613 92 15.01% 18 265 43.23% 7 256 41.76% 6
NYG 601 90 14.98% 19 271 45.09% 13 240 39.93% 17
NYJ 586 87 14.85% 20 258 44.03% 10 241 41.13% 8
PIT 569 82 14.41% 21 247 43.41% 8 240 42.18% 5
CLE 605 86 14.21% 22 277 45.79% 18 242 40.00% 16
HOU 484 67 13.84% 23 220 45.45% 16 197 40.70% 12
ATL 516 71 13.76% 24 270 52.33% 30 175 33.91% 28
BUF 561 77 13.73% 25 233 41.53% 1 251 44.74% 1
MIA 580 79 13.62% 26 265 45.69% 17 236 40.69% 13
ARI 625 85 13.60% 27 282 45.12% 14 258 41.28% 7
GB. 539 72 13.36% 28 260 48.24% 22 207 38.40% 20
WAS 514 68 13.23% 29 270 52.53% 31 176 34.24% 27
CHI 507 66 13.02% 30 249 49.11% 24 192 37.87% 21
PHI 670 73 10.90% 31 335 50.00% 27 262 39.10% 19
NE. 589 64 10.87% 32 272 46.18% 19 253 42.95% 4
It's interesting to see some teams are great at forcing failed completions, but do so at the expense of more completions over all.
Meanwhile, look at New England. They're pretty good at preventing completions, but not good at all at preventing successful passes. That's either bad tackling, or a tendency to let receivers get open deep. Yup. That's them Patriots.
(Note: this would be better with total pass plays including sacks, not just attempts. Anyone with time to make that fix, feel free.)