Writers of Pro Football Prospectus 2008

27 Mar 2014

Failed Completions 2013

by Scott Kacsmar

We have probably all been guilty at least once of judging a quarterback's performance by his incompletions and not acknowledging that some of the completions may actually have been bad plays too.

A completion that loses yards, a 5-yard gain on third-and-15, or the worst: a 2-yard pass on fourth-and-10. None of these plays are going to benefit the offense, but they help the quarterback's completion percentage and the receiver's reception total. Advanced stats are not quite to the level where we can credit the quarterback who throws a long incompletion on third-and-17 more than we credit the dump pass on third-and-13 -- hey, at least that guy tried to move the chains -- but tracking failed completions is a start.

What are failed completions? They are any complete passes that fail to gain enough yardage to count as a successful play based on these guidelines: 45 percent of needed yards on first down, 60 percent on second down, or 100 percent on third/fourth down.

I became very interested in failed completions after first reading about them on Football Outsiders a few years back. I kept track of them for years (without knowing any fancy name) when doing research on third downs, but it was good to see methods that could be used for every down-and-distance situation. The only criticism I think one could throw at this stat is that occasionally a failed completion on third down sets up a significantly shorter field goal or puts a team into field-goal range on what otherwise would have been a punt. It also helps when you can turn an awful third-and-long into a fourth-and-short that's manageable to go for, but overall I think those situations do not represent the vast majority of failed completions.

In preparing our totals for 2013 failed completions, I used only regular-season data. The total number of completions is slightly higher than the official NFL total since Football Outsiders includes backward (lateral) passes as pass plays and completions.

First, here is the rate of failed completions for every team along with where that team finished in pass offense DVOA.

Offense - 2013 Failed Completions
Rk Team Completions Failed Pct. Failed Yards PASS DVOA DVOA RK
1 SD 378 70 18.5% 236 52.5% 2
2 DET 371 70 18.9% 268 9.9% 16
3 DEN 461 88 19.1% 323 60.3% 1
4 SF 244 50 20.5% 250 31.8% 4
5 MIA 357 77 21.6% 278 4.0% 20
6 PHI 310 67 21.6% 256 30.0% 5
7 NE 380 84 22.1% 327 28.2% 6
8 DAL 376 84 22.3% 288 15.9% 10
9 SEA 269 61 22.7% 266 27.4% 8
10 NYG 325 75 23.1% 336 -17.9% 29
11 GB 366 85 23.2% 338 12.2% 13
12 ARI 365 85 23.3% 339 10.8% 15
13 IND 354 83 23.4% 260 8.3% 17
14 PIT 378 89 23.5% 309 23.5% 9
15 CIN 364 86 23.6% 264 13.5% 12
16 CHI 375 89 23.7% 322 28.0% 7
Rk Team Completions Failed Pct. Failed Yards PASS DVOA DVOA RK
17 NO 447 107 23.9% 357 35.9% 3
18 NYJ 266 65 24.4% 243 -15.9% 28
19 STL 301 74 24.6% 291 0.1% 21
20 WAS 358 90 25.1% 379 -13.9% 26
21 OAK 298 76 25.5% 276 -20.3% 31
22 CAR 292 75 25.7% 288 11.5% 14
23 KC 333 90 27.0% 314 6.7% 18
24 ATL 445 122 27.4% 432 14.7% 11
25 TEN 328 90 27.4% 294 5.6% 19
26 BUF 300 83 27.7% 312 -14.0% 27
27 HOU 372 106 28.5% 404 -19.7% 30
28 MIN 326 93 28.5% 335 -8.3% 23
29 TB 292 84 28.8% 311 -1.7% 22
30 JAC 349 101 28.9% 398 -24.2% 32
31 CLE 380 110 28.9% 417 -9.2% 25
32 BAL 363 112 30.9% 349 -9.0% 24
AVG 347.6 85.0 24.5% 314.4 - -

On average, 24.5 percent of completions in the NFL are not successful plays, averaging 3.7 yards per reception. As you probably expected, many of the elite passing games had the lowest rates of failed completions while the worst passing games had some of the highest. The correlation coefficient between failed completion rate and pass DVOA was -0.70, because DVOA is built on the concept of successful plays. Now some plays are bigger failures than others, which is where the fractional points in the DVOA "success point" system come into play, but it was no surprise to see strong correlation.

(DVOA, for example, does give credit for partial success for plays such as 13-yard completions on third-and-15.)

The Falcons led the league with 432 yards off of failed completions. That includes a minus-8-yard completion by defensive back Shann Schillinger on a blocked punt against the Jets. The completion actually saved a little bit of field position compared to a normal block, but this was not a typical play by any means.

No team had a larger differential in where they ranked in these two stats than the Giants, who were 10th at failed completion rate, but only 29th in DVOA. Of course, Eli Manning led the league in a special kind of failed completion: 27 passes completed to a player wearing the other team's jersey.

Next is a look at the 30 quarterbacks with at least 150 completions and their rate of failed completions. I also included the quarterback's DYAR and DVOA on these failed completions.

Quarterbacks: 2013 Failed Completions (Min. 150 Completions)
Rk Quarterback Completions Failed Pct. DYAR Rk DVOA Rk
1 17-P.Rivers 378 70 18.5% -197 19 -59.6% 16
2 18-P.Manning 450 84 18.7% -222 23 -58.3% 11
3 9-M.Stafford 371 70 18.9% -201 21 -64.5% 19
4 12-J.McCown 150 29 19.3% -114 3 -85.9% 30
5 7-C.Kaepernick 243 50 20.6% -46 1 -31.5% 1
6 17-R.Tannehill 355 76 21.4% -195 17 -61.4% 18
7 12-T.Brady 380 84 22.1% -196 18 -55.0% 6
8 9-T.Romo 343 77 22.4% -188 16 -54.2% 5
9 10-E.Manning 317 72 22.7% -163 11 -65.6% 21
10 12-A.Rodgers 193 44 22.8% -54 2 -38.3% 2
11 9-N.Foles 203 47 23.2% -158 10 -71.9% 25
12 3-R.Wilson 259 60 23.2% -131 4 -57.4% 8
13 10-R.Griffin 276 64 23.2% -153 8 -58.6% 13
14 3-C.Palmer 364 85 23.4% -199 20 -55.2% 7
15 7-B.Roethlisberger 376 89 23.7% -285 28 -68.4% 24
Rk Quarterback Completions Failed Pct. DYAR Rk DVOA Rk
16 14-A.Dalton 363 86 23.7% -326 29 -76.4% 29
17 12-A.Luck 347 83 23.9% -254 24 -64.7% 20
18 9-D.Brees 447 107 23.9% -271 27 -57.6% 10
19 7-G.Smith 247 60 24.3% -183 15 -73.7% 26
20 1-C.Newton 292 75 25.7% -182 14 -58.4% 12
21 11-A.Smith 308 80 26.0% -264 25 -75.1% 27
22 6-J.Cutler 225 60 26.7% -136 6 -58.9% 14
23 3-E.Manuel 180 49 27.2% -134 5 -66.7% 22
24 2-M.Ryan 439 120 27.3% -267 26 -52.6% 3
25 4-R.Fitzpatrick 217 61 28.1% -221 22 -75.1% 28
26 7-C.Henne 305 87 28.5% -179 12 -53.7% 4
27 8-M.Schaub 219 63 28.8% -158 9 -59.8% 17
28 8-M.Glennon 248 74 29.8% -181 13 -59.1% 15
29 17-J.Campbell 180 54 30.0% -137 7 -57.5% 9
30 5-J.Flacco 362 111 30.7% -336 30 -67.4% 23

In dark corners of the internet, there rages an irrational Joe Flacco vs. Matt Ryan debate. In 2013, they led the league in failed completions with Flacco having the worst rate of anyone. Jason Campbell was the only other quarterback to hit 30 percent for failed completions, living up to his reputation of checking down when he shouldn't.

Most of the best quarterbacks in the league have a top 10 rate, but Drew Brees (18th) is one big exception. Given the Saints had 171 receptions by running backs, maybe that's not a surprise.

Josh McCown's completely unexpected season comes with another oddity. He was one of only four quarterbacks to be under 20 percent for failed completions, yet he had the worst DVOA (-85.9%) of anyone on those plays. Expecting to see McCown with a high rate of his failed completions coming on third/fourth down, I was surprised to see he still ranked 11th:

Percentage of Failed Completions on 3rd/4th Down
Rk Quarterback Failed 3D Rate Rk Quarterback Failed 3D Rate
1 9-T.Romo 77 17 22.1% 16 7-B.Roethlisberger 89 31 34.8%
2 18-P.Manning 84 19 22.6% 17 8-M.Schaub 63 22 34.9%
3 4-R.Fitzpatrick 61 16 26.2% 18 3-C.Palmer 85 31 36.5%
4 2-M.Ryan 120 32 26.7% 19 3-R.Wilson 60 22 36.7%
5 5-J.Flacco 111 32 28.8% 20 12-T.Brady 84 31 36.9%
6 12-A.Luck 83 24 28.9% 21 17-J.Campbell 54 20 37.0%
7 1-C.Newton 75 22 29.3% 22 10-R.Griffin 64 24 37.5%
8 12-A.Rodgers 44 13 29.5% 23 11-A.Smith 80 30 37.5%
9 9-N.Foles 47 14 29.8% 24 7-G.Smith 60 23 38.3%
10 9-D.Brees 107 32 29.9% 25 8-M.Glennon 74 29 39.2%
11 12-J.McCown 29 9 31.0% 26 9-M.Stafford 70 29 41.4%
12 14-A.Dalton 86 27 31.4% 27 7-C.Henne 87 38 43.7%
13 17-P.Rivers 70 23 32.9% 28 17-R.Tannehill 76 34 44.7%
14 6-J.Cutler 60 20 33.3% 29 10-E.Manning 72 34 47.2%
15 3-E.Manuel 49 17 34.7% 30 7-C.Kaepernick 50 25 50.0%

Someone has to be on the receiving end of these plays. Finally, I looked at the failed completions for everyone with at least 30 receptions. The following table excludes running backs since they dominate these lists with all the short passes they catch. For those curious, Ben Tate had a league-high 62.9 failed completion rate.

Most Failed Completions Lowest Failed Completion Rate (WR/TE) Highest Failed Completion Rate (WR/TE)
Receiver Team Failed Receiver Team Rec. Failed Pct. Receiver Team Rec. Failed Pct.
11-J.Edelman NE 28 81-C.Johnson DET 84 3 3.6% 18-A.Sanders JAC 51 23 45.1%
88-P.Garcon WAS 27 12-M.Colston NO 75 6 8.0% 84-C.Patterson MIN 45 17 37.8%
13-K.Wright TEN 25 16-L.Moore NO 37 3 8.1% 89-J.Carlson MIN 32 11 34.4%
84-A.Brown PIT 24 89-J.Cotchery PIT 46 4 8.7% 15-D.Bess CLE 42 14 33.3%
18-A.Sanders JAC 23 15-M.Floyd ARI 65 6 9.2% 80-E.Bennett CHI 32 10 31.3%
84-J.Cameron CLE 20 85-K.Thompkins NE 32 3 9.4% 11-T.Austin STL 40 12 30.0%
80-A.Johnson HOU 19 10-D.Hopkins HOU 52 5 9.6% 82-K.Rudolph MIN 30 9 30.0%
11-L.Fitzgerald ARI 18 88-D.Bryant DAL 93 9 9.7% 18-G.Little CLE 41 12 29.3%
13-T.Hilton IND 18 82-M.Jones CIN 51 5 9.8% 22-D.McCluster KC 53 15 28.3%
83-M.Bennett CHI 18 13-K.Allen SD 71 7 9.9% 85-T.Eifert CIN 39 11 28.2%

It is fitting that the consensus-best receiver in the game, Calvin Johnson, had the lowest failed completion rate. That helps explain some of the Detroit numbers from above. Elsewhere in the NFC North, the Vikings have three of the seven worst failed completion rates, which probably says more about the unholy trinity of quarterback play (Christian Ponder, Josh Freeman and Matt Cassel) than the receivers.

This is a good spot to highlight the greatest failed completion of the 21st century. In Week 9, the Steelers faced a third-and-30 against the Patriots. Ben Roethlisberger completed a screen pass to Le'Veon Bell and after a missed tackle, Bell gained 29 yards on the final play of the first quarter. Technically, that's a failed completion. The Steelers did go for it on fourth-and-1 at the New England 42 and Bell converted on the ground, which never would have happened without the big gain on the screen.

That play is the longest completion since 1999 that came on third or fourth down but failed to convert for a first down.

In closing, let's look at how defenses fared at forcing failed completions. An alarming stat from 2013 would be the Carolina Panthers allowing a 66.6 completion rate. We know their overall defense was great, but the secondary was suspect. However, no defense was better at forcing failed completions than the Panthers:

Defenses - 2013 Failed Completions
Rk Team Completions Failed Pct.
1 CAR 377 128 34.0%
2 DET 339 97 28.6%
3 SEA 310 88 28.4%
4 NO 304 83 27.3%
5 CIN 363 98 27.0%
6 BAL 314 84 26.8%
7 KC 336 89 26.5%
8 TB 352 92 26.1%
9 OAK 375 98 26.1%
10 SF 348 90 25.9%
11 DEN 357 92 25.8%
12 IND 325 82 25.2%
13 NYJ 345 87 25.2%
14 NYG 361 90 24.9%
15 JAC 357 89 24.9%
16 PIT 329 82 24.9%
Rk Team Completions Failed Pct.
17 BUF 310 77 24.8%
18 TEN 344 83 24.1%
19 SD 364 87 23.9%
20 MIN 421 100 23.8%
21 CLE 363 86 23.7%
22 DAL 405 95 23.5%
23 HOU 287 67 23.3%
24 ARI 367 85 23.2%
25 MIA 344 79 23.0%
26 STL 356 79 22.2%
27 GB 332 72 21.7%
28 CHI 315 66 21.0%
29 ATL 341 71 20.8%
30 WAS 338 68 20.1%
31 NE 336 64 19.0%
32 PHI 408 73 17.9%

Posted by: Scott Kacsmar on 27 Mar 2014

56 comments, Last at 02 Apr 2014, 3:31pm by nat

Comments

1
by JoeyHarringtonsPiano :: Thu, 03/27/2014 - 12:18pm

great stuff.

"An alarming stat from 2013 would be the Carolina Panthers allowing a 66.6 completion rate. We know their overall defense was great, but the secondary was suspect. However, no defense was better at forcing failed completions than the Panthers:"

This what you like to see from advanced stats: confirming what our eyes are telling us (In this case, that the Panthers front 7 made up for a weak secondary by forcing the QB to get rid of the ball early and settle for checkdowns).

54
by panthersnbraves :: Wed, 04/02/2014 - 10:48am

... and then closing quickly to stop YAC.

2
by stanbrown :: Thu, 03/27/2014 - 1:13pm

I don't think there is anything significant to be learned from the identity of the receiver on a failed completion. Unless he didn't run his route properly, it's not his fault that the QB had to throw it to him. Perhaps, he doesn't make people miss on WR screens or quick under routes, but I'm not sure this stat is going to be much help. Blown up screens are usually the fault of a missed block, not the receiver.

36
by dcaslin :: Fri, 03/28/2014 - 12:17am

Isn't a common problem where a receiver cuts his route 1 yard short for an easy completion that fails to convert on 3rd and 10? While the QB could arguably share some blame there, that's mostly on the receiver right? (Admittedly this is only one of many possible causes)

39
by stanbrown :: Fri, 03/28/2014 - 7:51am

For receivers who want to stay employed, I don't see that all that often. It's more common to see one foolishly make moves after the catch trying to get big yardage and give up the first down.

3
by nat :: Thu, 03/27/2014 - 1:19pm

It seems like you're looking at the wrong ratio here. We don't really need to look at the percentage of completions that were failures. We need the percentage of pass plays that are failed completions.

For example, your chart shows Rivers and P. Manning as being the best at avoiding failed completions. But of your top five, it's actually Stafford and Kaepernick who are least likely to have a pass end in a failed completion (then Rivers, McCown, and Manning). And who knows about the rest of the list?

You're trying to get to a statement like "Joe Schmo-Thrower has an usually high tendency to complete passes that nonetheless are failures". But the ratio that you use can't get you there, can it? Joe might be perfectly good at choosing the right target to throw to, fine at hitting underneath routes in stride, but inaccurate on longer passes.

This is kind of like judging baseball players on the percentage of their hits that go for extra bases. It would be much more useful to study the percentage of their plate appearances that result in extra base hits.

22
by Eddo :: Thu, 03/27/2014 - 5:19pm

I disagree. This is interesting to note as a supplement to completion percentage. Sure, QB X might have a surprisingly high completion percentage, but that's because he also has a high failed completion percentage.

What you're describing is just the inverse of success rate.

30
by nat :: Thu, 03/27/2014 - 7:19pm

You've missed my point. I suggest he use the "failed completion percentage". What he's using is something different. It's percent of all pass plays that are failed completions divided by the completion percentage, or more simply, failures per completion.

It's analogous to Yards per Attempt vs. Yards per Completion. In the bad old days, people focused on Y/C, and worshipped mad bomber QBs. Now we know better.

4
by nat :: Thu, 03/27/2014 - 2:02pm

Sorry all for the formatting of the following list...

2103 Percentage of Pass Plays Resulting in Failed Completions

QB Pct. Rank
9-M.Stafford 10.64% 1
7-C.Kaepernick 10.96% 2
17-R.Tannehill 11.82% 3
17-P.Rivers 12.17% 4
10-E.Manning 12.20% 5
12-J.McCown 12.29% 6
7-G.Smith 12.37% 7
18-P.Manning 12.37% 8
12-T.Brady 12.63% 9
10-R.Griffin 12.93% 10
3-R.Wilson 13.25% 11
9-T.Romo 13.46% 12
9-N.Foles 13.54% 13
12-A.Luck 13.70% 14
3-C.Palmer 13.84% 15
14-A.Dalton 13.98% 16
12-A.Rodgers 14.19% 17
7-B.Roethlisberger 14.33% 18
1-C.Newton 14.62% 19
3-E.Manuel 14.63% 20
11-A.Smith 14.68% 21
9-D.Brees 15.57% 22
7-C.Henne 16.08% 23
8-M.Glennon 16.12% 24
6-J.Cutler 16.17% 25
17-J.Campbell 16.27% 26
4-R.Fitzpatrick 16.53% 27
8-M.Schaub 16.58% 28
5-J.Flacco 16.79% 29
2-M.Ryan 17.27% 30

Scott's chart misleads us for Geno Smith, Peyton Manning, Aaron Rodgers, and Matt Ryan. The rest are with 4 in rank.

5
by Scott Kacsmar :: Thu, 03/27/2014 - 2:10pm

If you use every pass, aren't you just piling on more failure (incompletions) and getting away from the original focus? All passes would penalize QBs who are very good at completing passes. Peyton, Rodgers and Ryan were all top 5 in completion percentage. Can't have a FC if you don't complete the pass. A 2-yard pass on 2nd-and-8 still beats having an incompletion.

10
by nat :: Thu, 03/27/2014 - 2:45pm

A more complete picture would show all the various rates: Successful completion, failed completion, incomplete, interception, sack, and a few odd cases involving fumbles and busted plays. We could look at all the different ways QBs fail on pass plays. But the numbers would all be rates per pass play.

I do like the idea of looking at failed completions. But the relevant question is each QB's propensity for throwing them. Using a percentage relative to completions just clouds that picture.

For example, you honestly thought that Flacco was more likely to throw a failed completion than Ryan and used Flacco's picture to headline the article. While it's close, it turns out that Ryan is the one more likely to throw one. It's just that Flacco is worse at other things (completing passes in general). Your choice of ratio led you to emphasize the wrong thing about each QB, and ultimately to the wrong photo. Oops.

Ditto for Peyton Manning. His season was good yet not all that unusual for avoiding failed completions. But your choice of ratio led you to that false conclusion.

40
by tbwhite :: Fri, 03/28/2014 - 11:32am

You're just giving people credit for not completing passes. Not completing a pass doesn't make them a better QB in my book.

QB A, attempts 400 passes, completes 200, 50 of which are failures
QB B, attempts 400 passes, completes 300, 75 of which are failures

According to the article both A and B have failure rates of 25%

You are suggesting that the correct numbers are 12.5% for A, and 18.8% for B, so A is the better QB by your measure. But, I think everyone would agree that B is the better QB.

I think the correct number to use is actually (incompletions + failed completions)/ attempted passes

For QB A that gives a value of 62.5%, and for QB B 43.8%

41
by nat :: Fri, 03/28/2014 - 1:13pm

Well, clearly QB B is more likely to throw a failed completion on any given play. That's indisputable.

That doesn't mean he's worse, and I never said so. It means he is more likely to do this one thing, which was the topic of the article.

QB B is also much more likely to throw a successful completion, with 75 more for the same number of attempts.

The complete picture with this data would be that QB B is more likely than QB A to throw failed completions (+25 out of 400 attempts), much more likely to throw successful completions (+75), and much less likely to throw incomplete passes (-100).

Your preferred number is 1 - success rate. That's a perfectly fine number, although it doesn't make a statement about failed completions themselves, which was the point of the article.

Scott's approach is a bit of a mess, and a step back from DVOA or DYAR. Sometimes it's fun to ask "How much could we figure out by just focusing on failed completions?" I just think he took it a step too far by excluding the total attempts from the ratio.

47
by Scott Kacsmar :: Mon, 03/31/2014 - 12:56pm

It's not my approach. I did the article the way it's been done in the past on this site and I see no problem with that. There's a level of skill involved in completing a pass, so let's harness that skill. If I'm looking at which QBs complete the most passes that are not successful plays, then why would I care about the passes they don't complete? Again, you're raining down more failure to cover up the original area of focus. We have other big-picture stats for that. Failed completions were never meant as a be-all and end-all stat, and I do think it matters a lot more on 3rd/4th down.

48
by nat :: Mon, 03/31/2014 - 1:34pm

If I'm looking at which QBs complete the most passes that are not successful plays, then why would I care about the passes they don't complete?

Why would you look at the successful passes they complete either? If "most failed completions" was what you wanted, you wouldn't use a ratio at all.

If you wanted to know which QB throws unsuccessful completions at the highest rate, then you would use a ratio of "failed completions" to all pass plays, not just to completions.

It's a classic blunder. It's like confusing Yards/Completion for Yards/Attempt. It's like confusing an offensive line that has a bad "quick sacks as percentage of all sacks" ratio with one that gives up "quick sacks" at a high rate. (Hint: you need to look at quick sacks per pass play to get the right answer).

If you're going to look at rate stats, it is critical to know the implications of the denominator you choose, and to pick the right one.

49
by Scott Kacsmar :: Mon, 03/31/2014 - 1:52pm

"If you wanted to know which QB throws unsuccessful completions at the highest rate, then you would use a ratio of "failed completions" to all pass plays, not just to completions."

That's what you think should be done. I disagree. If I'm looking at failed completions, I'm still looking at passes that were completed, so that's the ratio I care about.

Situational stats - http://www.cbssports.com/nfl/players/player/splits/2013/420095

See the "1ST %" stat? That's basically what I'm calculating, though each down has its own success rate and it's not always 100%. See how Matt Ryan has a 100% 1ST % on 4th down (3/3)? His actual conversion rate is 3/9 (33.3%), but 1ST % is looking at which percentage of completions gained a first down.

That's all we're doing here -- looking at what percentage of completions failed to gain enough yards to be considered successful plays.

50
by nat :: Mon, 03/31/2014 - 4:39pm

That's all fine. But you can't then say "Flacco having the worst rate of anyone...". Because he definitely does NOT throw failed incomplete passes at the highest rate. (Although his rate is not good!)

This is not just a terminology problem, or a matter of taste. You are doing the equivalent of looking at Joe Schmo's yards per completion and then saying "Joe Schmo gains passing yards at the worst rate." Completions is a bad choice of denominator if you want to reach conclusions like that. You need to use something like attempts.

51
by LionInAZ :: Tue, 04/01/2014 - 8:49pm

Any idiot could work out the stat that you think is so much more important with just 10 minutes worth of arithmetic based on the the data presented here and make their own conclusions. The failed completion data shown here is value-added data, and the results presented here are interesting in their own right. I can't understand why you insist on bashing the authors for not doing your preferred analysis. You could have easily worked up your own numbers and presented them for discussion, but instead you chose to criticize the article.

For what it's worth, I agreed that looking at failed completions plus incompletions was interesting, but your attitude is so obnoxious that I can't defend it as a positive contribution.

52
by nat :: Wed, 04/02/2014 - 9:56am

You could have easily worked up your own numbers and presented them for discussion, but instead you chose to criticize the article.

See comment #4 at the top of this very thread. Did you really want more of my "own numbers" than that?

Notice the follow on discussion. In the back and forth, we cover the problem with using failed completion rate as meaning simply "better QB", and the problem with using a "per completion" ratio was meaning "rate that it happens". There are examples, worked out math, appeals to authority (not always the best thing, but Scott is allowed if he wants to), and clarifying analogies.

Of course we can criticize what looks to be a flawed stats article. This is FO.

So please don't be so quick to post rude comments. Join the discussion instead, if it's a discussion that you want. Actually look at the numbers that are posted or post your own, if it's numbers that you need.

6
by mehllageman56 :: Thu, 03/27/2014 - 2:19pm

Pretty interesting grist for the mill. My takeaways: Tannehill ranks really high up on both lists, which fits what I see from him. Give him an offensive line, and he'd be really efficient. That EJ Manuel and Geno Smith aren't all the way on the bottom of the lists gives some hope for their futures. I'm also suprised that there were no Jets receivers on the failed completions list, but they probably didn't catch enough passes to qualify.

7
by Karl Cuba :: Thu, 03/27/2014 - 2:22pm

So Kaepernick is really good at avoiding failed completions, which makes sense for a guy who is a much better deep passer than he is at the short stuff.

But he stinks at it on third down. I wonder why, other than a sample size anomaly. Weird.

16
by Perfundle :: Thu, 03/27/2014 - 4:06pm

For a team that was pretty good at offense, the Niners had a surprisingly high percentage of third-down plays. They were in 8th place, behind the Ravens, Jets, Buccaneers, Raiders, Jaguars, Titans and Browns, pretty much all awful offenses. Because of that, Kaepernick had a very high percentage of his throws that occurred on third or fourth down, at 30.3%; for comparison, Romo was at 24.1% and P. Manning at 24.6%. Kaepernick still throws a high percentage of failed completions on third down, but at least he's now tied for 19th place at 31.3%, instead of dead-last. This is the full list:

P.Manning 19.0%
P.Rivers 22.5%
A.Rodgers 24.1%
R.Fitzpatrick 24.6%
J.McCown 25.0%
T.Romo 25.8%
M.Stafford 27.1%
N.Foles 27.5%
A.Dalton 27.6%
R.Wilson 27.8%
R.Griffin 28.6%
G.Smith 28.8%
D.Brees 29.1%
M.Ryan 29.4%
C.Newton 30.1%
J.Flacco 30.2%
T.Brady 31.0%
J.Cutler 31.3%
C.Kaepernick 31.3%
A.Luck 32.0%
B.Roethlisberger 32.3%
C.Palmer 32.6%
R.Tannehill 34.3%
E.Manuel 35.4%
E.Manning 36.2%
A.Smith 37.5%
M.Schaub 38.6%
M.Glennon 39.2%
J.Campbell 40.8%
C.Henne 42.2%

17
by Karl Cuba :: Thu, 03/27/2014 - 4:08pm

Thanks for that.

18
by Deelron :: Thu, 03/27/2014 - 4:19pm

I don't find it all that surprising that they had so many 3rd down plays, it jives pretty well with having running a ton of the time relatively ineffectively (14th DVOA, terrible Run Blocking stats) with an efficient (4th DVOA) passing attack.

24
by Karl Cuba :: Thu, 03/27/2014 - 6:17pm

I'd love to know how these rates correlate to average distance required to convert on 3rd and 4th.

28
by Karl Cuba :: Thu, 03/27/2014 - 6:43pm

OK, apparently the niners had the 6th longest yards to go on 3rd down. Isn't context nice?

Though it should be pointed out that teams know the niners love to run and so are probably stacking the box and making the pass game easier.

8
by Perfundle :: Thu, 03/27/2014 - 2:25pm

The QB DYAR on failed completions isn't very interesting, since it's basically an inverse ordering of the number of passes each QB threw. What does QB DVOA on non-failed completions look like? I would like to see if there are any surprise names on the bottom of that list.

9
by Scott Kacsmar :: Thu, 03/27/2014 - 2:36pm

"What does QB DVOA on non-failed completions look like?"

Here it is (min. 100 non-failed completions)

Rk QB DVOA Non-failed Comp.
1 9-N.Foles 224.7% 156
2 3-R.Wilson 197.1% 199
3 7-C.Kaepernick 196.3% 193
4 7-C.Keenum 190.2% 100
5 7-G.Smith 189.7% 187
6 10-K.Clemens 189.3% 104
7 16-M.Cassel 187.6% 108
8 1-C.Newton 184.0% 217
9 4-R.Fitzpatrick 183.4% 156
10 6-J.Cutler 182.9% 165
11 14-A.Dalton 182.8% 277
12 18-P.Manning 180.2% 366
13 12-A.Rodgers 179.3% 149
14 3-C.Palmer 178.5% 279
15 12-J.McCown 177.1% 121
16 9-D.Brees 175.8% 340
17 10-R.Griffin 174.4% 212
18 9-M.Stafford 173.2% 301
19 5-J.Flacco 173.2% 251
20 8-M.Glennon 172.5% 174
21 17-P.Rivers 172.2% 308
22 7-B.Roethlisberger 169.8% 287
23 17-J.Campbell 169.7% 126
24 12-T.Brady 166.4% 296
25 2-T.Pryor 166.2% 117
26 3-E.Manuel 166.1% 131
27 10-E.Manning 165.3% 245
28 9-T.Romo 163.6% 266
29 11-A.Smith 161.5% 228
30 8-M.Schaub 160.9% 156
31 7-C.Henne 160.6% 218
32 2-M.Ryan 160.6% 319
33 12-A.Luck 158.1% 264
34 17-R.Tannehill 155.1% 279
35 8-S.Bradford 152.5% 123
36 7-C.Ponder 150.2% 111

14
by Perfundle :: Thu, 03/27/2014 - 3:38pm

Thanks! Absolutely no mystery about the top three, while Romo and Ryan are a bit surprising at the bottom. But I guess that's partly why their DVOA rank is slightly below their DYAR one.

25
by Vince Verhei :: Thu, 03/27/2014 - 6:26pm

This is basically yards/completions. Correlation is .830. If we remove Nick Foles (who is way ahead of the pack in both categories), the correlation falls to .766, but that's still awfully strong.

31
by Perfundle :: Thu, 03/27/2014 - 7:33pm

True enough, but the outliers are of interest. Rodgers and Wilson, for instance, have the same YPC, and same percentage of failed completions, but Wilson has a noticeably better DVOA on "successful" completions, which suggests that his failed completions are shorter and his successful completions are longer.

27
by jmaron :: Thu, 03/27/2014 - 6:30pm

any list that has C. Ponder at the bottom must have some merit

11
by JonFrum :: Thu, 03/27/2014 - 2:55pm

So if a receiver drops a pass that would have been a 4 yard completion on first down, he looks better in this stat than if he caught it - and produced a 'failed completion' (less than 4.5 yards)?

21
by Eddo :: Thu, 03/27/2014 - 5:17pm

Only if you insist on interpreting it as being "better" or "worse". Sometimes, a stat is just a stat; there doesn't have to be a value judgement involved.

42
by scraps :: Fri, 03/28/2014 - 10:12pm

"Failed" in "Failed Completions" carries with it a value judgment.

43
by Jerry :: Fri, 03/28/2014 - 11:16pm

It's just treating "failure" as the opposite of "success".

45
by eggwasp :: Mon, 03/31/2014 - 5:35am

This is the problem with the FO arbitrary decision to impose a binary system (success v failure) on a continuum of data (yards gained on a play). It makes for a cute number, but move the boundary between success & failure and the stat would change. I've never understood why they do this. Sure, it means something for 3rd down (and even then, this is altered by game situation, whether in FG range (& who would kick it) etc.
Is gaining 4 vs 5 yards on first down really so different in predicting prolonging the drive? Why is the boundary here, not at 3 yards, or 6? Is there a public explanation of this?

46
by nat :: Mon, 03/31/2014 - 9:47am

From the "Our New Stats Explained" link (which they really should rename)

DVOA breaks down every single play of the NFL season, assigning each play a value based on both total yards and yards towards a first down, based on work done by Pete Palmer, Bob Carroll, and John Thorn in their seminal book, The Hidden Game of Football. On first down, a play is considered a success if it gains 45 percent of needed yards; on second down, a play needs to gain 60 percent of needed yards; on third or fourth down, only gaining a new first down is considered success.

That's less an explanation as an appeal to authority. I believe the idea is to call a play a success if it increases the likelihood of getting a first down on that set of downs.

But it doesn't matter very much, because everything is compared to an average for most of FO's work.

Every single play run in the NFL gets a “success value” based on this system, and then that number gets compared to the average success values of plays in similar situations for all players.

It's only in articles like this one that the somewhat arbitrary cutoff between success and failure becomes an issue. Scott seems to be trying to see how much useful information you can get from the raw "complete but not a success" numbers for passing. Your concern is valid for this article, but not for FO in general.

12
by wwhitman :: Thu, 03/27/2014 - 2:59pm

Interesting that Joe Flacco leads in all failed completions, but he had the fifth lowest percentage of failed 3rd and 4th down passes. Seems to fit with my observations this year: he made a lot of conversions in 3rd and 13 situations...after two bad plays put them in 3rd and 13. The short passing game for the Ravens stunk with Rice hurt, Pierce hurt, Pitta hurt, Dickson bad, and Anquan Boldin on another team, but there were a lot of successful deep slants to Smith and Jones when the offense was in a hole.

19
by iron_greg :: Thu, 03/27/2014 - 4:28pm

I had the same thought whit.

This analysis is great but it seems to penalize the QBs who had an unusually bad run game. the 2013 ravens were i think the 7th worst run game in NFL history. That's alot of 2nd and 12's, and 3rd and 11's.

For instance, let's say it is 2nd and 12 after another run gets stuffed in the backfield. Now you need 7 yards just to not count as a failed completion. But getting 7 yards suddenly becomes tough when everyone KNOWS you can't run (especially after a failed run play).

I would be curious to see their run game DVOA ranking next to the team's/QB's failed completion rate

20
by Scott Kacsmar :: Thu, 03/27/2014 - 5:06pm

I only have 2013 right now, but the correlation between 2013 FC rate and 2013 run DVOA was -0.36.

29
by iron_greg :: Thu, 03/27/2014 - 6:57pm

Thanks, Hmm well I guess its not as strong as it would seem to be conceptually.

13
by tuluse :: Thu, 03/27/2014 - 3:25pm

No one can accuse Matt Stafford of being captain checkdown.

15
by JoeyHarringtonsPiano :: Thu, 03/27/2014 - 4:01pm

His team might have made the playoffs if he actually checked down a little more in the last 7 games instead of taking so many unnecessary risks. (Admittedly, I'm being a bitter, armchair, Monday morning quarterback).

23
by Q :: Thu, 03/27/2014 - 6:11pm

I dislike the name "Failed Completion." We all know that each yard a team obtains increases the odds of it being the next team to score and reduces the odds of the other team scoring. A 5 yard gain on 3rd and 15 is obviously not ideal but has lowered the other team's expected points in comparison to if the pass was incomplete or only gained 1,2,3, or 4 yards.

A "failed" completion should only be a completion that does not reduce the other team's expected points:

1)A negative gain
2)A 0 yard gain
3)A gain on the last play of a half that does not result in points

26
by Perfundle :: Thu, 03/27/2014 - 6:29pm

"A 5 yard gain on 3rd and 15 is obviously not ideal but has lowered the other team's expected points in comparison to if the pass was incomplete"

Not always. If the completed pass takes 20 more seconds off the clock than the incompletion, then the incompletion might be preferable.

"A "failed" completion should only be a completion that does not reduce the other team's expected points."

By that standard, most of the failed completions discussed here match your criteria: a gain on 3rd-and-15 that does not pick up a first down usually decreases a team's chance of scoring relative to the 3rd-down chances, and it becomes more and more likely as the down distance goes down. To take an example, a team with a 3rd-and-15 at midfield should expect 0.54 points. A team with a 4th-and-1 at the opponent's 36 should only expect 0.32 points, whereas a team with a 1st-and-10 at the opponent's 35 should expect 2.96 points.

32
by Q :: Thu, 03/27/2014 - 7:46pm

1)You are right that in end of half situations the incompletion is preferred.

2)The comparison shouldn't be between gaining 5 on 3rd and 15 vs gaining the 1st down. The comparison should be between gaining the 5 vs an incompletion or gain of 1-4. A QB should naturally be trying to gain as many yards as yards as realistically possible on that play.

With field position being so important. Having your opponent start at their own 22 vs their own 27 does hold some value.

33
by Thok :: Thu, 03/27/2014 - 9:22pm

Do you have some idea what the year to year correlation of failed completion rate for defenses is? Because I look at that outlier for Carolina and see a lot of room for regression to the mean.

34
by Vince Verhei :: Thu, 03/27/2014 - 10:55pm

It is amazing how far they are ahead of everyone else. No. 2 DET is closer to No. 24 ARI than they are to Carolina.

35
by Vince Verhei :: Thu, 03/27/2014 - 11:03pm

Here is yet another way to look at the data. Matt Ryan had 120 failed completions, with 209 incomplete passes (including interceptions). So, 36.5% of his failed passes (not including sacks/DPI/etc.) were completed, the highest rate among starters:

PLAYER	FC	INC	FC%
2-M.Ryan	120	209	36.5%
9-D.Brees	106	202	34.4%
7-C.Ponder	42	87	32.6%
6-J.Cutler	60	125	32.4%
4-R.Fitzpatrick	61	130	31.9%
12-A.Rodgers	44	96	31.4%
8-M.Schaub	63	138	31.3%
7-C.Henne	87	195	30.9%
16-M.Cassel	45	101	30.8%
5-J.Flacco	111	250	30.7%
7-B.Roethlisberger	89	203	30.5%
8-M.Glennon	73	167	30.4%
17-P.Rivers	71	165	30.1%
1-C.Newton	75	178	29.6%
9-N.Foles	47	112	29.6%
11-A.Smith	80	196	29.0%
18-P.Manning	84	206	29.0%
3-C.Palmer	84	207	28.9%
3-R.Wilson	59	146	28.8%
9-T.Romo	77	191	28.7%
17-J.Campbell	54	135	28.6%
3-E.Manuel	49	124	28.3%
14-A.Dalton	86	221	28.0%
12-J.McCown	29	75	27.9%
12-A.Luck	83	227	26.8%
10-R.Griffin	64	178	26.4%
8-S.Bradford	36	103	25.9%
12-T.Brady	84	243	25.7%
2-T.Pryor	39	113	25.7%
17-R.Tannehill	76	229	24.9%
10-E.Manning	72	229	23.9%
7-G.Smith	60	195	23.5%
7-C.Kaepernick	50	173	22.4%
9-M.Stafford	70	257	21.4%

I'm not sure if Ryan's position is a good or bad thing, so I'll leave the analysis out of it, except to say there are a lot of good quarterbacks on either end of the list.

53
by nat :: Wed, 04/02/2014 - 10:06am

I suspect (without evidence) that the sweet spot is near the middle of this list, if it is anywhere. If your failures are split between too short completions and incompletions of all lengths, perhaps you are more balanced and attacking all parts of the field.

37
by dcaslin :: Fri, 03/28/2014 - 12:27am

Is the receiver failed completion list intentionally removing RB's or are there just not as many of them? I'm really surprised that Ray Rice isn't on that list... Or perhaps three-headed-mediocrity-beast of Rice/Pierce/Dallas Clark ended up spreading the failure around? (Similar question for Ryan and ATL; surprised he doesn't have a receiver on the bad list either)

38
by Scott Kacsmar :: Fri, 03/28/2014 - 1:12am

It's only for WR/TE.

44
by JamesPM2412 :: Sat, 03/29/2014 - 4:01pm

I took it only as a supplement to a qbs overall completion percentage. Just to see whos comp% might be inflated by settling for "failed completions" vs a qb who consistanly trys the more difficult pass to move the sticks but his overall comp% suffers because of it. So you're right its not a very complete picture but I dont think it was intended to be.

55
by panthersnbraves :: Wed, 04/02/2014 - 11:04am

Great stuff. I have to wrap my head around it a little bit. The problem I have is trying to figure out a way to give partial credit.

On a busted screen, throwing an incompletion at the receivers feet is a better over-all play than throwing a catchable ball. On third-and 12, if all of your downfield receivers are covered, throwing a 7 yard pass, and hoping the receiver makes someone miss is a play that at least has a chance. Panicking and throwing a 2-yard swing with a defender right there, rather than at least looking downfield for another moment or two is probably bad.

Given these things, I'll take away that the Panthers D was pretty good, and that Cam was more or less middle of the pack. I'm just not sure that I would want to try and parse things really finely.

56
by nat :: Wed, 04/02/2014 - 3:31pm

FWIW, here are similar stats per attempt (not per completion) for defenses in 2013, sorted by the topic of the article, failed completions:

Team Att [Fail Pct Rk] [Suc. Pct Rk] [Inc. Pct Rk]
CAR 563 128 22.74% 1 249 44.23% 12 186 33.04% 30
OAK 549 98 17.85% 2 277 50.46% 28 174 31.69% 31
DET 572 97 16.96% 3 242 42.31% 4 233 40.73% 11
TB. 547 92 16.82% 4 260 47.53% 20 195 35.65% 23
SEA 524 88 16.79% 5 222 42.37% 5 214 40.84% 10
NO. 507 83 16.37% 6 221 43.59% 9 203 40.04% 15
JAC 551 89 16.15% 7 268 48.64% 23 194 35.21% 24
CIN 614 98 15.96% 8 265 43.16% 6 251 40.88% 9
SD. 548 87 15.88% 9 277 50.55% 29 184 33.58% 29
MIN 648 100 15.43% 10 321 49.54% 25 227 35.03% 25
SF. 585 90 15.38% 11 258 44.10% 11 237 40.51% 14
IND 536 82 15.30% 12 243 45.34% 15 211 39.37% 18
DAL 623 95 15.25% 13 310 49.76% 26 218 34.99% 26
TEN 545 83 15.23% 14 261 47.89% 21 201 36.88% 22
BAL 552 84 15.22% 15 230 41.67% 2 238 43.12% 3
STL 521 79 15.16% 16 277 53.17% 32 165 31.67% 32
KC. 592 89 15.03% 17 247 41.72% 3 256 43.24% 2
DEN 613 92 15.01% 18 265 43.23% 7 256 41.76% 6
NYG 601 90 14.98% 19 271 45.09% 13 240 39.93% 17
NYJ 586 87 14.85% 20 258 44.03% 10 241 41.13% 8
PIT 569 82 14.41% 21 247 43.41% 8 240 42.18% 5
CLE 605 86 14.21% 22 277 45.79% 18 242 40.00% 16
HOU 484 67 13.84% 23 220 45.45% 16 197 40.70% 12
ATL 516 71 13.76% 24 270 52.33% 30 175 33.91% 28
BUF 561 77 13.73% 25 233 41.53% 1 251 44.74% 1
MIA 580 79 13.62% 26 265 45.69% 17 236 40.69% 13
ARI 625 85 13.60% 27 282 45.12% 14 258 41.28% 7
GB. 539 72 13.36% 28 260 48.24% 22 207 38.40% 20
WAS 514 68 13.23% 29 270 52.53% 31 176 34.24% 27
CHI 507 66 13.02% 30 249 49.11% 24 192 37.87% 21
PHI 670 73 10.90% 31 335 50.00% 27 262 39.10% 19
NE. 589 64 10.87% 32 272 46.18% 19 253 42.95% 4

It's interesting to see some teams are great at forcing failed completions, but do so at the expense of more completions over all.

Meanwhile, look at New England. They're pretty good at preventing completions, but not good at all at preventing successful passes. That's either bad tackling, or a tendency to let receivers get open deep. Yup. That's them Patriots.

(Note: this would be better with total pass plays including sacks, not just attempts. Anyone with time to make that fix, feel free.)