Writers of Pro Football Prospectus 2008

Most Recent FO Features


» Seventh Day Adventure: Week 13

The biggest game this week is the Iron Bowl, where the playoff hopes of Alabama, Auburn, and Georgia hang in the balance.

23 Nov 2015

ALEX: Week 11 and Screen Passes

by Scott Kacsmar

Typically, the lowest ALEX play each week has been a screen pass on third-and-long, which I like to call a "give-up play" as the offense has such overwhelming odds of converting given the reliance on YAC. While some teams like the draw play, the trend anymore is to just run a screen, gain a few yards and punt the ball away. We have also seen this year that once in a while that screen can work, just as the draw can work too if you ask the Miami Dolphins after Darren McFadden converted on third-and-14 this week.

But what kind of numbers are we talking about for screen success rate on third down? I looked at our charting files for 2013 and 2014 to get the data. I also added plays marked as "dumpoff," which are basically just checkdowns.

3rd-Down Play Type 2013 2014
Plays Conv% ALEX Plays Conv% ALEX
Dumpoff 144 18.1% -9.4 179 25.7% -9.4
Screen 324 20.1% -14.6 297 24.2% -14.2
WR Screen 140 20.0% -12.7 154 26.6% -12.5
RB Screen 168 21.4% -16.1 124 23.4% -16.1
TE Screen 16 6.3% -14.7 19 10.5% -14.8
Screen (1-3 yds to go) 28 60.7% -5.0 23 69.6% -5.7
Screen (4-7 yds to go) 55 29.1% -8.7 51 43.1% -8.0
Screen (8+ yds to go) 241 13.3% -17.0 223 15.2% -16.4

The first thing that stands out is that 74.7 percent of third-down screens are in long-yardage situations (8-plus yards to go). The average ALEX on those plays is nearly 17 yards short of the sticks. Earlier this season we looked at the average ALEX for all throws on each down, and the average third-down ALEX was about plus-1.3. The overall screen success rate was in the 20 to 25 percent range. Over the last two seasons, even the worst third-down passing offense (Jacksonville) was still at a 28.7 percent conversion rate (including sacks).

Third-down passing in medium situations (4 to 7 yards to go) converts about 44 percent of the time, so the screen is still a below-average call there. But where we really see the difference is on the long plays. Screens, which make up around a tenth of third-and-long pass calls, work about 14 percent of the time, compared to an overall success rate of over 25 percent. Plays negated by penalties are excluded here, but you have to think longer passes are more likely to generate positive penalties (illegal contact, holding, pass interference) for an offense compared to more negative penalties on screens for holding or offensive pass interference.

It's not that offenses should never throw screens in these situations, but the screen has certainly earned a reputation for being a safe, conservative call. When an offense does that in a situation that calls for aggressive play, then it's only fair to criticize the call.

Week 11's Most Conservative Plays

The Lowest ALEX

Teams: Minnesota vs. Green Bay
Situation: third-and-27 at own 14, tied 6-6 in second quarter with 13:18 left
Play: Teddy Bridgewater to Matt Asiata for 9 yards
Air yards: minus-8
ALEX: minus-35

This was set up after Bridgewater lost 18 yards on a terrible sack where he turned his back to the defense. However, this is exactly the kind of screen I am talking about, where it looks like a give-up play. The offense could run this without even communicating it, because the call is so automatic. Get a few yards and bring out the punt team. Maybe you do that a few times, but there are going to be situations where you have to push a little more. In a game with so many ticky-tack calls that produce automatic first downs, why not take some more shots on passes that will actually do something more than pad the completion percentage?

A Notable Failure of the Week

Teams: Miami vs. Dallas
Situation: third-and-17 at own 36, trailing 24-14 in fourth quarter with 7:15 left
Play: Ryan Tannehill to Lamar Miller for 11 yards
Air yards: 7
ALEX: minus-10

Tannehill is no stranger to this section, but what a lame way to get Miller his ninth and final touch of the game. It looked like Miami ran four deep routes just to clear the middle of the field for a dumpoff to Miller, who never had a shot at the first down. If you are going to run this play to set up fourth-and-manageable at midfield, then that's understandable, but Miami just punted on fourth down. Dallas then hogged the ball for six-plus minutes and Miami only had 64 seconds left when the offense got back on the field, still down two scores.

Week 11's Most Aggressive Plays

The Highest ALEX

Teams: Atlanta vs. Indianapolis
Situation: fourth-and-3 at own 39, trailing 24-21 in fourth quarter with 0:01 left
Play: Matt Ryan's Hail Mary intercepted by Coby Fleener
Air yards: 51
ALEX: plus-48

This is actually not aggressive at all since the only option for Atlanta was the Hail Mary on the game's final play. Ryan probably lost some throwing power by coming to a complete stop before releasing the ball, and his pass was obviously short of the end zone. This play also shows some of the flaws, or perhaps differences in methodology, in how air yards are calculated. I thought the pass was clearly intercepted around the 7-yard line (54 air yards), but the play-by-play says it was at the 10-yard line with a 2-yard return. This is something we could change in our own charting file. However, what a crazy way to end a game with a short Hail Mary getting intercepted by Coby Fleener.

A Notable Success of the Week

Teams: Green Bay at Minnesota
Situation: third-and-15 at own 15, leading 9-6 in second quarter with 1:53 left
Play: Aaron Rodgers to Jeff Janis (50-yard pass interference penalty)
Air yards: 50
ALEX: plus-35

I said last week I normally overlook defensive pass interference plays, but this was a significant one. It came in a crucial time with the Vikings looking to get the ball back in good field position before the half. In a game that featured many third-and-longs and frustrating ALEX decisions, it was refreshing to see Rodgers take a chance in single coverage with a deep ball to Jeff Janis. Terence Newman was all over the receiver for an easy call on the 50-yard flag for pass interference. Green Bay ended the drive with a touchdown to take a 16-6 lead, and it never had a lead smaller than six points the rest of the way.

2015 ALEX Rankings Thru Week 11

The following table shows where each qualified quarterback (minimum 26 passes) ranks in ALEX on third down only. There are also rankings for DVOA, average need yards (ranked from highest to lowest) and conversion rate.

Rk Quarterback Team ALEX CONV% Rk DVOA Rk Passes Avg. Need Rk
1 Ben Roethlisberger PIT 6.8 47.2% 5 55.1% 8 53 6.9 31
2 Carson Palmer ARI 4.2 52.4% 3 90.0% 3 82 7.1 26
3 Matt Hasselbeck IND 4.1 44.8% 12 1.6% 29 29 6.2 36
4 Ryan Fitzpatrick NYJ 3.8 40.7% 22 32.4% 19 91 7.4 22
5 Aaron Rodgers GB 3.3 38.6% 24 48.8% 10 83 7.7 19
6 Jay Cutler CHI 2.8 47.1% 6 67.8% 6 85 6.6 34
7 Cam Newton CAR 2.6 37.3% 26 31.7% 20 83 7.9 15
8 Andrew Luck IND 2.6 42.0% 17 55.8% 7 69 7.6 21
9 Johnny Manziel CLE 2.5 41.0% 21 95.4% 2 39 8.7 3
10 Blake Bortles JAC 2.4 38.8% 23 37.2% 18 98 8.0 12
11 Andy Dalton CIN 2.4 46.9% 7 75.5% 5 81 8.1 10
12 Russell Wilson SEA 2.1 41.6% 19 38.8% 17 77 7.0 28
13 Brian Hoyer HOU 2.0 42.4% 16 46.9% 11 66 7.8 17
14 Tyrod Taylor BUF 2.0 43.1% 14 50.6% 9 51 8.3 7
15 Joe Flacco BAL 1.8 34.0% 31 -10.2% 31 103 7.0 29
16 Peyton Manning DEN 1.6 32.7% 33 -37.5% 35 98 8.2 8
17 Josh McCown CLE 1.4 54.3% 1 126.6% 1 70 7.2 24
18 Matt Ryan ATL 1.4 46.8% 8 30.9% 21 109 6.6 33
Rk Quarterback Team ALEX CONV% Rk DVOA Rk Passes Avg. Need Rk
19 Kirk Cousins WAS 1.0 46.3% 9 43.5% 14 108 7.1 27
20 Tom Brady NE 0.9 53.8% 2 78.2% 4 78 6.9 32
21 Drew Brees NO 0.9 46.2% 10 43.3% 15 104 7.9 16
22 Matt Cassel DAL 0.8 37.8% 25 42.3% 16 37 6.9 30
23 Jameis Winston TB 0.8 42.0% 18 7.9% 27 100 8.2 9
24 Colin Kaepernick SF 0.7 33.8% 32 10.2% 26 74 7.7 18
25 Nick Foles STL 0.4 24.0% 36 -45.3% 36 96 8.6 4
26 Derek Carr OAK 0.4 43.6% 13 44.8% 12 101 7.6 20
27 Philip Rivers SD 0.4 42.6% 15 19.8% 24 101 8.3 6
28 Marcus Mariota TEN 0.4 41.4% 20 30.6% 22 70 8.0 11
29 Eli Manning NYG 0.3 45.1% 11 43.6% 13 102 7.2 25
30 Ryan Mallett HOU 0.2 35.7% 28 -36.8% 34 42 7.2 23
31 Tony Romo DAL 0.1 48.1% 4 -7.8% 30 27 6.5 35
32 Matthew Stafford DET 0.1 36.7% 27 -34.8% 33 98 7.9 14
33 Ryan Tannehill MIA -0.1 28.6% 34 2.8% 28 91 8.8 1
34 Teddy Bridgewater MIN -0.1 35.3% 29 23.0% 23 85 7.9 13
35 Sam Bradford PHI -1.0 25.0% 35 -19.5% 32 88 8.5 5
36 Alex Smith KC -3.8 34.1% 30 18.9% 25 85 8.8 2

I moved the minimum requirement down just enough this week to include current starters Tony Romo and Matt Hasselbeck. Oddly enough, they have the shortest average need yards on third down this season, but obviously neither has hit 30 plays yet.

Next, ALEX is presented in splits by distance: short (1 to 3 yards), medium (4 to 7 yards) and long (8-plus yards). The colors indicate where a player is well above average (darker green) versus below average (darker red). Those conversion rates are also shown with a ranking.

Carson Palmer is having a pretty incredible season on third-and-long, and it helps when you are so used to going vertical in Bruce Arians' offense. Tom Brady is two standard deviations above everyone in converting on third-and-medium, so that is something to keep an eye on against Buffalo tonight.

Note: these numbers are subject to change at season's end. The data for 2006-2014 is the same as what we use for stats like receiving plus-minus and YAC+, which excludes passes that are thrown away, batted at the line, or when the quarterback was hit in motion. The 2015 data currently includes all passes, but game charting will filter out those passes that were not truly aimed or intentional.

For those new to this metric, it is called Air Less Expected, or ALEX for short. ALEX measures the average difference between how far a quarterback threw a pass (air yards) and how many yards he needed for a first down. If a quarterback throws a pass 5 yards behind the line of scrimmage on third-and-15, that would be minus-20 ALEX. The best application of ALEX is to look at third and fourth down when it's really crucial to get 100 percent of the need yards to extend the drive.

Posted by: Scott Kacsmar on 23 Nov 2015

18 comments, Last at 24 Nov 2015, 6:51pm by OSS-117


by nat :: Mon, 11/23/2015 - 9:26pm

Cool. Thanks for getting some screen numbers. It's not quite enough to look at the different QBs, which is too bad, but it seems the charting data is a bit harder to marry up with conversion rates than you'd think. This advances the topic nicely, though.

I wouldn't call 15% vs. 25% overwhelmingly bad odds for converting. And if the average ALEX for the 8+ to go screens is really -17 and thus about 14 or 15 to go, that means screens in the 8+ to go group are significantly farther to go than other passes, which average about 11 or 12 to go. Who knows how much that matters. But the drop from the 4-7 group is pretty big. So even a few yards extra could explain a lot.

It's looking like screens aren't quite the "give up plays" we've been hearing about. Nor should they be a team's main tactic on third down. More like part of a mixed strategy to keep defenses from just camping out at the sticks. To really get the picture, you'd need to plot yards to go vs conversion rate for screens and non-screens. That's too much to really do. It would be different for each offense anyway.

by greybeard :: Tue, 11/24/2015 - 3:07am

I watched the KC game this week. And since I commented on this post last week I paid special attention to 3rd downs especially considering that ALEX was conceived by Scott to prove how bad a QB Alex is.
Overall they had a few aggressive calls and a few behind the LOS shorter screen calls for third downs. I am pretty sure they were positive ALEX until they had a 3rd and 21 and they threw a screen for something like -26 ALEX. Which is why Alex's ALEX is I think -3.8 for this week.
For that call: They had a comfortable lead and had no reason to be aggressive: 4th quarter, score 19-3.

KC scored 26 pts in 9 drives. Had comfortable lead throughout the game. And had quite a bit of long 3rd downs after they have some significant lead: 5, 1, 1, 5, 8, 9, 6, 4, 21, 6.
They were aggressive first half (202 passing yards, 5 plays of 20+ yards) and were in cruise control most of the second half where their longer 3rd downs was (about 50 yards of passing).
Which means that the -3.8 ALEX meant nothing for winning the game.
So here are my takeaways:
1) Averages are terrible way to measure something. The outliers kill.
2) When it comes what play is called it would be very context and opponent dependent, which ALEX does not take into account.
3) Teams that like screen passes either because they are weak at Oline or are great at with shift RBs or WRs will have much worse ALEX score though they probably would have less success with better ALEX if they had called something else.
4) Anthonio Brown is awesome

by FlippingADollar :: Tue, 11/24/2015 - 10:51am

Can't disagree with this. Andy loves the screen. The issue I have with it is that he forces Smith to do that even when we aren't in a game controlling situation. It was even crazier considering how well Denver's front 7 were playing. I just hope it doesn't become an issue in the last 6 games.

by Scott Kacsmar :: Tue, 11/24/2015 - 2:57pm

Smith was 3/8 on 3rd-down throws vs. SD, which don't include a sack, a scramble and a penalty for holding. As far as ALEX, just a ho-hum -3.9 day really. Still -1 without the 3rd-and-21 play.

I saw an offense that scored 12 points in the first half, then added a pick-six, had a TD drive that was sparked by Ware's 52-yard run, then Ware had all the yards on a 24-yard TD drive.

Don't know why anyone would bother to defend Smith's strategy, because it is definitely more about Smith than it is Reid. Only QBs lower than him in conversion rate this year are all benched and/or injured right now. You really have to coddle him with YAC, a defense and running game to win games.

by greybeard :: Tue, 11/24/2015 - 3:22pm

They scored 12 points in 4 drives in the first half. And the last one would have been most likely a TD if they had a little more time. They got a FG on 1st down at 6 yard line because of the lack of time. That is pretty good in my opinion. Especially when the opponents have 3 points and have not come closer than 34 yards to the KC end zone.

The 3rd down plays are not Smith's strategy. All behind LOS plays were screens. Smith does not call the plays.

I am not defending Smith. I am attacking this stupid stat using its poster child which happens to be Smith.

by LyleNM :: Tue, 11/24/2015 - 3:57pm

The 3rd down plays are not Smith's strategy. All behind LOS plays were screens. Smith does not call the plays.
You do realize, don't you, that you can still have a negative ALEX without throwing the ball behind the LOS? That's the point of the statistic - not that he's throwing behind the LOS, but that he's throwing short of the sticks.

by greybeard :: Tue, 11/24/2015 - 4:14pm

You do realize screen plays are usually about 5 -6 yards behind the LOS usually making it very terrible ALEX? Even worse than just throwing to LOS. And when dealing with averages that is terrible.

by Scott Kacsmar :: Tue, 11/24/2015 - 5:52pm

How many screens do you really think are being thrown? It's the 2-yard throw on third-and-10 that hurts to watch, and there's no denying Smith is routinely way conservative year after year at all distances.

by greybeard :: Tue, 11/24/2015 - 6:01pm

In this game I think 3 or 4 of the 3rd downs were screen plays. If i check it again I will let you know.

"there's no denying Smith is routinely way conservative year after year at all distances": Yet he has a DVOA of 19%. Which is pretty good. So what is your point: he should be aggressive and less successful so he can have better ALEX?

by Scott Kacsmar :: Tue, 11/24/2015 - 6:18pm

Is 19% "pretty good" when the average here is 30.6%? We know conversion rate is really the main goal anyway, and that's where he ranks 30th out of 36. The short passes for failed completions give him partial credit to help boost the DVOA/DYAR, but the offense is still walking off the field too frequently.

My whole point is we need to do more to acknowledge where QBs are throwing and separate that from what happens after the catch is made. Smith is clearly a QB who relies more on his teammates to make yards for him than someone who is throwing for a lot of air yards. One of this site's first basic principles was to stop treating all 5-yard gains as the same. We can take that a step further with the charting data on where the passes are being thrown.

I appreciate the interest and numbers from Eleutheria below. I don't expect ALEX and CONV% to have perfect or even very strong correlation. They never should. Football is way too complex for that. But there usually is something there each year in the 0.4 range, so it's not strong, not weak, or about what you expect. Stats will have their limitations. They don't know when a receiver is completely wide open because of a blown coverage, so that minus-10 throw is actually a good play. You don't know when a receiver will drop a pass. Stats won't predict when someone goes crazy and breaks three tackles and does their Mark Ingram in SB 25 impersonation. But the stats average out to show those plays are the outliers and not what you expect to happen on an average play. Football has a lot of randomness to it, but 10 yards is still 10 yards, and asking guys to cover long distances without getting tackled first remains a difficult thing to do in this game.

by greybeard :: Tue, 11/24/2015 - 6:40pm

A few different points:
How the hell is 30.6% is average? How is this DVOA thing measured where an average QB is 30.6%? It surely looks like DVOA dropped the ball here.

Conversion rate is very highly correlated with the distance. Smith has the longest average 3rd down distance. So it is expected to be he would be at the bottom of the list for the conversions.

Relying on teammates. I do not agree with this point as I know that he was never surrounded -with the exception of 2012- even average teammates. Look at what happened to Bowe/Avery/McGloin etc. They are not even starting anymore. He pretty much had a below average OLine for his entire career except for 2012, terrible receivers for his entire career (Battle, Morgan, and mostly injured Crabtree were his best receivers until this year, and even this year outside JMac he has a bunch of unproven second and third string receivers). He has been blessed with good RB play - though that also means longer 3rd downs- and TE play.
If these players that are not even starters elsewhere are the ones elevating his play and the reason for any success you are not watching closely.

Smith sells play action well, he sells the screens well and he plays for Andy Reid who loves screen passes and quick throws and he plays with Jamaal Charles who is a very nifty, shifty player. KC Oline has been quite bad in 2013 and 14 and in 15 as well until the last few games. All of these results in a lot of passes that are beyond the LOS. Therefore low ALEX but overall a maximizing-the-success strategy.

by greybeard :: Tue, 11/24/2015 - 4:11pm

Here is some 3rd and 21 and longer stats for you:

In the last 15 years in the first 3 quarters:
Happened 1703 times and got converted by 108 times.
Alex Smith converted 2 out of his 29. Payton Manning converted 3 times out of 27 and Tom Brady 3 times out of 45, Big Ben 1 out of 35.

When leading by 7 or more:
Happened 307 times, converted 23 times.
Alex Smith 0 out of 4. Brady 2 out of 17. Manning 1 out of 3. Big Ben 0 out of 13.

It is funny that you thought somehow DYAR and DVOA did not capture the badness of Smith and had to invent your own metric to prove your point.

by Scott Kacsmar :: Tue, 11/24/2015 - 5:51pm

I didn't invent anything for Alex Smith. I quantified an annoying part of quarterback play that Smith happens to be the poster boy for. It's really not my job to get you to correctly interpret the value of the data. I don't think 3rd-and-21+ represents a common, realistic situation in this game. Every defense has a massive advantage by that point.

by tuluse :: Tue, 11/24/2015 - 1:55pm

Scott, if you take the "worst case scenario" and treat all sacks as failed positive ALEX plays, how would that effect throwing past the sticks vs screens?

by Scott Kacsmar :: Tue, 11/24/2015 - 3:00pm

Sacks were already included in the overall third-down rates referenced above for medium and long situations (the 44 and 25 percents). Didn't include the sacks on screens, though there were like a handful in each of the last two seasons.

by tuluse :: Tue, 11/24/2015 - 3:28pm

Thanks Scott.

by Eleutheria :: Tue, 11/24/2015 - 3:22pm

Since I finally had time to do this properly, lets look at some regression analysis.

First, some simple correlations:
Alex vs 3rd down DVOA: 0.390
Alex vs Conversion%: 0.412
Distance vs 3rd down DVOA: -0.191
Distance vs Conversion%: -0.588

It's not surprising that ALEX correlates better with DVOA then Distance does, since DVOA adjusts for distance. Distance correlated better with Conversion though, which isn't too much of a surprise. I think everyone probably agrees that a bad QB can convert 3rd and short more frequently then a good QB can convert 3rd an long.

Time for some multivariable regressions:
The first one I did was predicting Conversion% based on ALEX, Distance and DVOA (all downs, not from the table above. Note: Now these numbers aren't 100% accurate, since I'm using DVOA from the QB stat page that hasn't been upgraded to include last weekends games, if people request it I'll do a quick update tomorrow when the QB page is updated), putting in the numbers spat out this regression formula:
Conv% = 0.0410*ALEX + 0.3640*Distance + 0.5637*DVOA

These numbers are in Standard Deviations, so a QB who's average yards needed is one standard deviation above average would (all else equal) have a Conversion% that's 0.3640 deviations above average.

So the correlation between ALEX and conversion% is gone. But I have a problem with using DVOA here, since 3rd down success is in the DVOA formula. So I estimated DVOA on 1st/2nd down (by subtracting the QBs 3rd down DVOA weighted based on % of plays that are 3rd downs for each QB)

Doing this provided the following regression formula:
Conv% = 0.1637*ALEX + 0.4295*Distance + 0.2694*FSDVOA
Correlation: .664
So ALEX still correlates with Conv%, though not as strongly as without factoring in the QBs DVOA. The correlation not surprisingly dropped, sin we're no longer using 3rd down DVOA.

Next I did the above two tests, but with 3rd down DVOA instead of conversion%, since 3rd down DVOA takes distance into account, I didn't include distance in this formula.
3rd down DVOA = 0.2235*ALEX + 0.4373*DVOA
Correlation: .562

3rd down DVOA = 0.3937*ALEX - 0.0134*FSDVOA
Correlation: .391

This is perhaps the most interesting one, since Fist/Second Down DVOA correlates negatively with 3rd down DVOA when you adjust for ALEX, and this correlation is slightly strong (0.390 vs 0.391) then Alex vs 3rd down DVOA.

These numbers can mean whatever you like, and they're not 100% perfect. My DVOA numbers are slightly off and I'm looking at each QB rather then individual plays (since I don't have that data).

by OSS-117 :: Tue, 11/24/2015 - 6:51pm

I watched the KC Pit game last year. KC was down 2 scores in Q4 and Reid was calling more vertical routes and Smith kept dumping it short. 3rd and 16, Reid called all verticals and left Smith a check down in Kelce. He never even looked down field and dumped it to Kelce before his receivers had a chance to get to the sticks. 2 yard completion bailed out by the worst call of 2014 when William Gay got flagged for apparently taunting his own teammate? Next play Reid went all verts again, but taking away Smith's safety valve, and he tucks it and scrambles. Just refused to ever uncork it. If it wasn't right in front of his face, a screen/flair, or checkdown, he wasn't throwing it. Thought how painful it must be to be a Chiefs fan and watch this turd. Or any split end knowing all you're doing is just getting a lot of cardio in.