Writers of Pro Football Prospectus 2008

Most Recent FO Features

SmithAle05-QB.jpg

» 2017 ALEX: Midseason Report

The latest ALEX update looks at the recent draft class that is struggling, the unusual Chicago strategy, and what's gotten into Alex Smith? We also looked at Tyrod Taylor's declining ALEX, but rising conversion rate that Buffalo just sent to the bench.

02 Nov 2015

ALEX: Week 8

by Scott Kacsmar

In the coming weeks we will be using ALEX to help analyze other statistics familiar to FO readers such as failed completions and success rate adjusted for down and distance. We will also get into the defensive numbers to see if any defenses are consistently good at forcing offenses into shorter throws. These studies take more time than a quick Monday write-up permits, so in the future we'll have the numbers prepared late in the week so that we can look at these things on a Monday along with the recap of the week's notable ALEX plays.

Since we are getting close to the midway point of the season, here is a look at which quarterbacks have finished first and last in ALEX on third down for each season from 2006 to 2014 (minimum 50 passes).

Season Leader ALEX Last ALEX
2006 Michael Vick 6.97 David Carr -0.80
2007 Sage Rosenfels 4.09 Brian Griese -1.63
2008 Tony Romo 3.62 Jason Campbell -2.62
2009 Philip Rivers 3.73 Trent Edwards -2.79
2010 Michael Vick 4.73 Jimmy Clausen -2.18
2011 Ben Roethlisberger 4.56 Kevin Kolb -2.78
2012 Drew Brees 4.26 Blaine Gabbert -2.40
2013 Matt McGloin 4.08 Alex Smith -2.15
2014 Drew Stanton 3.94 Alex Smith -2.04
2015 Ben Roethlisberger 9.11 Alex Smith -3.83

We will see if Ben Roethlisberger can join his current backup Michael Vick as the only quarterbacks to lead the league in ALEX twice. Roethlisberger is bound to regress to the mean eventually here. Alex Smith is looking for a three-peat and a new low on the other end, and the last-place finishers include some very notable captain checkdowns.

For those new to this metric, it is called Air Less Expected, or ALEX for short. ALEX measures the average difference between how far a quarterback threw a pass (air yards) and how many yards he needed for a first down. If a quarterback throws a pass five yards behind the line of scrimmage on third-and-15, that would be minus-20 ALEX. The best application of ALEX is to look at third and fourth down when it's really crucial to get 100 percent of the need yards to extend the drive. Here is where we review the week's most conservative and aggressive plays by ALEX on third and fourth downs.

Week 8's Most Conservative Plays

The Lowest ALEX

Teams: Cincinnati at Pittsburgh
Situation: third-and-34 at own 43, trailing 7-6 in third quarter with 11:05 left
Play: Andy Dalton to Mohamed Sanu for 7 yards
Air yards: 2
ALEX: minus-32

This game got ugly in the second half. A penalty for an illegal peelback block eventually led to a third-and-34 for the Bengals. Andy Dalton scrambled to his left, but eventually just dumped the ball down to Mohamed Sanu for a short gain before a punt from midfield.

Cincinnati actually has the lowest ALEX (minus-30) for a play that converted since 2006, and it was a total fluke. In 2009, Carson Palmer threw a TE screen three yards behind the line of scrimmage to Daniel Coates, who gained 23 yards before getting stripped of the ball from behind. Laveranues Coles came through with the recovery on what became a 38-yard gain and first down against the Packers.

A Notable Failure of the Week

Teams: New England vs. Miami
Situation: third-and-16 at own 26, tied 0-0 in first quarter with 12:07 left
Play: Tom Brady to Dion Lewis for 17 yards (first down)
Air yards: minus-4
ALEX: minus-20

This almost had to be a Miami play from Thursday night, but instead of going over the minus-25 ALEX dump to Jonas Gray, let's go to the defense's failure at the beginning of the game. This one really set the tone for the night as Miami had a good chance to get a quick stop on third-and-16. New England ran a simple running back screen, but the Dolphins weren't able to wrap up Dion Lewis short of the sticks, and he blew through a tackle attempt for the first down. The Patriots went on to score a touchdown and never trailed in the 36-7 win.

In 2015, plays with a minus-20 ALEX or lower on third or fourth down have converted just 3 of 61 times (4.9 percent). The Patriots have two of those conversions now with Danny Amendola getting the other on a bubble screen against the Colts.

Week 8's Most Aggressive Plays

The Highest ALEX

Teams: Pittsburgh vs. Cincinnati
Situation: third-and-5 at own 22, leading 7-6 in third quarter with 11:56 left
Play: Ben Roethlisberger pass intended for Antonio Brown intercepted by Reggie Nelson
Air yards: 48
ALEX: plus-43

This game was wild as both of the week's highest ALEX plays came on interceptions thrown by Ben Roethlisberger and Andy Dalton (plus-39 ALEX in the fourth quarter). Both served as punts, though Roethlisberger's bomb here was returned to the Pittsburgh 33. However, the Bengals moved backwards and that became the drive that led to our lowest ALEX play mentioned earlier on third-and-34.

A Notable Success of the Week

Teams: Baltimore vs. San Diego
Situation: third-and-1 at own 29, trailing 10-6 in second quarter with 9:24 left
Play: Joe Flacco to Chris Givens for 39 yards (first down)
Air yards: 39
ALEX: plus-38

This was a strange one, because the CBS broadcast changed the graphic from third down to second-and-2, but this was definitely a third-and-1. Baltimore almost appeared to attack it like a second-and-2, because Joe Flacco simply lined up in the shotgun and lofted a good deep ball down the sideline to Chris Givens for 39 yards. He beat Brandon Flowers, who has had his share of struggles in 2015. The Ravens finished this drive off with a touchdown.

2015 ALEX Rankings Thru Week 8

The following table shows where each qualified quarterback (minimum 30 passes) ranks in ALEX on third down only. There are also rankings for DYAR, average need yards (ranked from highest to lowest) and conversion rate.

Note: these numbers are subject to change at season's end. The data on 2006-2014 is the same as what we use for stats like receiving plus-minus and YAC+, which excludes passes that are thrown away, batted at the line or when the quarterback was hit in motion. The 2015 data currently includes all passes, but game charting will filter out those passes that were not truly aimed or intentional.

Rk Quarterback Team ALEX DYAR Rk Passes Avg. Need Rk CONV% Rk
1 Ben Roethlisberger PIT 9.1 302 4 36 7.1 25 55.6% 1
2 Carson Palmer ARI 5.6 423 2 62 6.4 30 51.6% 3
3 Ryan Fitzpatrick NYJ 5.3 236 13 60 6.8 28 46.7% 8
4 Blake Bortles JAC 3.8 161 15 66 7.8 14 37.9% 19
5 Russell Wilson SEA 3.0 156 16 62 6.2 32 41.9% 15
6 Andy Dalton CIN 3.0 238 12 54 8.2 9 48.1% 7
7 Tyrod Taylor BUF 2.5 90 21 37 8.4 8 40.5% 16
8 Jay Cutler CHI 2.1 294 6 59 6.7 29 49.2% 5
9 Joe Flacco BAL 1.9 -61 28 79 7.0 26 34.2% 27
10 Brian Hoyer HOU 1.9 241 11 61 8.0 13 42.6% 14
11 Aaron Rodgers GB 1.8 154 17 49 7.4 20 36.7% 22
12 Matt Ryan ATL 1.8 272 7 83 6.4 31 48.2% 6
13 Drew Brees NO 1.6 297 5 87 7.7 18 46.0% 10
14 Josh McCown CLE 1.4 578 1 70 7.2 22 54.3% 2
15 Cam Newton CAR 1.4 46 24 47 8.1 10 36.2% 24
16 Andrew Luck IND 1.1 150 18 45 8.4 5 37.8% 21
Rk Quarterback Team ALEX DYAR Rk Passes Avg. Need Rk CONV% Rk
17 Nick Foles STL 0.9 -146 30 69 8.4 7 26.1% 31
18 Kirk Cousins WAS 0.9 251 9 82 7.0 27 46.3% 9
19 Colin Kaepernick SF 0.7 80 23 74 7.7 16 33.8% 28
20 Teddy Bridgewater MIN 0.6 174 14 66 7.7 15 37.9% 20
21 Philip Rivers SD 0.5 121 20 82 7.7 17 43.9% 12
22 Derek Carr OAK 0.4 267 8 68 7.5 19 45.6% 11
23 Peyton Manning DEN 0.3 -162 32 79 8.0 12 32.9% 29
24 Ryan Mallett HOU 0.2 -73 29 42 7.2 21 35.7% 25
25 Matthew Stafford DET -0.1 -156 31 74 8.1 11 35.1% 26
26 Jameis Winston TB -0.2 28 25 70 8.7 3 40.0% 18
27 Marcus Mariota TEN -0.4 86 22 42 8.4 6 40.5% 17
28 Tom Brady NE -0.5 328 3 55 7.2 24 50.9% 4
29 Sam Bradford PHI -0.6 -54 27 71 8.6 4 25.4% 32
30 Ryan Tannehill MIA -0.7 28 26 63 8.9 1 27.0% 30
31 Eli Manning NYG -1.0 249 10 75 7.2 23 42.7% 13
32 Alex Smith KC -3.8 127 19 69 8.8 2 36.2% 23

Posted by: Scott Kacsmar on 02 Nov 2015

16 comments, Last at 04 Nov 2015, 11:58pm by alan frankel

Comments

1
by Jimmy Oz :: Mon, 11/02/2015 - 8:29pm

How about some splits on the 3rd/4th downs based on yards to get, like 1-3 yards, 3-6 yards, & 7+ or whatever?

3
by Scott Kacsmar :: Mon, 11/02/2015 - 9:05pm

I might be able to squeeze in six columns (SHORT, Rk, MID, Rk, LONG, Rk) at the end of the weekly table to present that.

2
by Hoodie_Sleeves :: Mon, 11/02/2015 - 8:47pm

Not really seeing anything that makes me think ALEX is meaningful - the quarterbacks at the top of the list don't seem to be better ones than the ones at the bottom - it seems to say something about style, but ranking them seems like it insinuates a quality judgement.

4
by Eleutheria :: Tue, 11/03/2015 - 3:05am

Well the general view among advanced analysts is that being aggressive on 3rd and 4th down is a good thing, so I think this is a relatively decent way of rating a QBs decision making.

7
by Hoodie_Sleeves :: Tue, 11/03/2015 - 2:44pm

"Being Aggressive" and "Air Yards" have very little to do with each other - and neither is a good indicator of decision making ability/performance.

Very often, the best decision isn't throwing to the guy who's the furthest down the field - which is what ALEX measures.

14
by Eleutheria :: Wed, 11/04/2015 - 11:23am

If you call short passes in third and long you will, over the long run, have a 3rd down efficiency that is lower then someone who calls long passes in third and long. Assuming two offenses of equal calibre.

I'm not saying you always need to be aggressive. I'm saying ALEX measures who aggressive a quarterback is on third/4th down, and in these situations being aggressive pays off.

5
by Jerry :: Tue, 11/03/2015 - 6:02am

Nobody's claiming ALEX is Meaningful. It's just interesting data.

6
by Scott Kacsmar :: Tue, 11/03/2015 - 8:03am

I wouldn't work on it if I didn't find it meaningful. I can also tell you at least one NFL team has been using the ALEX concept in how they approach their play-calling.

I don't think I ever posted the master list, but here's everyone under +1.0 ALEX from 2006-2014 (min. 200 third-down passes)

Josh Freeman 0.97
Nick Foles 0.96
Kyle Orton 0.95
Chad Henne 0.94
Sam Bradford 0.92
Matt Cassel 0.81
JaMarcus Russell 0.76
Ryan Tannehill 0.67
Shaun Hill 0.36
Matt Schaub 0.34
Brandon Weeden 0.29
Steve McNair 0.26
Robert Griffin III 0.16
Christian Ponder 0.05
Marc Bulger -0.17
Kurt Warner -0.32
Colt McCoy -0.37
Jason Campbell -0.52
Joey Harrington -0.55
Alex Smith -0.61
Trent Edwards -1.52
Blaine Gabbert -1.91

A few one-year wonders, Matt Schaub had an extended run of success, and this was old Arizona Kurt Warner, but overall the list speaks for itself in terms of the caliber of QB play.

It's pretty much impossible to find a QB who is below average in ALEX and is still consistently successful in this league. You can't live on expectation-beating YAC.

Like every other imperfect stat, you'll get some weird results from the top and bottom for a season or even a career. Look at sack rate. Guys like Randall Cunningha, Roger Staubach and Ben Roethlisberger are really low. Guys like Joey Harrington and Mike Tomczak are in the top 10. We've been looking at only quarterbacks so far, but this can easily be looked at for receivers too.

ALEX is definitely more about playing style and context. It's not a counting (production) stat, and it's really not an efficiency stat by itself either. It is in the same family of research as failed completion, REC +/- and YAC+. Stats that are looking at where a ball is being thrown and what happens after the catch point.

I think using data to put playing style into context is very important, because it helps you understand the other stats a player has. So if you see a QB has a low interception rate, you can dig into that by finding out how often he scrambles or takes sacks (afraid to pull the trigger), where he's throwing the ball, how often he throws deep, how far he's throwing relative to the sticks, what's his breakdown of incompletions (overthrows? defensed? throwaways?), dropped interceptions, etc.

8
by nat :: Tue, 11/03/2015 - 3:25pm

Interestingly enough, almost half (10 of 22) of those "very low ALEX" guys are in the top half for passer rating on third down for QBs with 200+ attempts 2006-2014.

That's pretty amazing, since a bunch of their passes would be check downs, that is, longer (more "aggressive") pass plays that had already failed so badly that there was no one to throw to, but that in ALEX get lumped with plays designed to exploit the defense's weaknesses by going short.

It's even more amazing when you consider that pretty much all of the third down sacks would be during the plays designed to go deeper, too, but do not show up in passer rating.

9
by tuluse :: Tue, 11/03/2015 - 3:28pm

If you are completing passes you pretty much get a good QB rating. This doesn't surprise me very much. Deeper passes are more likely to fall incomplete or be picked.

11
by nat :: Tue, 11/03/2015 - 4:17pm

Passer rating is a little more sophisticated than that. But I agree it's not perfect. I'd like to specifically reward first downs and penalize interceptions (since they reduce the first downs your opponent needs by giving up field position).

So I tried this: (First downs - 2xInts)/(pass plays)

That's an effective first down percentage, with a small adjustment for the lost field goal chances, lost field position, and possible pick sixes that come with interceptions. It doesn't reward yardage at all, only the ability to get (or negate through bad plays) first downs.

Half of the "low ALEX" QBs listed were in the top half by this stat.

Who knows? Perhaps there is a small effect somewhere in this mess after you deal with all the flaws. But as many QBs do well with low ALEX numbers on third down as do badly.

15
by Eleutheria :: Wed, 11/04/2015 - 11:29am

You're overrating how bad interceptions are in this, especially if we're talking about players with high ALEX, since an interception thrown 45 yards downfield is no different then a punt in terms of field position.

10
by Hoodie_Sleeves :: Tue, 11/03/2015 - 3:58pm

"It's pretty much impossible to find a QB who is below average in ALEX and is still consistently successful in this league. "

Tom Brady this year:

Alex -.5 Rk 24
Conv% - 50.9 Rk 4

Overall
DYAR - 828 - Rk 1
DVOA - 34.6- Rk 3

If you think your stat is a measure of "Quality" that's a huge problem. If you think it's about style, that makes perfect sense - and it's interesting.

As a quality stat, the stat is making the 'decision' that more 'air yards' is better - when that's very often not the case. In the case of a heavy pass rush - screens can be the highest success rate - and they're always hugely negative in ALEX.

I ran some correlations and regressions - and my general results were that ALEX correlated to Conv% at a much lower rate than overall QB DVOA, and even correlated at a lower rate than "Avg Yards to Go". It's just not particularly good at telling us how often QBs are going to get first downs in these situations - largely because it ignores the fact that higher ALEX plays are always going to be lower percentage plays. I wonder how well it would correlate to qb variance. At a glance, I'd guess well.

12
by Scott Kacsmar :: Tue, 11/03/2015 - 4:23pm

Watch where the numbers are by season's end. The 2015 Patriots haven't cracked the code on how to run screens or beat expectations with YAC. Most third-and-long screens lead to the offense coming off the field. That's just a fact.

On throws with minus-15 ALEX or worse, Tom Brady was 2/36 at converting on third down from 2006-2014. He is 2/2 this season because of screens to Amendola and Lewis. Which one do you think is the fluke? Brady's even 3/3 this year if we said minus-14. It's not going to continue.

Yes, it makes more sense for a team like NE to throw such short passes, because they have several players who are really good at getting YAC. Edelman always seem to fall forward, Lewis is very elusive, and Gronk is so hard to bring down quickly. But you can't live on these plays all season.

And 2015 is not very indicative of how Brady has fared with ALEX or his 3D conversion rate throughout his career.

13
by Scott Kacsmar :: Wed, 11/04/2015 - 6:15am

Don't know why I didn't say this in the first place, but we're talking about 55 plays for Brady. Those two screens with the minus-20 ALEX are the difference between him sitting at minus-0.54 and plus-0.19. It's still early in the season, so plays like that are going to have an impact on the numbers.

16
by alan frankel :: Wed, 11/04/2015 - 11:58pm

These stats don't seem to come out to any real conclusive conclusions. The most useful thing so far seems to be that the players that are taking the least amount of 3rd down snaps are the best players at the position. Also instead if using a players average Alex wich gets skewed by a couple of deep or short throws. use a frequency model that show how many times qb goes over or under the yrds needed. Another thing it's possible that going slightly over or slightly under the marker is better than going exactly for it so myb if we look at each play more individually we could find a reliable average proximity over and under that indicates smart qb play