Writers of Pro Football Prospectus 2008

Most Recent FO Features


» 2017 Defensive Personnel Analysis

Defenses have taken a wide variety of responses to the rise of 11 personnel. Is any one system better than another? And how has the rise of the "moneybacker" changed defensive philosophy?

16 Nov 2015

ALEX: Week 10 and Distance Index

by Scott Kacsmar

Week 10 offered some interesting cases of ALEX decisions, but before we get to that, I want to update last week's discussion about a more specific look at screen passes on third down. We have screens in our charting data, but it's not in the same format as the files for our ALEX and plus-minus data. So instead of trying to beat the Monday time crunch and produce something that may not be apples-to-apples and have errors, I will hold off on that particular study until a later time (mid-week or next Monday). Instead we will focus on the notable Week 10 plays and look at a new table for the season results.

Week 10's Most Conservative Plays

The Lowest ALEX

Teams: Philadelphia vs. Miami
Situation: third-and-17 at opponent 19, leading 16-3 in second quarter with 13:31 left
Play: Sam Bradford to Ryan Mathews for 5 yards
Air yards: minus-6
ALEX: minus-23

The Eagles were off to a great start, but stayed conservative with this little screen to Mathews for a meager gain. That brought on Caleb Sturgis for a 32-yard field goal, which he missed, and from that point the Dolphins started their comeback. We are going right back to this game, and with a different quarterback behind center in a far more important situation.

A Notable Failure of the Week

Teams: Philadelphia vs. Miami
Situation: fourth-and-10 at own 44, trailing 20-19 in fourth quarter with 1:01 left
Play: Mark Sanchez to Jordan Matthews for 6 yards
Air yards: 6
ALEX: minus-4

It is one thing to throw a screen when you are up two touchdowns on third-and-17. This was fourth-and-ballgame with the Eagles needing a game-winning field goal, and Mark Sanchez is no stranger to these situations. Miami got pretty good pressure with just a four-man rush, but Sanchez threw to the receiver with the second-shortest route depth. Matthews caught the ball with a Miami defender draped around him and was gang-tackled with no chance for any YAC.

The design of this play was questionable. For better or worse, Matthews is the Eagles' No. 1 wide receiver this season. He is the guy you probably want as a primary target on this play. You cannot design the play to where he lazily jogs a route and hooks at 6 yards on fourth-and-10. The route has to be deeper, sharper and run with a purpose just in case the quarterback has to throw it more quickly than he wants to. You cannot afford to have one of your top players run a route that short of the sticks with his back to the defense.

Unfortunately, it's not like we have a huge sample size of fourth-and-10 passes to look at, especially with minus-4 ALEX. We have six that specific from 2006-2014, and two converted. Overall, negative ALEX on fourth-and-10 only converts 33.9 percent of the time (sample of 56 plays). When ALEX is 1 to 5 yards beyond the sticks, then the conversion rate is 43.8 percent on 89 plays.

Week 10's Most Aggressive Plays

The Highest ALEX

Teams: Baltimore vs. Jacksonville
Situation: third-and-4 at own 48, trailing 7-0 in first quarter with 1:03 left
Play: Joe Flacco pass incomplete to Kamar Aiken
Air yards: 37
ALEX: plus-33

Flacco is no stranger to deep passes like this. Down a touchdown early, he attacked the right sideline, but Davon House had good coverage on the pass. House finished the game with four passes defensed and two interceptions.

A Notable Success of the Week

Teams: Pittsburgh vs. Cleveland
Situation: third-and-12 at own 11, leading 21-3 in third quarter with 9:30 left
Play: Ben Roethlisberger to Antonio Brown (39-yard pass interference penalty)
Air yards: 39
ALEX: plus-27

Ryan Tannehill did have a nice plus-18 ALEX throw to Lamar Miller with a safety in tight coverage, but I think we gave that game enough attention already. Normally I overlook defensive pass interference plays, but Ben Roethlisberger attacked the Browns so much on Sunday in his surprise performance from the bench, that it just deserves a mention. Officially, Roethlisberger threw for 379 yards, but also drew four pass interference penalties for 141 more yards. For reference, in 2014 Roethlisberger drew eight penalties for 156 yards, total. A number like 141 in one game is absurd. Roethlisberger had ALEX throws of plus-27 and plus-28 on third down to draw penalties on the Browns, with Antonio Brown the target on both plays.

2015 ALEX Rankings Thru Week 10

The following table shows where each qualified quarterback (minimum 30 passes) ranks in ALEX on third down only. There are also rankings for DVOA, average need yards (ranked from highest to lowest) and conversion rate.

Rk Quarterback Team ALEX CONV% Rk DVOA Rk Passes Avg. Need
1 Ben Roethlisberger PIT 6.8 47.2% 7 60.1% 7 53 6.9
2 Carson Palmer ARI 4.8 52.1% 3 90.9% 3 73 6.7
3 Ryan Fitzpatrick NYJ 3.6 41.5% 16 32.8% 18 82 7.3
4 Aaron Rodgers GB 3.0 37.5% 23 38.1% 16 72 7.7
5 Blake Bortles JAC 2.9 37.8% 22 29.6% 20 90 8.0
6 Andy Dalton CIN 2.8 51.6% 4 90.4% 4 62 8.0
7 Andrew Luck IND 2.6 42.0% 15 55.5% 8 69 7.6
8 Johnny Manziel CLE 2.5 41.0% 17 97.0% 2 39 8.7
9 Jay Cutler CHI 2.5 48.7% 5 68.9% 6 76 6.5
10 Cam Newton CAR 2.3 35.1% 26 21.1% 23 74 8.1
11 Joe Flacco BAL 2.3 35.6% 25 -17.5% 29 90 6.9
12 Russell Wilson SEA 2.0 39.1% 20 24.1% 21 69 6.9
13 Tyrod Taylor BUF 2.0 43.1% 13 52.2% 12 51 8.3
14 Brian Hoyer HOU 1.9 42.6% 14 52.9% 11 61 8.0
15 Peyton Manning DEN 1.6 32.7% 31 -37.0% 31 98 8.2
16 Matt Ryan ATL 1.5 46.3% 8 35.7% 17 95 6.6
17 Josh McCown CLE 1.4 54.3% 1 129.3% 1 70 7.2
Rk Quarterback Team ALEX CONV% Rk DVOA Rk Passes Avg. Need
18 Tom Brady NE 0.9 53.8% 2 80.3% 5 78 6.9
19 Kirk Cousins WAS 0.8 48.5% 6 55.4% 9 99 7.0
20 Teddy Bridgewater MIN 0.8 35.1% 27 21.2% 22 77 7.6
21 Drew Brees NO 0.8 46.2% 9 44.1% 15 104 7.9
22 Matt Cassel DAL 0.8 37.8% 21 45.6% 14 37 6.9
23 Colin Kaepernick SF 0.7 33.8% 29 12.2% 27 74 7.7
24 Jameis Winston TB 0.6 39.3% 19 -13.8% 28 89 8.3
25 Derek Carr OAK 0.5 45.2% 10 53.3% 10 93 7.5
26 Nick Foles STL 0.4 24.0% 34 -44.7% 33 96 8.6
27 Eli Manning NYG 0.3 45.1% 11 45.8% 13 102 7.2
28 Marcus Mariota TEN 0.2 41.0% 18 31.8% 19 61 7.9
29 Ryan Mallett HOU 0.2 35.7% 24 -40.9% 32 42 7.2
30 Matthew Stafford DET 0.0 34.8% 28 -46.7% 34 89 7.9
31 Ryan Tannehill MIA -0.1 30.9% 32 15.1% 24 81 8.3
32 Philip Rivers SD -0.2 43.5% 12 13.0% 26 92 8.1
33 Sam Bradford PHI -1.0 25.0% 33 -23.0% 30 88 8.5
34 Alex Smith KC -3.8 33.8% 30 14.3% 25 77 8.8

Well, "Third Down McCown" was one thing, but here is Johnny Manziel right behind him in DVOA on a higher ALEX. I thought Manziel was quite impressive in Pittsburgh this week, but we'll see how his season progresses, assuming the Browns are smart enough to keep starting him.

Next, ALEX is presented in splits by distance: short (1 to 3 yards), medium (4 to 7 yards) and long (8-plus yards). The colors indicate where a player is well above average (darker green) versus below average (darker red). Those conversion rates are also shown with a ranking.

Note: these numbers are subject to change at season's end. The data on 2006-2014 is the same as what we use for stats like receiving plus-minus and YAC+, which excludes passes that are thrown away or batted at the line, or when the quarterback was hit in motion. The 2015 data currently includes all passes, but game charting will filter out those passes that were not truly aimed or intentional.

For those new to this metric, it is called Air Less Expected, or ALEX for short. ALEX measures the average difference between how far a quarterback threw a pass (air yards) and how many yards he needed for a first down. If a quarterback throws a pass 5 yards behind the line of scrimmage on third-and-15, that would be minus-20 ALEX. The best application of ALEX is to look at third and fourth down when it's really crucial to get 100 percent of the need yards to extend the drive.

Posted by: Scott Kacsmar on 16 Nov 2015

18 comments, Last at 19 Nov 2015, 8:23am by nat


by RickD :: Mon, 11/16/2015 - 4:57pm

The mid-range numbers for Brady are interesting. He has the #1 conversion rate with an ALEX of merely 0.6. Apparently the Pats really specialize in throwing right to the 1st down marker.

By contrast, Cam Newton has a massive ALEX of 8.2 for the same range, but the lowest conversion rate.

Is there some anti-correlation going on here? At least for positive values of ALEX perhaps? Or am I stumbling upon some counter-indicative outliers?

by Scott Kacsmar :: Mon, 11/16/2015 - 9:19pm

I'm a bit under the weather right now, but I can check later to see the number of attempts for that range. Might not be a lot in 4-7 yards.

Just read Nick Foles was benched. He's managed to rank last in CONV% for each split here. Lot of dark red there.

The other thing that has stood out to me this week is Aaron Rodgers has climbed back up to 4th in ALEX. He has ranked in the top 4 in every season since 2008, so even if things aren't looking right in GB, that part of his game is at least returning to form. Just not having as much success this year.

by nat :: Mon, 11/16/2015 - 9:29pm

Feel better soon.

When you feel up to it, could you see if you can get conversion rates for screens with short/medium/long to go from standard FO data and charting? It might help tease apart play calling and in play check downs in their effects on ALEX and success. Screens aren't the full story, of course. But they are automatically the lowest ALEX plays. So it would be helpful to see how successful they are as a set.

by Scott Kacsmar :: Tue, 11/17/2015 - 3:34am

I started getting those screen numbers, but my concern was comparing it to all other passes and having the right sample for that. As a set by themselves, it won't be hard to look at them. I also want to look at screens for the other downs to see how they stack up in conversion rate. You would think a defense would play closer to the sticks on third-and-10 than they would on first-and-10 when the whole playbook is really available.

by nat :: Tue, 11/17/2015 - 8:09am

You should just tell us the short/med/long third down conversion rates for screens. It's not a perfect stat, but since there's an idea out there that screens are "give up plays" that seldom convert, it would be good to check anyway.

We can easily get conversion rates for all passes to compare. You should use the same time periods as you do for ALEX for clarity.

It might be hard to get other categories, such as designed short passes vs check downs. But that should not stop us from investigating those screens.

This isn't as hard as you're making it.

by nat :: Wed, 11/18/2015 - 11:04am

Stonewalling just looks bad. Buying time to comb through the data looking for new ways to slice or spin it just looks bad.

Just show what you said you would show:

What are the conversion rates for screens on third down at the three distances: Short (1-3 yards to go), Medium (4-7), and long (8+)?

Once you've done that, then people - including you - can investigate and discuss why the numbers might be what they are. But for now, it's just you appearing to hide data you don't FO readers to see.

by Scott Kacsmar :: Wed, 11/18/2015 - 1:06pm

When people tell me the comments section is getting worse to read, this is what they are referring to. Stonewalling? I have a life, and right now that includes needing to be up at certain times for home repairs and for visits to a nursing home to spend time with a family member. I have a job, and that job has specific deadlines early in the week. I am a bit more relaxed on Wednesdays and Thursdays during the season, and that is when I will take the time to put together screen data that can be used in Monday's article. I'm not going to post that data to appease one stubborn user in the comments section on a two-day-old article.

by nat :: Wed, 11/18/2015 - 4:10pm

Now you're trying diversionary tactics to avoid meeting your pledge to see and share what the actual data shows about third down screens. Please.

(cue stirring music,
a Football Outsiders flag seen waving on the distant battlements)

FO has always been about looking at the real data to inform opinions for both writers and commenters. Sometimes that means being man or woman enough to admit when we are wrong about some aspect of football or statistics, some piece of conventional, and perhaps dearly held wisdom. We argue. We disagree. But we will always agree to look to the data to find a step towards truth. We will always share the data we have.

This case is no different, and you (and I, and the rest of us) should not try to be exempt from those fine FO traditions.

It's time to put your cards on the table, Scott.

(dramatic pause)

It's time.

(the music pauses,
the camera turns to Mr. Kacsmar, who takes a deep breath, and says...)

by LyleNM :: Wed, 11/18/2015 - 5:05pm

While I am interested in seeing what comes out of this, this is the kind of comment that, were it my (free) website, would lead to a commenting ban.

Seriously, nat, we get that you don't think much of anything that Scott does, but throwing a tantrum because he's not providing a free service on the timescale you are demanding is extremely unlikely to accomplish anything useful.

by nat :: Wed, 11/18/2015 - 6:44pm

Was it the music? The waving flag? The "FO's finest traditions" shtick?
Was it ever expecting Scott to just "put his cards on the table"?

It was a joke. Didn't the music and camera stuff give it away?

Look. I get it. Scott looked at the data and now we can't really make him show it if he doesn't want to. Not ever. All we can do is to ask. And when he gets impolite and huffy, all we can really do is poke fun at him for holding back and then suck it up. Next week we can ask again.

Scott, if you are reading this, put away that email demanding a ban. That's not necessary, and I don't think justified.

I promise I will treat the data on third down screen conversion rates fairly. We all will. On my honor.

And I hope you are feeling better from earlier in the week. I've seen you posting in comment threads here and there, so I assume you're back on your feet. That's good news.

by Aaron Schatz :: Wed, 11/18/2015 - 11:52pm

It's amazing how much readers a) overestimate how many employees FO has ; b) overestimate the amount of time we all have to write and research each week; c) overestimate how easy it is to put data together from a number of different sources, especially since none of us are computer programmers or database specialists. Did you notice that we no longer are doing the volunteer charting project this year? We're partnering with a larger company for that because we are small. And we're still working out a lot of the technical database issues in that partnership.

Nobody is hiding anything from you. Nobody is holding any numbers back. We're trying to have time during the week to spend with our families. We're trying to have a little bit of leisure time. And we're trying to devote the majority of our work time to the things that make us money so we can feed ourselves and our children. I'm sorry for getting all angry and defensive in the comments, I really do try to avoid that weakness of being an Internet writer, but this pushed me. KNOCK IT THE F OFF. Your last comment may have been playful, but the demands before were not.

by nat :: Thu, 11/19/2015 - 8:23am

I think my last two posts had already knocked it off by making light of the issue. But I'll try to be more understanding. I will.

You've probably hit the nail on the head when you say people underestimate the difficulty of pulling data together. I know you have tons of observations about DVOA and success splits in Quick Reads. I had always envisioned that by that point of the week you had the simplest charting results tied to your list of plays, and that it would include the "screen" categorization for passes. It sounded like it was tying the ALEX table in unnecessarily that was causing delays. That's why I focused on conversion rate. I expected it to be the simplest, as well as the most easily interpreted stat. It really would clarify things to see it.

I'm glad nothing is being withheld. I'll try to be more patient.

Keep spending time with your family. Make FO a success. Remain true to your best ideals. As always, you run a great site.

by nat :: Tue, 11/17/2015 - 8:55am

Brady has 30 pass attempts on third down with 4-7 yards to go. That puts him in the top half of the league at 12th. Sample size isn't an issue for his number, at least comparatively. From the PFR play finder.

by vrao81 :: Mon, 11/16/2015 - 5:15pm

Remember that Alex doesn't account for the pass being complete or incomplete. So Newton is throwing a lot of deep passes, but they aren't being completed, because Devin Funchess, Corey Brown, and Ted Ginn Jr.

by Eleutheria :: Mon, 11/16/2015 - 6:38pm

So running a multivariable regression with ALEX, DVOA and DistanceNeeded as the variables, there is no correlation between Alex and conversion%.

Though since I'm using players not individual plays, it's not completely accurate.

by Moridin :: Tue, 11/17/2015 - 3:06am

Yeah, ultimately, I feel like ALEX isn't something that is going to apply universally well. Other than what Scott has already noted that throwing for a 1st down with the yards in the throw is a much better idea than short and hoping for YAC. Perhaps if the QBs are grouped by offensive philosophies?

Really, where I find this bottom table shines is in evaluating a QB & the offensive system he's in (and other players he's works with). If they convert well, then the QB is being utilized well in the system & w/ the correct players running routes, regardless of ALEX. If they have a specific weakness, you know that there's an issue. Or you can suck at all 3 (Foles) and either be a poor QB or a poor fit in that offense. Some QBs even have better 3rd & long conv% than med (Cam & Carr, Bridgewater, Palmer [though that's because his long is ridiculous], McCown & Mallet).

There are some pretty interesting splits where QB's rates are either average+ in 2 and bad in 1 or bad in 2 and average+ in 1. Like Cam and 3rd and medium. His throwing it long on medium plays is really hampering drive success. Derek Carr and his very good 3rd and short and 3rd and long, but crappy 3rd and medium. Or poster boy Alex Smith, who seems to have the same play package for 3rd and short as 3rd and long, both with terrible results (I assume WR & RB screens and dump offs). 3rd and medium though, he seems to throw around the sticks and has a respectable conversion, making it seem like the 3rd and medium play packages are a much better fit.

I certainly wanted to see Bridgewater's splits, and its interesting to see he's below average at 3rd/short while throwing at the sticks. Above average on 3rd/long throwing short of the sticks (he gets decent YAC and broken tackles on those obviously, which matches what I see of alot of his dump offs). And he sucks at 3rd/medium where he throws well beyond the sticks. Being only above 50% on short isn't good, but seeing that medium conversion means his play package and route choices are very poor for him, and is probably a huge reason for drive death for the Vikings.

I dunno, at the very least, it certainly seems like there's a lot of conversational pieces here. Aaron Rodgers probably isn't usually so terrible at 3rd/short, and he's going well past what he needs while sucking at it. I wonder if he's throwing longer this year on short downs or if his ALEX is normal and he's just not completing them.

by Eleutheria :: Tue, 11/17/2015 - 10:43pm

One thing I gage from the lower picture is that QBs like Mariota and Carr will probably soon see their 3rd down coversions regress to the mean.

Their ALEX scores on 3rd and long are bad, while their success rate is good.

by David C :: Tue, 11/17/2015 - 2:41am

Try restricting your sample to players with average distance needed of 8 yards or more.