ALEX: Week 9 and Short vs. Stick Throws
by Scott Kacsmar
The plan this week was to dig into the numbers on the most common third-down passing situations, but Sam Bradford's slow start on Sunday night caught my attention. He had a few minus-ALEX plays early in the game on third down that kept coming up short of the sticks. So what happens when we take our data for 2006-2014 and look at passes thrown just short of the sticks (minus-1 ALEX) compared to those thrown exactly at the sticks (0 ALEX)?
The results were pretty interesting. The following graph shows the completion percentage and conversion rate for each type of throw ("Short" and "Stick") at each third-down distance up to 16 yards or more. There were 3,902 Stick throws and 2,400 Short throws. Note that the Stick throw rates (red and green) are not 100 percent correlated because of a small number of plays where a receiver caught the ball at the sticks but still failed to pick up the first down -- usually lost fumbles, plus a handful of plays where a receiver lost the first down by running backwards after the catch. In fact, those things have happened 50 times. Demaryius Thomas did this twice last year on third-and-long. Roddy White is the only other receiver with two such plays.
The conversion rate is higher on Stick throws at every third-down distance, and often by more than double-digit percentage points. The strategies are at their closest with 1-2 yards to go when defenses are expecting a run, so you can often catch them by surprise. Once you get to third-and-3, the Short throw never converts more than 47 percent of the time, while the Stick throws convert at least 44.5 percent at every distance from 1 to 12 yards. Once you get past the 5-yard legal contact zone, there is a noticeable drop in conversion rate for both strategies, but it is a larger drop on the Short throws, especially starting at third-and-8.
The completion rates are usually about 10 percentage points higher on the Short throws, since the defense tends to keep the play in front of them -- "tackle the catch," as Dick LeBeau would teach. NFL defenses are actually not too bad at making the immediate tackle a yard short of the sticks. On a money situation like third-and-10, the Short completion rate soars back up to 70.3 percent, but only 36.1 percent of those throws are converted for a first down. Compare that to the Stick throws: the completion percentage drops to a lower 44.9 percent completion rate, but with 44.5 percent conversions. Offenses completing a 9-yard throw on third-and-10 have converted 51.4 percent of the time since 2006 (57-of-111), so it is about a 50/50 proposition if you go that route.
What about average gains? This next graph compares the yards per attempt (YPA) of each passing strategy. It is true that you could usually get more YAC on the Short throws, but the difference is very marginal in the range of 5 to 9 yards to go. On third down, conversion rate usually trumps everything else, but not in every situation.
Finally, for the risk averse, what about interceptions? This chart looks at interception rates. The Short throw usually is intercepted less often, though we see some odd splits like a rate twice as high as the Stick throw on third-and-3, which is universally a passing down in this game. The interception rate on third-and-9 is nearly three times higher on Short throws, though this speaks to the small sample sizes (average 217 Short throws for the 1- to 10-yard range) and the general flukiness of interceptions.
If you really want the conversion, no matter what the distance to gain is, on average you are better off throwing to the sticks than you are throwing 1 yard short. Yes, sometimes the best option is to get the ball to a wide-open guy as soon as possible even if he's short of the sticks, but on average you want to avoid the negative ALEX plays. It is one of the most frustrating things in the NFL to watch a team complete a third-down pass a yard short of the sticks, but we see it all the time. Sometimes the quarterback makes a bad throw, and sometimes the receiver just does not run his route to the proper depth.
We have been spending a lot of time on the quarterbacks, but receiver data for ALEX will be coming.
For those new to this metric, it is called Air Less Expected, or ALEX for short. ALEX measures the average difference between how far a quarterback threw a pass (air yards) and how many yards he needed for a first down. If a quarterback throws a pass 5 yards behind the line of scrimmage on third-and-15, that would be minus-20 ALEX. The best application of ALEX is to look at third and fourth downs when it's really crucial to get 100 percent of the needed yards to extend the drive. Here is where we review the week's most conservative and aggressive plays by ALEX on third and fourth downs.
Week 9's Most Conservative Plays
The Lowest ALEX
Teams: Buffalo vs. Miami
Situation: Third-and-23 at own 16, leading 19-14 in third quarter with 8:38 left
Play: Tyrod Taylor to LeSean McCoy for 14 yards
Air yards: minus-4
ALEX: minus-27
Miami tightened things up a bit at this point. Tyrod Taylor was sacked on first down, set up a terrible screen that lost 7 yards on second down, then threw another screen here to McCoy for a 14-yard failed completion. Taylor was 11-of-12 passing on the day, but had three failed completions and one incompletion on the four targets that did not go to Sammy Watkins.
A Notable Failure of the Week
Teams: Denver at Indianapolis
Situation: Third-and-14 at opponent 32, trailing 24-17 in fourth quarter with 11:04 left
Play: Peyton Manning to Demaryius Thomas for 17 yards (first down)
Air yards: 2
ALEX: minus-12
The Indianapolis defense did it again. Danny Amendola followed his blockers on a bubble screen (minus-20 ALEX) earlier this season to beat them, and Thomas got them here in another big spot. This looked more like 2013 Denver in the screen game than what we've seen since, but the key to the play was Colt Anderson, playing for an injured Mike Adams, unable to make the tackle near the catch point. Thomas had a quick, easy path to 17 yards on a drive that ended with a game-tying touchdown.
Week 9's Most Aggressive Plays
The Highest ALEX
Teams: St. Louis at Minnesota
Situation: Third-and-5 at own 39, trailing 10-0 in first quarter with 2:37 left
Play: Nick Foles to Kenny Britt for 55 yards (first down)
Air yards: 51
ALEX: plus-46
Nick Foles did not do much on Sunday, but he threw a nice ball here. The Rams tried this play on the first play of the game, but Foles ended up scrambling to throw a checkdown. This time he went deep and Britt beat Terence Newman down the middle of the field.
This is now the highest ALEX completion on third or fourth down in 2015.
A Notable Success of the Week
Teams: Buffalo vs. Miami
Situation: Third-and-14 at opponent 44, leading 19-14 in third quarter with 2:29 left
Play: Tyrod Taylor to Sammy Watkins for 44 yards (touchdown)
Air yards: 44
ALEX: plus-30
To Taylor's credit, those targets to Watkins were very productive on Sunday. He throws a pretty deep ball and Watkins was on the receiving end in the end zone for a touchdown from which Miami never recovered. In trying to keep up with Watkins, Brent Grimes fell down at the end of the play, a sight that has just been all too common for Miami fans.
2015 ALEX Rankings Through Week 9
The following table shows where each qualified quarterback (minimum 25 passes) ranks in ALEX on third down only. There are also rankings for DYAR, average need yards (ranked from highest to lowest), and conversion rate.
Note: these numbers are subject to change at season's end. The data on 2006-2014 is the same as what we use for stats like receiving plus-minus and YAC+, which excludes passes that are thrown away, batted at the line or when the quarterback was hit in motion. The 2015 data currently includes all passes, but game charting will filter out those passes that were not truly aimed or intentional.
| Rk | Quarterback | Team | ALEX | DYAR | Rk | Passes | Avg. Need | Rk | CONV% | Rk |
| 1 | Ben Roethlisberger | PIT | 7.7 | 299 | 7 | 48 | 7.0 | 30 | 50.0% | 5 |
| 2 | Carson Palmer | ARI | 5.6 | 416 | 2 | 62 | 6.4 | 34 | 51.6% | 4 |
| 3 | Johnny Manziel | CLE | 4.0 | 110 | 23 | 27 | 8.1 | 9 | 40.7% | 18 |
| 4 | Ryan Fitzpatrick | NYJ | 3.9 | 227 | 13 | 71 | 7.1 | 27 | 45.1% | 11 |
| 5 | Blake Bortles | JAC | 3.5 | 158 | 16 | 77 | 7.6 | 20 | 39.0% | 22 |
| 6 | Russell Wilson | SEA | 3.0 | 150 | 17 | 62 | 6.2 | 35 | 41.9% | 17 |
| 7 | Andy Dalton | CIN | 2.8 | 323 | 5 | 62 | 8.0 | 11 | 51.6% | 3 |
| 8 | Cam Newton | CAR | 2.7 | 107 | 24 | 68 | 7.9 | 13 | 33.8% | 30 |
| 9 | Andrew Luck | IND | 2.6 | 258 | 11 | 69 | 7.6 | 21 | 42.0% | 16 |
| 10 | Aaron Rodgers | GB | 2.3 | 166 | 15 | 58 | 7.7 | 17 | 36.2% | 24 |
| 11 | Tyrod Taylor | BUF | 2.3 | 144 | 19 | 43 | 8.5 | 3 | 44.2% | 13 |
| 12 | Jay Cutler | CHI | 2.1 | 288 | 8 | 59 | 6.7 | 32 | 49.2% | 6 |
| 13 | Joe Flacco | BAL | 1.9 | -75 | 30 | 79 | 7.0 | 28 | 34.2% | 29 |
| 14 | Brian Hoyer | HOU | 1.9 | 210 | 14 | 61 | 8.0 | 12 | 42.6% | 15 |
| 15 | Matt Ryan | ATL | 1.5 | 260 | 10 | 95 | 6.6 | 33 | 46.3% | 9 |
| 16 | Josh McCown | CLE | 1.4 | 570 | 1 | 70 | 7.2 | 25 | 54.3% | 1 |
| 17 | Drew Brees | NO | 1.2 | 357 | 4 | 97 | 7.8 | 14 | 47.4% | 7 |
| Rk | Quarterback | Team | ALEX | DYAR | Rk | Passes | Avg. Need | Rk | CONV% | Rk |
| 18 | Nick Foles | STL | 1.0 | -177 | 35 | 83 | 8.4 | 6 | 24.1% | 35 |
| 19 | Kirk Cousins | WAS | 0.8 | 317 | 6 | 93 | 7.2 | 23 | 47.3% | 8 |
| 20 | Teddy Bridgewater | MIN | 0.8 | 149 | 18 | 72 | 7.8 | 16 | 36.1% | 25 |
| 21 | Derek Carr | OAK | 0.8 | 240 | 12 | 80 | 7.4 | 22 | 45.0% | 12 |
| 22 | Peyton Manning | DEN | 0.8 | -97 | 33 | 91 | 8.1 | 8 | 35.2% | 27 |
| 23 | Colin Kaepernick | SF | 0.7 | 86 | 26 | 74 | 7.7 | 18 | 33.8% | 31 |
| 24 | Jameis Winston | TB | 0.5 | 13 | 29 | 80 | 8.4 | 5 | 40.0% | 20 |
| 25 | Philip Rivers | SD | 0.5 | 111 | 20 | 82 | 7.7 | 19 | 43.9% | 14 |
| 26 | Marcus Mariota | TEN | 0.3 | 111 | 22 | 52 | 7.8 | 15 | 40.4% | 19 |
| 27 | Ryan Mallett | HOU | 0.2 | -78 | 31 | 42 | 7.2 | 24 | 35.7% | 26 |
| 28 | Tom Brady | NE | 0.2 | 357 | 3 | 68 | 7.2 | 26 | 52.9% | 2 |
| 29 | Matt Cassel | DAL | 0.1 | 101 | 25 | 28 | 7.0 | 29 | 39.3% | 21 |
| 30 | Matthew Stafford | DET | -0.1 | -176 | 34 | 74 | 8.1 | 7 | 35.1% | 28 |
| 31 | Eli Manning | NYG | -0.2 | 262 | 9 | 90 | 6.9 | 31 | 45.6% | 10 |
| 32 | Ryan Tannehill | MIA | -0.3 | 44 | 28 | 69 | 8.6 | 2 | 29.0% | 33 |
| 33 | Sam Bradford | PHI | -0.6 | -78 | 32 | 82 | 8.4 | 4 | 24.4% | 34 |
| 34 | Brandon Weeden | DAL | -2.4 | 69 | 27 | 25 | 8.0 | 10 | 32.0% | 32 |
| 35 | Alex Smith | KC | -3.8 | 111 | 21 | 69 | 8.8 | 1 | 36.2% | 23 |
As requested, here is a look at the third-down splits by distance: short (1 to 3 yards), medium (4 to 7 yards) and long (8-plus yards). I added some color to give a bit of an index for where a player is well above average (darker green) versus below average (darker red). Players like Cam Newton and Tyrod Taylor have been more aggressive in the longer situations this season while Marcus Mariota and Derek Carr are more likely to be conservative there. Then you have Alex Smith, off playing his own sport again. Brandon Weeden hopefully won't be included in this much longer.
Comments
43 comments, Last at 08 Mar 2016, 7:28pm
#1 by greybeard // Nov 09, 2015 - 4:06pm
This is by far the most meaningless and useless statistics about football. #1 guy has less DYAR then #19 guy despite 7 more ALEX.
The last guy in the list has positive DYAR despite being 5 ALEX behind #13 guy which has negative DYAR.
Football is already a low sample size sports. On top of that it is also much more context sensitive than most other sports. Also 3rd down is mostly about who your best play maker outside the QB. Charles for KC and Brown for PIT. That shows.
#2 by David C // Nov 09, 2015 - 4:24pm
Correlations are rarely 1-1, but there is a correlation between ALEX and both conversion percentage and DVOA. Also, DYAR is a cumulative stat; ALEX is a noncumulative stat so should be compared with DVOA, another noncumulative stat.
#4 by greybeard // Nov 09, 2015 - 5:59pm
Of course correlations are rarely 1:1. What is the correlation to DVOA, % completion and DYAR to this stat?
Or put in another way, what is the value of this stat? Has Ben been be a better QB than Brady this year because he is 6 ALEX above. Are Bortles and Hoyer have been better QBs than Alex Smith and Eli Manning?
#6 by Thomas_beardown // Nov 09, 2015 - 6:33pm
It describes styles of play, not quality of play. Just like a stat that shows one receiver is a deep threat while another is a possession receiver. Both can be good or great or bad, but they're different types of players.
#14 by greybeard // Nov 09, 2015 - 8:11pm
Fair enough. I do not think Scott views it this way ("It's pretty much impossible to find a QB who is below average in ALEX and is still consistently successful in this league. You can't live on expectation-beating YAC.").
But whatever. I do not find there is anything to learn from this stat. I am glad you are getting better mileage out of it.
#9 by Jerry // Nov 09, 2015 - 6:46pm
ALEX tells you how far a quarterback throws the ball relative to the first down marker. Period.
If you don't find that interesting, you're free to ignore ALEX. If you want to know conversion percentage or DVOA, those are available. But complaining because this one piece of information doesn't correspond with how you think quarterbacks should be ranked is like complaining about a list of QBs ranked by their rushing DVOA. ALEX is just looking at a particular tree rather than trying to describe the forest.
#13 by greybeard // Nov 09, 2015 - 8:02pm
I did not complained about it because it does not fit my pre conceptions. I criticized it because it is a meaningless and useless metric. If you do not find my criticism of this metric interesting, you are free to ignore it.
#16 by Scott Kacsmar // Nov 09, 2015 - 8:39pm
You probably have no interest in air yards period, which is fine, but many of us happen to think it's very important to track where passes are thrown and to separate what happens after the catch point. Failed completions, REC +/- and YAC+ all play right into this. Does a QB have a lot of failed completions because he makes terrible ALEX decisions that aren't likely to work, or is it because his receivers don't get any YAC? We can use these stats to figure these things out.
The DYAR is there because someone is bound to ask for it. If people prefer DVOA, then we could do DVOA instead.
The stats, including ALEX, are ranked because it's easier to read a table that way. If I posted an average air yards table, you'd see guys like Manziel at the top and Brady near the bottom. That doesn't mean one is "better" than the other. It's just a stat that adds a lot of context to how the QB is playing this season.
And like with most stats, you learn the most from which players are at the top and bottom. And I'm telling you, consistently negative ALEX (throwing short of the sticks) is a poor third-down strategy, and it's most commonly used by bad NFL QBs. We have nearly a decade of data for this now.
#17 by greybeard // Nov 09, 2015 - 9:23pm
You are not talking about air yards period. You are talking about a subset of it, which is quite different.
And your statement "And I'm telling you, consistently negative ALEX (throwing short of the sticks) is a poor third-down strategy, and it's most commonly used by bad NFL QBs. We have nearly a decade of data for this now." Where is this data?
You have some data last week that suggest Michael Vick (twice), Sage Rosenfels and Matt Mcgloin and Drew Stanton leading the league in ALEX. Do you need more proof that it does not correlate to success?
Your poster child for this metric: Alex Smith has 111 DYAR on 3rd downs with 69 attempts. That is what Russell Wilson has for total DYAR (114) in 264 attempts, despite having 3.0 ALEX. 111 DYAR It would make Alex Smith 20th best QB in the league if he just handed off on 1st and 2nd downs and throw about 4 yards of shy of sticks every third down.
Your other poster child Weeden has 69 DYAR in 25 attempts. That is Bridgewater in 231 attempts, who happens to have positive ALEX.
#19 by Scott Kacsmar // Nov 10, 2015 - 1:11am
Where is this data?
Well it's not like I'm doing a book on this. Been slowly releasing more pieces to it. I put the list of the lowest ALEX QBs here: http://www.footballoutsiders.com/alex/2015/alex-week-8#comment-1007864
You have some data last week that suggest Michael Vick (twice), Sage Rosenfels and Matt Mcgloin and Drew Stanton leading the league in ALEX. Do you need more proof that it does not correlate to success?
All this does is show I probably need a higher minimum attempt qualifier for rankings. Just like how 2014 Derek Anderson was the last straw in using 100 attempts for DVOA/DYAR. As for Michael Vick, what's exactly wrong with his 2006 and 2010 seasons? A lot of people would say they are the finest seasons of his career (though I'm partial to 2002 over 2006).
Do you need to see all the guys like Aaron Rodgers, Andrew Luck, Ben Roethlisberger, Peyton Manning, Philip Rivers, etc. that rank so highly in ALEX then?
As for your last paragraph, I'm not sure DYAR is a good way to judge a QB's contributions to third-down success. Failed completions are going to be worth more than an incompletion, but failed completions are also shorter, easier throws. The guy who actually tries to throw for the conversion gets penalized there. The Chiefs have been a poor third-down offense this year, and Weeden sure wasn't helping Dallas, so I don't see how those are good examples at all of success.
#29 by greybeard // Nov 10, 2015 - 11:37am
So your data backs it but only you have access to it. We have to take your word for it I guess.
And why is DYAR not good? It adjusts for the opponent, which ALEX does not. It adjusts for the distance, which ALEX does not. It does not deal with averages, which is a bad way to look at data especially with small sample sizes - but Alex does. It is counting stat but you are including the number of attempts anyway, so easy to see the relationship.
Why ask me questions regarding the QBs highly rated for ALEX? You have the data. I do not. All I have is a lousy table from last week and your word for it.
#36 by Scott Kacsmar // Nov 10, 2015 - 5:35pm
How does ALEX not adjust for distance when that's half of the calculation? DYAR cares about results, not air yards.
I would have to dig into research on the defenses to see if there's any consistency there. Might find better luck on shorter situations, because I can't imagine any defense being consistently bad on third-and-long. Honestly, opponent adjustments may not be cut out for something like this. And they are such a tricky thing in the first place given all the changes and injuries that take place throughout a season. Look at how different the Chiefs-Broncos will look this week compared to Week 2. They won't be defending Jamaal Charles, and Smith won't be seeing Ware, Talib, Ray, etc.
And btw, 2008 Sage Rosenfels led all QBs in conversion rate when he led in ALEX. In fact, it was the 4th-highest conversion rate since 2006. He's the only QB not named Peyton, Brees, Romo, Palmer, Rivers, Roethlisberger or Rodgers to convert above 54% in a season.
#20 by Eleutheria // Nov 10, 2015 - 3:21am
But there's an important distinction between a deep throw on first/second down (where a short pass is normally preferable) then a deep throw on third and long (where a long pass is normally preferable).
It's an important distinction.
#21 by Eleutheria // Nov 10, 2015 - 3:31am
Correlation between ALEX and DYAR is 0.37, Correlation between ALEX and Conversions is 0.43
I'm not going to post the correlation between DYAR and 3rd down conversions, because 3rd down conversions are in the formula for DVOA, there's no surprise there's a correlation there. If Scott Kasmar could post DVOA on 1st and 2nd down only, then I'd look at the number.
#3 by nat // Nov 09, 2015 - 4:49pm
Whenever a "sticks" play design fails badly (via sack, scramble, or check down) it doesn't get counted in ALEX or worse gets counted as a "short" play. That's a fatal flaw in ALEX.
We're left with the stellar observation that those "sticks" plays that look open enough to draw a throw are better than the combination of all "short" plays that draw throws plus all "sticks" play check downs plus all "long" play check downs. Well, duh. Check downs are already failures, and just try to salvage a little field position and the occasional first down. And short throws - by design - tend to have their failures after the catch rather than before the throw.
And no type of play gets counted for the sacks and scrambles it generates. Hint: screens don't result in sacks very often. Nor do quick hitting short passes.
Until you come up with a way to at least give a reasonable estimate of those effects, ALEX remains a mostly useless stat that does not say what you claim it says.
#7 by Thomas_beardown // Nov 09, 2015 - 6:34pm
I wonder what it would look like if you assumed worst case scenario that every sack and QB rush was a failed stick throw. I expect it still shows throwing deep in a positive light because even on 3rd and long, sacks aren't *that* common.
#8 by nat // Nov 09, 2015 - 6:46pm
And check downs. And subtract them from the "short" plays...
I agree we don't know the answer. But we do know that ALEX is broken in a specific direction, and by a fair amount.
It will be interesting to see if Scott enthusiastically investigates this problem, or resists doing so. Any bets?
#11 by Thomas_beardown // Nov 09, 2015 - 7:07pm
Well there is a limit to work that can be put into a project. Checkdowns aren't marked in pbp so it would be a bit much to ask for that on a weekly basis.
Edit: I'm not sure removing checkdowns would be fair. I'm sure there are open or semi-open receivers on many plays where the QB decides to check down. Alex Smith isn't throwing 7 yards short of the first down because every receiver is within 3 yards of the LOS. A check down is a decision, it could be good, bad, or 50/50, but the point of ALEX is to track what decisions QBs/coaches are making on 3rd down.
#15 by Scott Kacsmar // Nov 09, 2015 - 8:29pm
Alex Smith and Aaron Rodgers are two of the most sacked QBs on 3rd down, yet they are polar opposites when it comes to ALEX. Some of the most productive checkdowns come from guys who hold onto the ball a long time and make something crazy happen to free up a receiver like Roethlisberger, Wilson and Luck have done. That can definitely be a throw to a short receiver, who may have left his blocking assignment. Sacks can happen for many reasons, and we don't know when the intent was clearly for a short throw or a long one, since so many (good) pass plays are designed to include both route depths. It's a real stretch to say any sack/scramble is equal to a Stick throw. Huge stretch.
What nat wants is basically impossible without going back, re-watching every single play and trying to account for some of these things. We don't chart plays as checkdowns. We have "dump passes" which is similar, but not inclusive. We don't chart all the routes run on a pass.
Fact is most screens on third-and-long are not going to convert for a first down. They're give-up plays, in the same way Mike Holmgren used to run draws to Mack Strong in Seattle.
Somehow I'm not surprised fans of the Chiefs and Patriots make up the anti-ALEX crowd. Some KC fans refuse to see how Smith limits their offense, while NE fans often overrate a QB's impact on YAC.
#23 by nat // Nov 10, 2015 - 7:38am
The fact that it is hard to extract screens or check downs from the play by play doesn't change my point at all. Unless you do, ALEX is always going to be broken badly. But you could look with an open mind and try to learn. Will you? It's a reasonable request, motivated by a desire to improve the stats.
You say screens are give up plays. But you do not have data to back that up. It's just your agenda-driven bias. It could also be that they are an effective part of a diverse strategy. Why not look?
This isn't about the Chiefs and Patriots. It about bad, agenda-driven stats and analysis. FO is supposed to be a cure for that. Now (here) it's become the disease.
By the way, if your charting includes "dump offs" it probably includes "screens" as a category, too. You could at least look at those as a set. Are they as bad as all other "short" passes? And how about those dump offs? And then you could look at non-screen, non-dump off passes as a set. Those would be a mix of first and second reads, designed plays mostly. It wouldn't be perfect, but you'd start to get an idea of whether intentionally going short was as bad as you say, or whether it was not seeing anyone open, giving up during the play and dumping the ball off was the issue.
#37 by Scott Kacsmar // Nov 10, 2015 - 5:40pm
Broken in your opinion, which I have no reason to value over the thoughts on ALEX from people involved in this business. No one's content is more agenda-driven here than your comments.
When it's third and really long, even Phil Simms knows a screen is a conservative call just like a draw is. In fact, he says it almost every week in those situations. "Expect a screen or draw." I think every week our lowest ALEX play has been a team just doing a screen in that situation. There's a big difference between third-and-short and third-and-10+. The stats get that even if you don't.
The charting includes screens, but the charting is a separate file from this.
#40 by nat // Nov 11, 2015 - 8:00am
But you can easily get success rate or conversion rate for screens, with down and distance. That's the point of the charting data, isn't it? We don't really need ALEX values for screens, because they are always close to the same air yards by definition.
So, what's the conversion rate with 1-3 yards to go for screens? 4-7? 8 or more?
How does that compare to all thrown passes? All pass plays including sacks and scrambles?
#18 by eagle97a // Nov 09, 2015 - 11:44pm
As usual QB metrics that are another version of the chicken or egg question; which came first or to put it in another way to which do we attribute a play the qb or the receiver? Which begs the question ,where is the attribution for OL and the other skill positions (and the coaching)? It is interesting in the sense of play design and general offensive philosophy but as in any football question we cannot assign definite weights to position contributions. Another metric to add to the growing number of questions we have about qb play.
#22 by Moridin // Nov 10, 2015 - 3:49am
Well, I like ALEX and I find the info an interesting supplement (don't want you to only hear complaints after all).
I do agree that DYAR isn't quite as appropriate as DVOA for these. Also, that last table is really a much better basis for presenting the data. As a combined table, showing % of occurrence of their split totals & the DVOA and/or conversion % for each split, it would allow a good picture to develop at how well the QBs are doing with their decisions in comparison, allowing for talent and offense design of course. I love being able to see, for instance, that Bridgewater on 3rd and short is barely averaging at the sticks, but on medium and long he's probably throwing to the same spots. But it would really feed the analysis to know how well he's doing on those splits by conversion (without knowing how often one is in s/m/l distances and how often their chosen throws are working out or not, hard to tell how they are succeeding/failing).
For examples, Tom Brady is rather low in ALEX (in fact, he really seems to average throwing about the same distance regardless of 3rd down distance) and yet has the 2nd best conversion rate. Palmer and Roeth are overthrowing the sticks usually, and yet have high conversions rates, where Manziel and Bortles overthrow the sticks as well but have much lower conv rates.
Not really for you of course, Scott, but can FO take the plunge and get a table script/package that makes the column headers sort? Just seems weird that in all this time, the bread and butter presentations of FO are still the most basic static HTML tables.
#27 by Eleutheria // Nov 10, 2015 - 10:35am
There is a medium correlation (0.43) between ALEX and 3rd down conversion.
Obviously Tom Brady is going to be better then Blake Bortles on 3rd downs, he's a much better QB. But the main point of this is unless you're an old QB who's clearly lost some arm strength (Tom Brady has completed just one 20+ yard throws this season) you're better off being aggressive on 3rd down.
#42 by Eleutheria // Nov 13, 2015 - 11:01am
So I ran a multivariable regression, correlation between ALEX+Yards to go with Conv% is 0.5927 and has a P value of 0.009 (aka statistically significant).
Edit: realized I had a typo in my formula, fixed the numbers. Effect size of Yards is twice the effect size of ALEX, not the 9 times that I had initially found.
#25 by nat // Nov 10, 2015 - 9:59am
The splits are interesting.
I wonder, what kind of ALEX ranks do the 5 best QBs at converting by pass at each distance have?
I used the PFR play finder, so I did teams, not QBs. Cleveland is listed with two values. I used 2015 to match the splits table.
1-3 yards needed: 18, 17or28, 16, 3, 11 (average 14th, median 16)
4-7 yards needed: 25, 33, 4, 26, 15 (average 21st, median 25)
8+ yards needed: 8, 17, 100r16, 24, 7 (average 14th, median 10or16))
With that very limited data set, it looks like a near average or lower ALEX strategy is the way to go to the head of the class. But mostly it looks like ALEX doesn't mean much.
My personal belief is that a strategy that attacks the whole field (screens, short passes, throws to the sticks and occasional deep passes) keeps the defense honest and works well with a decent QB. That's likely to be a below average ALEX, because conventional wisdom is to throw beyond the sticks as much as possible on third down. But it seems to work.
#26 by nat // Nov 10, 2015 - 10:16am
A quick follow up:
It looks just as bad for ALEX if you ask how successful the top 5 ALEX guys are at each distance. They just don't rank very well at getting first downs. They're kinda average as a group at each distance.
What we can't know from this data is whether the problem with high ALEX value QBs is not enough balance in play calls or poor post-snap decision making by some QBs.
Or maybe it's just that ALEX doesn't mean much.
#28 by Eleutheria // Nov 10, 2015 - 10:37am
sorry NAT, you're just looking at the average and best, without looking at what's in between.
The correlation between ALEX and conversion% is 0.43 and is statistically significant. High ALEX improves a QBs effectiveness on 3rd downs.
#30 by nat // Nov 10, 2015 - 11:50am
You're confusing cause and effect, and using correlation a bit naively, too.
Lower ALEX does not cause QBs to be worse on third down. Being bad causes their ALEX to be lower on third down. Remember those issues of sacks, scrambles, and check downs not being counted or being counted in the wrong group? Those issues affect bad QBs a lot more, because good QBs avoid those results. That's a problem for ALEX, because it's not accurately measuring what Scott claims it measures. And it creates the misleading correlation that you saw.
So what does the data say about average or better QBs? If we look at the top 16 QBs (by DYAR) what is the correlation between ALEX and Conversion %? It's very low: 0.056. Among average or better QBs, ALEX tells us essentially nothing about their third down effectiveness.
You might think that would be because they all use the same high ALEX strategy. You'd be wrong. They range from the highest ALEX score to 28th and 31st out of 35. The way to be good on third down is to be a good QB. There's no magic about air yards on third down for these QBs. The trick for good QBs is to have smart, results-oriented play calls and decisions, not necessarily calls and decisions biased towards more air yards.
On the other hand, there is some correlation between ALEX and Conversion % for the rest of the league, 0.324. It looks like once you're not a good QB, the worse you get, the less your passes are thrown longer distances. Why? You're getting sacked more, flushed out of the pocket more, or throwing check downs far too often to get good results. Even your play calling might be affected by your inability to go down field, so defenses can cheat closer to the line to stuff your short passes, too. No real surprise there, is there?
#39 by eagle97a // Nov 10, 2015 - 9:41pm
Agree with this observation but obviously ALEX will be heavily influenced by particular game plans and specific matchups and with the sample sizes among the different data sets the results wont be smooth and chunky I suppose.
#43 by Eleutheria // Mar 08, 2016 - 7:28pm
Just had an argument with someone who thinks that situation is irrelevant and argued that an 8 yard completion on 3rd and 15 is preferable to a 2 yard completion on 3rd and 1. (more yards per attempt)
This is why ALEX is necessary, please don't understand the importance of playing the situation.
I shared the above graph to him, hoping it changes his view.