ESPN: The Red Zone Efficiency Myth
This week's MNF feature looks at the question of red-zone efficiency, where the Kansas City Chiefs are one of the best offenses in the league, and the San Diego Chargers are one of the worst defenses. But does that really mean anything? We've already done research showing that there's very little correlation from year to year in the difference between overall efficiency and red-zone efficiency. Is the same true in season? Trends from the last five seasons suggests it is.
That means the idea of "red-zone efficiency" is essentially a myth. What matters is simply how good an offense or defense is overall. This has some interesting ramifications we'll need to explore in the coming months. Does this mean there's no point in comparing each team's red-zone efficiency when looking at who might win a single game? Would it make sense to change DVOA so that red-zone plays no longer are more important than other players? I also need to analyze drive chart stats to see if the more traditional definition of "red-zone efficiency" (i.e. touchdowns per drive) is more consistent from year to year, or from first half of the season to second half of the season, than red-zone DVOA is. All this is worth exploring.
Comments
18 comments, Last at 02 Nov 2011, 7:39pm
#1 by Jim Glass (not verified) // Oct 31, 2011 - 1:23am
Myth it is. There's a double fallacy in the numbers constantly cited by the TV commentators and reporters.
First, there is little to it in substance. (Good/bad Os are good/bad Os all over the field, what magic is there inside the 20 to reverse them?)
Second, to the modest extent there may be something to it, small sample size gives the cited numbers a bloated margin of error. Only a small percentage of plays are in the red zone -- and after 4, 6, 8 games, the number of plays is absurdly small to accurately detect a true small difference, or even give an accurate general reading.
Hey, this past week the Jets were rated #2 overall by you, with an offense ranked #29 by drive stats but which, you said, gets a big boost because of its top red-zone performance.
Watch Sanchez passing and Greene with his 3.8 running ... there's the argument from your own lyin' eyes that red zone play deserves demotion in the DVOA formula. At least as a predictive indicator (if not a backward looking explanatory one).
Test: Check back with the Jets after game #14, see where the O ranks in red zone play during their second-7 games of the season, compared to their first 7.
And I write as a Jets fan.
#5 by PackersRS (not verified) // Oct 31, 2011 - 4:51pm
The magic of a shortened field to work with. The Packers' D is so bad in yardage because they give up so many big plays. But inside the 20 receivers can't beat safeties deep. That's why they're 28th in yards per game but 9th in points per game. Also helps that they're 3rd in turnovers...
#6 by PackersRS (not verified) // Oct 31, 2011 - 4:55pm
Just to illustrate, they're 3rd in 20+ passing plays given and tied for 4th in 40+ passing plays given. But 7th in redzone scoring percentage.
#9 by Jim Glass (not verified) // Oct 31, 2011 - 8:15pm
"(Good/bad Os are good/bad Os all over the field, what magic is there inside the 20 to reverse them?)"
The magic of a shortened field to work with.
Yes, a nice familiar popular theory. Too bad it doesn't fit the facts.
Last year the big passing Packers offense was tied at #3 in red zone scoring rate (63%) -- because the short field took away its ability to score long?
Last year's Jets with their big rushing of 150 yards/game but Sanchez at QB sure don't look like they were hampered much by sudden inabilty to throw long from inside the 20, but were at the bottom in red zone efficiency, 44%.
This year's Jets have the same O, except worse -- the O-line and running game have collapsed (3.7 from 4.4 y/carry, 90 yds/game) -- but they have rocketed up to the very top in red zone efficiency.
"Short field inside the 20" explains all this ... how?
How small sample size can explain it is pretty obvious.
#11 by ppabich // Nov 01, 2011 - 4:08am
What I think the point is, is that the "20 yard red zone" is arbitrary. There is not a significant difference in how you call plays just beyond and before the 20 yard line. But, the closer you get to the end zone, clearly, the playbook is different. It becomes more difficult for an offense to operate, simply there is less space to work with.
Last year the big passing Packers offense was tied at #3 in red zone scoring rate (63%) -- because the short field took away its ability to score long?
The problem with that statement is, the reason why the Packers are good in the red zone is they have a QB that doesn't make mistakes (One red zone INT in his entire career) and can make exceptional throws.
You also use the sample size argument to disprove, red zone effectiveness. Shouldn't small samples not prove anything either way?
It is 100% true that a good offense, will be better than a bad offense in the red zone. The Jets and Chiefs will regress. But the fact is the game changes near the red zone, and certain teams, like Philly offensively, won't improve dramatically. They are always average to poor in the red zone, and don't have the personnel to be overly successful in that area. Red zone performance isn't necessarily correlated to future success, but it is an important part of the game of football. The margin for error is much smaller. (which could be why the correlation coeficiant is so low)
#15 by Jim Glass (not verified) // Nov 02, 2011 - 12:47am
The problem with that statement is, the reason why the Packers are good in the red zone is they have a QB that doesn't make mistakes (One red zone INT in his entire career) and can make exceptional throws.
Which is the same reason their offense is good outside the red zone. Again, no difference.
Note -- the claim was that a good passing offense like this becomes *less* effective in the red zone. But now, not? This makes the whole "explanation" of red zone play a just-so story. When the facts contradict the story, the story changes.
You also use the sample size argument to disprove, red zone effectiveness. Shouldn't small samples not prove anything either way?
I do not cite small sample size to disprove red zone effectiveness.
I say, first, the data *contradicts* the claims of red zone effectiveness. Then, to quote myself:
"Second, to the modest extent there may be something to it, small sample size gives the cited numbers a bloated margin of error. Only a small percentage of plays are in the red zone -- and after 4, 6, 8 games, the number of plays is absurdly small to accurately detect a true small difference, or even give an accurate general reading."
That is, the sample size is too small to support the claim, even if it were true. "Can't support" is very different from "contradict".
In plain English this just means that when the talking heads during the game go one about "this team is #8 in red zone offense against a team that is #14 in red zone defense" they are blathering about statistical meaninglessness.
But if one wants it, here's a more detailed statistical examination of play inside and outside the red zone -- all hail the star red zone QB: Trent Edwards!
Red zone performance isn't necessarily correlated to future success
Which demonstrates its meaninglessness. If it doesn't contribute to success in the future, it is *not* a skill. See: Trent Edwards.
Look, recovering 5 fumbles in a game last week was very very important, and explains why you won that game in the past. It doesn't mean you have a skill at recovering fumbles that going forward will win you future games. It is the same with red zone play.
#17 by PackersRS (not verified) // Nov 02, 2011 - 7:10pm
There isn't a difference when it's Rodgers throwing, when he can fit into really tight windows.
When it's Sanchez throwing the ball... Receivers don't get as open in the redzone as they do with bigger spaces.
#18 by Thomas_beardown // Nov 02, 2011 - 7:39pm
I think what he was trying to say is that Rodgers is so good he overcomes the added difficulty of red zone play.
#16 by PackersRS (not verified) // Nov 02, 2011 - 7:07pm
So because the Packers were able to create big plays consistantly they wouldn't be able to score in short yardage? And because the Jets were able to run well it necessarily means they should score in short yardage?
With that kind of logic you reach the conclusion that short yardage doesn't have anything to do with redzone efficiency?
So you're completely disregarding playcalling? Nevermind that the Packers had a FB last year (Kuhn) that was a monster converting 3rd downs and making touchdowns. It doesn't matter that the Packers had a QB that had the ability to run, and that said QB was a monster in the redzone, throwing 60+ TDs and just 1 INT in his career.
Nevermind that with a shortened field and without the treat of the pass, particularly with safeties not having to worry about Santonio Holmes getting behind them, that opponents would focus on the run when playing the Jets in the redzone.
#2 by Kal // Oct 31, 2011 - 2:47am
I do think some teams are consistently going to be better than others based on scheme in the red zone, both offense and defense. Teams that use zone, for instance, I suspect are better in the red zone than they are in other zones due to the lack of space for those zone plays; they effectively have more space to cover. Similarly, I think teams that rely on stretching the field as a primary offensive weapon are going to suffer in the red zone compared to teams that are better at short passes and running the ball.
But DVOA doesn't measure scheme, so that's tougher.
Mostly, I think it's especially silly because it's a percentage-based number when you really care about number of actual times. If a team converts 100% of their red zone attempts for TDs but only gets to the red zone once a game, what does that matter? (that is basically the Jets in a nutshell). Drive efficiency is probably more predictive and more useful of a way to measure a team's strength, especially when you factor in things like average starting position gained.
#4 by zenbitz // Oct 31, 2011 - 4:01pm
your first point seems to be the lead to explain any observed RZ effect. So you would kind of expect some kind of effect - although perhaps it washes out if you efficiency metric is scheme agnostic.
However - lets assume there there IS no effect. Maybe the deal is that 20 yards is not really that short a distance for scheme to matter. A 20 yard bomb or sprint is not really that much different from a 40 yarder. Maybe zone/short scheme matters more inside the 10 or the 5. However, halving or quartering your sample size doesn't make things any easier.
I guess you could code "scheme" as some kind of varianceDVOA/DVOA. With high variance= boom or bust offense and low variance = grind it out. No distinction between run/pass here, it's the 21st century. (Although something like 2*(passDVOA-runDVOA)/DVOA might make some sense too.
#3 by Joseph // Oct 31, 2011 - 2:13pm
Aaron, I'm going to suggest something, and let you run with the idea to see where the better fit is.
My idea, based on your intro, is: Less "bonus" for positive red-zone plays (except for the TD), and heavier penalties (than right now) for RZ failures. IMO, this would hurt teams with bad O's (less chance to accumulate bonuses and negative points for FG's when they get close without scoring the TD) and help good offenses, who theoretically should score more TD's and have more RZ opportunities. This might explain DVOA's long-term affair with PHI and this year's fling with NYJ.
A team who gets to the RZ prob will prob. accumulate enough "positive points" to overcome any negative RZ penalties for a non-TD outcome, whereas recovering a fumble on the opp. 15 and having a drive of "4 plays, -3 yds, FG" will be a overall negative (as it should be--you should be able to at least get inside the 10, and prob. score the TD).
#7 by Julian Simington (not verified) // Oct 31, 2011 - 5:44pm
interesting thing is that while i agree overall,i think many would agree that certain skillsets are more valuable in the redzone. for example amongst receivers, guys who can get separation on short routes are going to be more valuable than burners who stretch the field. prob don't see anything like this with massive impact b/c teams do a pretty good job of balancing personnel, but it merits consideration
#8 by LionInAZ (not verified) // Oct 31, 2011 - 6:57pm
I'm going to be simplistic about this question.
If you're not efficient in the red zone, then you're depending a lot on big plays to score. That makes a team much more susceptible to being shut out of the end zone by a good defense. It's a dangerous game to depend a lot on big plays unless you can score early and often.
#10 by zenbitz // Oct 31, 2011 - 11:26pm
no you can also be "dependent" on getting to the red zone more often.
#12 by nat // Nov 01, 2011 - 9:21am
Aaron,
By all means revisit the "red-zone bonus" in DVOA. You've shown that red zone performance is not more predictive of future red zone performance than overall performance. And giving the red zone extra weight is effectively reducing your sample size, which hurts most of your analysis, and has the worst effect on single-game analysis.
DVOA is interesting in part because it can show us how and why per-play efficiency leads to victory, and when and why it sometimes doesn't. The red zone bonus interferes with that.
#13 by Eggwasp (not verified) // Nov 01, 2011 - 10:20am
Is it really more difficult to advance the ball in the red zone? Do more drives stall here than anywhere else on the field? And if they do - is some of that merely conservative playcalling (given the potential FG attempt) rather than a change in genuine drop in efficiency for individual playcalls?
Does anyone know of any research on this?
#14 by Nathan // Nov 01, 2011 - 12:26pm
Anecdotally, spread-based offenses seem suffer in the red zone because there is so much less field to spread. This is mitigated by having a mobile QB.