Writers of Pro Football Prospectus 2008

Most Recent FO Features

TomlniMik07.jpg

» Clutch Encounters: Week 4

Blowout week, but not for the Steelers. Do they play down to the competition? Also: bad Foles, Bridgewater's debut, and did J.J. Watt just end EJ Manuel's career in Buffalo?

24 Jun 2010

Counteracting the Draw Play

by Aaron Schatz

Today we continue rolling out some numbers from the 2009 game charting project, hopefully giving everybody something interesting to discuss while we're busy finishing FOA 2010 and the KUBIAK fantasy projections.

Last week, I posted stats for which offenses were most efficient running draw plays. Today, we'll look at defense on those same plays. To remind readers, game charters are generally told to mark a draw play on a run where the offensive linemen take their first step backwards to pass block rather than firing out to run block. A run from shotgun is not necessarily a draw play.

It will surprise nobody that Indianapolis faced more draws than any other defense -- it's a play designed to take advantage of a team with fast defensive ends who are always thinking pass rush. However, draws against the Colts didn't seem to be any more effective than other runs. And draws were actually a bad strategy choice against most of the other defenses known for heavy pass rush and poor run defense. Cincinnati had the league's best DVOA against draws, and San Diego, Denver, and New Orleans all did better against draw plays than against runs in general.

Baltimore was the team with the biggest difference between total run defense and defense against draws; but lest anyone think we've found a good strategy against the Ravens, beware of small sample size -- we're only talking 20 plays here. If you want defenses who struggled against draws based on more reasonable samples, we've got San Francisco, Philadelphia, Chicago, and Houston.

Defense Draws RB Runs Draws as
Pct of Runs
Yds Rank DVOA
on Draws
Rank DVOA
All Runs
Rank
CIN 23 345 7% 3.3 2 -62.1% 1 0.3% 21
PIT 19 321 6% 3.4 3 -46.3% 2 -11.1% 9
MIN 31 306 10% 3.2 1 -45.4% 3 -17.9% 1
MIA 25 387 6% 4.7 15 -34.0% 4 5.0% 24
SD 32 365 9% 3.6 4 -31.7% 5 4.4% 23
DEN 33 419 8% 4.1 5 -16.3% 6 -1.5% 18
JAC 29 397 7% 6.9 29 -15.0% 7 -12.0% 7
SEA 32 394 8% 4.2 6 -14.9% 8 -8.9% 10
GB 27 333 8% 5.2 18 -12.6% 9 -14.4% 5
NYJ 20 373 5% 4.7 14 -11.7% 10 -12.0% 8
ATL 31 377 8% 5.8 25 -7.6% 11 -16.1% 2
NO 24 395 6% 4.3 7 -6.9% 12 7.7% 29
KC 25 464 5% 4.8 16 -3.1% 13 10.7% 31
IND 59 425 14% 5.2 19 1.0% 14 0.2% 20
ARI 23 352 7% 6.6 28 1.1% 15 -4.6% 14
WAS 29 385 8% 4.7 12 1.9% 16 -14.3% 6
Defense Draws RB Runs Draws as
Pct of Runs
Yds Rank DVOA
on Draws
Rank DVOA
All Runs
Rank
TB 35 474 7% 7.3 30 2.6% 17 7.6% 28
TEN 44 349 13% 4.4 10 2.8% 18 5.8% 26
DAL 25 324 8% 5.2 21 3.5% 19 -8.2% 11
SF 38 363 10% 5.1 17 5.6% 20 -14.5% 4
CLE 21 447 5% 4.4 9 6.4% 21 8.4% 30
NE 31 361 9% 6.1 27 13.4% 22 -7.8% 12
DET 26 406 6% 4.6 11 13.4% 23 -1.7% 17
BUF 32 471 7% 4.7 13 15.9% 24 6.8% 27
NYG 36 376 10% 5.3 22 16.0% 25 0.9% 22
CAR 46 398 12% 5.2 20 21.7% 26 0.1% 19
PHI 42 367 11% 5.8 24 22.7% 27 -7.6% 13
BAL 20 376 5% 4.4 8 25.9% 28 -14.7% 3
CHI 30 408 7% 5.7 23 27.9% 29 -2.9% 15
HOU 42 334 13% 7.6 32 36.8% 30 -2.9% 16
STL 26 448 6% 5.9 26 50.4% 31 10.8% 32
OAK 33 485 7% 7.4 31 50.9% 32 5.1% 25
NFL 31 388 8% 5.2 2.1% -2.5%

Posted by: Aaron Schatz on 24 Jun 2010

21 comments, Last at 01 Jul 2010, 11:18am by jimbohead

Comments

1
by BGNoMore (not verified) :: Thu, 06/24/2010 - 9:27pm

I don't see the point in this. Fifteen teams have a sample size of 30 or fewer, rendering them automatically statistically irrelevant. And while I don't have the raw data to conduct the relevant tests, my best guess is that none of the teams' data surpass even the loosest reasonable definition of statistically relevant. Just look at the sample sizes. I'm sorry, but this is just a noisy waste of time.

2
by BigDerf :: Thu, 06/24/2010 - 10:50pm

And because it doesn't fit your definition of statistically relevant then we can draw no insight at all from the data? Not a single thing? We can't throw out the 15 teams and just look at the teams with solid sample sizes? We can't be slightly surprised that Indy isn't horrible on draw plays? (It did stand to reason if you didn't see them much this year) And we can't be thankful that we are getting a free look at the fruits of hours and hours and hours of work?

3
by PerlStalker :: Thu, 06/24/2010 - 10:52pm

Sample size is always an issue in Football compared to sports such as Baseball, Basketball or Hockey due to the significantly smaller number of games played. That doesn't mean that we shouldn't look at what data there is simply because there's isn't much of it.

DVOA does a reasonable job of dealing with small sample sizes but there will always be times, like this, where there isn't as much data as you would like. So, rather than giving up on the whole thing, let's make do with what we've got, eh?

4
by Karl Cuba :: Fri, 06/25/2010 - 9:20am

Well I've had a look at this data a few times and I can't see any trends worth a damn. I looked to see if 3-4 or 4-3 teams were more susceptible to draws and I se no trend.

Why, for example, are the 49ers so bad against draws when their conventional run defense is very good? All can think of is that they use very different personnel on passing downs, taking Franklin, Sopoaga, Lawson and Spikes off the field and adding Mark Roman (shudder) amongst others. To really get into the numbers you'd need to include those sorts of factors for all teams.

It does seem to be a rather noisy data set though.

5
by Mr Shush :: Fri, 06/25/2010 - 9:34am

No surprise to me that the Texans were bad on draws - and it looks like teams may have picked this up on film and run more draws than usual against them. There also seems to be a pretty reasonable orthodox explanation: the Texans defense is one of the youngest units in football, and their system is pretty aggressive in a lot of ways, with linemen encouraged to shoot gaps instead of tying up blockers, aggressive, run-first linebackers at all three spots and a strong safety in Pollard who is very much a downhill player and generally lines up in the box, coupled with a secondary who (Pollard aside) don't play the run very well*. That certainly meshes with my subjective impression from watching the team.

*Dunta Robinson has historically been very good in run support, but he certainly wasn't in 2009, whether due to lingering (perhaps psychological) after-effects of his injury, or as a response to his unhappiness with his contract situation.

6
by Rich Conley (not verified) :: Fri, 06/25/2010 - 10:20am

"That doesn't mean that we shouldn't look at what data there is simply because there's isn't much of it."

No, it doesn't mean we shouldn't look at it, but in a lot of cases, it means that the data doesn't tell us anything.

9
by Karl Cuba :: Fri, 06/25/2010 - 11:32am

Maybe you should spend your time looking at one of the other sites that provide statistics on draw plays compared with all other running plays across the league, for FREE.

Why not try this one:

www.inyourf**kingdreams.com

15
by Bobman :: Fri, 06/25/2010 - 9:49pm

I keep trying that site, maybe six or seven times now, and frankly, it sucks. A total waste of my time. Last night for example, my wife had this skimpy outfit on and I said "Not tonight, dear, I'm on-line." Four, five hours later--I'm pounding espressos to make sure I stay awake not don;t miss anything--I'm not sure if the URL is bad or the server was down or what, but it was like staring at a blank screen.

Oh... wait....

Long story short, I agree with you.

7
by Aaron Schatz :: Fri, 06/25/2010 - 10:38am

We're going to lock the "this article is pointless" people in a room with the "why doesn't FO run any free articles anymore" people and just let you fight it out. :)

8
by jimbohead :: Fri, 06/25/2010 - 11:14am

popcorn available only to ESPN subscribers.

almost makes me want to spend the $5...

12
by Jeff M. (not verified) :: Fri, 06/25/2010 - 2:45pm

Maybe lock them in a room to take a statistical inference class until they stop making assertions like "sample size smaller than 30 means automatically statistically irrelevant"...

13
by Scott C :: Fri, 06/25/2010 - 6:36pm

Yeah, there is actually a lot that can be said here. Sure, the error bars are huge on lots of individual teams, but aggregate conclusions can be made. For example, 3-4 and 4-3 defenses don't seem to be statistically any different versus draw plays.

And even for teams with low sample sizes, the range of the error bars for draw vs non-draw run plays tells us something. Perhaps one can say with 80% confidence that low sample team X is actually worse against the draw than normal runs. not a 99% confidence but its still something and not 'useless'.

10
by Phill (not verified) :: Fri, 06/25/2010 - 11:40am

Assuming everyone is utilising rational game-playing strategies (which they aren't, but let's let that slide) - and ignoring sample size effects (hah!) - you might expect that draw plays and non-draw runs would have the same DVOA. A team that is selling out on the pass defense will be susceptible to draws, so the offense run more draw plays, so the defense has to play with awareness of the draw play, improving their success vs the draw and worsening the success vs the pass via a weaker pass rush. In the the large number limit (and add other caveats here) all offensive options have the same chance of success.

While obviously that isn't going to apply in the real world of a limited number of plays per game, limited games per season, and a host of other factors coming in to play calling apart from aiming for Pareto efficiency, it has to have some kind of effect. Teams that are found to be very vulnerable to the draw will be a) exploited and b) adapt. Large differences between the DVOAs for draws and normal runs will tend to be self-limiting.

Which is all ultimately an argument that in the limit, you'd expect the numbers to be dominated by statistical noise.

It is always worth looking at though, simply because you might well find some strange outliers. But the proof of the pudding is in watching the actual plays, rather than the stats. You can see more clearly if a team is correcting a vulnerability to draws on tape than by analysing the stats. The stats however might point you at the teams most worth looking at.

14
by Scott C :: Fri, 06/25/2010 - 6:40pm

Absolutely -- with sample sizes this small, looking at the plays is much more informative. You can watch 30 draw plays and easily say "Team Y was victimized by draws in unusual situations (1st down) but otherwise appeared typical". or "Wary veteran LB's on team Z sniffed out draws consistently".

16
by Aaron Schatz :: Sat, 06/26/2010 - 12:11pm

Man, I would love to eventually have some sort of deal with the league, either direct or through our ESPN relationship, that allowed us to actually show video of plays to either a) help explain or explore interesting stats or b) help illustrate concepts from Walkthrough and Cover-3.

Believe me, it's on the long-term radar.

17
by Bobman :: Mon, 06/28/2010 - 2:59am

Way, way, WAY, w-a-y totally f-ing cool.

With all the play diagrams you have here, how cool would it be to have the diagram and the actual clip side by side? As I noted above, way cool.

18
by ChaosOnion (not verified) :: Mon, 06/28/2010 - 11:27am

I would totally pay money for that.

11
by Joseph :: Fri, 06/25/2010 - 12:31pm

Can't speak for other teams, but I can give you my $.02 about the Saints. They were horrible at run D, but when teams had to pass to catch up, they couldn't use regular runs. A common situation for a draw is 3rd & medium, where a decent run could pick up the first down. The Saints like to blitz a lot, even on 3rd down. It's hard for that RB to get free when he's supposed to be blocking the free blitzer, who is now the free tackler. Also, in general, isn't half the point of the blitz to make the QB get rid of the ball before he wants to, making him throw the ball to an outlet receiver or one running a short route (i.e., short of the sticks)? That means extra DB's can play short routes and can tackle the RB who gets out of the backfield before he picks up the first down.

19
by puscifer222 (not verified) :: Wed, 06/30/2010 - 10:14am

Just wondering with regards to the 3-4 vs. 4-3 angle. Is it safe to say that more than not, the 3-4 defenses align more like a typical 4-3 (putting OLB's on the line to rush the passer) on passing downs/situations?

20
by tuluse :: Wed, 06/30/2010 - 6:48pm

No, it's not.

Pittsburgh for instance loves to run the 11 angry men defense, where they have everyone moving around before the snap so the offense has no idea what's going on.

21
by jimbohead :: Thu, 07/01/2010 - 11:18am

If by aligning like a 4-3, you mean running a nickel package with 4 down lineman, then occasionally (my understanding is 3-4 hybrid teams like the 9ers a couple years ago, and the pats now, tend to do this). If a 3-4 team is playing passing downs in its base defense, then, even if they have a particular OLB that always rushes in those situations, the splits will be different (DEs directly over tackles, WOLB split wide of the LT for instance).

Also, nickel packages vary from team to team. For instance, you could simply replace a MLB with a DB, still have both your OLBs line up wide of the tackles 2-4 yds deep, and run some sort of 3-3, with your extra DB dropping into the middle of a cover 3. There are a million variations, and 3-4 defenses tend to be highly variable. Furthermore, not every 3-4 team uses nickel consistently in passing situations, as Tuluse notes.