In this week's Varsity Numbers, Bill Connelly takes a page out of baseball's playbook and attempts to isolate power from efficiency.
13 Oct 2005
by Mike Tanier
The 2001 Arizona Cardinals had a very poor defense. They allowed 5,695 offensive yards, the fourth worst figure in the league. They allowed 322 offensive points, ninth worst in the league. Their defensive DVOA of 13.7% was third worst in the NFL (click here for a DVOA explanation).
The middle linebacker for that Cardinals team, Ronald McKinnon, finished sixth in the NFL with 98 tackles. Safety Kwame Lassiter finished second among defensive backs with 83 tackles. Two other linebackers, Rob Fredrickson and Raynoch Thompson, finished among the top 50 linebackers in tackles. In all, Cardinals linebackers recorded 259 tackles, the third highest total in the NFL, while the secondary racked up 343 tackles, second-highest in the NFL.
So how did a lousy defense rack up so many tackles? It turns out that the 2001 Cardinals are just a typical example of the rule, not the exception. Bad teams generally have very good raw tackle stats. Good teams, especially teams with good defenses, produce low tackle totals.
In 2004, the Colts led the NFL in tackles, with 788. They were followed by the Raiders, Seahawks, Saints and Giants. All were below the league average in DVOA, with the Raiders and Saints among the worst defensive teams in the NFL. Three of the best teams in the NFL according to defensive DVOA, the Bills, Broncos, and Steelers, finished 29th, 31st, and 32nd in tackles. The Dolphins, another good defense, ranked 30th. The Redskins ranked 26th.
If DVOA isn't your bag, try a stat like Offensive Points allowed. The results are the same: the top five teams in tackles all allowed an average of over 21 offensive points per game. The four teams at the bottom averaged 18 or fewer points per game. Or just look at the team names: you don't need stats to tell you that the Raiders had a bad defense and the Bills and Steelers had great defenses in 2004.
The trend continues this season. The 49ers, Jets, Rams and Raiders have very high tackle totals. The Redskins and Buccaneers have extremely low totals.
The inverse relationship between tackle totals and quality isn't perfect; statistical relationships rarely are. The Eagles, a team with a good defense, ranked sixth in tackles in 2004. The Chiefs, with as bad a defense as you'll ever see, ranked 27th. Similarly, the Colts have a high tackle total this season, while the Vikings and Texans have relatively low totals.
But perform a simple linear regression (a method to find out if one variable is directly related to another, as explained here) and the trend is evident. As defensive DVOA goes up, tackles go up -- and that's bad, because the higher the DVOA, the worse the defense. The "r" value is about .375, the sign of a definite (but not absolute) relationship. Use offensive points allowed or some other relevant stat in place of DVOA and you get comparable results.
So face it: if the starters on your favorite team, particularly the linebackers and defensive backs, have high tackle totals, it's probably a sign that the defense isn't very good.
At first, the relationship between tackles and defensive quality seems counter-intuitive. But the more you think about it, the more sense it makes.
For defensive players to make tackles, what has to happen? Obviously, the other team must execute a play. Furthermore, the play has to end with a tackle. That eliminates touchdowns, interceptions, incomplete passes, and plays where the back or receiver steps out of bounds without being forced. In other words, most tackles come at the end of either a) handoffs or b) completed passes.
So clearly, bad defenses have an advantage. Opponents can run more plays against them, because they can't stop drives as well as good defenses can. Because bad defenses cannot stop the pass, opponents can complete more passes against them. Because bad defenses usually belong to bad teams, opponents often have the lead, meaning more handoffs to eat the clock, and more effort on the part of the offense to stay in bounds.
All of which means one thing: more opportunities to make tackles.
There are three effects that act to counter-balance the forces that lead to more tackles for bad defenses:
1. When teams are milking the clock, they often run fewer total plays. This appears to have a very minor effect on tackle totals, one offset by the fact that so many of these slowly-executed plays are handoffs in the middle of the field.
2. Teams with effective quick-strike offenses tend to a) put their defenses on the field more often and b) force opponents to speed up the tempo of their play, resulting in more plays for the opponent and more tackle opportunities. This is why the Eagles had a high tackle total despite a good defense, and why the Colts led the league in tackles despite an average defense in 2004. But this factor is also offset somewhat by the fact that a good defense, even in prevent mode, will force lots of incomplete passes when holding a lead.
3. Defenses that allow lots of big plays will have lower tackle totals because, frankly, they can't tackle anyone. If opponents can go 80 yards in three plays, then the defense will probably only record two tackles. This explains why the Chiefs had a low tackle total in 2004; the Texans are having similar problems this year.
But none of these factors can truly stem the tackle tide. Tackles depend on completed passes and handoffs more than any other variable, and bad teams allow more than their share of both.
Take a team's handoffs and their completed passes, add them together, and subtract touchdowns. The result is Tackle Opportunities. Tackle Opportunities have a very high correlation with solo defensive tackles: for stat fans out there, the r-value is a healthy .802. The average team gets about 720 Tackle Opportunities and records 700 solo tackles: out-of-bounds plays and other oddities make up the difference (note that players often earn a tackle for forcing a ball carrier out of bounds).
The range of Tackle Opportunities was 603 to 800 in 2004; the Steelers recorded the 603, the Raiders the 800. The 200-unit range in Tackle Opportunities is roughly consistent from year to year. Actual tackles also fall into a relatively neat 200-unit range, and the standard deviations for both are large: 43.6 for Tackle Opportunities, 47.7 for tackles.
What does all of this number crunching mean? It means that bad defenses have about 200 tackles to split up among their defenders that great defenses don't. That comes down to 11 tackles per defensive position, though the tackles aren't really distributed that evenly. And the standard deviation suggests that the difference between a somewhat above average team and a somewhat below average team can amount to 95 tackles: almost nine per starting position.
Think about that in terms of Danny Clark, who had 99 solo tackles for the Raiders in 2004, or Robert Griffith, with 92 solo tackles for the Browns. Or look at the other side of the spectrum: John Lynch's 48 tackles for Denver, James Farrior's 68 tackles for the Steelers.
Powerful forces were distorting these totals, elevating Clark and Griffith, punishing Lynch and Farrior. Those same forces are helping Jonathan Vilma right now. Vilma is a solid player on a decent defense, but the Jets' offense is so bad that Vilma is always on the field. He leads the league with 47 solo tackles. Under other circumstances, Vilma could play at the same level but register far fewer tackles.
When you realize that tackles aren't evenly distributed among players, then the forces shaping defensive statistics become even stronger. Only 25% of all tackles are made by defensive linemen, 32% by linebackers, and 43% by defensive backs. Yes, almost half of all tackles are made in the secondary.
If defensive backs make the most tackles, then why are the NFL's tackle leaders always linebackers? It's simple, really. The 221 tackles made by an average team's linebackers are usually split three ways, sometimes four. The 302 secondary tackles are always split at least four ways, with fifth and sixth defensive backs playing so regularly that they eat up a fair percentage of the totals.
You might expect that better defenses make more tackles on the defensive line, while weaker defenses make more tackles in the secondary. That's somewhat true, but there's a lot of noise in the data: 3-4 vs. 4-3 defenses, players who can be classified as either linemen or linebackers, and the effect of a great defense spending the fourth quarter in prevent mode all pollute the data. But there's a slight positive correlation between good defenses and line tackles (gummed up by 3-4 teams like the Steelers) and a significantly stronger negative correlation between defensive quality and secondary tackles. The linebacker data also has a slight negative correlation; since linebackers and DBs make far more tackles than linemen, the result is a fairly firm negative correlation between tackles and quality overall.
Remember that 200 tackle split between the best and worst teams? If 43% of the tackles go to the secondary, that's 86 tackles. You can give 20 to each starter and let the nickelback keep the rest. The linebackers get 64 tackles; in a 4-3, that means another 20 for each starter, plus four for the backups. That leaves 40 for the lineman, but in reality, if the defense is truly bad, those linemen will claim a smaller percentage of the booty, meaning more tackles for the linebackers and DBs.
Those 20-tackle swings separate the best teams from the worst teams, but with high standard deviations, even the difference between decent and mediocre can mean 11 or 12 tackles at positions like middle linebacker or strong safety, tackles gained not on merit but due to circumstance. A dozen tackles is the difference between 12th and 40th on the league leaders list.
If you haven't figured it out yet, it means that tackle totals are almost completely useless for evaluating the performance of individual players. And justifying the quality of your team's defense by saying that "Joe Linebacker is third in the league in tackles" is almost precisely counterproductive.
So if tackle totals are meaningless, is it possible to come up with a good statistic to measure the performance of an individual defender?
Yes, but it takes a lot of work. With some massaging, we can get a little truth out of tackle totals. Roland Beech's method of breaking down tackles by yardage and situation (which you can find at Beech's Two Minute Warning and in Pro Football Prospectus 2005) shows that some defenders make more significant tackles, while some just clean up the mess after 12-yard gains. Beech's technique makes it easier to single out defensive tackles who make all of their tackles right at the line of scrimmage or expose frauds at weakside linebacker who only make stops downfield.
A quick-and-dirty way to determine a defender's contribution is to take the player's tackles and divide them by the team's tackles. Players like Derrick Brooks and Ray Lewis shine in the "percent of team's tackles" category, showing that high tackle totals don't always come from easy opportunities.
But even these approaches are limited. Beech's methods can't help a linebacker whose front four is so terrible that he's forced to make lots of downfield tackles. Some coaches funnel plays to their top linebackers, inflating their tackle totals and screwing up the team percentages. And cornerbacks present the thorniest problem: if they do their jobs right, they won't have to tackle anyone, because the receivers they cover won't be open.
One major problem lies not with the plays that are counted, but the plays that aren't counted: incomplete passes. Good defenses force a disproportionate amount of incomplete passes when compared to bad defenses. In 2004, opponents executed over 950 plays against the Bills; 201 resulted in incomplete passes, over 21%. Opponents executed over 1,046 plays against the Raiders, but just 186 passes fell incomplete, or under 18%.
Many of those incomplete passes represent good plays by individual defenders: tight coverage in the secondary, good pressure from the front four, or a linebacker blitz that forces a bad throw. These efforts are sometimes counted in hard-to-find categories like "hurries" and "passes defensed," but these stats aren't much help. Hurries can still lead to completions, and the player who breaks up the pass is actually the one guy whose receiver may have appeared to be open: shouldn't his teammates get more credit than he does?
At Football Outsiders, we're always on the prowl for better statistics, and we're eager to break the "tackle code" and create a reliable metric for individual defenders. We've already come up with some useful techniques; for example, we break down DVOA by primary receiver, secondary receiver, slot receivers, and so on. If a team always shuts down their opponents' tight ends, we know about it. By the end of this season, our game charting project will allow us to determine who should get credit for such an accomplishment: the free safety, a linebacker, a nickel defender, or some combination of defenders. Our Adjusted Line Yards statistic can be used to determine whether a team's run defense is stout up the middle, to the left, or to the right. Through game charting, we'll find out who deserves the credit: the linebacker making the plays, the lineman eating the double teams, or someone else.
But while we toil away in the Football Outsiders laboratory, the moral of the story remains simple. Read tackle totals if you want, use them in your fantasy league if you want, but don't use them to compare players. Take several grains of salt before trying to interpret them. And never forget that a high tackle total could be the sign of a great player, but it's more likely to be the result of playing for a lousy team.
(Note: only solo tackles on defensive plays were counted in this study, not assists or special teams tackles.)
32 comments, Last at 08 Nov 2005, 4:14pm by Gio Loz