Guest columnist Zachary O. Binney fact-checks a story in a national publication and finds that everyone makes mistakes.
30 Aug 2013
by Danny Tuccitto
Welcome back for the sixth and final installment of our anniversary series on individual DYAR and DVOA records since 1991. If you need to get up to speed or would just like to reminisce, here are the links to the previous installments:
Over the course of the series, I've made several observations, both explicitly and implicitly, about the difficulties inherent in football analytics. In "Best Running Backs," I made a purely statistical case for Priest Holmes' Hall of Fame candidacy knowing full well that statistics aren't everything (as many commenters astutely pointed out). In "Best and Worst Wide Receivers," I cited the problem of separating a wideout's value from that of his quarterback and fellow pass catchers. The real issue underlying both of these is one of process versus outcome. Even knowing the strengths and weaknesses of the methods used to come up with a number, we're still often guilty of focusing too heavily on the number itself.
Back when Doug Drinen created Pro Football Reference's approximate value (AV) statistic, he spent several thousand words detailing the methods behind it. Buried in those words, and apparently forgotten over the past five years by those of us who care about process, is a preliminary, admittedly arbitrary, yet nevertheless elegant attempt to assign an average proportion of passing, running, receiving, and blocking for each offensive position.
For tight ends, Drinen allotted 30 percent of their value to blocking, while setting the other stat-driven positions to zero. Admittedly, the impact of a running back's (or wide receiver's) blocking ability on his total value as a player is higher than zero percent -- Goddammit Donald! -- but even if we gave them non-zero blocking value, I think we would still all agree that the proportion for tight ends would remain significantly higher than the other positions. For our purposes today, the specifics aren't nearly as important as the fact that (a) tight ends have much more of their individual value wrapped up in blocking, and (b) blocking isn't in the play-by-play, which is what forms the basis of DYAR and DVOA. Like Walter Johnson said, "You can't hit what you can't see."
I've chosen to wax philosophical here for two reasons. First, it will become apparent very quickly that the extent to which a tight end's responsibilities diverge from that 70-30 split ends up making a huge difference in his DVOA and DYAR. Second, the rankings are so dominated by a handful of predictable tight ends that I could stop the column right now and go straight to a red light challenge in the comments section.
Instead, here are the best and worst receiving DVOA seasons by a tight end since 1991:
Yes, Antonio Gates was ridiculous in 2010, but no one would mistake him for a great blocker. That's not a slight; it is what it is. Rickey Dudley was a "below average blocker." Tony McGee was "not the best blocker" and only was a blocker because the Bengals often needed him to be a blocker. Jermichael Finley was a poor run blocker prior to this month. Visanthe Shiancoe is "one of the worst run-blocking tight ends in the league." Matt Schobel had the dreaded tag of "finesse blocker." Troy Drayton was a converted wide receiver.
Meanwhile, Howard Cross was "probably the best of all tight ends in the league at blocking." Daniel Graham was "one of the best pass-blocking tight ends in the NFL" in 2010. I could go on.
By my count, outside of total packages like Rob Gronkowski, Martellus Bennett, and Ken Dilger, two of whom -- spoiler alert! -- rank among the best 22 tight ends since 1991, these lists are loosely based on Stealers Wheel lyrics: Receivers to the left of us, blockers to the right.
I wonder if this changes when we switch from DVOA to DYAR. The relevant table is displayed below:
Why yes, yes it does, but the reason is obvious: DYAR depends on volume, so the best blockers (i.e., the worst receivers) will tend to get fewer targets. It takes a legendarily awful performance over a small sample of targets for a blocking tight end to produce an all-time awful DYAR. Hence, only Graham and Cross show up on the list of worsts.
Otherwise, it's a who's who of tight ends that you will recognize from fantasy football waiver wires of years gone by -- and Brandon Pettigrew (and Freddie Jones, but more on him later). Pettigrew's 2012 season in Detroit was the tight end version of Chris Chambers' 2006 season in Miami. The only way a player that inefficient gets that many targets in the pass offense is if (a) he's coming off a big year, and (b) his team is basically forced to keep throwing him the ball due to a dearth of competent pass catchers. Obviously, Detroit had Calvin Johnson last season (and, thankfully for Lions fans, still have him), but when a quarterback shatters the record for most pass attempts in a single season, Megatron can't be the only one on the receiving end. (Of course, it also doesn't help when your No. 2 receiver turns into "The Lifeguard" midway through the season.)
That said, it's tough to absolve Pettigrew of much blame for last year. Although his 58 percent catch rate (10th-lowest among qualifying tight ends) was actually higher than the 49 percent posted by teammate Tony Scheffler's (second-lowest), Pettigrew's average target was 6.3 yards downfield (sixth-lowest), while Scheffler led all qualifying tight ends with an average target 12.5 yards downfield. Catching that few passes on that many short targets translates to a receiving plus-minus of -8.1, which was the worst at his position. What's more, as mentioned in Football Outsiders Almanac 2013, Pettigrew has dropped more passes over the past three seasons than any other tight end. In other words, the best explanation for Pettigrew's appearance on this list is that he was treating a ton of Matthew Stafford passes as if each one was the "Deadliest Catch."
Now, if we've learned anything thus far in this series, I hope it's that the good players can have horrible games. Do you think that trend applies to tight ends? Well, here are the 10 best and worst games since 1991 (box scores linked in the "Week column; asterisk means the team won):
|Best Receiving DYAR, Game, 1991-2012|
|Worst Receiving DYAR, Game, 1991-2012|
Why yes, yes it does. Exactly half of the bottom 10 tight end receiving DYARs in a game came from players who you will see in our table of best careers, probably the least shocking of which is Jeremy Shockey. Here's the thing about his pair of all-time bad performances in 2006: He actually finished the year above replacement. In the two games featured in the table, he had -99 receiving DYAR; in the other 13 games that year he amassed 121 DYAR. Why? Well, aside from the fact that Shockey had already proven himself to have multiple personalities as a rookie in 2002 playoffs, Eli Manning and offensive coordinator John Hufnagel gave him 98 other opportunities to redeem himself. And unlike, Chambers and Pettigrew, he did. (In case you're wondering why Shockey beat out Dallas Clark despite an identical stat line, it's mainly because Clark's game came against the 14th-ranked Chiefs defense, while Shockey's came against 21st-ranked Buccaneers.)
The list of bests is mostly what we've seen so far: a handful of great tight ends figuring prominently. So, let's take a trip to Whoville and admire Ethan Horton. In 1991, Horton had 57 catches for 650 yards and five touchdowns, which means that he averaged 3.3 catches for 35.1 yards and 0.3 touchdowns in his other 15 games. In 1991, Horton had 133 DYAR; I'll leave the absurd math to you on this one. In 1991, Horton made the Pro Bowl.
How did that happen? Well, it's because 1991 was arguably the worst year for tight ends in our database. Horton's DYAR ranked third; in 2012, it would have ranked sixth. DYAR's most valuable tight end in 1991 was Eric Green, with 170. Only four of the past 22 seasons have seen the No. 1 tight end fail to eclipse 200 DYAR, and Green's was the lowest DYAR of them all. But if you really want a crystallization of how irrelevant the tight end position was back then, here are PFR's yardage rankings for tight ends in 1991. Pro Bowlers Horton, Marv Cook, Jay Novacek, and Steve Jordan ranked second through fifth -- behind Oilers slot receiver Ernest Givins. (Rivers McCown would vehemently disapprove if that was [redacted].)
Next up, the worst career receiving DYARs according to a simple sum, a weighted sum, and an average of the player's six best seasons (asterisk means the tight end is still active):
Mostly, it's the blocking tight end All-Stars, folks. That's the only way players like Mark Bruener and Justin Peelle show up here despite career lengths in the double digits. Bruener registered one horrible year of receiving DYAR in 1999 (-100), but it came on only 35 targets. The other 12 years involved minor fluctuations between his second-worst DYAR (-41 in 1998) and his best DYAR (37 in 2000). The same goes for Peelle. If not for his one bad year with the 2005 Chargers (-71 receiving DYAR in 20 targets), he wouldn't be anywhere near this list. It's safe to say both Bruener and Peelle remained on NFL rosters as long as they did because teams coveted their blocking ability, so it's kind of awkward to discuss their value only in terms of receiving. Such is the life of a blocking tight end; such is the state of limitations in play-by-play analysis.
On the flip side, the fact that there are so few receiving tight ends on these lists makes the ones who do appear look really bad. Freddie Jones deserves every bit of his No. 1 ranking according to aggregate DYAR. Since 1991, 19 different tight ends have had a season worse than -100 receiving DYAR. Jones is the only one to have accomplished that "feat" twice: -158 DYAR in 1998 and -108 DYAR in 2002. And yet, because he played on a couple of awful teams (turn-of-the-millenium Chargers and Cardinals), he too owes his standing in the table to a case of Brandon Pettigrew syndrome (nee Chris Chambers syndrome). Jones' 702 targets is by far the highest of any tight end above; only Stephen Alexander (429) and the aforementioned Pettigrew (393) broke 300.
Finally, below are the best career receiving DYARs according to a simple sum, a weighted sum, and an average of the back's six best seasons (asterisk means the tight end is still active):
As I'm sure everyone expected, Tony Gonzalez pretty much blows away the field here. Since we're talking a lot about methods today, I'll use Gonzalez as an example of how the peak versus longevity argument goes off the rails sometimes. As I said in "Best Running Backs," hardly anyone cites a player's peak if he had a long career. As you can see in the table, the more we slide over to the peak side of the continuum, the less dominant Gonzalez appears, to the point where he drops out of the top spot according to the average of his best six seasons. Here's the thing: His 16th season (i.e., 2012) was the third-best season of his career according to receiving DYAR. Research shows that tight ends decline after age 29. Since he turned 30 in 2006, Gonzalez has made six Pro Bowls and two All-Pro first teams. In that time, he's amassed 1,316 receiving DYAR. Stated differently, if Gonzalez' career began when the average tight end declines, he would still own the third-best career since 1991.
Back in May, Chase Stuart asked, "Can Jason Witten catch Jerry Rice's career receptions record?" The answer was "probably not, but he's a lot closer than you think." Can he catch Gonzalez in aggregate receiving DYAR? I think our answer is the same. Witten just completed his age-30 season, in which he amassed 192 DYAR, and he currently stands 1,369 DYAR behind Gonzalez. Conveniently, I just told you that Gonzalez had 1,316 DYAR after age 29. A quick back-of-the-envelope calculation tells us that, in order to catch Gonzalez, Witten would basically have to be the second coming of Gonzalez. Given his injury history, I don't think that's likely. (And in case you're wondering, Shannon Sharpe checks in at 2,075 DYAR with only one season prior to 1991. We'll be introducing DVOA ratings for 1990 next week, but I can tell you that Sharpe had a mere 5 receiving DYAR as a rookie, so his career will rank no better than third in perpetuity.)
To close out this series, I'll offer an anecdote that hopefully helps tie all of these anniversary columns together. Like many of you, my earliest memories of working with football stats came in the context of finding an advantage in fantasy football. When I went off to college, and ultimately graduate school, the field of football analytics didn't exist, and so my area of study was Sport Psychology, something fascinating to this boyhood fan of Joe Montana. In 2004, I discovered a little site called Football Outsiders, and, as an aspiring academic, I eventually found the Journal of Quantitative Analysis in Sports. The very first issue of said journal had an article entitled "Football's Hilbert Problems" and authored by Aaron Schatz, the guy running that little site I discovered the year before. The moment I read it, I was hooked, and so began a personal journey away from Sports Psychology towards football analytics: "You can get this kind of stuff published in an academic journal? People take this stuff seriously?" Without Aaron blazing a trail 10 years ago, I wouldn't be here, and I don't think the field would be as popular as it is today. For that, I think we all owe him a debt of gratitude.
As far as the meat of "Football's Hilbert Problems" goes, it's behind a pay wall, but the basic premise was to list the 10 most-vexing problems confronting those who analyze football statistics:
Looking back years later, it's amazing to see how far we've come. Certain official scorers in certain stadiums are still nuts, but play-by-play is standardized for the most part, and we now have things like snap counts that have been added. Game charting projects, here and elsewhere, have gone a long way towards having at least something in the way of non-standard defensive stats. We've found that a deep roster is better than one with stars and scrubs. Analytics departments around the league have developed sophisticated ways to valuate free agents, and it's practically conventional wisdom that free agency is a real-world example of the winner's curse. Park and weather effects have been accounted for. Draft pick values are a dead horse at this point. FO's various rookie projection models have helped translate college stats to the NFL.
The only Hilbert problems that haven't progressed that far are blocking, player comparisons, and strategy. It's possible to measure blocking (we do), but error abounds when you don't know the blocking assignments. Also, there's the point I've belabored today: Whether or not you can measure it in a reliable way, how much of a tight end's value is wrapped up in his blocking ability? Doug Drinen guessed 30 percent, but establishing a more concrete proportion has been something largely ignored in the football analytics community. Similarly, game charting projects and the NFL's release of All-22 video has illuminated data about in-game strategies and tactics, but it's been used primarily as a way to describe the past (i.e., Team X lined up in 11 personnel 48 percent of the time last year) rather than to predict the future or research how specific strategies relate to winning statistically; and without being privy to the minutes of coaches meetings, it will be difficult to take that leap.
Finally, there's the inability to compare players independent of their teammates, which was the problem I highlighted in "Best and Worst Wide Receivers." To wit, Aaron wrote the following eight years ago (DPAR is the ancestor of DYAR):
DPAR cannot be used to compare players across positions, however, because it gives full value for passing plays to both quarterback and receiver, and does not adjust based on the quality of surrounding players on each team. As discussed above, much of the data needed to fully correct for the imperfections of DPAR does not currently exist.
It still doesn't exist. As FO enters its second decade, major progress on nine of the 10 Hilbert problems that Aaron delineated at the beginning of its first decade is not shabby at all. As loyal readers, you've been here to watch the process unfold. We hope you stick around for the day when that most intractable of Hilbert problems gets solved. Does 2023 work for you?
15 comments, Last at 03 Sep 2013, 5:39pm by Fartzone