Instant replay review is one of the cornerstones of the modern NFL. The process and its myriad special rules have been internalized and constantly debated. Mike Kurtz wonders: is it worth it?
01 Dec 2011
by Mike Tanier
MEL TUCKER: Good morning, gentlemen. It sure has been a week of upheaval, but our job as Jaguars coaches is to avoid distractions and get ready for the Chargers. So listen up...
DIRK KOETTER: No, you listen up! I’m the one that calls the shots around here!
TUCKER: What are you talking about?
KOETTER: I have been running this team for years. I have implanted a mind control chip in your brain. You cannot resist my will.
TUCKER: That is crazy. I ... zzzzzzaappp ... Gah! Don’t call a timeout in the final seconds! Run a stupid play at the goal line!
KOETTER: You see? Just like your predecessor, you have no free will! And I have implanted chips in coaches around the league. Observe!
TUCKER: Marty Morhinweg and Jim Washburn? What are you doing here?
JIM WASHBURN: How did that son of a gun get to you? We had the whole Eagles defense protecting you. Oh yeah, right. Well, I will tackle you and rip it off myself ... zzzzzzaappp ... Gah! Agree with Juan Castillo! Agree with Juan Castillo!
KOETTER: You see, my device forces you to make the most ridiculous decision imaginable. I have guaranteed us a Monday night win by attaching one to our rival coach. Right, Norv?
NORV TURNER: I will run the ball three straight times to try to set up a career-long field goal by my journeyman kicker in overtime, giving the opposing folk-hero quarterback, who cannot actually throw, a short field to work with so he can win with zone-read handoffs.
TUCKER: I cannot help but notice that you did not push any buttons, there.
KOETTER: Yeah, with Norv it is more of a failsafe. But look at how it works on Raheem Morris!
RAHEEM MORRIS: zzzzzzaappp ... Gah! Spike the ball! ...zzzzzzaappp ... Spike it again! ... zzzzzzaappp ... Keep spiking!
TUCKER: I ... I cannot resist your will. All I can do is circumvent the system by giving public credit to you for your decisions. That must be what Jack Del Rio was doing. He wasn’t deflecting blame! He was trying to warn America!
KOETTER: And he was fired. Now it is too late. My plan reaches the highest levels of the NFL. Right, Mister Elway?
JOHN ELWAY: Maybe I should offer more support for my incredibly popular quarterback, build some good will from this unlikely winning streak, and try to set up some kind of long-range plan to wean him off this option offense and into something a little more viable ... zzzzzzaappp ... Gah! Backhanded criticism! Backhanded criticism!
TUCKER: Funny, I don’t remember our conference room having space for this many people.
KOETTER: My plan is coming to fruition! Elway and John Fox will be fired by the Broncos. I will be hired and have access to a great defense and the most beloved quarterback in the world! Morris becomes my defensive coordinator. LeSean McCoy gets frustrated with underuse in Philly and joins my Broncos. And no matter how stupid I make Norv, he will keep his job and I get two easy wins per year!
BOB COSTAS: Not so fast! You did not count on the vigilance of one of the most trusted sports reporters in the nation!
KOETTER: Yes I did. This mind control device has a "celebration" mode.
TUCKER: ... zzzzzzaappp ... Gah! I ... I’m only dancing. John?
JOHN ELWAY: I am also boogying.
BOB COSTAS: This is mesmerizing. I must tell society to stop watching the Kardashians and wasting their leisure time doing what they choose to do and instead listen to a short, middle-aged man as he conflates a moment of questionable taste into proof of society’s ills.
KOETTER: With Costas neutralized, nothing can stop me!
NICK NOVAK: Hey, Coach Norv. I hope I am not interrupting, but I pulled the car around back and grabbed the dry cleaning, and I just wanted to see if you guys needed anything. Why are you doing the Electric Slide? Did someone get married?
TUCKER: Stop Koetter, kicker boy! He has a mind control ... zzzzzzaappp ... cut the quarterback three days before the season opener!
MORNHINWEG: His power source is in that Gatorade cooler. Short it out by ... zzzzzzaappp ... Chad Hall is an all-purpose offensive threat!
TURNER: I have no advice to offer. Do as you please and all shall fall into line, or not.
NOVAK: Let’s see. They want me to short something out. Why don’t I kneel down beside this Gatorade cooler, reach into my pants, and ... pull out some wire cutters to snip a few connections.
TUCKER: I am free! I am free! The Jaguars and the NFL are saved!
COSTAS: Yes, Mel Tucker, and if all you peons in the audience can tear yourselves away from super-violent video games and mindless reality shows to listen to me, your intellectual and moral superior, for just a few minutes, there’s an important lesson to be learned here. It’s about lazy humor. Given the tone of the times, it is probably too much to expect a knucklehead blogger to be funny in the inventive style of Jack Benny or Eddie Cantor. Instead, we get this: a dull mash-up of the week’s events with a predictable setup, recycled jokes, potty humor, and lame broadsides at a broadcast legend in place of a finale. Worst of all, it traffics in the kind of meta-commentary that references itself as it goes, finally reaching an obnoxious moment when it is undercutting its own premise in the name of humor. Here’s an idea: why not write about what is actually going on in the games, instead of cramming a bunch of headlines into one thudding, obvious skit?
TUCKER: That is great advice, Mr. Costas.
TURNER: Okay, I’m confused. Should I send the field goal unit out now?
The table below shows the teams with the most holding penalties by wide receivers:
|Wide Receiver Holding Penalties, 2011|
|Team||WR Holding Penalties|
|Nine other teams||1|
The count is not perfect. It does not include holds by tight ends or backs lined up at wide receiver. Special teams holds may be in the mix -– Courtney Roby has a holding penalty, for example, that may well have come on a kick return. Monday night’s numbers are not included, but since the Saints have one receiver hold (by Roby) and the Giants zero, I will not bother updating the numbers unless I see Victor Cruz dragging Jabari Greer around the field. I may have missed some third-string receiver somewhere. But you get the point: the Panthers have committed three or four teams worth of wide receiver holds in eleven games.
Steve Smith leads NFL receivers with four holds. Legedu Naanee is second with three. Wire-fighting expert Jerome Simpson of the Bengals and Michael Jenkins of the Vikings each have two. No other receiver has more than one. Smith and Naanee are clearly over-achieving.
So what is going on? First, Smith and Naanee are blocking. Hard. The Panthers rank first in the NFL in runs off right end and fourth off left end. Their offense incorporates a lot of lateral running: by the backs, by Cam Newton on designed runs, and by Smith on end-arounds. Naanee has a reputation as a blocking specialist, and he is living up to it. When receivers are constantly blocking in open space on outside runs, the chances that they will hold and get called for it go way up.
That said, the Panthers have come by most of these holds honestly. I went back and looked at all seven. A couple of Naanee’s were somewhat tacky. Against the Vikings, he blew up Brandon Burton, driving him from the slot to a point about 15 yards downfield before burying him. It was more of a pancake or a bulldoze than a hold, and I think Naanee drew the call because the block was so good it just drew attention to itself. On a Smith end-around against the Saints, Naanee motioned into a tight formation, controlled his cornerback at the point of attack, then performed a cut block as Smith raced past him for a significant gain. I did not see a clear hold, but the defender had a hard time escaping the block, so there was probably more going on than great positioning.
But then, there are Smith’s penalties. He held Chad Greenway on a critical fourth-quarter play against the Vikings, and while it was a fine effort by Smith, it was also an open-field hold. Smith tackled a defender in frustration on a screen pass during garbage time in the Titans game. He also bear-hugged a Lions cornerback on a sweep. Smith got called for a more questionable hold against the Colts, but again, it was a hold, albeit a subtle one, in the open field.
Ron Rivera has complained that the Panthers have not yet earned the benefit of the doubt with officials. It’s possible that he has a point: if the Panthers had a reputation as a great team, then Naanee’s borderline holds might be shrugged off as great plays. The tendencies of officials are out of Rivera and Rob Chudzinski’s hands. Their job is to keep the Panthers receivers blocking hard for the backs, Newton, and one another, while at the same time trying to cut down on penalties that almost always erase good plays. If Rivera wants the benefit of the doubt, he needs to stop Smith from tackling defenders.
From my Inbox:
I recently read an article on another site touting a new method for rating quarterbacks. It was called the Super Duper Rating or something. Unlike DVOA or ESPN’s Total Quarterback Rating, this new stat is based on the old efficiency rating, but souped up to include sacks and scrambling.
This new stat starts with basic quarterback numbers: attempts, completions, yards, touchdowns, and interceptions. But sacks and rushing attempts are added to attempts, rushing attempts are added to completions, rushing yardage is added to (and sack yards subtracted from) yards, rushing touchdowns are added to touchdowns, and fumbles are added to interceptions (I am not sure if that is fumbles lost or all fumbles). The new figures are then run through the old quarterback rating formula, with the 2.375 and all of that.
This new Super Duper Rating was offered as statistical evidence by a major media outlet that Tim Tebow is actually doing an excellent job at quarterback. I don’t want to debate about Tebow, though. I am just wondering if this stat has any merit, because I get the impression it is all mumbo jumbo. I would love your thoughts. Thanks!
Regular readers know that I don’t have a major problem with the NFL passer efficiency rating. It can be useful at times. The major flaw it has, other than being over-engineered, is that it was designed to consider 1970-era passing statistics "average." That causes a big problem when it comes to completion percentage, a ratio whose "average" value has gone up about 20 percent in 40 years. Completion percentage is another stat that gets undue criticism by some analysts. It has some value as an evaluative tool, but not when you consistently mistake today’s 60-percent "average" quarterback as being a better player than a late 1960s sharpshooter.
Completion percentage operates in its own little wilderness in the formula. A quarterback who goes 10-of-20 for 120 yards, no touchdowns, and no interceptions, earns a 68.8 rating. Crank the completions up to 12 (60 percent) without changing anything else, and his rating jumps to 77.1. Kick it up to 14 completions (70 percent) and the rating reaches 85.4. It’s certainly arguable that the 14 completion game was better than the 10 completion game, because the quarterback’s six yards per attempt were more evenly distributed. But those eight-point jumps in rating are operating independently of actual production.
The system thresholds at 80 percent, or 16 completions, with a rating of 91.6, which would rank eighth in the NFL right now if the numbers were projected to a full season. A quarterback who completes 80 percent of his passes, despite averaging only 7.5 yards per completion, would probably be a valuable asset. But again, these completion percentage bonuses operate totally independently of the yardage, in their own little play land.
Now, let’s look at the Super Duper Stat. I like the idea of calling sacks passing attempts and sack yardage lost passing yardage. I can live with the idea of adding fumbles to interceptions, assuming that someone took the time to remove aborted snaps and other events that have little to do with a quarterback’s true performance. And combining rushing stats with passing stats is laudable in theory.
But there is a huge problem, which I bet has readers are screaming at their monitors right now: The Super Duper Stat adds rushing attempts to pass attempts. And it adds rushing attempts to pass completions. So every time a quarterback runs, the system treats it like a completed pass.
And completion percentage is the one stat that the old efficiency rating grossly over-rewards.
The distortions caused by a non-running quarterback are minor. The distortions caused by an Aaron Rodgers or Michael Vick will be somewhat greater, but even seven or eight carries per game are going to be drowned out by 35 passes, and a little boost to Rodgers’ or Vick’s stats for running prowess gibes with common sense.
But then, along comes a quarterback who passes 18 times and runs 22 times in a game. The quarterback’s 9-of-18 completion rate does not move the needle on old efficiency rate, which considered 50 percent "average." But give that quarterback 22 more "completions," and efficiency rating does a happy dance at the site of a 77.5 "completion percentage." It starts assigning value that has nothing to do with actual yardage gained.
That last point is the one that needs to be hammered home, because heaven knows I have gotten enough Tebow hate mail. This system is not awarding points for rushing well. It is awarding points for rushing, period. For illustration’s sake, let’s go back to the Jets game, because the old efficiency rating was happy with Tebow’s numbers against the Chargers. A 9-of-20, 104-yard, no goodies stat line earns a 61.2 by the old rating. Let’s add eight rushing attempts to the mix, but with a catch: instead of adding Tebow’s 68 real yards, let’s give him 24 yards and no touchdowns on those carries. This is not a fourth-quarter hero, but any old quarterback who averages a measly three yards per rush but happened to run about 40 percent as often as he threw.
By the Super Duper formula, this quarterback’s rating goes up to 71.7. A ten-point jump! Not because of epic scrambling ability, but simply because of eight carries.
Let’s really test the parameters. Let’s make a quarterback go 0-for-4 passing, but carry the ball 16 times for 32 yards. The Super Duper rating? An impressive 79.6.
These distortions would only be obvious in an extreme case like Tebow, but they are prevalent throughout the system and create a distinct advantage for rushing quarterbacks. Quarterbacks on winning teams who get to kneel two or three times per game would also be overvalued, because unless the designers of this stat eliminate kneel plays, they get to boost their completion percentages. Because extra carries tend to lower yards per attempt and touchdown percentages, the effects sometimes come out in the wash. But those extra carries can also lower interception rates, causing the Super Duper Ratings to slide off of the old efficiency rating for reasons that have nothing to do with anything. It’s an incredible source of statistical noise that makes a bad situation worse when using an already out-of-date metric.
Can quarterback rushing statistics be added to passing statistics? I would not recommend doing it with a tool that was specifically designed to reward passing and was developed by Nixon-era actuaries. If I absolutely had to soup up efficiency rating with rushing stats, I can think of two ways to try it. First, I might only include scrambles, which are clearly marked in the play-by-play and can be teased out of the raw rushes with a little effort. That takes care of kneels and sneaks and limits the quarterback’s passing evaluation to things that actually happened when he tried to pass. Secondly, I might consider limiting "completions" to first downs gained while scrambling. It’s a jerry-rig, but it provides an almost workable solution to the problem of calling every attempt to run a "completion."
Both of the solutions described above require extra work that goes beyond grabbing raw numbers off Pro Football Reference or NFL.com and slamming them into Excel. The designers of this Super Duper Stat, unlike us or the ESPN Total QBR guys, have apparently declined to do that, opting instead to repackage the low-lying fruit in a new box. That leads me to my final thought, not about the value of this goofy stat itself (nothing, as it has nothing that the old rating does not offer), but about the quality of the reasoning behind it.
I have tinkered with many failed statistical ideas over the years, and Football Outsiders has piloted, adjusted, fiddled with, and a few times thrown out whole models and methods. A statistical method can look good in the spreadsheet and spit out hundreds of encouraging results before revealing some terrible flaw. There is nothing wrong with being wrong, for having to do a headslap and an apology, or with pointing to some anomalous result and saying "this is odd, and it may be a sign of a bigger problem, but it’s the best result we can currently provide."
But there is a major problem with taking a strange, anomalous result, ignoring all warning signs, and trumpeting it as some hidden truth in the name of making money on an article because it happens to say what you want it to say about a currently popular player.
The problems with this method are glaringly obvious to anyone who has ever worked with the old efficiency rating, and I find it impossible to believe that the designers didn’t see red flags when they were setting up their spreadsheets. If they calculated completion percentage in a separate cell, as most of us would, they saw completion rates shooting up from 50 to 77.5 and 45 to 60.7 (the Jets game). They must have noticed that these changes were extreme, that they affected one player more than others, and that they represented essentially zero value added to the quarterback’s performance (remember, the yards themselves are separate). They could have realized the obvious flaw in their method, or at least said something to the effect that "Tebow is blowing up our system." Instead, they spun this critical conceptual error into "Only our stats can see how well Tebow is doing, but yours can’t."
That was lazy, irresponsible, dishonest, and frankly embarrassing to my entire profession.
The best thing I can say about this Super Duper rating is that whoever designed it could not be any more backward in their thinking. They took an ancient statistic and tried to put fins on it. We have been using play-by-play for eight years, and we now have a database going back another decade before that. The Total QBR guys are working with play-by-play and charting. There are other sites using variations on charting. I don’t agree with a lot of their results, and I compete head-to-head with them for cash money at times, but I have to respect that their digging may yield different, perhaps more interesting information than our digging on occasion. Data like third-down splits is not hard to find if you want to create your own Triple Deluxe Efficiency Rating by awarding a third-down bonus or something.
The future lies in more granular data, whether it is separating out yards after catch, differentiating passes by type, or going to the game tape with a compass and protractor and figuring accuracy down to inches (a bad idea, but an example of what’s possible). Grabbing stats off the back of a football card and twiddling with them is not amateurish, because amateurism implies effort and love. It’s just weak and juvenile. And I am guessing this Super Duper Rating will disappear once it starts spitting out results its designers don’t like or cannot profit from.
I am appearing at The Field House in Center City Philadelphia on Thursday night for the Eagles-Seahawks game. Lest you think that it will be nothing but me screaming for three-and-a-half hours, this essay explains my complex relationship with the only team I ever truly loved.
I don’t really hate the Eagles. We are just going through some problems in our relationship right now.
I have been accused of Eagles negativity on Twitter, on the message boards, and even in our inter-office emails. Considering the vitriolic commentary that accompanies the Eagles even on playoff runs, I must be getting pretty extreme to stand out from the crowd. But familiarity breeds contempt, as the saying goes. Like the old guy at the water cooler who grouses nonstop about the wife he could not possibly live without, I have grown so used to the Eagles’ quirks, ticks, and shortcomings that I must remind myself to see the good in them, even in the best of times. And these are not the best of times.
My point is best illustrated by a long, rambling story.
Over the Thanksgiving weekend, I took my sons to see The Muppets (awesome!), then sat down for a few minutes with my birthday gift: Batman: Arkham City for Xbox 360. The boys sat to watch me play, with Mikey sent away at the first of many, many obscenities. C.J. has heard such language -– he is in third grade, after all -– and I allowed him to stay, though I closed the door to make sure my wife did not hear him hearing such language. I needed him to get me through some of the levels that require the kind of twitchy button mashing skills it was impossible to develop in the Atari era. I also get motion sick playing first person games, so C.J. is often in charge of walking through corridors and finding things.
Batman’s morality is starting to trouble me now that I have to explain his actions to a nine-year-old. He does not kill, which seems laudable until you think about what he does do. He delivers the kind of beatings that can result in internal bleeding or broken limbs, then leaves perpetrators defenseless on the streets of Gotham, a city of about four million people with one competent cop. I am sure none of the goons he beats up have health insurance -– perhaps Harvey Dent flips a coin and insures them if it comes up "heads," making him 50 percent more generous than many employers -– and The Joker is not the kind of guy to set up an infirmary or drop his henchmen off in the ER.
So Batman, with his sanctity for human life, leaves criminals to go into shock in the gutter, then get punished for their efforts by some psychotic crime lord. C.J. and I also watched some Indiana Jones this weekend, and Indy isn’t above offing the occasional bad guy in self defense. Not just Nazis, either, but tribesmen and the like. Given the choice of getting shot by Indy or left to suffer after having an arm broken and spleen ruptured by Batman, I know which one I would choose.
Anyway, Batman wasn’t beating up anyone with me controlling him, because I could not figure out the controls. Bats got beaten up badly by a couple of guys with baseball bats, the greatest superhero in the world pummeled by the middle of the Texas Rangers lineup. I kept mixing up the Strike button with the Pose Menacingly button, a problem Dominique Rodgers-Cromartie shares. Later, there was a labyrinth of corridors, which I followed until I literally threw up. It was not Batman’s finest hour, as he stood repeating the same prompt over and over while I retched nearby.
C.J. was better at dispatching goons, but C.J. has the childish habit of button-mashing through cut scenes, because kid time is too valuable to waste on advancing the plot. So Kevin Conroy would intone "Now that I have found this new gadget, I can open locked doors. All I have to do to use it is..." mash, mash, mash, and we would be left groping along some warehouse until I got the heaves again.
But I did not get angry at my son, even as he knocked on the bathroom door while I was heaving to say that he learned how to fire batarangs and leveled up my character. That is not what this story is about.
C.J. eventually went to bed, and my wife and I curled up on the couch and realized that we don’t watch any television shows regularly anymore. So we ended up watching the 2009 Rock ‘n’ Roll Hall of Fame Anniversary Concert on Palladia, a station that shows nothing but an Iron Maiden documentary and lots of promos involving a butterfly. I try to catch the Rock ‘n’ Roll HoF induction ceremonies when they are televised, because they are a hoot. Old R&B performers stumble through their past hits with someone like Fergie horning in to gain some credibility. Rock dinosaurs pretending they like each other. Uninspired renditions of songs that changed the world by artists who long ago moved past the stage in life when the songs mattered to them. And, of course, the kinds of teamups that cause brain pain: Nelly and Robbie Robertson, Booker T. and the MGs and Armored Saint, C-Lo Green and Crowded House joining Tom Waits to perform "He Ain’t Heavy, He’s My Brother."
The anniversary concert was every bit as good as an induction ceremony, with a senseless Sting-Stevie Wonder teamup and Metallica bringing both Lou Reed and Ray Davies on stage after aiming straight for the middle of the diverse audience by starting with "For Whom the Bell Tolls." Then, a very bored-looking U2 appeared, slogged through "Vertigo," and invited Bruce Springsteen on stage.
Now, I am a huge fan of both U2 and Springsteen. I have even followed them through the long third acts of their careers, buying albums as their respective relevance unraveled. I liked Springsteen’s Magic and U2’s No Line on the Horizon even though those albums were nothing compared to the best work by either performer. So I should have been delighted when they took the stage, with South Jersey’s own Patti Smith in tow, no less.
Instead, I thought oh no, here it comes.
Smith’s presence made "Because the Night" obligatory, and it was another perfunctory rendition of a song Springsteen often drags out in concert. Smith was then sent away, lest she perform one of her challenging, interesting songs. Springsteen hung around. "You know, once he takes the stage, he doesn’t leave," my wife said, and as a veteran view of HoF inductions, I knew she was right. Springsteen inserts himself everywhere, trying to perform with everybody. For all of his achievements, the world did not need his version of "London Calling." Everything Springsteen performs becomes Springsteenized, robbed of any lilt and replaced with the gravel voice and jackhammer guitar.
Meanwhile. Bono feels the need to explain everything, because he is deeper than all of us put together. Springsteen used to be an explainer, too, and old concert bootlegs are filled with 10 minute spoken word interludes for songs like "Independence Day," which had a pretty freakin’ obvious message in the first place. Bono reached the nadir of explanation around Rattle and Hum, which was ten minutes of music discussed on stage for an hour-and-a-half. Bono has eased back, like Springsteen, but not nearly enough, and he decided to help us decode one of his trickier songs by explaining that it is about a spiritual journey. The song? "Still haven’t Found What I’m Looking For." Bet you puzzled over the meaning of that one for years, right?
So there they were, two of my favorite performers, two legends of contemporary music, being completely non-entertaining as they indulged their worst instincts. But I could not turn away, because I am a Springsteen and U2 fan. I kept waiting for it to get better. Maybe they would rip into some song I don’t expect or do something truly magical. I felt obligated to watch and feel exasperated.
And that is how I feel about the Eagles. I am stuck watching them. I know their foibles. I have a hard time getting excited about a Giants win sandwiched between two ugly losses. I was optimistic, to a fault, through an entire era, and now I am stuck supporting a lame facsimile of a system that worked much better long ago. It’s like listening to Working on a Dream, except that the Eagles never quite produced a seminal Darkness on the Edge of Town under Andy Reid. And yet, I have articles to write and a book to promote, so I have to keep watching.
So the negativity is inevitable, and I plan for this to be the last Eagles-centric Walkthrough of the season, because they are no longer relevant to a national audience. Thursday’s appearance at The Field House will be my last Eagles game for a while, because my professional time is better spent on the Ravens or Steelers. It is not a divorce, or even a trial separation. It’s a working vacation, a chance to get a little space. I hope to come back reinvigorated.
Now, off to confront the Joker, then enjoy some ginger ale and crackers.
86 comments, Last at 05 Dec 2011, 11:12am by Noah of Arkadia