Writers of Pro Football Prospectus 2008

printable version

Extra Points

Articles from around the Web

Need Help with KUBIAK or Premium?

blog advertising is good for you


04 Oct 2011

Michael Vick and Roughing the Passer

Following the Eagles' Week 3 loss against the Giants, Michael Vick went on a bit of a rant about the state of officiating in the NFL as it pertained to him. The rare "I'm not complaining, but I am complaining," statement. Former NFL Director of Officiating Mike Pereira went on a barnstorming tour to show his support for the officials and to counteract Vick's claims, ending with a piece on FoxSports.com that measured each players' Roughing the Passer penalties against his pass attempts.

Now, if you're a long-time Football Outsiders reader, that sentence probably inspired the same red flag that came to my mind: pass attempts are a poor denominator. Not only would it shift things in favor of players who didn't throw often, it would also make certain superstar quarterbacks with a proclivity for throwing the ball often show up lower on the chart.

I decided to run the numbers and compare the Roughing the Passer penalties to our quarterback knockdown numbers (i.e. QB hits plus sacks). This contains data from 2009 through Week 3 of this season. Plays cancelled by penalty are included (after all, a sack with a Roughing the Passer penalty doesn't count in the PBP, but it sure counts to the quarterback).

Here's your big caveat: Roughing the Passer penalties are fairly rare. There are only about three or four of them, total, called in any given week. Don't make sweeping conclusions based on this data. (And a small caveat: Not every Roughing the Passer penalty is called on a quarterback knockdown. A knockdown technically means the quarterback hit the ground or was sacked, but some Roughing the Passer penalties are called simply for helmet contact.)

With those limitations in mind, here are my observations:

  • Vick actually draws flags at about a league-average rate overall. He's drawn four flags since he started seeing regular quarterback time again in 2010, one this season. Per our data, he was knocked down 21 times in the first three weeks of this season. Sam Bradford, who was knocked down 30 times, lead the league. The 32 quarterbacks I selected for this sample (players who were knocked down 70 or more times over the 2009-2010 seasons) averaged a Roughing the Passer flag every 31 knockdowns. So while Vick only has one Roughing the Passer penalty so far this year, it's not like he's getting hosed. That puts him ahead of schedule. (Though other quarterbacks are certainly further ahead of him, of course.)
  • On the other hand, there may be a bit of a precedent to whine publicly about unfair officiating. After Tom Brady drew two Roughing the Passer flags against Baltimore in 2009, the Ravens hemmed and hawed about it. After that, Brady and Peyton Manning, the two pre-eminent quarterbacks in the NFL this past decade, both drew just one flag for it in 2010. Granted, Brady's knockdowns went down about 30 percent in 2010, but he drew five in 2009 (Manning drew three). Brady hasn't drawn one yet in 2011 either, though Ryan Fitzpatrick drew two in their Week 3 game in Buffalo. Drew Brees, Aaron Rodgers, and Philip Rivers have stayed roughly constant.
  • In that spirit, it may be time for Tony Romo, Matt Schaub, Joe Flacco, and Josh Freeman to start complaining (or having someone else do it for them). The four of them are all huge outliers -- combining for just five total flags in 620 quarterback knockdowns. Romo did finally draw one in Week 4 that I haven't counted down here -- all it took was Ndamukong Suh nearly destroying him. For that matter, David Garrard drew zero flags in 210 knockdowns, but I'd imagine he's more worried about finding employment than lobbying for favorable calls at this point.

Here's what the raw data looks like:

Roughing the Passer Penalties / Quarterback Knockdowns, 2009-Week 3 2011
Quarterback KD's RTP penalties KD's/RTP
David Garrard 210 0 INF
Joe Flacco 180 1 180
Tony Romo 132 1 132
Josh Freeman 124 1 124
Matt Schaub 184 2 92
Matt Hasselbeck 138 2 69
Brett Favre 148 3 49.3
Alex Smith 133 3 44.3
Eli Manning 132 3 44
Donovan McNabb 162 4 40.5
Kurt Warner 79 2 39.5
Ben Roethlisberger 156 4 39
Derek Anderson 72 2 36
Mark Sanchez 103 3 34.3
Aaron Rodgers 165 5 33
Matt Cassel 160 5 32
Jason Campbell 162 6 27
Michael Vick 106 4 26.5
Matt Ryan 149 6 24.8
Chad Henne 144 6 24
Matthew Stafford 93 4 23.2
Trent Edwards 88 4 22
Tom Brady 131 6 21.8
Drew Brees 131 6 21.8
Carson Palmer 130 6 21.6
Peyton Manning 86 4 21.5
Philip Rivers 138 7 19.7
Jay Cutler 194 10 19.4
Shaun Hill 76 4 19
Kyle Orton 133 8 16.6
Sam Bradford 113 7 16.1
Ryan Fitzpatrick 128 9 14.2
League Average 133.7 4.3 31

Posted by: Rivers McCown on 04 Oct 2011

45 comments, Last at 01 Jan 2013, 7:13am by mano

Comments

1
by jk (not verified) :: Wed, 10/05/2011 - 12:55pm

Been waiting for you to correct Pereira, good work, thanks.

2
by Chazzz (not verified) :: Wed, 10/05/2011 - 12:56pm

Something about the phrase "Infinite Garrard Knockdowns" makes me chuckle.

3
by bubqr :: Wed, 10/05/2011 - 1:06pm

Can I suggest another factor that comes STRONGLY to mind when looking at this data set, but defies #FO's rule number one ? I I guess I'm not the only one seeing it.

32
by bubqr :: Thu, 10/06/2011 - 6:24am

Well, didn't take long for someone to go there explicitly. Oh well.

4
by wickerman :: Wed, 10/05/2011 - 1:18pm

Is is statistically significant that 3 of the top 10 on that list are black (#1, 4, 10)? And that there are no black quarterbacks in the bottom 14?

10
by Anonymouss (not verified) :: Wed, 10/05/2011 - 3:31pm

I'd venture to say it's due to mobility in some ways; slow white QBs vs agile Black QBs tends to be the situation. So it may be comparing apples to oranges in terms of trying to wrangle race into the equation.

13
by evenchunkiermonkey :: Wed, 10/05/2011 - 4:42pm

IDK, is it statistically significant that Brett Favre, Tony Romo, and Eli Manning are also at the top of the 'no call list' while Shaun Hill gets more flags?

Do the referees have a crush on Jay Cutler? Would David Garrard fall over in a light breeze? And do they call roughing in the Backyard Wrangler League?

Favre's sexy texts aside, there are a lot of QB's the NFL and other corporations use as the face of the league/brand/whatever that find themselves with no love from the officials, while the QB with the most flags is the guy whos so unlikeable that no one wants him endorsing anything (Cutler) I don't see anything that would really imply favoritism for stars.

But that's really not the substance of the questions you're posing.

It seems that you are implying there may be a racist bent to the officiating, but asking why there are 3 black QBs getting no RtP calls, while the top 14 include only white QBs misses a more significant point. The question you ought to be asking is:

"Is it statistically significant that of the 32 QB's shown here only 5 are black and one Hispanic when the make-up of the rest of the league is so much less lily-white?"

Inevitably trying to answer this question on a public message board will lead to all kinds of rhetoric. You'll know that there will be no further intelligent discussion after you first see the phrase "fast-twitch muscle" after which the conversation devolves into a heated debate centered on evolution with one side favoring white supremacy and another side favoring black power until eventually some one will mention "affirmative action" at which point the white power people will call everyone who disagrees with them "whiny liberals" and the black power people will call everyone who disagrees with them "idiot red-necks" and before you know it all discussion about institutional racism has ceded to a full blown flame war.

And its a shame because the question "Why are only 10% of coaches in college football black men, when 50% of the players are black men?" is more compelling then "When did Jim Tressel know what he knew and what should be done about it?"

15
by Joshua Northey (not verified) :: Wed, 10/05/2011 - 5:44pm

I think in the big picture answer to almost all of these questions is that "black" median family income is 60% of median family income.

Socio-economic status drives almost everything else. Athletics is one of the few fields where it socio-economic status can be transcended because it is so results oriented and not very education dependent.

Coaching on the other hand is less results oriented and more education dependent. Any discussion of problems involving race that doesn't start with the huge difference between the races typical socio-economic status is doomed to failure.

17
by Keith(1) (not verified) :: Wed, 10/05/2011 - 5:51pm

Discussions beginning with any sort of boundary are limiting us as a culture, and it does nothing on this particular football website. We can all agree to disagree and be done with this thread.

26
by Aaron Brooks Go... :: Wed, 10/05/2011 - 11:39pm

I'm not going to overly elaborate, but 100 some years of Olympic games suggests that there is a strong ancestral role in athletic performance, independent of socio-economic status.

This also explains why you don't see any players of Indian descent in the NFL.

34
by Joshua Northey (not verified) :: Thu, 10/06/2011 - 9:11am

I am surprised people cannot have a conversation about this without tip-toeing around it. We are all adults here.

There are certainly racial differences in athletic performance, but the results oriented nature of athletics is also why socio-economic status doesn't stop that higher generic level of performance from being ignored. Particularly the SE factors explain the coaching disparity. Coaches are not using the same skill set, and the position is much less results oriented.

In a lot of professions results are very hard to quantify, so socio-economic status drives hiring a lot more than aptitude.

As just a stupid example when I came out of college with a liberal arts degree from a state school it didn't matter much that I was extremely bright. I graded standardized tests, bounced around from job to job despite always having excellent performance reviews. Then I suddenly got a position at a in the legal department at a bank doing high level work. How did that happen? It happened because the position had a very results oriented structure and thus the selection process mainly involved a test, which I aced.

If it had been a more typical position with a more typical hiring process a early twenties kid from a poor family with a bad attitude and no social connections would have had no shot at the job.

Likewise if you hired athletes with a typical process you would see a much more lily-white NFL.

29
by GBPack (not verified) :: Thu, 10/06/2011 - 12:00am

It may also be worth mentioning that out of 32 teams, 26 start white QBs, or approximately 80%. Given that 32 is a fairly small sample size, this really isn't all that out of line with the 72.4% of people in this country that the 2010 census presumes are white. I feel that people sure like to forget about actual proportions when examining racial outcomes, not to mention the disparity created in other fields through huge gaps in education, experience, and affluence where we somehow expect that there "should" be a certain proportion of race x,y, or z in a given occupation. With logic like this, perhaps we should demand that 72% of running backs be white or that 72% of the NBA should be white. In arguments about race in more intellectual fields than football, people with an axe to grind tend to ignore data about Asians vs. "European-Americans," or whatever we should call people of that ancestry, as this doesn't help their case much when it comes to claims of racial injustice against non-whites. Whether or not this has to do with fundamental differences between groups is largely beside this point. We shouldn't expect equal outcomes when dealing with groups of people from significantly different cultural backgrounds.

20
by John Doe (not verified) :: Wed, 10/05/2011 - 6:19pm

There are three non-white athletes in the bad third of the list, two in the middle, and one at the top third. Considering that there are only six non-white athletes on the list, that the set of all Quarterbacks is small, and that the number of RTP penalties is REALLY small.. I would tend towards it just being coincidence. If one of three at the bad end was on the good end it would be near perfect distribution, deriving racism from such a small deviation from what should be the case on average is stretching it a bit.

21
by The Powers That Be :: Wed, 10/05/2011 - 7:10pm

Um, who's the non-white athlete in the "top" third?

22
by Keith(1) (not verified) :: Wed, 10/05/2011 - 7:17pm

Sam Bradford is Cherokee.

23
by John Doe (not verified) :: Wed, 10/05/2011 - 8:10pm

Shaun Hill is not white. He has a smaller sample size than most, but the sample sizes are all tiny anyway.

25
by Bowl Game Anomaly :: Wed, 10/05/2011 - 10:08pm

Shaun Hill isn't white? I didn't know that, and I rooted for him at Maryland.

33
by John Doe (not verified) :: Thu, 10/06/2011 - 8:43am

Damnit, I was mistaking him for Shaun King.. who is not white. Still, the sample size is far too small to draw conclusions of racism. With so few RTP calls, and such a small deviation, the Refs would really have to work hard to sneak some racism in.

27
by Aaron Brooks Go... :: Wed, 10/05/2011 - 11:56pm

Got any back-up? Ted Hill was white, and I'm yet to see a photo or description of his mother, Trudy, beyond that she's a school nurse.

28
by Aaron Brooks Go... :: Wed, 10/05/2011 - 11:57pm

Got any back-up? Ted Hill was white, and I'm yet to see a photo or description of his mother, Trudy, beyond that she's a school nurse.

5
by Verifiable (not verified) :: Wed, 10/05/2011 - 1:44pm

I was comparing knockdowns to sacks for some of these QB's and got useful (?) some non-surprising results. Ben Roethlisberger takes a sack over 60% of the time he is hit, Peyton Manning takes a sack on less than one third of his hits. Sanchez was worst, of the few I looked at, at getting sacked 62% of the time he is hit. I also looked at hits per dropback, where dropbacks were atts+sacks (not sure if rushes should be in there too). Garrard got hit on over 20% of his dropbacks (ouch!), Manning got hit on 6.7%

6
by Aaron Brooks Go... :: Wed, 10/05/2011 - 1:56pm

Do hits on runs count as QB hits?

12
by Verifiable (not verified) :: Wed, 10/05/2011 - 4:11pm

Based on my look at the data no. Roethlisberger is listed with 156 hits, he had 96 sacks and 86 rushes for a total of 182, which exceeds the hits. Sacks and rush data from PFR

35
by bengt (not verified) :: Thu, 10/06/2011 - 10:55am

That would seem to indicate that he 'gave himself up by sliding with his feet forward' close to 26 times.

7
by Neal (not verified) :: Wed, 10/05/2011 - 1:57pm

My tally has Garrard, McNabb, Campbell and Vick with 640 knockdowns and 14 RTPs, for a ratio of 1 RTP per 46 knockdowns. Everyone else is 1 per 29. In order to achieve parity, you would need to throw two additional flags on hits against each of these four.

9
by The Powers That Be :: Wed, 10/05/2011 - 2:59pm

If you put Josh Freeman on the correct side of the ledger, the numbers move to 50.9 and 28.6. No idea about the statistical significance of that.

30
by Aaron Brooks Go... :: Thu, 10/06/2011 - 12:05am

Poor Jason Campbell. One of these is not like the others.

Other than complexion, he should really be traded with Aaron Rodgers for that list.

37
by Ariel (not verified) :: Thu, 10/06/2011 - 12:26pm

Meh, to think that Jason Campbell and Aaron Rodgers were taken back-to-back.

38
by justanothersteve :: Thu, 10/06/2011 - 3:18pm

And I know a few Packers fans who were upset at the time because they thought Campbell had a better future.

39
by Aaron Brooks Go... :: Thu, 10/06/2011 - 5:20pm

It's still hard to judge Campbell, oddly enough. The poor guy has had a different QB coach every year since his freshman year at Auburn.

Contrast this with Peyton Manning, who has run the same offense every year since his freshman year at Tennessee.

40
by tuluse :: Thu, 10/06/2011 - 6:20pm

I think at this point campbell is what he is. Any effects from all the changing offenses is probably permanent at this point.

8
by Jonathan (not verified) :: Wed, 10/05/2011 - 2:28pm

You need multiple coders to actually look at the hits. If there are multiple coders and intercoder reliability measures provided I might actually buy this as being legitimate. Until then this analysis is not really helpful especially with such a rare event.

11
by steveNC (not verified) :: Wed, 10/05/2011 - 3:33pm

The last column should be labeled KD's/penalties.

14
by Joseph :: Wed, 10/05/2011 - 5:05pm

I don't know how to do the math for standard deviations, but my look at the numbers gives me the idea that the top 6 names on the list (Hass & above) are the only guys that are really out of whack. Every one else looks to be within 1 SD of the average. To me, that says that only those 6 guys aren't getting the call from the ref--and everybody else just needs to remember that training camp is over, and your jersey is the same color as everyone else's.
As far as "correcting Mike P." goes, I think both parties reached the same conclusion (although FO's math is obviously better).

16
by Keith(1) (not verified) :: Wed, 10/05/2011 - 5:49pm

My takeaway from this is that quarterbacks that went to Harvard are favored so much more than quarterbacks from all other schools. Boycotting!

18
by MJK :: Wed, 10/05/2011 - 6:00pm

Brady hasn't drawn one yet in 2011 either, though Ryan Fitzpatrick drew two in their Week 3 game in Buffalo

Brady has actually been hit twice at or below the knee this season (causing every Patriots fan to catch their breath), and hasn't drawn a flag either time. Notably, at least one of those hits did result in a fine for the defensive player. So the league definitely thinks that Brady SHOULD have at least one more RTP flag than he has gotten.

19
by horn :: Wed, 10/05/2011 - 6:10pm

The 2 worst non-calls I can think of were both Carolina v McNabb.

1) NFC_C game, McNabb trips and falls to ground in backfield, defenseless, DE Rucker comes in and spears him, breaking his rib, no flag. Probably the main reason PHL lost that game since it happened Q2.

2) 2009, McNabb rushes into the endzone from 3 yds out, another DL - Damione Lewis hits him while he, again, is lying defenseless on the ground. Broken rib again. While in the ENDZONE.

I've seen McNabb take more late hits the last 10 years than Matt Millen's reputation.

24
by Intropy :: Wed, 10/05/2011 - 8:34pm

Is there a significant variability in the rates at which various officials call the foul? Looking forward to the upcoming Referee adjusted RTP per hit metric.

41
by RichC (not verified) :: Thu, 10/06/2011 - 9:40pm

For pretty much every study I've seen of other penalties, the officiating crew is more important than the teams involved, so I'd be very surprised if this isn't the case.

31
by Eggwasp (not verified) :: Thu, 10/06/2011 - 6:18am

Is the data adjusted for how many times the QB plays the Raiders?

36
by Bee (not verified) :: Thu, 10/06/2011 - 11:58am

No real way to measure it, because it's by nature subjective, but I think part of the question is whether defensive players are going after certain QBs more than others, and if they're getting away with it more against some than others.

I don't think these numbers really tell us anything at all, because we're assuming that every QB gets flagrantly hit at about the same rate, but that might not be true. What if defensive players really are targeting Vick more than other QBs? (I'm not saying they are.) If that's true, and he gets calls at the average rate, then he IS getting ripped off.

Just saying.

42
by RichC (not verified) :: Thu, 10/06/2011 - 9:42pm

What if Vick gets hit worse because he turns himself into a runner and is no longer protected by the QB rules, but doesn't realize that?

43
by RickD :: Fri, 10/07/2011 - 11:42pm

Your analysis is fine, but your intro is, um, weird.


Now, if you're a long-time Football Outsiders reader, that sentence probably inspired the same red flag that came to my mind: pass attempts are a poor denominator. Not only would it shift things in favor of players who didn't throw often, it would also make certain superstar quarterbacks with a proclivity for throwing the ball often show up lower on the chart.

...because the number of QB hits is constant, regardless of how many passes are made?

Peireira error, but the criticism here you imply is itself flawed. QBs who throw the ball a lot are not less likely to draw roughing calls simply because their denominator is high. No, they're less likely to draw roughing calls because most of them have good pass protection (as a necessity) and aren't hit as often per pass plays as the QBs at the top of the list.

Thankfully, you moved on and looked at the proper denominator: QB hits.

44
by Blah (not verified) :: Mon, 10/10/2011 - 9:38am

"Hemmed and hawed" doesn't mean "complained." Am I really the only person who noticed this?

45
by mano (not verified) :: Tue, 01/01/2013 - 7:13am

Walker is out this week http://www.fresh-tests.com/exam/156-215.75.htm with a broken jaw (wierdly kneed in the head accidentially in the week 16 Seahawks game). Not positive how replacing him 156-215.75 tests with non-receiving threat Peelle is going to mess up those plays (fewer TE arounds for positive).

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

More information about formatting options