Where our work appears on mainstream media outlets
NOW DISCOUNTED 40%
Discounted for midseason; buy the PDF version here for just $7.50.Buy print version here for $22.95.
Click here to buy PDF version
Official Account: @fboutsiders
Scott Kacsmar: @FO_ScottKacsmar
Rivers McCown: @FO_RiversMcCown
Ben Muth: @FO_WordofMuth
Aaron Schatz: @FO_ASchatz
Danny Tuccitto: @FO_DTuccitto
Vince Verhei: @FO_VVerhei
-- plus --
Andy Benoit: @Andy_Benoit
Bill Connelly: @SBN_BillC
J.J. Cooper: @jjcoop36
Cian Fahey: @Cianaf
Brian Fremeau: @bcfremeau
Tom Gower: @ThomasGower
Matt Hinton: @MattRHinton
Mike Ridley: @TheMikeRidley
Mike Tanier: @MikeTanier
Matt Waldman: @MattWaldman
Rob Weintraub: @robwein
25 Sep 2006
Not my best effort, but I'm a bit under the weather.
Posted by: Aaron Schatz on 25 Sep 2006
56 comments, Last at
29 Sep 2006, 11:12pm by
I'm sorry, but I don't see how Westbrook with 13 more yards and 2 more touchdowns ends up less valuable than Ladell Betts. Especially when he made his 71 yard TD run with one of the most vicious stiff arms you'll ever see from a 200 lb. running back. I know its just numbers being crunched into the formula, but I don't see it. You'll have to explain that one to me.
I feel the Jets are handling Pennington very well overall, but they should trust him on even the difficult underneath throws. He's not without an arm; he can zip every one of the short and medium passes with the best of them, it's just that he can't translate the speed and technique in those fast throws underneath into the strength needed to get it deep as consistently as he should.
Regarding Dre' Bly (why the apostrophe?), I think he's been consistently underrated for several years. He even made two pro bowls ('03 and '04), but I still don't hear his name mentioned among the elite corners.
#1: One thing is that DVOA does not favor RBs that break huge runs and otherwise are mediocre. Pretty much everything over 20 yards is the same, one way or another - and I think the cap is 40+. So Westbrook doesn't rush the ball well compared to Betts, and there ya go. It's also in comparison to the league averages; if you rush for 5 yards on first down, this is quite a bit more positive than rushing for 3 on first.
But I didn't see either game, so I can't intelligently speak to how the stats are kind of lying in this case.
One thing that is left out of the "[Palmer] spiked the ball to stop the clock, then took a timeout" is the fact that he took the timeout with one second left on the play clock after audibling at the line of scrimmage. No, it doesn't make it any better, but it doesn't make him look like an idiot either.
i don't know much about QB mechanics, but i'm glad i wasn't the only who noticed that Grossman takes a ton of steps on his drop.
Shout out to Kellen (a.k.a. "the much maligned") Winslow 2.
I sure hope Maurice Jones Drew keeps this up. He looked like the real deal running outside, but then the Indy Dome amplifies his strengths. He's short but weighs 220 - he could hold up to Warrick Dunn level carries as an everydown back I think.
I wonder if DPAR takes into consideration the 15 yd unsportsmanlike conduct penalty after Winslow's first catch...
Aaron, Feel better.
I am surprised that Manning, Brady, and McNair are all as high as they are. Is is the lack of turnovers from the three seasoned vets? I initially looked at Palmer's 4 TDs vs Pitt and envied them. Then I looked at his TO line and rethought things.
Aaron, any chance you can re-adjust the ratings for just this one week to include ass-kicking toughness, nudge little Simms up a dozen places or so? Easy to add in a category such as "played despite internal organ failure" worth 6-8 PAR pts? Hell, leave it in for good, since it would only be used once a decade, if that often.(NOTE: This would not apply to brain cramps, Mr. Warner and Mr. Culpepper)
Easy to add in a category such as â€œplayed despite internal organ failureâ€? worth 6-8 PAR pts?
This is clearly discriminatory against ROBO players, who have no internal organs.
On Westbrook versus Betts: Betts had 16 carries to Westbrook's 8. That's double the opportunities for successful plays, which has to carry a lot of weight in DPAR.
Centrifuge: Good point, but ROBO-players sneer at your pity. They need no breaks, you lowly mortal. If they had salivary glands, they would spit at your sympathy.
Hey, I just noticed Peyton Manning has the highest rushing PAR of any QB this week. How often does that happen? No need to tell me. Maybe, just maybe, the last time was the week he had his 33 yard naked boot against the Bills in 2001 (I think). And it was all downhill after that with 5 consecutive losses. But if that happens again this season, the message will be clear: keep him from running so damn much!
I bet ROBO-QB has pity for Eli's DPAR through the first 3 quarters.
A question about PAR at the end of halves: it doesn't take into account getting out of bounds, does it? It occurred to me to ask while watching tonights Saints-Falcons game. At the end of the 1st half, the Saints are at roughly midfield and trying to get into FG range with about 20 seconds left and no timeouts. Reggie Bush catches a pass at about the line of scrimmage, maybe 10 yards from the sideline. Instead of running forward and gaining, say, 8 yards, he sprints diagonally to the sidelines and goes out of bounds for a two yard gain.
I'm guessing that DPAR and DVOA penalize him for that choice, even though it was actually a very smart play. Does anyone know the story on that?
9: For Manning, McNabb and Brady's high DPAR, I'm sure it had a lot to do with no turnovers and very few sacks. I think Brady only got sacked once, and I'm sure the other two didn't get sacked much either.
I love the work you guys do, but Brady's numbers are one of the reasons these measures can lie about the impact on the game. He was 2-11 on third down conversions, Got 80 yards and 10-10 on the TD drive when denver played pure prevent Defense with no DB's within 7 yards of the LOS, and his ONLY TD of the game and scoring drive of the game.
Conversely, Denver's defense, which did something no other team has done since 1942 in not allowing a TD in the first 11 quarters of the season, will get penalized for playing Prevent pass defense in order to win a game.
The strategy the coaches used to secure a victory, which is the ONLY thing that matters to them, makes Denver's defense look weaker than it was against the PATS. They traded yards and points for time, a strategy I despise but can understand when leading 17-0 in the fourth quarter.
Overall, when you measure each plays success, teams that are leading by large margins in the fourth quarter ala Denver and SEA last week get penalized for playing winning football. And, QB's that have played horrendously for three quarters get inflated numbers thanks to those defenses using bend but do not give up a big play strategy.
It is one of the primary reasons the Broncos were 13-3 last year and were not as dominant in Defensive VOA being third in points allowed in the NFL. They held the lead in 14 of 16 games entering the fourth quarter last year. Teams were forced to throw the ball and go for it on fourth downs. Those fourth quarters ate up a lot of the VOA compared to the first three.
I'm pretty sure that (D)VOA and (D)PAR take all of that into consideration. You can't just completely ignore garbage time stats, but IIRC they aren't weighted very heavily.
And, QBâ€™s that have played horrendously for three quarters get inflated numbers thanks to those defenses using bend but do not give up a big play strategy.
The entire complaint would make a lot more sense if the system didn't have an innate adjustment for this, considering you're comparing things to the average performance of QBs who are down by 17 in the fourth quarter, and Brady still beat that average.
He was 2-11 on third down conversions
Average yardage to go on those 3rd downs: 8 yards. He wasn't exactly being given manageable 3rd downs for most of the evening.
Got 80 yards and 10-10 on the TD drive when denver played pure prevent Defense with no DBâ€™s within 7 yards of the LOS, and his ONLY TD of the game and scoring drive of the game.
If Denver was really playing prevent that entire drive, they're idiots. That was at the beginning of the 4th quarter. 2 TDs and a field goal are completely reasonable in an entire quarter.
Yes, they do factor garbage time, but obviously not well enough that Brady would be a top 5 QB in their projections.
And, not enough that a defense that was ahead by 10+ points going into the fourth would be valued as low as they were in VOA.
IMHO, the quantatative numbers have not caught up enough to the qualitative aspects of the game. Especailly, when attempting to measure CB's in coverage by who was closest to the ball and not by knowing the coverages they were playing.
At 13:03 in the 4th quarter, the Pats had the ball, losing 17-0. They then scored at TD at 9:18. I'd argue that their chances of winning went up significantly by scoring, and that Brady's DPAR numbers should reflect that.
Of course, if Denver really was in a prevent defense, then that should be taken into account, if possible. Does the game charting project track prevent defenses, and if so, is there a noticeable change in DVOA for prevent defenses vs. non-prevent defenses?
I don't think Brady should be considered a top 5 QB this week, since he threw a lot more than the other QB's on the list. On the other hand, in the context of when he was throwing (again, excluding the effect of prevent defenses), he was above average.
So, is it too early to start start the 'McNabb for MVP' bandwagon?
Through 3 games (albeit against 3 bad teams):
960 yds (1st in the league)
8.50 ypa (2nd behind Sexy Rexy)
7 TDs (2nd behind Eli)
1 INT (T-5th with more atts than anyone he's tied with; behind Huard & Rivers with less than 50 atts each, and Bulger and Alex Smith)
105.3 Rating (3rd behing Carr and Rivers).
Anyone got any thoughts about the potential advantages of taking the 5 yard penalty and preserving your TO for later?
I realize it's impossible to know early in the 3rd Quarter that you will need the TO at the end, but I wonder if there aren't certain situations in which you might just sacrifice the yardage. Any thoughts?
That has long been a pet peeve of mine as an Eagles fan where timeouts are frequently burned because the plays don't get in on time. To me, the situational difference between say 2nd and 11 and 2nd and 16 is definitely not worth using a timeout. The difference between 3rd and 1 and 3rd 6 is another matter, however. I would love to see some sort of quantitative look at this.
It's not too early for "McNabb for MVP" based on his raw stats, but he's put up those numbers against the Texans, Giants, and 49ers, all lowest-tier pass defenses. If he keeps this up through the Week 5 game against Dallas, he's the frontrunner.
Remember, DPAR rewards players with a lot of throws, rushes, and receptions. I bet Brady's DVOA is lower because his completion percentage isn't great, espeically on 3rd down, but he had a lot of attempts and those attempts netted a lot of yards, first downs, and the Patriots' only points. At the same time, he did nothing to hurt the Patriots chances--no interceptions, no sacks, and probably the 3rd down conversion rate wasn't that below average considering what he was facing. Hence, a good PAR. It's probably deserving given that Brady was the ONLY offesne the Patriots had going that day--the Broncos completely shut down their run attack.
The PAR system is telling us something along the lines that only four QB's played well enough this week to do better in the situation Brady found himself.
Incidentally, it wasn't quite the way you paint it. The Broncos weren't cleverly playing a prevent D to trade points for time. As some people have pointed out, that's stupid with only a 3 score lead and an entire quarter to go. Their D had been playing hard all game, and finally got tired, and the Patriots switched to a hurry up to exploit that. They actually had the Broncos D so exhausted that players were throwing up on the field! I'm pretty sure that was not a calculated tactic. If the Pats D had then forced a 3-and out after scoring rather than letting the Bronco's Offense drive halfway down the field, then the Broncos D wouldn't have been as rested on the next drive and things might have turned out differently. Or a bad turnover break could have happened for the Broncos. Going prevent and letting the Patriots score at that point would have been stupid, and I'm giving Shanahan more credit than that.
Re 23 and 24:
I bet Bill Krasker's model (at footballcommentary.com) could say something about timeouts versus yards. As I recall, he's quatitatively estimated the value of 5 yards of field position, and of a timout, as a function of current field positions, score, and time remaining, in terms of probability of winning the game. I don't know if his model is finely graded enough to judge different down and distance situations--I think he just estimates probabilty states at the beginnings and ends of drives--but it might be a place to start.
what about Brian Westbrook?
in 3 games he has 132, 124, and 164 combined yards and 5 TDs.
Of course, if Denver really was in a prevent defense, then that should be taken into account, if possible.
Why? You'll never be able to tell "moron defensive scheme" from "bad defense" anyway, and how is it bad for a QB to take advantage of a bad defensive scheme anyway? Isn't that what good QBs do?
If Brady didn't deserve to be a top 5 QB this week, who deserved to be there instead of him? All the QBs below him either had more turnovers and sacks, or a lower completion percentage. Overall, it wasn't a very good day for QBs.
I thoroughly respect and understand the work that FO does. They have created some wonderful analysis tools and I have read the Prospectus from cover to cover. The point I am trying to make is that statistics sometimes fail to tell the story accurately. The analysis of what happened and Aaron's own reaction in his comments about Brady bears this out.
However, your condescending tone imparts some form bias into your response. Calling Denver's coaching staff idiots after completely shutting out three NFL teams from Scoring a TD in 11 quarters and 10 red zone possessions is not only inflammatory but infers you could do better and are smarter.
Now on to the relevant parts of your post:
1. Yes, they did play pure Prevent on that drive, and they would only be statistical idiots for trading yards and points for a finite amount of time and opportunities. They took NE completely out of their gameplan through three and a half quarters, and somehow Brady is a top 5 QB on the day?
2. That Brady had 8 yards to go on third down is a credit to the defense on the first two downs as well. Brady having incomplete's and short dumpoffs on first and second down and being ineffective there as well contributed to a lot of those long third down's.
3. Finally, Denver does not have a pass rush from the front four alone. They had the worst adjusted sack rate in the NFL last year. So color me unimpressed when Brady beats that prevent going no huddle.
As I said earlier, I like FO and applaud their efforts. But sometimes no matter how well a metric is set up, it tells an incomplete and tainted story.
The charting project does indeed consider how many players rush in any given pass play, which would help infer prevent defenses. It's not perfect, but it helps.
Average yardage to go on those 3rd downs: 8 yards. He wasnâ€™t exactly being given manageable 3rd downs for most of the evening.
So that would give him high PAR but low (negative?) VOA, right? Do we assume a replacement-level QB can't get an average of 8 yards on non-manageable 3rd downs?
It's an interesting discussion. I do believe Brady's PAR should be relatively high, considering the situation - going on impression, haven't looked at the numbers. However, his VOA should be low, and his DVOA slightly higher; Denver's defense was good, IMO...
I agree with most of your points, but I wonder about #1. It has nothing to do with Brady's worthiness as a top-5 QB for Sunday, BTW...
Do you believe a defense playing prevent early in the 4th, ahead by 17, is good coaching/scheme? Obviously this goes coupled with an offense set to kill the clock, but it seems to me that playing prevent THAT early in a game would be questionable, at best - considering, if you allow a score in, say, 5 minutes, the opposing offense still has about 10 mins to score 10 points... not entirely impossible, even considering you're trying to kill the clock.
Just a thought.
Sergio, I am not happy they employed that strategy at all. I think the prevent says we do not trust you to make the plays you have made all day. It requires changing from an aggressive to passive mode that most defenses can not handle and the players make more mental errors trying no to screw up, than just playing.
BTW, I think Tom Brady is a phenomenal QB. He is the definition of elite IMHO as he makes average players good.
The whole point of my post is that when the coaches take the foot off the pedal, whether ill advised or not, it skews the data. The data is no longer as accurately measured when defenses will gladly give up a five yard completion underneath and in bounds.
This is where it gets tricky. How can you account for that accurately? This is where a measure of qualitative analysis could be implemented IMHO. A mixed method approach would certainly render the data more accurately in that situation. However, I understand FO's approach in why they do what they do and how they measure. It is what it is.
I think you're confusing PAR and VOA. PAR is a measure of production, not value. Brady having the 5th highest PAR isn't saying that he was the 5th best QB on Sunday. It's saying that he contributed the 5th most production. That'll happen when you have 10 more attempts than the 2nd highest guy (Alex Smith). It's also the reason why, when DVOA becomes available, the tables are sorted by DVOA and not DPAR.
Calling Denverâ€™s coaching staff idiots
I wasn't calling them idiots. I said they'd be idiots if they ran a prevent defense while only up 17 points at the beginning of the 4th quarter.
I don't buy that they did. It certainly didn't seem so watching the game. Given that the scoring drive only took four minutes, that's not exactly a useful trade of time for points. With 9 minutes left to go, needing a TD and a FG, four minutes per TD drive puts you in a very dangerous situation.
It looked more like Denver got stuck in a bad situation because New England went to a no-huddle offense and Denver was left without the proper defense on the field.
"Why? Youâ€™ll never be able to tell â€œmoron defensive schemeâ€? from â€œbad defenseâ€? anyway, and how is it bad for a QB to take advantage of a bad defensive scheme anyway? Isnâ€™t that what good QBs do?"
DPAR and DVOA already adjust for defenses, because you can't evaluate an offense without acocunting for the defense it faces. In principal, if the data is available, there's no reason not to account for how the defense is set up. There's already an observed difference in performance based on the down and the number of rushers. With a few more year's worth of data, there's no reason not to include this information, especially since it helps to distinguish between the players on the field and the coaching on the sidelines.
In principal, if the data is available, thereâ€™s no reason not to account for how the defense is set up.
What I meant was that you'll never know how the defense is set up without knowing the defensive alignments the coaches called. You can easily have two alignments that look identical, but where the players have different assignments and are a totally different defense.
That is, you'll never tell the difference between bad defense - someone misinterpreting their assignment - and a bad defensive scheme - i.e. everyone executing their assignments but the offense taking advantage.
Daytwa tends to play better when Capt Bly (ask Joey) is out of the lineup
Re Maurice Jones-Drew. Is height or weight or speed a more important characterisitc of an NFL RB?
Here's a list of the 9 RBs who finished in the top 10 of the RB DPAR rankings more than once in 2003-2005, plus Drew and two others:
I don't have the speed numbers, but Jones would probably be in the upper half of the list.
The backs listed above:
Name HT WT
Alexander 5-11 225
James 6-0 220
Rudi Johnson 5-10 228
Larry Johnson 6-1 230
Barber 5-10 205
Tomlinson 5-10 221
Portis 5-11 212
Martin 5-11 210
Holmes 5-9 213
Jones-Drew 5-7 212
Taylor 6-1 226
Westbrook 5-8 203
If height matters for an RB, Jones-Drew would be an outlier. But if weight is more important, then Drew is in the midst of many good backs.
I strongly disagree that Denver was playing prevent. What you were seeing was a situation where Denver's fourth CB was playing on the outside due to injury, and they had him playing well off the line so Gabriel couldn't get behind him.
At the time, New England had all three time outs. To go into a Prevent at the beginning of the fourth quarter absolutely would be idiotic. New England got the ball back twice more in that game, with time stil left on the clock. I know the Broncos D had been stopping the Pats O all day, But there's no way that Shanahan calls a Prevent that leaves the ball in Brady's hands down by ten with (potentially) nine minutes left.
What you were seeing was a situation where Denverâ€™s fourth CB was playing on the outside due to injury
Man, that sounds familiar! Sounds like Philly's strategy versus the Giants!
Clearly, Tom Brady is worse than Eli Manning.
Thank you for clearing that up. That said, it doesn't make me as optimistic about Gabriel knowing that...being able to beat Denver's 4th CB does not translate into being the #1 1/2 WR the Pats need him to be. Since that was the first significant PT he'd been getting, I thought maybe he was actually doing something good, but it sounds more like he was just exploiting a serious mismatch.
Karl Paymah clearly constitutes a serious mismatch.
ABW (#43 )--
Getting a wide receiver on the field who can reliably exploit a favorable match-up, would be a significant step for the 2006 Patriots' offense.
Concerning the prevent defense:
1. It all depends on your definition. Is prevent 3 pass rushers dropping 8 or can 4 pass rushers with 7 dropping be prevent? Answer: its both. It depends on the style of the coverage and the depth adjustments to those coverages.
2. On the second play of that drive, Reche Caldwell committed PI on a play Bailey had position to make an interception. The throw looked a little outside, but I have seen Champ make more difficult INT's. The Pats committed a penalty to stop a TO IMHO.
3. Denver plays off man coverage, almost exclusively, with a 5-7 yard cushion. On that drive, they presented themselves with 7-9 yard cushions on the outside exclusively. They rushed no more than 4, except on a five man zone blitz. No Safety was within 12 yards of the LOS at the snap, and twenty at the time of the throws.
4. Al Michaels has no idea what Zone coverage is. Denver played red Zone Cover four on the TD. Paymah played his Zone. Brady made his best throw of the day splitting Lynch and Gold. Good Under route by Faulk to clear the other LB Williams from the spot Brady threw the ball.
I post here because people are open to seeing things in a different light using metrics as a tool. It does not replace other tools like game film and performance grading. The limitations on the statisitical measures will always miss the opportunity cost of not making the correct plays when they are there to be made. That was my point in going in this direction.
I will never be upset with this kind of analysis, it is groundbreaking. I also will not forego other methods and replace them with such a young model. FO is a great service to a bunch of people who want to get more in depth than others.
Regarding post 46:
It should read, "I also will not completely forego other methods" in the final paragraph.
So let's see â€¦ Ladell Betts is playing great, Clinton Portis is feeling much better â€¦ what was the point of the T.J. Duckett trade again?
They had to figure out SOME way to send their draft picks to Denver this year. Apparently, Tags and/or Goodall nixed the idea of just sending them over.
Just a little note defending both Carson Palmer and Ben Rothlesburger...
If I am correct, all their interceptions came throwing against a very brutal wind and neither made any bad passes with the wind, which was where Palmer threw his TD's. Shouldn't that be considered somewhat?
Corrolary to #49, how much will the later adjustments for good defenses make if after six or eight games the steelers are one of the top defences in the leauge and the Bengals are on top in number of intereceptions? I though Palmer won the game for the Bengals, and that perhaps actual points scored should be even more of a factor? Carson had a crappy day by some measures but when they were unable to drive the ball at all he connected on some mid-range touchdowns that provided the points needed to win the game.
taking out Brian westbrook's long carry of 71 yards
7 carries 46 yards
taking out ladell betts longest carry of 26 yards
15 carries 98 yards
There is no way in hell Betts had a better game than westbrook, not a chance.
This really does prove that these numbers need tweaking
Wow how can you agrue that Denver was not running a preventdefense on the patriots Td drive.
That was CLEARLY PREVENT!!!!
now was is desicion to play prevent...clearly another story
but serisosly if you watched that game and didnt think that was prevent then you have no idea what you are talking about
aaron, in quick reads under "How PAR works" you have the following descriptive entry:
Players receive bonuses when they play well against good defenses, and they don't get rated as world-beaters when they shred the 49ers.
Isn't that actually a description of DPAR? Especially on the FOX entries, it might be a little confusing to someone who is not familiar with the outsiders site and runs across DPAR and DVOA later in the season.
Plus, it may be about time to insert "Texans" in place of "49ers".
Iâ€™m sorry, but I donâ€™t see how Westbrook with 13 more yards and 2 more touchdowns ends up less valuable than Ladell Betts.
It's PAR, not VOA. Betts had more good runs than Westbrook did, just because he rushed more, and there's value in being a player that can handle a load.
Besides, the difference is 0.4 PAR. Basically it's the same - Westbrook contributed a lot on a few drives, Betts contributed less, but on more drives.
Rushers sent by Denver on the "prevent" drive:
4, 4, 4, 3, 4, 6, 4, 3, 4, 4, 4
I always though prevent was thee rushers and everybody else 25 yards back. Denver had two safties deep, two linebackers and 3 corners within 7 yards of the LOS. Now, they were playing the WRs loose, but it's not quite a prevent defense, unless you consider the cover-2 to be prevent.
In this week's Varsity Numbers, Bill Connelly takes a page out of baseball's playbook and attempts to isolate power from efficiency.
See All XP | NFL XP | College XP
© Football Outsiders, Inc. // site design by B:COMPLEX Creative :: site architecture by Grossmont Designs // Partner of USA TODAY Sports Digital Properties