|
AMERICA'S BEST-SELLING 
Click here for more info. KUBIAK UPDATE ROTOWORLD.COM The perfect complement
 |
» Printable version
Why Doesn’t Bill Polian’s S–t Work in the Playoffs?“My shit doesn’t work in the playoffs.” - Billy Beane, as quoted by Michael Lewis in Moneyball Let’s just get that quote out of the way — you and I both know that it’s an essential part of any sabermetric discussion of playoff performance. Beane’s Axiom doesn’t just apply, of course, to the Oakland Athletics, Jeremy Giambi, the playoffs, or even baseball; it’s a simple way of pointing out that the variance over a seven-game sample size is impossible to control for in the same way that you can over the course of a 162-game season. Beane’s statement is as hopeless as it is simple. It’s also incorrect. My colleagues at Baseball Prospectus analyzed Beane’s statement in last year’s Baseball Between The Numbers, in a chapter titled, naturally, “Why Doesn’t Billy Beane’s Shit Work In The Playoffs?” You can read an edited version of the chapter on ESPN here. What Nate Silver and Dayn Perry found were that the traditional ideas of what led to playoff success, namely pitching and defense, were correct. The three variables that were the most likely predictors of postseason performance were the performance of the team’s closer, the strikeout rates of their pitchers, and the performance of the team’s defense. Offensive performance showed no such significance in predicting postseason success. It’s with this work in mind that I decided to do the same research on predicting performance within the NFL playoffs here at Football Outsiders. While the work done on baseball looked at data from 1972 on, we at FO currently have DVOA figures going back to 1997, leaving nine years of playoff performances to be analyzed. While the traditional data is available for seasons before 1997, our advanced DVOA metric does a better job of adjusting for context and proves a truer measure of teams’ abilities and performances. (For anyone new to Football Outsiders, our DVOA stats break down every single play, measuring success towards both a touchdown and a first down, and compare to a league baseline based on situation and opponent. More is explained here. “Weighted” DVOA is a version of DVOA that gives more weight to late-season games over early-season games.) For example, there’s been a lot of talk this year about how the Colts have the worst rush defense, perhaps in football history, with their high yards per carry allowed given as the reasoning behind such a claim. That ignores the fact that the Colts are often ahead in games and are putting their defense in situations where the average team gives up more running yards and yards per carry than they would if they, say, were the Raiders and losing all the time. DVOA measures a team’s performance versus the average performance of a team in the same situation, as opposed to all situations. Of course, the Colts still can’t stop the run — they rank 31st in run defense DVOA — but they’re not even the worst team in the playoffs. That honor goes to the Jets, who profile as slightly worse than Indianapolis. To measure the success of each team within the playoffs, I created a quick and dirty metric similar to Nate and Dayn’s Playoff Score Points. The idea is to reward teams that win the Super Bowl over all else, but also to note the performance of teams who perform well without doing so. In that vein, each team was given:
I am amenable to the idea that this system might punish teams that get a first round bye slightly, but at the same time, those teams simply didn’t need to perform as well once they made it to the playoffs as a team that went from the Wild Card to the Super Bowl did. Furthermore, a team that wins two home games and the Super Bowl would earn nine points; a team that won three road games but lost the Super Bowl would receive the same figure. I don’t think it’s outlandish to suggest that both of those teams played very well. The maximum number of points a team could receive would be 14, which Pittsburgh achieved last year; the minimum, obviously, would be 0. 40% (44) of the teams who’ve made it to the playoffs over the last nine years (108) have not earned a single point. After compiling the Playoff Score Points for each playoff team since 1997, the numbers were then correlated against seventeen of our metrics: Offensive and Defensive DVOA (both weighted and overall, as well as rushing and passing-specific numbers for both), Special Teams DVOA (again weighted and overall, as well as the individual kicking/punting unit statistics), and finally, Overall and Weighted DVOA. What did I find? In short, that team defense rules the roost and a team’s momentum ending the season has no effect in the playoffs. Take a look at these correlation coefficients:
(Remember that the defensive correlations are working with performance metrics where a negative number is preferred, so an inverse correlation would actually be preferable.) As you can see, defensive performance is a stronger predictor of playoff success than offensive performance across the board. While the defensive figures aren’t an exact correlation or close to it, remember that the small sample size of the NFL, even across the 16-game regular season, does normally result in lower coefficients and more variability than in other sports. In addition, a team’s weighted DVOA is no better at predicting playoff performance than its DVOA over the course of the season as a whole. Overall, a team’s DVOA over the course of the entire season (.317) had a stronger correlation with playoff success than its weighted DVOA (.264). (Ed. note: Originally here there was a whole section about teams on five-game winning streaks not winning Super Bowls, except we apparently did something wrong with the research and missed a couple of streaks, so I just took it out. Nothing to see here. — Aaron) With regards to special teams, while having good special teams clearly doesn’t hurt, it was surprising to find what particular unit didn’t really matter:
That’s right — apparently, having a good field goal kicker and/or a good kickoff man is entirely irrelevant to playoff success. This seems to jibe with Aaron Schatz’s research in the New York Times about the lack of consistency in kicker performance on field goals. (Ed. note: Then again, the same article pointed out that kickoff distance is one of the most consistent stats in the league, so I don’t know if that article and this one are really related — Aaron) OK, let’s summarize so far:
While those are some good overarching principles, there’s more to be said about predicting postseason performance. In their piece, for example, Nate Silver and Dayn Perry pointed out that pitching is more important than hitting, but specifically that having a good closer and a staff with a good strikeout rate is crucial. What does Football Outsiders have that analyzes performance of the same level of precision? Well, how about the brand spanking new Football Outsiders Premium Stat Database? What’s it got, you ask? Well, DVOA for each team during each season split about 45 different ways both offensive and defensively, that’s what! It has each team’s performance separated by down, distance, score gap, field zone, season split, home/road, and even close and late (for you Tom Brady fans out there) situations. Remember Bill James’ Favorite Toy? This is my new favorite toy. (Ed. note: The beta version of the Premium Stat Database available for free during this year’s playoffs doesn’t have all these splits in it yet, but this gives you an idea of what’s coming next year. — Aaron) Bringing the splits into the discussion provide some talking points for analysis and some insight on the upcoming playoffs. OffenseSplitting the offensive data gave even more credence to the idea that regular season offensive performance has little or nothing to do with playoff success. The strongest offensive correlation (Third Quarter DVOA, at .147) would be the 20th strongest correlation if it were a defensive stat. Other, seemingly random statistics join Third Quarter DVOA at the top of the list: Second-and-Medium DVOA, Red Zone Passing DVOA, DVOA in the “Middle” Zone, and Offensive DVOA at Home. The prevalence of these unrelated statistics as the “strongest” correlations would speak more to a small sample size than to any sort of predictive value. DefenseOn the other hand, the defensive splits reveal several fascinating aspects of what’s consistent with playoff success, including items and ideas that are consistent with what we’ve found in analyzing season-to-season performance. Case in point: The weakest correlation between defensive performance and playoff success? Third-and-long DVOA (at .110; remember that a perfectly accurate defensive metric would be -1, so this is actually a correlation of -.110), followed by Third/Fourth Down DVOA and Third-and-Short DVOA. In much the same way that success on third down isn’t predictive of future success, the same holds for a team’s postseason performance. Another example? Well, FO’s 15th Precept says, “Championship teams are generally defined by their ability to dominate inferior opponents, not their ability to win close games.” (For more information, please read this article by Aaron from 2005.) That follows, to an extent, in the playoffs. A team’s defensive performance while winning a game by 9+ points (-.227) is almost exactly as predictive as a team’s defensive performance in games that are close and late (-.230). While this doesn’t represent a team’s ability to get into situations where they’d be up by a large margin, it does display their ability to keep the score that way. What correlates strongest with postseason success? Actually, how a team does on first down:
These correlations are significantly stronger than those on the offensive side. The data seems to point to two skills being predictors of post-season success:
The prevalence of Road DVOA being important can be considered a form of selection bias — after all, teams who have the highest Playoff Score Points will have played well on the road — but this shows that there is some predictive ability for regular season DVOA on the road when it comes to the playoffs. Well, on the defensive side at least; the correlation between PSP and Offensive DVOA on the Road is a measly .018. Now, the important question: What does this all mean for the 2006 Playoffs? Obviously, a greater correlation between defense and winning championships doesn’t mean that an offense-first team CAN’T win the Super Bowl. But given the lack of any reliable or significant relationship between offensive performance in the regular season and playoff success, it’s hard to recommend teams like Indianapolis, New Orleans, the New York Jets, Seattle, or Dallas. Now, you might argue that Dallas’ defense (16th in DVOA) doesn’t belong in a group with the rest of those teams. When it comes to looking at their overall defense, perhaps; but once you break the unit’s performance down into its particular splits, it reveals that Dallas is the team most uniquely suited for a failure in the postseason. Remember that list of the ten strongest correlations between defensive performance and playoff success I listed above? Here’s how the Dallas defense ranks in each of those categories:
As you can see, Dallas has a terrible defense on first down (they are above-average on third down), and is poor at stopping teams inside the red zone. Not even Carrie Underwood and a delicious hamburger can help that. Let’s look at the top seeds in each conference, along with that defensive juggernaut in Baltimore, and one mystery team:
The team whose shit might, in fact, work in the playoffs? The New England Patriots. They do extremely well in each of these categories except for rushing DVOA over the second half of the year. Does this mean the Patriots are the favorites? No. They’re still most likely going to have to play a playoff game at either San Diego or Baltimore, and that could very well be too much for them to overcome. But if they do make it through the AFC minefield and are hoisting up the Vince Lombardi Trophy on February 4th, well, you can tone down the platitudes about respect from Rodney Harrison and Tom Brady’s clutch gritty manly leaderness by the media and tell your friends that the Patriots are a team just built for the playoffs — and that you knew all along they wouldn’t be facing Dallas. (As a final note, I want to point out one of the interesting ramifications of this research. We previously discovered that defense varies from year to year more than offense does. Combine that idea with this idea and you are left with a dilemma for general managers: Focus on offense, and your team is more likely to be in the playoff hunt every year — but less likely to actually win it all. Which is more important — getting into the tournament, or having a better chance to win it once you get there? I’m sure we’ll be continuing this conversation in Pro Football Prospectus 2007. — Aaron) 168 Comments » | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
I haven’t read the article yet, but I wanted to say that based on the title … to paraphrase Cameron Crowe …
You had me at s–t. You, had, me, a, s–t.
:: RF — 1/4/2007 @ 12:31 pm
If you are a GM or a coach I think there is no question that it is more important to build a team that can get into the playoffs than to build a team to win once you are there, at least if you’re worried about job security.
:: Harry — 1/4/2007 @ 12:40 pm
Wow, as a Patriots fan, you give me some hope!
But it makes me fear Baltimore the most.
:: Athelas — 1/4/2007 @ 12:40 pm
Not sure why they weren’t included, but the 2000 Ravens finished the regular season on a 7-game winning streak.
:: Jimbo — 1/4/2007 @ 12:42 pm
Typical FO New England Homerism.
:)
Great article. It’s interesting how much Baltimore is favored over SD using this system, but more interesting is how dominant NE is here.
Where does NO rank? Heck, can we get the splits for all the teams and what their overall “SB worthiness” is?
:: kal — 1/4/2007 @ 12:43 pm
Is there anyway we can see the stats (as in the ones that correlate to playoff success listed above) for the other playoff teams?
:: TheWedge — 1/4/2007 @ 12:45 pm
Sorry, Bill missed the Ravens… will add. Just to make sure people understand: This doesn’t tell you who WILL win, it tells you who might have a better chance than their overall DVOA or win-loss record might indicate. And of course, we’re not saying this is set in stone or that this is the last time we’ll look at this research.
:: Aaron Schatz — 1/4/2007 @ 12:48 pm
Wow. Nice work, Bill. You’ve come up with the a solid statistical basis for “Offense wins games, defense wins championships” here.
Of course, something confuses me about the offense/defense logic here. The teams which are best known for coming up short in the playoffs in recent seasons are the Colts, Eagles and Steelers. The Colts certainly fit the model. The Steelers equally obviously don’t - they’ve been a consistently strong defensive team who were generally torpedoed by quarterback play worthy of Rex Grossman. The Eagles haven’t been (according to DVOA, anyway) consistently stronger on one side of the ball than the other under Andy Reid.
This is probably just small sample size, though. Specific examples don’t work so well.
:: Ilanin — 1/4/2007 @ 12:54 pm
Nice article, Bill.
I agree that it would be neat to see the stats for all the playoff teams–I bet Philly looks really good.
:: Joshua — 1/4/2007 @ 12:56 pm
Ahhhhh!
Another long year as a Colts fan living in New England?! I can see it now…
Please say you’re just kidding…
:: Purds — 1/4/2007 @ 12:58 pm
When you measure a correlation coefficient from a sample of data, it comes with an uncertainty. Offhand I’d guess most of the uncertainties in the coefficients measured here are on the order of +-0.10, though you should be able to calculate it directly from the data you have.
When you have many quantities, all with significant uncertainties, and then you choose the largest or smallest of the quantities, you often end up looking at noise. I imagine that, for instance, the difference observed between the effect of first down DVOA and red zone DVOA is probably small compared to the uncertainties on both of them. (Though you can calculate and tell me if I’m wrong.)
On the same note, how statistically significant is the observed difference of 0.156 between defensive and offensive correlation?
:: Andrew Foland — 1/4/2007 @ 1:00 pm
Didn’t the NE Pats also go on a 9 game streak to finish 2001?
:: Tones — 1/4/2007 @ 1:00 pm
Nice article. I’m feeling OK about my Baltimore wins it all prediction now. They beat your dark horse New England team in 5/7 defensive metrics. I don’t think Rivers will perform nearly as well as he did early in the year. His arc this year will be very similar to Big Ben’s first dramatic tragedy in the playoffs. They’ll win a tight game against the Pats and then fall to the more experienced Ravens in the championship. I’d buy into the Pats darkhorse theory, but they didn’t handle Jason Taylor well at all and will be unable to stop Merriman.
No NFC team will be able to stop the Ravens.
:: Viva Pedro — 1/4/2007 @ 1:01 pm
#8
The Steelers/Eagles could have just been unlucky and run into better teams. They could have just been the second best team every year.
:: TheWedge — 1/4/2007 @ 1:02 pm
To look at this on a more macro level, no team has won the Super Bowl without winning at least 11 regular season games since the 1988 49ers, who won 10. Since the NFL went to a 16-game schedule, they’re the only Super Bowl champ with fewer than 11 regular-season wins (aside from the 1982 Redskins, who went 8-1 in a strike-shortened season).
If that holds, the only teams with a shot at winning the Super Bowl this year are the Chargers, Ravens, Patriots, Colts and Bears. The Bears are the only NFC team that even has a chance, which I think is probably not such a controversial notion.
:: Boots Day — 1/4/2007 @ 1:06 pm
Nice work. Just took a look this morning and the DVOA’s for the Super Bowl teams since 1997. The losers on average actually had better offenses than the winners, but the winners had much much stronger D’s. Only the ‘03 Pats (in one of the biggest upsets of all time) and the ‘98 Broncos (with a pretty darn good offense) won the Big One with a D rated lower than 6th overall.
:: Steve — 1/4/2007 @ 1:13 pm
When you measure a correlation coefficient from a sample of data, it comes with an uncertainty.
Not without either a resampling technique (see here) or the assumption of normally-distributed data, which is almost guaranteed to be wrong (only teams which reach the playoffs are involved - therefore, there’s a cut on things which lead to regular season wins).
Using the bootstrap resampling tool on that page would probably be an interesting idea, but the small size of the data set might limit things.
:: Pat — 1/4/2007 @ 1:16 pm
#14: Like the Colts, in 2003, 2004, and 2005, who lost to the Super Bowl winner in the playoffs every year.
:: Pat — 1/4/2007 @ 1:18 pm
as a fan, i’m all for the “make the playoffs every year” school of thought. “flukey” teams that put up a great record and then drop right back down the next year get no love!
:: joel in providence — 1/4/2007 @ 1:22 pm
I think once you add in teams going back to the early 80s offensive DVOA will become more important.
Nine years is a really small sample size, and the failures of Indy and maybe Philly make offense look less important than it might actually be.
Some really impressive offensive teams won or reached the Super Bowl in the 90s and 80s but the 00s have been dominated mostly be defensive team. While I really like the idea presented by this article, I don’t know how confident we can be in the actual numbers.
:: navin — 1/4/2007 @ 1:26 pm
Good article, but I must jump in with my English pet peeve:
This research doesn’t “jive” with Aaron’s research. This reasarch, however, might jibe with Aaron’s research.
:: James G — 1/4/2007 @ 1:27 pm
also, weren’t the 2004 eagles AND steelers both among the top all-time teams in DVOA? yet both were unlucky to have the 2004 patriots at their absolute peak…. god 2004 was just bananas for football fans.
:: joel in providence — 1/4/2007 @ 1:28 pm
“Which is more important — getting into the tournament, or having a better chance to win it once you get there?”
Flags fly forever.
:: B — 1/4/2007 @ 1:30 pm
I’m confused as to why you point out that the Patriots are the team that is best built for the playoffs when the Ravens rank better than them in 5 of the 7 categories, and are only one spot behind them in another. Homerism?
:: bmw1 — 1/4/2007 @ 1:32 pm
Bill:
Speaking of uncertainty, though, part of the problem you’ve got here is that your data points aren’t necessarily independent. The Patriots, with Adam Vinatieri, have had bad kickoff DVOAs in 2001, 2004, and 2005, and they were ‘meh’ in 2003. That’s because Vinatieri isn’t that good a kickoff kicker. That’s one data point you’re treating as four, and given that the Patriots won the Super Bowl in 3 of those years, and earned 2 points in 2005, that could be a big part of the reason why the kicking DVOA has such a low correlation.
Think of it this way: because some of the points with the biggest lever arm (NE 2001, 10 points, NE 2003, 9 points, NE 2004, 10 points) all have poor kicking DVOA, the correlation says “hell, kicking DVOA doesn’t seem to matter” whereas all it’s really saying is “Adam Vinatieri’s kickoffs didn’t hurt the Patriots” - but that could easily be coming from something else (all the defensive metrics) rather than any intrinsic unimportance of kicking.
:: Pat — 1/4/2007 @ 1:32 pm
Really interesting article and the Stat Database is going to be awesome.
Since the article kicks off with Polian, can I get the Top 7 Playoff Stats for the Colts? I suspect that they are not nearly as bad as you’d expect, probably better than Dallas.
:: turbohappy — 1/4/2007 @ 1:33 pm
24: It’s because the ratings in the top categories are more important than the ratings at the bottom categories. Also, Bill is a Giants fan, so we should all feel bad for him.
:: B — 1/4/2007 @ 1:38 pm
Re: 23
And if don’t get into the play-offs you don’t have a job if you are the coach or GM.
If you are the owner in a some markets you aren’t making money.
I’m a Colts fan, and I while I hate the fact, I’m realistic enough to realize that the Colts are built to attract people to the games. How do you do that? By winning games and generating hype and excitement.
I’m sure that Irsay and other owners wish they were in markets where they could field an awful team and still sell out. The Packers are a small market team that sells out consistently, but from what I’ve heard from friends who live in WI is that Packers tickets while still not easy to get are noticebly easier to obtain than they were when they were a play-off team.
:: Frick — 1/4/2007 @ 1:43 pm
Interesting stat from the Stat Database for the Colts defense:
Q1: 16th
Q2: 30th
Q3: 15th
Q4: 31st
Correlates with my assertion that there’s not as big of difference between this year and last year as it seems - last year they were just enough better in Q1 that they didn’t get tired in Q2.
:: turbohappy — 1/4/2007 @ 1:43 pm
Any comments on exactly why this would be the case?
With baseball, there are many tangible differences between 162 games and seven games. In a seven game series, you can be do or die with your pitching staff, play your best starter three times a la Jack Morris, etc. Good pitching beats good hitting in the playoffs generally because you can give an otherwise unsustainable number of innings to your best pitchers. All those good hitting stats were racked up playing against 6 innings of good pitching and 3 innings of mediocre pitching a game. It’s different facing nine good innings a game. Anyway, you see where I’m going with this. How is playoff football that intrinsically different?
In football, that doesn’t seem to be the case. Football games are pretty do or die. Fatuous Bill Simmons theorizing nothwithstanding, I don’t think that teams do “Milton Berle” games during the season, “saving” themselves for the playoffs. The whole Guts and Stomps thing tends to point this way as well.
So, I guess you can color me unconvinced, at least initially. With only the 9 years of data, this could be pushed along by the Pats dynasty (which was also an outlier in the Guts and Stomps thing). Here’s a thought, though–could the whole cold weather advantage thing be the real driving force behind this? If cold weather teams tend to be more defensive oriented (I don’t know if this is the case, but it doesn’t seem outlandish), then those teams would be overperforming in January, and would have more postseason success.
Anyway, I really enjoyed this article, just trying to think things through.
:: Chris — 1/4/2007 @ 1:45 pm
re 12:
It was 6 games I think, but should still be included.
:: steelberger1 — 1/4/2007 @ 2:00 pm
“Which is more important — getting into the tournament, or having a better chance to win it once you get there?”
I want a team that will get to the playoffs consistently and hope that one year the matchups break my way and/or luck is on my side. If you believe (like I do) that playoffs are a crapshoot, then you want to be in the postseason as often as possible. Building a great playoff team isn’t all that helpful when you’re watching the playoffs from your couch.
:: Rocco — 1/4/2007 @ 2:00 pm
re 12:
It was 6 games I think, but should still be included in the article.
:: steelberger1 — 1/4/2007 @ 2:02 pm
That’s “predictive” not “predicative”.
:: Rob — 1/4/2007 @ 2:04 pm
I wonder how the winning streak theory would work if the last game wasnt included when teams had no chance to improve their seed (i.e. resting players).
:: steelberger1 — 1/4/2007 @ 2:04 pm
Ask the 2001 Steelers if special teams matter in the playoffs….it hurts just to remember it.
:: steelberger1 — 1/4/2007 @ 2:06 pm
“This seems to jive with Aaron Schatz’s research in the New York Times about the lack of consistency in kicker performance on field goals.”
That should be jibe, not jive. As Barbara Billingsley might say, “Jive ass fool don’t got no brains anyhow.” (Sorry, I couldn’t resist.)
:: Marko — 1/4/2007 @ 2:09 pm
It seems Bill may have messed something up on the winning streaks. I’ve e-mailed him about it but he’s on vacation, I’m going to wait for him to fix it so we make sure we don’t miss any. No need to keep bringing up examples; we know it’s an error. I’ll change the word “jibe” too. This is what I get for editing an article myself instead of handing it over to Tim Gerheim.
:: Aaron Schatz — 1/4/2007 @ 2:09 pm
#21: I didn’t see your post before my post - I hadn’t refreshed my screen for a while. Obviously, we have a similar pet peeve.
:: Marko — 1/4/2007 @ 2:13 pm
For large samples, you can compute the standard error of the correlation coefficient without using resampling or assuming normality.
:: econometrician guy — 1/4/2007 @ 2:17 pm
For large samples, you can compute the standard error of the correlation coefficient without using resampling or assuming normality.
Considering football stats live in the world of 16-game seasons, there’s a reason why I ignored that possibility. 68 points distributed from 1 to 14 isn’t a large sample.
:: Pat — 1/4/2007 @ 2:52 pm
With that database available, I may never work again.
:: Countertorque — 1/4/2007 @ 2:53 pm
Considering only 5 game win streaks is not enough.
Consider 10 game winning streaks at the end of the season and look again.
Somewhere between 5 and 10 game winning streaks at the end of a season, the correlation with playoff prowess grows dramatically.
:: Scott C. — 1/4/2007 @ 3:13 pm
Love the hamburger quote. That’s high comedy there.
:: elibolender — 1/4/2007 @ 3:14 pm
#30 - I was thinking along the same lines at first. That Pats represented nearly 5% of the sample population and would probably skew the results in a way to point to them as having one of the stronger correlations. As I began to think of the specific teams, however, I realized that none of the Patriots teams are really all that similar in what their player makeups and strengths were.
By the way, nice thought provoking article. I also would love to see a larger population, but I don’t know how reliable those would be given rule changes and the like. Is there a point in time that FO would be hesitant to “mix” data from one era to the next? The start of free agency would probably one of those dates. The change in the pass interference rules would have to be another.
:: bsr — 1/4/2007 @ 3:17 pm
Purds,
Come on, it’s not like it’s a surprise.
Turbohappy, that Dfeensive DVOA variance from quarter to quarter is actually predictable for small fast guys but potentially good news. If Indy does play according to the script and the O gets a lead, their 15th ranked D will not necessarily face 10 runs on a 12-play drive and tire out like they do. If they can keep the opponents to a roughly 50/50 run/pass balance, I bet their 2nd and 4th qtr D rankings rise to, maybe 20-22.
Yes, compared to Baltimore it’s laughable and I am grasping at straws, but a defensive DVOA of 15/20 from quarter to quarter might be all they need. I sure as hell know that 15/31 is NOT what they need.
And Pat, this might be the 4th year in a row Indy loses to the eventual SB champ. Some claim to fame. They’re Buffalo Junior. Odds favor it if they lose to either Balt in Balt or beat them to face SD. And if they win it all and break the streak, well, what was the title of that Bill Simmons book a couple years ago….?
:: Bobman — 1/4/2007 @ 3:22 pm
#42 - I’m wondering if we can get a thread devoted to nuggets we’ve dug out of the data base. Seeing that the Colts have the #1 first half offense and the Chiefs have the #31 first half defense makes me wonder if Indy will put the game out of reach early.
And the Ravens third quarter D is incredible. Rex Ryan must be making some kind of halftime adjustments — why isn’t he getting mentioned as possible head coach material?
:: Jimbo — 1/4/2007 @ 3:22 pm
As a Sociology major, I have learned there are three types of lies: lies, damned lies and statistics.
Clearly FO forgot to include the statistical impact of running backs wearing No. 21 in blue and white and the likelihood of a former basketball player catching touchdown passes as a tight end.
I added these to FO’s calculations, and the clear cut Super Bowl winner is the Chargers.
Trust me on this…I’m edumacated.
GO CHARGERS!!!
:: Go Chargers — 1/4/2007 @ 3:24 pm
I’m looking at the premium stats database-Thank You Aaron!-and the Colts’ offense has a 175.3% DVOA on third and long? How is that even possible?
That would explain why Peyton-Reggie-Marvin’s DPAr are so high. 175.3%? Wow.
:: throughthelookingglass — 1/4/2007 @ 3:33 pm
Viva Pedro #13:
No NFC team will be able to stop the Ravens.
That’s not the real question. Its not whether the Ravens can be stopped - clearly their pop-gun offense can be stopped. Its whether the NFC team can find a way to get something going against the Ravens defense before the defense forces them into a costly mistake. The “weakest” point on the Ravens statistically is runs middle/guard and off right tackle, passing to Tight Ends, and passing to #1 (hello Samari!) and other wideouts. Just a cursory check suggests their worst NFC match-ups would be the Eagles and Giants offensively, since those “weaknesses” are the offensive strengths of those two teams.
:: Andrew — 1/4/2007 @ 3:39 pm
I hate to ask this because it seems so cheesy but given we only have nine years to work with ..
What are the correlations if you remove some of the most obvious, cliche examples of each of these (IE, Colts being offense heavy and failing alot, do non-Colts teams with good offense correlate noticably better? Do defense-first teams only correlate well because of Baltimore and Tampa Bay? Do all those things the Pats do well only correlate highest because they’ve done them well and won alot of games the last few years?)
It seems like such a small sample size is prone to heavy adulteration by the specific examples because some of these teams have been involved every year. Naturally anything the Pats have done consistantly well through their run will correlate well with playoff success. But that doesn’t mean it does universally - if it doesn’t correlate at least somewhat with them omitted, then it’s very possible they adulterate the sample.
Of course, you knew this, but I’d like to see myself anyway.
:: Mike — 1/4/2007 @ 3:44 pm
A breif early 90’s rundown of Superbowl winners
1990 Giants- Powerhouse defense with very low turnover (only 5ints on 311 attempts) but effiecient passing game and grind it out running game
1991 Redskins- Led the league in pass offense and pass defense in YPA. Near league leaders in lowest int% on offense and highest int% on defense. Run game wasn’t great on offense but it didn’t need to be and held onto the ball. Had many stomps 260 pt differntial with only 224 pts allowed
1992 1993 1994 1995 Cowboys Niners- Cowboyw won 3 out of the 4 Superbowls but the Niners were right there with them Again these teams were dominant on both sides of the ball.
I think Indy has a chance if they can generate some INT’s with their pass defense but I wouldn’t make them the favorites. I think it’s a crapshoot between NE, BAL, and SD with INDY having an outside shot if they can get a few good games out of thier defense specifically in the INT or fumble dept. Thier pass defense isn’t terrible just mediocre.
:: Dave Brude — 1/4/2007 @ 3:50 pm
A breif early 90’s rundown of Superbowl winners
1990 Giants- Powerhouse defense with very low turnover (only 5ints on 311 attempts) but effiecient passing game and grind it out running game
1991 Redskins- Led the league in pass offense and pass defense in YPA. Near league leaders in lowest int% on offense and highest int% on defense. Run game wasn’t great on offense but it didn’t need to be and held onto the ball. Had many stomps 260 pt differntial with only 224 pts allowed
1992 1993 1994 1995 Cowboys Niners- Cowboyw won 3 out of the 4 Superbowls but the Niners were right there with them Again these teams were dominant on both sides of the ball.
I think Indy has a chance if they can generate some INT’s with their pass defense but I wouldn’t make them the favorites. I think it’s a crapshoot between NE, BAL, and SD with INDY having an outside shot if they can get a few good games out of thier defense specifically in the INT or fumble dept. Thier pass defense isn’t terrible just mediocre.
:: Dave Brude — 1/4/2007 @ 3:50 pm
Hey guys,
I’m on a real crappy computer filled with spyware at the MGM Grand so I can’t answer questions too thoroughly. When I get back, I’ll be able to do some more research on the questions that have come up.
Just a few quick notes:
- I’M SORRY about the jive/jibe thing. I am ashamed of myself and my family isn’t too proud of me either.
- The reason the Ravens didn’t come up as winning their last five games, even though they did, was because I was only looking at games 13-17 and the Ravens had a bye. That’s my bad — I should’ve stated that. I would imagine that the traditional idea of “momentum” would consider a bye week to be something that impedes that momentum, but I think that’s something we can look at in the future, too.
- I really wish this Bonzi Buddy would go away.
- Obviously the small sample size does play an issue in complicating the reliability of the findings. I’m not worried so much about teams repeating with the same core group of players because they’d be representing 4 or 5 of 110 teams, and if the Patriots win so much despite not having a solid kickoff guy, it would then follow that a kickoff guy wouldn’t be that important! Otherwise, the Patriots wouldn’t have won. I don’t think that logic is fallacious but I’m quite hung over.
:: Bill Barnwell — 1/4/2007 @ 4:16 pm
One other thing about the streaks. Adding NE to the list, 7 of 14 made it to the big dance. 11 of 14 made it to their conference championship.
When I read that chart, I find a streak to be a whole lot more meaningful that Bill did.
Otherwise, I think it is one of his better pieces.
:: Oswlek — 1/4/2007 @ 4:18 pm
that 1991 redskins offense was positively bonkers. maybe this is just my selective memory but i seem to remember rypien comleting an absurd number of long td passes. like he’d have 3 or 4 BOMBs per game.
:: joel in providence — 1/4/2007 @ 4:21 pm
The basic gist of “defense wins championships” can be seen statistically in the Pythagorean Wins formula.
The Super Bowl winner has been the Pythagorean Win leader in 10 out of the past 16 years. The #2 team won in another 2 of those years.
The Pythagorean Win formula mainly varies with changes in points scored against, which is primarily affected by defense. Scoring 50 more points in a year is far less important than preventing the scoring of 50 more points.
For example, a 400-300 team has a Pythagorean Win of 10.24. If they go 450-300, it rises to 11.08. But if they go 400-250, it rises to 11.51.
If a higher Pythagorean Wins total is predictive of Super Bowl victory, defense is far more important, which is also why we see so many Super Bowl champs be among the top 6 in preventing scoring. The only recent champ not in the top 6 in preventing scoring was the 98 Broncos, and they were 7th.
:: Andrew — 1/4/2007 @ 4:22 pm
Why again are the five-game winning streaks not predictive when twice as many teams won the super bowl as would be expected if all playoff teams were random, and *half* the teams made it to the SB?
:: hgfalling — 1/4/2007 @ 4:37 pm
The 91 Redskins outscored their opponents by a more than 2-to-1 margin for the season (485 to 224). And I think their sack differential was more than +40, in large part because Rypien was sacked 9 times all season. The Ravens have a +43 sack differential this year, which is pretty amazing.
:: Jimbo — 1/4/2007 @ 4:40 pm
Mike #51, I wondered about this too–if you take out the frequent outliers (Indy on O and TB/Balt on D) do the numbers stand up? I suspect they do–think the year Oakland faced TB in the SB. D won big-time. Can’t blame it all on Barrett Robbins.
Dave #53, keep in mind Indy’s D has faced the fewest passes of anyone, so they may be mediocre, or they may be worse if seriously challenged. I doubt Brady will throw 5 picks against them again (did he really throw 5? And NE only lost by one TD?) They are currently playing safeties that rank 5 and 6 on the depth chart, and even with Sanders back, he’s not 100% and not exactly a cover guy.
Bill Barnwell, thanks for your dedication, emailing from vacation and while hung over to boot! The true sign of a great professional.
:: Bobman — 1/4/2007 @ 4:45 pm
I’m pretty skeptical of the all the stats presented here. None of those correlation coefficients are significant. Most show downright zero correlation, suggesting that Billy Beane’s theory, if applied to football, is correct.
The only thing that looks at all statistically relevant is the momemtum thing, which you discounted. Despite the incredibly small sample size. 3/13 teams won the superbowl with momentum and only 6/95 won the superbowl without momentum. If you are going to make a claim about those numbers either way, it seems to me to have to be that momentum is the only statistically relevant stat going into the playoffs.
And look one further, 7/13 teams entering the playoffs made it to the superbowl. which means only 11/95 teams without momentum made it to the final match.
Here’s the study, using your methodolgy. Rank all playoff teams from last 9 years by winning percentage over last 5,6, and 7 games. Now rank all playoff teams from last 9 years based on your playoff points system. Find the correlation between the rankings, i’ll wager it blows all the DVOA correlations out of the water.
Why? Cause your performace over the last half of the season is far more relevant than your performance over the entire season. How many hundreds of anecdotal bits of evidence can we find just within the NFC East this year…
:)
Cheers, i did enjoy reading it though, despite the doubting thomas-y-ness.
:: jacob — 1/4/2007 @ 5:12 pm
I’d like to see how some NFC teams lacking a QB named Grossman rank here. Like Philadelphia, for example, who seems like the only team in the NFC with a chance. I know that their red zone D is boosted by a ridiculous number of stops and turnovers/TDs in “and goal” situations, but your database software crashed before I could look up any stats on my own.
:: billsfan — 1/4/2007 @ 5:12 pm
I agree the logic’s not totally fallacious, but I suppose I’m just getting stuck on the smallness in absolute terms of the correlations we’re dealing with - it’s just so realistically possible that the Pats or someone else won despite x or y and not because of it due to the sample size, and I would think that any team that’s advanced in the playoffs 2-3 times would represent a significant sample because 40% of the teams were 1 and dones, after all.
With all that said, I really enjoyed the article, I just wonder how profound the stilting effect is of those most successful teams.
:: Mike — 1/4/2007 @ 5:16 pm
I think others have mentioned it, but the statistics professor in me advises everyone to treat the splits with caution as it involves many comparisons therefore raising the chance of a spurious result. (especially with a sample size too small to support it)
But the results seem to make some sense. Imagine good offense vs. good defense. For the offense to score, they must either sustain a drive against a defense that does not allow sustained drives, or they must get really lucky. The defense makes one or two pretty good plays at any point in the drive and they get off the field. Good offense must count on a string of successes to score (or a lucky bomb).
:: BD — 1/4/2007 @ 5:20 pm
To be fair in interpreting the results, consider that we’re talking about the outcome of a couple 60 minute contests. Random error is bound to be there (Bills fans… can you say wide right?).
The article doesn’t address statistical significance per se, but a significant correlation around .2 or .3 is definately NOT zero! In some fields, a correlation of .3 can be considered a fairly robust effect. With all the bounces of the ball, snowy Januarys, tuck rules, etc. I would say a .25 correlation is meaningful.
:: BD — 1/4/2007 @ 5:26 pm
Bobman, it was “Now I Can Die In Peace”, written by the guy who a) “created” the five-year moratorium and b) promptly broke it about 5000 times with respect to the Red Sox, so take that with a grain of salt. (It’s not a bad book, but if you can’t read all the way through some of his columns, I’d skip it.)
It’s one thing to say that a team like the Colts should make a change to shoot for one big flag instead of a number of little ones (you know, they put up flags for everything these days, and I mean everyone, not just the Colts or just NFL teams), but sometimes you make a change and end up running the franchise into the ground because you hired a completely incompetent individual and refuse to acknowledge it, and what’s worse, not only does he refuse to acknowledge he’s incapable, he thinks it’s some badge of honor to be terrible at your job but to keep on doing it anyway, as if years of sucking would suddenly make you good. (Yes, the Lions were kind of in that position - not as good as the Colts, obviously, but six playoff appearances in the ’90s and one NFC title game appearance was a hell of a lot better than the decade before or the decade after.)
I don’t think more variance in defense means that you’re better off building your team around offense. I think it means that you have to focus more on expected value of your defense than actual value: it’s more likely that your offense really is bad if it has a bad year than that your defense really is bad if it has a bad year. Or, to put it in playoff terms, if you have a mediocre defense, it might need a bit of help or a lot of help, but you shouldn’t necessarily clean house - you should try to swap out bad parts for better parts and wait for it to gel. On the other hand, if you have a mediocre offense, it probably is what you think it is, but even if you make it better, you won’t necessarily go farther in the playoffs.
:: zlionsfan — 1/4/2007 @ 5:27 pm
None of those correlation coefficients are significant.
The size of a correlation coefficient does not have anything to do with its significance. Its significance is determined by more complicated measures (most people think of p-values, but in this case, getting a p-value from a chi-squared is probably not accurate at all).
You can have small but significant correlations. It happens all the time.
I’m not worried so much about teams repeating with the same core group of players because they’d be representing 4 or 5 of 110 teams,
Bill:
The Patriots don’t represent 4 or 5 of 110 teams. They represent 3 out of 8 teams that have won the Super Bowl - which is a much bigger sample - and teams that have won the Super Bowl, by definition, dominate the large-value of Playoff Score Points.
Think of it on a graph: the Patriots (well, Adam Vinatieri, obviously) represent three out of 8 points on the “large PSP” side of the graph. And they’re all low, or negative. Even if all 5 of the other 8 Super Bowl winners had positive kicking DVOA (they don’t), that would still look like a weak correlation. They have a lot of lever arm on the fit, and it’s all due to one point.
and if the Patriots win so much despite not having a solid kickoff guy, it would then follow that a kickoff guy wouldn’t be that important!
Not necessarily - it could be that there’s something else consistent about the Patriots over that time period that allows them to win in spite of poor kickoffs. That’s the problem - it’s basically one data point.
Then again, of course: the other Super Bowl winners’ kicking DVOA were 6.6, -4.8, -0.7, 4.1, 9.3, and -1.6, or an average of 2.2. So it almost definitely doesn’t matter anyway.
:: Pat — 1/4/2007 @ 5:42 pm
Never mind, it’s working again.
As far as I can tell from the premium stat preview, the Eagles are 7th late and close (Saints 20th, Giants 22nd), 19th on first down (Giants 9th, Saints 14th), and 8th red zone (Saints 6th). FWIW, Philly was #1 in goal-to-go defense.
:: billsfan — 1/4/2007 @ 5:46 pm
65:
Another good example is Kevin Dyson not having long enough arms. And I hate you.
:: billsfan — 1/4/2007 @ 5:55 pm
Great Article
:: Lincoln — 1/4/2007 @ 6:08 pm
First, on the five-game winning streak subject:
DVOA correlates better with playoff success than weighted DVOA. That tells you that the results of early games are still relevant enough to be predictive.
But on the other hand, the teams that have gone on 5+ game winning streaks going into the playoffs have done spectacularly well. Why is that?
Well, if you’re good enough to win five or more games in a row, then you’re probably a really good football team. A team that has a five-game winning streak probably went 11-5 or 12-4, minimum. Of course that sort of team is going to do well.
:: Yaguar — 1/4/2007 @ 6:11 pm
I just took out the winning streaks thing. We missed a couple teams which won Super Bowls, which makes Bill’s point look wrong, except that we also probably missed a couple teams that didn’t win Super Bowls and nobody is bringing them up to make Bill’s point look better, so frankly we should just re-do the whole thing at some point. The article works fine without it so just go about your business.
:: Aaron Schatz — 1/4/2007 @ 6:12 pm
I was wondering if it was possible to do analysis of DVOA involving only teams with a winning record to see if any interesting trends develop.
The title of the article focuses on the Colts losses in the playoffs. In looking back over the last three years, why did they lose?
2003 and 2004 — the Pats were superior in both lines. The wind was also a factor in 2003(never understood why people focus on cold when it was clearly wind affecting ball flight). The other thing that is obvious to anyone going back to look at those two games is that the Colt receivers could not get open. On defense, the Colts couldn’t get a pass rush in 03 and couldn’t stop the run in 04. Brady’s failures were what kept the 03 game close and the 04 game close until the 2d half when the Pats finally took the ball away from him and pounded it every play.
In 05, Steelers just absolutely overwhelmed the Colt O-line. People focus on the blitz, but it wasn’t a blitzing LB coming unblocked that created the havoc. It was all the Steeler pass rushers dominating their blockers. It looked exactly like a replay of the Charger game from the reg season.
So what is the “explanation”? The Colts play with the smallest margin of error of any offense. Their run is totally dependent on the pass. Their pass is predicated on having a WR/DB mismatch somewhere and the belief that wherever it is on the field, they (18) will find it and get the ball there (and before your unblocked blitzer can reach him). They don’t need real good pass pro, barely adequate will do. They don’t need wide open receivers, a tiny opening will do. They believe their offense can outscore what you do to their defense.
What is amazing over the last few years is that they can take barely adequate pass pro and small mismatches with WRs and put up 40 points. Even a slight improvement in pass rush &/or a slight improvement in coverage can just about shut them a down.
The problem — their pass pro is barely adequate. If the pass rush from the def front four manhandles the O-line it all goes to crap. If the front four stops the run without help (see Jax, Tenn this year), they can get 7 defenders totally focused on 4 receivers. If the front also dominates with a pass rush, it gets really, really ugly.
When the Colts face big physical D lines with some ability to pass rush and decent secondaries, they get embarassed. Because they have no answer. They can’t knock anyone off the ball. They can’t win with defense or special teams. They absolutely must have a little time to execute the precision pass game because it is all they have to win with.
:: stan — 1/4/2007 @ 6:21 pm
re: 50
“Its not whether the Ravens can be stopped - clearly their pop-gun offense can be stopped. Its whether the NFC team can find a way to get something going against the Ravens defense before the defense forces them into a costly mistake.”
First off, the Ravens offense has been a top ten unit since Billick took over, its hardly pop-gun. There are only 2 elite defenses in the playoffs, the Bears and the Ravens (the next team is NE at -8.3%), and the Bears D has been falling apart of late, so I don’t see any team in the playoffs that could possibly shut them down.
What worries me is the punt and kick returns. Evers since B.J. Sams broke his leg, Cory Ross has been the return man, he’s a pretty green rookie. I can definitely see him muffing a punt in a key situation. Clayton is probably a good return man but he hasn’t seen any game action, so he is prone to making a mistake as well.
“The “weakestâ€? point on the Ravens statistically is runs middle/guard and off right tackle”
The Ravens rank 2nd runs to mid/guard (3.78ypc) and 7th at right tackle (3.5ypc), but 1st at right end (1.99ypc). No area of the run D is weak, maybe a RB busted one long run off right tackle that skewed the average. I don’t think coach watching tape would think running at RT would seriously exploit the Ravens D.
I think the problem with trying to attack Rolle downfield, where he’s been burned several times this season, is that then the QB has to sit in the pocket for a while , and if that reciever is doubled or not open, the QB will likely be crushed by Suggs, Scott, etc. A team would need a real burner as a #2 reciever (assuming McCalister covers their best WR) to cause problems, the only team that really has that is the Colts. Of course, if the Ravens cover a teams best reciever with Rolle (see Lee Evans TD), then they are screwed.
:: jonnyblazin — 1/4/2007 @ 6:27 pm
I agree with Pat. And, of course, there is a similar point with offensive teams in that the Colts, with their high powered offense, have consistently underperformed in the playoffs. Last year, they would have gotten zero PSP. And, like Pat’s point, it’s really one data point. It’s basically been the same Colts offense for each of those seasons.
Kudos to Bill Barnwell for crunching the numbers, but I would have liked to have seen him posit a causal mechanism for why good defense would trump good offense in the playoffs. In baseball, a plausible one is fairly easy to identify, but I can’t see why one aspect of a team’s performance would overpredict in the playoffs.
:: Chris — 1/4/2007 @ 6:47 pm
#73 - You are underselling the OL and WR of the colts by a large margin. Saying that they have barely adequate pass protection or wide recievers that barely get open is a gross understatement. Both are exceptional.
:: bsr — 1/4/2007 @ 7:26 pm
A few thoughts on why defense might play a more significant role in recent NFL playoffs:
* Refs call fewer penalties in the playoffs, making it easier for defenses to disrupt timing of offenses (I realize last year might be an exception)
* Since LT (the original), the disparity in speed between offensive players and defensive ones has diminished. Montana and Rice played many more games vs. old-style linebackers and DEs (big, slow guys) than current “West Coast” offenses do.
Just a few thoughts… I’m not claiming any sort of substantiation other than defensive dominance in playoffs recently.
:: ElTiante — 1/4/2007 @ 7:29 pm
One thing missing from the article (please correct me if I’m wrong). It’s nice to quote correlation coefficients, but are these numbers significant? At what level?
As a suggestion for further research, I think it would be interesting to assign dummy variables to things like “3-4 defense,” “4-3 defense,” “predominant Cover 2 scheme” and do a regression to try and determine significance between the points formula you mention and these factors.
In general, I would have liked to hear more discussion of why the factors you tested were chosen - I’m assuming it’s a data thing. Perhaps doing some multivariate analysis of the factors you found most compelling?
:: RF — 1/4/2007 @ 7:37 pm
Adding to my comment in #78, there may be a line editors want to walk with regard to the statistical methods and writing an article which needs to transcent the statistically inclined crowd. That sign, things like the level of significance are very important. Perhaps using tags and endnotes to say “significant at the X level” would be enough to have the information yet not detract from the article as written.
:: RF — 1/4/2007 @ 7:51 pm
Some statistical quibbles…
First off, it is quite likely that the playoff score points system makes the correlation coefficients weaker. It would be instructive to rerun the analysis using a simple dummy variable approach with wins-losses as 1-0. This way victories in certain games by certain teams don’t have more weight. If, for example, the last 9 superbowl winners all had excellent run defenses, the small sample size assures of biasing our statistic by using the weighted playoff score system. This is further exacerbated by using linear regressions and a non-linear point system.
Statistical quibble #2: “momentum” shouldn’t be judged by wins\losses. Football Outsiders correctly understands that a team can lose based on bad luck. Instead, a teams DVOA growth rate (or stability at a high number) prior to the playoffs should be used. I understand that this introduces elements of a time series analysis into the picture, but really this would be much more accurate.
Quibble 3: Using linear regressions for playoff success is probably always going to yield low correlations. It is quite probable that the relationship is either log-linear or exponential, meaning that either small improvements at the low end are more important or small improvements at the high end are more important.
To understand 3, think about it this way: Is the difference between a team with a run DVOA of 0% and -5% more, less, or the same as a difference of -20% and -25%? One is the difference between an average defense and a slightly better than average defense, the other is the difference between a good defense and a best-in-the year, exceptional defense. There are many teams in the 0% to -15% range in any year, but few ever crack -25%. The result is that changes in DVOA at the extremes are likely indicative of something intrinsically ‘more valuable’ and need to be weighted more heavily by using an exponential model for the regression.
:: David B — 1/4/2007 @ 7:52 pm
Perhaps doing some multivariate analysis of the factors you found most compelling?
Hey, there’s only 64 data points with non-zero “playoff score points” - eliminating additional degrees of freedom (especially when several of those data points are likely covariant, as I mentioned before) is just going to murder you.
It would be instructive to rerun the analysis using a simple dummy variable approach with wins-losses as 1-0.
Skip that. Why not just use DVOA in the playoffs itself? You’ve got a better measure of a team’s performance in the playoffs. Why not use it?
:: Pat — 1/4/2007 @ 8:06 pm
76,
The Colts offensive line is barely adequate. They can’t run block (in any ordinarily understood definition of the term). Check the FO stats and see that they always rank among the very worst in short yardage situations every year. They are absolutely pitiful if you need to pick up a yard and run a straight ahead power.
If you want to see how soft they are (and why Joey Porter called them out on it), watch the tapes of the SD or Pitt games last year or the Dallas, Jax or even Tenn game this year.
I don’t believe I said anything about the WRs not being good.
:: stan — 1/4/2007 @ 8:11 pm
Re 81:
Don’t want to use DVOA for the same reason the As don’t win the post season: having the most statistically succesful night doesn’t mean you’ll win (even in baseball). Ultimately, it’s a game about winning, so what you want to know is if we’ve developed a statistical measure which tells us anything about why someone might win in the playoffs. For all we know, the eagles might go play the giants, dominate both sides of the ball for 3 quarters, rack up a 14 point lead, and then lose the game on a series of questionable calls and remarkable fumbles that fly forward to waiting Giants in the end zone.
The purpose of this excercise is to see if we can now identify which trends lead to playoff success…as defined by winning.
:: David B — 1/4/2007 @ 8:19 pm
Just a thought - how would the correlations change if some points were added for earning a bye? One team gets a bye and loses its first game, another doesn’t and goes 1-1. Which has done better in the playoffs? Neither - they came equally close to winning the championship. Just thinking, tack on two points for a bye, run the numbers real quick, and see what happens.
:: Trogdor — 1/4/2007 @ 8:25 pm
On 84…it’s a good thought, but it would skew the results by telling you who is more likely to get a bye, rather than who is more likely to win a playoff game. Since the rest of the results are about likelihood to win a playoff game, it won’t look right (and will probably reduce both confidence and correlation).
:: David B — 1/4/2007 @ 8:29 pm
76,
More on Colts’ O-line — did you watch them struggle in pass pro against the Giants in week one or the Pats? Pats should have had about a half dozen sacks that night.
:: stan — 1/4/2007 @ 8:29 pm
I had the same thought as trogdor. There are 5 possible outcomes for a playoff team:
Lose in rd 1 (wild card rd)
Lose in rd 2 (divisional rd)
Lose in conference championship
Loss in Super Bowl
Win in Super Bowl
I’m not sure the system of assigning playoff score points reflects these outcomes very well.
I tried this:
any loss = 0 pts
1st rd win = 1 pt
Bye = 1 pt (getting bye into 2d rd is same as winning 1st rd)
2nd rd win = 2
conf championship win = 3
Super Bowl win = 4
This translates to these pts per outcome:
Lose in rd 1 = 0 pts
Lose in rd 2 = 1 pt
Lose in conference championship = 3 pts
Loss in Super Bowl = 6 pts
Win in Super Bowl = 10 pts
From ‘97-’05, this gives the following correlation coefficients:
Total DVOA = 0.43
Off DVOA = 0.14
Def DVOA = -0.27
ST DVOA = 0.17
Wtd DVOA = 0.40
Non Adj VOA = 0.47
Est Wins = 0.48
Pyth Wins = 0.50
Actual Wins = 0.52
Defense appears to more strongly correlate with success than offense (it would be instructive to look at traditional stats to see how they correlate compared to DVOA).
Total DVOA looks much better - and still beats weighted DVOA. However, actual wins (among others) beats everything. I think this is because assigning one point to getting a bye biases the results - byes correlate completely with actual wins (and probably very strongly with Non-Adj VOA). Also HFA probably has a significant impact here, so the author’s system of giving extra points for road wins might be better.
I didn’t try to run the correlations for the various splits. What is clear to me though, is that the size of the correlation is affected by the system of assigning points. This system probably needs to be examined much more closely before proceeding further in assessing what predicts playoff success.
:: MRH — 1/4/2007 @ 9:11 pm
HFA since NFL went to 12 team playoffs:
Rd 1: 43-21 0.672
Rd 2: 51-13 0.797
Conf Champ: 19-13 0.594
Teams with a bye (home in rd 2) do better at home than home teams with no bye (home in rd 1 and conf champ game). I think this is relatively well-known, but thought I’d put the numbers up (apologies for any errors in counting).
If the playoff point systme is going to reward road wins over home wins, is 3-2 the right ratio? Should it be larger? Should it vary by round? I think these questions need to be addressed by further research.
:: MRH — 1/4/2007 @ 9:29 pm
#82 & #86 - They don’t rank highly in power situations because they are built around a stretch play run scheme and are more light and quick then powerful. However, that doesn’t mean they are barely adequate, even in run blocking. They rank highly in all other DVOA stats in both run blocking and pass blocking. They also have two probowlers on that line. Furthermore, the one traditional way to rattle Manning is with pressure up the middle. If the line was barely adequate, he would be rattled much more often then he is. There is no evidence to suggest that they are anything other a very good line. If anything, if the colts went to a more traditional power rushing line, you would then see a big decrease in pass protection.
As for the Pats and Giants games, both have good defensive lines. Especially the Pats who have pretty much manhandled every offensive line they have faced this year. It isn’t fair to judge them based on one or two games where there could simply be bad matchups.
Rereading your post, I do see now that you weren’t discrediting the WRs. My bad. I do have an answer for you about the cold effecting the passing game however. It isn’t just the wind. The ball itself is much harder and heavier, which makes it more difficult handle and different to throw.
:: bsr — 1/4/2007 @ 9:50 pm
dudes, i love your stuff when i can read it. when you go all numbers and correlations on me i’m lost. suggestion: try to tease this information, which i’m sure would fascinate me if i paid attention in maths class, into somrthing resembling a narrative.
best, mark
:: mark cook — 1/4/2007 @ 9:53 pm
I really enjoyed reading this article. It is a good starting point for a deeper statistical analysis. Besides that it is also a good starting point to find what #75 called “causal mechanism” behind the data.
To me it seems that the reason why the “First Down Passing DVOA” and the “First Down DVOA” are so important is that these situation are likely to see the most variation in plays by the offense.
Therefore it is more difficult for the coaching stuff and the players to detect what the offense really wants to do with this play and stop them from doing so without guessing totally wrong.
A defense with a good “First Down DVOA”-value is a sign of team that has only few holes in its defense and has intelligent coaches/players. Generally if a defense can deal with complex situations like 1-10 it should also be able to deal with situation where the number of different plays they are facing is smaller.
A good “First Down Passing DVOA”-value indicates that a team doesn’t get burned by a long pass. This enables the defense to stop the drive later on (to force a punt or only allow a field goal). It also helps at the end of the halves when teams often have to move the ball quickly (which leads to a passing play). If your defense has a higher prob of stopping this it gives you a vital egde in close games.
When looking at the chances of the two teams that rank best in the presented categories I would give the Patriots an edge over the Ravens in an actual game but I would give the Ravens an edge over the Patriots when it comes to playing the other teams.
To me the Bengals (before they decided that the play-offs would not matter to them this season) laid out the blue print of how to beat the Ravens: You have to have complex offense which keeps the Ravens defese guessing whats coming up because they have to defend a large field. You then have to move the ball without giving up turnovers on only 3 or 4 possesions. the most important point is to avoid that the Ravens get 10 points through their defense. If you can do that and still move the football for 13 or 17 points you have taken away the biggest asset of the Ravens. The Bengals did it as did the Patriots in a similar game against the Bears: Create a low scoring contest in which your offense is the bigger thread at the end of the two halves than your oponent one.
Looking at the Patriots their biggest weakness in my opinion is their lack of experience in the defense due to injuries. I prepared for the upcoming Patriots - Jets match by looking at previous two games this season. The Jets had the most success when they went no huddle thus limiting the influence of the coaching stuff and asking the defense if they could make the right reads on their own. In the first game the Patriots got the job done but in the second game they did not. In this game I believe they had more injuries in their defense and it made a difference under this circumstances.
I think, against the Patriots you have to have an experienced QB whom you trust to make his own decisions when it really matters. This would rule out the Chargers for me.
:: jbochow — 1/4/2007 @ 10:01 pm
On the question of why defense should be more important than offense in the playoffs relative to the regular season, I have a tentative suggestion. Coaches and players speak of a radical difference of approach going into playoff games. I submit that perhaps the most important way in which this manifests itself might be a change in the proportion of general to opponent-specific elements in training. That is, perhaps teams stop doing so much work on fundamentals or familiarity with the entire playbook, and look more closely at what is contained in this week’s gameplan only. Let’s call this premise 1.
We now need another premise: that it is in some sense easier or more effective to adjust one’s gameplan for a specific opponent on defense than offense. This seems intuitively plausible. An offense will have six of the same highly specialised personnel (QB and OL) on the field for every single play, and they will be lining up in pretty much the same place. The defense has no such constraint, so more variations in personnel and alignment are possible. Moreover, the most important element of the defense - the pass defense - has to be geared primarily around the tendencies of only one opposing player - the quarterback - where an offense has to be adjusted to match up well against and be prepared for the tendencies of an entire unit.
So:
1. More emphasis is placed on gameplanning and game-specific practice for playoff games than for those in the regular season.
2. Additional game-specific emphasis improves defensive performance more than offensive performance (and the pure speculation element - this improvement is a multiplier-type effect, not the addition of a constant).
Therefore
3. Defense is more important than offense in the playoffs compared to the regular season.
Now, I’m not saying either 1 or 2 is true, but they do seem plausible to me, and if true they go some way towards explaining the facts.
:: Mr Shush — 1/4/2007 @