Writers of Pro Football Prospectus 2008

05 Apr 2011

The Ratio of Relative Importance

Northwestern student Keith Goldner is working on an Expected Points metric that has some promise; here, he attempts to measure the impact of each facet of the game.

Posted by: Bill Barnwell on 05 Apr 2011

15 comments, Last at 08 Apr 2011, 11:41am by keithgoldner


by Randy Hedberg (not verified) :: Tue, 04/05/2011 - 2:53pm

Didn't FO really start when someone suggested looking beyond correlations and explained why "run to win" was a fallacy? This seems like it might make correlations easier to understand, but the net effect on our understanding of the game is pretty minimal.

Brian Cook of mgoblog.com once critiqued an article by sports economist David Berri by saying "When you've got a hammer, everything looks like a nail. Berri's hammer is regression analysis, and he goes about hitting everything he can find with it until he finds something that seems vaguely nail-like from a certain angle." I'm worried that something as simple as this would fall prey to that kind of trap.

Cook's critique is here: http://mgoblog.com/content/sports-economists-always-wrong-about-everythi...

by Temo :: Fri, 04/08/2011 - 10:47am

Interesting read, thanks.

by BaronFoobarstein :: Tue, 04/05/2011 - 3:19pm

You can do pretty much the same math with DVOA. And in both cases it's not telling you "offense matters more than defense;" it's telling you "we measure offense better than we measure defense."

by Jerry :: Tue, 04/05/2011 - 6:15pm

DVOA measures defense the same way it measures offense.

by tuluse :: Tue, 04/05/2011 - 6:31pm

Right, however they are two different things.

by Bobman :: Wed, 04/06/2011 - 11:14pm

That, my friend, is one of the funniest, awesomest, most word-efficient comments I have seen here in a long, long time. I plan to steal it and use it at some point... maybe when my son says, "I play catcher just like I play running back."

by bradluen :: Fri, 04/08/2011 - 1:25am

It's telling you offenses are more variable than defenses. If you believe in DVOA, you knew this already. If you don't believe in DVOA, it's easy enough to show -- the SD of Points For (across team seasons) is bigger than the SD of Points Against.

by BaronFoobarstein :: Fri, 04/08/2011 - 3:54am

I don't think that offensive scores having higher variance than defensive scores implies that they will correlate better with wins.

by ABW (not verified) :: Tue, 04/05/2011 - 6:05pm

So...I took their data, pasted it into Excel, and then entered the equation they got from doing the regression, and tried to see what the equation predicted the wins would be for various teams. Here are some of the the results I got:

2010 Exp Wins
ARI -622.1682
ATL 332.1576
BAL 249.2379
BUF -228.1708
CAR -702.735

Then I tried with the "Off Rating" and "Def Rating" instead of "Off NEP" and "Def NEP", and that gave me this:

2010 Other Exp Wins
ARI -17.1455
ATL 78.9301
BAL 57.8899
BUF 13.6851
CAR -67.775

Either someone needs to explain what I am doing wrong, or I am going to conclude that doing this sort of regression analysis is not a very useful way to analyze this data.

by Samson151 :: Tue, 04/05/2011 - 9:20pm

Nope, you have it right. I fully expect Carolina to lose 57 games next season. They'll find a way.

by justanothersteve :: Wed, 04/06/2011 - 12:45pm

So the owners asking for an 18 game season was only a subterfuge?

by Temo :: Wed, 04/06/2011 - 11:32pm

There is a way: When a vindictive Jerry Richardson forces his team into barnstorming tours versus area high schools in punishment for their temerity during labor negotiations.

by poboy :: Tue, 04/05/2011 - 10:27pm

He mentioned in the article that he normalized the ratings before running the regression. I'm guessing you used non-normalized ratings?

by bradluen :: Fri, 04/08/2011 - 2:04am

The main draw here is that you can split credit between offense, defense, and special teams in a reasonably transparent way.

Running my own regression, I get that a win is about 41 NEP. The 2010 Patriots had 270.5 NEP, so they should have been around a 8 + 270/41 = 14.6 win team. Of the 6.6 wins above average, 5.4 were due to offense and 1.2 were were due to special teams; defense was basically average. This doesn't quite square with DVOA, but there are no opponent adjustments.

The Panthers played like a 2.8 win team according to NEP. Their offense lost 4.4 wins, and their special teams lost 0.6 wins. Pretty close to DVOA.

Goldner's normalisation presumably gives you a slightly better picture than this, but not much more.

The explanatory power isn't magic or anything. NEP are almost the same thing as raw points, and we shouldn't be surprised that raw points explain wins pretty well.

by keithgoldner :: Fri, 04/08/2011 - 11:41am

Yeah that's the main goal I'm trying to accomplish. I am actually going to talk about opponent adjustments and scaling the NEP into wins, just like you did above, in the next week or so. Thanks for all the feedback