Writers of Pro Football Prospectus 2008

29 Sep 2010

Team Drive-based Rating (TeDR) Debut

(I'd publish this on my own site, only I don't have much of one and anyway I can't guarantee I'd maintain and improve it over the years. So I'm offering it to FO to discuss as well as blogging it.)

Several years ago, while in college, I became interested in the mayhem that is trying to predict the NFL. At first, I was merely curious to see if I could get playoff teams correct from one year to another, but then I got interested in the week-to-week problem. I made several abortive attempts (college is bad for concentration on other things), and then abandoned the project once I started reading TMQ and his dismissal of the complicated systems.

This year, I came back to it. The project I'm working on is drive-based. I'm not yet sold on play-by-play, even with DVOA, as a solid predictor; besides, I don't have time.

The system produces a rating for each team, based on three stats: yards per possession (or "drive"), yards per point, and points per drive.

Some features:
- I do count penalties, sort of. Offensive penalties will affect an offense's rating, and defensive a defense's.
- Turnovers are counted as negative, with additional penalties if the opponent scores off the turnover.
- I do not count yards in the endzone; anything that happens behind the pylons is 0 (or I guess 100, though I don't know what that would be). I can't justify this, but it primarily affects returns, which often lose a yard or two from their "official" NFL-published length.
- I do count returns, whether interception, kick, punt, or fumble, as part of a possession even though I use the term "drive". This is against for simplicity's sake, even if it conflates special teams in with offense and defense.
- I make no account of punts, other than to mark them as a non-scoring drive that didn't end in a turnover. A muffed punt is not a possession for the returning team unless recovered by them.

From these I manufacture a rating for the drive, offense and defense; the team's offensive or defensive rating is then the average of all drives. I currently generate three rankings from these: a "raw" rank which averages the ranks of a team's offense and defense; a combined rank which generates a team rating from the offense and defense ratings, and ranks them; and adjusted or "versus" rank based on a comparison of a team's offense to all defenses and vice versa. The last two so far tend very close or identical; the first tends to differ quite a bit.

A sample complete rating (actual rating, after three weeks):

Pittsburgh Steelers: Offense 23.54 (#9), Defense -21.38 (#1). Ranks: Average 5, #1. Combined 22.46, #2. Adjusted 99.41, #3.

Full ranking, adjusted rank, no ratings published:

1. Atlanta Falcons
2. Indianapolis Colts
3. Pittsburgh Steelers
4. Green Bay Packers
5. New England Patriots
6. Kansas City Chiefs
7. Philadelphia Eagles
8. Tennessee Titans
9. New York Jets
10. Seattle Seahawks
11. San Diego Chargers
12. Chicago Bears
13. St Louis Rams
14. Miami Dolphins
15. Denver Broncos
16. Cincinnati Bengals
17. Baltimore Ravens
18. Minnesota Vikings
19. Dallas Cowboys
20. New Orleans Saints
21. Washington Redskins
22. Tampa Bay Buccaneers
23. Detroit Lions
24. Cleveland Browns
25. Oakland Raiders
26. Houston Texans
27. Arizona Cardinals
28. New York Giants
29. Carolina Panthers
30. Buffalo Bills
31. San Francisco 49ers
32. Jacksonville Jaguars

Unfortunately, I have neither time nor data to truly adjust this for opponents/strength of schedule. I'm sure you can all find your own team which looks way out of place right now; the sore thumb is of course the Texans. Because of the turnover penalties, I think the system may bias towards good defenses, but I'm going to run it unchanged for at least a full season before I panic and start adjusting.

If you want to see my full data and formulas, email me at marshaldiaz, which is a gmail address. I can provide the spreadsheet I do the majority of the work on in at least xls or the native ods (OpenOffice) formats, maybe more.
Want to know how a team ended up where it is in the ranking? Ask me: I'll give at least a brief rundown, maybe some numbers.
Want to help collect data? I've been pulling data from NFL play-by-play, published on the gamecenters each week. This takes 15-25 minutes per game; I'll throw together a game-record sheet and email it to you: you fill it out (either from NFL or another play-by-play) and email it back. This could speed things up immensely.
Have ideas for improvement? I'd like to hear them, although I personally won't implement them this year - I'll send you a copy of my sheet and you can modify it as you like.

Formulas, method, and ranking data copyright Jonathan Frank, 2010. You can use this information freely for discussion, demonstration, ranting arguments with other fans, and discrediting other people's rankings and conclusions. If you're making money betting using my data, please let me know but I do not demand a cut. (If you want to give me a percentage anyway, go right ahead.) Ranking system and rankings may not be sold, rented, pimped, or otherwise directly used to make yourself money (not that I think they're solid enough yet to make money). You know, the usual copyright stuff.

Posted by: Jonadan on 29 Sep 2010

13 replies , Last at 18 Oct 2010, 1:04pm by Jonadan

Re: Team Drive-based Rating (TeDR) Debut
by Maximum :: Thu, 09/30/2010 - 12:28am

I like it. I think it could be a really useful tool. I mean you've got to wait to see if it turns out to have meaningful predictive value going forward but I don't see why it shouldn't. If i were going to suggest the next steps to improve this system I would probably suggest including pass interference as a positive for the offense because my feeling is usually your dealing with what would have been completions on those sorts of plays. Also eventually you're probably going to want to start adjusting for opponents, but neither of those things are pressing or represent a major issue with your rankings they're just things I'd think at some point in the future you could tweak.

Re: Team Drive-based Rating (TeDR) Debut
by tuluse :: Thu, 09/30/2010 - 7:08pm

Want to help collect data? I've been pulling data from NFL play-by-play, published on the gamecenters each week. This takes 15-25 minutes per game;

It shouldn't be hard to write a script that does this automatically.

additional penalties if the opponent scores off the turnover.

FO has done research that shows this is pretty random. Are you sure this useful for predictive purposes?

Re: Team Drive-based Rating (TeDR) Debut
by Jonadan :: Fri, 10/01/2010 - 10:22am

No, I was assuming it. This is a small enough change I'm going to take a look and see how it changes things. Thanks for the note.

Re: Team Drive-based Rating (TeDR) Debut
by Jonadan :: Fri, 10/01/2010 - 3:26pm

Dropping all turnovers to the same score results in the following rank changes:

1. Atlanta Falcons
2. Pittsburgh Steelers (+1)
3. Indianapolis Colts (-1)
4. Green Bay Packers
5. Philadelphia Eagles (+2)
6. New England Patriots (-1)
7. Tennessee Titans (+1)
8. Kansas City Chiefs (-1)
9. New York Jets
10. Chicago Bears (+2)
11. Seattle Seahawks (-1)
12. San Diego Chargers (-1)
13. St Louis Rams
14. Miami Dolphins
15. Minnesota Vikings (+3)
16. Cincinnati Bengals
17. Dallas Cowboys (+2)
18. Denver Broncos (-3)
19. New Orleans Saints (+1)
20. Baltimore Ravens (-3)
21. Tampa Bay Buccaneers (+1)
22. Cleveland Browns (+2)
23. Washington Redskins (-2)
24. Detroit Lions (-1)
25. Oakland Raiders
26. Houston Texans
27. Arizona Cardinals
28. Carolina Panthers (+1)
29. New York Giants (-1)
30. San Francisco 49ers (+1)
31. Buffalo Bills (-1)
32. Jacksonville Jaguars

Based on only three games, there are very few significant moves here. I'm going to arbitrarily define a move of more than 2 as significant for predictive purposes, giving the following:

15. Minnesota Vikings (+3). The Vikings so far give up points almost half the time they turn it over, but have few takeaways of their own and only one score off a turnover.
18. Denver Broncos (-3). The Broncos have few turnovers either way, so reducing them all to the same flattens them out as they so far have scored on more than they allow points on.
20. Baltimore Ravens (-3). Not sure why the Ravens dropped. They turn the ball over a lot, and haven't generated many turnovers themselves, so it shouldn't have affected them that much - unless they're turning the ball over more than other teams, and they were just lucky.

My conclusion: Based on 3 weeks of data, I like the feel of the original metric better. But I will keep an eye on it.

Re: Team Drive-based Rating (TeDR) Debut
by DeltaWhiskey :: Fri, 10/01/2010 - 12:16pm

ATL is clearly ranked too high because DVOA says so. DVOA is way better than this. DVOA use a way more complex formula that is shrouded in many more mysteries.

There, got that off my chest. Now for a question, and one that I have for DVOA as well. Are you using some sort of statistical modeling procedures to construct these calculations and determine the appropriate weight for the cost of a turnover?

Re: Team Drive-based Rating (TeDR) Debut
by Jonadan :: Fri, 10/01/2010 - 3:17pm

No, I'm using randomly assigned values given my own personal feel for football. As we get deeper in the season I may try to finagle the value to reflect things more accurately, but at this point I haven't gotten that far.

Re: Team Drive-based Rating (TeDR) Debut
by DeltaWhiskey :: Fri, 10/01/2010 - 4:24pm

Why not take your various values and use some sort of regression analysis?

Latest Round of Adjustments
by Jonadan :: Mon, 10/04/2010 - 2:40pm

This week made it really clear that my treatment of special teams is faulty (Exhibit A: Seattle). I'm slowly coming around to the position that even a good ST unit just isn't predictable enough in the long term to base huge chunks of a ranking on. Especially since I basically only count returns (e.g. Hester, Washington).

Gonna have fun working out how to adjust this. Part of me is tempted to just dump the project, but a single abysmal week doesn't seem like an entirely valid reason to waste the previous 3 weeks' work.

Re: Latest Round of Adjustments
by Jonadan :: Tue, 10/05/2010 - 7:10pm

And then the Pats go and do that.

Dammit, I'm leaving this thing alone until I have a full season's worth of looks to see how off I am where.

Re: scripting. If I had any idea how to do it, you're right, it's probably pretty possible. But I don't.

Re: Team Drive-based Rating (TeDR) Debut
by Jonadan :: Wed, 10/06/2010 - 10:42pm

Well, I've fought this for a while, and apparently CSS doesn't work here so direct export -> copy/paste won't work. I need to learn some more about computers, I think.

The full table, with the base TeDR, is here; apparently it will download, I don't know why. (See above comment about computers.) The numbers below are the ones I've adjusted after Week 4, which I'm calling VELA (Versus-Entire League Adjusted) because I like acronyms. If you've got questions I'll be glad to discuss them.

1. New England Patriots 74.76
2. New York Jets 41.05
3. Pittsburgh Steelers 38.10
4. San Diego Chargers 37.11
5. Green Bay Packers 35.96
6. Atlanta Falcons 34.06
7. Kansas City Chiefs 33.86
8. Indianapolis Colts 33.76
9. Philadelphia Eagles 21.56
10. St Louis Rams 20.26
11. Tennessee Titans 15.88
12. Denver Broncos 6.04
13. Baltimore Ravens 1.77
14. Seattle Seahawks 1.71
15. Washington Redskins 1.36
16. Chicago Bears 0.72
17. Cincinnati Bengals -1.18
18. Dallas Cowboys -4.18
19. Minnesota Vikings -4.82
20. New Orleans Saints -5.46
21. Tampa Bay Buccaneers -7.04
22. Houston Texans -12.37
23. Cleveland Browns -13.43
24. New York Giants -14.09
25. Detroit Lions -17.00
26. Miami Dolphins -28.22
27. Oakland Raiders -32.55
28. Carolina Panthers -34.87
29. San Francisco 49ers -45.68
30. Jacksonville Jaguars -52.48
31. Arizona Cardinals -57.77
32. Buffalo Bills -64.25

Re: Team Drive-based Rating (TeDR) Debut
by Jonadan :: Sun, 10/10/2010 - 10:23pm

Okay, I've got this ranking up separately here:


Will continue to update and improve there, if you want to comment go there.

Re: Team Drive-based Rating (TeDR) Debut
by StatFan81 (not verified) :: Fri, 10/15/2010 - 12:00pm

How is your model using Yards per point?

I would think that yards per point might have an inverse relationship with team rating. Scoring points without a lot of yards might be a good sign for a team. However, it might also be good to be able to score after long drives.

Re: Team Drive-based Rating (TeDR) Debut
by Jonadan :: Mon, 10/18/2010 - 1:04pm

Never mind all this, I don't have the time to keep it up. Guess it was bound to happen.

Login or register to post comments