Writers of Pro Football Prospectus 2008

Most Recent FO Features

WentzCar16-1.jpg

» Week 11 DVOA Ratings

DVOA has finally climbed on board the Wentz Wagon! The Eagles move into the No. 1 spot, but they aren't the only strong, well-balanced team in the NFL this year. New Orleans, Pittsburgh, and the Los Angeles Rams make this one of the best seasons ever for multiple teams over 30% in DVOA, and Minnesota isn't far behind.

18 Sep 2015

VN: A (Ratings) Change Will Do You Good

by Bill Connelly

So I made a pretty significant change in the way S&P+ is measured last year, and it only recently struck me that I haven't talked about it much here at Football Outsiders. I wrote a few posts about it at Football Study Hall last winter, but all I've done here is rewrite the actual S&P+ page.

So let's ask and answer a few questions regarding the whys and hows of this, the second major update I've made to my system of ratings. We'll start with the obvious.

Why? How?

I talked about that a little here. Basically, I liked S&P+ but thought it could be better and far more stable, so I started tinkering.

F/+ and S&P+ are good measures. They do just well enough in their predictive ability, and they pass the eyeball test for probably 95 percent of FBS teams, so I pretty frequently use them as reference points for what my eyes see. But they can always be better, and I'm always looking to make them better. (Quick differentiation for beginners: the S&P+ measure is mine, and F/+ is the combination of my numbers and Brian Fremeau's FEI.)

I mentioned a couple of times in November that I was tinkering with an overall ratings redesign. In early 2013, I rolled out a new approach to S&P+ that involved drive ratings, and with the Five Factors concept I've tossed around over the last year, I've been trying to figure out whether that could not only improve my analysis, but improve my ratings as well.

In November at Football Outsiders, I did some serious over-fitting to figure out how close I could get to explaining a team's percentage of points scored with success rate, IsoPPP, red zone success rate, turnovers and turnovers luck, sacks, and special teams. I got pretty close.

I started to realize over time that these ratings should fit far closer to a normal distribution than either my S&P+ or our combined F/+ actually do, so that, plus my Five Factors ideas, pushed me in this direction.

What Are Some Of The Most Obvious Changes?

Since I was more or less starting from scratch in how I approach this, I took the time to add in some other features as well. For instance, I introduced a concept called second-order wins.

There are exponents available for turning college football points into a Pythagorean win percentage, but I'm more interested in another concept: second-order wins. That basically takes the same idea but uses advanced stats of some sort to determine not simply what you did score and allow, but what you should have scored and allowed.

My new ratings are based on margins in categories related to my Five Factors: efficiency, explosiveness, field position, finishing drives, and turnovers/luck. As I flesh the system out with previous years of data, I'm able to basically use these margins to determine both a team's most likely scoring margin in a given game and, based on the plays that took place, that team's likelihood of winning a given game.

To further explain the second part of that last sentence, it basically says "If you took all the plays in this game, tossed them up in the air, and had them land in a random order, you'd win this game XX% of the time." It is a single-game win likelihood concept, and with it, we can look at wins and losses not as zeroes and ones, but as percentages. And if you're winning a lot of "You'd have won this game 60 percent of the time" games, you're probably getting a little bit lucky. And as with everything else, that luck is likely to change over time.

In the official S&P+ page, you can now find second-order wins next to teams' win totals.

Plus, since you can really present your ratings in any of about a hundred different ways, I began strapping S&P+ to the normal scoring distribution. I thought that the 100-point scale I've used for years -- in which a perfectly average offense or defense is given a 100.0 score, anything better is above 100, anything worse is below -- was fine, but presenting S&P+ as an adjusted scoring margin is more relatable. The current ratings show Ohio State's S&P+ rating as 29.5, meaning the Buckeyes are 29.5 points better than the average team. Poor Eastern Michigan, meanwhile, is 23.2 points worse (but rising!).

One issue that I began to realize in doing it this way, however, was that scoring averages change from year to year. College football has become a more offense-friendly game over the last 10 years, so if someone had a plus-25.0 rating in 2005, it wouldn't necessarily mean the same as a plus-25.0 rating in 2015. That's a problem since one of the most fun reasons for playing with ratings like this is comparing teams from different seasons.

Really, I could have simply shared the percentile, but I still like the adjusted margin concept, so I share both.

That's why you also see a percentile grade in the S&P+ tables. Percentiles are a concept most of us understand from our old standardized testing days in school, so when you see that Ohio State is at the 99.1 percentile, you know that the Buckeyes are pretty awesome.

And while the differences between years aren't enormous, they're there. In 2014, national champion and top-ranked S&P+ team Ohio State finished with a plus-30.2 rating, which landed the Buckeyes at 99.5 percent on the percentile scale. Meanwhile, in 2005, the top-ranked team was USC: 99.6 percent, but only plus-29.7. USC and, by decimal points, Texas grade out higher than 2014 Ohio State, but both had a lower S&P+ margin.

So You Were Looking For Something More Stable. Did You Find It?

I think so! Here's a comparison of the current S&P+ ratings through two weeks of the 2015 season, and what the old S&P+ process would have produced through the same number of games.

Team Actual S&P+ Actual Rank Previous Method Rank Diff
Ohio State 29.5 1 5 4
Alabama 27.3 2 6 4
Ole Miss 23.2 3 2 -1
Georgia 22.9 4 1 -3
Oregon 22.3 5 12 7
Baylor 21.6 6 18 12
UCLA 20.6 7 7 0
USC 19.7 8 8 0
Michigan State 19.5 9 23 14
TCU 19.2 10 20 10
Georgia Tech 19.1 11 9 -2
LSU 18.9 12 17 5
Florida State 18.8 13 4 -9
Notre Dame 17.6 14 22 8
Clemson 17.4 15 3 -12
Arkansas 16.7 16 14 -2
Oklahoma 16.5 17 16 -1
Auburn 16.5 18 19 1
Stanford 16.1 19 13 -6
Texas A&M 14.3 20 11 -9
Boise State 14.3 21 21 0
Missouri 14.2 22 41 19
Arizona State 13.9 23 30 7
Wisconsin 13.6 24 25 1
Tennessee 13.2 25 10 -15
Team Actual S&P+ Actual Rank Previous Method Rank Diff
Virginia Tech 13.2 26 28 2
Mississippi State 12.6 27 24 -3
South Carolina 12.1 28 29 1
Kansas State 11.9 29 40 11
Miami 10.6 30 34 4
Nebraska 10.4 31 26 -5
West Virginia 10.4 32 15 -17
Arizona 10.0 33 36 3
Michigan 10.0 34 27 -7
Utah 8.6 35 44 9
Louisville 8.6 36 35 -1
Penn State 8.4 37 32 -5
Florida 8.0 38 31 -7
Oklahoma State 8.0 39 49 10
Duke 7.3 40 55 15
BYU 7.3 41 48 7
Pittsburgh 7.2 42 46 4
North Carolina 6.6 43 45 2
Marshall 6.1 44 42 -2
N.C. State 6.0 45 50 5
Boston College 5.4 46 37 -9
California 5.0 47 47 0
Texas 4.8 48 38 -10
Texas Tech 4.2 49 57 8
Cincinnati 3.9 50 53 3
Team Actual S&P+ Actual Rank Previous Method Rank Diff
Minnesota 3.9 51 33 -18
Louisiana Tech 3.8 52 56 4
Iowa 3.6 53 52 -1
Temple 3.4 54 54 0
Navy 3.4 55 61 6
Northwestern 3.0 56 63 7
Memphis 2.4 57 65 8
Western Kentucky 2.2 58 59 1
Illinois 2.1 59 43 -16
Virginia 2.0 60 39 -21
Colorado State 1.5 61 78 17
Utah State 1.4 62 77 15
Washington 1.3 63 69 6
Northern Illinois 1.2 64 87 23
Toledo 0.6 65 67 2
Central Florida -0.1 66 71 5
Kentucky -0.2 67 51 -16
Washington State -0.5 68 68 0
Maryland -1.0 69 58 -11
San Diego State -1.2 70 75 5
East Carolina -1.6 71 84 13
Georgia Southern -1.6 72 62 -10
Air Force -2.5 73 74 1
Purdue -3.0 74 64 -10
Ball State -3.2 75 106 31
Team Actual S&P+ Actual Rank Previous Method Rank Diff
Western Michigan -3.3 76 80 4
Syracuse -3.4 77 76 -1
Houston -3.5 78 102 24
Arkansas State -3.6 79 60 -19
Vanderbilt -3.6 80 66 -14
Rutgers -3.9 81 79 -2
Appalachian State -4.0 82 72 -10
Indiana -4.3 83 82 -1
Nevada -4.4 84 94 10
Oregon State -4.5 85 89 4
Middle Tennessee -5.2 86 95 9
Iowa State -5.2 87 85 -2
Colorado -5.4 88 73 -15
UL-Lafayette -5.7 89 90 1
Tulsa -6.1 90 101 11
Kent State -6.5 91 70 -21
Wake Forest -6.5 92 86 -6
Bowling Green -6.7 93 99 6
Ohio -6.9 94 98 4
Central Michigan -7.2 95 93 -2
San Jose State -7.3 96 88 -8
Rice -8.4 97 103 6
New Mexico -8.8 98 113 15
UL-Monroe -9.2 99 91 -8
Fresno State -9.3 100 92 -8
Team Actual S&P+ Actual Rank Previous Method Rank Diff
Florida Atlantic -9.5 101 97 -4
Tulane -9.6 102 81 -21
South Florida -9.8 103 83 -20
Florida International -10.2 104 116 12
Texas State -10.3 105 96 -9
Old Dominion -10.7 106 110 4
Connecticut -11.0 107 115 8
Charlotte -11.4 108 105 -3
Southern Miss -12.2 109 117 8
South Alabama -12.6 110 111 1
Georgia State -13.4 111 100 -11
Hawaii -13.6 112 109 -3
Buffalo -13.6 113 127 14
SMU -14.3 114 118 4
Akron -14.5 115 119 4
Massachusetts -14.8 116 121 5
Wyoming -15.4 117 126 9
UTEP -15.7 118 107 -11
Miami (Ohio) -15.7 119 122 3
Troy -15.9 120 123 3
Army -16.2 121 125 4
Kansas -16.5 122 104 -18
UNLV -17.2 123 112 -11
Idaho -18.0 124 114 -10
New Mexico State -18.0 125 120 -5
North Texas -18.5 126 128 2
UTSA -18.8 127 108 -19
Eastern Michigan -23.2 128 124 -4

I accepted as part of the process that early-season rankings would be super-volatile with the old method. I mostly liked the end results, so I lived through it. Still, it would have made me a little queasy to head into Week 3 with an S&P+ top five of Georgia, Ole Miss, Clemson, Florida State, and then Ohio State.

The new system looks more stable and performs at the same level or higher from a predictions standpoint.

And here's the most interesting part: with the old method, I was phasing preseason projections out over a period of about seven weeks. At this point, projections would still make up about 70 or 75 percent of the ratings.

With the new system, which clearly looks more stable, I'm already down to 50 percent weight for the projections, and all projections will be phased out after Week 4. That's exciting to me (though it must be said that strange things excite me).

I still have some updating to do, by the way. The Offensive S&P+ and Defensive S&P+ pages will begin to get updated next week, and while I intended to do this long ago, I still need to update the offense and defense pages for 2005-13. Soon, I promise!

Posted by: Bill Connelly on 18 Sep 2015

1 comment, Last at 18 Sep 2017, 4:03pm by fileorgin

Comments

1
by fileorgin :: Mon, 09/18/2017 - 4:03pm

https://psiphon.me/
https://droid4x.org/ is a tool designed for smart
phones and Windows PC’s which allows the users to bypass the censorship
that their service provider has set and surfed the part of the web that is not allowed to be accessed in their part of the world.