Writers of Pro Football Prospectus 2008

Most Recent FO Features

JohnsonKer18.jpg

» Seventh Day Adventure: Week 13

The biggest game this week is the Iron Bowl, where the playoff hopes of Alabama, Auburn, and Georgia hang in the balance.

28 Sep 2016

FEI Week 4: Ratings Check

by Brian Fremeau

It is still very early in the college football season, but we are at the point at which every team has played at least two games against FBS opponents, and a significant portion of the year's non-conference games are already in the books. To date, 150 of the 194 games played between FBS opponents (77.3 percent) have been non-conference matchups. Only 48 of the 518 remaining regular season FBS games (9.3 percent) are non-conference matchups, and most of those remaining non-conference games involve only four teams: independents Army, BYU, Massachusetts, and Notre Dame.

There is more connectivity in game results between teams in different conferences right now than between those within the same conference. Western Michigan has the nation's best record against the Big Ten (2-0). Boise State sits alongside Stanford with the best record against the Pac-12 (2-0) to date. Oklahoma has more common opponents with the AAC and Big Ten so far (losses to Houston and Ohio State) than it does with any of its fellow Big 12 teams (no games yet).

FEI ratings at this point in the season still lean significantly on preseason projections (43 percent). We still need many more of those conference games to connect game results more reliably. If Oklahoma rolls through its Big 12 schedule, it will reflect even more favorably on the Cougars and Buckeyes. If Oklahoma struggles, those victories over the Sooners may not be as meaningful for those programs. The FEI ratings don't reliably know enough yet about Oklahoma, Houston, or Ohio State based only on games played this season, so preseason projections serve to supplement those results for now.

Preseason FEI ratings have correctly identified game winners in 147 out of the 194 games played to date. FEI ratings updated weekly based on a combination of preseason data and game results have also correctly identified game winners in 147 out of the 194 games played to date.

I received a few inquiries in the last week or so about the teams in which the FEI ratings and Bill Connelly's S&P+ ratings differ most significantly. The Florida Gators (7th in S&P+ this week, 50th in FEI) and West Virginia Mountaineers (48th in S&P+, 11th in FEI) are two of the most apparent examples, but they're not alone. There are ten teams in which S&P+ and FEI disagree by at least 30 ranking positions, five times as many teams that differed by 30 or more spots in last year's final rankings. 30 teams differ by at least 20 ranking positions, while only 11 teams did so at the end of last year. The average difference in ranking position between S&P+ and FEI at the end of last year was 8.04 spots; the average difference is 12.56 spots this week.

One of the biggest reasons Bill and I combine our two rating systems to produce F/+ is to identify these differences and dig a little deeper into the different narratives that play-by-play and drive data might be telling us. Last season, for example, Houston was very successful in terms of drive results -- 11th nationally in net points per drive, and top 20 ranks in both offensive and defensive points per drive. But the Cougars weren't as consistently successful at the play-by-play level -- 43rd in offensive success rate, 41st on defense -- and were the beneficiary of significant turnover luck (65 percent fumble recovery rate). Those play-by-play and drive efficiency distinctions had a big impact on the difference between Houston's 2015 FEI rating (10th overall) and its S&P+ rating (44th) at the end of the year.

The distinctions between the two systems this year may have something to do with play-by-play versus drive efficiency, but the distinctive ways in which S&P+ and FEI handle opponent adjustments, particularly early in the year, may be an even bigger factor. Consider Florida. The Gators rank 44th in offensive success rate through four games, and 16th in defensive success rate at the play level. At the drive level, Florida ranks 40th in offensive points per drive and ninth in defensive points per drive -- similar numbers; in fact slightly more favorable drive success than play success. So why do the Gators rank so low in drive-based FEI and so high in play-based S&P+?

Distinctions in the way we work with opponent adjustments are significant here. The teams that have been exceptionally efficient in terms of unadjusted data are similar in both systems -- Ohio State, Michigan, Louisville, Miami, and Alabama all rank among the top 10 in S&P+ success rates and in the top 10 in my raw game efficiency data. Only three of those teams (Alabama, Ohio State, and Louisville) have played a top-30 opponent so far this year according to FEI, and those teams rank more highly than Michigan (10th; best wins against No. 54 Penn State and No. 58 Colorado) and Miami (29th; best win against No. 67 Appalachian State) as a result. All five of those teams rank among the top six in S&P+.

Florida has the 15th best game efficiency rating on the year, but the Gators lost to Tennessee this weekend, their only game against a top-90 opponent so far this year. The FEI ratings aren't particularly impressed with dominant wins over bad or awful opponents since those results could also be produced by an average team. S&P+ rates the Gators with the potential to be much better than average because it weights game results themselves a bit more than the level of competition those results were produced against.

On the flip side of that is West Virginia -- 34th in game efficiency, 32nd in offensive success rate, 49th in defensive success rate, all marks of a solid but not spectacularly efficient team. Why then does FEI rate the Moutaineers so much higher (11th) than S&P+ (48th)? They have only played the 75th toughest schedule to date, with FBS wins over Missouri and BYU. This is actually something I may need to take a closer look at in the offseason, but my opponent adjustment method appears to be especially rewarding for teams like the Mountaineers who don't have any results against poor opponents. Michigan State is another example -- unexceptional raw efficiency rates, good FBS opponents, 47th in S&P+, 16th in FEI.

With FEI and S&P+, we both attempt to strike a balance between raw game data and opponent adjustments, but the different weights on these factors will produce some dramatically different results. And of course, the ratings of a given team's opponents are still fueled in part by preseason projections. Three weeks from now, that won't be the case, and both my FEI ratings and Bill's S&P+ ratings will be produced exclusively from 2016 game results. I'll plan on taking another look at the distinctions between the two rating systems later in the year.

FEI Ratings Through Week 4

The Fremeau Efficiency Index (FEI) is a college football rating system based on opponent-adjusted drive efficiency. Approximately 20,000 possessions are contested annually in FBS vs. FBS games. First-half clock-kills and end-of-game garbage drives and scores are filtered out. Unadjusted game efficiency (GE) is a measure of net success on non-garbage possessions, and opponent adjustments are calculated with special emphasis placed on quality performances against good teams, win or lose. Overall SOS ratings represent the likelihood than an elite team (two standard deviations better than average) would go undefeated against the given team's entire schedule.

FEI ratings through the first six weeks of the season are based in part on preseason projection data. Preseason ratings (PRE) represent approximately 43 percent of this week's ratings. Strength of schedule ratings for games played to date (SOP) are also provided. Ratings for all teams are linked here.

Rk Team Rec FEI GE Rk SOS Rk PRE Rk SOP Rk
1 Alabama 4-0 .332 .318 8 .079 9 .290 1 .455 16
2 Clemson 3-0 .297 .112 32 .241 51 .235 3 .630 33
3 Stanford 3-0 .272 .163 25 .251 53 .183 7 .610 29
4 Ohio State 3-0 .243 .418 4 .171 31 .154 12 .745 58
5 Texas A&M 3-0 .238 .139 28 .045 5 .111 24 .547 25
6 Mississippi 1-2 .229 .105 33 .043 4 .182 8 .256 6
7 Houston 3-0 .223 .298 10 .382 76 .104 26 .720 53
8 Louisville 4-0 .221 .472 2 .109 16 .137 17 .757 61
9 LSU 1-2 .217 -.016 70 .050 6 .236 2 .554 26
10 Michigan 4-0 .216 .392 5 .187 37 .123 22 .923 107
11 West Virginia 2-0 .197 .097 34 .370 75 .056 41 .819 75
12 Florida State 2-1 .193 -.031 74 .107 14 .160 11 .303 8
13 Wisconsin 4-0 .176 .211 18 .107 15 .058 40 .444 15
14 UCLA 2-2 .170 .041 54 .123 19 .133 20 .227 4
15 Tennessee 4-0 .166 .127 29 .128 20 .161 10 .840 85
Rk Team Rec FEI GE Rk SOS Rk PRE Rk SOP Rk
16 Michigan State 1-1 .166 -.097 90 .225 48 .141 16 .668 38
17 Oregon 1-2 .165 .049 51 .197 42 .187 5 .697 44
18 Nebraska 4-0 .152 .214 17 .183 35 .134 19 .780 67
19 Oklahoma 1-2 .139 -.012 68 .137 21 .200 4 .420 12
20 Utah 3-0 .138 .118 30 .308 66 .091 30 .839 84
21 Baylor 3-0 .131 .239 12 .305 65 .181 9 .930 108
22 Arkansas 3-1 .129 .060 48 .060 7 .134 18 .493 20
23 BYU 1-3 .123 -.015 69 .191 38 .104 27 .431 13
24 Auburn 2-2 .123 .063 46 .019 1 .103 28 .248 5
25 Boise State 3-0 .119 .222 16 .718 121 .109 25 .940 111
26 Iowa 3-0 .112 .327 7 .414 85 .085 32 .970 121
27 Notre Dame 1-3 .109 .062 47 .256 55 .145 14 .734 57
28 Georgia Tech 2-1 .109 .071 44 .225 49 .081 35 .487 19
29 Miami 2-0 .104 .304 9 .325 69 .038 48 .939 110
30 Western Michigan 3-0 .102 .201 20 .779 126 .028 59 .933 109

Posted by: Brian Fremeau on 28 Sep 2016

3 comments, Last at 01 Oct 2016, 1:29am by peepshowmopguy

Comments

1
by danplatt17 :: Thu, 09/29/2016 - 10:29am

With the high AP ranking, I'm surprised to see the Washington Huskies ranked so low... I assume that's a function of their weak schedule so far?

2
by Brian Fremeau :: Thu, 09/29/2016 - 12:12pm

Yes, they haven't played a top-70 opponent so far according to FEI. Similar problem that held Florida back before the Tennessee game.

3
by peepshowmopguy :: Sat, 10/01/2016 - 1:29am

So any guesses what that might look like now that UW trounced Stanford?