Writers of Pro Football Prospectus 2008

Most Recent FO Features

JohnsonKer18.jpg

» Seventh Day Adventure: Week 13

The biggest game this week is the Iron Bowl, where the playoff hopes of Alabama, Auburn, and Georgia hang in the balance.

07 Dec 2016

FEI Week 14: Hits and Misses

by Brian Fremeau

We have reached the conclusion of the regular season, and it's time to take stock of how well the FEI preseason projections fared. The top two seeds in the College Football Playoff -- Alabama and Clemson -- had the two highest mean win projections to start the year. They were also the national champion and national runner-up last year, of course, and pretty much everyone had the Crimson Tide and Tigers back in the playoff hunt again this season. Both teams reloaded and both delivered.

Ohio State and Washington, the No. 3 and No. 4 seeds in the playoff, both delivered and exceeded expectations this year. FEI projections had both the Buckeyes and Huskies ranked among the top 15 to start the year, but both were expected to trip up one or two more times than either actually did. The FEI ratings fully responded only after the Buckeyes notched a dominant road victory over Oklahoma in Week 3 and after the Huskies obliterated Stanford and Oregon in consecutive weeks in early October.

How should we measure the success of the projection model? Projecting the best team in four of the Power 5 conferences isn't really something to boast about when pretty much every system and poll also forecasted success for Alabama, Clemson, Ohio State, and Oklahoma (projected No. 5 by FEI). There were certainly as many big misses. How did the projection model perform overall?

The correlation of the preseason FEI ratings to the Week 14 FEI ratings stands at 0.69, consistent with preseason projection results over the last several years. Thirty-eight out of 128 teams were projected within one-quarter of one standard deviation of their current FEI rating. Sixty-two were projected within one-half of one standard deviation, and 102 were projected within one standard deviation.

That leaves 26 teams that the FEI projections whiffed on the most, by more than one standard deviation. Six of those teams are in the Big Ten.

Penn State (0.009 projected FEI, No. 66; 0.188 current FEI, No. 6) made one of the most impressive runs over the second half of the season, culminating in a Big Ten championship and trip to the Rose Bowl. The Nittany Lions had zero 90th percentile opponent-adjusted performances over the first half of the year and three times recorded a single-game performance at the 46th percentile or worse. Wins over Ohio State, Iowa, and Wisconsin over the final seven weeks of the year all now rank among the 50 best wins of the year.

Michigan (0.123 projected FEI, No. 22; 0.257 current FEI, No. 3) had one of the hottest starts of the year, recording a 90th percentile victory in each of its first five games before cooling off somewhat down the stretch. The Wolverines' overtime loss to Ohio State on November 26 ranks as the best loss any team has taken this year, and was likely the only thing standing between Michigan and the playoff berth claimed by the Buckeyes.

Wisconsin (0.058 projected FEI, No. 40; 0.192 current FEI, No. 7) overcame one of the nation's toughest schedules to win 10 games on the strength of one of the nation's best defenses. Their three losses against teams ranked No. 2, No. 3, and No. 6 in FEI this week each came by a single score.

Ohio State (0.154 projected FEI, No. 12; 0.295 current FEI, No. 2) is the other Big Ten team that exceeded its projection by more than one standard deviation.

Between the regular season and the conference championship game, a full round-robin was contested between those four teams. Since most of those games were competitive, those teams each benefitted in the FEI ratings by their collective strength. And they rose toward the top of the FEI ratings overall on the strength of several key out-of-conference victories, wins over No. 11 and Pac-12 runner up Colorado, No. 13 LSU, and No. 14 Oklahoma. There were a total of 32 losses by teams currently ranked in the FEI top 15 this year, and the top four teams in the Big Ten handed out nine of those losses.

The entire Big Ten was not underrated by FEI to start the season, however. Some were overrated significantly. Michigan State (0.141 projected FEI, No. 16; -0.014 current FEI, No. 63) had a very disappointing year, and Rutgers (-0.038 projected No. 81; -0.199 current FEI, No. 122) had a spectacularly poor one. Both of those teams are also counted among the teams that ended the year more than one standard deviation separated from where they began.

The Big Ten was certainly a conference of extremes, which makes any debate about the merits and challenges of other leagues more complex. My interactive strength of schedule chart, now updated with bowl games, helps illustrate the problem. The best teams in the Big Ten all played one another, and all have a very good bowl opponent still to come. For an elite team, any of the schedules of Ohio State, Michigan, Penn State, or Wisconsin would be very challenging to navigate unscathed, and those team schedules rank among the toughest through that lens.

But shifting the focus to how an average team would do against the same schedule, and the Big Ten doesn't appear any more distinctive than the other conferences. Games against the Big Ten's best would be a huge challenge, but there are plenty of wins to be had against the bulk of the league. In fact, Alabama's schedule -- very few heavyweights, but totally loaded with top 40 opponents -- ranks as the toughest schedule for an average team.

One more round of data points will roll in over the course of bowl season, then it's time to start revisions on the projection model to make next year's forecast even more accurate.

FEI Ratings Through Week 14

The Fremeau Efficiency Index (FEI) is a college football rating system based on opponent-adjusted drive efficiency. Approximately 20,000 possessions are contested annually in FBS vs. FBS games. First-half clock-kills and end-of-game garbage drives and scores are filtered out. Unadjusted game efficiency (GE) is a measure of net success on non-garbage possessions, and opponent adjustments are calculated with special emphasis placed on quality performances against good teams, win or lose. Overall SOS ratings represent the likelihood than an elite team (two standard deviations better than average) would go undefeated against the given team's entire schedule.

Offensive FEI (OFEI) is value generated per offensive non-garbage possession adjusted for the strength of opponent defenses faced. Defensive FEI (DFEI) is value generated per opponent offensive non-garbage possession adjusted for the strength of opponent offenses faced. Special Teams Efficiency (STE) is the average value generated per non-garbage possession by a team's non-offensive and non-defensive units. Strength of schedule ratings for games played to date (SOP) are also provided.

Ratings for all teams are linked here.

Rk Team Rec FEI GE Rk SOS Rk OFEI Rk DFEI Rk STE Rk SOP Rk
1 Alabama 12-0 .338 .317 2 .102 22 .77 16 1.92 1 .03 50 .213 37
2 Ohio State 11-1 .295 .275 4 .074 8 1.04 7 1.28 2 .02 55 .136 22
3 Michigan 10-2 .257 .288 3 .087 13 .99 8 .98 7 .21 2 .110 13
4 Washington 11-1 .253 .331 1 .097 18 1.11 5 1.22 3 .07 26 .441 88
5 Clemson 11-1 .253 .183 9 .094 16 .93 10 .71 11 .00 62 .278 53
6 Penn State 11-2 .197 .138 20 .063 5 .70 20 .33 35 .07 25 .119 16
7 Wisconsin 10-3 .192 .140 19 .049 2 .37 37 1.02 6 -.02 74 .095 7
8 Auburn 7-4 .181 .157 12 .057 4 .52 29 .81 9 .08 17 .075 4
9 Western Michigan 12-0 .181 .271 5 .496 101 1.36 2 .39 28 .05 30 .722 123
10 Boise State 10-2 .180 .208 8 .497 102 .76 17 .76 10 -.08 112 .510 98
11 Colorado 9-3 .177 .091 34 .076 10 .16 52 1.19 4 -.09 116 .163 25
12 USC 9-3 .175 .179 10 .043 1 .60 25 .42 27 .15 4 .061 2
13 LSU 6-4 .172 .123 24 .091 14 .50 30 1.04 5 -.16 127 .114 15
14 Oklahoma 10-2 .171 .157 13 .167 35 1.18 3 .04 55 .02 52 .228 41
15 Miami 7-4 .168 .119 25 .262 56 .09 60 .53 18 .07 22 .306 55
Rk Team Rec FEI GE Rk SOS Rk OFEI Rk DFEI Rk STE Rk SOP Rk
16 Florida State 8-3 .154 .087 36 .092 15 .79 15 .20 46 -.09 115 .175 29
17 Louisville 9-3 .151 .222 6 .186 40 .89 11 .57 15 -.04 89 .247 48
18 Houston 8-3 .144 .137 21 .376 76 .32 40 .53 17 .04 38 .398 74
19 Washington State 8-3 .136 .146 17 .199 42 .85 13 .26 43 .04 34 .216 38
20 Virginia Tech 8-4 .131 .108 29 .177 37 .11 57 .46 25 .04 43 .342 62
21 Western Kentucky 9-3 .127 .213 7 .144 30 .96 9 .28 39 .12 9 .172 28
22 Pittsburgh 7-4 .125 .051 48 .115 26 .89 12 -.44 95 .03 45 .127 18
23 Iowa 8-3 .124 .110 28 .212 46 .25 44 .37 30 .11 11 .239 44
24 Tennessee 7-4 .118 .048 49 .168 36 .55 27 -.08 66 .07 21 .177 30
25 Temple 9-3 .117 .161 11 .337 67 .23 46 .46 24 .12 10 .398 75
26 West Virginia 9-2 .115 .093 32 .349 68 .24 45 .55 16 -.04 90 .458 89
27 North Carolina 6-4 .115 .042 50 .283 58 .43 35 -.38 90 .11 13 .323 60
28 South Florida 9-2 .114 .142 18 .479 98 1.08 6 -.15 72 .00 61 .484 95
29 BYU 7-4 .112 .098 31 .364 73 .01 65 .46 23 .14 5 .392 72
30 Memphis 7-4 .110 .147 16 .395 82 .56 26 -.07 65 .21 1 .471 90

Posted by: Brian Fremeau on 07 Dec 2016

2 comments, Last at 11 Dec 2016, 10:00pm by Brian Fremeau

Comments

1
by roguerouge :: Sun, 12/11/2016 - 6:15am

What happened with Navy? Back to back losses, after last week's article on its historic offense?

2
by Brian Fremeau :: Sun, 12/11/2016 - 10:00pm

Jinxed. And a QB injury.