FEI Week 14: Hits and Misses

by Brian Fremeau

We have reached the conclusion of the regular season, and it's time to take stock of how well the FEI preseason projections fared. The top two seeds in the College Football Playoff -- Alabama and Clemson -- had the two highest mean win projections to start the year. They were also the national champion and national runner-up last year, of course, and pretty much everyone had the Crimson Tide and Tigers back in the playoff hunt again this season. Both teams reloaded and both delivered.

Ohio State and Washington, the No. 3 and No. 4 seeds in the playoff, both delivered and exceeded expectations this year. FEI projections had both the Buckeyes and Huskies ranked among the top 15 to start the year, but both were expected to trip up one or two more times than either actually did. The FEI ratings fully responded only after the Buckeyes notched a dominant road victory over Oklahoma in Week 3 and after the Huskies obliterated Stanford and Oregon in consecutive weeks in early October.

How should we measure the success of the projection model? Projecting the best team in four of the Power 5 conferences isn't really something to boast about when pretty much every system and poll also forecasted success for Alabama, Clemson, Ohio State, and Oklahoma (projected No. 5 by FEI). There were certainly as many big misses. How did the projection model perform overall?

The correlation of the preseason FEI ratings to the Week 14 FEI ratings stands at 0.69, consistent with preseason projection results over the last several years. Thirty-eight out of 128 teams were projected within one-quarter of one standard deviation of their current FEI rating. Sixty-two were projected within one-half of one standard deviation, and 102 were projected within one standard deviation.

That leaves 26 teams that the FEI projections whiffed on the most, by more than one standard deviation. Six of those teams are in the Big Ten.

Penn State (0.009 projected FEI, No. 66; 0.188 current FEI, No. 6) made one of the most impressive runs over the second half of the season, culminating in a Big Ten championship and trip to the Rose Bowl. The Nittany Lions had zero 90th percentile opponent-adjusted performances over the first half of the year and three times recorded a single-game performance at the 46th percentile or worse. Wins over Ohio State, Iowa, and Wisconsin over the final seven weeks of the year all now rank among the 50 best wins of the year.

Michigan (0.123 projected FEI, No. 22; 0.257 current FEI, No. 3) had one of the hottest starts of the year, recording a 90th percentile victory in each of its first five games before cooling off somewhat down the stretch. The Wolverines' overtime loss to Ohio State on November 26 ranks as the best loss any team has taken this year, and was likely the only thing standing between Michigan and the playoff berth claimed by the Buckeyes.

Wisconsin (0.058 projected FEI, No. 40; 0.192 current FEI, No. 7) overcame one of the nation's toughest schedules to win 10 games on the strength of one of the nation's best defenses. Their three losses against teams ranked No. 2, No. 3, and No. 6 in FEI this week each came by a single score.

Ohio State (0.154 projected FEI, No. 12; 0.295 current FEI, No. 2) is the other Big Ten team that exceeded its projection by more than one standard deviation.

Between the regular season and the conference championship game, a full round-robin was contested between those four teams. Since most of those games were competitive, those teams each benefitted in the FEI ratings by their collective strength. And they rose toward the top of the FEI ratings overall on the strength of several key out-of-conference victories, wins over No. 11 and Pac-12 runner up Colorado, No. 13 LSU, and No. 14 Oklahoma. There were a total of 32 losses by teams currently ranked in the FEI top 15 this year, and the top four teams in the Big Ten handed out nine of those losses.

The entire Big Ten was not underrated by FEI to start the season, however. Some were overrated significantly. Michigan State (0.141 projected FEI, No. 16; -0.014 current FEI, No. 63) had a very disappointing year, and Rutgers (-0.038 projected No. 81; -0.199 current FEI, No. 122) had a spectacularly poor one. Both of those teams are also counted among the teams that ended the year more than one standard deviation separated from where they began.

The Big Ten was certainly a conference of extremes, which makes any debate about the merits and challenges of other leagues more complex. My interactive strength of schedule chart, now updated with bowl games, helps illustrate the problem. The best teams in the Big Ten all played one another, and all have a very good bowl opponent still to come. For an elite team, any of the schedules of Ohio State, Michigan, Penn State, or Wisconsin would be very challenging to navigate unscathed, and those team schedules rank among the toughest through that lens.

But shifting the focus to how an average team would do against the same schedule, and the Big Ten doesn't appear any more distinctive than the other conferences. Games against the Big Ten's best would be a huge challenge, but there are plenty of wins to be had against the bulk of the league. In fact, Alabama's schedule -- very few heavyweights, but totally loaded with top 40 opponents -- ranks as the toughest schedule for an average team.

One more round of data points will roll in over the course of bowl season, then it's time to start revisions on the projection model to make next year's forecast even more accurate.

FEI Ratings Through Week 14

The Fremeau Efficiency Index (FEI) is a college football rating system based on opponent-adjusted drive efficiency. Approximately 20,000 possessions are contested annually in FBS vs. FBS games. First-half clock-kills and end-of-game garbage drives and scores are filtered out. Unadjusted game efficiency (GE) is a measure of net success on non-garbage possessions, and opponent adjustments are calculated with special emphasis placed on quality performances against good teams, win or lose. Overall SOS ratings represent the likelihood than an elite team (two standard deviations better than average) would go undefeated against the given team's entire schedule.

Offensive FEI (OFEI) is value generated per offensive non-garbage possession adjusted for the strength of opponent defenses faced. Defensive FEI (DFEI) is value generated per opponent offensive non-garbage possession adjusted for the strength of opponent offenses faced. Special Teams Efficiency (STE) is the average value generated per non-garbage possession by a team's non-offensive and non-defensive units. Strength of schedule ratings for games played to date (SOP) are also provided.

Ratings for all teams are linked here.