FEI 2019: New and Improved

by Brian Fremeau

This year, I'm introducing a new version of the Fremeau Efficiency Index (FEI) college football ratings, which were first conceived of in 2002 and developed in the years since as an effort to better understand the game and better evaluate successful teams and units. This latest iteration of the FEI ratings system is the outcome of a significant offseason project I undertook oriented around two key goals. The first goal was to produce a more accessible and intuitive format for the ratings, and the second was to make the system more predictive.

One risk with producing and publishing a college football rating system is to draw more attention to the ranking of teams than to the rating of teams. It's natural to fixate on arguments like "Team A is five ranking positions better than Team B," as if that number of ranking positions is more valuable data than the delta and meaning of the respective ratings of the two teams. But the ranking, of course, is just the order in which the ratings are sorted and doesn't carry any inherent meaning of its own. That problem was aggravated by the uncertain meaning of my old FEI ratings format. What, for instance, did Alabama's 0.326 FEI rating mean? How do we compare that number with Clemson's 0.311 rating? Did the 0.015 delta between those ratings mean something specific? Was it referring to a percentage, or something else? Could it be easily interpreted and understood?

I tested a potential new version of the FEI ratings output last year with something I called Adjusted Possession Advantage (APA), but I hadn't had the opportunity then to run APA for past seasons and scrutinize its utility as a predictive or retrodictive measure. I did like the format, however -- a better way to cleanly represent a team's per-possession advantage over another team rather than the nebulous old FEI format. So I committed to restructuring the output of the new FEI ratings to adopt this format. The stat pages for overall FEI ratings, offensive FEI ratings, defensive FEI ratings, and special teams FEI ratings since 2007 have all been updated on this site and share the same format, now more clearly defined as a per-possession representation of team and unit strength. I have also published at-a-glance team history pages of all of the FEI ratings since 2007 on my own site.

The formatting issue was only part of the problem, of course. Was FEI optimized to predict, and if so, was it any good at doing so? To be honest, I had never given this question its due until this past offseason. I have been utilizing FEI as a forecasting tool for many years and had made several efforts to produce better predictions for seasons and for individual games. But I needed to take a much more thorough approach and figure out what FEI is and what it can and should do. I was particularly focused on challenging my own assumptions. For instance, FEI had for many years weighted certain results more than others. Games between teams of similar strength, for instance, were given more weight in the formula than games in which one of the opponents was far outmatched. Did the weights need to be tweaked? Were the weights needed at all? Was the weighting methodology a feature or a bug? Further, how would an optimized FEI rating perform better than one that is not optimized?

I decided first to focus my work on creating a revised rating model that produced more accurate individual game projections. That is, how well did a given week's FEI rating forecast the outcome of the next week's games? Since FEI ratings require possession data inputs, I chose to focus my attention specifically on the predictive quality of FEI ratings to forecast games in the second half of seasons, from Week 7 through the end of the postseason. And to measure accuracy, I focused on measuring the delta between game scoring margin predictions and the actual game scoring margin results, then calculating the mean error of the predictions. The new FEI ratings were ultimately the most predictive, producing the lowest mean error. They were also the most simplified in terms of their calculation. None of the game weights I tested proved to be better than removing the weights altogether.

I then investigated the season predictive power of the ratings. Were weighted five-year ratings histories still solid baseline predictors of next-year success? Did transitional data inputs like returning production and recruiting ratings make those predictions even more accurate? Yes, and the new FEI ratings outperformed the old ratings every step of the way.

It's important to point out that better predictive ratings and a better output format did not necessarily result in "more palatable" or "better eye test" ratings. Alabama still ranks ahead of Clemson in 2018, and in fact, several of the national championship teams since 2007 are not ranked No. 1 in that year's new FEI ratings. This is definitely a feature, not a bug. FEI isn't designed to name a champion, it's designed to measure per-possession efficiency and to apply that measurement to make better predictions of future success. Dismissing a rating system that "looks wrong" is not the right approach, something I had to come to grips with myself. Especially given evidence of the new system's effectiveness to do what it is designed to do and make better predictions, ratings that "look wrong" are an invitation to dig a little deeper and uncover an insight about what they actually do reveal.

With that introduction, I present the 2019 Projected FEI Ratings. Applied over the last three seasons, this preseason projection methodology resulted in successfully forecasting the winner of 69.2 percent of all regular season games. More importantly, and since new possession data will be recorded and will better inform the projection model throughout the year, the preseason Projected FEI ratings have a 76.8 percent win percentage for Weeks 0-2 when no such in-season data is available. (Betting lines successfully projected the winner of only 75.3 percent of games in Weeks 0-2 over the same span). For those early week projections, the mean error (average difference between projected margin and actual margin) was 11.6 points per game, a projection improvement of more than 1.5 points per game over the old FEI system.

Projected FEI ratings (PFEI) represent the per-possession scoring advantage a team would be expected to have on a neutral field over an average opponent. Projected losses (PL) represents the average number of losses expected based on the team's PFEI rating and that of its opponents produced from individual game win likelihoods. Projected OFEI Offense ratings (POFEI) and Projected DFEI Defense ratings (PDFEI) represent per-possession advantages for each unit.

There isn't much variance between these projections and the national consensus at the very top, and that's not at all surprising. Alabama and Clemson have met in the College Football Playoff four straight seasons (and have won 11 of the last 12 CFP games played) and are best positioned to be in the running again for playoff bids this fall. Right behind them are programs that have the talent, recent history, and pedigree to be in the mix as well: Georgia, Ohio State, Oklahoma, and LSU. But after that, there are some notable differences between FEI and the national consensus.

No. 50 Texas Longhorns

Texas racked up a couple of impressive victories in 2018 (they went 1-1 against Oklahoma and beat Georgia) and has earned attention as a potential playoff contender as a result. Among other accolades, the Associated Press writers voted the Longhorns No. 10 in their preseason poll. So what is FEI seeing that is dragging them all the way down to No. 50? Their projection input data simply isn't very impressive, and that's all this system has to work with. Texas ranked No. 28 in last season's final FEI ratings, coupling those great wins with a number of very pedestrian results against weak opponents. They also have very little production returning this fall in comparison to other possible contending programs. Even if FEI is blind to Texas' 2019 upside potential, it would take a major leap in efficiency on both sides of the ball to comfortably forecast the Longhorns winning double-digit regular season games.

No. 27 Oregon Ducks

The Ducks are also a dark-horse playoff contender in the eyes of some prognosticators (No. 11 in the Associated Press poll), but FEI doesn't see it. They have solid returning production numbers, but not a very strong recent history. The Ducks have only one win against an FEI top-30 opponent in the last three seasons combined, and nine of their 18 losses in the same span have come against teams ranked outside of the top 30. With games away from Autzen Stadium against Auburn and Stanford in September, they'll have opportunities to impress, but will have to prove it before FEI will recognize that potential.

No. 22 Texas A&M Aggies

The Aggies don't jump out as terribly overrated or underrated, but are worth discussing due to their unusually challenging schedule. They'll face No. 1 Alabama, No. 2 Clemson, No. 3 Georgia, and three other SEC West teams ranked in the FEI top 10. If Texas A&M were projected to be an elite team in terms of possession efficiency, they would still struggle to navigate that kind of slate without suffering at least two losses. Since FEI projects them to be merely good and not elite, they are staring down five or six projected losses, with worst-case scenarios looming if they suffer any injury setbacks. No team in the FEI projected top 30 has anywhere near as many landmines to avoid.

No. 7 Mississippi State Bulldogs
No. 8 Auburn Tigers

Speaking of those SEC West teams, FEI is demonstrably higher on these programs in comparison to the national consensus, certainly as it relates to their ranking if not to their respective forecasted records. The Bulldogs project to have the nation's third-best defense this fall according to FEI and still project to lose four times. Auburn projects to have a top-20 offense and defense (one of only eight teams with such projected balance) but has 4.5 projected losses due to their brutal schedule. The rankings may appear to be lofty for Mississippi State and Auburn, but the loss projections are right in line with national betting win totals.

No. 11 Appalachian State Mountaineers

There hasn't been much buzz this offseason with regard to Group of Five teams that might make noise, in part because the volume of noise those non-power programs can muster isn't likely to move the needle with the playoff committee. Appalachian State has solid possession efficiency inputs and will couple that with a very manageable schedule. They may not face a credible threat to derail an undefeated season until Halloween, so a 10-2 regular season (or better) projects to be more likely than not. They won't rack up any wins that would turn the committee's head to break into the final four, but they are best positioned according to FEI to grab a major bowl bid just the same.

It should be repeated that FEI isn't designed to be infallible. It very obviously has blind spots to transitional intangibles that can't easily be measured. And though it is optimized to predict the bulk of FBS games as accurately as possible, it should be fully expected to miss wildly on several teams. My gut says the No. 50 Texas projection is a prime candidate for a wild swing and miss, but it isn't prompting me to redesign the system around that outlier. I'm excited to see whether FEI is mostly right or dead wrong on this and the other 129 FBS teams. Either way, I'm encouraged to acknowledge FEI's blind spots while still celebrating its effective qualities. I hope you'll share the same perspective and follow along throughout the year to examine and challenge the weekly ratings along with me here at Football Outsiders.