Week 10 FEI Ratings

by Brian Fremeau

In last week’s Seventh Day Adventure [1] column, I opined that the 2009 college football season may be careening toward an unsatisfactory conclusion similar to 2004. With a lack of connectivity among the top contenders and few other challengers on the road ahead, the end of the regular season may present us with a murky mess of multiple undefeated teams, including more than two from BCS conferences. Iowa then promptly lost at home Saturday to an inferior Northwestern squad, bowing out of the BCS title chase and clearing up some of the murkiness.

There was something inevitable about the Hawkeyes’ loss. They had been living dangerously way too often this year, pulling out fourth quarter comebacks and narrow victories ever since an opening weekend escape from FCS Northern Iowa. Once quarterback Ricky Stanzi was sidelined early against the Wildcats, late-game heroics were out of the question. If ever there were a circumstance primed for an upset, this was it.

Iowa, a 14-point favorite over Northwestern heading into the game, was one of four double-digit favorites to fall on Saturday. Notre Dame (-12) fell victim to Navy, Louisiana-Lafayette (+13.5) edged Arkansas State, and Baylor (+14.5) dropped Missouri. The Iowa loss is the only one of the four with significant national implications, but does that make it the biggest upset? Better yet, are upsets best measured by Vegas lines set in advance of the game, or should they only be judged retroactively at the end of the season?

Vegas betting lines aren’t precisely designed to predict game outcomes. But since they are designed to balance wagers on either side of the line, they do represent the fair estimate of the betting public’s consensus prediction for the game. And they’re fairly accurate, too. Since 2003, teams favored to win by Vegas have a .746 overall winning percentage. Double-digit favorites over the same span have a .874 winning percentage.

Perceptions of the betting public in advance of a game are one thing, but should results and upset judgments be reevaluated once new data is collected? USC may have been a favorite at Oregon when the two teams faced off on Halloween, but the outcome of that game and others certainly can change our perspective on which was the lesser team in the match-up. We can use a rating system like FEI to judge upsets retroactively, but since new game data can shake up the ratings weekly, it is probably best reserved for end-of-year assessments.

If the season ended today, Iowa’s loss to Northwestern is the least likely outcome based on current FEI ratings. Two other games have a larger FEI rating delta between the teams involved in the upset -- No. 9 Clemson lost to No. 70 Maryland, and No. 32 Houston lost to No. 99 UTEP -- but both of those were played in the weaker team’s stadium. Iowa’s home loss stands alone, for now. If the Hawkeyes keep losing, or Northwestern finishes strong, it may not measure as a very significant upset after all.

I ran a comprehensive assessment over the last six seasons (2003-2008) of the number of times teams with a lower FEI end-of-year rating defeated an opponent with a higher one. Such "upsets" occurred in 21.8 percent of all games played in that span. In games in which the FEI rating delta exceeded 0.1 (roughly equivalent to a touchdown difference on a neutral field over an average number of possessions), upsets were recorded in 11.3 percent of such games.

As it turns out, Iowa was on the wrong end of more retroactive FEI upset losses than any other team in the country from 2003-2008. Not only that, but the Hawkeyes haven’t really countered those setbacks with many upset wins of their own.

Other observations: UCLA rates as the most schizophrenic team in the country during the last few years. More than a third of its games played between 2003 and 2008 involved an upset of some kind -- 13 went in favor of the Bruins and 14 went against. Also, Boston College and Virginia Tech have posted several significant upset losses, but they haven’t knocked off any teams rated significantly stronger than them.

Of course, this data may not provide evidence of upsets as much as it may reveal FEI biases. The Eagles and Hokies would not appear on this list at all if the system didn’t rate them so high in the first place. FEI certainly appears to like the ACC more than most, and that may be due in part to the relative weight given to certain upsets over others. Generally, close-game upsets aren’t heavily punished by the FEI formula, and ACC teams often play tighter games than other leagues. Combined with a good out-of-conference track record (the ACC was strong, though unheralded, by this measure last year), and we can start to understand where those conference trends in the ratings are coming from.

On the flip side, the team that has posted the most upset victories since 2003 is Toledo. They have 16 upset wins against only five upset losses. Most of the teams at the top of this list are of the BCS conference variety, though: UCLA (13), Texas A&M (12), Indiana (12), and yes, Northwestern (12) appear among the top 10.

As for Iowa, they haven’t fallen out of the Big Ten title picture officially yet, and their BCS ranking is still strong. But without Stanzi heading into a likely conference-deciding showdown with Ohio State, Vegas isn’t buying it. Ohio State is a 17-point favorite this weekend, equivalent to an 86 percent likelihood of victory according to Vegas line trends. Not very promising, but there’s always a chance.

Week 10 FEI Top 25

The principles of the Fremeau Efficiency Index (FEI) can be found here [3]. Like DVOA, FEI rewards playing well against good teams, win or lose, and punishes losing to poor teams more harshly than it rewards defeating poor teams. Unlike DVOA, it is drive-based, not play-by-play based, and it is specifically engineered to measure the college game.

FEI is the opponent-adjusted value of Game Efficiency (GE), a measurement of the success rate of a team scoring and preventing opponent scoring throughout the non-garbage-time possessions of a game. Like DVOA, it represents a team's efficiency value over average. Strength of Schedule (SOS) is calculated as the likelihood that an elite team would win every game on the given team's schedule to date. SOS listed here does not include future games scheduled.

Only games between FBS teams are considered in the FEI calculations. Preseason projections are not a factor in the current calculations. Current FEI ratings are a function of results of games played through Nov. 8.

FEI ratings for all 120 FBS teams are now listed in the stats page section of FootballOutsiders.com. Click here for current ratings [4]; the pull-down menu in the stats section directs you to 2007 and 2008 ratings.