Writers of Pro Football Prospectus 2008

Most Recent FO Features


» The Deep Ball Project

Guest columnist John Kinsley breaks down the tape of every deep pass in the NFL in 2017 and comes away with a shocking conclusion: even without Andrew Luck, the Colts had the best long-ball quarterback in the league.

08 Oct 2014

FEI Week 6: What Just Happened?

by Brian Fremeau

A big reason why I started working with college football data in the first place was to answer the question "What just happened?" I've told the story a few times here about how I began kicking around the building blocks of FEI while watching Notre Dame lose to Boston College in 2002, a game in which the Eagles' offense never advanced the ball across midfield. "What just happened?" that day was a boatload of turnovers and poor end-of-drive execution for the Fighting Irish. I wanted to quantify it, and that game kickstarted the data collection and analysis I continue to do today.

I run new FEI ratings each week and am rarely stunned by the results. Team X had a certain rating last week, they won or lost, and FEI responds to that new data. The interconnectedness of results is critical, but most weekends are fairly easy to decipher. Conference Y has a strong non-conference weekend and a group of teams gets a boost in the FEI ratings. My gut doesn't always agree, but it isn't often that I'm stumped about exactly "What just happened?" This was that kind of weekend.

The FEI game projections I publish on my site had their best results of the year in Week 6 against the spread (33-23-1). But it was the worst weekend to date for straight up picks (37-20) and particularly poor for high probability picks -- five of the 15 teams with an FEI win likelihood of at least 90 percent lost. Vegas was wrong about those games as well; BYU, UTSA, USC, Oregon, and Western Kentucky were favored to win by an average of 16 points each. They all lost, by an average of 8.2 points.

Those weren't the only upsets. Kentucky beat South Carolina (87 percent win likelihood according to FEI), Utah beat UCLA (77 percent), Purdue beat Illinois (72 percent), California beat Washington State (72 percent). Based on the FEI game projections, the likelihood that all nine of those upsets would occur last weekend was 0.0 percent. The likelihood that three or fewer of those nine games would result in an upset was 98 percent.

Obviously, the FEI game projections were way off the mark, which means the FEI ratings themselves were likely off the mark as well. But the answer to the question of "What just happened?" is awfully murky at this point in the year. The FEI ratings model makes sense of last weekend with circular logic. Arizona jumps 31 spots in the rankings because they beat Oregon on the road, and California's win, combined with their narrow loss to Arizona a few weeks ago, combined with their win over Northwestern in Week 1, combined with Northwestern's win over Wisconsin, launches the Bears 44 ranking spots to No. 9. UCLA lost to Utah, but they whipped Arizona State a few weeks back, so they get a boost by the Sun Devils' win over USC, plus Memphis' win over Cincinnati.

Preseason projection data still influences the FEI ratings at this point in the year, accounting for 10 to 15 percent of each team's rating. Though not set up specifically with this purpose, the FEI ratings model optimizes for mean wins. All but two of the FEI top 40 teams this week have an actual win total within 0.7 wins of their expected mean wins to date. The two exceptions are Arizona (+0.9 mean wins) and Washington State (-1.2 mean wins). FEI doesn't look right in comparison to the polls, or in comparison to last week's FEI ratings for that matter. But FEI does look right according to results on the field and expected results.

The top 40 teams in FEI had an average change in rank from last week to this week of 13 ranking spots. That is very unstable, but a number of big upsets will do that. I expect the ratings to really stabilize in a few weeks. The data visualization below illustrates the instability of the FEI ratings over the first half of last season before teams finally settled in around Week 9.

FEI 2013 Team Ratings By Week

Projected FEI ratings had a .788 correlation with end of year FEI ratings in 2013. Week 6 ratings had a .872 correlation with end of year FEI. Week 9 ratings had a .952 correlation. We'll never be able to predict every game outcome, but we're getting closer to the point in the season when the chaos begins to make some sense.

FEI 2014 Week 6 Ratings

The Fremeau Efficiency Index (FEI) is a college football rating system based on opponent-adjusted drive efficiency. Nearly 20,000 possessions are contested annually in FBS vs. FBS games. First-half clock-kills and end-of-game garbage drives and scores are filtered out. Game Efficiency (GE) is a function of the starting field position and outcome of non-garbage possessions. Opponent adjustments are calculated with special emphasis placed on quality performances against good teams, win or lose.

Strength of Schedule (SOS) is calculated as the likelihood that an "elite team" (two standard deviations above average) would win every game on the given team's schedule. SOS listed here includes all regular season games scheduled. A multifaceted approach to measuring schedule strength is available here.

Mean Wins (FBS MW) represent the average total games a team with the given FEI rating should expect to win against its complete schedule of FBS opponents. Remaining Mean Wins (FBS RMW) represent the average total games a team with the given FEI rating should expect to win against the remaining opponents on its schedule.

These ratings are partially influenced by preseason projections, a function of Program FEI ratings, previous-year FEI and garbage time data, previous-year turnover-neutral, special teams-neutral, and field position-neutral FEI, returning starters, recruiting success, and quarterback reliance. As the season progresses and actual 2014 data continues to be collected, the weight given to projection data will be reduced each week until Week 7, at which point it will be eliminated from the rankings entirely. Offensive and defensive FEI ratings will also debut in Week 7.

Rank Team FBS
1 Arizona 5-0 .273 33 .112 31 .045 8 9.3 5.2
2 UCLA 4-1 .257 9 .097 37 .051 10 8.9 4.9
3 TCU 3-0 .239 20 .256 7 .329 76 9.5 7.0
4 Oklahoma 4-1 .239 7 .263 6 .287 62 10.4 6.3
5 Oregon 3-1 .238 1 .153 18 .042 6 7.5 4.6
6 Mississippi 5-0 .237 11 .306 3 .170 35 8.7 4.3
7 Auburn 5-0 .230 3 .286 4 .080 12 8.0 3.6
8 Utah 3-1 .226 24 .146 21 .051 9 7.3 4.4
9 California 3-1 .225 53 .016 60 .044 7 7.1 4.7
10 Mississippi State 5-0 .204 12 .269 5 .152 27 8.2 3.9
11 Georgia Tech 4-0 .203 32 .123 27 .278 59 8.9 5.6
12 USC 3-2 .200 8 .123 28 .023 1 7.0 3.4
Rank Team FBS
13 Alabama 4-1 .196 4 .225 12 .139 23 7.7 3.8
14 Florida State 4-0 .196 2 .148 19 .297 64 8.6 5.2
15 Notre Dame 5-0 .185 15 .238 11 .148 26 8.7 4.4
16 Ohio State 4-1 .184 17 .246 9 .281 61 9.5 5.4
17 Virginia Tech 3-2 .183 28 .077 44 .272 56 8.3 4.8
18 Stanford 2-2 .179 6 .094 38 .029 3 5.7 3.1
19 Baylor 4-0 .167 16 .388 2 .258 52 8.3 4.7
20 Missouri 3-1 .166 10 .128 24 .263 54 8.0 4.9
21 Texas A&M 4-1 .165 5 .145 22 .084 14 6.8 3.3
22 Georgia 4-1 .164 14 .243 10 .166 33 7.3 3.3
23 Washington State 1-4 .161 36 -.045 79 .036 4 5.1 2.9
24 Arizona State 3-1 .158 61 .023 57 .051 11 5.7 3.3
25 Louisville 4-1 .154 29 .146 20 .222 47 7.6 3.6

Posted by: Brian Fremeau on 08 Oct 2014

6 comments, Last at 08 Oct 2014, 6:44pm by Brian Fremeau


by Scott C :: Wed, 10/08/2014 - 2:34pm

I'm a Bears fan, but calling that game over Washington State a "win" is tough. Having the opposing kicker miss a game winning 19 yard FG isn't repeatable. Their offense is very good, their defense is atrocious. We saw this same game against Arizona with a slightly different last-second outcome.

Does FEI weigh the win that much, or was the actual drive efficiency in that game good? Both offenses were nearly unstoppable, but I'd expect huge negative value on the defensive side to counteract that.

At this point I expect every Cal game to end up a close high scoring game. I can see that they are dangerous and can beat good teams with some luck, but even the optimist in me can't fathom #9.

by Brian Fremeau :: Wed, 10/08/2014 - 2:58pm

Everyone in the Pac-12 is getting boosted by everyone else in the Pac-12. Cal isn't particularly efficient overall (60th in raw game efficiency) and they've tallied a lower percentage of available yards than their opponent in three FBS games so far: http://www.bcftoys.com/2014-game-splits#california

I can't fathom Cal remaining at No.9 either, but I can fathom the internal circular logic of why Cal is there now (played No. 1 close, wins over Nos. 23, 29, and 61).

by Lance :: Wed, 10/08/2014 - 3:37pm

Shoot. And Oklahoma State gets booted from the top 25. OU gets beat, but moves up from 7 to 4, perhaps because the team that beat them jumped from 20 to 3? I'm lost.

by Brian Fremeau :: Wed, 10/08/2014 - 6:44pm

Didn't help Oklahoma State that UTSA and Texas Tech had lousy weekends as well.

by dude6935 :: Wed, 10/08/2014 - 5:41pm

It seems your FEI is kind of binary. Either you score from midfield or you don't. Whereas the S&P+ assigns success on a play by play basis. So LSU can totally ball out on first down, then go 0 for 13 on 3rd down and still have a respectable S&P+ ranking - but its FEI ranking is dismal.

by Brian Fremeau :: Wed, 10/08/2014 - 6:43pm

Success is certainly measured differently in the two approaches, though most teams don't have extreme differences between their play success rate and drive success rate.