Writers of Pro Football Prospectus 2008

01 Nov 2017

FEI Week 9 Ratings

by Brian Fremeau

The weekly release of College Football Playoff rankings have often served a reliable purpose for those of us who work with data to evaluate teams, even if their meaningfulness in determining which teams will contend for the national championship at season’s end is dubious. We get a glimpse at the factors that matter most to the selection committee, and even if our own algorithms have different priorities, I find it’s helpful to better understand what priorities others are bringing to the table.

Tuesday night’s release of the first CFP top 25 rankings for this year wasn’t earth-shattering, and some of the same principles that inform the FEI ratings were clearly influential to the committee as well. Schedule strength matters, and it matters more than the value of a digit in a given team’s loss column. Undefeated Wisconsin and Miami were ranked 9th and 10th respectively by the committee this week, and six teams already saddled with a loss were ranked ahead of the Badgers and Hurricanes.

This week’s FEI ratings also have the same six one-loss teams -- Notre Dame, Clemson, Oklahoma, Ohio State, Penn State, TCU -- ranked ahead of Wisconsin and Miami, albeit in a different order. And for good measure, a pair of two-loss teams that have played well and have faced good schedules to date – Iowa State and Auburn – are ranked ahead of the Badgers and Hurricanes in my opponent-adjusted drive efficiency ratings as well. Largely, FEI and the committee rankings are in solid agreement at the top, with one glaring exception.

It’s Alabama. This is at least the third week I’ve called out why Alabama isn’t getting love from the FEI ratings, but their precipitous drop to No. 13 overall this week requires special attention. I’ll save everyone a tweet and just ask myself: “What in the world is this computer system thinking, and what am I going to do about it?”

We’re in uncharted territory with the Crimson Tide. We’ve had teams in Power 5 conferences play very well but not get much love in the opponent-adjusted ratings due to a relatively weak schedule, but I don’t recall a year in which my system has valued a Power 5 schedule so poorly this deep in the year. Alabama has played the 120th-toughest schedule to date. An elite team would have been expected to lose only 0.18 times on average against their schedule to date. The only teams with a worse schedule strength rating to date are Louisiana Lafayette, Idaho, Arkansas State, Coastal Carolina, Southern Mississippi, Texas State, Massachusetts, Western Kentucky, UAB, and UTSA. Woof.

Alabama's SOS-to-date ranking dropped significantly this past weekend, when nearly everyone they had previously played appeared to take a synchronized nosedive. Florida State lost 35-3 to Boston College, Fresno State lost by 10 to UNLV, Colorado State lost by 17 to Air Force, Vanderbilt lost to South Carolina, Texas A&M lost by 21 to Mississippi State, and Tennessee lost to Kentucky. The only team Alabama has already defeated that won last weekend, Arkansas, knocked off another team Alabama has played already, Ole Miss. Alabama had a bye, everyone they’d played to date tanked, and Alabama’s opponent-adjusted ratings plummeted. At this point in the year, Florida State is hanging on to the No. 49 ranking in FEI, and that game remains the only opponent Alabama has played that is ranked in the FEI top 50. There isn’t a single other team in the FEI top 50 that hasn’t played at least one top-40 opponent.

That this phenomenon is happening to Alabama, the most dominant program by far over the last decade, is what makes this so unusual. The Crimson Tide rank No. 1 in raw game efficiency, No. 1 in net points per drive, and they're one of only four teams with top-10 rankings in opponent-adjusted offense and defense. They've trailed in a game on only 1.3 percent of non-garbage drives to date, and they've let by at least 17 points on 44.5 percent of non-garbage drives to date. They've been crushing opponents. Isn’t there something wrong with a system that doesn’t recognize Alabama’s dominance for what it is?

To be honest, I’m curious about that myself. On the one hand, I can make an argument that if we took Alabama’s name off of its stats and slapped Central Florida’s name on them against the weak slate that the Crimson Tide have faced, we wouldn’t even blink about the No. 13 overall ranking. And that’s exactly what the algorithm is doing.

But on the other hand, Alabama has looked just as dominant this year, if not moreso, as it has for most of the last decade. And those previous Alabama teams were both truly dominant against weak opponents, and consistently very strong against good opponents. This team simply hasn’t had the opportunity to face good opponents yet. But it will soon. Starting this week, Alabama will play three of its final four games of the year against LSU (No. 39 in this week’s FEI), at Mississippi State (No. 16), and at Auburn (No. 12). If Alabama dominates each of those opponents as it has against its opponents to date, there will be no question the Tide are worthy of the lofty playoff position they will inevitably claim.

But if FEI is right, and that's a big if, Alabama won’t be as dominant against these stronger teams and perhaps they will be tripped up once or twice down the stretch. I’m not counting on it, but I’m not ruling it out either. I’m just curious to find out what will happen with a team and program that has been so unusually untested this deep into the year.

FEI 2017 Week 9 Ratings

The Fremeau Efficiency Index (FEI) is a college football rating system based on opponent-adjusted drive efficiency. Approximately 20,000 possessions are contested anually in FBS vs. FBS games. First-half clock-kills and end-of-game garbage drives and scores are filtered out. Game efficiency (GE) is a measure of net success on non-garbage possessions, and FEI opponent adjustments are calculated with special emphasis placed on quality performances against good teams, win or lose. SOS ratings represent the average number of losses an elite team (two standard deviations better than average) would have against the team's entire schedule.

Offensive FEI (OFEI) is value generated per drive adjusted for starting field position and opponent defenses faced. Defensive FEI (DFEI) is value generated per opponent drive adjusted for starting field position and opponent offenses faced. Special Teams Efficiency (STE) is the average value generated per possession by a team's non-offensive and non-defensive units.

Ratings for all 130 teams can be found here.

1 Georgia 7-0 .320 .358 3 1.57 27 3.58 5 .38 1 .04 38
2 Clemson 7-1 .293 .249 10 1.34 41 2.87 21 .84 4 -.03 82
3 Notre Dame 7-1 .289 .258 8 1.83 16 3.62 4 1.03 7 .00 58
4 Ohio State 7-1 .267 .331 5 1.33 42 3.79 2 1.02 6 .01 55
5 TCU 6-1 .252 .222 12 1.84 14 2.64 32 1.35 13 .22 1
6 Oklahoma 7-1 .247 .194 16 2.07 5 4.13 1 2.23 64 -.09 118
7 Penn State 7-1 .246 .281 6 1.27 47 3.24 9 1.20 11 .18 3
8 Iowa State 5-2 .231 .131 19 1.71 20 2.57 35 1.46 18 .06 28
9 Central Florida 6-0 .227 .358 2 .46 106 3.54 6 1.42 17 .10 14
10 Washington 6-1 .222 .347 4 .58 95 3.13 11 .96 5 .07 25
11 Oklahoma State 7-1 .219 .247 11 1.69 23 3.70 3 1.53 20 -.14 127
12 Auburn 5-2 .216 .251 9 1.92 11 2.74 27 .74 2 .04 39
13 Alabama 8-0 .210 .406 1 .98 66 3.33 8 .78 3 .10 16
14 Wisconsin 8-0 .185 .264 7 .57 96 2.69 30 1.05 8 .04 35
15 Miami 6-0 .171 .106 25 .97 68 2.66 31 1.60 23 -.07 110
16 Mississippi State 5-2 .166 .202 15 1.64 25 3.02 15 1.41 16 .04 42
17 Virginia Tech 6-1 .152 .221 13 1.31 43 2.34 49 1.06 9 .12 9
18 USC 7-2 .142 .076 34 1.46 34 3.07 12 2.07 53 -.03 81
19 Iowa 5-3 .136 .077 32 1.95 10 2.33 51 1.69 27 .05 32
20 Texas 4-4 .126 .062 41 2.22 1 1.76 104 1.12 10 -.01 67
21 Arizona 5-2 .119 .111 23 .76 81 3.47 7 2.61 94 -.04 96
22 Stanford 6-2 .118 .143 18 1.46 33 2.82 24 2.27 65 .10 12
23 North Carolina State 5-2 .117 .062 42 1.79 17 2.83 23 2.08 55 -.06 102
24 Michigan 6-2 .111 .096 27 1.70 22 2.05 78 1.38 15 .06 27
25 Washington State 6-2 .109 .095 28 1.13 56 2.09 71 1.22 12 -.06 105
26 Navy 5-2 .107 .064 40 1.49 32 2.69 29 1.96 46 .00 61
27 Houston 5-3 .104 .052 46 .65 88 1.98 84 1.55 21 .00 65
28 Boston College 5-4 .101 .008 60 1.92 12 2.33 52 1.61 24 .06 31
29 Georgia Tech 3-3 .100 .069 37 1.89 13 2.92 19 1.94 44 -.07 106
30 Michigan State 6-2 .097 .043 52 1.96 9 2.08 74 1.46 19 -.07 112

Posted by: Brian Fremeau on 01 Nov 2017

5 comments, Last at 03 Nov 2017, 3:59pm by Happy Fun Paul


by techvet :: Wed, 11/01/2017 - 6:21pm

Twq questions:

1. Question about garbage time: When Notre Dame had a 7+ minute drive at the end of the NC State game and ended the game inside the Wolfpack 10-yard line, does it get credit for that time-consuming drive or is that considered garbage time because they already had a 3-TD lead?

2. When Notre Dame had a punt blocked for a touchdown, did that help NC State's STE as much as it hurt Notre Dame's?

by Aaron Brooks Go... :: Thu, 11/02/2017 - 11:28am

Better example than UCF -- TCU or Boise State circa 2010.

Alabama's results this year are almost identical to those two about a decade ago. We were fine burying them around 10th and making them play each other every year in the Fiesta Bowl.

If Alabama wants to schedule the Little Sisters of Mercy, they should go bowling accordingly.

by NJBammer :: Thu, 11/02/2017 - 1:11pm

Bama opened with FSU, which was ranked #3 at the time. They turned out to be not deserving of such a ranking but it's not like they opened with a little sisters of mercy as you suggest.

Boise State in 2010 opened with #13 VA Tech. This was the best team they played all year and they barely won 33-30, foreshadowing their future loss to #19 Nevada. Bama Gobsmacked every team they played this season, it silly to compare the two teams.

by Aaron Brooks Go... :: Fri, 11/03/2017 - 8:21am

Is it?

Bama hasn't played anyone as good as Kaepernick's Nevada team.

by Happy Fun Paul :: Fri, 11/03/2017 - 3:59pm

"Bama opened with FSU, which was ranked #3 at the time. They turned out to be not deserving of such a ranking but it's not like they opened with a little sisters of mercy as you suggest."

Agreed, except for the "turned out to be not deserving" part.

One of the problems facing any objective football ratings system is that, with so few data points, the system has to assume that each team has a "true" ability level and that any game-to-game variations from that are just random. And usually, that's a good enough assumption.

Sometimes, however, a team can change drastically for the worse due to injuries. Florida State this year is one such example. Lost for the year is, most obviously, QB Deondre Francois with a knee injury (suffered with about 6 minutes left in the Alabama game), but also two offensive linemen, Jauan Williams (shoulder) and Baveon Johnson (knee). Three more OLs also missed time, Derrick Kelly (leg), Landon Dickerson (ankle), and Cole Minshew. (Source: https://www.tomahawknation.com/)

Put another way, the FSU team that lost to Boston College 35-3 bore little resemblance, especially on offense, to the FSU team that Alabama battled -- quite successfully-- for 54 minutes. Alabama's opponent Texas A&M also made a downhill plunge after facing Bama, albeit less drastically. Other teams on Bama's schedule, especially SEC teams, just happened to be having down years, but no one expected that at the start of the season.

Brian's right that the situation will sort itself out as Alabama faces better opponents. A similar thing happened some years back in the NFL when the early-to-mid-season Manning-led Colts had a strength-of-schedule so low that it broke Aaron's system. You don't want to make a computerized ranking system that tries to adjust for extreme cases that make bad precedent, especially if it means chasing week-to-week changes. You make a system that (1) assumes that teams don't drastically change in quality over the sources of the season and (2) maximizes predictive strength later in the season, when there's more data... knowing that there will be an occasional weird glitch in ratings mid-season.

However, it's incorrect to say (as Brian implies) that Alabama has been so untested that we can't say anything about their quality (Alabama won solidly over FSU and T A&M, who were pretty impressive opponents when Bama played them, even if the system doesn't know that).

It's even more incorrect to assume that Alabama intentionally scheduled weak opponents. The past few years, Bama has started the season against about as highly-ranked opponent as they could find (Wisconsin [#20], USC [#20, but #3 at the end-of-year AP rankings], FSU [#3]). The rest of their non-league schedule has been less impressive, admittedly, but until this year, their league schedule made up for that.