Writers of Pro Football Prospectus 2008

07 Dec 2011

FEI: Last Stop Before Bowling

by Brian Fremeau

It appears that I struck a nerve with this week’s ESPN article “defending” the BCS. From my perspective, the headline was a bit more sensational than the content, and I honestly did not intend to trumpet the BCS system and methodology as the best way for college football to conclude its season. I was trumpeting the fact that the winner of the BCS championship consistently has a better claim to the title of “best team” than the winner in sports that host multi-round playoffs.

One (of many) ways the BCS fails us is that even though the best team wins, it rarely plays the second-best team in the championship. According to FEI, in each of the last five seasons, the national champion hasn’t faced the second-best team in the BCS title game (though 2009 champion Alabama did play Florida in the SEC title game to conclude the regular season). FEI isn’t an infallible authority on “best” or “second-best,” but as the ESPN article points out, many other computer systems come to the same conclusion. Compared to the murkiness of determining the second-, third-, fourth- and fifth-best teams in the nation, the post-BCS national championship is clear as the crystal football the title team hoists over its head.

Still, we are justifiably dissatisfied with college football’s postseason. Count me in that group, but let's not adopt a playoff simply because it isn’t the BCS. I’ve long been convinced that an equitable playoff model would include automatic qualifiers from every conference, plus enough at-large selections so as to properly acknowledge the vast gulf in power between the best and worst conferences. A 24-team playoff (11 autobids, 13 at-large) achieves that balance with a distribution very similar to that of the 64-team NCAA basketball tournament (roughly the same balance of autobids/at-large bids and 20 percent representation of the total pool of teams). A 24-team college football playoff might work in theory but it is logistically unsound. It also doesn’t address the more fundamental problem plaguing college football’s post-season: the lack of connectivity in its regular season.

I want the best teams to play one another on the field as much as anyone. But several, if not many, of those games could and should be scheduled in the regular season before we even get to developing a new playoff selection process. Several excellent suggestions to implement improvements in the regular season were made by FO readers the other day including:

Reader "Corrections": First, make playing a AA school an automatic disqualifier. Second, require playing at least 2 other non conference AQ teams that meet a certain threshold (say bowl-eligible in two of the last 3 years)or a non conference non AQ that is bowl-elgible in say 4 of its last 5.

Reader "Kal":Determine your schedule based on a rotation. Year 1, everyone in the Pac-12 plays everyone in the SEC in two games. If they were the #1 team or #2 team they play the previous year's top 2 teams. If you were #11 or #12 you play the previous year's 11 and 12 teams. In addition, the Pac-12 also plays one team in the Big 12. Year 2, the Pac-12 plays everyone in the Big-12 twice. They play the Big 10 once. Year 3, they play everyone in the Big 10 twice. They play everyone in the SEC once.

These are ambitious ideas that would be impossible to implement without consensus from all the major conferences. Could we convince every team in every conference to surrender two of its non-conference matchups per year to something like a coordinate annual interconference challenge? It would have to be compensated, but that’s exactly where the sponsors and TV partners can make a big difference.

There will be 35 bowl games played between December 17 and January 9. That’s about 15 bowls too many. If the standard for bowl eligibility was elevated to having seven wins versus FBS competition, only 44 teams would qualify this year. If games against FCS competition didn’t count toward qualifying for bowls, many teams would scramble to replace those matchups with weak FBS opponents. However, if better matchups were incentivized, we might have something. What if R&L Carriers, Beef ‘O’ Brady’s, Little Caesers, and the San Diego County Credit Union were to throw their money toward promoting the Big Ten/Big 12 and SEC/Pac-12 challenges rather than the bowl games they currently sponsor?

There were only seven non-conference regular season games played this year between teams that ended up in the final Associated Press top-25 poll. Two of those games involved LSU, wins over Pac-12 champion Oregon on a neutral field and over Big East champion West Virginia in Morgantown. Those games are a huge reason why we are so confident that LSU is the best in the nation. The absence of more of those kinds of games is why we aren’t as sure about the other contenders.

The other reason we can’t be so sure is because the best teams in the nation can’t be expected to play like the best teams in the nation week in and week out. The graphic above illustrates all of the GFEI (opponent-adjusted single game performance) data to date this season. The top teams have the best overall distributions of GFEI data, but for some, the game performances are wildly variable.

Oklahoma State registered two of the three best single-game GFEI performances of the season, against Baylor on October 29 and against Oklahoma last weekend. But the overtime loss to a middling Iowa State receives significant weight and neutralizes those two performances.

A few other outliers provide interesting insight into the components of FEI. Alabama had a very good year, and strangely enough, the Crimson Tide’s worst GFEI performance came against Kent State, a game the Tide would eventually win 48-7. The final score was a blowout, but the game wasn’t a particularly efficient effort for Alabama and it contained an unusually high 30 non-garbage possessions. As a reference, Kansas State efficiently whipped Kent State a few weeks later in only 14 non-garbage possessions. That result alone doesn’t drag down Alabama too far since it doesn’t receive too much relevance weight, but in comparison to the other top-10 teams, it is a noticeable outlier.

Among the other games of note identified in this graphic, Oregon’s best game came against Stanford, the fourth best single-game GFEI in the nation. But almost every other top-10 team in FEI had at least two performances better than the Ducks’ second-best GFEI against Nevada. USC and Michigan State both had rather weak outlier victories, both in home games against FEI No. 102 Minnesota. Wisconsin gets credit for having had the "best" worst game; that is, the Badgers never had a terrible day, dominating the weakest opponents they faced and falling twice at the very end of regulation against decent FEI opponents.

There have been 679 FBS vs. FBS college football games to date, meaning that there have been a total of 1,358 individual GFEI performances. The top teams turned in most of the top performances, of course. LSU had eight games this year that rank among the top-10 percent of games played this season according to GFEI. Oklahoma State had seven such games. Alabama had six. Everyone in the top 10 had at least three such games. Combined, the FEI top-10 teams turned in 37 percent of the best games played this year.

If given the opportunity, every one of these teams has proven it has the capacity to play like one of the best teams in the country. Each would be capable of winning a sequence of games in a playoff format to win a championship if one were created. Several other teams outside this list probably would be able to as well. As such a system is devised, let’s not lose the valuable regular season data we do have to generate a strong field in the first place. In fact, let’s make an effort to get more of that data going forward.

FEI Week 14 Top 25

The Fremeau Efficiency Index (FEI) rewards playing well against good teams, win or lose, and punishes losing to poor teams more harshly than it rewards defeating poor teams. FEI is drive-based, not play-by-play based, and it is specifically engineered to measure the college game.

FEI is the opponent-adjusted value of Game Efficiency (GE), a measurement of the success rate of a team scoring and preventing opponent scoring throughout the non-garbage-time possessions of a game. FEI represents a team's efficiency value over average. Strength of Schedule (SOS) is calculated as the likelihood that an "elite team" (two standard deviations above average) would win every game on the given team's schedule. "SOS Pvs" represents only games played to date. "SOS Fut" represents only remaining scheduled games including bowl games.

Mean Wins (FBS MW) represent the average total games a team with the given FEI rating should expect to win against its complete schedule of FBS opponents. Remaining Mean Wins (FBS RMW) represent the average expected team wins for games scheduled but not yet played.

Offensive FEI (OFEI), Defensive FEI (DFEI), Special Teams Efficiency (STE) are also provided, along with Field Position Advantage (FPA), the share of the value of total starting field position earned by each team against its opponents.

Only games between FBS teams are considered in the FEI calculations. These FEI ratings are a function of results of games played through December 3. The ratings for all FBS teams can be found here. You can also find OFEI, DFEI, and STE on their own pages.

Rk Team FBS
FEI Last
1 LSU 12-0 .318 1 .313 2 .143 22 .609 4 11.4 0.7 .454 8 -.719 2 3.207 3 .586 2
2 Oklahoma State 11-1 .297 2 .270 7 .239 46 .676 5 11.5 0.7 .354 20 -.748 1 2.344 9 .557 4
3 Wisconsin 10-2 .255 4 .335 1 .247 48 .684 6 11.1 0.6 .704 2 -.295 29 .629 46 .526 22
4 Alabama 10-1 .252 3 .315 3 .254 51 .395 1 9.9 0.3 .356 18 -.669 3 -.674 78 .521 31
5 Stanford 11-1 .220 6 .258 6 .353 79 .459 2 10.8 0.3 .443 11 -.492 10 .546 49 .500 61
6 Oregon 10-2 .217 5 .246 8 .147 24 .602 3 10.2 0.4 .380 14 -.542 7 1.182 30 .515 36
7 Michigan 10-2 .213 7 .213 10 .270 55 .813 19 10.6 0.7 .507 5 -.419 17 .124 59 .521 32
8 USC 10-2 .198 9 .158 14 .215 40 - - 9.6 - .493 7 -.299 26 1.403 26 .524 26
9 Michigan State 9-3 .193 11 .139 15 .091 3 .812 18 9.0 0.6 .174 40 -.528 8 2.084 14 .542 12
10 Boise State 11-1 .191 8 .311 5 .469 96 .885 36 11.3 0.7 .310 26 -.290 30 3.396 2 .610 1
11 Oklahoma 9-3 .180 10 .179 9 .090 2 .866 30 9.3 0.7 .372 16 -.434 15 .427 53 .554 6
12 Florida State 7-4 .173 14 .147 17 .368 81 .814 20 9.1 0.6 .046 52 -.454 13 3.645 1 .573 3
Rk Team FBS
FEI Last
13 Kansas State 9-2 .170 12 .081 35 .131 15 .801 15 8.3 0.6 .317 25 -.331 23 2.297 12 .540 13
14 South Carolina 9-2 .156 17 .104 26 .347 77 .806 16 8.9 0.5 .095 47 -.562 6 -2.949 112 .489 74
15 Clemson 9-3 .152 31 .088 38 .225 43 .806 17 8.9 0.5 .355 19 -.299 27 .871 39 .510 41
16 Arkansas 9-2 .150 16 .119 20 .093 6 .767 12 8.1 0.4 .400 13 -.093 53 1.437 23 .525 25
17 Nebraska 8-3 .147 18 .083 32 .157 26 .791 13 7.9 0.5 .242 30 -.200 43 2.173 13 .527 21
18 West Virginia 8-3 .147 23 .069 36 .152 25 .798 14 7.8 0.5 .495 6 -.183 45 -1.177 86 .479 89
19 Georgia 9-3 .144 15 .111 16 .167 28 .726 8 8.9 0.4 .230 32 -.437 14 -.874 80 .501 60
20 Virginia Tech 10-2 .143 13 .091 18 .351 78 .691 7 8.9 0.3 .277 28 -.269 32 -.053 66 .508 46
21 Notre Dame 8-4 .142 19 .125 19 .136 20 .762 11 8.1 0.4 .342 22 -.412 18 .761 42 .492 69
22 Cincinnati 8-3 .140 20 .108 23 .292 62 .942 43 8.2 0.7 .345 21 -.273 31 1.252 28 .539 14
23 Texas A&M 6-6 .136 22 .116 21 .134 17 .943 44 8.5 0.7 .371 17 -.263 33 .370 56 .505 55
24 Rutgers 7-4 .135 24 .052 42 .309 65 .966 55 8.2 0.8 .036 56 -.642 4 2.428 8 .523 27
25 Pittsburgh 5-6 .128 27 .036 56 .218 42 .985 64 7.7 0.9 .053 50 -.381 20 1.682 19 .514 38

Posted by: Brian Fremeau on 07 Dec 2011

25 comments, Last at 09 Dec 2011, 12:21am by Dennis


by zlionsfan :: Wed, 12/07/2011 - 2:26pm

I agree with you about 24 being the most likely number for an NCAA-run playoff system, but I think the existing I-AA playoff system provides a pretty good framework to use as a starting point: higher seeds host until round X, then games at a predetermined site. (I actually don't mind cutting out the bowls entirely, but realize that the money they represent won't go away quietly, so it might be necessary to include some of those locations as sites.)

Questions that aren't answered by I-AA teams might be covered by the NFL, because they use basically the same system. The main difference would be crowd size - how to manage more games at stadiums seating 90-100K instead of mostly 60-80K.

Sure, it's a lot of games to play for the teams that make the Final Four ... but then this isn't about the teams, is it? (Or more specifically, it's not about the players.) It's just one or two more games than I-AA teams play.

by KK Probs (not verified) :: Wed, 12/07/2011 - 2:32pm

Have you ever correlated this last set of regular season ratings with the pre-season ratings, for all teams? Or maybe did a chart correlating how each week of the season correlates with this week's ratings? That might be interesting.

It seems like it would make more sense to do it now than after the bowls because the 2-5 week layoffs before the bowl games might introduce variables that wouldn't be present in a 14-week stretch where teams typically play every week. (Although, just from a fun standpoint, a post-bowl game correlation would be nice to know, too.)

by Brian Fremeau :: Wed, 12/07/2011 - 2:42pm

Nice suggestion. Ran some quick numbers. Here are the week by week correlations of preseason FEI projected data for 2011 with each week's FEI ratings. Remember that preseason data was included in the weekly FEI for each of the first six weeks.

Week 1: .970
Week 2: .936
Week 3: .900
Week 4: .715
Week 5: .765
Week 6: .735
Week 7: .752
Week 8: .758
Week 9: .773
Week 10: .781
Week 11: .791
Week 12: .787
Week 13: .791
Week 14: .788

by HokeyPokey (not verified) :: Wed, 12/07/2011 - 2:40pm

I notice that OSU ended up #1 in DFEI at the end of the season, and I think that reflects what I have noticed throughout the year, that OSU has the best defense for the Big 12.

That doesn't mean they have the best defense period, or the best defense in the Big 12, but playing against and playing with the hurry up, spread offenses that flourish there, OSU has the best solution. You can see that OSU's best defensive performances have been against the most pass happy teams - OU, Tech, and Baylor. When you are facing a fast spread attack week in and week out (something that OOC oppoents never have to do) you have to find ways to keep your defense from fading down the stretch. OSU's defense doesn't try to stop every play, but it does turn the tables on occasion and make sure to capitalize on opposing mistakes. This philosophy doesn't work as well against the run, but the defense is built to mirror OSU's quick scoring offense that can take the lead early, but put the defense on the field more than it would like to be.

by Brian Fremeau :: Wed, 12/07/2011 - 2:46pm

Oklahoma State had the two best single-game DFEI performances of the year against Baylor and Oklahoma. I'm with you, that the Oklahoma State defense is very underrated due to pace, garbage time, and opponent strength. But I also think that if given the opportunity to play very good offenses, LSU and Alabama may have been even more impressive. We don't have much data, unfortunately, to have had them prove that on the field.

by HokeyPokey (not verified) :: Wed, 12/07/2011 - 3:10pm

I think that Alabama or LSU have a good chance of being even better in one game (say a bowl game or an OOC like Arkansas-A&M) but I wonder if they could maintain their defense over a season, in the SEC you have to deal with injuries from physical play week in and week out, in the Big 12, you are having to run and run and run and run. I would like to see how those big Bama linebackers make it through a Big 12 schedule.

by NYMike :: Wed, 12/07/2011 - 4:16pm

I've suspected (admittedly as a fan) that Wisconsin is one of the more under-appreciated teams this year. Your statistics seem to bear that out. How many calls did we hear from pundits asking for the Badgers in the NCG (or the SEC redux, as some of us atheists are calling it). Yet there they are, arguably, as good as Alabama. And don't give me "they had their chance." Sure they did, but so did Alabama. I'm not arguing for Wisconsin in the NCG; I'm arguing for anyone BUT 'Bama.

The claims that this year's big game, largely a product of SEC self-promotion (look at the coaches' poll) "clearly" represents the two best teams in the country are a stretch if we're kind, and just plain silly if we're honest. Of course, in 2006, when Michigan lost by three points on the road to number one Ohio the State University, there could be no rematch, because Michigan had their chance. This year, when Alabama lost at home to LSU, didn't win its conference, heck, didn't win its *division*, the rematch seemed to be a foregone conclusion. Nothing Oklahoma State could do, including destroying the number 10 team in the country, could derail to hype machine.

Ironically, the computers were willing to give us a fresh contest. It's the people that failed.

by Dennis :: Wed, 12/07/2011 - 10:25pm

The fundamental problem with the BCS is that the formula was continually altered to make the objective data less of a factor. It is now at the point where it is essentially a complete non-factor since the polls are 2/3 of the weight.

by Sancho (not verified) :: Wed, 12/07/2011 - 5:10pm

If one wants to be revolutionary, here it goes:

A 64-team league, divided into four Conferences (Pacific, Atlantic, Southern and Big) of 16 teams each.

Pacific would be build around the Pac-12. The Atlantic, the ACC; the Southern, SEC and Big-12; and Big, around Big-10 and Big East). Basically, it is North (Big), South (Southern), East (Atlantic) and West (Pacific).

-Regular Season-

Each Conference would be divided into two divisions of 8 teams. Pacific/Atlantic would have North and South division; Southern and Big, East and West.

Teams would play 3 free-to-choose games (non-Conference opponents); 7 against divisional opponents; and 2 against teams from the other division of its same conference (also free-to-choose).

The divisional winners would face each other in Conference Championship games.

-The BCS Bowls-

The Big champion and the Pacific champion would play each other at the Rose Bowl. The Atlantic champion and the Southern champion would play each other either in the Fiesta Bowl.

The winners of those bowls would play in the National Championship Game, that would rotate between the Sugar Bowl and the Orange Bowl.

The remaining bowl (Sugar or Orange) would see the best two non-conference-champion teams facing each other.

-Other bowls-

All other bowls could be decided in any criteria they believe its good.

-Promotion Bowls-

The worst-recorded team from each Conference (the 16th place) would face one of the four best non-league teams with its league credential on the line. The winners would play in the league next season; the losers would become a non-league school. How the four non-league school would be chosen is up to debate.

by Sancho (not verified) :: Wed, 12/07/2011 - 5:28pm

Post-season would be basically a 8-team playoff hidden under a form similar to the current structure.


There is an alien 'either' written above. The right sentence would read: "The Atlantic champion and the Southern champion would play each other in the Fiesta Bowl."

by Bill Connelly's innermost thoughts (not verified) :: Wed, 12/07/2011 - 5:11pm

After spending so much time the past week insisting that LSU and Alabama were clearly the two best teams when taking into account every play of the season -- even after Oklahoma State's destruction of Oklahoma -- Alabama comes up fourth in FEI.

by Dennis :: Wed, 12/07/2011 - 10:22pm

"I was trumpeting the fact that the winner of the BCS championship consistently has a better claim to the title of “best team” than the winner in sports that host multi-round playoffs."

But the purpose of the playoffs isn't to determine the "best team", it's to determine a champion. It should be obvious that these are not necessarily the same thing, and quite often are not. If you just want to anoint the best team, you wouldn't have playoffs or a championship game at all.

by Kal :: Wed, 12/07/2011 - 11:17pm

And for the vast majority of college, we didn't.

I guess that's kind of lost on people - that having the champion isn't synonymous with being the best team, and celebrating the best team is something good too.

That being said, right now we have the worst of both worlds.

by Dennis :: Thu, 12/08/2011 - 5:47pm

There's nothing wrong with celebrating the best team, but the vast majority of fans want a true champion. The current system fails miserably at that.

by sswoods (not verified) :: Thu, 12/08/2011 - 12:11am

Let's say, for discussion's sake, that in 2012 the best team in college football is ... UNLV. For the sake of discussion, UNLV is the greatest collection of talent, coaching, and chemistry to EVER grace the college football scene. Now, they don't have a great schedule, being in the MWC and all, plus their non-con isn't great either. But they are the best. They are unproven, having not played other contenders, but that doesn't change the fact that they are the best team. Under the current system - or the prior system - there is a 0% chance for them to be National Champions, or even play in the Championship Game. It sure seems unlikely that UNLV will be that good next year, or any non-AQ school would be, but we have no way of actually knowing. If settling the matter on the field isn't capable of determining the best team - I understand the arguments that claim that - then what on earth does? Computer ratings built around the opinions of the programmers' perspective of what makes a good team? Popular vote? If Navy played the exact same schedule as LSU this year, with the exact same results, Navy wouldn't be playing in the title game. We have absolutely no idea if the BCS has done a good job of determining the best team each year or most years or any year, we have only the eye test, a test that favors the exciting over the methodical, the sensational over the efficient, style over substance to compare final results.

by Kal :: Thu, 12/08/2011 - 4:13am

"Under the current system - or the prior system - there is a 0% chance for them to be National Champions, or even play in the Championship Game."

Actually that's kind of untrue. In the prior system the AP could vote them national champions. They did that with BYU, and it's not like BYU was in a major conference. They just looked good, had great talent and beat the hell out of everyone they played.

Last year boise State was two wins away (them beating Nevada and Auburn or Oregon losing one of their games) from going to the title game.

Let's not make it out that these teams don't have a chance at all.

If Navy beat the Pac-12 conference champion, the Big East conference champion, an 11-1 Alabama team, a 10-3 Georgia team, a 10-2 Arkansas team and did three of those eon the road? You bet your ass they'd be in the NCG game.

We have a lot of good reasons to suspect that the BCS does a fairly good if not perfect job of matching up #1 and #2. As Brian pointed out, last year Auburn (#1) played Oregon (#4) by FEI. The year before that it was a #1 vs. #3 I believe. You can get a lot of objective rating systems put together and get an amortized result, but one thing that strikes me as interesting is that the teams at the top of FEI/S&P often - not always, but often - are the same teams getting voted highly. This isn't like the NFL where there is so much parity that you can't make reasonable arguments about whether one team is better than another. This is where quite often there are only a handful of truly dominant teams in a season, and they tend to show it early and often.

by NYMike :: Thu, 12/08/2011 - 11:39am

I understand everything you're saying. But this year, we have 1 vs 4 in FEI, and 4 is 4 because they already lost to 1. The best thing that could happen for talk radio would be for Alabama to win 9-6 in OT. Also about the worst thing for fair-minded people.

by Adam H (not verified) :: Thu, 12/08/2011 - 11:45am

"let's not adopt a playoff simply because it isn’t the BCS"

Ah, but the BCS is in fact a playoff. The discussion is whether it should be expanded or not. In my humble opinion, a 4 team playoff would be better than a 2 team playoff, as historically the #3-4 teams tend to have a decent claim to the national championship. And if either were to beat two top-4 teams on top of their season accomplishments, they would surely be the undisputed champions.

by Brian Fremeau :: Thu, 12/08/2011 - 12:28pm

I agree with this. Which four teams should get in this year?

by crw78 :: Thu, 12/08/2011 - 5:36pm

Just take the top 4 conference champions as determined by BCS standings. If we want the regular season to mean something (and I do), then use the regular season to determine who the top 4 teams that won their conference are, regardless of whether it's an AQ conference or not, and play it off. As you said in your article, there's so little connectivity between the teams, it's impossible to say that Team A is better than Team B because it won this conference, which is a tougher conference. While LSU and Alabama certainly LOOK great and might be the two best teams, and the SEC is usually the best conference, the truth is most of the teams in that conference were not very good this year.

In a perfect world (in my mind), a committee would be formed to determine who the 4 best conference winners are, with the computer ratings used as a tool to help them make the decision (a la RPI in basketball). Forget about the polls, they are a complete conflict of interest (at least the coaches poll is).

This year, according to me, the 4 best conference champions would be LSU, Oklahoma State, Oregon, and Wisconsin.

by Brian Fremeau :: Thu, 12/08/2011 - 6:02pm

Not bad, though I imagine there are many different and equally good ideas for criteria by which to select four teams. I don't agree that in order for the regular season so mean something we need to select four conference champions. Not only are there differences in strength between conferences, there are vastly different schedule strengths within conferences. If Georgia, for instance, had defeated LSU in the SEC title game, I would have a hard time accepting that LSU was not still one of the best teams in the nation.

by Dennis :: Fri, 12/09/2011 - 12:21am

But the point is to crown a champion, not to pick the best team. Upsets happen and the best team doesn't always win. That's why they play the games.

by MattN (not verified) :: Thu, 12/08/2011 - 11:10pm

Best idea I've heard so far, crw78.

by MattN (not verified) :: Thu, 12/08/2011 - 11:33pm

only problem I see is that you might have the SEC, Big 10, Pac 12 and Big 12 champs as de facto AQs. Maybe make it the top 6 conference champs and give 1 and 2 a first round bye?

by MattN (not verified) :: Thu, 12/08/2011 - 11:36pm

or, to address Brian's concern, the top 5 conference champs with one vote-in slot.