FEI Week 10: Blind Side

by Brian Fremeau
If you haven’t seen it yet, there is an interesting online exercise offered by Tom Fornelli of CBS Sports, a blind test for fans to remove their own bias and select two teams to play for a national championship game based on resume alone. Spoiler alert: based on results to date, the overwhelming consensus is that the profiles of Kansas State and Notre Dame trump Alabama and Oregon. That is to say, the overwhelming consensus is that the specific factors selected for the blind test favor Kansas State and Notre Dame over Alabama and Oregon.
The factors Fornelli uses aren’t completely unsound. Wins over top-20 and top-40 opponents, opponent winning percentage, average margin of victory, and average opponent rank are all measures that are perfectly reasonable to use as part of a fair evaluation of a team. But let’s not pretend this exercise means much.
The use of opponent records and margin of victory are problematic without the context of tempo, garbage time, and individual game results. I challenge the use of average opponent rating instead of evaluating performances against opponents from the top down. And three of the five measures selected by Fornelli use the overall ratings published by Jeff Sagarin (top-20 wins, top-40 wins, average opponent rank). Sagarin’s ratings are well-respected and sound, but why would selected components of those ratings trump the overall ratings themselves? If Sagarin’s system is so useful in helping fans determine who has the best profile, why wouldn’t we trust Sagarin’s system to distinguish between these teams? Sagarin ranks Alabama No. 1, Kansas State No. 2, Notre Dame No. 3, and Oregon No. 4, and (surprise!), does not trump up top-20 and top-40 victories as final determining factors.
If I offered up a similar exercise and asked you to select teams based on selected portions of the FEI ratings, I would imagine that you would challenge the method by which those categories were chosen. Wins over top-5 FEI teams? Kansas State and Notre Dame have one apiece. Wins over top-10 FEI teams? Kansas State, Notre Dame, Florida, LSU, Georgia, Washington, and North Carolina State all have one apiece. Are wins over selected categories a better measure than the overall ratings themselves, or should Oregon and Alabama rank behind all of the teams I just named?
The truth, of course, is that the Sagarin ratings themselves (and FEI, S&P+, F/+, etc.) are far more sophisticated evaluations of team profiles than the Fornelli exercise because they consider many factors rather than just a hand-picked few. It may help fans and voters to remove as much noise as possible and focus on a few select statistical categories as they evaluate teams, and that’s okay. But this exercise doesn’t allow for the user to seek and weight factors that are important to them individually. And it strips away the best asset of computer systems: their ability to actually make some sense out of the noise rather than reduce it.
Computer ranking systems don’t have a monopoly on the truth, but we need to better understand them and use them properly as tools to evaluate what has happened and what may happen going forward.
Week 10 Revisionist Box Scores
This weekly feature identifies the games played each week that were most impacted by turnovers, special teams, field position, or some combination of the three. The neutralized margin of victory is a function of the point values earned and surrendered based on field position and expected scoring rates.
Week 10 Games In Which Total Turnover Value Exceeded Non-Garbage Final Score Margin | |||||||||||||||||
Date | Winning Team | Non-Garbage Final Score |
Losing Team | TTV + |
TTV - |
TTV Net |
TO Neutral Score Margin |
||||||||||
11/1 | Middle Tennessee | 34-27 | Western Kentucky | 7.3 | 0.0 | 7.3 | -0.3 | ||||||||||
11/3 | Army | 34-21 | Air Force | 15.4 | 0.0 | 15.4 | -2.4 | ||||||||||
11/3 | Florida | 14-7 | Missouri | 16.5 | 3.7 | 12.8 | -5.8 | ||||||||||
11/3 | Florida International | 28-20 | South Alabama | 13.3 | 3.3 | 10.0 | -2.0 | ||||||||||
11/3 | Kansas State | 44-30 | Oklahoma State | 20.0 | 0.0 | 20.0 | -6.0 | ||||||||||
11/3 | Rice | 49-47 | Tulane | 22.5 | 13.7 | 8.8 | -6.8 | ||||||||||
11/3 | South Florida | 13-6 | Connecticut | 12.0 | 3.6 | 8.4 | -1.4 |
Week 10 Games In Which Special Teams Value Exceeded Non-Garbage Final Score Margin | |||||||||||||||||
Date | Winning Team | Non-Garbage Final Score |
Losing Team | STV + |
STV Neutral Score Margin |
||||||||||||
11/1 | Middle Tennessee | 34-27 | Western Kentucky | 10.4 | -3.4 | ||||||||||||
11/3 | Rice | 49-47 | Tulane | 3.6 | -1.6 | ||||||||||||
11/3 | San Diego State | 21-19 | Boise State | 10.8 | -8.8 |
Week 10 Games In Which Field Position Value Exceeded Non-Garbage Final Score Margin | |||||||||||||||||
Date | Winning Team | Non-Garbage Final Score |
Losing Team | FPV + |
FPV - |
FPV Net |
FPV Neutral Score Margin |
||||||||||
11/1 | Middle Tennessee | 34-27 | Western Kentucky | 22.7 | 15.6 | 7.1 | -0.1 | ||||||||||
11/3 | Rice | 49-47 | Tulane | 39.6 | 28.2 | 11.4 | -9.4 | ||||||||||
11/3 | San Diego State | 21-19 | Boise State | 24.8 | 13.1 | 11.7 | -9.7 |
2012 totals to date:
- Net Total Turnover Value was the difference in 80 of 511 FBS games (15.7 percent)
- Net Special Teams Value was the difference in 39 of 511 FBS games (7.6 percent)
- Net Field Position Value was the difference in 48 of 511 FBS games (9.4 percent)
- Turnovers, Special Teams and/or Field Position was the difference in 111 of 511 FBS games (21.7 percent)
2012 Game Splits for all teams, including the offensive, defensive, special teams, field position, and turnover values recorded in each FBS game are provided here.
FEI Week 10 Top 25
[ad placeholder 3]
The Fremeau Efficiency Index (FEI) rewards playing well against good teams, win or lose, and punishes losing to poor teams more harshly than it rewards defeating poor teams. FEI is drive-based and it is specifically engineered to measure the college game. FEI is the opponent-adjusted value of Game Efficiency (GE), a measurement of the success rate of a team scoring and preventing opponent scoring throughout the non-garbage-time possessions of a game. FEI represents a team's efficiency value over average.
Other definitions:
- SOS Pvs: Strength of schedule to date, based on the likelihood of an elite team going undefeated against the given team's schedule to date.
- SOS Fut: Strength of schedule, based on the likelihood of an elite team going undefeated against the given team's remaining schedule.
- FBS MW: Mean Wins, the average number of games a team with the given FEI rating would be expected to win against its entire schedule.
- FBS RMW: Remaining Mean Wins, the average number of games a team with the given FEI rating would be expected to win against its remaining schedule.
- OFEI: Offensive FEI, the opponent-adjusted efficiency of the given team's offense.
- DFEI: Defensive FEI, the opponent-adjusted efficiency of the given team's defense.
- STE: Special Teams Efficiency, the scoring value earned by field goal, punt and kickoff units measured in points per average game.
- FPA: Field Position Advantage, the share of the value of total starting field position earned by each team against its opponents.
These FEI ratings are a function of results of games played through November 3rd. The ratings for all FBS teams, including FEI splits for Offense, Defense, and Special Teams can be found here. Program FEI (five-year weighted) ratings and other supplemental drive-based data can be found here.
Rk | Team | FBS Rec |
FEI | LW | GE | GE Rk |
SOS Pvs |
Rk | SOS Fut |
Rk | FBS MW |
FBS RMW |
OFEI | Rk | DFEI | Rk | STE | Rk | FPA | Rk |
1 | Kansas State | 8-0 | .318 | 1 | .317 | 3 | .268 | 33 | .646 | 34 | 9.9 | 2.7 | .428 | 13 | -.648 | 6 | 3.273 | 3 | .582 | 1 |
2 | Alabama | 9-0 | .295 | 3 | .358 | 2 | .419 | 63 | .757 | 53 | 10.1 | 1.8 | .359 | 18 | -.676 | 4 | 2.825 | 7 | .566 | 6 |
3 | Notre Dame | 9-0 | .289 | 2 | .153 | 19 | .211 | 23 | .707 | 40 | 10.5 | 2.7 | .600 | 3 | -.725 | 1 | -1.013 | 92 | .485 | 78 |
4 | Oklahoma | 5-2 | .276 | 5 | .245 | 6 | .143 | 9 | .673 | 37 | 8.9 | 3.6 | .577 | 4 | -.482 | 17 | 1.388 | 29 | .507 | 54 |
5 | Oregon | 8-0 | .272 | 4 | .393 | 1 | .539 | 85 | .467 | 13 | 9.7 | 2.3 | .418 | 14 | -.546 | 11 | 1.493 | 27 | .542 | 16 |
6 | Florida | 8-1 | .240 | 6 | .154 | 18 | .272 | 35 | .593 | 27 | 8.9 | 1.5 | .067 | 52 | -.699 | 3 | 3.288 | 2 | .548 | 11 |
7 | Oregon State | 7-1 | .236 | 7 | .101 | 34 | .323 | 45 | .457 | 12 | 8.6 | 2.1 | .438 | 12 | -.559 | 9 | .477 | 46 | .504 | 58 |
8 | Florida State | 6-1 | .215 | 8 | .280 | 4 | .648 | 102 | .645 | 33 | 8.4 | 2.2 | .100 | 44 | -.654 | 5 | 1.959 | 20 | .563 | 7 |
9 | Texas A&M | 6-2 | .210 | 9 | .196 | 9 | .449 | 66 | .376 | 10 | 7.7 | 1.2 | .551 | 6 | -.238 | 34 | -.484 | 75 | .509 | 52 |
10 | Ohio State | 10-0 | .207 | 10 | .170 | 14 | .354 | 51 | .709 | 42 | 9.8 | 1.4 | .502 | 10 | -.462 | 18 | -1.140 | 95 | .503 | 62 |
11 | LSU | 6-2 | .200 | 11 | .137 | 23 | .147 | 10 | .912 | 85 | 8.3 | 2.7 | .117 | 42 | -.568 | 8 | 1.050 | 34 | .554 | 9 |
12 | Nebraska | 6-2 | .184 | 21 | .052 | 46 | .192 | 19 | .756 | 52 | 7.8 | 2.4 | .630 | 2 | -.306 | 29 | -2.134 | 106 | .439 | 121 |
Rk | Team | FBS Rec |
FEI | LW | GE | GE Rk |
SOS Pvs |
Rk | SOS Fut |
Rk | FBS MW |
FBS RMW |
OFEI | Rk | DFEI | Rk | STE | Rk | FPA | Rk |
13 | Cincinnati | 4-2 | .164 | 15 | .122 | 29 | .559 | 91 | .866 | 76 | 8.0 | 3.4 | .321 | 21 | -.558 | 10 | .656 | 44 | .533 | 26 |
14 | UCLA | 7-2 | .158 | 35 | .125 | 27 | .418 | 62 | .741 | 50 | 9.0 | 2.2 | .256 | 29 | -.520 | 15 | -.801 | 85 | .549 | 10 |
15 | South Carolina | 7-2 | .158 | 19 | .162 | 15 | .264 | 29 | .717 | 45 | 8.0 | 1.4 | .019 | 59 | -.540 | 12 | -.847 | 86 | .503 | 63 |
16 | Georgia | 8-1 | .149 | 27 | .193 | 11 | .384 | 54 | .950 | 102 | 8.7 | 1.7 | .291 | 25 | -.309 | 28 | .107 | 58 | .526 | 34 |
17 | BYU | 4-4 | .148 | 20 | .092 | 36 | .141 | 8 | .850 | 73 | 7.4 | 2.6 | .095 | 46 | -.540 | 13 | -1.184 | 97 | .498 | 65 |
18 | Michigan State | 5-5 | .147 | 17 | .041 | 49 | .161 | 13 | .867 | 77 | 7.6 | 1.4 | -.019 | 62 | -.717 | 2 | -.337 | 71 | .492 | 73 |
19 | USC | 6-3 | .147 | 13 | .144 | 21 | .275 | 37 | .387 | 11 | 7.6 | 1.4 | .317 | 23 | -.119 | 50 | 1.979 | 19 | .526 | 35 |
20 | Clemson | 7-1 | .146 | 30 | .174 | 13 | .522 | 81 | .815 | 61 | 8.6 | 2.3 | .447 | 11 | .000 | 61 | 2.784 | 8 | .535 | 23 |
21 | Stanford | 7-2 | .145 | 12 | .150 | 20 | .273 | 36 | .239 | 3 | 7.5 | 0.9 | -.057 | 70 | -.578 | 7 | 1.656 | 23 | .572 | 4 |
22 | Oklahoma State | 4-3 | .143 | 16 | .073 | 42 | .195 | 20 | .366 | 9 | 6.9 | 2.4 | .545 | 7 | -.126 | 47 | -.006 | 64 | .448 | 114 |
23 | Boise State | 7-2 | .143 | 18 | .186 | 12 | .534 | 84 | .905 | 82 | 10.0 | 2.7 | .118 | 41 | -.524 | 14 | -.604 | 78 | .545 | 12 |
24 | Louisville | 8-0 | .139 | 22 | .136 | 24 | .644 | 101 | .677 | 38 | 8.6 | 2.1 | .515 | 9 | -.349 | 23 | -2.473 | 114 | .454 | 106 |
25 | Texas | 7-2 | .137 | 32 | .091 | 37 | .277 | 38 | .269 | 4 | 7.8 | 1.5 | .250 | 30 | -.036 | 57 | 3.183 | 6 | .570 | 5 |
1 Re: FEI Week 10: Blind Side
Michigan has had a pretty unusual ride in the FEI this year. Last week was a good example. With Michigan significantly better than Minnesota in all FEI measures, the game was listed as a toss up.
Do you think the 2 games against Alabama and ND are one of the reasons FEI does not seem to like Michigan this year?
2 Re: FEI Week 10: Blind Side
Really remarkable things being done by K-State with a roster with nowhere close to as many top recruits as Alabama and Oregon have. Factoring in the talent he has to work with, Bill Snyder has got to be one of the very best coaches of all time.
3 Re: FEI Week 10: Blind Side
Nowhere else on the site to comment on this, but Darrell Royal has passed away today. Rest in peace, Coach.
4 Re: FEI Week 10: Blind Side
"...but why would selected components of those ratings trump the overall ratings themselves?"
They wouldn't, if the goal is to find the "strongest team," or predict the better team, or somesuch.
What I think Fornelli is trying to get at, however awkwardly, is something more like "most deserving team." Tommy Tomlinson, over on "Sports on Earth", does this more explicitly; he refers to ranking teams in order of the "best resumés." As amusing as Tomlinson's columns are, I'm not sure it is at all possible to make an objective ranking of "most deserving," because there will always be criteria that run contrary to each other, in a way that can't be weighted.
Example: Teams A and B have the same record. Team A had a tougher strength-of-schedule, but Team B beat Team A head-to-head. It's not at all clear which team is "more deserving." We set up arbitrary tie-breakers to decide these things, but makes the tie-breakers "acceptable" isn't that they're "correct," it's that they're set up in advance so everyone knows the rules.
What people like about playoffs is that they're considered a fair way, with agreed-upon rules, of determining "most deserving." Playoffs often result in a champion who is not considered the strongest team, but that doesn't matter-- the rules are the same for everybody, and the champion "won out" while others did not. Case closed.
When discussing college football rankings, I think it would be great to explicitly separate out "strongest team" from "best resumé / most deserving"-- it would help clarify a lot of inherently unsolvable (i.e. "dumb") arguments.
Comments
4 comments, Last at 07 Nov 2012, 2:31pm