FEI Week 10: Blind Side

by Brian Fremeau

If you haven’t seen it yet, there is an interesting online exercise offered by Tom Fornelli of CBS Sports, a blind test for fans to remove their own bias and select two teams to play for a national championship game based on resume alone. Spoiler alert: based on results to date, the overwhelming consensus is that the profiles of Kansas State and Notre Dame trump Alabama and Oregon. That is to say, the overwhelming consensus is that the specific factors selected for the blind test favor Kansas State and Notre Dame over Alabama and Oregon.

The factors Fornelli uses aren’t completely unsound. Wins over top-20 and top-40 opponents, opponent winning percentage, average margin of victory, and average opponent rank are all measures that are perfectly reasonable to use as part of a fair evaluation of a team. But let’s not pretend this exercise means much.

The use of opponent records and margin of victory are problematic without the context of tempo, garbage time, and individual game results. I challenge the use of average opponent rating instead of evaluating performances against opponents from the top down. And three of the five measures selected by Fornelli use the overall ratings published by Jeff Sagarin (top-20 wins, top-40 wins, average opponent rank). Sagarin’s ratings are well-respected and sound, but why would selected components of those ratings trump the overall ratings themselves? If Sagarin’s system is so useful in helping fans determine who has the best profile, why wouldn’t we trust Sagarin’s system to distinguish between these teams? Sagarin ranks Alabama No. 1, Kansas State No. 2, Notre Dame No. 3, and Oregon No. 4, and (surprise!), does not trump up top-20 and top-40 victories as final determining factors.

If I offered up a similar exercise and asked you to select teams based on selected portions of the FEI ratings, I would imagine that you would challenge the method by which those categories were chosen. Wins over top-5 FEI teams? Kansas State and Notre Dame have one apiece. Wins over top-10 FEI teams? Kansas State, Notre Dame, Florida, LSU, Georgia, Washington, and North Carolina State all have one apiece. Are wins over selected categories a better measure than the overall ratings themselves, or should Oregon and Alabama rank behind all of the teams I just named?

The truth, of course, is that the Sagarin ratings themselves (and FEI, S&P+, F/+, etc.) are far more sophisticated evaluations of team profiles than the Fornelli exercise because they consider many factors rather than just a hand-picked few. It may help fans and voters to remove as much noise as possible and focus on a few select statistical categories as they evaluate teams, and that’s okay. But this exercise doesn’t allow for the user to seek and weight factors that are important to them individually. And it strips away the best asset of computer systems: their ability to actually make some sense out of the noise rather than reduce it.

Computer ranking systems don’t have a monopoly on the truth, but we need to better understand them and use them properly as tools to evaluate what has happened and what may happen going forward.

Week 10 Revisionist Box Scores

This weekly feature identifies the games played each week that were most impacted by turnovers, special teams, field position, or some combination of the three. The neutralized margin of victory is a function of the point values earned and surrendered based on field position and expected scoring rates.

2012 totals to date:

• Net Total Turnover Value was the difference in 80 of 511 FBS games (15.7 percent)
• Net Special Teams Value was the difference in 39 of 511 FBS games (7.6 percent)
• Net Field Position Value was the difference in 48 of 511 FBS games (9.4 percent)
• Turnovers, Special Teams and/or Field Position was the difference in 111 of 511 FBS games (21.7 percent)

2012 Game Splits for all teams, including the offensive, defensive, special teams, field position, and turnover values recorded in each FBS game are provided here.

FEI Week 10 Top 25

The Fremeau Efficiency Index (FEI) rewards playing well against good teams, win or lose, and punishes losing to poor teams more harshly than it rewards defeating poor teams. FEI is drive-based and it is specifically engineered to measure the college game. FEI is the opponent-adjusted value of Game Efficiency (GE), a measurement of the success rate of a team scoring and preventing opponent scoring throughout the non-garbage-time possessions of a game. FEI represents a team's efficiency value over average.

Other definitions:

• SOS Pvs: Strength of schedule to date, based on the likelihood of an elite team going undefeated against the given team's schedule to date.
• SOS Fut: Strength of schedule, based on the likelihood of an elite team going undefeated against the given team's remaining schedule.
• FBS MW: Mean Wins, the average number of games a team with the given FEI rating would be expected to win against its entire schedule.
• FBS RMW: Remaining Mean Wins, the average number of games a team with the given FEI rating would be expected to win against its remaining schedule.
• OFEI: Offensive FEI, the opponent-adjusted efficiency of the given team's offense.
• DFEI: Defensive FEI, the opponent-adjusted efficiency of the given team's defense.
• STE: Special Teams Efficiency, the scoring value earned by field goal, punt and kickoff units measured in points per average game.
• FPA: Field Position Advantage, the share of the value of total starting field position earned by each team against its opponents.

These FEI ratings are a function of results of games played through November 3rd. The ratings for all FBS teams, including FEI splits for Offense, Defense, and Special Teams can be found here. Program FEI (five-year weighted) ratings and other supplemental drive-based data can be found here.