Writers of Pro Football Prospectus 2008

26 Nov 2014

FEI Week 13: Undefeated and Unloved

by Brian Fremeau

For the first time since October 11, the Florida State Seminoles played a football game in which they did not fall behind their opponent, but their victory over Boston College on Saturday was yet another dogfight. With the game tied 17-17, the Eagles manufactured a 17-play, 67-yard drive that took nearly nine minutes off the clock in the fourth quarter. That drive came up empty in the end, however, and Jameis Winston led the Seminoles down the field over the final four minutes of the game to the Boston College 8-yard line for a chip shot field goal and a 20-17 victory.

Florida State has faced criticism this season for not being dominant. A year ago, they throttled their FBS opponents by an average regular season victory margin of 42 points per game. This year, Florida State hasn't beaten a single FBS opponent by that margin, and they have recorded five victories by a final margin of six or fewer points each. The close calls appear to be factoring into the College Football Playoff selection committee deliberations as well. The committee has yet to rank Florida State No. 1 in any of their weekly rankings, and for the third straight week, FSU ranks behind at least one team saddled with a loss. The Seminoles have an unblemished record, but a significantly blemished perception.

The FEI ratings haven't had the Seminoles on top since Week 4, but with an overall opponent-adjusted efficiency ranking of No. 4 this week, FEI does think more highly of Florida State's body of work than most other computer systems. And the Seminoles do remain No. 1 in terms of degree of difficulty.

I started publishing degree of difficulty ratings (DOD) this year as a way to attempt to approximate what the playoff selection committee priorities are likely to be. DOD ratings are based on current FEI ratings, but instead of measuring efficiency against schedule, DOD measures record against schedule. How difficult would it be for an elite team (two standard deviations better than average) to play a given team's schedule to date and achieve that team's record?

Prior to this week, I had been publishing the DOD rankings only for teams with two or fewer losses because these were the only teams likely to have any hope of claiming one of the four bids in the end. But the rankings after the top 10 or so were misleading because I was ignoring three-loss teams that would clearly rank ahead of two-loss teams. I've revised my approach to now rank the top 25 teams according to DOD regardless of record. I've also gone back and calculated the top 10 pre-bowl DOD rankings for each year of the FEI era, 2003 to present, in order to provide a broader perspective.

The current College Football Playoff top four are the same group of four teams ranked at the top of this week's DOD rankings, albeit in a different order. In the previous 11 seasons, the top four DOD rankings represented the same four teams in the BCS top four only twice: 2004 and 2011. The top four in the DOD rankings were all ranked among the top five teams in the BCS final standings five times in that span. I'm not seeking to develop something that would mimic the BCS and CFP top four, but DOD does seek to approximate their priorities, and it appears to do a pretty good job overall on that front.

Another interesting observation is of the DOD ratings themselves, which, by definition, represent the likelihood of an elite team having the given team's record. Because of this standardized definition, comparisons of teams from year to year are possible. Since 2003, the three teams that achieved the highest degree of difficulty heading into bowl season were the 2010 Auburn Tigers (.081 DOD), the 2009 Alabama Crimson Tide (.105 DOD) and the 2004 USC Trojans (.122 DOD). All three teams went on to win the BCS national championship game in that season.

DOD should not be confused with a power rating or projection rating, however. The next two teams on the list of highest regular season degree of difficulty achieved were the 2012 Notre Dame Fighting Irish (.128 DOD) and the 2011 LSU Tigers (.147 DOD), both of which were crushed by championship Alabama teams in BCS title games. Neither the 2012 Crimson Tide (.611 DOD) nor the 2011 Crimson Tide (.705 DOD) had particularly strong DOD ratings.

What value might DOD actually provide? This is tricky, because I think DOD creeps into "deserve" versus "best" territory, which can be awfully contentious and challenging to debate. That territory is why there is a selection committee and not a computer formula in the first place. If DOD or other similar metrics could provide some value, I think it is in standardizing these questions with more consistently concrete answers. The committee isn't wrong if it arrives at a different conclusion, but as I've said before, I hope they are answering their own questions consistently.

Degree of Difficulty through Week 13

The degree of difficulty rankings (DOD) are based on current FEI ratings, but instead of measuring efficiency against schedule, DOD measures record against schedule. How difficult would it be for an elite team (two standard deviations better than average) to play a given team's schedule to date and achieve that team's record?

My hypothesis is that the inaugural College Football Playoff selection committee is likely to value and reward something akin to DOD through their process and deliberations. The current selection committee rankings are provided in the table for comparison.

Degree of Difficulty: Record Against Schedule To Date
DOD
Rank
Team Record DOD FEI FEI
Rank
SOS
Pvs
Rank SOS Rank Win
Out
CFP
1 Florida State 10-0 .241 .256 4 .241 51 .225 55 .899 3
2 Alabama 9-1 .463 .297 2 .141 24 .103 23 .752 1
3 Oregon 9-1 .526 .306 1 .159 30 .149 38 .956 2
4 Mississippi State 9-1 .575 .229 10 .157 28 .093 21 .456 4
5 UCLA 9-2 .649 .254 5 .088 17 .079 17 .858 8
6 TCU 8-1 .672 .242 7 .270 58 .253 60 .887 5
7 Ohio State 10-1 .708 .233 8 .324 68 .319 71 .966 6
8 Arizona 9-2 .737 .218 13 .079 11 .063 11 .676 11
9 Baylor 8-1 .753 .229 11 .343 71 .286 68 .745 7
10 Georgia 8-2 .839 .273 3 .192 40 .134 33 .670 9
11 Marshall 10-0 .860 .116 36 .860 128 .847 128 .902 24
12 Colorado State 9-1 .876 .125 33 .508 93 .471 93 .669 -
DOD
Rank
Team Record DOD FEI FEI
Rank
SOS
Pvs
Rank SOS Rank Win
Out
CFP
13 Kansas State 7-2 .881 .159 21 .202 42 .114 27 .236 12
14 Arizona State 8-2 .882 .190 15 .221 46 .132 32 .324 13
15 Auburn 7-3 .884 .231 9 .072 9 .027 1 .248 15
16 Georgia Tech 8-2 .896 .247 6 .263 55 .122 29 .330 16
17 Clemson 7-3 .901 .158 22 .067 4 .062 9 .731 21
18 Michigan State 8-2 .913 .157 23 .183 36 .163 46 .675 10
19 Missouri 8-2 .937 .156 25 .282 61 .235 57 .534 17
20 Louisville 7-3 .947 .201 14 .161 32 .158 42 .981 22
21 Mississippi 7-3 .950 .219 12 .142 25 .104 24 .544 19
22 Wisconsin 8-2 .981 .181 17 .496 91 .439 88 .719 14
23 Duke 7-3 .982 .153 27 .212 44 .209 54 .965 -
24 Boise State 9-2 .984 .089 42 .445 85 .431 85 .708 23
25 Minnesota 7-3 .988 .133 30 .243 52 .162 45 .281 18
DOD - Likelihood that an elite team would have the given team's record against the given team's schedule
SOS Pvs - Strength of Schedule of all games played to date
SOS - Strength of Schedule of all games scheduled
Win Out - Likelihood given team will win all of its remaining games
CFP - College Football Playoff selection committee ranking

FEI 2014 Week 13 Ratings

The Fremeau Efficiency Index (FEI) is a college football rating system based on opponent-adjusted drive efficiency. Nearly 20,000 possessions are contested annually in FBS vs. FBS games. First-half clock-kills and end-of-game garbage drives and scores are filtered out. Game Efficiency (GE) is a function of the starting field position and outcome of non-garbage possessions. Opponent adjustments are calculated with special emphasis placed on quality performances against good teams, win or lose.

Strength of Schedule (SOS) is calculated as the likelihood that an "elite team" (two standard deviations above average) would win every game on the given team's schedule. SOS listed here includes all regular season games scheduled. A multifaceted approach to measuring schedule strength is available here and visualized here.

Mean Wins (FBS MW) represent the average total games a team with the given FEI rating should expect to win against its complete schedule of FBS opponents. Remaining Mean Wins (FBS RMW) represent the average total games a team with the given FEI rating should expect to win against the remaining opponents on its schedule.

Offensive FEI (OFEI) is opponent-adjusted offensive efficiency. Defensive FEI (DFEI) is opponent-adjusted defensive efficiency. Special Teams Efficiency (STE) is the composite efficiency of field goal kickoff, punt, and return units. Field Position Advantage (FPA) is the average share of field position value generated by offense, defense, and special teams efficiency.

These ratings are exclusively produced from 2014 game data and are not influenced by preseason projections. Complete ratings and ratings splits for all 128 FBS teams are available here. Supplemental data including points per drive, game splits, and game factors are available as well.

Rk Team FBS
Rec
FEI LW GE GE
Rk
SOS Rk
FBS
MW
FBS
RMW
OFEI Rk DFEI Rk STE Rk FPA Rk
1 Oregon 9-1 .306 1 .246 4 .149 38 9.6 1.0 .768 3 -.311 29 1.798 11 .543 13
2 Alabama 9-1 .297 2 .205 9 .103 23 9.2 .8 .605 8 -.592 7 -.777 97 .481 101
3 Georgia 8-2 .273 4 .241 5 .134 33 9.1 .7 .616 7 -.371 24 2.437 6 .591 1
4 Florida State 10-0 .256 5 .135 19 .225 55 9.3 .9 .664 6 -.433 16 .735 36 .497 67
5 UCLA 9-2 .254 7 .079 36 .079 17 9.3 .9 .692 5 -.306 32 .280 55 .496 68
6 Georgia Tech 8-2 .247 8 .164 14 .122 29 8.6 .3 .910 1 -.075 58 1.445 17 .556 8
7 TCU 8-1 .242 10 .201 10 .253 60 9.4 1.9 .403 17 -.571 8 1.567 14 .557 7
8 Ohio State 10-1 .233 9 .264 2 .319 71 10.5 1.0 .518 11 -.426 17 .948 27 .566 3
9 Auburn 7-3 .231 12 .103 26 .027 1 7.3 .2 .773 2 -.278 38 -.589 90 .504 51
10 Mississippi State 9-1 .229 15 .233 6 .093 21 8.5 .5 .341 24 -.610 5 .527 49 .530 25
11 Baylor 8-1 .229 6 .252 3 .286 68 9.3 1.7 .426 16 -.415 19 .784 34 .561 6
12 Mississippi 7-3 .219 3 .166 13 .104 24 7.9 .5 .165 43 -.669 4 .727 37 .537 16
Rk Team FBS
Rec
FEI LW GE GE
Rk
SOS Rk
FBS
MW
FBS
RMW
OFEI Rk DFEI Rk STE Rk FPA Rk
13 Arizona 9-2 .218 16 .110 23 .063 11 8.9 .7 .382 18 -.302 33 .240 58 .554 9
14 Louisville 7-3 .201 19 .093 29 .158 42 8.2 1.0 .168 42 -.678 3 -.638 91 .482 98
15 Arizona State 8-2 .190 14 .076 38 .132 32 7.5 .3 .363 22 -.389 22 .709 39 .532 23
16 USC 7-4 .187 11 .104 25 .075 14 8.0 .7 .255 33 -.309 31 .506 50 .512 39
17 Wisconsin 8-2 .181 21 .232 7 .439 88 9.1 .7 .313 29 -.459 13 .142 63 .501 56
18 Oklahoma 8-3 .175 20 .190 11 .225 56 9.3 1.0 .507 12 -.201 45 2.116 10 .529 26
19 Utah 6-4 .170 13 .024 56 .069 13 6.7 .9 -.106 70 -.484 11 2.659 5 .535 18
20 Arkansas 5-5 .169 34 .094 28 .049 5 6.4 .5 .326 26 -.412 20 -.758 94 .518 34
21 Kansas State 7-2 .159 28 .154 15 .114 27 7.5 1.2 .308 30 -.287 36 2.349 8 .543 14
22 Clemson 7-3 .158 23 .082 33 .062 9 6.8 .7 -.011 60 -.774 1 -.902 102 .505 50
23 Michigan State 8-2 .157 24 .213 8 .163 46 8.1 .7 .318 28 -.164 50 .978 26 .553 10
24 Miami 5-5 .157 17 .076 37 .068 12 6.3 .8 .491 13 -.326 27 -.509 88 .486 89
25 Missouri 8-2 .156 27 .107 24 .235 57 7.9 .5 -.027 62 -.512 10 .852 31 .501 57

Posted by: Brian Fremeau on 26 Nov 2014

7 comments, Last at 29 Nov 2014, 10:06am by Brian Fremeau

Comments

1
by gmoney_714 :: Wed, 11/26/2014 - 2:54pm

Great analysis Brian. Thank you

2
by Kal :: Wed, 11/26/2014 - 6:28pm

Asked this over at football study hall. Why is USC ranked higher than notre dame? Notre dame has a higher ofei, dfei, special teams and field position yet is below USC.

Any ideas?

3
by Thok :: Thu, 11/27/2014 - 9:56am

Presumably USC is rewarded for its higher strength of schedule.

4
by Kal :: Thu, 11/27/2014 - 1:16pm

That's supposed to be baked in to the OFEI, DFEI and SFEI rankings. Those aren't raw efficiency rankings; these are adjusted.

I was thinking it was some kind of Simpson's paradox effect manifesting, but that doesn't make a whole lot of sense here from what I understand of how the three values combine to overall FEI. But maybe?

5
by Brian Fremeau :: Thu, 11/27/2014 - 3:36pm

FEI is a function of raw game efficiency adjusted for opponents. USC has a better overall raw game efficiency and a stronger set of opponents.

OFEI is a function of raw offensive efficiency adjusted for opponent defenses. DFEI is a function of raw defensive efficiency adjusted for opponents offenses. For OFEI, DFEI and FEI, a "relevance" factor gives a weight to each game result. You can see the game-by-game results for FEI, OFEI, and DFEI here:
Notre Dame: http://www.bcftoys.com/2014-game-factors#notre-dame
USC: http://www.bcftoys.com/2014-game-factors#usc

Special Teams Efficiency and Field Position Advantage numbers are not opponent-adjusted. It should be noted that FPA data is also tied into offense, defense, and special teams data, each of which contribute to field position.

6
by Kal :: Sat, 11/29/2014 - 3:17am

Okay, so it was a bit of Simpson's Paradox. From what you're saying it sounds like OFEI and DFEI aren't weighted when you're combining the values to get the one true OFEI for a team - is that right? So the OFEI value in the table is essentially some normalized value of all the OFEI values, but isn't considering game weight?

Or is it that OFEI is OE * opponent defensive strength of schedule, and DFEI = DE * opponent offensive strength of schedule, but that isn't the same as FEI = GE * overall strength of schedule? If so, that seems like a bit of an oddity overall; ranking FEI as a different product of strength of schedule compared to offense/defensive SOS is at least misleading and possibly ending up in something that might not be as good overall.

7
by Brian Fremeau :: Sat, 11/29/2014 - 10:06am

OFEI and DFEI do factor in game weight, just like FEI does. OFEI is OE adjusted for opponent defense, DFEI is DE adjusted for opponent offense, and FEI is GE adjusted for opponent overall strength.

Another factor in the opponent adjustments is where the games were played. Note the USC and Notre Dame GFEI results against Stanford. Similar GE results (.031, #592 overall for ND; .050, #522 overall for USC) but not-as-similar GFEI results (.195, #358 overall for ND; .360, #158 overall for USC) because ND hosted Stanford and USC played them on the road. To date, USC has played twice as many true road games this year than ND.