Writers of Pro Football Prospectus 2008

01 Dec 2010

FEI: Circular Logic

by Brian Fremeau

The final weekend of head-to-head matchups between the top teams in the SEC West further solidified those teams' lofty ratings in this week's FEI standings. Auburn topped Alabama (with precisely the final score FEI forecasted) and Arkansas knocked off LSU. The winners received a slight bump. The losers held their ground. In the end, they all benefitted from playing one another.

There were six games played between a combination of Auburn, Alabama, Arkansas, and LSU. Five of the six games were decided by a single score: 24-17, 31-23, 24-21, 24-20, 28-27. All but one of the 12 individual team performances ranked among the Top 100 single-game GFEI games of the year. Against everyone else, these four teams combined to win 31 of 32 FBS games by an average margin of 18 points, and the one loss was to No. 10 South Carolina. Are the SEC West team ratings artificially inflated?

In the last few years, FEI has been criticized for inflating ACC team ratings due to a similar connectivity boost. When a group of teams play one another very close and play teams outside the circle close, FEI tends to reward the entire group more than conventional wisdom would suggest. Whether those ratings are wrong or not is difficult to judge without more non-conference game data, but inevitably, a closed rating system must employ some degree of circular logic.

Auburn is No. 1 in part because Alabama is No. 2, Arkansas is No. 4, and LSU is No. 8. And those teams are ranked where they are in part because of Auburn's rating. The same logic applies to every team, but the connections may be weaker. TCU hasn't played a team ranked within 25 places of itself according to FEI. Comparing a dominant performance against a very weak opponent to a modest performance against a very strong opponent isn't easy, and FEI may not have all the answers. But whether you or I agree with every team rating isn't as important as our understanding of the system logic that produced that rating.

FEI Drive Summary Breakdown: Nevada vs. Boise State

Previous drive summary breakdowns:

The FEI system "watches" each game as a series of alternating possession and breaks down the scoreboard value of each possession to the offensive, defensive, and special teams units that contributed to that value.

In the table below, the white fields represent basic box score drive data. The yellow fields represent the component values that contribute to each team's scoring, according to FEI efficiency metrics. Offensive drive value (ODV) is the scoring value produced (or forfeited) by the offense in the given drive, including the value of driving into field goal range. Field goal value (FGV) is the value produced (or forfeited) by the field goal unit on attempts. Extra point value (XPV) is the value produced by the point-after unit touchdown drives, including two-point attempts. Field position value (FPV) is the expected score value of the drive based on starting field position alone. The gray fields represent the component values of FPV. Automatic field position value (Auto) is unearned drive value based on national average score expectations from possession alone. Punt or kickoff return value (Ret) is the value above or below average produced from the kick return that initiated the drive. The remaining value (Seq) is produced from the sequence of defensive and special teams events that preceded the drive.

The total score values include only non-garbage possessions and are the only data that is included in the weekly FEI ratings. When applicable, garbage possession data is provided as a reference, including first-half clock-kill possessions. The start of garbage time at the end of a game is calculated retroactively from the game's conclusion as a function of the score margin and the remaining possessions to be played.

First Half Non-Garbage Possessions
Drive Possession Initiated Start P Y End Result Utah SDSU ODV FGV XPV FPV Seq Ret Auto
1 Nevada Kickoff own 23 7 38 opp 39 Interception 0 0 -1.38 - - 1.38 - -.22 1.60
2 Boise State Interception own 17 11 67 opp 16 FG 0 3 1.18 .67 - 1.16 -.70 - 1.86
3 Nevada Kickoff own 32 4 19 opp 49 Punt 0 3 -1.75 - - 1.75 -.26 .15 1.86
4 Boise State Punt own 13 13 87 - Touchdown 0 10 5.94 - .04 1.02 -.69 -.15 1.86
5 Nevada Kickoff own 9 8 22 own 31 Punt 0 10 -.89 - - .89 -.26 -.71 1.86
6 Boise State Punt opp 48 6 48 - Touchdown 0 17 4.21 - .04 2.75 -.17 1.06 1.86
7 Nevada Kickoff own 30 4 70 - Touchdown 7 17 5.30 - .04 1.66 -.26 .06 1.86
8 Boise State Kickoff own 15 3 85 - Touchdown 7 24 5.87 - .04 1.09 -.26 -.51 1.86
9 Nevada Kickoff own 24 5 21 own 45 Punt 7 24 -1.42 - - 1.42 -.26 -.18 1.86
10 Boise State Punt own 13 3 12 own 25 Half 7 24 -1.02 - - 1.02 -.57 -.27 1.86
Second Half Non-Garbage Possessions
Drive Possession Initiated Start P Y End Result Utah SDSU ODV FGV XPV FPV Seq Ret Auto
11 Boise State Kickoff own 23 7 31 opp 46 Punt 7 24 -1.38 - - 1.38 - -.22 1.60
12 Nevada Punt own 20 15 63 opp 17 Failed FG 7 24 1.03 -2.30 - 1.26 -.74 .14 1.86
13 Boise State Failed FG own 16 5 12 own 28 Punt 7 24 -1.12 - - 1.12 -.74 - 1.86
14 Nevada Punt own 42 9 58 - Touchdown 14 24 4.74 - .04 2.22 -.05 .42 1.86
15 Boise State Kickoff own 20 3 -2 own 18 Punt 14 24 -1.26 - - 1.26 -.26 -.34 1.86
16 Nevada Punt own 32 5 68 - Touchdown 21 24 5.21 - .04 1.75 .41 -.52 1.86
17 Boise State Kickoff own 49 3 8 opp 43 Punt 21 24 -2.59 - - 2.59 -.26 .99 1.86
18 Nevada Punt own 7 15 87 opp 6 FG 24 24 1.80 .36 - .84 -.78 -.25 1.86
19 Boise State Kickoff own 21 1 79 - Touchdown 24 31 5.66 - .04 1.30 -.26 -.30 1.86
20 Nevada Kickoff own 21 14 79 - Touchdown 31 31 5.66 - .04 1.30 -.26 -.30 1.86
21 Boise State Kickoff own 38 2 53 opp 9 Failed FG 31 31 .51 -2.54 - 2.03 -.26 .43 1.86
Overtime Possessions
Drive Possession Initiated Start P Y End Result Utah SDSU ODV FGV XPV FPV Seq Ret Auto
22 Boise State Overtime opp 25 6 13 opp 12 Failed FG 31 31 -1.75 -2.45 - 4.20 - - 4.20
23 Nevada Overtime opp 25 4 8 opp 17 FG 34 31 -1.90 .70 - 4.20 - - 4.20
Nevada Total 34 16.41 -1.24 .16 18.67 -2.46 -1.41 22.54
Boise State Total 31 14.24 -4.32 .16 20.92 -4.17 .69 24.40

Unlike my esteemed colleague, I did not watch the thrilling finale to one of the best college football games of the season. When halftime arrived and Boise State appeared to be safely in control of the game, I turned off ESPN3 and went to bed. It was, admittedly, a bit of a thrill learning of the details of Nevada's comeback via a frenzied scan of the previous night's tweets, but I sure wish I would have watched it unfold live.

The drive summary breakdown paints a picture of two halves, with the first controlled by the Broncos and the second by the Wolf Pack. I found a few data points particularly interesting. First, there was only one drive that started in opponent territory, Boise State's 48-yard touchdown drive that opened up a 17-0 lead. Fourteen of the game's 20 other regulation drives started at or inside the team's own 25-yard line. Punt and kickoff returns were generally well defended on both sides. The big exception was Boise State's kickoff return after Nevada pulled with in three points, 24-21, early in the fourth quarter.

Boise State started only 51 yards from the end zone. On its 22 previous non-garbage drives from midfield or closer this year, the Broncos had scored 19 times, 18 of which were touchdowns. They have been lethal on short fields in 2010, but eight yards later, they had squandered the second-best field position of the night and punted the ball back to Colin Kapernick and the explosive Nevada offense.

And then there were the missed field goals. Had the first one been successful, we might well have blamed a Nevada loss on the blocked field goal that concluded their first possession of the second half. Instead, the two Boise State misses were the costliest individual plays of the game. As coach Chris Petersen has said, there were many other plays that could have been made to change the outcome. But there aren't many other plays in a football game that have such a direct impact on score value.

The last thing worth mentioning is how the drive summary breakdown may not be a great fit for all drives in every game. Note the relative value of starting field position, offensive value, and field goal value in Boise State's final possession in regulation. The Kellen Moore bomb and diving catch by Titus Young were unbelievably valuable plays, but the drive summary doesn't recognize them as such. Starting field position on a team's own 38-yard line might be worth two points on average, but not with only 13 seconds on the clock. The Moore/Young play probably deserves more than 20 percent of the potential value of that drive.

Likewise, note the fixed value of 4.20 automatic points for starting field position in overtime. It may be appropriate to value overtime drives from the opponent's 25-yard line the same as in regulation. But the goal of Nevada's final possession wasn't a touchdown since they only needed a field goal to win. These kinds of particularities in drive context might need to be examined for future drive summary analysis.

Three and Out

Previous drive stat splits and team resumes:

In the tables below, the Game Efficiency, Offensive FEI, Defensive FEI, and "Game" FEI (GFEI) for each team in each game is provided. The ranking of those individual unit and game performances is also provided. Note that there have been 663 FBS vs. FBS game played to date, meaning that there have been 1,326 individual game performances for each category.

The opponent FEI ranking is also provided, as well as a general relevance factor (Rel) for the particular GFEI, OFEI, and DFEI results for that team in that game. As stated in the FEI principles, my system rewards playing well against good teams, win or lose, and punishes losing to poor teams more harshly than it rewards defeating poor teams. In the formula, the relevance factor is partly a function of the relative ratings of the two teams. Across all games, the least relevant results receive about one-eighth as much weight as the most relevant results. For simplicity, I've generalized the relevance data here into three equally distributed categories, High, Med, and Low.

No. 3 Stanford Cardinal (10-1)
Date Wk Opponent Result Opp
FEI
Opp
OFEI
Opp
DFEI
GE GE
Rk
FPA FPA
Rk
OFEI OFEI
Rk
DFEI DFEI
Rk
GFEI GFEI
Rk
Rel
9/11 2 at UCLA W 35-0 82 100 72 .500 67 .591 182 .526 419 -.238 320 .418 99 Low
9/18 3 Wake Forest W 68-24 105 98 95 .724 15 .590 184 1.546 31 .261 728 .336 184 Low
9/25 4 at Notre Dame W 37-14 31 52 28 .300 207 .536 439 .800 262 -.797 67 .579 28 Med
10/2 5 at Oregon L 31-52 6 16 18 -.148 947 .487 726 .999 169 .143 629 .424 95 High
10/9 6 USC W 37-35 28 40 63 .029 612 .527 501 .925 203 .756 1066 .179 370 Med
10/23 8 Washington State W 38-28 100 93 119 .270 238 .596 158 -.117 907 1.100 1211 -.081 804 Low
10/30 9 at Washington W 41-0 74 76 86 .554 51 .628 74 .418 491 -.775 72 .525 39 Low
11/6 10 Arizona W 42-17 42 19 76 .420 121 .485 742 1.459 49 -.881 44 .463 71 Low
11/13 11 at Arizona State W 17-13 41 65 40 .052 543 .471 833 .285 598 -.258 303 .245 287 Low
11/20 12 at California W 48-14 89 110 70 .905 3 .560 313 2.271 3 -.161 387 .786 4 Low
11/27 13 Oregon State W 38-0 44 59 81 .603 35 .483 754 1.252 91 -1.042 25 .630 15 Low

Has there been any team playing better football in the last two weeks than Stanford? I discussed last week a few of the reasons the Cardinal jumped ahead of Oregon in the FEI ratings, despite the fact that Oregon beat Stanford 52-31 on October 2. That head-to-head matchup is one thing, but Stanford's games versus four other Pac-10 foes -- Arizona, California, Oregon State, and Washington -- were the best GFEI ratings posted against those teams.

It's interesting to note the relevance value of each game for Stanford. Individually, the importance of a given Low relevance game may only be 40 percent as relevant as the Oregon game, but the total relevance of those games is still greater than that of the Oregon, USC, and Notre Dame games combined.

No. 7 Ohio State Buckeyes (11-1)
Date Wk Opponent Result Opp
FEI
Opp
OFEI
Opp
DFEI
GE GE
Rk
FPA FPA
Rk
OFEI OFEI
Rk
DFEI DFEI
Rk
GFEI GFEI
Rk
Rel
9/2 1 Marshall W 45-7 103 109 52 .526 59 .535 446 1.254 90 -.020 481 .152 402 Low
9/11 2 Miami W 36-24 19 29 3 .132 399 .534 454 1.034 157 -.875 46 .411 105 Med
9/18 3 Ohio W 43-7 81 89 62 .468 94 .579 227 .197 663 -.751 78 .242 290 Low
9/25 4 Eastern Michigan W 73-20 115 84 120 .531 56 .581 219 .149 704 .587 956 -.011 683 Low
10/2 5 at Illinois W 24-13 30 49 21 .137 397 .494 692 .807 256 -.447 209 .424 94 Med
10/9 6 Indiana W 38-10 90 45 97 .556 48 .558 324 .596 381 -1.032 26 .288 238 Low
10/16 7 at Wisconsin L 18-31 9 5 30 -.232 1043 .447 974 .801 261 .115 603 .284 246 High
10/23 8 Purdue W 49-0 88 108 43 .667 24 .548 377 1.675 24 -.205 357 .407 106 Low
10/30 9 at Minnesota W 52-10 72 42 98 .421 119 .635 65 .128 720 -.565 152 .400 112 Low
11/13 11 Penn State W 38-14 53 72 53 .243 275 .667 23 .363 534 .230 705 .207 331 Low
11/20 12 at Iowa W 20-17 21 28 15 .048 554 .479 785 1.049 153 -.246 312 .443 89 Med
11/27 13 Michigan W 37-7 47 2 104 .429 116 .601 137 .046 790 -1.900 1 .444 88 Med

Ohio State turned in the single best opponent-adjusted defensive performance of the year on Saturday against hated rival Michigan. The Wolverines turned the ball over (via fumble, interception or downs) on five of their first seven drives against the Buckeyes. Like Stanford, Ohio State ranks ahead of the one team it lost to, Wisconsin. Unlike Stanford, the Buckeyes likely wouldn't be much higher in the FEI ratings if it had won that game.

No. 15 TCU Horned Frogs (11-0)
Date Wk Opponent Result Opp
FEI
Opp
OFEI
Opp
DFEI
GE GE
Rk
FPA FPA
Rk
OFEI OFEI
Rk
DFEI DFEI
Rk
GFEI GFEI
Rk
Rel
9/4 1 Oregon State W 30-21 44 59 81 .129 407 .506 633 .396 506 .220 697 .155 399 Med
9/18 3 Baylor W 45-10 71 33 106 .471 93 .530 487 1.000 168 -.717 91 .318 207 Low
9/24 4 at SMU W 41-24 69 51 48 .211 309 .564 296 .781 272 .023 523 .221 316 Low
10/2 5 at Colorado State W 27-0 119 117 116 .286 220 .524 525 -.861 1235 .048 544 -.225 1009 Low
10/9 6 Wyoming W 45-0 112 104 114 .905 2 .562 304 1.223 101 -.248 308 .420 96 Low
10/16 7 BYU W 31-3 75 99 51 .273 230 .636 64 .004 826 -.236 322 .098 499 Low
10/23 8 Air Force W 38-7 43 31 61 .490 79 .526 517 2.087 8 -.610 132 .517 42 Med
10/30 9 at UNLV W 48-6 116 114 112 .487 80 .640 60 .002 827 .579 945 -.001 663 Low
11/6 10 at Utah W 47-7 48 69 36 .556 46 .598 149 1.080 141 -.865 50 .712 6 Med
11/13 11 San Diego State W 40-35 52 26 57 .045 561 .526 515 .250 617 -.284 295 .012 633 Med
11/27 13 at New Mexico W 66-17 118 118 109 .385 139 .555 347 -.254 994 .674 1012 -.118 859 Low

With Boise State's loss last week, TCU moved into position to potentially grab a BCS championship game bid if Auburn or Oregon were to slip. FEI doesn't think the Horned Frogs are worthy of that distinction, having not played a single opponent currently ranked in the Top 40. Only Kent State, Middle Tennessee, and Bowling Green share that distinction. The current FEI Top 25 faced an average of 4.9 Top 40 opponents each.

If you have a suggestion for an FEI team resume you'd like to see, drop me a line on Twitter (@bcfremeau) or in the comment section here. I'm happy to answer data inquiries or provide team resume tables for bloggers interested in investigating the data themselves.

FEI Week 13 Top 25

The principles of the Fremeau Efficiency Index (FEI) can be found here. FEI rewards playing well against good teams, win or lose, and punishes losing to poor teams more harshly than it rewards defeating poor teams. FEI is drive-based, not play-by-play based, and it is specifically engineered to measure the college game.

FEI is the opponent-adjusted value of Game Efficiency (GE), a measurement of the success rate of a team scoring and preventing opponent scoring throughout the non-garbage-time possessions of a game. FEI represents a team's efficiency value over average. Strength of Schedule (SOS) is calculated as the likelihood that an elite team (two standard deviations above average) would win every game on the given team's schedule to date. SOS listed here does not include future games scheduled.

Mean Wins (FBS MW) represent the average total games a team with the given FEI rating should expect to win against its complete schedule of FBS opponents. Remaining Mean Wins (FBS RW) represent the average expected team wins for games scheduled but not yet played.

Offensive FEI (OFEI) and Defensive FEI (DFEI) are the opponent-adjusted ratings of all non-garbage-time drives from scrimmage. Field Position Advantage (FPA) is the share of the value of total starting field position for the season earned by each team against its opponents. Field Goal Efficiency (FGE) is the point value per field goal attempt earned by the field goal unit.

Only games between FBS teams are considered in the FEI calculations. The FEI ratings published here are a function of the results of games played through November 27.

FEI ratings for all 120 FBS teams are listed in the stats page section of FootballOutsiders.com. Click here for current ratings; the pull-down menu in the stats section directs you to 2007 through 2009 ratings. There are also now separate pages for offensive and defensive FEI ratings for 2010.

Rk Team FBS
Rec
FEI LW
Rk
GE GE
Rk
SOS SOS
Rk
FBS
MW
FBS
RW
OFEI OFEI
Rk
DFEI DFEI
Rk
FPA FPA
Rk
FGE FGE
Rk
1 Auburn 11-0 .324 1 .154 17 .093 11 10.3 0.7 .805 1 -.408 13 .511 48 .134 52
2 Alabama 8-3 .263 2 .235 9 .061 8 8.6 - .513 4 -.358 19 .550 10 .253 37
3 Stanford 10-1 .262 4 .321 4 .286 63 9.7 - .420 9 -.338 22 .537 16 .208 44
4 Arkansas 9-2 .261 6 .145 18 .042 1 8.3 - .649 3 -.409 12 .499 59 .398 22
5 Virginia Tech 10-1 .258 3 .254 7 .191 38 10.0 0.7 .400 11 -.432 9 .563 3 .676 4
6 Oregon 10-0 .253 5 .285 5 .393 80 9.7 0.8 .341 16 -.373 18 .557 6 .088 58
7 Ohio State 11-1 .232 9 .324 3 .229 49 10.2 - .325 18 -.474 6 .557 5 .124 55
8 LSU 9-2 .229 8 .094 29 .049 3 7.9 - .185 35 -.353 20 .586 2 .602 7
9 Wisconsin 10-1 .225 10 .257 6 .284 62 9.3 - .487 5 -.268 30 .548 13 .467 17
10 South Carolina 8-3 .224 13 .161 15 .061 7 8.2 0.3 .478 6 -.381 16 .555 8 .052 64
11 Oklahoma 10-2 .223 15 .173 14 .206 43 10.5 0.6 .360 14 -.463 7 .532 21 .223 40
12 Boise State 10-1 .221 7 .413 1 .419 88 10.8 1.0 .374 13 -.324 26 .544 14 .040 66
Rk Team FBS
Rec
FEI LW
Rk
GE GE
Rk
SOS SOS
Rk
FBS
MW
FBS
RW
OFEI OFEI
Rk
DFEI DFEI
Rk
FPA FPA
Rk
FGE FGE
Rk
13 Nebraska 9-2 .204 12 .205 11 .314 65 9.7 0.4 .236 24 -.436 8 .549 11 .819 2
14 West Virginia 7-3 .199 23 .132 21 .240 51 8.6 1.0 .055 47 -.679 1 .506 51 -.126 85
15 TCU 11-0 .198 11 .336 2 .728 117 10.2 - .286 20 -.420 10 .561 4 .126 54
16 Missouri 9-2 .197 14 .143 19 .253 54 8.7 - .220 27 -.499 5 .518 41 .419 20
17 North Carolina State 7-4 .188 16 .082 30 .180 35 7.9 - .184 36 -.418 11 .518 40 -.073 81
18 Florida State 8-3 .188 21 .106 25 .137 19 8.1 0.3 .429 8 -.182 34 .532 22 .060 63
19 Miami 6-5 .178 17 .035 49 .114 13 7.1 - .218 29 -.517 3 .492 72 .199 45
20 Oklahoma State 10-2 .172 19 .183 13 .384 79 9.7 - .348 15 -.264 31 .524 31 .550 14
21 Iowa 6-5 .164 18 .137 20 .325 67 8.2 - .219 28 -.385 15 .522 35 -.015 78
22 Michigan State 10-1 .155 22 .103 26 .329 69 8.2 - .331 17 -.374 17 .481 84 .588 8
23 Texas A&M 8-3 .147 25 .081 31 .165 28 7.2 - .152 41 -.502 4 .464 103 .279 31
24 Pittsburgh 5-5 .141 20 .058 41 .276 60 7.1 0.7 .249 23 -.205 33 .507 50 .009 74
25 Clemson 5-6 .137 24 .030 51 .059 5 6.1 - .035 56 -.550 2 .498 61 -.589 110

Posted by: Brian Fremeau on 01 Dec 2010

9 comments, Last at 03 Dec 2010, 7:19pm by cfn_ms

Comments

1
by Scott P. (not verified) :: Wed, 12/01/2010 - 3:06pm

"Has there been any team playing better football in the last two weeks than Stanford?"

Wisconsin.

2
by Kal :: Wed, 12/01/2010 - 9:00pm

Nah. Stanford's played significantly better opponents than Wisconsin has and done significantly better against them. Beating up on Indiana is impressive with 3rd stringers, but at some level it's just not as interesting as beating a very hard Cal team at home by 34 points in a game that wasn't that close.

4
by Will :: Thu, 12/02/2010 - 12:43am

Is there any game that's shaping up better than the Rose Bowl? Wisconsin vs Oregon/Stanford/TCU are all great games.

Will

3
by TomTom (not verified) :: Wed, 12/01/2010 - 11:40pm

What is the deal with SOS being so sporadic? Stanford and Oregon being 63 and 80 differ greatly from, say, the well respected Sanagrin ratings. Looking at Stanford and Oregon's schedules, they obviously got lucky with the home/away schedule, so perhaps it is the weight put onto those? What affect will the bowl games have on a circular situation, such as the SEC West, too?

5
by Will :: Thu, 12/02/2010 - 12:45am

FEI's SoS is (I believe) based on the chance of going undefeated, not on overall strength. The SEC West teams have all played 3 or 4 games where they legitimately had a good chance to lose, whereas Oregon and Stanford only had one (each other).

Will

6
by Brian Fremeau :: Thu, 12/02/2010 - 10:24am

This is correct.

Some time this off-season, I plan to run a version of my SOS methodology with Sagarin's ratings in order to demonstrate the difference. The underlying ratings would be the same, but the SOS methodology and goal are very different. An average of a team's opponent ratings is a lousy way to describe the difficulty of its schedule.

9
by cfn_ms :: Fri, 12/03/2010 - 7:19pm

I wouldn't say it's a lousy way to describe it, but I would say that it's limited. I don't think that, for instance, it makes much sense to rate Rutgers's schedule based on the odds of an elite team running the table.

I would suggest that, for a more complete picture of SOS, you'd instead run a couple different things under the SOS banner:

1) average opponent rating (it's still a reasonable default, even if it is limited)
2) likelihood of an elite team going undefeated against it
3) likelihood of an avg team hitting 6-6 against it (which is far more relevant for the bulk majority of 1-A teams IMO)

My guess is that for some teams you'd get three materially different numbers, and for most at least one of the three would be a good deal off from the other two.

7
by ammek :: Thu, 12/02/2010 - 10:46am

FEI doesn't seem to have a 'variance' stat like DVOA does; would it be a useful addition? I'd certainly like to know whether the award for "What theā€¦?" team of the year should indeed go to California.

8
by Bill Connelly :: Thu, 12/02/2010 - 10:53am

I ran standard deviation on VN a couple of weeks ago, and yeah...California and Oregon State were by far the two most high-variability teams...