Writers of Pro Football Prospectus 2008

11 Nov 2009

Week 10 FEI Ratings

by Brian Fremeau

In last week’s Seventh Day Adventure column, I opined that the 2009 college football season may be careening toward an unsatisfactory conclusion similar to 2004. With a lack of connectivity among the top contenders and few other challengers on the road ahead, the end of the regular season may present us with a murky mess of multiple undefeated teams, including more than two from BCS conferences. Iowa then promptly lost at home Saturday to an inferior Northwestern squad, bowing out of the BCS title chase and clearing up some of the murkiness.

There was something inevitable about the Hawkeyes’ loss. They had been living dangerously way too often this year, pulling out fourth quarter comebacks and narrow victories ever since an opening weekend escape from FCS Northern Iowa. Once quarterback Ricky Stanzi was sidelined early against the Wildcats, late-game heroics were out of the question. If ever there were a circumstance primed for an upset, this was it.

Iowa, a 14-point favorite over Northwestern heading into the game, was one of four double-digit favorites to fall on Saturday. Notre Dame (-12) fell victim to Navy, Louisiana-Lafayette (+13.5) edged Arkansas State, and Baylor (+14.5) dropped Missouri. The Iowa loss is the only one of the four with significant national implications, but does that make it the biggest upset? Better yet, are upsets best measured by Vegas lines set in advance of the game, or should they only be judged retroactively at the end of the season?

Vegas betting lines aren’t precisely designed to predict game outcomes. But since they are designed to balance wagers on either side of the line, they do represent the fair estimate of the betting public’s consensus prediction for the game. And they’re fairly accurate, too. Since 2003, teams favored to win by Vegas have a .746 overall winning percentage. Double-digit favorites over the same span have a .874 winning percentage.

Perceptions of the betting public in advance of a game are one thing, but should results and upset judgments be reevaluated once new data is collected? USC may have been a favorite at Oregon when the two teams faced off on Halloween, but the outcome of that game and others certainly can change our perspective on which was the lesser team in the match-up. We can use a rating system like FEI to judge upsets retroactively, but since new game data can shake up the ratings weekly, it is probably best reserved for end-of-year assessments.

If the season ended today, Iowa’s loss to Northwestern is the least likely outcome based on current FEI ratings. Two other games have a larger FEI rating delta between the teams involved in the upset -- No. 9 Clemson lost to No. 70 Maryland, and No. 32 Houston lost to No. 99 UTEP -- but both of those were played in the weaker team’s stadium. Iowa’s home loss stands alone, for now. If the Hawkeyes keep losing, or Northwestern finishes strong, it may not measure as a very significant upset after all.

I ran a comprehensive assessment over the last six seasons (2003-2008) of the number of times teams with a lower FEI end-of-year rating defeated an opponent with a higher one. Such "upsets" occurred in 21.8 percent of all games played in that span. In games in which the FEI rating delta exceeded 0.1 (roughly equivalent to a touchdown difference on a neutral field over an average number of possessions), upsets were recorded in 11.3 percent of such games.

As it turns out, Iowa was on the wrong end of more retroactive FEI upset losses than any other team in the country from 2003-2008. Not only that, but the Hawkeyes haven’t really countered those setbacks with many upset wins of their own.

Teams Most Victimized by Upsets According to FEI, 2003-2008
Team 2003-2008
FBS Games
Upset
Losses
Upset Losses
>0.1 Delta FEI
Upset
Wins
Upset Wins
>0.1 Delta FEI
Iowa 72 16 6 3 2
UAB 68 16 4 8 1
New Mexico 71 16 1 9 2
Pittsburgh 69 15 4 7 2
Virginia Tech 75 14 4 4 0
Kent State 65 14 2 7 1
Boston College 74 14 6 8 0
UCLA 75 14 3 13 5

Other observations: UCLA rates as the most schizophrenic team in the country during the last few years. More than a third of its games played between 2003 and 2008 involved an upset of some kind -- 13 went in favor of the Bruins and 14 went against. Also, Boston College and Virginia Tech have posted several significant upset losses, but they haven’t knocked off any teams rated significantly stronger than them.

Of course, this data may not provide evidence of upsets as much as it may reveal FEI biases. The Eagles and Hokies would not appear on this list at all if the system didn’t rate them so high in the first place. FEI certainly appears to like the ACC more than most, and that may be due in part to the relative weight given to certain upsets over others. Generally, close-game upsets aren’t heavily punished by the FEI formula, and ACC teams often play tighter games than other leagues. Combined with a good out-of-conference track record (the ACC was strong, though unheralded, by this measure last year), and we can start to understand where those conference trends in the ratings are coming from.

On the flip side, the team that has posted the most upset victories since 2003 is Toledo. They have 16 upset wins against only five upset losses. Most of the teams at the top of this list are of the BCS conference variety, though: UCLA (13), Texas A&M (12), Indiana (12), and yes, Northwestern (12) appear among the top 10.

As for Iowa, they haven’t fallen out of the Big Ten title picture officially yet, and their BCS ranking is still strong. But without Stanzi heading into a likely conference-deciding showdown with Ohio State, Vegas isn’t buying it. Ohio State is a 17-point favorite this weekend, equivalent to an 86 percent likelihood of victory according to Vegas line trends. Not very promising, but there’s always a chance.

Week 10 FEI Top 25

The principles of the Fremeau Efficiency Index (FEI) can be found here. Like DVOA, FEI rewards playing well against good teams, win or lose, and punishes losing to poor teams more harshly than it rewards defeating poor teams. Unlike DVOA, it is drive-based, not play-by-play based, and it is specifically engineered to measure the college game.

FEI is the opponent-adjusted value of Game Efficiency (GE), a measurement of the success rate of a team scoring and preventing opponent scoring throughout the non-garbage-time possessions of a game. Like DVOA, it represents a team's efficiency value over average. Strength of Schedule (SOS) is calculated as the likelihood that an elite team would win every game on the given team's schedule to date. SOS listed here does not include future games scheduled.

Only games between FBS teams are considered in the FEI calculations. Preseason projections are not a factor in the current calculations. Current FEI ratings are a function of results of games played through Nov. 8.

FEI ratings for all 120 FBS teams are now listed in the stats page section of FootballOutsiders.com. Click here for current ratings; the pull-down menu in the stats section directs you to 2007 and 2008 ratings.

Rank Team FBS
W-L
FEI Last
Wk
GE GE
Rk
SOS SOS
Rk
OE OE
Rk
Off
FEI
OFEI
Rk
DE DE
Rk
Def
FEI
DFEI
Rk
FPA FPA
Rk
1 Alabama 9-0 .272 1 .283 5 .257 16 .116 40 .368 13 -.685 1 -.601 4 .537 22
2 TCU 8-0 .257 4 .335 3 .678 100 .290 22 .297 22 -.662 5 -.475 12 .553 11
3 Florida 8-0 .257 2 .280 6 .530 76 .167 33 .328 18 -.639 6 -.585 7 .541 18
4 Cincinnati 8-0 .242 7 .320 4 .622 90 1.113 1 .589 4 -.209 31 -.223 32 .495 67
5 Texas 9-0 .234 5 .375 1 .447 60 .408 14 .352 14 -.681 2 -.454 14 .574 5
6 Georgia Tech 8-1 .233 8 .106 31 .134 4 .483 10 .744 1 .235 94 -.084 48 .504 56
7 Virginia Tech 6-3 .230 10 .149 18 .085 1 .096 47 .408 10 -.378 21 -.422 18 .555 10
8 Miami 6-2 .228 11 .087 36 .129 2 .209 29 .443 8 -.094 42 -.435 16 .506 50
9 Clemson 5-3 .227 15 .124 28 .130 3 -.202 85 .138 34 -.420 18 -.632 3 .553 12
10 Oregon 7-2 .226 3 .210 9 .348 35 .397 15 .448 7 -.422 17 -.394 19 .514 39
11 Iowa 8-1 .211 6 .135 22 .433 56 -.078 68 .107 41 -.550 9 -.683 2 .511 45
12 Ohio State 8-2 .202 20 .238 7 .319 28 .016 57 .104 43 -.667 4 -.593 6 .575 4
Rank Team FBS
W-L
FEI Last
Wk
GE GE
Rk
SOS SOS
Rk
OE OE
Rk
Off
FEI
OFEI
Rk
DE DE
Rk
Def
FEI
DFEI
Rk
FPA FPA
Rk
13 Boise State 8-0 .201 9 .370 2 .534 77 .372 16 .140 33 -.574 8 -.449 15 .578 3
14 USC 7-2 .195 14 .117 30 .350 36 .171 32 .306 20 -.326 23 -.351 21 .505 51
15 Arizona 5-2 .192 16 .138 21 .360 40 .247 26 .215 28 -.181 34 -.265 27 .539 20
16 Pittsburgh 7-1 .190 12 .232 8 .593 85 .455 12 .334 17 -.103 39 -.094 46 .557 8
17 Oklahoma 4-4 .179 13 .162 15 .324 30 .138 36 .126 35 -.531 11 -.553 8 .491 70
18 Oregon State 5-3 .172 23 .070 37 .187 10 .259 25 .308 19 .189 86 .011 58 .536 24
19 Stanford 6-3 .156 25 .133 26 .289 22 .503 9 .636 3 .308 98 .316 101 .557 9
20 LSU 7-2 .156 19 .161 16 .259 17 -.053 63 .050 55 -.498 13 -.432 17 .547 15
21 North Carolina 4-3 .154 26 .002 55 .308 26 -.235 94 -.107 73 -.396 19 -.596 5 .439 113
22 Penn State 7-2 .152 17 .204 10 .332 32 .216 27 .216 27 -.668 3 -.485 11 .499 64
23 Florida State 3-5 .148 18 .008 53 .167 6 .325 18 .563 6 .583 115 .242 94 .537 21
24 Boston College 5-3 .147 21 .053 42 .250 15 .019 55 .062 53 -.428 16 -.742 1 .482 83
25 Texas Tech 5-3 .144 22 .181 13 .543 80 .475 11 .410 9 -.331 22 -.160 37 .502 59

Posted by: Brian Fremeau on 11 Nov 2009

5 comments, Last at 13 Nov 2009, 12:29am by Brian Fremeau

Comments

1
by Currently Anonymous (not verified) :: Wed, 11/11/2009 - 1:34pm

Looks like the ratings for all 120 teams haven't been updated yet..

2
by Brian Fremeau :: Wed, 11/11/2009 - 2:04pm

Fixed, thanks.

3
by Jetspete :: Wed, 11/11/2009 - 5:25pm

four in the top ten and seven in the top 25 for the ACC, which had multiple teams lose to I-aa opponents?

4
by xLittleP :: Wed, 11/11/2009 - 6:26pm

We also have seven of the top ten schedules.

VPI (1) has played Alabama, Nebraska (and ECU for what it's worth)
Miami (2) has played Oklahoma
Clemson (3) has played TCU
Georgia Tech (4) has played Mississippi State and Vanderbilt (disclosure: my team)
Florida State (6) has played BYU and South Florida
Virginia (7) has played TCU and Indiana (and yes, William and Mary)
Wake Forest (9) has played Baylor, Stanford, and Navy

Not to mention that they've all played each other. Brian, does the SOS take into account opponents' opponents? Given the teams included in the list above, I'm not sure that it would make a difference if it were (or weren't). Does it include games against FCS opponents?

Another good question is, Does it include future opponents? I doubt that it would, but it should be noted that three of the teams above have regular season finales with the SEC team in their state (Florida, Georgia, and South Carolina).

5
by Brian Fremeau :: Fri, 11/13/2009 - 12:29am

Doesn't take into account opponent's opponents directly, no, and it does not include FCS games or future opponents. It is calculated as a function of the strength (FEI rating) of the individual opponents themselves (which are themselves a function of their performances against their opponents).