Writers of Pro Football Prospectus 2008

printable version

Fremeau Efficiency Ratings

College football power ratings and analysis

Need Help with KUBIAK or Premium?

blog advertising is good for you


Most Recent FO Features

ManningEli04.jpg

» Scramble for the Ball: Super Bowl XLVI Recap

Mike and Tom play nice for once and highlight a few commercials that made them smile. Plus: prop bet results, the FO Staff Playoff League, and the results of our first ever Playoff Fantasy Challenge!

19 Nov 2008

FEI Week 12 Ratings

by Brian Fremeau

The Fremeau Efficiency Index principles and methodology can be found here. Like DVOA, FEI rewards playing well against good teams, win or lose, and punishes losing to poor teams more harshly than it rewards defeating poor teams. Unlike DVOA, it is drive-based, not play-by-play based, and it is specifically engineered to measure the college game.

FEI is the opponent-adjusted value of Game Efficiency, a measurement of the success rate of a team scoring and preventing opponent scoring throughout the non-garbage-time possessions of a game. Like DVOA, it represents a team's efficiency value over average. Strength of Schedule is calculated from a privileged perspective (explained here) and represents the likelihood that an Elite team (top 5) would post an undefeated record against the given team's opponents to date.

The following ratings are calculated based on data from all FBS games played through Saturday, November 15. Only games between FBS teams are considered.

Rank Team Record FEI Last Week vs. Top 10 vs. Top 40 GE GE Rank SOS SOS Rank
1 Florida 9-1 0.309 1 0-0 3-1 0.460 1 0.413 59
2 North Carolina 6-3 0.290 3 1-0 6-3 0.153 17 0.171 10
3 Penn State 9-1 0.284 2 1-0 4-1 0.331 8 0.368 51
4 Texas 10-1 0.266 4 1-1 3-1 0.364 4 0.258 34
5 USC 9-1 0.266 6 1-0 3-1 0.397 2 0.442 64
6 Texas Tech 8-0 0.258 5 1-0 2-0 0.379 3 0.456 67
7 Oklahoma 8-1 0.217 8 0-1 2-1 0.337 6 0.392 55
8 Alabama 11-0 0.212 7 0-0 3-0 0.294 9 0.486 76
9 Ohio State 8-2 0.210 10 0-2 3-2 0.145 18 0.170 9
10 Georgia Tech 5-3 0.197 11 0-1 3-3 0.069 33 0.149 6
11 Boston College 6-3 0.193 19 0-2 3-3 0.077 30 0.145 5
12 Virginia Tech 5-4 0.189 12 2-0 3-4 0.034 45 0.125 4
Rank Team Record FEI Last Week vs. Top 10 vs. Top 40
GE GE Rank SOS SOS Rank
13 Florida State 5-3 0.188 9 0-1 3-3 0.055 37 0.210 21
14 Ball State 9-0 0.169 13 0-0 0-0 0.338 5 0.785 117
15 Iowa 6-4 0.159 14 1-0 2-3 0.126 19 0.246 33
16 Mississippi 5-4 0.159 17 1-1 1-3 0.110 21 0.199 15
17 Pittsburgh 7-2 0.158 15 0-0 2-1 0.099 23 0.479 73
18 Missouri 8-2 0.157 16 0-1 1-2 0.242 11 0.396 56
19 Miami 6-3 0.154 20 0-2 3-3 0.036 42 0.084 1
20 Wake Forest 6-4 0.138 18 0-0 4-2 0.018 53 0.277 39
21 Michigan State 9-2 0.134 22 0-1 3-2 0.104 22 0.345 45
22 Connecticut 6-3 0.133 21 0-1 2-3 0.073 31 0.275 38
23 Utah 10-0 0.129 25 0-0 2-0 0.262 10 0.680 105
24 Boise State 9-0 0.127 24 0-0 0-0 0.337 7 0.762 114
25 Georgia 8-2 0.121 23 0-2 1-2 0.049 41 0.216 26

Adjusted Offensive Efficiency and Adjusted Defensive Efficiency are the opponent-adjusted values of Offensive Efficiency and Defensive Efficiency, explained here. Like FEI, the multiple-order adjustments are weighted according to both the strength of the opponent and the relative significance of the result; efficiency against a team's best competition faced is given more relevance weight. AOE and ADE represent a team's value over/under average. Positive AOE and negative ADE are the most valuable.

Rank Team Record AOE AOE Rank ADE ADE Rank OE OE Rank DE DE Rank
1 Florida 9-1 0.433 5 -0.445 6 0.550 10 -0.661 1
2 North Carolina 6-3 0.223 24 -0.531 1 -0.082 62 -0.348 15
3 Penn State 9-1 0.512 3 -0.409 8 0.444 13 -0.488 6
4 Texas 10-1 0.506 4 -0.382 12 0.920 2 -0.197 29
5 USC 9-1 0.400 6 -0.446 5 0.478 12 -0.612 2
6 Texas Tech 8-0 0.565 2 -0.349 16 1.134 1 0.001 67
7 Oklahoma 8-1 0.568 1 -0.196 34 0.899 3 -0.133 44
8 Alabama 11-0 0.254 23 -0.397 11 0.115 38 -0.600 3
9 Ohio State 8-2 0.216 25 -0.453 4 0.016 51 -0.412 8
10 Georgia Tech 5-3 0.343 12 -0.289 20 -0.031 58 -0.283 20
11 Boston College 6-3 0.194 28 -0.484 2 -0.076 60 -0.460 7
12 Virginia Tech 5-4 0.027 61 -0.381 13 -0.213 85 -0.166 34
Rank Team Record AOE AOE Rank ADE ADE Rank
OE OE Rank DE DE Rank
13 Florida State 5-3 0.284 18 -0.252 23 -0.025 55 -0.130 46
14 Ball State 9-0 0.334 14 -0.252 22 0.788 5 -0.308 18
15 Iowa 6-4 0.109 45 -0.464 3 -0.027 56 -0.398 9
16 Mississippi 5-4 0.272 20 -0.231 28 0.115 37 -0.224 27
17 Pittsburgh 7-2 0.358 11 -0.149 42 0.267 22 -0.035 58
18 Missouri 8-2 0.365 10 -0.088 50 0.741 6 0.068 76
19 Miami 6-3 0.099 50 -0.061 53 -0.206 82 -0.099 50
20 Wake Forest 6-4 -0.136 80 -0.404 9 -0.327 101 -0.178 31
21 Michigan State 9-2 0.203 27 -0.204 31 -0.107 64 -0.238 25
22 Connecticut 6-3 0.107 47 -0.363 14 -0.028 57 -0.357 13
23 Utah 10-0 0.129 41 -0.201 33 0.191 29 -0.362 11
24 Boise State 9-0 0.144 37 -0.216 29 0.442 14 -0.533 5
25 Georgia 8-2 0.385 8 -0.061 54 0.157 35 -0.090 52

The Week 12 FEI Ratings for all 120 FBS teams can be found here. Expanded FEI Ratings data can be found here.

It started as an early-season eyebrow-raising fling with Virginia Tech that hardly seemed worth fretting over. But as the season progressed, FEI started hanging out more and more with the Atlantic Coast Conference, two-timing Georgia Tech and Wake Forest, texting North Carolina while canoodling Duke. It was clearly not healthy behavior, but what could be done? Florida State, Miami and Boston College? Oh please, not Clemson! With just a couple of weeks left in the college football regular season, FEI appears to be one or two drinks away from a trip to the tattoo parlor. It's time for an intervention.

The Tarheels lost their third game of the season and moved up from No. 3 to No. 2 in the FEI rankings. Why? Start by adding up the total margin of defeat in North Carolina's three losses: eight points. Then consider that all nine UNC opponents have overall records at or above .500 and all nine appear in this week's FEI top 40. Impressed? I didn't think so.

There must be a bug in the system, and it has to have something to do with North Carolina and the opponent adjustments, right?. As we saw three weeks ago, simply eliminating the conference from the books might be satisfactory, but we would certainly rather isolate and fix the problem than ignore it. So let's ask the following: Why does FEI think North Carolina is better than, say, undefeated Alabama?


Efficiency Breakdown: Alabama and North Carolina
Stat Alabama North Carolina
Offensive Efficiency 0.115 (38) -0.082 (62)
Defensive Efficiency -0.600 (3) -0.348 (15)
FIeld Position Efficiency 0.181 (10) 0.309 (3)
STSE -0.010 (66) 0.001 (50)
Opponent STSE -0.032 (19) -0.003 (63)
Game Efficiency 0.294 (9) 0.153 (17)
Adjusted Offensive Efficiency 0.254 (23) 0.223 (24)
Adjusted Defensive Efficiency -0.397 (11) -0.531 (1)
FEI 0.212 (8) 0.290 (3)

STSE and Field Position Efficiency were introduced and explained last week. Overall, the unadjusted efficiency data favors Alabama as expected, but UNC isn't too far behind -- quite respectable for a three-loss team, in fact. Both teams hang their hat on defense and controlling field position, and both have blown open games on defensive and special teams return scores (calculated as part of FPE). But when we make the FEI opponent adjustments, the blemished ACC team jumps significantly ahead of the unblemished SEC team. That can't be right, can it?

Defensively, neither Alabama nor North Carolina have allowed more than 24 non-garbage points in a game this season, though neither team has been tested by a high-powered offense yet either. Alabama has held seven opponents to ten points or less, but five of those seven teams rate in the bottom third nationally in offensive efficiency. North Carolina's opponents aren't lighting up scoreboards either, but they have been eight percent more efficient offensively than Alabama's opponents. On defense, North Carolina's opponents are 11 percent stronger than Alabama's, and the Tarheels were most productive against the best of the bunch, Boston College. These numbers are useful when considering the two teams and two conferences, but they are hardly conclusive. What other factors are in play?

I mentioned in a comment a few weeks back that the schizophrenic nature of the conference might be vexing the formula. One measure of that chaos is "ranking violation," a retrodictive measure of the percentage of games played this season in which a team has lost to an opponent currently rated below it. ACC teams have accounted for 18 such occurrences in 106 total games played (17 percent), second only to Conference USA. Another measure is variance in adjusted game efficiency; as expected, Maryland (defeated California, lost to Middle Tennessee) and Virginia (crushed by USC, Duke and Connecticut, beat North Carolina, Georgia Tech and Maryland) lead the way nationally in that category.

An even bigger factor might be FEI's treatment of close games and blowouts. Average non-garbage margin of victory in ACC conference games this season is 9.6 points, a full touchdown less than the average non-garbage margin of victory in the other conferences. Virtually no one is getting blown out in the ACC, and as Oklahoma State, Ohio State, Missouri, LSU, Georgia, and more can attest, FEI will punish blowout losses swiftly. There is logic behind this strategy, but apparently the flip side of that coin means a tightly contested league will receive a broader boost than one that keeps beating down its opposition. An off-season FEI analysis will need to take a closer look at this.

Lastly, I mentioned this three weeks ago but it bears repeating: The ACC is leading the way in inter-conference play in 2008 whether we like it or not. They haven't slain any giants, but Miami did hold the Florida Gators to their lowest offensive output of the year, better than any SEC foe. But the real tests for the league are still ahead. If FEI is wrong about the ACC like we all believe it must be, then South Carolina, Georgia, Vanderbilt, and Florida should have their way with Clemson, Georgia Tech, Wake Forest, and Florida State. And if those four hold their own, well, we're all getting tattoos. Who's in?

Posted by: Brian Fremeau on 19 Nov 2008

9 comments, Last at 22 Nov 2008, 11:43pm by David227

Comments

1
by Travis :: Wed, 11/19/2008 - 11:36am

The ACC is leading the way in inter-conference play in 2008 whether we like it or not.

If "leading the way" equals "best record in inter-BCS conference games", true.

FWIW, none of the BCS computers have the ACC as the top conference. The Colley Rankings have the ACC a distant third behind the Big 12 and the SEC, while the other 5 have the ACC anywhere from a close second (Sagarin, Massey, Anderson & Hester) to fifth (Wolfe).

2
by Statistical_Anon (not verified) :: Wed, 11/19/2008 - 12:55pm

You don't reveal enough of the details of the formulas to make it clear, but some speculation..

You're treating the drive as the unit of observation. Teams that tend to be more successful on offensive drives, and prevent successful drives on defense, rate highly, and are expected to win more in the future. Your measure basically boils down to the average rate of success. This means that a team that has a 50 point win and a 10 point loss against constant competition will rate similarly to a team with two 20 point wins against the same constant competition (if I follow correctly). But this cannot be correct. Certainly volatility of success matters; and you have the drive-level information. So suppose that, instead of using just average information, you use information on the reliability (ie, precision, variance, whatever metric you prefer) to help forecast future results. For instance, using the Pythagorean Wins formula would do the job: comparing a +20, no variation team to a +20, high variation team in Pythagorean Wins will tell you they are different, and how. This should punish conferences like the ACC, where there is a set of relatively elite teams that are +10 on average, but high variation.

3
by dbostedo :: Wed, 11/19/2008 - 3:02pm

"For instance, using the Pythagorean Wins formula would do the job: comparing a +20, no variation team to a +20, high variation team in Pythagorean Wins will tell you they are different, and how."

Actually, doesn't the Pythag. ignore variation and just go by absolute points scored and allowed? In that case, the formula doesn't take into account whether you won by 20 twice, or by 50 once. For example, if the two game scores were 60 - 0, and 0 - 20, your PF and PA are 60 and 20. If you won both games 30 - 10, your PF and PA are still 60 and 20. Your pythag. is the same either way.

4
by Joseph :: Wed, 11/19/2008 - 3:58pm

Brian, maybe a baseline for each league based on their out-of-conference games might help. First, most teams play their OOC games early in the year. Second, if you do a baseline for the CONFERENCE, this should balance out those who schedule cupcakes and those who try to play other BCS teams OOC. Another tip--as time goes on, you could base your baseline on the last 4 years to increase sample size--maybe even weighting the earlier years less to reflect graduating/exiting talent.
Another idea--point reduction (in some way) for losses. Although this may make your numbers look like the AP poll, there needs to be some way to weigh down the teams that appear to be rated too high (ACC teams, Ole Miss) based on their overall record. My guess is that FEI doesn't know that those teams had just enough bad drives in certain games to lose them. Example: Florida--just enough bad luck in one game to hurt, although not kill, their championship aspirations. Otherwise, we are giddy with the possibility of the SEC championship game being a national semifinal with the OU-TTU game this Sat. being the other. (Which, although Texas (and maybe USC) fans would be SCREAMING, would match up the four best teams in the land (providing UF would have beaten Ole Miss).
Texas fans, here's your caveat. If you replace OU, that game has already been played--TTU beat you, and (theoretically) would await the winner of the SEC CG.

5
by Statistic_Anon (not verified) :: Wed, 11/19/2008 - 4:30pm

D'oh, you're right about Pyth. Wins. Still, I think you could extrapolate something pretty painlessly. What you want to capture is that teams which are consistently pretty good (ie, 2002 OSU) are different from teams that are inconsistently alternating between very good and mediocre (ie, Clemson before this year). Without seeing all the formulas, it seems to me that the problem here is that the ACC is a bunch of mediocre inconsistent teams which are being treated as just their average outcome.

And if that doesn't fail, I'm sure we can work out more theories..

6
by Love is like a bottle of gin (not verified) :: Wed, 11/19/2008 - 5:35pm

Imagine a world where you have teams A, B in conference 1, Q, R in conference 2, and Y, Z in conference 3. Imagine also you have football clairvoyance in this world and know that AB are "5" strong +/-3, teams QR are "6" strong +/-3, and teams YZ are "7" strong +/-3.

Now lets pretend that the teams only play intra-conference except for one game each. In these games:

A beats Q
B beats Y
Z beats R

Nothing surprising has happened. I think the ACC thing has entirely to do with the relative lack on connectivity. I would predict the ACC will sink as more connectivity is established, but even still in such systems it is very easy for models to prop up "undeserving" conferences based on a few outlier results because it has no other information to go on (and indeed shouldn't).

7
by jb (not verified) :: Sat, 11/22/2008 - 5:49am

Your strength of schedule is WAY, WAY too strong in this.. It needs to be weakened!!

This is the source of most of your problems. I found going halfway between the FEI and GE is producing FAR better results.

8
by jb (not verified) :: Sat, 11/22/2008 - 6:05am

Going halfway between FEI and GE produces this top 25..

#1 Florida .385
#2 USC .331
#3 Texas Tech .319
#4 Texas .315
#5 Penn State .307
#6 Oklahoma .277
#7 Ball State .254
#8 Alabama .253
#9 Boise State .232
#10 North Carolina .222
#11 Missouri .200
#12 Utah .196
#13 Ohio State .178
#14 TCU .163
#15 Iowa .142
#16 Oklahoma State .140
#17 Mississippi .135
#18 Boston College .134
#19 Georgia Tech .133
#20 Pittsburgh .129
#21 BYU .127
#22 Florida State .122
#23 Arizona .119
#24 Michigan State .118
#25 Virginia Tech .112

Not saying it necessarily needs to be weakened this much, but teams definently getting overpunished/overrewarded based on SoS.

9
by David227 (not verified) :: Sat, 11/22/2008 - 11:43pm

NC State 41 NC 10. I would say that the system needs alot of work. You really don't need a computer to know that NC was not the #2 team in the country and should never have moved up in the face of a loss to Maryland. Yet that is what happened in the FEI.

You can say what you want about close losses, but they are still losses...and perhaps the ACC is really a dreadful conference this year and none of the teams really deserve to be ranked as high as they are.

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

More information about formatting options

CAPTCHA
To skip this, please log in.