Writers of Pro Football Prospectus 2008

Most Recent FO Features

LuckAnd12.jpg

» Scramble Over/Unders: The Souths

Mike and Tom's tour of the league gets musical! Listen to the strangely up-tempo Blinded by the Luck, the haunting melancholy of The Schiano Man, and the endless march of the Saints.

24 Oct 2007

Fremeau Efficiency Index: Week 8

by Brian Fremeau

The Fremeau Efficiency Index (FEI) principles and methodology can be found here. Like DVOA, FEI rewards playing well against good teams, win or lose, and punishes losing to poor teams more harshly than it rewards defeating poor teams. Unlike DVOA, it is drive-based, not play-by-play based, and it is specifically engineered to measure the college game.

FEI is the opponent-adjusted value of Game Efficiency, a measurement of the success rate of a team scoring and preventing opponent scoring throughout the non-garbage-time possessions of a game. Like DVOA, it represents a team's efficiency value over average.

Only games between FBS (Division 1-A) teams are considered. The Week 8 Ratings represent the results of games played through Sunday, October 21, 2007.


Rank Team W-L FEI Last Wk GE GE Rank
1 LSU (7-1) 0.304 1 0.260 8
2 South Florida (5-1) 0.280 2 0.170 17
3 Florida (4-2) 0.237 7 0.139 23
4 Oregon (6-1) 0.236 6 0.274 7
5 Auburn (5-3) 0.235 3 0.117 26
6 West Virginia (6-1) 0.222 8 0.329 3
7 Oklahoma (7-1) 0.209 5 0.287 6
8 Ohio State (7-0) 0.192 13 0.364 1
9 Boston College (7-0) 0.188 16 0.250 10
10 Arizona State (7-0) 0.185 12 0.289 5
11 Kentucky (5-2) 0.183 4 0.073 37
12 Kansas (6-0) 0.180 11 0.338 2
Rank Team W-L FEI Last Wk GE GE Rank
13 Missouri (5-1) 0.177 17 0.209 11
14 Rutgers (5-1) 0.177 32 0.209 11
15 California (5-2) 0.173 9 0.110 27
16 Cincinnati (5-2) 0.164 15 0.179 16
17 Virginia (7-1) 0.161 25 0.050 45
18 Alabama (5-2) 0.160 29 0.085 33
19 Kansas State (3-3) 0.155 14 0.085 34
20 Georgia Tech (4-3) 0.149 22 0.090 31
21 Connecticut (5-1) 0.145 33 0.158 19
22 Arkansas (3-3) 0.139 24 0.156 20
23 South Carolina (5-2) 0.138 10 0.040 48
24 Florida State (4-3) 0.138 20 0.061 42
25 BYU (4-2) 0.138 18 0.136 25

Click here for rankings of all 119 teams.

Wait a second. Didn't South Florida lose last Thursday? Didn't the national media close the book on their Cinderella season? Didn't we all agree to relegate the Bulls to the bottom of the one-loss heap?

Not FEI, and unlike the polls, the reason behind FEI's rating of USF has nothing to do with the name on the uniform or the history of the program.

LSU and South Florida are currently the only programs in the country with multiple wins over top-10 FEI teams, and each have dropped one game to a top-15 FEI team. But while LSU maintains a comfortable position at No. 3 in the BCS, South Florida tumbled to No. 10 in the BCS this week and now has virtually no shot at the championship game. USF finds itself behind, among others, one-loss Oklahoma and one-loss West Virginia in the BCS, two good teams, but neither with a better resume than the Bulls. The Sooners lost to a now four-loss Colorado team, and the Mountaineers lost to none other than South Florida themselves. Poll anchoring and loss-consequences are one thing, but can WVU over USF on a ballot truly be justified?

The one top-10 team bested both by LSU and South Florida is Auburn, a team that hangs in at No. 5 despite another loss over the weekend. Three-loss teams don't typically garner this much respect, but the Tigers have faced a murderous schedule to date and have fared pretty well. Auburn is one of four teams with multiple victories over FEI top-20 teams, they have faced more top-10 (three) and top-20 (four) teams than any other team in the FEI top-25, and they lost to both No. 1 and No. 2 on the final play of the game.

Here's a look at this week's Underrated and Overrated teams in the polls.

Underrated

Rutgers (4-2; NR BCS, No. 25 AP, No. 14 FEI)
By knocking off South Florida, Rutgers earned the biggest FEI bump from last week, and put itself in position to make another run at the Big East title that eluded them last season. Unlike last season, though, Rutgers is doing more with less, at least as far as field position is concerned. They have faced the largest drop-off in Field Position Advantage among FBS teams from last season, plummeting from an eight yard per possession advantage in 2006 to a nearly seven yard per possession disadvantage in 2007. Their excellent running game has barely missed a step, however, and spearheaded the victory last Thursday despite facing a six-yard per possession field position deficit for the game to USF. Up next are contests with No. 6 West Virginia and No. 21 Connecticut, two winnable games regardless of FPA. If Rutgers can shift the field in their favor, watch out.

Georgia Tech (4-3; NR BCS, NR AP, No. 20 FEI)
The Yellow Jackets haven't notched a true marquee victory yet this season, so why the FEI love? Georgia Tech has faced as many top-40 opponents as any other team, winning two of five, and as good teams are supposed to do, they have manhandled the weak competition they have faced. The schedule sets up nicely for a strong finish, with an off weekend to prepare for a date next Thursday with Virginia Tech, followed by a relatively light ACC slate until the annual rivalry game with Georgia.

Overrated

Virginia Tech (5-1; No. 8 BCS, No. 8 AP, No. 33 FEI)
Virginia Tech's best win over No. 31 Clemson also happens to be their only victory over a team with a winning record. Against LSU in Week 2, the Hokies were as uncompetitive as any of the Tiger opponents this season. Several top teams have seemingly earned their lofty poll ranking by default this season, perhaps none more so than Virginia Tech. A Thursday night showdown with undefeated Boston College (another team with a little more to prove) gives the Hokies a second opportunity in the spotlight to assert themselves and actually earn poll recognition.

USC (6-1; No. 12 BCS, No. 9 AP, No. 34 FEI)
In the eyes of poll voters, the Trojans virtually cashed in a get-out-of-jail-free card with their 38-0 thrashing of Notre Dame on Saturday. Why hasn't FEI yet forgiven the "historic" loss to Stanford on October 6? Well, not only has USC not yet defeated a top-40 opponent, they haven't yet played one. In fact, four Southern Cal victories have come against teams ranked among the bottom 40 in FEI, and facing their toughest competition to date, USC escaped with victories over No. 51 Washington and No. 61 Arizona. With five FEI top-40 match-ups coming up, including No. 4 Oregon this week, plenty of opportunities remain for USC to improve their resume. Until they do, the jury is out.

FBS vs FCS

On Saturday afternoon, North Dakota State defeated Minnesota 27-21 in the Metrodome, the seventh loss in an entirely forgettable season for the Golden Gophers. Yet Minnesota's FEI ranking climbed two spots this week and their record remains listed at 1-6. What gives?

As mentioned in the lead-in to each week's ratings, FEI considers only games between FBS (formerly known as Division-1A) opponents. All other FBS games, both wins and losses to FCS (formerly Division-1AA) teams, are ignored. None of the 60 victories by FBS over FCS teams, nor the eight losses, impact the weekly FEI ratings in any way. In any other year, ignoring those 68 games would go largely unnoticed. But one result seems to make this year different. See if you can pick it out:

  • Minnesota lost to North Dakota State 21-27
  • Central Michigan lost to North Dakota State 14-44
  • Marshall lost to New Hampshire 35-48
  • Louisiana Lafayette lost to McNeese State 17-38
  • Northern Illinois lost to Southern Illinois 31-34
  • Iowa State lost to Northern Iowa 13-24
  • Rice lost to Nicholls State 14-16
  • Michigan lost to Appalachian State 32-34

I haven't received one inquiry this season about how FEI is going to handle Louisiana Lafayette's loss. I likewise haven't received one inquiry this season about how to deal with Ohio State's Week 1 victory over Youngstown State. Why does that epic Michigan game deserve special treatment?

FEI is built on the principle that wins over poor teams are relatively unimportant while losses, particularly to poor teams, are given significant weight. By this standard, a fair evaluation of Michigan ought to include the Appalachian State data point in some fashion. But ignoring that outcome has less to do with a fair evaluation of Michigan than it does with a fair evaluation of Appalachian State. With only one game against FBS competition, can we properly determine how good or bad Appalachian State truly is? Fellow FCS team North Dakota State has played (and won) twice as many games against FBS competition -- can we confidently assign a strength value to them, either?

The challenge of any college football ranking system to properly evaluate 119 teams is difficult enough, particularly with most teams within that large set playing only 11 or 12 games total, seven or more against a small subset of conference teams. And yet, the "connectivity" of each FBS team to every other FBS team dwarfs that of FCS teams. At regular season's end, the average FBS team will have played 11 FBS opponents, and have at least one common opponent with 53 other teams. But Appalachian State will share only one common opponent (Michigan) with each of only 11 other FBS teams. By the second generation of connectivity, an average FBS team will share more than 25 common opponent-opponents with every other FBS team. Appalachian State, meanwhile, will share an average of only two common opponent-opponents with less than half of the FBS teams by season's end.

By this measure of connectivity, the individual game data point for each FBS vs. FCS game is about ten times less valuable than the individual game data point in FBS vs. FBS games. Appalachian State's victory over Michigan may still be the headline of the year, but is it statistically significant in the context of Michigan's other 11 games this season?

While some computer ranking systems have employed different solutions to this problem, I refuse to assign an arbitrary value to Appalachian State, particularly one that would carry anywhere near the same weight as the other opponents on Michigan's schedule. Though it is certainly possible that the Wolverines could "unjustly" disrupt the national championship picture by ignoring this one result, I believe that a fair evaluation of Michigan can be made from their other 11 games, just as a fair evaluation of their rival Ohio State can be made by excluding its victory over Youngstown State.

Posted by: Brian Fremeau on 24 Oct 2007

27 comments, Last at 26 Oct 2007, 7:10pm by zlionsfan

Comments

1
by billsfan (not verified) :: Wed, 10/24/2007 - 4:05pm

Christian Cash Advance Loans? Almost makes me long for the Catholic Match Girl.

2
by billsfan (not verified) :: Wed, 10/24/2007 - 4:11pm

Seriously, though, BC-VT should be a good clash of the overrateds. I've yet to be impressed by a single performance from BC this season. They've gotten quite good at playing down to their opponents, especially in the Army and UMass games. Guess we'll find out how overrated they are on Thurs.

3
by pawnking (not verified) :: Wed, 10/24/2007 - 4:12pm

Interesting project you have here. Very good points on USF and USC.

4
by Antigonos (not verified) :: Wed, 10/24/2007 - 4:14pm

I'd imagine Rutgers drop in average field position is directly related to having Jeremy Ito take over kick off and punting duties. He is just terrible at both of them...especially punting.

5
by Dennis (not verified) :: Wed, 10/24/2007 - 4:44pm

Nice work. I totally agree about USF and USC. If Oklahoma and USF switched schedules and results, Ok would be ranked higher. USF was dropped solely because of their name.

Likewise, if Rutgers played the USC schedule, they wouldn't be ranked.

6
by lionsbob (not verified) :: Wed, 10/24/2007 - 4:53pm

Rutgers did play the USC schedule but had USF last week and thats about it. I don't mean to take away from Norfolk State.

7
by Brian (not verified) :: Wed, 10/24/2007 - 5:43pm

billsfan, imagine the effect if BC actually wins tonight - and sails through the rest of the year with the possibility of playing for a national title.
That might be the straw that broke the BCS's back and finally creates a legitimate playoff system.

8
by mmm... sacrilicious (not verified) :: Wed, 10/24/2007 - 5:47pm

Given this system, I'm actually wondering how Michigan is rated so low. Trying to avoid being a homer, they have beaten the #30 and 32 teams, the latter on the road, and their "only" loss is to the #4 team.

I'm not saying they're top 10 material, but I would have thought they definitely have a better resume than some of the teams above them, for example:

BYU: Best win #50, losses #29, 73
Cincinnati: Best win #14, losses #39, 52
Connecticut: Best win #39, loss to #17
Arkansas: Best win #64 (!) losses to #5, 11, 18

BYU and Connecticut I simply fail to explain. Cincinnati would be possible if good wins are worth a LOT more than bad losses. And Arkansas would be possible only if playing very well against good opponents but actually beating nobody significant were rewarded.

I realize this is a drive-by-drive metric, which probably explains a lot of these inconsistencies. And I love these types of systems. However, it appears that this system is significantly flawed, unless I'm missing something.

9
by Dwayne (not verified) :: Wed, 10/24/2007 - 6:01pm

This is the first time I have read this column, but his rankings seem to be right on.
One feeling I have always had is that it is hard to keep a team ranked higher than teams that you feel they would lose to. That is my only issue with USF. Whatever else they may have accomplished, and it is fun to see this program grow, I feel they would lose to several of the teams ranked below them.

10
by PHn (not verified) :: Wed, 10/24/2007 - 6:18pm

5:

Likewise, if Rutgers played the USC schedule, they wouldn't be ranked.

Hmm.

Conference schedules are a given, and conferences have up and down years, so conference games are essentially equal, subject to the vagaries of relative conference strength.

How about non-conference games?

USC: Idaho, Nebraska, and Notre Dame.

Rutgers: Buffalo, Navy, Norfolk State, and Army.

While it's true that Nebraska and ND are both having bad to terrible years, they are both historically good programs and those games were scheduled years ago. No one can say that USC went out of its way to schedule sub-par non-conference opponents.

The same cannot be said for Rutgers. None of its non-conference games are against historically strong competition. Having games against the service academies is laudable though hardly a strong test for a ranked team. But Buffalo and Norfolk State? Heh heh heh.

11
by billsfan (not verified) :: Wed, 10/24/2007 - 6:53pm

Brian,

BC's alumni stadium is visible from my office window. I'm enjoying this joke at the BCS's expense for as long as it lasts.

I don't think the BCS system is likely to be replaced by a tournament. The big conferences pull in too much money from the BCS system, and revenue from a playoff would be too equitably distributed. Besides, they've already had controversial non-championships. If BC wins out and wins the BCS championship, they'll be undefeated, which wouldn't be so bad. If they lose to an undefeated Ohio State, that wouldn't be so bad, either. If BC loses, a real team will take their place in the BCS Championship.

12
by Dennis (not verified) :: Wed, 10/24/2007 - 7:02pm

My point about Rutgers/USC is if Rutgers had played the schedule USC has thus far with the same record, they would be unranked, unlike USC. USC is only ranked currently because of their history and their preseason ranking.

And besides USF, Rutgers has played Cincinatti, who is significantly better than anyone USC has played, and Maryland, who is probably as good or better than anyone USC has played.

I'm not saying Rutgers played a weak non-conference schedule, because obviously they have. But so has USC. The fact that Nebraska and ND are "historically strong" is irrelevant to ranking teams this season. If Rutgers played USC's schedule they would barely be in the also receiving votes.

The fact that USC is in the top 10 in both the Harris and Coaches polls shows what a joke the human rankings are because they have done nothing on the field to merit the ranking.

13
by young curmudgeon (not verified) :: Wed, 10/24/2007 - 7:08pm

Your rationale makes some sense, but I still don't see how you can fail to penalize Michigan for its Appalachian State loss and feel comfortable with your system. The idea that you don't reward FBS teams for beating FCS teams, so why penalize them for losing to one seems "logical, but wrong" if you know what I mean. It may offend statistician sensibilities, but perhaps a random or subjective value assignment would be the best way to handle this. I think there are 120 FBS (the Division 1A nomenclature was so much better, BTW) teams. Let's say beating an FCS team would carry the same weight as beating the 110th ranked FCS team, while losing to one would carry the same weight as losing to the 50th ranked team. Yes, it's utterly subjective, but at least you'd be following the college football mantra "every game counts." And, in the case of FBS losing to FCS, that game should count for something (namely, to the detriment of the FBS team.)

This isn't, by the way, specifically directed at Michigan--I tend to root against Ohio State and against Notre Dame, so I end up hoping to hear "Hail to the Victors" in many games.

If this off-the-top-of-my-head suggestion doesn't work, this website attracts a lot of attention from intelligent people and people who are very adept at the use of statistics. I bet one of them could devise a workable scheme.

14
by Dennis (not verified) :: Wed, 10/24/2007 - 7:10pm

Re 9 One feeling I have always had is that it is hard to keep a team ranked higher than teams that you feel they would lose to.

This goes back to whether the rankings are descriptive or predictive. If you want to determine the best teams based on what they have actually done, then the rankings need to be descriptive.

Most voters rank teams predictively - who they think would beat who, which is why human polls are useless for determining the best teams.

15
by Dwayne (not verified) :: Wed, 10/24/2007 - 9:32pm

#14 There has to be some of both in the rankings, predictive and descriptive. Teams that are undefeated are rewarded by rising up the ranks, but you cannot end your season like 1984 with BYU as #1 just because they beat a bunch of cupcakes.

16
by young curmudgeon (not verified) :: Wed, 10/24/2007 - 9:41pm

re 13: intelligent people and people who are very adept at the use of statistics.

Urk! Looking at this in print, I realized that it might read somewhat differently than I intended--there is NO implication that the two categories are mutually exclusive.

17
by blacksuit (not verified) :: Wed, 10/24/2007 - 10:46pm

Re 8:

Arkansas loses on the road to Bama by 3, at home to Auburn by 2, and at home to Kentucky (who scored 3 4th quarter TDs) by 13 points.

I don't think the 20th ranking is a sign of an error of the system at all. It's a team that played really well, but came up a little short against some of the best teams in college football.

Mabye in some parallel dimension, a couple of little things go the other way and Arkansas is sitting at 5-1. Some years you just get bad bounces, and this system doesn't penalize teams for that.

18
by PaulH (not verified) :: Thu, 10/25/2007 - 2:15am

17,

I agree completely about Arkansas. The Hogs were leading Alabama with 15 seconds to go, leading Auburn with under a minute to go, and led for three quarters against Kentucky. I know they don't exactly have any major wins, but they've played really well against three ranked teams.

Moreover, the Hogs have what most consider to be the best player in the country (McFadden), likely the best running game in the country, and one of the SEC's best defenses (currently 4th in scoring defense). At some point, a great rushing attack and a good defense is going to get them an upset win.

Moreover, Marcus Monk is going to return, and that will help if but for only upgrading a super-crappy passing attack to just a crappy passing attack.

The Hogs still have Arkansas, Tennessee, and LSU on the schedule, and they are going to take all three of those right down to the wire, and they'll probably win a game or two.

The LSU game, in particular, will be very interesting. The Hogs have played the Bayou Bengals very close the past two years, and these guys will very much be looking for the upset. They know, and their coach knows, an upset over LSU could be just what it takes to save Nutt's job, and they'll play hard. Those who think that if LSU gets past Alabama that they will easily breeze into the SEC title game are sorely mistaken.

I'm not a Hog fan, but I've seen them play several times this year, and it's fairly easy to see that they are much better than they get credit for being.

19
by James G (not verified) :: Thu, 10/25/2007 - 7:36am

#8 - like DVOA, this system is more than who beat who. It's drive based and schedule adjusted, but your drive efficiency still matters.

Brian - I wonder if you could do an FEI calculation for the NFL and compare its results to DVOA. We could see how drive-based and play-based ranking systems compare.

20
by tlt (not verified) :: Thu, 10/25/2007 - 2:21pm

obviously, any rating that places a 4-3 team from a lackluster conference over a 5-1 team from a premier conference has got its head up its, well, ain't that a snake in the gr-ass?

hello?!! is anyone in there?!! how efficiently do you think a rating scheme honed on the professional game will transfer to the amateur ranks, one set aside by enormous variance in talent, and by youth?

sheesh.

21
by lionsbob (not verified) :: Thu, 10/25/2007 - 3:32pm

#18

How pumped are you for the November 3rd LSU-Alabama game. If Alabama somehow pulls it out-it will be amazing. I think they can do it, but it depends on what Les Miles/Crowton decides to show up....

Though I am looking more forward for next season, we only lose 3 bigtime players in Hall, Castille, and Gilberry and some role/solid players like Mustin, Cadell, Britt, and Brown....but man alive we return what 17-18 starters-plenty with 3 years of starting experience and what looks to be a top 5 recruiting class to pick through as well. I know 2009 was the target year, but I am getting pumped for next season as well.

22
by beargoggles (not verified) :: Thu, 10/25/2007 - 3:53pm

Re: 20

By a 5-1 team from a power conference, I assume you mean USC.

Not a good argument because so far they've played all the cupcakes in their power conference. So as well stated already, they haven't actually accomplished jack.

No doubt they're talented, and with games against all the good teams in the conference still ahead, they have plenty of chances to move up in the system, but as of this moment, their best wins are against horrible Notre Dame and Nebraska teams, with a bunch of squeakers thrown in. Rutgers has accomplished more. And probably Georgia Tech also.

23
by Kal (not verified) :: Thu, 10/25/2007 - 7:58pm

#20: Because just because you've won games does not mean you're a good team; it matters that you won, who you beat and how you beat them.

Ask OSU and Michigan last year about their winning records and how meaningful they were in determining who had a better team. Ask Nebraska and Colorado in 2001. Just because you win every game doesn't mean you're a good team, and it certainly doesn't mean that because you've got a better record than another team, you're automatically a better team.

24
by ChrisFromNJ (not verified) :: Thu, 10/25/2007 - 8:07pm

#22:

No, clearly he was taking about the football powerhouse that is the University of Connecticut. There are multiple 4-3 teams above USC. ;)

25
by David (not verified) :: Thu, 10/25/2007 - 8:24pm

Understand the logic to simply ignore the FCS games, but there is something wrong when you ignore a loss to a FCS team. Yes you will also get inforamtion on the team from the other games they play, but you are ignoring a loss that says something about the team as well. To throw it out on the grounds that you can't measure it, points out one of the funadamental flaws of any computer system and the advantage that the human system will always have. Yes the human polls can be flawed and people can vote with their hearts, but at least they are smart enough to know that you don't discount a loss to a small school program on the basis that you don't know how good the small school is compared to the other 119 big schools.

26
by Dwayne (not verified) :: Fri, 10/26/2007 - 9:39am

#23, Great point. A huge problem with the rankings is that you know you will fall any time you lose. If you lose to #1, there is a chance you are still the second best team in the nation, but the rankings will drop you. These debates are part of what is great about college football.

27
by zlionsfan (not verified) :: Fri, 10/26/2007 - 7:10pm

Brian, you're starting to make it big ... the subjective-system comments are trolling in. Er, I mean rolling in.

Re comments about UM-ASU: he explained why he wasn't including it. I'm not sure I can explain it better, but I can try.

Think about selling old board games on eBay. If you sell a game that enough other people have sold, you can get a feel for the market value of the game. If you sell one that enough people in the same region have sold, the value may be even more accurate.

If you try to sell a game that only two other people have sold, it's really, really hard to determine the market value of your game. Maybe it sold for US$5 and US$3. Was it just another old game? Was it a really valuable game, but the sellers just didn't know? Was it a really cheap game, but the buyers wanted it badly?

You can set an arbitrary value for it, like US$60, but that doesn't mean the game is worth $60, and if you're trying to calculate net profit, it looks silly to show that you lost $30 for the month because you made $20 on games with known market values and you "lost" $50 on a game because you decided it was worth $60.

Polls have no advantages. They're based on the myth that a person can make an accurate subjective judgment about the relative strength of the top 30 to 40 teams and adjust that judgment on a weekly basis while still doing their regular work.