Writers of Pro Football Prospectus 2008

Most Recent FO Features

HarrisChr11.jpg

» Film Room: Chris Harris

Is Harris one of the league's top cover corners, or a product of the system in which he plays? Cian Fahey says the answer lies somewhere in the middle.

03 Dec 2008

FEI Week 14 Ratings

by Brian Fremeau

The Fremeau Efficiency Index principles and methodology can be found here. Like DVOA, FEI rewards playing well against good teams, win or lose, and punishes losing to poor teams more harshly than it rewards defeating poor teams. Unlike DVOA, it is drive-based, not play-by-play based, and it is specifically engineered to measure the college game.

FEI is the opponent-adjusted value of Game Efficiency, a measurement of the success rate of a team scoring and preventing opponent scoring throughout the non-garbage-time possessions of a game. Like DVOA, it represents a team's efficiency value over average. Strength of Schedule is calculated from a privileged perspective (explained here) and represents the likelihood that an Elite team (top 5) would post an undefeated record against the given team's opponents to date.

The following ratings are calculated based on data from all FBS games played through Saturday, November 29. Only games between FBS teams are considered.

Rank Team Record FEI Last Week vs. Top 10 vs. Top 40 GE GE Rank SOS SOS Rank
1 Florida 10-1 0.323 1 1-0 4-1 0.455 1 0.314 51
2 Penn State 10-1 0.282 2 0-0 2-1 0.354 4 0.380 62
3 Oklahoma 10-1 0.277 4 0-1 4-1 0.347 5 0.226 38
4 USC 10-1 0.271 5 0-0 4-0 0.398 2 0.419 73
5 Texas 11-1 0.269 3 1-0 3-1 0.391 3 0.260 43
6 North Carolina 7-4 0.250 6 2-0 5-4 0.105 21 0.136 14
7 Georgia Tech 7-3 0.234 7 2-1 5-3 0.096 22 0.114 11
8 Alabama 12-0 0.229 8 0-0 3-0 0.306 8 0.407 68
9 Boston College 8-3 0.213 10 1-2 5-3 0.079 27 0.098 5
10 Florida State 6-4 0.200 9 0-3 5-4 0.047 39 0.057 2
11 Ohio State 9-2 0.188 12 0-2 1-2 0.168 16 0.205 32
12 Texas Tech 9-1 0.186 11 1-1 2-1 0.232 11 0.247 41
Rank Team Record FEI Last Week vs. Top 10 vs. Top 40
GE GE Rank SOS SOS Rank
13 Virginia Tech 7-4 0.179 14 2-2 4-4 0.046 40 0.124 12
14 Pittsburgh 8-3 0.176 17 0-0 4-2 0.079 26 0.275 46
15 Clemson 5-5 0.175 20 1-3 4-5 0.029 47 0.109 10
16 Mississippi 7-4 0.174 13 1-1 1-3 0.155 18 0.158 22
17 Iowa 7-4 0.171 15 1-0 1-2 0.170 15 0.277 47
18 Ball State 11-0 0.171 16 0-0 1-0 0.320 7 0.713 116
19 Cincinnati 9-2 0.161 19 0-1 4-2 0.084 25 0.195 30
20 Wake Forest 7-5 0.160 21 1-1 4-5 0.028 48 0.147 19
21 Missouri 8-3 0.140 18 0-1 0-2 0.213 12 0.407 69
22 Connecticut 6-4 0.139 23 0-1 2-4 0.061 30 0.254 42
23 Boise State 11-0 0.133 26 0-0 1-0 0.330 6 0.712 115
24 Utah 11-0 0.131 24 0-0 1-0 0.261 9 0.660 109
25 West Virginia 6-4 0.129 25 0-0 2-3 0.066 28 0.338 58

Adjusted Offensive Efficiency and Adjusted Defensive Efficiency are the opponent-adjusted values of Offensive Efficiency and Defensive Efficiency, explained here. Like FEI, the multiple-order adjustments are weighted according to both the strength of the opponent and the relative significance of the result; efficiency against a team's best competition faced is given more relevance weight. AOE and ADE represent a team's value over/under average. Positive AOE and negative ADE are the most valuable.

Rank Team Record AOE AOE Rank ADE ADE Rank OE OE Rank DE DE Rank
1 Florida 10-1 0.455 6 -0.473 3 0.549 10 -0.658 1
2 Penn State 10-1 0.510 3 -0.397 12 0.522 11 -0.506 6
3 Oklahoma 10-1 0.626 1 -0.323 18 0.896 3 -0.071 50
4 USC 10-1 0.399 8 -0.476 2 0.438 13 -0.650 2
5 Texas 11-1 0.493 4 -0.433 8 0.971 2 -0.249 23
6 North Carolina 7-4 0.179 32 -0.498 1 -0.111 69 -0.321 16
7 Georgia Tech 7-3 0.364 13 -0.309 21 0.075 41 -0.183 32
8 Alabama 12-0 0.285 18 -0.411 10 0.126 38 -0.637 3
9 Boston College 8-3 0.206 28 -0.470 4 -0.126 71 -0.458 8
10 Florida State 6-4 0.247 22 -0.302 22 -0.040 57 -0.126 40
11 Ohio State 9-2 0.187 29 -0.389 14 0.031 49 -0.437 10
12 Texas Tech 9-1 0.543 2 -0.244 27 1.011 1 0.136 85
Rank Team Record AOE AOE Rank ADE ADE Rank
OE OE Rank DE DE Rank
13 Virginia Tech 7-4 0.040 57 -0.361 15 -0.237 91 -0.237 25
14 Pittsburgh 8-3 0.378 11 -0.191 37 0.181 31 -0.041 58
15 Clemson 5-5 0.114 42 -0.468 5 -0.220 86 -0.383 11
16 Mississippi 7-4 0.274 20 -0.287 23 0.151 33 -0.342 14
17 Iowa 7-4 0.133 38 -0.446 7 0.048 45 -0.465 7
18 Ball State 11-0 0.323 16 -0.234 29 0.764 5 -0.216 27
19 Cincinnati 9-2 0.085 50 -0.389 13 -0.076 63 -0.297 18
20 Wake Forest 7-5 -0.089 73 -0.462 6 -0.355 107 -0.258 22
21 Missouri 8-3 0.355 14 -0.067 56 0.697 6 0.080 77
22 Connecticut 6-4 0.097 48 -0.427 9 -0.075 62 -0.362 12
23 Boise State 11-0 0.108 45 -0.309 20 0.420 14 -0.558 5
24 Utah 11-0 0.080 51 -0.226 30 0.205 28 -0.327 15
25 West Virginia 6-4 0.122 40 -0.282 24 -0.098 67 -0.310 17

The Week 14 FEI Ratings for all 120 FBS teams can be found here. Expanded FEI Ratings data can be found here.

Playoffs?!?

If ever there were a need for a college football playoff, it's this year. Not for the national championship, mind you, but for the Big 12 Conference title. Through 48 conference games, there are certainly more than two teams deserving of their shot, and a six-team, eight-team, or even a plus-one model would be better than what we've got.

If the season were shortened by eliminating those annoying non-conference games, a multiple-round playoff wouldn't even interrupt finals for the student-athletes. The first rounds could be played at home stadium sites and the finals could rotate through Kansas City, Dallas and San Antonio. Think of all of the revenue such a playoff would generate!

Here's the six-team model I propose: There would be two automatic bids per division, and two wild cards taken from either division. This year, the North would send Missouri and Nebraska and the South would send Texas, Oklahoma, Texas Tech and Oklahoma State to the playoff. For the following simulation, seeding was determined according to record first, followed by a double-elimination round-robin rock-paper-scissors battle to break ties. Scores were simulated by actual players from each team on an actual football field.

Wild Card Round
No. 5 Missouri def. No. 4 Nebraska (52-17) in Lincoln, Neb., October 4
No. 3 Texas Tech def. No. 6 Oklahoma State (56-20) in Lubbock, Texas, November 8

Semifinal Round
No. 2 Oklahoma def. No. 3 Texas Tech (65-21) in Norman, Okla., November 22
No. 1 Texas def. No. 5 Missouri (56-31) in Austin, Texas, October 18

Championship
No. 1 Texas def. No. 2 Oklahoma (45-35) in Dallas, Texas, October 11

Never mind the impact on the space-time continuum, I think this playoff model might really work. The first two rounds could be a bunch of blowout yawners, but the title game looks like an outstanding battle. And the best part is we get an undisputed champion of the conference!

Ahem.

There has been plenty of lamenting this week regarding the BCS-determined outcome of the Big 12 Conference South Division, and a retro-fitted hyper-playoff certainly seems as viable a solution to the problem as any. College football's annual crisis always has something to do with the BCS, and once again the best solution to the problem seems to be foresight. As obscure or unlikely as this year's nationally-implicated Big 12 tiebreaking scenario may have seemed months or years ago, it doesn't excuse the fact that the selection of a conference champion or division champion should have nothing to do with results of games outside the conference. Right? The team I follow most closely may be independent, but I'm pretty sure conferences are fundamentally built on this principle. Well, that and revenue.

That said, if the Big 12 South cannot fully sort itself out through conference records, head-to-head matchups and common opponents, what else is appropriate for breaking ties? I think a BCS-like formula that excludes out-of-conference data would be a fine way to resolve the issue, but to my knowledge, the voters and computers have not been asked to produce a conference-only perspective. So let's use FEI.

In this week's FEI ratings, Oklahoma holds a narrow lead over Texas, but that rating includes Game Efficiency data from Sooners victories over No. 19 Cincinnati and No. 27 TCU, whereas Texas' best out-of-conference victory came against No. 68 Rice. To produce the following Big 12 Conference FEI ratings, I plugged in conference games exclusively into the formula.


Big 12 Conference FEI Ratings Excluding All Non-Conference Games
Rank Team W-L Conf. FEI Conf. GE CGE Rank Conf. SOS CSOS Rank
1 Texas 7-1 0.263 0.315 1 0.342 7
2 Oklahoma 7-1 0.250 0.310 2 0.382 8
3 Texas Tech 7-1 0.182 0.203 3 0.317 6
4 Missouri 5-3 0.100 0.174 4 0.440 10
5 Oklahoma State 5-3 0.080 0.082 5 0.182 2
6 Baylor 2-6 -0.047 -0.082 7 0.179 1
7 Kansas 4-4 -0.066 -0.108 8 0.247 4
8 Nebraska 5-3 -0.067 -0.062 6 0.307 5
9 Kansas State 2-6 -0.139 -0.157 9 0.436 9
10 Colorado 2-6 -0.159 -0.178 10 0.483 11
11 Iowa State 0-8 -0.187 -0.226 11 0.718 12
12 Texas A&M 2-6 -0.211 -0.271 12 0.234 3

Texas slides ahead of Oklahoma in this conference-only output, but why? We can answer that question with Calibrated Adjusted Game Efficiency (CAGE). As first explained last season, FEI is calculated from multiple order Game Efficiency data modified to adjust for the strength of the opponent and relevance of the data point to an individual team. The Calibrated AGE data removes the relevance adjustment in order to directly compare individual performances of different teams.


Top Five Big 12 Conference Performances According to CAGE
Rank Team Opponent CAGE GE Non-Garbage Score Final Score
1 Oklahoma vs. Texas Tech 0.804 0.543 52-14 65-21
2 Texas Tech at Kansas 0.636 0.706 56-14 63-21
3 Texas Tech vs. Oklahoma State 0.611 0.518 49-20 56-20
4 Texas vs. Missouri 0.546 0.420 42-17 56-31
5 Texas vs. Oklahoma 0.543 0.130 45-35 45-35

Of the 96 Big 12 CAGE ratings (one for each team in the 48 games), 13 of the top 16 individual team performances were turned in by the big three, Texas, Oklahoma and Oklahoma State. The very best single-game conference performance came in Oklahoma's rout of Texas Tech two weeks ago, far ahead of the Longhorns' best wins over Missouri and Oklahoma in back-to-back weeks in October. However, Texas' worst CAGE rating came in their last-second loss to Texas Tech (0.105), a rating better than three Oklahoma performances: versus Texas (0.087), Kansas (0.065), and Kansas State (0.016). Plus, according to variance in CAGE, Texas was the third most consistent team in the Big 12 (0.025); Oklahoma (0.060) and Texas Tech (0.119) were the least consistent.

This evaluation doesn't exactly settle the debate, but the FEI statistical evidence supports sending the Longhorns to Kansas City over Oklahoma on the merits of conference play. Texas had a stronger overall average Game Efficiency, was more consistent, and had fewer weak performances. By the same CAGE measure, Missouri is the best team from the North division by a big margin, but that's not saying much either. And the most significant part of the Big 12 Conference FEI breakdown might be the total disparity in strength of schedule.

Are Iowa State and Baylor really members of the same conference? The Bears' conference opponents went a combined 32-24 against teams other than Baylor; the Cyclones' opponents went 19-37 against conference teams other than Iowa State. The SOS rating listed above is calculated as the likelihood of an elite team going undefeated against the given team's schedule, and Baylor's conference schedule is four times harder than Iowa State's. Adding in out-of-conference games, Iowa State's overall schedule is the easiest among every BCS conference team, and is even easier than 18 team schedules from non-BCS conferences. If Cyclones fans boast about the strength of the Big 12 this offseason, ask them when they are planning on joining the league.

It's not the Cyclones' fault, of course, that they skipped playing Texas, Oklahoma and Texas Tech in 2008, but it does illustrate the danger in categorically making assumptions about teams based on other assumptions of conferences as a whole. The Pac-10 plays a round robin, but USC's conference strength of schedule is much weaker than the rest of the league, simply because the Trojans don't have to play against themselves. Ditto Florida and Alabama in what has been a mostly disappointing SEC, though they'll get their chance against one another this weekend.

Back to the matter at hand. Did the Big 12 get it right or wrong? Texas Tech played their way out both by no-showing in Norman, but also by not dominating Baylor, Nebraska, and Texas A&M. FEI says the best team overall, Oklahoma, won the bid, Texas deserved it, and it's a really close call either way. Like others, I maintain that arguments are good for college football, no solutions are perfect, and better metrics can help.

Posted by: Brian Fremeau on 03 Dec 2008

17 comments, Last at 07 Dec 2008, 2:07am by Ross

Comments

1
by Travis :: Wed, 12/03/2008 - 12:19pm

Alternatively:
Wild Card Round
No. 5 Kansas def. No. 4 Missouri (40-37) in Kansas City, Mo., November 29
No. 3 Texas def. No. 6 Oklahoma State (28-24) in Austin, Texas, October 25

Semifinal Round
No. 1 Oklahoma def. No. 5 Kansas (45-31) in Norman, Okla., October 18
No. 2 Texas Tech def. No. 3 Texas (39-33) in Lubbock, Texas, November 1

Championship
No. 1 Oklahoma def. No. 2 Texas Tech (65-21) in Norman, Okla., November 22

or

Wild Card Round
No. 3 Oklahoma def. No. 6 Nebraska (62-28) in Norman, Okla., November 1
No. 5 Oklahoma State def. No. 4 Missouri (28-23) in Columbia, Mo., October 11

Semifinal Round
No. 1 Texas Tech def. No. 5 Oklahoma State (56-20) in Lubbock, Texas, November 8
No. 2 Texas def. No. 3 Oklahoma (45-35) in Dallas, Texas, October 11

Championship
No. 1 Texas Tech def. No. 2 Texas (39-33) in Lubbock, Texas, November 1

2
by Bravoatoc (not verified) :: Wed, 12/03/2008 - 12:41pm

Seeding for the above hyper-playoff was determined by order of finish in the BCS.

4
by Travis :: Wed, 12/03/2008 - 1:21pm

For the following simulation, seeding was determined according to record first, followed by a double-elimination round-robin rock-paper-scissors battle to break ties.

By those standards, Oklahoma, Texas, and Texas Tech could be anywhere in the top 3.

(FWIW, the seeding in the original simulation by that standard should have been Missouri 4/5, Oklahoma State 4/5, and Nebraska #6.)

3
by Tom Gower :: Wed, 12/03/2008 - 1:20pm

No, it wasn't. Here are the B12 teams ranked in the BCS top 25:
1. Oklahoma
2. Texas
3. Texas Tech
4. Oklahoma State
5. Mizzou
and I'll add in
6. Nebraska

Here's your BCS playoffs. As a result of their byes, OU and UT get to play on the road in the semifinal round. Mizzou also gets to host in the first round because they won their division. Them's the breaks.

Wild Card:
#3 Texas Tech over #6 Nebraska
#4 Oklahoma State over #5 Mizzou (game at Mizzou)

Semifinals:
#3 Texas Tech over #2 Texas (game at TTU)
#1 Oklahoma over #4 Oklahoma State (game at OkSU)

Final:
#1 Oklahoma over #3 Texas Tech

5
by mm (not verified) :: Wed, 12/03/2008 - 1:40pm

I actually don't mind if, after several other tiebreakers fail to produce a winner, they resort to looking at non-conference games. I understand some might feel differently.

What I found silly, though, was Texas claiming the SEC/ACC tiebreaker procedure which uses the BCS to pick the worst team of the 3 was better than the Big-12 which uses the BCS to pick the best team of the 3.

6
by BK (not verified) :: Wed, 12/03/2008 - 2:03pm

"What I found silly, though, was Texas claiming the SEC/ACC tiebreaker procedure which uses the BCS to pick the worst team of the 3 was better than the Big-12 which uses the BCS to pick the best team of the 3."

While it may be silly, after eliminating #3 thru BCS, the ACC/SEC revert back to head-to-head, which has been the de facto way of breaking ties most any sport. So if Big 12 modifies the tie breaker procedure for the future to state: Top 2 BCS teams head-to-head record will be used to break tie (as opposed to eliminating #3 - maybe this won't hurt peoples feelings as much!)

8
by mm (not verified) :: Wed, 12/03/2008 - 3:47pm

That, I believe, is how its worded in the ACC and SEC, but it's the same thing. You're acknowleding that head to head does not break the tie. To say that the BCS has a good opinion on the last team and not the first is absurd.

I have to re-emphasize this...you are ignoring head to head to pick a champion. To say that it's OK to ignore head to head to throw out Texas Tech but not OK to ignore head to head to pick Oklahoma is something absurd.

To decide the division champion you take the following steps:
1) pick the team with the best conference record. If there is a tie, emliminate the rest and go to step 2.
2) pick the team with the best head to head record between the tied teams. If more than 2 are tied for this, eliminate the rest and go to 3.
3) pick the team that had the best record against the next best team in the division(in order of finish). If more than 2 are tied, eliminate the rest. Repeat for each next best team if necessary. If more than 2 are still tied, eliminate the rest and go to 4.
4) pick the team with the best record against common conference opponents. If more than 2 are tied, eliminate the rest and go to 5.
5) pick the team with the best BCS standing. If more than two are tied, eliminate the rest and go to 6.
6) pick the team with the best winning percentage.
7) draw

At each step you are looking for the best team according to the criteria. Yes, if you end up with a 2 team tie you then can go back to step 2 to break that tie, because you'd rather not go further down the list of tie-breakers. But at each step (before 7) you are looking for the best. To get to step 5 with a 3 way tie, have it pick a clear winner of the 3, and then ignore that clear winner because you want Texas to get in is inconsistent.

Look at the NFL tiebreakers for a comparison. At every step it also looks for the best team according to criteria. Only if exactly 2 teams are tied according to the criteria does it go back to head to head.

7
by Travis :: Wed, 12/03/2008 - 3:36pm

The NFL tiebreakers work in the same fashion as the Big 12's. If one team beats all the other teams in a certain tiebreaker, that team advances, rather than dropping the lowest team and re-starting from the top tiebreaker.

Hypothetical example:
Denver beat New England.
New England beat Indianapolis.
Indianapolis beat Denver.

All three teams win their division, finish with 15-1 records, are 11-1 in the AFC, and 1-0 vs. common opponents (not enough to matter, FWIW).

New England's strength-of-victory (win-loss percentage of defeated opponents) is .500, Indy's is .600, Denver's is .400.

In a three-way tiebreaker, Indianapolis would get the #1 seed based on the best strength-of-victory among the three. (Had Indy and New England TIED for best strength-of-victory, it would reset to a two-team tiebreaker between the two, and New England's head-to-head win would be the tiebreaker).

9
by mm (not verified) :: Wed, 12/03/2008 - 4:01pm

I just want to say that the reason my post came out 10 minutes after you while saying much the same thing (although in a much more rambling fashion) is only because I'm slow at composing and editing my comments, and not because I was copying you.

I also think every sports journalist who has printed something lambasting Oklahoma's selection should be required to print an article of equal length lambasting the NFL's tie-breaking procedures. Every one who has taken time on air attacking Oklahoma's selection must be forced to give equal air time attacking the NFL.

11
by Tom Gower :: Wed, 12/03/2008 - 5:41pm

Actually, there's one part of the NFL tiebreaker system that is quite flawed that nearly came into play in 2006. That year, TEN, NYJ, and PIT could have ended up in a tie at 9-7 for the last wild card spot. TEN would have lost tiebreakers to NYJ individually based on head-to-head result and PIT individually based on record against common opponents, but would have won a 3-way tiebreaker based on strength of victory. As a TEN fan, this result would have pleased me greatly, but as a critic of systems design, I don't like it. In a multi-way tie, the first tiebreaker to be applied should be the 2-team tiebreaker rules to each pair. If Team A beats both Team B and Team C in the 2-team tiebreaker, it should be declared the winner of the tiebreaker. Similarly, if Team A loses to both Team B and Team C in a 2-team tiebreaker, Team A should be in 3rd place and the winning of the 2-team tiebreaker between Teams B and C should come out on top.

That's somewhat of a special case, and something you don't have to worry much about in a college tiebreaker because the teams in question will have played each other (possible exception: B10, whose lack of a conference CG mostly mitigates this problem (B10 Rose Bowl berth tiebreaker rules)), but very well could happen in the NFL.

10
by Travis :: Wed, 12/03/2008 - 4:24pm

Hypothetical scenario in which the ACC and SEC tiebreakers lead to a questionable result:

Alabama beats Auburn by 50.
Auburn beats LSU by 1.
LSU beats Alabama by 1.

All three win all their other games, but Alabama wins theirs in blowout fashion, while Auburn and LSU squeak by. Alabama also schedules BCS non-conference opponents, while Auburn and LSU schedule Sun Belt and I-AA teams.

Final BCS standings: Alabama #2, LSU #7, Auburn #8.

LSU goes to the SEC championship game.

12
by Lance :: Wed, 12/03/2008 - 7:35pm

Sort of shocked that Oklahoma State dropped out of the top 25!

13
by Mikey Benny (not verified) :: Thu, 12/04/2008 - 3:06pm

Can someone explain to me how FSU, at 6-4, is #10? Thanks!

14
by RickD :: Thu, 12/04/2008 - 3:08pm

Are the pollsters systematically screwing over USC and Penn State with their over-hyped love of the SEC and Big 12? Your ratings would suggest that is the case.

Why is the off week Texas had against Texas Tech more forgivable than Penn State's loss at Iowa, or USC's loss at Oregon State?

15
by A. nonymous (not verified) :: Fri, 12/05/2008 - 4:47am

I'm guessing it's because Texas Tech is a lot better than Iowa or Oregon State, and TTU won on a last-second miracle play.

16
by RickD :: Fri, 12/05/2008 - 3:51pm

But how do we know Texas Tech is a "lot better" than anybody outside of the Big 12? Their non-conference schedule was Nevada, UMass, and Eastern Washington. And neither Texas nor Oklahoma has a meaningful non-conference win. (Or should we count Cincinnati?)

How can we tell if an entire conference is overrated when there is so little meaningful inter-conference play?

And the BCS system exacerbates this problem - universities now have the incentive to load up on patsies rather than play meaningful games, since any kind of loss is punished irrationally.

We see this kind of thing every year. We really get no feel as to the relative strengths of the conferences until the bowl games have been played, at which point it's too late.

17
by Ross (not verified) :: Sun, 12/07/2008 - 2:07am

Yea, they should have beaten Iowa...

However, when you look at Iowa's losses, you realize that they were a MUCH better team than their record indicated. Penn State lost on a last second field goal in BRUTAL conditions ON THE ROAD.

They should get a shot for the MNC. Ugh.