Writers of Pro Football Prospectus 2008

Most Recent FO Features

KellyChi13.jpg

» The Week In Quotes: September 19, 2014

Yes, people are still talking about Adrian Peterson, Greg Hardy, and Ray McDonald. But hey, we also get Chip Kelly exchanging pleasantries with reporters.

11 Jan 2012

Final 2011 FEI Ratings

by Brian Fremeau

The college football season abruptly concluded on Monday night with the Alabama Crimson Tide turning in the best game performance of the year. They deserve every bit of the celebration that has followed, and there’s no asterisk on the national championship as far as I’m concerned. Now, I’ll try to explain why Alabama ranks third in FEI and why I’m not putting an asterisk on that either.

Since I started working with FEI data in 2003, the rating system has consistently agreed with the consensus regarding the national champion. Though there have been frequent arguments about the teams given the opportunity to play in the two-team BCS championship format, the team hoisting the crystal football at the end of the year has always had a legitimate claim to the nation’s best end-of-year resume as well. The BCS system rewards the "best overall season," a distinction from playoff formats that reward the "best end-of-year run."

Of course, any team can be beaten on any given day. LSU’s distinction of having had the best overall season heading into the championship didn’t make them invincible, and I never claimed otherwise. What I’ve been wondering about in the aftermath of their loss in the title game is: do they still own the nation’s best overall season? The final FEI ratings support that claim.

I knew this kind of scenario would happen eventually. It’s happened before, of course. I don’t have FEI data for 2002, but I wouldn’t be surprised if the system considered the Miami Hurricanes to have had the better overall season despite a loss to the Ohio State Buckeyes in overtime in the championship. I’ve long thought that the Notre Dame Fighting Irish were likely the nation’s best team in 1989 and the second-best team in 1988, even though the on-field results in those seasons suggest the opposite.

It takes a particularly unique set of circumstances to win a national championship. According to FEI, this year’s LSU Tigers were significantly better than the 2007 and 2003 Tigers that did win the title. Neither of those teams had to face a team rated as highly as Alabama even once, while this year’s edition faced the Tide twice, both away from Baton Rouge. The Oregon game this year (also away from home and LSU’s third-toughest test according to FEI) would be the equivalent of the toughest opponent LSU faced in 2003 or 2007. The Tigers finished this year with the No. 1 strength of schedule and won 12 FBS games.

However, they didn't win when it counted most. In fact, they barely competed. The other two teams in the conversation, Alabama and Oklahoma State, were undefeated in regulation (a claim that helped vault LSU fortuitously into the 2007 championship game). The Crimson Tide's only loss to LSU trumped Oklahoma State's loss to Iowa State to position Alabama for the rematch, but the win column heavily favored the Cowboys. Alabama only played three FEI top-30 opponents, winning two, while Oklahoma State posted seven such victories. Only by playing the FEI No. 1 twice did Alabama's schedule edge out that of the Pokes.

Here are the top-to-bottom resumes of LSU, Oklahoma State, and Alabama according to GFEI, opponent-adjusted single-game efficiency.

Alabama GFEI Resume LSU GFEI Resume Oklahoma State GFEI Resume
Opponent Final GFEI GFEI
Rk
Opponent Final GFEI GFEI
Rk
Opponent Final GFEI GFEI
Rk
LSU W 21-0 .912 1 Alabama W 9-6 .716 7 Oklahoma W 44-10 .872 2
Florida W 38-10 .588 22 Oregon W 40-27 .670 11 Baylor W 59-24 .766 4
Vanderbilt W 34-0 .582 24 West Virginia W 47-21 .608 17 Texas Tech W 66-6 .710 9
Arkansas W 38-14 .572 27 Mississippi W 52-3 .586 23 Missouri W 45-24 .505 50
LSU L 6-9 .513 48 Georgia W 42-10 .565 29 Stanford W 41-38 .455 73
Penn State W 27-11 .459 72 Auburn W 45-10 .562 32 Tulsa W 59-33 .450 76
Tennessee W 37-6 .398 125 Arkansas W 41-17 .560 33 Texas A&M W 30-29 .420 99
Auburn W 42-14 .337 181 Tennessee W 38-7 .403 120 Texas W 38-26 .409 112
Mississippi W 52-7 .311 207 Florida W 41-11 .389 136 Kansas W 70-28 .320 192
Mississippi State W 24-7 .281 246 Alabama L 0-21 .317 198 Kansas State W 52-45 .315 201
North Texas W 41-0 .032 688 Mississippi State W 19-6 .313 206 Arizona W 37-14 .156 446
Kent State W 48-7 -.082 900 Western Kentucky W 42-9 .152 453 Iowa State L 31-37 .035 681
Kentucky W 35-7 .139 473 Louisiana Lafayette W 61-34 .033 685

Alabama's final game was the most impressive victory of the year without a doubt. Based on that performance, I have tremendous confidence that the Crimson Tide would also have navigated the tougher schedules of Oklahoma State and LSU just as successfully as those two teams did. However, FEI doesn't have the data to back up that confidence. Alabama didn't play those games.

I've always been convinced that a computer rating system cannot be much more than a lens to help us focus on the big picture. We can continue to work to improve our metrics, collect more and better data, and make that big picture even more clear. But the role of FEI isn't to crown a national champion. In years in which FEI and the national championship game result agree, it can be tempting to insist that the ratings mean more than they do. This year is a good reminder to not fall into that trap.

Games Of The Year

The tables below represent the best single-game performances in Offensive FEI, Defensive FEI, and overall Game FEI of the 2011 season. It is important to note that the individual OFEI, DFEI and GFEI data points are subjected to a relevance factor when calculating the season-long OFEI, DFEI and FEI team ratings. Oklahoma State, for instance, had three of the top-10 GFEI performances of the year, but the result against Texas Tech receives very little weight in Oklahoma State's overall FEI rating.

Offensive FEI Single-Game Performances of the Year
OFEI Date Team Opponent Opp DFEI Result
2.550 10/8 Alabama vs. Vanderbilt 48 W 34-0
2.472 9/17 Navy at South Carolina 7 L 21-24
2.424 10/1 Arizona at USC 29 L 41-48
2.423 10/15 Texas A&M vs. Baylor 86 W 55-28
2.421 10/8 Miami at Virginia Tech 31 L 35-38
2.359 9/10 Tennessee vs. Cincinnati 30 W 45-23
2.322 10/22 Houston vs. Marshall 67 W 63-28
2.321 11/19 Baylor vs. Oklahoma 12 W 45-38
2.309 1/2 Stanford vs. Oklahoma State 3 L 38-41
2.246 10/29 Oklahoma at Kansas State 19 W 58-17
Defensive FEI Single-Game Performances of the Year
DFEI Date Team Opponent Opp OFEI Result
-2.606 10/29 Oklahoma State vs. Baylor 1 W 59-24
-1.879 1/9 Alabama vs. LSU 17 W 21-0
-1.877 12/3 Oklahoma State vs. Oklahoma 18 W 44-10
-1.719 11/19 South Florida vs. Miami 3 L 3-6
-1.658 11/19 North Carolina State vs. Clemson 22 W 37-13
-1.525 10/1 Boise State vs. Nevada 38 W 30-10
-1.511 11/19 Rutgers vs. Cincinnati 25 W 20-3
-1.441 10/22 Miami vs. Georgia Tech 16 W 24-7
-1.413 11/5 Alabama vs. LSU 17 L 6-9
-1.367 10/7 Boise State at Fresno State 44 W 57-7
FEI Single-Game Performances of the Year
GFEI Date Team Opponent Opp FEI Result
0.912 1/9 Alabama vs. LSU 1 W 21-0
0.872 12/3 Oklahoma State vs. Oklahoma 4 W 44-10
0.779 10/29 Oklahoma at Kansas State 18 W 58-17
0.766 10/29 Oklahoma State vs. Baylor 29 W 59-24
0.738 1/4 West Virginia vs. Clemson 32 W 70-33
0.729 11/12 Oregon at Stanford 7 W 53-30
0.716 11/5 LSU at Alabama 3 W 9-6
0.714 9/17 Wisconsin vs. Northern Illinois 53 W 49-7
0.710 11/12 Oklahoma State at Texas Tech 77 W 66-6
0.699 10/21 Syracuse vs. West Virginia 17 W 49-23

FEI Data Correlations

We're not going to put together our projections for the 2012 season until recruiting and spring practice wrap up, but as has been the case in year's past, the foundations for next season expectations are formed in previous year data. I've already updated the 2011 Program FEI ratings, a five-year weighted drive efficiency metric that forms the core of my annual FEI projections. Below, I've plotted out each of the drive unit metrics I measure and their correlations to both same-season winning percentage and correlations from year-to-year. Light blue plots actually represent a negative correlation to win percentage, but are plotted as a positive correlation to allow for easier comparisons.

FEI Final Top 25

The Fremeau Efficiency Index (FEI) rewards playing well against good teams, win or lose, and punishes losing to poor teams more harshly than it rewards defeating poor teams. FEI is drive-based, not play-by-play based, and it is specifically engineered to measure the college game.

FEI is the opponent-adjusted value of Game Efficiency (GE), a measurement of the success rate of a team scoring and preventing opponent scoring throughout the non-garbage-time possessions of a game. FEI represents a team's efficiency value over average. Strength of Schedule (SOS) is calculated as the likelihood that an "elite team" (two standard deviations above average) would win every game on the given team's schedule.

Mean Wins (FBS MW) represent the average total games a team with the given FEI rating should expect to win against its complete schedule of FBS opponents. Remaining Mean Wins (FBS RMW) represent the average expected team wins for games scheduled but not yet played.

Offensive FEI (OFEI), Defensive FEI (DFEI), Special Teams Efficiency (STE) are also provided, along with Field Position Advantage (FPA), the share of the value of total starting field position earned by each team against its opponents.

Only games between FBS teams are considered in the FEI calculations. These FEI ratings are a function of results of games played through January 9. The ratings for all FBS teams can be found here. You can also find OFEI, DFEI, and STE on their own pages.

Rk Team FBS
Rec
FEI GE GE
Rk
SOS SOS
Rk
FBS
MW
OFEI Rk DFEI Rk STE Rk FPA Rk
1 LSU 12-1 .318 .270 4 .033 1 11.0 .391 17 -.747 2 2.830 6 .572 2
2 Oklahoma State 12-1 .311 .255 5 .120 26 11.5 .414 12 -.728 3 2.297 7 .558 5
3 Alabama 11-1 .306 .317 2 .091 16 10.4 .418 11 -.780 1 -.407 72 .528 19
4 Oklahoma 10-3 .240 .185 12 .062 7 10.2 .386 18 -.464 12 .861 43 .559 4
5 Wisconsin 10-3 .235 .300 3 .189 38 10.9 .745 2 -.139 49 .903 40 .531 17
6 Oregon 11-2 .232 .238 6 .102 20 10.6 .461 10 -.549 5 .955 39 .508 44
7 Stanford 11-2 .214 .234 8 .138 29 10.6 .505 6 -.446 13 -.077 63 .493 70
8 Boise State 12-1 .213 .329 1 .415 91 11.6 .308 23 -.357 21 3.771 1 .611 1
9 Michigan 11-2 .202 .203 11 .264 58 10.7 .472 9 -.414 16 .513 50 .523 26
10 USC 10-2 .193 .162 13 .207 42 9.5 .487 8 -.295 29 1.403 23 .524 23
11 Michigan State 10-3 .187 .129 16 .095 18 9.0 .150 41 -.542 6 1.610 19 .539 11
12 Florida State 8-4 .180 .140 15 .287 68 9.2 .052 54 -.510 10 3.590 2 .560 3
Rk Team FBS
Rec
FEI GE GE
Rk
SOS SOS
Rk
FBS
MW
OFEI Rk DFEI Rk STE Rk FPA Rk
13 South Carolina 10-2 .178 .118 19 .284 67 9.4 .139 43 -.541 7 -3.159 112 .490 76
14 Arkansas 10-2 .176 .125 17 .042 3 8.2 .408 14 -.185 41 2.061 9 .532 16
15 Notre Dame 8-5 .171 .113 21 .122 27 8.9 .281 26 -.407 18 .455 53 .498 64
16 Texas A&M 7-6 .170 .121 18 .087 14 8.9 .359 19 -.298 27 .554 48 .507 46
17 West Virginia 9-3 .170 .109 22 .154 32 8.6 .530 4 -.216 35 -1.109 83 .490 75
18 Kansas State 9-3 .163 .064 38 .068 9 7.7 .256 29 -.397 19 1.229 28 .528 21
19 Georgia 9-4 .152 .100 24 .112 22 8.8 .201 34 -.469 11 -.472 76 .498 63
20 Missouri 7-5 .150 .067 36 .072 11 7.5 .411 13 -.165 46 .327 56 .503 53
21 TCU 10-2 .143 .209 10 .377 84 9.7 .333 21 -.184 42 1.405 22 .538 13
22 Houston 12-1 .141 .236 7 .566 109 11.3 .498 7 -.085 55 -.577 78 .503 54
23 Cincinnati 9-3 .134 .107 23 .302 71 8.2 .297 25 -.289 30 1.616 18 .542 10
24 Rutgers 8-4 .129 .065 37 .347 77 8.3 .056 52 -.596 4 1.962 12 .519 31
25 Southern Mississippi 11-2 .121 .156 14 .546 107 10.6 -.082 72 -.375 20 1.565 20 .550 8

Posted by: Brian Fremeau on 11 Jan 2012

43 comments, Last at 08 Dec 2012, 3:41am by john123456

Comments

1
by Will Allen :: Wed, 01/11/2012 - 1:11pm

Gosh, to have "tremendous confidence" that Alabama would have succesfully navigated LSU's and OK State's tougher schedules as well as those two teams did, largely based upon a single game that occured more than forty days after the previous game, completely ignores variance in human performance, and random chance. Really? "Tremendous confidence"?

We, all of us, really have a tendency to convince ourselves that we know more than we actually do.

2
by c0rrections (not verified) :: Wed, 01/11/2012 - 1:55pm

Did Alabama show much variance all year?

6
by Will Allen :: Wed, 01/11/2012 - 2:42pm

There are two teams in a football game, and measuring variance over 12 games isn't all that meaningful. Adavanced stats are helpful in expanding sample sizes in football, but even then, the samples just are not large enough to provide "tremendous confidence", as I understand that term.

7
by Aaron Brooks Go... :: Wed, 01/11/2012 - 2:56pm

Yes. They have the best game and the 900th best game of the year.

Twice, Alabama played poorly enough to lose to Iowa State.

8
by Will Allen :: Wed, 01/11/2012 - 3:07pm

I'm a little surprised by that. I didn't bother to look, because I just don't think measuring variance with such small samples is very illuminative, but when you factor the variance of Alabama, with the variance of all of LSU's and OK State's opponents, and then toss in random chance, I just can't see how anybody could ever have "tremendous confidence" regarding the precise win-loss records over a 12 or 13 game schedule, involving 3 good football teams.

9
by Brian Fremeau :: Wed, 01/11/2012 - 3:18pm

"Tremendous confidence" was a poor choice. My intention was to say that Alabama at their best was better than anyone else at their best this year.

I'm not sure its illuminating either, but here is the OFEI, DFEI and GFEI variance data for each team (lower ranking means less variance):

LSU
OFEI No. 10
DFEI No. 45
GFEI No. 44

Oklahoma State
OFEI No. 41
DFEI No. 110
GFEI No. 100

Alabama
OFEI No. 80
DFEI No. 100
GFEI No. 102

12
by Will Allen :: Wed, 01/11/2012 - 4:00pm

Yeah, I don't mean to knock down Alabama. Every facet of their game Monday night was extremely impressive, except their field goal kicking, even after factoring how tremendously bad LSU's offensive execution was, independent of the opponent's defensive performance. I also think Alabama likely was better in the first game, excepting field goal kicking, not that we should give the Tide a pass for failing to have a decent kicker on their roster.

I just think we all have a tendency to overrate our ability to predict how athletic events would unfold. Which is kind of funny, since the reason we keep watching them is because we get surprised constantly.

10
by AnonymousBoob (not verified) :: Wed, 01/11/2012 - 3:41pm

How do you know? The two games you point to they won by 41 points each. You really think the focus (and personnel decisions) in two 41 point blow outs over overmatched foes would've been the same in a close, intra-conference matchup?

BTW, in those two games, Alabama was up 20-0 and 24-0 at half. They outgained their opponents 586-169 and 482-90.

Maybe it's me, but I have a hard time looking at these two performances as evidence that Alabama would lose a conference game at Iowa State.

11
by Viliphied (not verified) :: Wed, 01/11/2012 - 4:00pm

Those games tell me far more about flaws in the FEI system than it does about Alabama's performances in those games.

13
by Viliphied (not verified) :: Wed, 01/11/2012 - 4:10pm

Honestly, how can a game that Alabama A) Won by 41 points, B) Was up 21 in the 1st quarter, C) started rotating in 2nd/3rd teamers in the 2nd quarter, and D)Outgained their opponent by almost 400 yards result in a negative FEI? One of the worst FEI games of the year? Really?

17
by AnonymousBoob (not verified) :: Wed, 01/11/2012 - 5:14pm

I don't blame the system. I would guess it is virtually impossible to weed out situations in which total mismatches result in these types of results. To put it bluntly, Alabama and Kent State shouldn't be playing competitive football...even Alabama's worst day wouldn't result in a close game. There is simply too much of a difference in talent. Any empirical analysis is going to struggle to adequately account and articulate true team performance in these types of environments.

I blame the interpretation of the system. For example, the post above that states, unequivocally, that Alabama played two games this season in which it's performance would have resulted in losses to Iowa State. However, as shown, in those two games they thoroughly dominated the opposition. Taking the empirical analysis without the context of the game is the mistake.

19
by Brian Fremeau :: Wed, 01/11/2012 - 10:22pm

Good questions. I think there is probably something here that needs attention, to be honest. First, a little bit of context.

It isn't one of the worst GFEI games of the year. It was below average, but there were 1430 individual GFEI performances this season (715 FBS games). It wasn't among the bottom third in GFEI.

Against Kent State, three other teams had a better GFEI this year: Kansas State (GFEI No. 197), Northern Illinois (GFEI No. 585) and Temple (GFEI No. 872). The problem wasn't that Alabama didn't ultimately crush Kent State, but they didn't do so very efficienty at all. The non-garbage finals for each:

Kansas State led 34-0 through 14 game possessions
NIU led 40-3 through 23 game possessions
Temple led 31-9 through 20 game possessions
Alabama led 38-7 through 30 game possessions

At the end of the day, who cares? They all dominated Kent State and for a team as good as Alabama, it shouldn't make much of a difference. With the relevance factor added in, it works out to roughly 1/20th of Alabama's overall rating. Maybe that's where things need to be addressed. Should the relevance of that data point be much weaker than that? How much weaker?

22
by Aaron Brooks Go... :: Thu, 01/12/2012 - 10:54am

Why should scheduling cupcakes confer any benefit?

25
by cfn_ms :: Thu, 01/12/2012 - 12:49pm

It's not conferring a benefit, it's conferring a penalty. The question Fremau is talking about is how big of a penalty it should be, or if the data is just so irrelevant given the divergent talent of the two teams that it should instead just be ignored.

41
by AnonymousBoob (not verified) :: Wed, 01/18/2012 - 1:20am

Very good question and I have no idea what the correct answer is.

21
by Aaron Brooks Go... :: Thu, 01/12/2012 - 10:53am

If it makes you feel better, Alabama's listing doesn't even include a poor game they played against Georgia Southern, because they were an FCS school. That was a 10-pt game late in the 3rd, and GA Southern out-rushed Alabama. Not showing that game is sort of like FEI analyzing U-M in 2007, sans the home loss to Appalachian State, and saying they had a pretty good year.

Alabama was fully capable of laying an egg. They just lucked into doing so at home against a poor MAC team and a late season FCS team.

30
by AnonymousBoob (not verified) :: Thu, 01/12/2012 - 5:51pm

See that's problem. You are saying they laid an egg against an inferior MAC team in which they absolutely dominated. Did they dominate as much as they should have? Maybe not. But it didn't matter. There was no chance they were losing, the game wasn't close, and no one should draw any conclusions from their performance.

31
by Aaron Brooks Go... :: Thu, 01/12/2012 - 7:24pm

But when the argument from the Triangle crowd runs that Alabama wouldn't have lost to Iowa State, those arguments are less compelling. Alabama played, on three occasions, poorly enough that an Iowa State-level competitor could have beaten them. Or conversely, had the Oklahoma State team that showed up to play Iowa State played GA St, Kent State, or North Texas instead, they likely would have won anyway.

If Brian's still reading this, just out of curiosity, what were Temple's FEIs against Kent State and Penn State?

32
by Brian Fremeau :: Thu, 01/12/2012 - 9:58pm

Temple vs Kent State: GFEI No. 872
Temple vs Penn State: GFEI No. 702

33
by Brian Fremeau :: Thu, 01/12/2012 - 10:25pm

Let's explore this a bit more. There might be something here.

With how well they played defensively, it might be true that they had "no chance of losing". But five Alabama turnovers could have kept Kent State in the game more than they ultimately did. One interception set up Kent State in a goal-to-go situation. Still, perhaps the defensive dominance is undervalued with Game Efficiency/GFEI?

You also said "No one should draw any conclusions from their performance", which might be useful, but let's be more specific. Is it because the game wasn't ultimately close, or because Alabama is so much better than Kent State to make it irrelevant? Both?

35
by Aaron Brooks Go... :: Fri, 01/13/2012 - 12:15am

I'm not sure defense is underrated. Great defenses are more prone to short-term variance than great offenses are. In other words, you need fewer fluke successes to beat a great defense than a great offense.

Sort of like how Denver beat Chicago. Chicago kicked the crap out of Denver defensively for 55 minutes, but because their lead was so small, 5 minutes of success caught Denver up.

36
by Will Allen :: Fri, 01/13/2012 - 3:51am

Also, when your field goal kicker isn't a sure thing to get the ball wet, if he were kicking off the Navy pier in Mobile Bay, any game that managed to stay close is a game that could easily be lost.

40
by AnonymousBoob (not verified) :: Wed, 01/18/2012 - 1:14am

I think it's clearly both in this case. There is no chance that Alabama played with the type of emotion, preparation, or with the same skilled players that they would in a typical SEC game (due to backups getting more playing time than usual). The game was never going to be a contest and would be something the coaching staff almost certainly would've used for experimentation and exploration rather than maximizing margin of victory.

In the end, Alabama was up by 21 points after the end of the first quarter. Kent State didn't have a drive of longer than 16 yards until the 4th quarter (and even then, maxed out with a drive of 37 yards against, I am presuming, 3rd and 4th stringers). While a TO gave Kent State a 3 yard TD drive, that cut the lead only to 17, after which Alabama promptly scored on the next two possessions. It was a massacre on the football field, even if the numbers on the scoreboard were relatively tame.

With all that said, I can certainly understand the difficulty in weeding out games like this. How is an empirical system expected to determine which blowouts should be judged and which should not. How much credit should a game like this get versus Alabama's blow out of Ole Miss? Further, how does one judge the Ga Southern game, a D1-AA opponent (albeit a quality D1-AA opponent) stuck smack dab in the middle of the SEC schedule? It's not an easy question and requires someone much smarter than I to provide an answer.

Nevertheless, I shudder whenever anyone uses a blowout such as the Kent State-Alabama game when trying to compare across team performances.

3
by Viliphied (not verified) :: Wed, 01/11/2012 - 1:57pm

"Alabama only played three FEI top-30 opponents, winning two, while Oklahoma State posted seven such victories. Alabama played the FEI No. 1 twice and still finished with an easier schedule than the Pokes."

SOS Rank: Ok St. - 26, Ala - 16

Am I missing something?

5
by Brian Fremeau :: Wed, 01/11/2012 - 2:27pm

The sentence you quoted was erroneous and I'll correct it.

Strength of schedule means something very specific here. It is the likelihood that a team two standard deviations better than average would be expected to go undefeated against the entire schedule. By playing LSU twice, Alabama's likelihood of an undefeated season is lower than Oklahoma State's.

It might be fair to say that Oklahoma State's schedule would measure as tougher for a team 1.5 standard deviations better than average. Or it might be fair to say that it would be easier to play Alabama's schedule and finish with 2 or fewer losses than to play Oklahoma State's schedule and finish with 2 or fewer losses. These are all distinctly different ways to measure schedule strength.

4
by rj1 :: Wed, 01/11/2012 - 2:09pm

Gamecocks' special teams must've really sucked.

14
by Authentic Sellout (not verified) :: Wed, 01/11/2012 - 4:22pm

Times like this every team should play on Bourbon Street to test their discipline on and off the field.

16
by Kevin from Philly :: Wed, 01/11/2012 - 5:02pm

I played on Bourbon street once. Much like Alabama, I hurt pretty bad afterwards. And I didn't score.

15
by Kal :: Wed, 01/11/2012 - 4:59pm

Interesting. The FEI rankings are very different than the actual results on the field.

For instance, Oregon beat Wisconsin. Oregon is ranked less than Wisconsin.
Alabama dominated LSU. Alabama is worse than LSU.
Michigan State split with Wisconsin on two close games, yet is significantly further away on ranking.
Arguably, Stanford should have beaten OKST, yet ranks significantly lower than OKST.

I understand that all of these could fall under 'any given saturday' or whatever you'd like, but it's interesting to see how many do not work given direct matchups with the two teams. The only one that appears to work properly is the OKST/Oklahoma matchup and maybe the Oregon/Stanford one. The intuition is obviously that FEI is flawed in some way; at the very least, it indicates that FEI might not be as predictively useful.

18
by Brian Fremeau :: Wed, 01/11/2012 - 9:44pm

Well, you're going to have ranking violations no matter what. If you are intent on designing a system to have as few ranking violations as possible, that's fine, but I certainly don't think head-to-head results should trump the other 10 or 11 data points on the season as a rule.

FEI 2011 week-by-week projection results can be found here:
http://www.bcftoys.com/projections

23
by Aaron Brooks Go... :: Thu, 01/12/2012 - 11:03am

Those rankings, though, are style and strategy agnostic. Which is fine, but neglects a large portion of head-to-head interaction.

As a for instance, U-M was ranked higher than MSU heading into bowl season, but VT was much happier to play U-M than MSU, because VT is built to play teams like U-M (smaller, fast, misdirection team) and not teams like MSU (big lines, pro-style power team).

Stanford struggled with athletic teams with big-play potential -- Oregon, OK State, USC -- because their defense isn't fast and they couldn't just ball-control their way into a big lead.

LSU struggled with big lines and LBs who can play the option. Which is why they ran over Oregon and WVU (small defenses), but struggled mightily with Alabama and Georgia.

OK State was actually pretty solid against the pass, which is why they did just fine with Oklahoma, Baylor, and Texas Tech, but struggled against the run, hence why Iowa State and Stanford did well.

This is especially notable in martial arts, where your draw is often far more important than your seed. Some guys just can't beat other fighters with certain styles who, from a seed- or talent-perspective, they should obliterate.

28
by Will Allen :: Thu, 01/12/2012 - 2:39pm

Yep, that is a good part of whta makes the game fun to watch, and why, for me, a tournament of a small number of elite teams is the most fun to watch.

29
by Will Allen :: Thu, 01/12/2012 - 2:39pm

Yep, that is a good part of whta makes the game fun to watch, and why, for me, a tournament of a small number of elite teams is the most fun to watch.

26
by cfn_ms :: Thu, 01/12/2012 - 1:04pm

I agree with Brian's theoretical point here (that emphasizing ranking violations and H2H should NOT be a primary concern), though I disagree with his numbers in the Oregon-Wisconsin example.

Looking at his table it's fairly obvious why Wisconsin rates above Oregon (game score is .300 vs .238, a big Badger edge despite the worse record; and schedule strength difference isn't rated as huge). That said, I really have to wonder why Wisconsin's schedule is rated close to Oregon's.

The Badgers had a somewhat unfavorable 9-game BIG draw (missing Michigan, Iowa, Northwestern, though they played MSU twice) compared to Oregon's slightly unfavorable 10-game P12 draw (missing only a Utah team that went 4-5 in the lesser South division), and had an atrocious non-conference schedule (with a AA AND #119 by FEI UNLV compared to Oregon getting #1 LSU). That strikes me as a substantial difference, which should be even greater given that FEI's SOS basis (odds of undefeated) seems to over-weight the toughest opponent or two (which Oregon wins in a landslide unless I'm REALLY mis-understanding the system). Brian, any thoughts on that one?

39
by Brian Fremeau :: Fri, 01/13/2012 - 2:27pm

The difference really comes down to the LSU game. Without it, Wisconsin's schedule measures as more difficult than Oregon's.

Roughly equivalent:
Oregon .310 SOS (at Stanford, home vs USC, vs Wisconsin)
Wisconsin .314 SOS (at Michigan St., vs Michigan St., vs Oregon)
There would be a bigger difference if Wisconsin had one of these three at home.

The rest (leaving out Ore/LSU):
Oregon .825 SOS
Wisconsin .604 SOS
Wisconsin's games at Ohio State, at Illinois, vs Penn State, and vs Nebraska are all tougher according to FEI than Oregon's next toughest game vs Arizona State.

The game against LSU makes the biggest difference as you suspected to push Oregon's SOS as tougher than Wisconsin's.

27
by Kal :: Thu, 01/12/2012 - 2:06pm

Brian, I'm not talking about ranking issues. That's an interesting point, but not the one I'm making.

I'm saying that even before the bowl games were played FEI was predicting things like LSU beating Alabama, Wisconsin beating Oregon, Oklahoma State routing Stanford, etc. When the games played out very differently. The final rankings doesn't surprise me given the prior rankings.

What is interesting to me is that it wasn't that good as far as being a good predictive tool, which is something other folks below have stated.

I do think that looking at raw stats or even advanced, opponent-adjusted stats is ultimately not going to be super useful as a predictive tool because of matchups. As an example: I predicted strongly that USC would be very competitive with Oregon despite S&P and FEI saying I was nuts because I knew how Oregon had struggled with containing top-quality wideouts this year; I was similarly not worried about Stanford because they didn't have any. In both cases matchups dictated strongly what happened in the game. Similarly, I did an analysis using S&P and FEI for Oregon/Wisconsin that showed how much of an advantage Oregon had on standard downs and special teams over Wisconsin; all of Oregon's big plays came on standard downs.

That's not going to be the case in all games, and there are a lot of other factors that determine who wins, obviously, but I think this is something that FEI and other systems might be able to improve on - be able to see large mismatches and predict based on similar mismatches seen other places how things should go.

34
by cfn_ms :: Thu, 01/12/2012 - 11:52pm

This bowl season was a disappointment for FEI bowl confidence picks (though far from awful), but last year it did really well. I don't think it's fair to draw conclusions that are too deep based just on one season, and especially based on a couple specific games.

37
by Kal :: Fri, 01/13/2012 - 11:19am

Well, one can draw conclusions that it is at least inconsistent. Similar to how we look at DVOA differently between seasons because of rule changes or trends we can start doing similar things with FEI and S&P.

38
by Brian Fremeau :: Fri, 01/13/2012 - 12:48pm

Oddly enough, FEI has a consistently back-and-forth success rate with bowl picks. Since I've been working with projections, even years have been much more successful than odd years.

24
by Adam H (not verified) :: Thu, 01/12/2012 - 11:56am

Not to mention Oklahoma State being ranked ahead of Iowa State!!!! Clearly these rankings are flawed!

20
by Alexander :: Thu, 01/12/2012 - 5:08am

Looking forward to the F+ ratings.

42
by sera (not verified) :: Mon, 10/15/2012 - 2:29pm

This bowl season was a disappointment for FEI bowl confidence picks (though far from awful), but last year it did really well. I don't think it's fair to draw conclusions that are too deep based just on one season, and especially based on a couple specific games.

evim şahane-mobilya modelleri-bedava chat-chat-islami rüya tabirleri-mynet

43
by john123456 (not verified) :: Sat, 12/08/2012 - 3:41am

Examsmafia.com provides its customers genuine and quality study material. The study material for the certification 70-417 exam is thoroughly researched and prepared by IT industry experts who have deep knowledge and exposure of the candidates’ needs and problems. The 70-417 study material is also being constantly updated and amendments are being made whenever required in order to keep the content valid and relevant. Thus there is no question of authenticity with the 70-417 exam material provided by http://www.examsmafia.com/70-417.html. Our 70-417 study material is to the point, interactive and very easy in learning. Thus candidates from varying academic backgrounds find it the best and easy way of preparation for their certification 70-417 exams.