Writers of Pro Football Prospectus 2008

08 Sep 2010

FEI: The Boise State Dilemma

by Brian Fremeau

Boise State essentially clinched an undefeated season with one fourth-quarter drive Monday night against Virginia Tech. College football's regular season "playoff" has barely begun -- 35 percent of teams haven't yet played a fellow FBS opponent -- but it sure feels like the Broncos have already won their bracket. Whether or not their bracket will connect to the BCS championship is the question that will loom over the 2010 season.

If Boise State is one of a handful of undefeated teams at the end of the year, should the Broncos get the nod based on their history of meeting big game challenges? Should a one-loss team from a power conference earn a bid over an undefeated Boise State? How much should strength of schedule factor into the national championship race?

If you follow FEI, you know that my adjustments for opponent strength are significant factors in the formula. FEI is not concerned with identifying "deserving" candidates. It is designed to isolate the strength of each team in terms of its ability to maximize scoring opportunities and minimize those of its opponent. And because there are stark differences between the best and worst teams in college football, the lens through which those raw efficiency metrics should be viewed is critical.

WAC teams are worse than SEC teams. Much worse. That doesn't mean that there is a ceiling for Boise State in the FEI ratings, but it does mean that the Broncos will need to dominate the rest of its schedule to ascend past its projected rating. It would help if a few other WAC teams ascended as well via strong non-conference play, too. But there is little doubt that Boise State's overall strength of schedule will be weaker than most, if not all, of the other national title contenders at season's end. But how much weaker?

Most computer ranking systems use a simple average to calculate strength of schedule. FEI takes a unique approach to SOS calculations that can tell us much more about the relative difficulty of each team's slate of opponents. How difficult is it for a team to win every game? That is a fundamentally different question than "What is the average strength of a given set of opponents?" And it is a better one, too.

If a team played 10 average opponents, how might its strength of schedule change by playing two more teams? Winning two more games must be more difficult than not playing the games at all, even if the opponents are weak and victory is nearly guaranteed in both games. But if we used a simple average to calculate strength of schedule, then the 12-game schedule would actually be measured as weaker than the 10-game one because the addition of two weak teams would drag down the average of the entire slate. The average misleads us.

Instead, we'd like to find out the likelihood of victory in each of the games. The only problem is that the likelihood of victory depends on the strength of both teams, not just the opponent, and the ease of a schedule depends on our perspective. Teams at the top of the college football world would have more success against a bunch of average teams than they would against a schedule that included a handful of elite teams and a handful of chumps. The worst teams in the country would have more success against the latter group, however, because they'd have a shot to win a few games against the chumps.

If we want to measure the likelihood of an undefeated season, we have to take the perspective of an elite team. Only upper echelon teams can reasonably expect to go undefeated in a 12- to 14-game schedule. I chose to standardize SOS from the perspective of a fictional elite team, one with an FEI rating of two standard deviations above average, roughly equivalent to a typical top five team.

To calculate the likelihood of victory, I charted the relationship between power differential and winning percentage for both home and away teams during the last seven seasons. Power differential represents the standard deviation delta between the two teams' end-of-year FEI ratings. The best fit polynomial trendlines for those results are charted here.

Note the nonlinear relationship of these factors and the slight bump in the lines near the center of the graphic. The value of home field peaks when the teams are of relatively equal strength. Note also that a home team with a power disadvantage of roughly 0.2 standard deviations has a 50 percent win likelihood. Conversely, a road team must be better than 0.2 standard deviations in power than its opponent if it expects to win on the road.

Based on this regression model, I calculate the likelihood of victory for our fictional elite team against every game on every team's schedule. The product of those individual game win likelihoods is the overall probability that the elite team could run the table. The lower the probability, the tougher the schedule. SOS isn't permanently fixed at this point in the year. It will fluctuate throughout the season as FEI ratings change, but we get a pretty good idea now regarding the differences among Boise State and other title contenders.

As it currently stands, the Broncos total 2010 schedule ranks 103rd among all FBS teams. A typical elite, top five team would have a 57.9 percent likelihood of winning every game. And the biggest hurdle is already in the rearview mirror. An elite team would be expected to have a better than 94 percent chance of winning each of its remaining games (Oregon State is the next toughest challenge, and they have to visit the blue turf), and an 80.5 percent chance of running the table the rest of the way. That's the fifth-easiest remaining schedule in all of college football.

It isn't as though all automatic qualifying programs have tremendously difficult schedules in comparison, however. Texas and Ohio State rank 62nd and 59th, respectively, by this SOS measure. But it isn't enough to say their schedules are roughly 40 spots tougher than Boise State's. How much tougher is 40 spots?

Let's compare the schedules of Ohio State and Boise State, currently neck and neck in the Associated Press poll at No. 2 and No. 3. FEI produces a strength of schedule rating of .332 for Ohio State, a 33 percent likelihood that an elite team would win every game. If Boise State's schedule were played twice -- that is, 24 total games, two trips to FedEx Field to face the Hokies, two at home against Oregon State, etc... -- it would still be easier than the Buckeyes' 59th ranked schedule. An elite team would have a better chance of going 24-0 against Boise State's slate twice (33.5 percent) than it would going 12-0 against Ohio State's slate this year (33.2 percent).

This phenomenon is not isolated to Boise State or other teams with weak SOS ratings. A 24-0 record against Ohio State's schedule twice would still be easier (11.0 percent) than 12-0 would be against LSU's fifth-ranked schedule (10.8 percent).

Before this gets into Total Cereal territory, we can frame the difference with a little more subtlety. If Boise State replaced one non-conference game other than Virginia Tech this year with another heavyweight, their schedule strength would be on par with Ohio State and Texas. Playing a single game against either Florida or Alabama instead of Wyoming or Toledo, for instance, would produce an SOS at around .320. Adding in one more game of the caliber of Virginia Tech would improve SOS to around .400. The depth of the WAC certainly hurts the Broncos cause, and an SEC-level strength of schedule would be nearly impossible to achieve (it would require three or four elite non-conference dates, at least). But one more marquee matchup would do wonders, both in terms of popular opinion and SOS calculations.

This is not meant to be a criticism of Boise State's scheduling philosophy, but rather an objective approach to the question of schedule strength. There are huge differences in college football between conferences and team schedules. And as long as the BCS pits only two teams against one another for the title, the challenge of weighing performance against opposition requires this level of scrutiny.

FEI Week 1 Top 25

The principles of the Fremeau Efficiency Index (FEI) can be found here. FEI rewards playing well against good teams, win or lose, and punishes losing to poor teams more harshly than it rewards defeating poor teams. FEI is drive-based, not play-by-play based, and it is specifically engineered to measure the college game.

FEI is the opponent-adjusted value of Game Efficiency (GE), a measurement of the success rate of a team scoring and preventing opponent scoring throughout the non-garbage-time possessions of a game. FEI represents a team's efficiency value over average. Strength of Schedule (SOS) is calculated as the likelihood that an "elite team" (two standard deviations above average) would win every game on the given team's schedule to date. SOS listed here includes future games scheduled.

Mean Wins (FBS MW) represent the average total games a team with the given FEI rating should expect to win against its complete schedule of FBS opponents. Remaining Mean Wins (FBS RMW) represent the average expected team wins for games scheduled but not yet played.

Only games between FBS teams are considered in the FEI calculations. Since limited data is available in the early part of the season, preseason projections are factored into the current ratings. The weight given to projected data will be reduced each week until Week 7, when it will be eliminated entirely. Offensive and defensive FEI ratings will also debut in Week 7.

These FEI ratings are a function of results of games played through September 6.

FEI ratings for all 120 FBS teams are now listed in the stats page section of FootballOutsiders.com. Click here for current ratings; the pull-down menu in the stats section directs you to 2007 through 2009 ratings.

Rank Team FBS
W-L
FEI Last
Wk
GE GE
Rk
SOS SOS
Rk
FBS
MW
FBS
RMW
1 Florida 1-0 .270 1 .270 14 .158 19 9.2 8.2
2 Alabama 1-0 .223 3 .556 5 .135 11 8.2 7.2
3 Texas 1-0 .209 2 .207 20 .353 62 10.4 9.4
4 Ohio State 1-0 .205 5 .526 7 .332 59 10.1 9.1
5 Georgia 1-0 .202 9 .605 2 .155 17 8.0 7.0
6 Oregon 1-0 .200 12 .919 1 .292 49 8.8 7.8
7 Virginia Tech 0-1 .197 4 -.035 46 .185 27 8.2 7.6
8 Georgia Tech 0-0 .192 6 n/a n/a .113 6 7.6 7.6
9 West Virginia 0-0 .183 10 n/a n/a .366 65 8.9 8.9
10 Clemson 1-0 .181 11 .357 10 .138 12 7.5 6.5
11 LSU 1-0 .179 14 .071 31 .108 5 7.2 6.6
12 USC 1-0 .176 7 .271 13 .318 55 10.1 9.2
Rank Team FBS
W-L
FEI Last
Wk
GE GE
Rk
SOS SOS
Rk
FBS
MW
FBS
RMW
13 Boise State 1-0 .176 15 .035 33 .579 103 10.9 10.5
14 Penn State 0-0 .172 13 n/a n/a .208 31 8.2 8.2
15 Oklahoma 1-0 .168 8 .083 29 .267 40 9.0 8.1
16 Auburn 1-0 .151 16 .224 19 .166 21 6.9 6.0
17 Miami 0-0 .146 17 n/a n/a .102 3 6.1 6.1
18 Florida State 0-0 .146 18 n/a n/a .129 8 6.2 6.2
19 Boston College 0-0 .142 19 n/a n/a .275 41 7.3 7.3
20 Iowa 0-0 .142 21 n/a n/a .322 58 7.7 7.7
21 North Carolina 0-1 .142 20 -.071 48 .144 13 6.3 5.9
22 TCU 1-0 .140 23 .107 25 .688 116 9.7 8.9
23 Texas Tech 1-0 .135 22 .095 27 .362 64 8.0 7.0
24 Notre Dame 1-0 .134 24 .132 22 .284 47 8.5 7.7
25 Nebraska 1-0 .129 29 .488 9 .449 82 8.1 7.1

Posted by: Brian Fremeau on 08 Sep 2010

42 comments, Last at 17 Sep 2010, 1:30pm by cfn_ms

Comments

1
by Big Johnson :: Thu, 09/09/2010 - 1:03am

how heavily does preseason rankings affect future rank? The SOS column seems extremely useless at this point because we have had half a week to decide who is strong and who isnt. So if preseason rank has anything to do with future SOS then it would be extremely hard to break rank by the end of the season. Ive always wondered this with the NFL as well (although not as much due to more connectivity between teams) but with college it seems something as simple as that can keep a conference on top, and keep a team from winning any tie breakers over any other teams with the same record.

5
by Brian Fremeau :: Thu, 09/09/2010 - 7:29am

Preseason projected FEI ratings will influence FEI through the first six weeks of the season. So yes, the SOS ratings are subsequently influenced at this point. But while there will certainly be change and some fluctuation, we do have a pretty good picture of where Boise State's SOS stands and will stand relative to the elite opponents.

By Week 7, all projected data will be removed from the FEI and SOS calculations.

13
by TV_Pete (not verified) :: Thu, 09/09/2010 - 11:36am

Personally, I think the combined Polls should be no larger of a factor than the FEI and Sagarin Ranking (Predictor is a little variable and ELO-Chess used now is forced to ignore margin of victory and too much else).

So, yes, my ideal BCS rankings would be:

1 part Polls (Harris, AP and Coaches with votes released for all of them that count)

1 part Computer Ranking (FEI and Sagarin) - nice and non-deterministic, used by Week 7.

Of course, I wouldn't mind a classic bowl structure through January 1st or 2nd and then run the computer rankings and polls and play the #1 vs. #2 after some decent out of conference competition. Let Boise State play in the Sugar Bowl against 'Bama and then see how the rankings fall.

We don't need a playoff, which makes the regular season games less meaningful. What was the last great (season-changing) regular season game you remember from the NHL, NFL, NBA, College Basketball, or MLB? There aren't any, in my mind, since the only thing that matters is qualifying for the tournament at the end of the season.

14
by jints2468 :: Thu, 09/09/2010 - 11:49am

The BCS doesn't want a predictive/true talent computer rankings. They want a computer system that grades what has happened on the field. Hence, the difference between Sagarin's PREDICTOR and ELO_CHESS rankings.

41
by TV_Pete (not verified) :: Wed, 09/15/2010 - 12:19am

I think what they really want is a grade system that matches what they want to see in the polls.

The Sagarin ranking and FEI predictor may be better rankings, in my mind, and are certainly less biased.

42
by cfn_ms :: Fri, 09/17/2010 - 1:30pm

I think that what the BCS wants is a set of computer systems that gives results that aren't horribly unreasonable, that they don't have to monitor or interact with, and that isn't hugely controversial with the public (which is why they slice out models seemingly any time there's an outcry over one or more).

In that sense, they've got exactly what they want. Of course, most of us wish there was a higher set of standards they'd want to adhere to, but unfortunately that's not the case.

2
by cfn_ms :: Thu, 09/09/2010 - 2:32am

Are you really saying that Oregon St only has a 1 in 20 chance of winning at Boise? That seems like a pretty solid stretch IMO. Yes, Boise SHOULD win that game, but I'd say more like 1 in 8 upset chance - a meaningful spread but far from a lock.

Potentially ditto for Nevada, though I'm less sure about their end.

I think that it's quite premature to substantially speculate that Boise will run the table, as well as what the records of the top AQ teams would be. We could see another 2 undefeated teams like 2009, 2005, 2004, or 2002. Or we could see only one AQ team with less than 2 losses, like 2007 (Kansas's schedule was such a joke they were basically a non-AQ, missing OK, Texas, and anyone w/ a non-conference pulse) or 2006. In other words, even if Boise does run the table (FAR from guaranteed), history seems to indicate that it's very possible they'll either be obviously shut out (b/c there will be 2 undefeated AQ's) or obviously in (if there's one or zero AQ's with fewer than 2 losses).

And, of course, that completely ignores TCU, who also has to be in the discussion if they run the table.

6
by Brian Fremeau :: Thu, 09/09/2010 - 7:41am

According to the relative FEI ratings of both teams, and the regression of program power versus win likelihood, yes. Boise State currently has a 94 percent chance of beating Oregon State. (If BSU were rated more highly by FEI, it would be higher of course).

Yes, it is way to early to talk about guarantees -- the intro paragraph was a bit of an exaggeration. But an 80 percent likelihood of running the table is very very high for this point in the season.

I did ignore TCU for this particular commentary, and their SOS is actually rated as weaker than Boise State's right now. A breakdown of TCU would be very similar at this point.

3
by cfn_ms :: Thu, 09/09/2010 - 2:41am

1) Why is Florida #1? Do week 1 numbers just not mean much if you were high on a team before the year? Or did the FEI numbers actually like what the Gators did?

2) Why are you only showing SOS through the end of the year? Shouldn't you also show SOS for games played through now? Or is it just early for those numbers to mean anything?

3) Do you "believe" your numbers now, or does it take a few weeks before you really think they're solid?

4) LSU got a bump despite playing close against a UNC team that was missing a boatload of players. Is this something FEI adjusts for?

5) Why was Oregon's GE rating almost a 1.0, while Bama's was not much more than 0.5? Was there really that much of a meaningful difference between 72-0 and 48-3? Did Bama do something "wrong" in their game, or do you just have to totally and completely annihilate an opponent before FEI really likes the in-game numbers?

7
by Brian Fremeau :: Thu, 09/09/2010 - 7:54am

1. Florida was #1 by such a wide margin to start the year, their week 1 performance didn't impact their ranking too much. Preseason projected data will influence these ratings over the first six weeks of the year. This is because there is little data to work with in the early part of the season and projected data has a strong correlation to next year performance so it is a good substitute.

2. SOS could theoretically be calculated for only games played to date (Boise State has the best 1-game SOS for 1-0 teams so far). I'll consider adding a separate column next week.

3. The numbers are going to be relatively consistent throughout the season. By last week's Projected Win Expectations (based entirely on projected data), it did well. About as expected: http://bcftoys.blogspot.com/2010/09/fei-forecasts-week-1-sep-2-6.html

4. FEI does not account for the depleted UNC team. LSU and UNC got a little bit of a bump for simply playing a close game against one another.

5. Oregon's GE was extreme in the game. They scored touchdowns on 8 of their 11 first half possessions. Alabama scored touchdowns on only 4 of 8 first half possessions and gave up a field goal themselves. GE can be pretty volatile through the early part of the season, which is why projected data helps.

31
by cfn_ms :: Fri, 09/10/2010 - 3:22am

For point 5, GE will certainly be volatile since it's just one game, but I think it's extremely instructive as to what the model is actually doing. Assuming I'm reading it right, 1.0 is the max you can get, with 0.0 being a tie (not allowed in CFB).

Personally, I think it's just weird that a 48-3 Alabama win would get less than a 0.6 on that scale. It's the sort of result which makes me think you're giving substantially too much weight to extreme blowouts. Is there any evidence that winning 72-0 vs 48-3 is meaningful in terms of predicting future success? I would suspect the answer is no, but I could certainly be wrong.

Also, while we're at it, I saw that Bama's 48-3 win was given a lesser game rating than South Carolina's 41-13 win. Given that Bama's halftime lead was bigger, and the net yardage difference was MUCH bigger (SC's was almost a tie), I have to ask how that one came about. Was it because SC scored on almost every possession? Something else? Because that's something which doesn't quite seem right when looking at the games side by side.

4
by Kibbles :: Thu, 09/09/2010 - 2:52am

Good article, Brian. This is exactly why I'm pissed off that Boise State won. It'd be like if Florida seceded from the SEC and scheduled no one but creampuffs next season. Yes, they'd go undefeated. Yes, they'd have a recent track record. But should they really go to the National Championship for doing something that 25 other teams in the nation could have easily done (going undefeated against a joke of a schedule)? Absolutely not.

Out of curiosity, in anticipation of an "undefeated Boise vs. 1-loss _____" argument, could you calculate the likelihood of Boise State going undefeated and compare it to the likelihood of several of the top teams ('Bama, Florida, OSU, Texas) finishing with 1 loss? In other words, can you figure whether it'd be more impressive to finish with just 1 loss against 'Bama's schedule or to finish with no losses against Boise's? How about 2 losses? 3 losses? How many losses does 'Bama have to suffer before we can say that an undefeated Boise State team accomplished a more difficult task?

8
by trump (not verified) :: Thu, 09/09/2010 - 8:23am

I agree....it would be more interesting to know comparative difficulties of Boise State undefeated vs. other top contenders w/ one loss, as inevitably that is where the controversy will be. If Ohio State, Alabama, etc. go undefeated as well as Boise State, no one will call it a controversy if Boise State is ranked lower.

11
by gaaaah (not verified) :: Thu, 09/09/2010 - 10:56am

The thing is, if a team can go undefeated and should not have a chance for the National Championship, why even let them into Division I?

In a perfectly fair world with no money involved, there needs to be some serious contraction of Division I. Cut it down to 40 or so teams, make them play most of their games against each other to minimize cupcakes, and if there's only one or two undefeated teams, put them in the damn championship game.

Granted, there's so many ways this would never work (Mainly money) that it'll never happen. But I'm just frustrated with perfection being meaningless because of these schedule issues. Winning the right to play in the 5th place game doesn't seem like a worthy reward for it.

20
by Portmanteur (not verified) :: Thu, 09/09/2010 - 2:08pm

Don't worry, in the near future there will be four sixteen-team superconferences. And it will work mainly because of the money, not in spite of it.

17
by mysm (not verified) :: Thu, 09/09/2010 - 12:49pm

Kibbles has it backward. If Boise can go undefeated, and beat at least two good teams (one of them two time zones away in a virtual away game) then why SHOULDN'T they get a shot at the NC? I keep reading how trivial it is to run the table against a schedule like this, and yet only Boise, TCU and Utah seem to do it. Then all the big-conference partisans announce how overrated they are. Then the guppies beat a whale in the bowl game, and we hear how it doesn't mean anything. Lather, rinse, repeat.

FEI continuously underrates teams like Boise, because it measures what they have (allegedly) proven but NOT what they are CAPABLE of. It is telling that the combined metric of FEI/+ does better than FEI at predicting games. Va Tech is still ranked 6 spots above BSU after losing at home to them. That is silly.

And before y'all start calling me a Bronco, I've never been within 500 miles of Boise. I'm a Georgia Tech grad/fan, and I'm telling you that Boise could have easily gone undefeated last year against the ACC. If you think otherwise, then you're simply not paying attention.

21
by Portmanteur (not verified) :: Thu, 09/09/2010 - 2:20pm

Also a Tech grad, but I disagree that we should be deciding who should be playing for the national title based on what we think teams may or may not be "capable" of. By the end of the season, all 120 teams should have proven their worth. If Boise State didn't play enough good teams to prove that they're one of the top two teams, then why should we just take their word for it? And isn't putting a team in the national title based on what one believes them to be capable using the same flawed logic of preseason polling? How can anyone say Alabama is the number one team in the nation when they haven't proven it on the field? How could you say at the end of the season that Boise State belongs in the championship game when they won't have proven it on the field, either?

23
by Eddo :: Thu, 09/09/2010 - 2:59pm

I think you're right; we shouldn't base this on capability, but actual results.

And if Boise State is the only undefeated team, then every other team has proven that they can lose, whereas Boise is only capable of losing. Ranking any one-loss team ahead of Boise State is simply based on that team's capability, as opposed to their results, isn't it?

27
by mysm (not verified) :: Thu, 09/09/2010 - 8:10pm

I wasn't clear enough the first time. Eddo's post is better, and shorter. But I also want to point out that EVERYBODY is assuming capability. BCS conference people ASSUME that BCS schools are capable because, well, they're BCS conferences. Their games are ASSUMED to be harder by FEI. It flirts with circular reasoning.

28
by cfn_ms :: Thu, 09/09/2010 - 10:09pm

by mid-season, no preseason assumptions factor into numbers. therefore if the model thinks that the SEC is better, it's because the results on the field against other conferences support that idea. nothing circular about that.

32
by Eddo :: Fri, 09/10/2010 - 10:18am

Preseason assumptions may not factor into the numbers portion of the BCS formula (or FEI, but that's irrelevant to the BCS), but they sure have a large effect on the polls, which are weighted twice as much. Why do you think Texas played for the national title last year, instead of Cincinnati (whom the computers liked better)?

36
by cfn_ms :: Sat, 09/11/2010 - 2:31pm

Probably because the voters could factor in the fact that Cincy won by the hair of their chinny-chin-chin against:

UConn
WV
Pitt

and didn't exactly dominate:
Oregon St
Fresno
Illinois

Meanwhile Texas had a schedule probably in the ballpark of Cincy's and waxed most of their foes (OK and Neb were close, but all others were double-digits, including a bunch 20+)

37
by Eddo :: Sun, 09/12/2010 - 10:22am

Fair enough, and 2009 was probably a bad example on my part. But it's tough to deny that poll inertia is a real factor.

38
by cfn_ms :: Mon, 09/13/2010 - 1:31am

From week to week, I'd agree. But I have yet to see ANY convincing argument that preseason polls have any meaningful causal relationship on final polls. They correlate reasonably well, mainly because the pollsters are, in fact, better than guys randomly picking names out of a hat (that's not to say they're unbiased, they are, just that the biases are different than hanging on to preseason rankings).

It's actually VERY hard to find a decent example of this. Fans typically blame this when they don't like final rankings (i.e. "we'd have been higher, but those pollsters screw teams originally ranked poorly, nevermind the warts on our resume, it's totally the pollsters' fault"). And so it's become a popular meme.

IMO this really picked up steam with 2004 Auburn, whose fans are convinced that:
1) they were screwed
2) it was because of preseason polls

Unfortunately for the argument:
1) they weren't screwed (USC and Oklahoma had MUCH tougher schedules; the SEC was down in 2004 and Auburn's OOC was a mammoth joke)
2) they actually PASSED Oklahoma for a week or two in the polls, but dropped back down after they struggled in some close games (making it tough to back up the "poll intertia" argument)

Of course, they're married to the idea that they really DID deserve that title shot, despite a substantial amount of evidence to the contrary. And so the meme picks up momentum. Though to be fair, they're far from the only fanbase who incorrectly subscribes to the theory, just the one that seems to be the most insistent on it.

39
by Eddo :: Mon, 09/13/2010 - 9:50am

You make some really good points, and I definitely couldn't find a very good example (I noticed all the same things about Auburn when I looked to see if they fit).

I disagree with the following (albeit in an academic way):
"From week to week, I'd agree [that poll inertia exists]. But I have yet to see ANY convincing argument that preseason polls have any meaningful causal relationship on final polls."

I think that, if each individual week correlated with the previous week due to poll inertia, then the final polls *must* correlate with week one in that sense. Week one leads to week two, week two leads to week three, etc, all the way to week (final - 1) leading to the final poll. Wouldn't the transitive property of polling(TM) apply?

(* OK, now I'm being pedantic. In my heart, I realize that you are totally correct in the practical, big-picture sense, cfn_ms.)

40
by cfn_ms :: Mon, 09/13/2010 - 2:08pm

(note: the following is pure speculation, not backed up by research, but I'm pretty sure I'm right)

I would say that, for most weeks, a highly-ranked team will either have a bye or convincingly beat a team they were much better than (which pollsters seem to treat as a bye more or less). So they basically stay in place, maybe getting passed by teams on the way up or on the way down. So in that sense, the week to week rankings correlate pretty well.

However, pretty much every team has at least a couple of games that are more meaningful, either because they lose, struggle against someone much worse, or because they've beaten someone legitimately good. In those weeks, you can see more of a movement in the rankings for that team, and it's really those weeks that define a team's ranking.

In the case of 2004 Auburn, for instance, their resume was positively defined by bludgeoning Tennessee early, and maybe a couple of other games that I forget. And they were also the beneficiary of Oklahoma struggling against a couple of Big 12 teams they shouldn't have had issues with, which was why Auburn passed them for a week or two. But then Auburn's resume was negatively impacted by their way too close win against a poor Alabama team (and maybe one other close win, I forget), and at that point, the distance between them and Oklahoma was small enough that they could be dropped back down. And since nothing particularly bad came up for Oklahoma, the rankings then stayed the way they were.

So while week to week there's decent correlation (especially starting mid-season; the early season rankings are more fluid b/c people tend not to feel extremely strong about where they put teams before anyone has developed a resume), by season's end every team has had enough resume-defining moments that it's hard to really see their final rankings as being a function of their initial ranking (and for a team like Boise, who doesn't have enough resume-defining moments, they still will generally get passed, either on the way up or the way down, by enough other teams to make their initial ranking not especially meaningful).

P.S. In your guts, you know I'm nuts :P

30
by Brian Fremeau :: Fri, 09/10/2010 - 1:25am

I ran Kibbles' suggested comparison with the following results. Taking Florida's schedule, an elite team would have a 50.5% likelihood to finish 11-1 or better, and a 81.1% chance of 10-2 or better.

Taking the current FEI No. 1 schedule, Mississippi State's, an elite team would have a 38.0% chance of finishing 11-1 or better and a 71.8% chance of 10-2 or better.

Taking Ohio State's schedule, an elite team would have a 73.7% chance of 11-1 or better.

So, based on this methodology, it's safe to say an undefeated Boise State season would be less difficult to achieve than an 11-1 season in the SEC, but more difficult than a 10-2 SEC season or an 11-1 "average schedule" season.

33
by Jeff Fogle :: Fri, 09/10/2010 - 1:10pm

Really like this way of looking at it. But, am I right BF that we're only looking at the strength of opponents here rather than the difficulty of potential sequences? So, playing a road game at South Carolina the week after a bye would count the same as playing a road game at South Carolina immediately on the heels of playing at Arkansas and vs. Florida the prior two weeks...or playing a road game at Tennessee after a bye would count the same as playing at Tennessee in the fifth straight week of conference wars with the sequence: at Ark, vs. Fla, at SC, vs. Ole Miss, at Tennessee?

To me, there's much more "difficulty" in playing an SEC schedule than a WAC schedule that goes beyond the "strength" of the opponent on a given day. You have to have a "depth of power" so to speak that allows you to keep hitting people in the fatigue spots. Not saying Boise doesn't have this to some degree. But, if the methodology suggests that Boise State going undefeated is "more difficult than a 10-2 SEC season," but doesn't account for the grueling nature of playing very physical games over successive weeks without a break...then it may not be meaningful enough to persuade in the debate.

Alabama doesn't just have to beat all of its opponents. It has to beat them in the midst of an exhausting gauntlet that Boise State doesn't have to face. Boise's big games are relatively rare, and spread out. Beating Virginia Tech was certainly impressive. Could they do it three weeks straight. Or, could they beat Arkansas, Florida, and South Carolina in succession...then stay up for Ole Miss and Tennessee. With three of the five on the road? Maybe.

All teams have physical and emotional letdowns. Handicappers have been taking advantage of this for years. But, Boise's letdown comes against Cal-Davis (last year) or New Mexico State or somebody in a game they're going to win anyway. The eventual SEC champ will have had to deal with tougher challenges when flat.

I'm tempted to suggest adding this sort of element into the model mix here. But, there's already a pretty good spread between BCS conferences and the mid majors. Plus, it will just make the ACC look stronger in FEI because of the games against each other, and I'm still of the view that FEI (mysteriously) over-ranks ACC teams. But...if the conversation is about "strength" of schedule and "difficulty" of schedule, "difficulty" goes beyond just the ratings of the opponents in my view.

I know I've made this case before. New people in the discussion...and I think this point needs to be on the radar again this year now that Boise's going to be in BCS discussions the whole way. And, if we're trying to figure out the "percentage" chances of whether going 10-2 vs. an SEC schedule is harder than going 12-0 with a WAC schedule plus VT/Oregon State...that gauntlet element should be considered.

34
by Kibbles :: Sat, 09/11/2010 - 4:43am

Thanks Brian, that jives with what I intuitively felt.

Quick question, though. Were you including the possibility of an SEC Championship Game in the SEC schedule, and if so, how?

9
by Jetspete :: Thu, 09/09/2010 - 8:46am

There are factors that no one is taking into consideration in terms of schedule. If Boise played LSU's schedule, theoretically they would be a Southern school and have access to a greater talent pool than what they have in Idaho. Plus, geographically there are more major conference schools in the East than in the West. Just because they dont play a "top 6" conference schedule does not mean they are not deserving of a trip to a national title game.

And for the record, Boise is playing Oregon State this year, a team that has been in the Pac 10 title discussion for two years now. That is a second good non-conference game. But let's say they played LSU in early October, at no point does that still guarantee them a spot in the national title game because of voter indifference to the lesser conferences. Case in point, TCU had good wins last year and they were left out of the title game. Voters control the BCS. If this was 2004 and the formula was less voter defined, than yes i agree Boise should play a second "big game." But we should not penalize Boise for adhering to a system that was created to keep them out!

10
by Muldrake (not verified) :: Thu, 09/09/2010 - 10:18am

But they aren't in the South and don't have a greater access to talent pools. They shouldn't be given a bonus in the polls just because they're in Boise.

As for adhering to a system that is designed to keep them out, Boise stepped up from FCS and their playoff system to FBS after the Bowl Coalition (predecessor to the BCS) was in place. They aren't like TCU, which had the BCS thrust upon them. Boise elected to join a system they knew would be stacked against them. So they need to schedule to impress, and their continued refusal to do that by insisting on huge paychecks or 1-1 games.

12
by Thunderbolt of ... :: Thu, 09/09/2010 - 11:25am

Agreed. If Bethune-Cookman was a FBS school, they'd probably recruit better, but they aren't. I want to see the best two college football teams on the field in the BCS Championship, regardless of the (legal... ahem, USC) circumstances that led to their success.

24
by MTR (not verified) :: Thu, 09/09/2010 - 4:06pm

You make it sound like Florida and Ohio State are begging to play Bosie, or they refused invitations to join the Pac 10 and Big 10. Come on, all the real power teams play cupcakes outside their conference. The only exceptions are huge TV paydays, like USC-Notre Dame. The elite schools want nothing to do with Bosie or TCU - there's no money or prestige for winning and their season is torpedoed if they lose.

35
by Kibbles :: Sat, 09/11/2010 - 5:01am

Two years ago when Utah went undefeated and was shut out of the championship game in favor of two 1-loss teams, it originally had Texas on its schedule. That game was dropped... at UTAH'S request. So there are obviously some holes in this "the big boys want nothing to do with the mid-majors" stuff.

Look, the simple fact is that the big schools are all about the payday. They'll play anyone if the money is right. The biggest problem that they have is that the Boise States of the world are demanding home-and-homes, which means less money for the big schools. If Boise State agreed to a 1-way trip with cash compensation similar to (or even greater than) the patsies most big schools are already scheduling, those big schools would JUMP at the chance. Boise State doesn't want to do it because Boise State doesn't want to play 3 non-conference road games. To that I say "tough". You want to earn some respect with that joke of a conference schedule, you better have the toughest non-conference schedule in the nation.

Personally, I have no sympathy for the Boise States and the Utahs of the world, always whining that things aren't given to them. You know what? 30 years ago, Miami and Florida State were on the outside of the power structure looking in, too. You know how they became respected powerhouses? They scheduled with an "anyone, anywhere, anytime" philosophy, and then they backed it up on the field. None of this "one big-time non-conference game and then hope to backdoor your way into the championship game" nonsense. Take a look at Florida State's schedule in 1981, for instance. 6 road games vs. 5 home games. 5 games against top-25 opponents. 18th ranked SoS. In 1982, their SoS was 20th. In 1983, their SoS was 5th, and they again played more road games than home games. In 1984 it was 10th. In 1985 it was 22nd. Their schedule was one of the 10 toughest every single season from 1987 to 2000. Miami went through exactly the same process. Neither team would have DREAMED of begging their way into the championship game with the 100th toughest SoS in college football.

I'm not a fan of the whole "good ole' days" nostalgia thing, but in my opinion, if a program wants respect, that's how you go out and get it. Schedule more than 1.5 tough games a year, go on the road if you have to, and win more than you have any business winning.

It's not that there's no mechanism in place for Boise State or Utah or BYU or TCU to win a championship. It's that none of those teams find the mechanism in place all that appealing, and they'd rather everyone make it easy for them. National championships aren't supposed to be easy.

15
by Alexander :: Thu, 09/09/2010 - 12:42pm

I'm slightly confused, so I'm gonna make an assumption: That these early season rankings are not like VOA at the beginning of the NFL season. The formula said Boise did better(barely) than VT in the game correct? VT is ranked higher because they have something anchoring them higher right?

If we pretend that this is week 7, and VT played Boise 7 times with the exact same plays, Boise would be ahead of VT correct?

16
by dbostedo :: Thu, 09/09/2010 - 12:46pm

As stated above, the pre-season rankings are still a factor. So VT started high enough that they are still ahead of Boise State. Once we're 7 weeks in, the preseason rankings won't count for anything and things may have shaken out by then.

22
by Alexander :: Thu, 09/09/2010 - 2:49pm

Thanks, just trying to clarify.

18
by Joseph :: Thu, 09/09/2010 - 12:53pm

To follow up on some of these comments:
If you were a voter, and you have undefeated Boise St., and say, a 1-loss Big 10 team (OSU), a 1-loss SEC team (FLA/ALA), and a 1-loss Big 12 team (Texas)--do you have Boise St. vs. 1 of these others for the championship? [I say yes--what say you?]

19
by zlionsfan :: Thu, 09/09/2010 - 1:54pm

I'm not convinced another marquee matchup would make that much of a difference. Doubters will simply say the same things after that game they're saying now (Boise State didn't dominate, they played a sloppy game, etc etc); they'll attribute the outcome to the BCS team having an off night or something like that. The WAC portion of their schedule will come up, and the polls that are based on random people checking out box scores and ordering teams based on whatever crosses their minds that day will gradually drop the Broncos and teams like them no matter who they played to start the season.

And if the Broncos were to lose that game, they'd be cast back into the pile of pretenders regardless of how they finished the season. It doesn't matter if that team were actually good enough to compete in the SEC that year: the loss would be taken as "proof" that they weren't.

The BCS wasn't set up to be fair, it was set up to pair two power-conference teams to make more money. All the in-season buildup is simply a charade to fool people into thinking that it's similar to championships in other sports ... at the end, it comes down to who coaches (or assistant SIDs or whoever actually fills out the ballots) and writers want to pick, not who's the best or who has the best resume or whatever. It's great for arguments but terrible for determine a champion.

25
by Jeff Fogle :: Thu, 09/09/2010 - 5:47pm

Any thoughts from anyone (authors included) on where Florida "really" stands in the big picture right now. You'd think a yardage dead heat with FEI #79 would basically be "signature significance" that Florida isn't #1 (I understand why FO methodologies would still have them there because they're backloaded with prior data and respect for recent program performance). Couldn't imagine a similarity score type approach that would find a superpower who gained about 25 yards through three quarters of their first game...particularly a home game vs. a poor opponent.

Do they fall like USC and OU did last year...to about 4 losses but still getting respect in various methodologies because of overall team talent? Do they fix the offensive line woes (more than just snaps...you'd have to figure they'd fix snaps!) and return to top five in a finger snap?

People kept expecting USC and OU to right the ship, yet inconsistency plagued them throughout the season.

Is Florida top 5? About 20th? Could they drop even further?

Lacking earlier examples makes this an interesting case study I'd think. And, it may end up showing the dangers of ranking on program performance early in the season just because outliers can happen at the top. Some stuff's just unpredictable.

29
by cfn_ms :: Thu, 09/09/2010 - 10:11pm

I'm inclined to think they're somewhere between 10 - 15 based on what I saw, but I could definitely be wrong. It's just so early now, who the hell really knows anything? Florida could fix OL issues, run the table and win it all. Or they could lose five. Or anything in between.

26
by jpeta :: Thu, 09/09/2010 - 7:01pm

Brian, Why no listing this year of OE and DE by team as well as FPA rank? Those were very insightful metrics to look at upcoming match-ups and, in the case of FPA, sometimes revealed unsustainable performance. You do great work.