Writers of Pro Football Prospectus 2008

Most Recent FO Features

ReplayIns14.jpg

» Scramble for the Ball: Getting it Right?

Instant replay review is one of the cornerstones of the modern NFL. The process and its myriad special rules have been internalized and constantly debated. Mike Kurtz wonders: is it worth it?

08 Jan 2014

Final 2013 FEI Ratings

by Brian Fremeau

There was a moment late in the second quarter of the BCS Championship game that many point to as the most pivotal play of the game. Florida State was trailing Auburn 21-3 and quarterback Jameis Winston had just thrown an incomplete pass on third-and-4 at the FSU 40-yard line. The Seminoles had been outgained by about 100 yards to that point in the game, an almost identical score and yardage deficit faced by Notre Dame in a blowout loss to Alabama in the BCS title game one year earlier. Florida State appeared to be flopping in a similar manner on the biggest stage of the year.

And then Jimbo Fisher called a fake punt, and Karlos Williams sprinted for a first down. Winston added a spectacular 21-yard scramble later in the drive and Florida State scored one play later to cut the deficit to 11 and head into halftime with a little bit of confidence.

That fake punt doesn’t register with FEI. It was one of several important plays in an 11-play, 66-yard touchdown drive, but the total value of the drive wouldn’t have been different in the eyes of FEI if FSU had converted on third down, threw a 66-yard touchdown pass at the start of the possession, or any other combination of plays to reach the end zone.

It was the most valuable possession for Florida State in the first half, but the Seminoles had three key possessions in the second half of the game that were even more significant. Florida State ultimately won the national championship by earning its most valuable offensive, defensive, and special teams possessions in the second half of a game that featured big plays from both teams in all three phases.

Defense

After FSU added a field goal on its second drive of the second half to pull to within eight points, the Tigers and Seminoles traded a pair of three-and-outs. Auburn’s punt team outperformed Florida State’s in that sequence, and Auburn started its next drive on its own 43-yard line. It was the Tigers’ third-best starting field position of the night and Auburn had converted both of its other short field opportunities into touchdowns. On the season, Auburn had scored an average of 3.9 points per drive on possessions started within 60 yards of the end zone.

Florida State allowed Nick Marshall to run 13 yards into FSU territory on the first play of the drive, but then stonewalled the Auburn offense on its next three plays to force a punt. Not many highlight reels of the championship game will feature plays on that particular possession because of the fireworks that followed, but it was the single most valuable defensive sequence of the game, denying valuable field position and earning 2.3 points of scoring margin value according to the Game Splits formula.

Special Teams

Auburn started its drives almost 200 yards closer to the end zone than Florida State in the game due to strong special teams play on punts, kickoffs, and returns. A missed field goal for the Tigers in the first half proved to be a significant special teams miscue, but not as significant as the Auburn kickoff late in the fourth quarter after Auburn took a 24-20 lead.

Kermit Whitfield streaked 100 yards down the sideline, untouched to the end zone and Florida State’s first lead of the game. According to the Game Splits formula, Whitfield's play was worth a total of 5.4 points, the most valuable single event of the game. Florida State’s kickoff return efficiency ranking jumped from 43rd before the bowls to No. 4 after the bowls based on that spectacular play. FSU’s overall special teams efficiency jumped 24 spots. Auburn’s kickoff efficiency dropped 27 spots. It was a big play.

Offense

Auburn answered with a 75-yard touchdown drive and reclaimed a 31-27 lead with just under 90 seconds left in the game. The Tigers special teams stepped up on the ensuing kickoff, tackling Whitfield at the Seminoles’ 20-yard line, 80 yards from victory. Winston was outstanding down the stretch, connecting on 6-of-7 passes including a quick strike to Rashad Greene that he took 49 yards on the second play, and a touchdown pass to Kelvin Benjamin on first-and-goal at the 2-yard line with only seconds left in the game.

Based on Game Splits, Florida State’s final 80-yard touchdown drive was worth a total of 5.9 points of scoring value, more than tripling the total scoring margin value FSU had earned on offense to that point in the game. Auburn’s defense held Florida State to its least productive offensive game of the season, but Winston and the Seminoles delivered in the biggest spot.

Legacy

Florida State’s win ranks 44th nationally in terms of opponent-adjusted single game efficiency (GFEI), the Seminoles’ seventh top-50 single-game performance of the year. No other team had more than three top-50 GFEI games this year. In the last four seasons, only the 2011 LSU Tigers also posted seven top-50 GFEI performances in a single season. Only ten programs have posted at least seven top-50 GFEI performances in the last four years combined.

Top 50 GFEI Performances since 2010
Team 2010 2011 2012 2013 Total
Alabama Crimson Tide 6 5 5 2 18
LSU Tigers 2 7 2 2 13
Stanford Cardinal 5 3 1 3 12
Oregon Ducks 2 2 5 3 12
Oklahoma State Cowboys 1 4 2 2 9
Oklahoma Sooners 1 3 2 3 9
Florida State Seminoles 1 0 0 7 8
Auburn Tigers 4 0 0 3 7
South Carolina Gamecocks 5 0 1 1 7
Georgia Bulldogs 1 1 4 1 7

The Seminoles don’t go down in FEI history as an all-time top team due to overall schedule strength, however. Florida State’s strength of schedule, including the win against Auburn, ranks 65th. In the SEC’s seven year run of BCS championships, the lowest SOS ranking of any of those title teams was 2012 Alabama, with the 31st toughest schedule (hampered in part by FEI’s unfavorable rating of the Crimson Tide’s title game foe, Notre Dame).

Florida State’s 2013 season does rank 10th in the "all-time" FEI ratings dating back to 2003, and they are kings of the college football world at the end of the BCS era. Next season welcomes a new four-team playoff format to FBS, a long-awaited change in a two-team postseason structure that has prevented many strong contenders over the years from having an opportunity to settle the national championship debate on the field.

For everyone that claims that the entire college football season is the playoff, I threw together an official bracket for 2013 over the weekend. It works, and yet it is a ridiculous exercise, designed retroactively to fit imperfect results into a perfect conclusion. Kind of like the BCS.

Click here or on the image below for a larger version of the 2013 College Football Playoff Bracket.

FEI Final Top 25

The Fremeau Efficiency Index (FEI) rewards playing well against good teams, win or lose, and punishes losing to poor teams more harshly than it rewards defeating poor teams. FEI is drive-based and it is specifically engineered to measure the college game. FEI is the opponent-adjusted value of Game Efficiency (GE), a measurement of the success rate of a team scoring and preventing opponent scoring throughout the non-garbage-time possessions of a game. FEI represents a team's efficiency value over average.

Other definitions:

  • SOS: Strength of schedule, based on the likelihood of an elite team going undefeated against the given team's entire schedule.
  • FBS MW: Mean Wins, the average number of games a team with the given FEI rating would be expected to win against its entire schedule.
  • OFEI: Offensive FEI, the opponent-adjusted efficiency of the given team's offense.
  • DFEI: Defensive FEI, the opponent-adjusted efficiency of the given team's defense.
  • STE: Special Teams Efficiency, the scoring value earned by field goal, punt and kickoff units measured in points per average game.
  • FPA: Field Position Advantage, the share of the value of total starting field position earned by each team against its opponents.

Overall FEI ratings and FEI splits for Offense, Defense, Special Teams, and Field Position can be found using the drop-down menus provided above. Program FEI (five-year weighted) ratings and other supplemental drive-based data can be found here.

Rk Team FBS
Rec
FEI GE GE
Rk
SOS Rk
FBS
MW
OFEI Rk DFEI Rk STE Rk FPA Rk
1 Florida State 13-0 .319 .466 1 .236 65 12.1 .499 12 -.653 5 1.343 27 .562 2
2 Stanford 11-3 .311 .183 10 .030 6 11.5 .412 22 -.742 1 3.352 2 .561 3
3 Alabama 10-2 .289 .270 4 .097 28 10.1 .551 9 -.555 7 3.457 1 .560 4
4 Auburn 11-2 .271 .126 25 .033 7 10.1 .586 7 -.475 10 2.184 7 .516 44
5 Missouri 11-2 .263 .165 15 .072 23 10.4 .437 18 -.529 8 .085 65 .533 21
6 South Carolina 10-2 .257 .120 26 .099 30 9.7 .696 2 -.468 11 -2.201 114 .473 95
7 Oregon 10-2 .254 .285 3 .077 25 9.7 .612 4 -.422 16 1.379 26 .549 9
8 Michigan State 12-1 .252 .187 9 .108 31 10.8 .267 33 -.733 2 1.292 29 .558 5
9 Oklahoma State 9-3 .245 .154 17 .153 43 9.6 .397 24 -.717 3 -.451 85 .535 19
10 Arizona State 9-4 .239 .100 33 .020 1 9.0 .597 6 -.387 22 .225 60 .540 15
11 Baylor 10-2 .235 .254 5 .150 41 9.7 .551 10 -.355 27 -.589 88 .541 14
12 UCLA 10-3 .229 .149 20 .025 3 9.1 .501 11 -.417 18 1.686 18 .530 26
Rk Team FBS
Rec
FEI GE GE
Rk
SOS Rk
FBS
MW
OFEI Rk DFEI Rk STE Rk FPA Rk
13 Ohio State 11-2 .216 .235 7 .239 67 10.9 .624 3 -.148 42 2.563 5 .550 8
14 Oklahoma 11-2 .215 .110 29 .056 15 9.6 .409 23 -.443 15 1.027 37 .529 27
15 LSU 9-3 .213 .139 22 .064 19 8.6 .601 5 -.199 38 1.918 10 .507 53
16 Washington 8-4 .202 .138 23 .039 10 8.0 .476 13 -.384 24 1.564 19 .518 41
17 Central Florida 12-1 .193 .202 8 .231 64 10.5 .467 15 -.243 33 1.459 23 .520 37
18 Georgia 7-5 .192 .057 46 .066 20 7.8 .567 8 -.129 46 .074 66 .480 91
19 Wisconsin 8-4 .186 .173 12 .113 34 8.6 .286 31 -.459 13 .617 50 .520 35
20 USC 10-4 .184 .107 31 .076 24 10.0 .314 28 -.643 6 .671 46 .519 40
21 Louisville 11-1 .181 .296 2 .552 107 10.6 .361 25 -.397 21 1.337 28 .531 25
22 Clemson 9-2 .170 .151 19 .115 35 7.6 .343 26 -.422 17 -.241 78 .509 49
23 Notre Dame 9-4 .163 .059 44 .055 14 8.5 .417 20 -.293 30 -.340 82 .483 87
24 Texas A&M 8-4 .162 .120 27 .060 18 7.6 .738 1 .193 87 1.544 21 .550 7
25 Kansas State 8-4 .160 .157 16 .172 53 8.6 .460 16 -.230 35 1.162 33 .517 42

Posted by: Brian Fremeau on 08 Jan 2014

10 comments, Last at 09 Jan 2014, 6:20pm by cfn_ms

Comments

1
by Aaron Brooks Go... :: Wed, 01/08/2014 - 2:18pm

"Kermit Whitfield streaked 100 yards down the sideline, untouched to the end zone and Florida State’s first lead of the game."

Second. FSU also led 3-0.

2
by Spectral (not verified) :: Wed, 01/08/2014 - 5:56pm

I've got to say, as much as I love the work done and try to understand it in context, and grasp that subjective feelings don't really matter, I have trouble with a system that spits back Stanford at second and Michigan St. at eighth. I have no vested interest in either team, that just looks weird to me.

4
by Brian Fremeau :: Thu, 01/09/2014 - 10:07am

It looks weird to me too, honestly. But when I think about why it looks weird, I only arrive at the conclusion that to correct it, either I need to heavily emphasize head to head matchups (which may make many other things look really weird) or heavily de-emphasize opponent adjustments (which has also produced unsatisfactory results when I have explored this in the past).

Aside from the BCS title game, all the other BCS game results have ranking violations in the final FEI ratings:

Alabama > Oklahoma
Baylor > Central Florida
Ohio State > Clemson
Stanford > Michigan State

Weird, I guess, but not when you weight the other 12 games each of those teams played moreso than the head-to-head matchup.

5
by Flounder :: Thu, 01/09/2014 - 10:22am

I think it just reveals the inherent inconsistency between statistically measuring the level of performance of a team on every play/drive over the course of a season and a win or loss in an individual game.

I'm an MSU fan, and I don't really quibble that Stanford was probably "better" over the course of the whole season. MSU's offense was cover your eyes awful for the first three games. MSU was simply better in the Rose Bowl. I would have loved to see MSU play either FSU or Auburn - I think both would have been fascinating games.

Do FEI ratings discount early games at all, or are all games weighted the same? Certainly, the terrible offense at the start of the season would drag their statistics down.

6
by cfn_ms :: Thu, 01/09/2014 - 12:44pm

The other question is, how much should you weigh a really close result? When two teams play head to head on a neutral field and one wins a very close game, should it overwhelm the other 12 games of reuslts? I'm strongly inclined to think not, so I don't really have a problem w/ a system showing Stanford > MSU and certainly no problem w/ Bama > Oklahoma.

7
by Flounder :: Thu, 01/09/2014 - 2:21pm

Definitely.

My biased eyes do feel comfortable though that MSU was superior to Stanford at season's end.

I saw a game where MSU appeared to be the more physically dominant team. The luck ran about even - while Stanford dropped two potential picks, all three fumbles in the game were recovered by Stanford. MSU was just better, and I think in a hypothetical world of an infinite number of Rose Bowls, MSU comes out on top 60-65% of the time.

And I like Kirk Herbstreet, but he was definitely pushing the narrative in the first quarter that MSU "doesn't see these types of athletes" in the Big Ten, which is just patently untrue. For example, Tyler Gafney is good, but he's not as good a college running back as Carlos Hydge, Ameer Abdullah, and probably either of the WI running backs (though MSU didn't play WI this year). Of course, anyone who has watched a significant amount of MSU football this year knows that most of the yards/points that MSU gives up happens in the first quarter (several time this season in shit through a tin horn fashion), and then the clamps are put on.

And of course, I think it's important to recognize that while statistical rankings like these are informative and have a certain amount of predictive power, in the end who won and who lost is the ultimate measuring stick and there's nothing wrong with that.

8
by Brian Fremeau :: Thu, 01/09/2014 - 4:13pm

If Auburn had beaten FSU (say, 37-34 on a touchdown on that final play), FEI likely would have been impacted almost imperceptibly.

9
by Aaron Brooks Go... :: Thu, 01/09/2014 - 5:27pm

How much changes if you give Wisconsin that FG at the end of the ASU game? Because it seems like "ASU didn't suck!" just cascaded through the Pac-12 strength perception.

10
by cfn_ms :: Thu, 01/09/2014 - 6:20pm

Actually the bigger thing was the generally across the board OOC success from the Pac-12. 1-2 vs Notre Dame didn't help at all, but most of the rest (other than Oregon State's awful AA loss) was really good. Wazzu nearly won at Auburn, the league swept the Mountain West, UCLA pummeled Nebraska in Lincoln, USC pummeled BC, etc. And there was a real dearth of outright bad OOC results, whereas plenty of leagues had some stinkers, not just from the bottom of the league, but at times from the middle as well.

3
by Alexander :: Wed, 01/08/2014 - 6:11pm

Good to see, that, at least here, my eyes did not deceive me when I thought that FSU was the best team in a very mediocre year, and was a mediocre champion.