Writers of Pro Football Prospectus 2008

Most Recent FO Features

HarvinPer09.jpg

» Impact of the NFL's Kickoff Rule Change

After three NFL seasons of kicking off from the 35-yard line, what has been the impact on touchbacks, returns, field position, scoring and injuries? Also, is this rule responsible for a record number of big comebacks?

28 Nov 2012

FEI Week 13: Better To Be Lucky And Good

by Brian Fremeau

Notre Dame clinched its bid to the BCS national championship game on Saturday with a 22-13 victory over USC. As Matt Hinton described it, the victory was another "low scoring but utterly convincing victory" for the Irish, the culmination of an unexpected undefeated campaign led by an elite defense that won almost every battle in the red zone to carry the team to several close victories.

Are the Irish lucky to be undefeated? Certainly a missed chip-shot field goal by Pittsburgh in overtime would have derailed the championship run, and a play here and there in victories over Stanford, BYU, Purdue, and Michigan may have made the difference between victory and defeat. But there were blown opportunities by Notre Dame in each of those games that could have made each less of a nail-biter and allowed Notre Dame to win more comfortably. Are the Irish lucky to be undefeated or were several of their opponents lucky to hang in with the Irish?

That’s a complicated question, but we have some data points that can help answer it. First, we can use the mean wins calculation that I publish each week as a way to track what might be considered each team’s over-achievement or under-achievement this year. According to my analysis, the FEI rating and the strength of opposition faced produces an expected mean wins data point for each team: the average number of games a team with the given FEI rating would be expected to win against its entire FBS schedule. The difference between the team’s mean wins and its actual wins is one indicator of good or bad fortune over the course of the year.

Mean Wins at least 1.5 Games Over Actual Wins
FEI
Rank
Team Mean
Wins
Actual
Wins
Delta
114 East Carolina 3.5 7 +3.5
67 Middle Tennessee 6.2 8 +2.1
118 UTSA 2.0 4 +2.0
9 Ohio State 10.0 12 +2.0
87 Navy 4.7 6 +1.9
93 Duke 3.1 5 +1.9
121 Colorado State 2.1 4 +1.9
58 Ball State 7.3 9 +1.7
47 Kent State 8.7 10 +1.6
2 Notre Dame 10.4 12 +1.6
107 Wake Forest 2.5 4 +1.5
55 Washington 4.5 6 +1.5
Mean Wins at least 1.5 Games Under Actual Wins
FEI
Rank
Team Mean
Wins
Actual
Wins
Delta
68 South Florida 3.8 2 -1.5
70 Indiana 4.5 3 -1.5
74 Troy 6.6 5 -1.6
96 UTEP 4.7 3 -1.7
84 Florida International 4.8 3 -1.8
88 Boston College 2.8 1 -1.8
81 Western Michigan 5.0 3 -2.0
106 UNLV 4.1 2 -2.1
99 South Alabama 4.3 1 -2.6
115 Southern Mississippi 2.8 0 -2.8
101 Akron 3.4 0 -3.4

By this measure, Notre Dame has been one of the top beneficiaries of good fortune in 2012, but not exceptionally so. The undefeated Auburn Tigers were also +1.6 mean wins in 2010. The undefeated Alabama Crimson Tide were +1.5 mean wins in 2009. Top teams play games against top competition, and the probability of winning them all is rarely very high. Even teams that only play a few games against top-level competition are still statistically more likely to lose one than win them all. Is it a measure of good fortune that the Irish are undefeated? Yes, but it isn’t dumb luck. It’s well within the realm of probability.

There are 85 teams that have a record within one game of their mean wins expectation. The Irish finished last season with 0.9 wins below expectation, and they were 0.7 wins below expectation in 2010. That means that Brian Kelly’s three-year record of 28-10 is exactly what their FEI ratings over the last three years would have projected against their schedule. It doesn’t even out perfectly for all teams, but over the course of time, every program should probably play within a narrow range of their overall mean wins.

There are other ways to measure over-achievement and under-achievement. The "revisionist box scores" I publish each week are an attempt to identify the games in which turnovers, special teams, and field position were the difference between winning and losing. It isn’t as though teams aren’t deserving of close call victories due to those factors, but stripping away those factors does give us a better sense of which teams "played well enough to win." And it is another indicator of teams that have been the biggest beneficiaries of good fortune due to those elements.

Notre Dame did win one game in which the margin gained on turnovers exceeded the scoring margin in the game. Against Michigan, the Irish collected 12.7 points in total turnover value and won the game 13-6. But in most of their other victories this year, the Irish were overcoming turnover, special teams, or field position deficits.

Take the Pittsburgh game, for instance, the triple-overtime thriller against a mediocre opponent that nearly cost them a trip to Miami. The Irish gave up 14.0 points in turnover value in the game and still eked out a victory. In FBS games this year, only two teams overcame a larger turnover deficit and won. Duke beat Memphis despite an 18.8 point deficit in turnover value, and Virginia beat Penn State despite a 15.7 point deficit in turnover value.

The Irish won twice in 2012 despite losing the field position, turnover, and special teams battles in a single game: close victories over Purdue and BYU. Over the course of the season, Notre Dame won 10 games in which it had a deficit in at least one of those three factors, and 8 games in which it had a deficit in at least two of those factors. No other team overcame as many combined field position, turnover, and special deficits as Notre Dame. Does that make them lucky?

From the perspective of FEI, these elements are actually a big reason why the Irish are considered one of the best teams in the country, not one of the luckiest. Notre Dame’s opponent-adjusted offense is ranked eighth, and its opponent-adjusted defense is ranked second. It’s the non-offensive and non-defensive stuff like special teams (88th) that are holding back the Irish from passing the eye test in more people’s eyes. These might be the factors that hold the Irish back from winning the BCS title game as well, but it shouldn’t be a reason to knock them for what they have accomplished to date.

Week 13 Revisionist Box Scores

This weekly feature identifies the games played each week that were most impacted by turnovers, special teams, field position, or some combination of the three. The neutralized margin of victory is a function of the point values earned and surrendered based on field position and expected scoring rates.

Week 13 Games In Which Total Turnover Value Exceeded Non-Garbage Final Score Margin
Date Winning Team Non-Garbage
Final Score
Losing Team TTV
+
TTV
-
TTV
Net
TO Neutral
Score Margin
11/22 TCU 20-13 Texas 13.6 4.1 9.5 -2.5
11/23 Arizona State 41-34 Arizona 15.0 7.5 7.5 -0.5
11/23 East Carolina 65-59 Marshall 11.6 0.0 11.6 -5.6
11/23 LSU 20-13 Arkansas 7.8 0.0 7.8 -0.8
11/23 Utah 42-35 Colorado 17.3 4.3 13.0 -6.0
11/24 Baylor 52-45 Texas Tech 18.1 0.0 18.1 -11.1
11/24 Florida 30-20 Florida State 17.3 3.7 14.6 -4.6
11/24 Louisiana Monroe 23-17 Florida International 11.9 3.5 8.4 -2.4
11/24 Middle Tennessee 24-21 Troy 28.1 7.8 20.3 -17.3
11/24 Ohio State 26-21 Michigan 13.9 7.3 6.6 -1.6
11/24 Penn State 24-21 Wisconsin 3.9 0.0 3.9 -0.9
11/24 SMU 35-27 Tulsa 10.1 0.0 10.1 -2.1

Week 13 Games In Which Special Teams Value Exceeded Non-Garbage Final Score Margin
Date Winning Team Non-Garbage
Final Score
Losing Team STV
+
STV Neutral
Score Margin
11/23 LSU 20-13 Arkansas 7.5 -0.5
11/23 Washington State 31-28 Washington 5.8 -2.8
11/24 Connecticut 23-20 Louisville 4.1 -1.1
11/24 Middle Tennessee 24-21 Troy 5.6 -2.6
11/24 Oklahoma 51-48 Oklahoma State 3.6 -0.6
11/24 UTSA 38-31 Texas State 10.1 -3.1
11/24 Western Kentucky 25-24 North Texas 1.3 -0.3

Week 13 Games In Which Field Position Value Exceeded Non-Garbage Final Score Margin
Date Winning Team Non-Garbage
Final Score
Losing Team FPV
+
FPV
-
FPV
Net
FPV Neutral
Score Margin
11/23 East Carolina 65-59 Marshall 39.1 27.5 11.6 -5.6
11/23 Utah 42-35 Colorado 39.6 30.5 9.1 -2.1
11/23 Miami 52-45 Duke 28.2 20.1 8.1 -1.1
11/24 Middle Tennessee 24-21 Troy 36.8 24.8 12.0 -9.0
11/24 Oklahoma 51-48 Oklahoma State 31.7 27.6 4.1 -1.1
11/24 UTSA 38-31 Texas State 29.3 19.3 10.0 -3.0
11/24 Western Kentucky 25-24 North Texas 29.3 25.8 3.5 -2.5

2012 totals to date:

  • Net Total Turnover Value was the difference in 110 of 678 FBS games (16.2 percent)
  • Net Special Teams Value was the difference in 53 of 678 FBS games (7.8 percent)
  • Net Field Position Value was the difference in 66 of 678 FBS games (9.7 percent)
  • Turnovers, Special Teams and/or Field Position was the difference in 153 of 678 FBS games (22.6 percent)

2012 Game Splits for all teams, including the offensive, defensive, special teams, field position, and turnover values recorded in each FBS game are provided here.

FEI Week 13 Top 25

The Fremeau Efficiency Index (FEI) rewards playing well against good teams, win or lose, and punishes losing to poor teams more harshly than it rewards defeating poor teams. FEI is drive-based and it is specifically engineered to measure the college game. FEI is the opponent-adjusted value of Game Efficiency (GE), a measurement of the success rate of a team scoring and preventing opponent scoring throughout the non-garbage-time possessions of a game. FEI represents a team's efficiency value over average.

Other definitions:

  • SOS Pvs: Strength of schedule to date, based on the likelihood of an elite team going undefeated against the given team's schedule to date.
  • SOS Tot: Strength of schedule, based on the likelihood of an elite team going undefeated against the given team's entire schedule. Conference championship games and bowl games are not yet included.
  • FBS MW: Mean Wins, the average number of games a team with the given FEI rating would be expected to win against its entire schedule.
  • FBS RMW: Remaining Mean Wins, the average number of games a team with the given FEI rating would be expected to win against its remaining schedule.
  • OFEI: Offensive FEI, the opponent-adjusted efficiency of the given team's offense.
  • DFEI: Defensive FEI, the opponent-adjusted efficiency of the given team's defense.
  • STE: Special Teams Efficiency, the scoring value earned by field goal, punt and kickoff units measured in points per average game.
  • FPA: Field Position Advantage, the share of the value of total starting field position earned by each team against its opponents.

These FEI ratings are a function of results of games played through November 24th. The ratings for all FBS teams, including FEI splits for Offense, Defense, and Special Teams can be found here. Program FEI (five-year weighted) ratings and other supplemental drive-based data can be found here.

Rk Team FBS
Rec
FEI LW GE GE
Rk
SOS
Pvs
Rk SOS
Tot
Rk FBS
MW
FBS
RMW
OFEI Rk DFEI Rk STE Rk FPA Rk
1 Oregon 10-1 .292 3 .342 2 .264 54 .264 58 9.9 - .487 10 -.573 8 .807 40 .542 16
2 Notre Dame 12-0 .284 2 .193 10 .142 27 .142 31 10.4 - .510 8 -.742 2 -.768 88 .495 67
3 Alabama 10-1 .275 1 .352 1 .298 60 .211 45 10.5 0.7 .328 20 -.573 9 2.486 8 .566 3
4 Kansas State 9-1 .272 4 .242 6 .139 26 .116 19 9.0 0.8 .330 19 -.577 7 4.279 1 .587 1
5 Oklahoma 8-2 .270 5 .175 13 .126 20 .092 11 8.7 0.7 .520 5 -.518 13 1.406 27 .520 41
6 Florida 10-1 .262 7 .143 19 .131 23 .131 28 9.0 - .113 48 -.767 1 3.165 2 .548 10
7 Texas A&M 8-2 .245 6 .212 8 .189 38 .189 40 8.2 - .677 2 -.271 31 -.595 83 .505 59
8 Stanford 10-2 .225 10 .134 22 .069 4 .061 4 9.9 0.8 .121 45 -.715 3 1.758 19 .556 6
9 Ohio State 12-0 .220 9 .153 18 .248 51 .248 54 10.0 - .518 6 -.466 17 -.551 81 .505 58
10 Nebraska 9-2 .205 11 .093 35 .152 29 .123 25 8.8 0.7 .494 9 -.486 15 -1.620 106 .459 106
11 Georgia 10-1 .204 14 .258 3 .305 62 .166 36 9.6 0.3 .376 16 -.434 22 .423 52 .530 26
12 Oklahoma State 6-4 .193 15 .112 28 .114 14 .085 9 7.5 0.6 .532 3 -.277 30 1.781 17 .484 81
Rk Team FBS
Rec
FEI LW GE GE
Rk
SOS
Pvs
Rk SOS
Tot
Rk FBS
MW
FBS
RMW
OFEI Rk DFEI Rk STE Rk FPA Rk
13 South Carolina 9-2 .192 16 .177 12 .163 32 .163 34 8.4 - .140 41 -.589 6 -.201 72 .504 60
14 LSU 9-2 .190 13 .126 27 .118 16 .118 20 8.0 - .131 42 -.547 11 1.973 14 .553 8
15 Oregon State 8-3 .177 8 .076 38 .121 17 .121 21 7.5 - .396 14 -.337 27 .134 56 .499 65
16 Florida State 8-2 .175 12 .220 7 .439 83 .422 83 8.9 0.8 .083 53 -.517 14 1.718 20 .546 13
17 Texas 8-3 .169 17 .095 33 .188 37 .088 10 8.0 0.2 .346 18 -.149 42 2.336 11 .555 7
18 Cincinnati 6-3 .144 24 .129 25 .585 105 .563 105 7.8 0.8 .180 37 -.593 5 .455 50 .532 25
19 Wisconsin 6-5 .144 19 .108 29 .174 35 .123 24 7.6 0.3 .110 50 -.470 16 .057 60 .542 17
20 TCU 6-4 .143 37 .020 57 .159 30 .102 16 6.5 0.3 -.143 81 -.546 12 -.073 65 .520 42
21 Michigan 8-4 .142 22 .132 23 .060 3 .060 3 7.5 - .288 25 -.403 23 .614 46 .483 82
22 Northwestern 8-3 .140 25 .069 41 .236 50 .236 53 7.6 - .269 27 -.142 43 3.058 4 .532 23
23 Baylor 5-5 .138 31 .047 46 .103 12 .082 7 6.2 0.4 .821 1 .419 107 -1.373 100 .508 54
24 UCLA 9-3 .135 18 .093 34 .310 63 .180 38 8.8 0.2 .265 28 -.341 26 -.847 89 .545 14
25 USC 7-5 .134 20 .102 31 .084 7 .084 8 7.3 - .279 26 -.174 37 1.540 22 .519 43

Posted by: Brian Fremeau on 28 Nov 2012

5 comments, Last at 30 Nov 2012, 2:13pm by KyleJones

Comments

1
by Kal :: Wed, 11/28/2012 - 8:11pm

Huh.

I was really expecting you to make more out of Oregon getting such a bump from playing Oregon State. For the first time all year their game efficiency actually went down a bit, but they went up hugely in overall value.

2
by Brian Fremeau :: Wed, 11/28/2012 - 9:06pm

Yeah, I didn't mean to diss the Ducks, but since the top four teams have shifted almost on a weekly basis over the last month, I was planning to hold off on the FEI dissection until next week. Oregon's bump was a product of the win over Oregon State as well as Stanford's win over UCLA. The magnitude of the bump has to do with the relevance factor that is used -- Oregon's final pair of games were much more relevant than the bulk of their season.

3
by Anger...rising (not verified) :: Thu, 11/29/2012 - 1:23pm

Until "egregious officiating decisions that kept scoring drives alive" gets added to the stable of Revisionist Box Score categories, any analysis of Notre Dame's luck will be incomplete.

4
by Brian Fremeau :: Thu, 11/29/2012 - 5:14pm

You are correct, I'm not including officiating data that assisted Notre Dame or its opponents in games.

5
by KyleJones (not verified) :: Fri, 11/30/2012 - 2:13pm

Kinda OT, but I from here I went to the F+ ratings, and then to S&P, and there's gotta be something wrong with using that as a ranking. I mean, MSU number 6? Arkansas above Texas?

I'm not saying it can't have a use in terms of measuring offensive and defensive effectiveness, but using it to rank teams overall (like in the F+) results in some very non-intuitive (and imo incorrect) results.