by Brian Fremeau

Another Saturday is in the books, and another day full of surprises shook up the college football universe. In the last two weeks, Alabama, Kansas State, and Oregon were each positioned in the driver’s seat with a clear path to the BCS national championship game. Then, each lost a game as a double-digit favorite. Notre Dame now finds itself at the top of the BCS standings, looking to avoid a similar fate against rival Southern Cal.

There’s nothing specific in the data we track that would have predicted the recent poll carnage with any degree of certainty. The numbers did suggest that Oregon was more likely than not to stumble at some point in the home stretch, and Stanford’s defense did pose a major threat to the Ducks’ offensive attack, but FEI still projected Oregon to win by two touchdowns last weekend. Baylor was certainly capable of big plays and efficient offense against Kansas State, but FEI didn’t expect the Bears defense to play their best game of the season and turn it into a rout.

Projecting single games is a bit of a crap shoot, of course, and we need to see the forest for the (Stanford) trees. How successful are aggregated game-by-game projections? How successful are season-long projections? The projections aren’t something I’ve focused on too much in the weekly FEI column here at Football Outsiders, but I have been tracking the data at my personal website. There is still much more analysis to be done, but I’m satisfied with the progress so far. And I’m interested in making more sophisticated projections by including more data going forward.

I’ve been playing around recently with a new model I developed to calculate team similarity scores. I calculate a variety of metrics each week that characterize the offensive, defensive, special teams, and overall success and efficiency of each FBS team. The overall FEI rating is the most important of these metrics, but I’ve calculated the relationship between dozens of other stats (my drive-based stats, as well as many common stats) and each is important in its own context.

To build a new team similarity model, I selected 25 measures to define each team’s efficiency profile. The profile includes the opponent-adjusted overall FEI, offensive FEI, and defensive FEI ratings, as well as unadjusted data such as game efficiency, field-position advantage, special-teams efficiency, and 19 other offense, defense, and special teams splits. The significance of each measure in the profile was weighted in accordance with that stat’s correlation with overall winning percentage. I then compared the efficiency profile of the contenders against the efficiency profiles of each FBS team in the last five seasons, 599 teams in all.

By this analysis, the team with the most similar efficiency profile to the 2012 Notre Dame Fighting Irish is the 2010 Oklahoma Sooners, as illustrated in the chart below.

I find this team similarity model to be really interesting, but what can actually be learned from it? For starters, it is a good reminder that similarities in advanced stats don’t always translate to traditional statistics. The 2010 Oklahoma team scored at least 40 points seven times, whereas this year’s Notre Dame team has only done so twice. The Irish have only given up 20 points once this year, but the 2010 Sooners did so ten times. There might not be any obvious similarities on the surface, and yet, the opponent-adjusted data suggests that the profiles are quite similar. If this year’s Notre Dame team had played the 2010 Oklahoma schedule, should it have expected to have similar results to the Sooners? If the 2010 Sooners had played Notre Dame’s schedule this year, would its results have been expected to be similar to the Irish? That's the idea, at least.

I’m also interested in exploring the significance of the outlier data. 2010 Oklahoma and 2012 Notre Dame are a close match in many categories, but they are dramatically dissimilar in terms of special-teams efficiency. The Sooners ranked 27th in that category in 2010, whereas Notre Dame currently ranks 90th in that category. The Irish had a special-teams deficit by FEI's numbers in eight of their 11 games this year, and while it hasn’t cost them a victory yet – there have been a couple of close calls. The closest calls came in their narrow victories over Purdue and Pittsburgh, games in which the Irish lost 8.8 points in special teams combined.

Here are the efficiency profiles most similar to the current FEI top-10 teams:

There are many more things that can be done with the team similarity project, including an incorporation of that complete efficiency profile in the weekly projections. Currently, my weekly FEI projections are produced from a formula derived from aggregate FEI data. But what if, instead, we ran a similarity exercise comparing data for both teams in a given matchup against every FBS game played over the last five seasons (3576 games in all)?

Using FEI, OFEI, DFEI, and STE data only, I compared the Notre Dame and USC matchup against every game over the last five seasons to find matchups that were most similar across those efficiency data points. The most similar game was a 38-14 victory by Alabama over Arkansas last season. That is, Notre Dame’s profile compared with 2011 Alabama’s profile and USC’s profile compared with 2011 Arkansas’ profile produced the closest game match.

Not every similar matchup favors a victory for Notre Dame, of course. Oregon’s 2007 losses to California (24-31) and Arizona (24-34), LSU’s 2007 losses to Arkansas (48-50 in triple overtime) and Kentucky (37-43 in triple overtime), and USC’s 2008 loss to Oregon State (21-27) all rank among the 25 closest game similarity matches for this weekend’s ND-USC game.

If we aggregate the data, can we use it to make a more precise projection for the game? Taking the 25 most similar games, the Irish would be expected to win 80 percent of the time by an average final score of 33-24. Taking the 50 most similar games, the Irish would be expected to win 84 percent of the time by an average final score of 36-23. Taking the 100 most similar games, the Irish would be expected to win 80 percent of the time by an average final score of 35-23. Using my current methodology, FEI projects the Irish to have a 71 percent chance of victory by a score of 30-20.

This project is still in its infancy, but I’m looking forward to doing more with projections in the coming weeks. Any feedback would be appreciated.

Week 12 Revisionist Box Scores

This weekly feature identifies the games played each week that were most impacted by turnovers, special teams, field position, or some combination of the three. The neutralized margin of victory is a function of the point values earned and surrendered based on field position and expected scoring rates.

2012 totals to date:

• Net Total Turnover Value was the difference in 98 of 621 FBS games (15.8 percent)
• Net Special Teams Value was the difference in 46 of 621 FBS games (7.4 percent)
• Net Field Position Value was the difference in 59 of 621 FBS games (9.5 percent)
• Turnovers, Special Teams and/or Field Position was the difference in 135 of 621 FBS games (21.7 percent)

2012 Game Splits for all teams, including the offensive, defensive, special teams, field position, and turnover values recorded in each FBS game are provided here.

FEI Week 12 Top 25

The Fremeau Efficiency Index (FEI) rewards playing well against good teams, win or lose, and punishes losing to poor teams more harshly than it rewards defeating poor teams. FEI is drive-based and it is specifically engineered to measure the college game. FEI is the opponent-adjusted value of Game Efficiency (GE), a measurement of the success rate of a team scoring and preventing opponent scoring throughout the non-garbage-time possessions of a game. FEI represents a team's efficiency value over average.

Other definitions:

• SOS Pvs: Strength of schedule to date, based on the likelihood of an elite team going undefeated against the given team's schedule to date.
• SOS Tot: Strength of schedule, based on the likelihood of an elite team going undefeated against the given team's entire schedule. Conference championship games and bowl games are not yet included.
• FBS MW: Mean Wins, the average number of games a team with the given FEI rating would be expected to win against its entire schedule.
• FBS RMW: Remaining Mean Wins, the average number of games a team with the given FEI rating would be expected to win against its remaining schedule.
• OFEI: Offensive FEI, the opponent-adjusted efficiency of the given team's offense.
• DFEI: Defensive FEI, the opponent-adjusted efficiency of the given team's defense.
• STE: Special Teams Efficiency, the scoring value earned by field goal, punt and kickoff units measured in points per average game.
• FPA: Field Position Advantage, the share of the value of total starting field position earned by each team against its opponents.

These FEI ratings are a function of results of games played through November 17th. The ratings for all FBS teams, including FEI splits for Offense, Defense, and Special Teams can be found here. Program FEI (five-year weighted) ratings and other supplemental drive-based data can be found here.

Posted by: Brian Fremeau on 21 Nov 2012

4 comments, Last at 23 Nov 2012, 1:16am by IrishGush