Writers of Pro Football Prospectus 2008

Most Recent FO Features


» QBASE 2017

How iffy is the 2017 quarterback class? Even our highest-rated prospect has us questioning the numbers. While the top of the draft is full of risky picks, though, we might have found a solid mid-round sleeper.

20 Aug 2014

FEI Primer: Elite Characteristics

by Brian Fremeau

There will be four teams in the inaugural College Football Playoff at the end of the season. Only 3.1 percent of the 128 FBS teams will be selected, the teams judged by a committee to be the best of the best. Conference championships, strength of schedule, head-to-head competition, video and statistical analysis will all be part of the committee’s process and deliberations. What common characteristics will distinguish these four teams above all others?

There are no certainties at this point, but we can consider other models to provide a bit of insight. For model building, I like using the Massey Consensus Rankings, a collection of more than 100 college football rating systems compiled to produce a consensus ranking order of teams throughout the year. Each input in the Massey Consensus has its own criteria and each may prioritize different aspects of team performances and results. But the overall consensus is a pretty solid and reliable indicator of which teams had the best seasons.

What kind of seasons do elite teams have? I looked at the last ten years (2004 to 2013) and broke up the Massey Consensus into nine categories of teams: Elite (ranked No. 1 to No. 5), Very Good (No. 6 to No. 15), Good (No. 16 to No. 30), Above Average (No. 31 to No. 50), Average (No. 51 to No. 70), Below Average (No. 71 to No. 90), Bad (No. 91 to No. 105), Very Bad (No. 106 to No. 116), Awful (No. 117 or worse). By this definition, there were 50 Elite teams in the last ten seasons with a combined FBS record of 581-55 (.914). Against a 12-game schedule it averages out to 10.97 wins per team. Against a 13-game schedule it averages out to 11.88 wins per team.

Elite teams win 91.4 percent of their FBS games overall, but how does that break down against different groups of opponents? The table below summarizes Elite team performances against the other Massey Consensus team types over the last ten seasons.

Elite Teams Against Opponent Team Types
Opponent Type Record Win Pct. MOV Home Away Neutral
Elite 24-24 0.500 0.0 4-3 3-4 17-17
Very Good 69-19 0.784 10.9 25-7 15-10 29-2
Good 92-8 0.920 16.6 39-2 33-5 20-1
Above Avg 121-2 0.984 20.8 64-0 54-2 3-0
Average 102-1 0.990 26.1 54-1 45-0 3-0
Below Avg 85-1 0.988 30.2 48-1 37-0 0-0
Bad 53-0 1.000 35.9 30-0 22-0 1-0
Very Bad 22-0 1.000 40.2 11-0 11-0 0-0
Awful 13-0 1.000 47.2 11-0 2-0 0-0
Total 581-55 0.914 21.8 286-14 222-21 73-20

Not surprisingly, Elite teams almost never lose to average or weak opponents. In the last ten seasons against the Above Average, Average, Below Average, Bad, Very Bad, and Awful team types, Elite teams have lost only four times in 400 games. Each of those losses was enough to sink the BCS championship dreams of the Elite teams involved:

  • 12/2/2006: USC (11-2, No. 4 Massey Consensus) lost 13-9 to UCLA (7-6, No. 32)
  • 10/6/2007: USC (11-2, No. 5 Massey Consensus) lost 24-23 to Stanford (4-8, No. 77)
  • 12/1/2007: West Virginia (11-2, No. 3 Massey Consensus) lost 13-9 to Pittsburgh (5-7, No. 64)
  • 11/18/2011: Oklahoma State (12-1, No. 3 Massey Consensus) lost 37-31 to Iowa State (6-7, No. 48)

Only 14 of the 55 losses suffered by Elite teams in the last ten years came in Elite team home games, and 11 of those came against non-Elite opponents. When Elite teams lose, they don’t lose by much. Of the 31 losses suffered by Elite teams to non-Elite teams since 2004, only seven were decided by 10 or more points and the average margin of defeat was less than a touchdown (6.7 points). None of the Elite losses to Good opponents or worse were decided by more than eight points.

It is not particularly remarkable that Elite teams have enjoyed these levels of success, but the table provides some clarity with regards to the expected performance achievements of top teams. The frequency of opponents in each category may be of particular relevance. Elite teams played an average of 2.7 games against Elite or Very Good opponents over the last ten years, but only an average of 1.6 top-15 opponents in regular season games. Elite teams won 68.4 percent of their games against top-15 opponents since 2004. How will the selection committee compare teams that play three or four opponents of that caliber against those that play only one or none? The overall records of those teams certainly won’t tell the same story.

In the FEI ratings system, I define an Elite team as two standard deviations better than average. In the last ten seasons, the only team that ranked above that threshold in the FEI ratings but did not rank among the top five in the Massey Consensus was Stanford last year. FEI ranked the Cardinal No. 2 overall with an 11-3 FBS record; they were No. 6 in the Massey Consensus. Stanford’s results against Massey and FEI team types are listed below.

Stanford 2013 Record Against Team Type
Opponent Type Massey FEI
Elite 0-1 0-0
Very Good 2-0 4-1
Good 4-1 2-2
Above Avg 1-0 1-0
Average 1-1 1-0
Below Avg 1-0 1-0
Bad 0-0 1-0
Very Bad 2-0 1-0
Awful 0-0 0-0

Stanford had one major flaw in its record last season, a 27-21 loss on the road against 5-7 Utah. The Massey Consensus ranked Utah No. 51 (the top of the Average category) and FEI ranked Utah No. 30 (the bottom of the Good category). That loss was similar to the four games listed above that bounced teams out of the BCS championship. Would a similar loss be as costly for a team in the College Football Playoff hunt? Or could an overall profile with wins against other top opponents be enough to overcome a bad loss?

Like the other systems included in the Massey Consensus, the FEI ratings take a particular approach to answering those questions. We aren’t selecting teams to compete in the College Football Playoff, but our goal of using opponent-adjusted data to determine the best teams is not too dissimilar from the selection committee’s goal. The emphasis each places on the value of wins and losses will likely be different, but as mentioned last week, hopefully theirs will also be informed and consistent. I’ll present the final preseason projected FEI ratings for 2014 next week before the regular season kicks off and all of our expectations are recalibrated.

Posted by: Brian Fremeau on 20 Aug 2014

1 comment, Last at 23 Aug 2014, 11:39am by Dad2sca


by Dad2sca :: Sat, 08/23/2014 - 11:39am

Excellent as always. Looking forward to the FEI 2014!!!