In this week's Varsity Numbers, Bill Connelly revisits some measures and concepts: Adjusted Scores, Covariance, and momentum (or whatever you choose to call it).
24 Jul 2011
Guest Column by Kevin Haynes
It’s impossible to follow college football without crafting narratives in your head that explain the sport’s on-field results. Tim Tebow gave an emotional speech and promised to work harder than anyone else, and that’s why his Florida Gators won the 2008 BCS title. Charlie Weis thought too highly of his own abilities, and that’s why Notre Dame struggled under his watch. Pete Carroll’s contagious optimism swept through USC like wildfire, and that’s why the Trojans were arguably the best team of the last ten years.
Yet no matter the season, no matter the decade, college football has a startling tendency to finish according to the same tired script: Ohio State finishes at or near the top of the Big Ten, Duke flounders in the ACC, and the national championship goes to (or at least through) the SEC. This isn’t to say the sport doesn’t have surprises, years where upstarts like Boise State run a Statue of Liberty play to catch the old guard napping. But for the most part, when Oklahoma and Florida State slip from prominence, they aren’t replaced by a New Mexico, Eastern Michigan, Temple or Baylor. Instead, the beneficiary is from within the elite cartel: teams like Nebraska, Penn State, Florida, Texas, and Alabama. The spread of national championships among teams in college football is so distinct that it follows what mathematicians refer to as a "power law distribution," meaning large values are rare and small values are common, and that the two extremes occur at a predictable and formulaic rate. Other examples of power law distributions can be found in the spread of World Series wins among Major League Baseball teams (there’s the Yankees, and then there’s everyone else), city sizes within countries and states (a small number of big cities, countless small towns and villages), blockbusters in Hollywood per year (a couple Avatars, numerous Hurt Lockers), and even the spread of individual wealth within societies (few rich people, lots of poor).
|Final AP Rankings at 10-year intervals from 1969-2009|
|1||Texas||Alabama||Miami (FL)||Florida State||Alabama|
|2||Penn State||USC||Notre Dame||Virginia Tech||Texas|
|4||Ohio State||Ohio State||Colorado||Wisconsin||Boise State|
|5||Notre Dame||Houston||Tennessee||Michigan||Ohio State|
|6||Missouri||Florida State||Auburn||Kansas State||TCU|
The peculiar thing about these power law distributions, though, is that they’re typically formed not by choice, but by circumstance. The Yankees, for instance, are historically the best baseball team largely because they have the most money, but the Yankees only have the most money because they’re in the country’s largest market. New York is the country’s largest market for a few reasons, but mostly because it possesses a perfect natural harbor that to this day grants its inhabitants a competitive advantage over any city in the United States. Avatar squashed The Hurt Locker at the box office not because it’s necessarily a better movie, but because it possesses a matchless pedigree: a holiday release, top notch special effects (in 3D!), aliens, and the surefire brand of James Cameron.
With this distribution in mind, the question becomes: are games being won and lost because of the makeup of the competing players, or because of the makeup of the competing universities? Is there something unique to the sport’s competitive structure that makes Alabama inherently more likely to be a powerhouse than Iowa State? If so, this finding could force us to reexamine the way we think about the sport, and provide us a new way to evaluate and explain outcomes: one that doesn’t rely on tear-filled speeches and journalistic half-truths. If we get really lucky, we might even finally understand how Penn State, with an 84-year-old head coach who doesn’t wear a headset on the sideline, still manages to be competitive while Indiana can’t field a quality team no matter how many times the Hoosiers redesign their uniforms.
To determine whether this resource-centric assumption about outcomes in college football holds up under scrutiny, we turn to the statistical technique of multiple regression. Beginning with a "kitchen sink" model that includes as many variables as possible -- from the population and median income of each BCS conference school’s respective city to their average low temperatures in January to the tenure, age, and overall record of their head coach -- and then narrowing the list of predictors until we’re left with only the measures that remain influential and/or statistically significant. We wind up with a model that predicts, on average, the exact regular season records of over 20 percent of BCS teams from 2005 to 2010 essentially without error. In addition, 56 percent of its estimates are within one win of the actual totals, 81 percent of them are within two wins, and nearly 100 percent of them are within three wins.
Notable predictors at the year-to-year level include the number of wins a team had in its previous season, the number of returning starters it has on offense and defense (returning offensive starters matter almost twice as much as defensive starters), and, surprisingly enough, a team’s Academic Progress Rate score (teams that keep more players on track to graduate do, in fact, perform better on the field). It’s the significance of the factors that do not change from year to year, however, that begins to explain why the same group of teams consistently populate college football’s top ten. The model declares that the outcomes of college football games can overwhelmingly be attributed to three fundamental and relatively fixed factors: the academic standards held by the universities at-large, the amount of money they dedicate to their football programs, and the quality of high school talent found in the universities’ respective states.
What ultimately emerges when we combine the effects of these factors at the school level is a single quantitative measure that in many ways explains the disparate landscape of modern college football. The above figure shows how a handful of schools’ composite "starting positions" compare. With a distinct competitive advantage we see Ohio State and Alabama -- two extremely well-funded programs with softer academic standards located in states that typically produce a surplus of football talent. On the other end of the spectrum, we have Vanderbilt and Duke -- two cash-starved programs at academically demanding universities that also have the misfortune of being located in talent-poor states. Between these two extremes we see schools such as Wisconsin and USC, who have well-funded programs with strong traditions, but also at least one significant hurdle to overcome: Wisconsin is a fairly poor state to recruit in, and USC has extremely high academic standards. The values shown above indicate how many regular season games each of the selected teams should be expected to win if all the model’s fluid measures (number of returning starters, APR scores, etc.) were equal across the board. In other words, the competitive advantages inherent to Alabama (less-than-rigorous academic standards, an almost unmatched budget, and a talent-rich state) are so plentiful that the Crimson Tide should be expected to win roughly five more games than the Commodores even if a season rolls along where both teams have the same number of starters returning and their players are making the same progress toward earning their degrees.
Unfortunately for any nefarious plans to corner the sports betting market, predicting the result of an upcoming season with the accuracy we had from 2006-2010 is near impossible, as a number of the measures we’d like to use in the resource-centric model don’t become available until after that season ends. However, it’s also a lot of fun, so for the sake of argument we’ll use current 2011 data wherever possible to estimate the outcome of the 2011 campaign (number of starters returning on both sides of the ball, last season’s wins, etc.) and fill in the gaps with 2010 data wherever feasible (program budgets, academic standards, state talent levels). In places where we can’t feasibly fill the gap – such as APR scores and post-season Strength of Schedule indicators – we’ll just leave those measures out of the model entirely and cross our fingers that what’s left is robust enough to generate decent results. (Note that these predictions are not related to the forecasts in the new Football Outsiders College Football Almanac 2011.)
|*Because of data inefficiencies, non-BCS teams such as
Boise State and TCU were not yet applicable to the model.
For the purposes of comparison, we’ve posted the resource-centric model’s regular-season win predictions against noted college football guru Phil Steele’s top 10 teams for 2011. A quick glance at the results yields a number of similarities. Both Steele and the model agree that Alabama is primed to make a run at another national championship, but should face stiff competition for the SEC West from LSU. Oklahoma and Texas A&M (!) look like the front-runners in the Big 12, while Oregon could once again take advantage of USC’s sanctions to cruise to a third straight BCS bowl. Virginia Tech and Florida State seem destined to meet in the ACC Championship game, though it does not appear either will be a serious contender for the BCS title. The list and the system differ most over who should be the top team in the SEC East: the model holds that South Carolina (8.5 predicted wins) and Florida (7.1 predicted wins) are more likely to challenge for the SEC title, and that even Tennessee (6.6 predicted wins) should have a slight edge over the Bulldogs.
The other significant dissimilarity lies at the hands of Notre Dame. "I KNOW the Irish underachieve EVERY year," Steele writes in his 2011 College Football Preview about Notre Dame’s chances for the upcoming season, repeating an assertion that in college football has become as customary as Lee Corso donning mascot headgear and Nick Saban handing out grayshirts. According to the resource model (which hasn’t seen Rudy and isn’t familiar with the legends of George Gipp and Knute Rockne, much less Rocket Ismail and Lou Holtz), the Fighting Irish should have a relatively solid season, but a return to the Top 10 might still remain a couple wins out of reach. The model doesn’t yet understand how good of a coach Brian Kelly might be (we’ll show in part two of this piece that Notre Dame might have finally made the right hire), and it certainly can’t quantify the sort of leadership a player like Manti Te’o provides, but it does understand the fact that Notre Dame’s rigid academic standards and talent-poor home state of Indiana curse the program with a sizable competitive disadvantage that its strong budget and prominent reputation are not likely to overcome. No matter how well Kelly understands his X's and O's and Te’o inspires his defense.
|Phil Steele's Notre Dame: Constantly Underachieving or Habitually Overrated?|
|Year||Steele's Preseason Ranking||Model's Win Projection||Actual Wins|
For this reason, continuing to classify the Irish as habitual "underachievers" in the way the media typically does may be nothing more than a harsh exaggeration of the power of the Notre Dame brand. Instead, it might be more appropriate to only view the Irish as "overachievers" in seasons in which they win 10 or 11 games, and as simply "achievers" in all the rest. Case in point: if Northwestern wins eight games for three straight years, we’d be inclined to consider a run like that for the Wildcats as an unmistakable success. If Notre Dame does the same, though, we’d probably hear discussion about whether the Irish should once again gut the program in an effort to return the esteemed Catholic university to the apex of college football. But are these two schools really that different?
In terms of past success, national reputation, and expectations: heck yeah they are. Northwestern, as a program, has never truly made a run at a national championship, isn’t discussed on ESPN unless they upset a traditional power, and has to be at least somewhat satisfied with seven or eight wins. Notre Dame, on the other hand, has the most national championships in the sport, is discussed on network TV ad nauseam, and in a typical season, feels disappointment if its football team doesn’t win at least 10 games.
Compare their modern-day resources and restrictions, though, and the two schools actually have a lot in common. Both are small, private universities that only accept top tier students. Both are located in talent-poor states (Illinois is actually stronger than Indiana), and while Notre Dame has a far superior budget, the gap is not so large as to make the two schools complete strangers. While we can and should still expect Notre Dame to experience 10-win seasons from time-to-time due to the strength of its brand, the idea that the Fighting Irish are merely taking a sabbatical from the top tier of college football is almost certainly mistaken. For whatever reason -- be it larger television contract for conference teams, increased academic standards, or some other element we don’t have time to investigate here -- it appears that that the best days of the Fighting Irish might very well be a thing of the past.
(Next week: a look at what this model might tell us about coaches over the last decade.)
Kevin Haynes is a statistician in Cincinnati, Ohio. He earned a B.A. in English and Creative Writing at Clemson University and has a M.A. in Applied Economics from the University of Cincinnati. A lifelong Michigan fan, Kevin has spent the last four years making excuses for what’s been happening on the field in Ann Arbor, and swears that this season he won’t get too excited if the Wolverines beat Notre Dame.
Football Outsiders is always accepting guest columns that have a unique perspective on either the NFL or college football. Send your ideas or samples to mailbag-at-footballoutsiders.com.
27 comments, Last at 27 Jul 2011, 7:20pm by Aaron Schatz