Looking back at FEI's preseason projections, we find that most teams did about what they were supposed to do -- but not in the Big Ten, where things got screwy.
19 Feb 2010
by Bill Connelly
I got solid feedback from the two ESPN Insider pieces I wrote around signing day (found here and here), so I thought I would use the VN space to expand on the conversation. If you hate football recruiting and feel that even the smallest discussion of it makes you want to go take a shower, you might just want to skip this column ... or at least scroll down to the Random Golf Clap section.
The two ESPN pieces referenced above both dealt with college football recruiting and its link to on-field success. The first dealt with correlations and the predictive power of recruiting rankings; the second was more of a reaction to the 2010 class -- which teams saw upgrades and downgrades in their talent level, and which have recently overachieved or underachieved compared to what their recruiting rankings would predict. The conclusions ranged from duh-worthy (of course recruiting rankings matter, though there was a bit of personal surprise with the magnitude) to surprising and hard to explain (how the hell are Texas and Alabama overachievers?). Now let's expand on what we know to this point.
Recruiting success and on-field success are a chicken-or-egg concept. One leads to the other, which leads back to the other, but which came first in the process? Can you succeed at the highest level without great recruiting? Can you recruit consistently well over a period of years without succeeding at a high level on the field? The causation is too complicated to determine, but there is a correlation discussion that I think can take place, and I'm trying to get it started.
To illustrate my point, I want to reference a recent "Why Recruiting Matters" post from Carl Bialik of the Wall Street Journal's The Daily Fix blog. Bialik, with whom I had an e-mail conversation on the topic, said this:
There's certainly a link, but I'm not yet convinced that all of the correlation he finds is the result of recruiting breeding success. Good college-football teams tend to stay good, and also to recruit well. But that doesn't mean one causes the other. Surely the talent of incoming players is important, but so are facilities, other attributes of a university that are attractive -- such as climate and convenience -- and coaching. Those factors likely help top programs recruit, and also get the most out of their recruits.
Coaching and development are clearly important to the success of a program; for more on that, see the "Overachievers and Underachievers" section below. However, I think the above passage shows where we can get our minds spinning in circles on the topic. The point of that first Insider piece was to show that good rankings are correlated to good performance; I was looking for correlation rather than causation, and I think that's an important point to make. I can't tell you which comes first, the chicken or the egg, but I can tell you that chickens lay eggs now, and I can tell you that, if you recruit at a very high level over a period of 4-5 years, it will almost certainly reflect well on the field. And if your recruiting begins to suffer, it will almost certainly start to show on the field, maybe not right away, but a few years into the future.
In most years, you will still have to coach wonderfully and overachieve a little bit to bring home a national title, but as a whole, 4- and 5-year average recruiting rankings are almost as predictive to success as 4- and 5-year on-field performance is.
|Correlation between a team's F/+ performance* and common projection factors|
|Last Year's Performance||Last Year's S&P+ Rating: 0.754
Last Year's F/+ Rating: 0.744
Last Year's FEI Rating: 0.720
|Historical Performance||3-Year History: 0.738
4-Year History: 0.738
5-Year History: 0.729
|Recruiting Performance||5-Year Rivals Ranking Avg.: 0.673
5-Year Rivals Ranking Weighted Avg.**: 0.670
4-Year Rivals Points Avg.: 0.654
5-Year Rivals Points Avg.: 0.653
* The F/+ measure is the combination of S&P+ and FEI ratings, a measure referenced in Varsity Numbers often and featured in last year's Football Outsiders Almanac.
** The weighted average is an attempt to emphasize the importance of both experience and talent; it more heavily weights the classes from 3-5 years ago than the most recent classes.
It really does say a lot about the continuity involved in college football that you could come up with a pretty strong prediction of success for a team based solely on recent performance and recruiting, and not taking into account returning starters, recent turnover margin, or any of the primary year-to-year change factors. Obviously you can make a better prediction bringing more factors into the fold, but all of the correlations involved with both recent performance and recruiting are quite strong.
But as we know, the FBS level of college football is quite diverse, giving a playing field to both LSU and Louisiana-Lafayette, Florida and Florida International. What works in the SEC or Big 12 probably is not the same as what works in the Sun Belt or WAC. And sure enough, if we break out these correlations between major conferences (those whose champions get automatic bids into a BCS bowl) and mid-majors (those who don't), different stories emerge.
|Correlation between F/+ performance and common projection factors, AQ and non-AQ|
|Category||Factor/Correlation||AQ Conf.||Non-AQ Conf.|
|Last Year's Performance||Last Year's F/+
Last Year's S&P+
Last Year's FEI
Last Year's Point Differential
Last Year's Win %
|Historical Performance||2-Year History
|Recruiting Performance||4-Year ESPN Grade Avg.
5-Year Rivals Points Avg.
4-Year Rivals Points Avg.
5-Year Rivals Rank Avg.
5-Year Weighted Rivals Rank Avg.
|Correlation between change in F/+ performance and common projection factors, AQ and non-AQ|
|Category||Factor/Correlation||AQ Conf.||Non-AQ Conf.|
|Year-to-Year Change Factors||Last Year's Turnover Margin
Last Year's Pct. of Fum. Rec.
Offensive Starters Returning
Offensive Pct. of Lettermen Returning
Ret. Starting Quarterback
Defensive Starters Returning
Defensive Pct. of Lettermen Returning
There is quite a bit to digest here, but here are some potential conclusions:
Here's the strangest point made in either one of the Insider recruiting pieces: Texas, Alabama, and Florida were among the top 10 overachievers in 2008-09, according to what their recruiting rankings project. Here's how that list was established: for the 2008 and 2009 season, a team's F/+ rating was compared to the F/+ projected for them using a regression formula based on the relationship between F/+ and 5-year Rivals averages.
As soon as the second Insider piece was published, I regretted not expanding the sample to include 2007. The idea with the 2-year sample was to keep the data as immediate as possible (since people tend to want to make immediate conclusions when it comes to recruiting data), but a 3-year sample would have told a more comprehensive story, and for this piece, we're including 2007.
The biggest single-season overachievers from 2007-09 are as follows:
1. TCU 2009 (Projected F/+: 195.6, Actual F/+: 258.9, Difference: +63.3)
2. TCU 2008 (Projected: 194.7, Actual: 248.9, Difference: +54.2)
3. Boise State 2009 (Projected: 193.1, Actual: 241.7, Difference: +48.6)
4. Boise State 2008 (Projected: 191.8, Actual: 238.9, Difference: +47.1)
5. Cincinnati 2009 (Projected: 187.9, Actual: 234.8, Difference: +46.9)
The biggest single-season underachievers:
1. Washington State 2008 (Projected: 203.1, Actual: 144.5, Difference: -58.6)
2. Washington State 2009 (Projected: 195.8, Actual: 138.0, Difference: -57.8)
3. Washington 2009 (Projected: 213.0, Actual: 156.9, Difference: -56.1)
4. Texas A&M 2008 (Projected: 220.9, Actual: 171.9, Difference: -49.0)
5. Notre Dame 2007 (Projected: 221.9, Actual: 183.8, Difference: -38.2)
So this was the standard used for the underachievers and overachievers list.
Here is the three-year overachiever list, with numbers.
1. TCU (+44.0/season)
2. Cincinnati (+43.5)
3. Boise State (+39.1)
4. Florida (+35.1)
5. Navy (+32.7)
6. Air Force (+32.6)
7. BYU (+28.1)
8. Connecticut (+25.7)
9. Utah (+24.2)
10. Penn State (+21.7)
The expansion to a three-year sample bumped Texas (+18.6) to 16th and Alabama (+18.2) to 17th, thanks to their underachieving 2007 seasons.
You can make this list in two ways. You can either win with average (or worse) recruits, or win big with good (or great) recruits. Florida's recruiting-based projected F/+ was second behind USC in all three seasons (2007-09), but their actual F/+ output suggests that recruiting can only get you so far. Coaching, execution, and a little bit of overachieving are what you need to win (or contend for) titles.
Here is the three-year underachiever list.
1. Washington State (-39.3/season)
2. San Diego State (-28.2)
3. Texas A&M (-27.0)
4. Iowa State (-23.8)
5. Kansas State (-23.6)
6. Washington (-22.1)
7. Syracuse (-21.8)
8. Notre Dame (-21.7)
9. Colorado (-20.9)
10. Michigan (-20.3)
The fact that Notre Dame has upgraded from Charlie Weis, who led the Irish to the eigth-place spot on the underachievers list, to Brian Kelly, who led Cincinnati to second place on the overachievers list, tells you all you need to know about why Notre Dame fans should be absolutely ecstatic about their coaching upgrade. Experience and system fit still matter, and we'll see what Kelly has to work with in the 2010 season. But Kelly has proven himself more than almost any other coach in recent seasons, and if anybody can take the Irish back to the Top 10, Kelly seems to be the man to (eventually) do it.
My other regret with the second Insider piece: I used means instead of medians in averaging a conference's achievement numbers to determine which conferences had most over- and underachieved. Below is a 3-year look at FBS conferences with both the average and median "projected versus actual F/+" data.
|Median and Avg. Differences Per Conference
Between Projected F/+ and F/+ (2007-09)
Part of the headline of the second Insider piece was "Pac-10 in decline," and that may not have been entirely fair. The Pac-10 still does not fare well in stretching the sample to three years, but the use of medians as the guideline instead of averages helps them significantly, as it minimizes the anchor that is Washington State.
The median data also brings all but two conferences within +/- 5.0 of 0.0, meaning that over time, most conferences achieve at roughly the level expected from recruiting rankings. However, the fact that only two conference earned a positive median suggests that more teams tend to underachieve, while the ones who overachieve do it rather significantly.
For the second Insider piece, we compared teams' 2005-06 recruiting rankings to their initial 2010 rankings to determine who may be in the process of upgrading or downgrading their talent level. Here, we will do something slightly different. Using the same regression formulas referenced above, we can determine a projected F/+ based on teams' 5-year recruiting averages.
Below is a list of the 10 teams whose recruiting-based projected 2010 are most improved over their 2009 projected data. This will show you whose talent level has improved the most from last season. None of these improvement numbers are huge, because (a) regression equations are going to usually err on the conservative side of things (i.e. no tremendously high or low numbers), and (b) we're only talking about one recruiting class here, and nobody's talent level changes to a historic level with one class.
|Most Improved Major Conference Teams for 2010
According to Recruiting Rankings
While Brian Kelly never had a ton of talent to work with (according to the recruiting rankings), he made the most of it. And now he's leaving incoming coach Butch Jones with a fuller cupboard, even though Cincinnati is still among the bottom of the list in terms of major conference recruiting rankings. Meanwhile, Steve Sarkisian is doing a nice job of upgrading Washington's talent, and Missouri, the Big 12 North's biggest overachiever over the last three seasons, is upgrading its own as well.
|Least Improved Teams for 2010 According to Recruiting Rankings|
The Big Ten has been better than most conferences at outperforming recruiting rankings. The conference is going to have to hope that continues, with six teams' projected ratings falling at least four spots and just two (Penn State and Minnesota) improving.
To date, everything I have done with recruiting rankings has been at the team level. Eventually the goal is to reach rough conclusions based on individual talent as well -- what can one generally expect from a 5-star, first-year starter at quarterback, or a true freshman running back, etc. This, however, is a nice step forward in how to view recruiting rankings and the impact they may or may not have on your team's chances for success in the fall.
To something three decades in the making.
Close your eyes and imagine this for a moment. It is the BCS National Championship game. We are tied with five seconds left. Team A is lining up for the title-winning field goal. It is blocked! Team B recovers it and takes it the other way. They're going to score! Announcers, fans, players, coaches ... they're all going wild. As Team B's returner is about to score, the moment strikes him. He's about to win the national title in the most exciting title game finish ever. As he crosses the 5-yard line, the moment overtakes him, and he begins to high-step, or point to the stands, or raise his arms in victory ... and he is penalized for excessive celebration. And, with the NCAA's new, horrific rule change, it is a spot foul. No touchdown. The game goes to overtime.
I realize that with the No Fun League as its role model, the college football game has always been likely to over-penalize celebration, writing on eye black, or any other form of actual personality, but this a terrible rule.
This was pretty much the best debut for anybody, ever. And it's a lot harder to sneak up on a country with your utter dominance at the professional level, now, isn't it?
In honor of recruiting, Signing Day, hat ceremonies, etc.
"I Can't Sign My Name" by Big Joe Williams
"If I Could Only Win Your Love" by Emmylou Harris
"Kerosene Hat" by Cracker
"The Kerry Recruit" by The Dubliners
"Leopard-Skin Pill-Box Hat" by Bob Dylan
"New Recruit" by Eric Clapton
"Passing the Hat" by Cold War Kids
"Signs" by Tesla
"Street Signs" by Ozomatli
"Wasted Words" by Allman Brothers Band
Back in mid-December, when the Big Ten announced that it would be looking into expansion of its league, it also announced that the process would take at least one year, possibly two. It's going to be a long 10-22 months. So far, we've had Missouri's governor lobbying for Mizzou (and offending Texas Tech and Oklahoma State in the process), a wildfire "Pitt to the Big Ten! It's a done deal!!!" rumor spreading quickly despite not even being 1 percent true, the revelation that the Big Ten may have contacted Texas, and now the loud throat-clearing coming from Nebraska athletic director Tom Osborne.
Granted, the last couple of developments have added some entertainment value to the mix. First, we've gotten a healthy dose of Texas arrogance. Exhibit A: "We're the good-looking girl at the dance." Exhibit B: "I'm more convinced than ever that if Texas makes a move in the future, it could be a football-only move. Think Notre Dame. Football in one conference. Basketball and the non-revenue sports in another conference or regional alliance." (Fans of other Big 12 teams thank you, from the bottom of their comparatively cash-strapped hearts, for thinking about maybe, kind of, sort of sticking around to help them out in other sports after taking your football revenue to another conference. That's so kind of you.) Texas arrogance is actually a little bit refreshing -- they're not trying to convince you they're great and they hold all the cards because they know you already know they do. The "I can't help that we're Texas, but we are, and you know it," vibe is different from other forms of arrogance; it is both more jarring and more acceptable.
Then there's Nebraska. Certain SI reporters (both of whom I really enjoy) were tripping all over themselves to proclaim what a great choice the Huskers would be, but ... is this true? Nebraska is a "national name" in football in that they were great for decades, but a) since Eric Crouch left after the 2001 season, they are 63-40 (Missouri is 63-39) and are not one of the 30 FBS teams to have finished in a season's final AP Top 10 (for what it is worth, half of the Big 12 North -- Kansas, Kansas State, Missouri -- is on that list of 30 teams); b) while their football attendance obviously dwarfs that of any of the other Big Ten candidates north of Austin, theirs is dwarfed by that of Michigan, Ohio State and Penn State, somewhat minimizing the accomplishment; c) Omaha's market is smaller than that of Springfield, MO, while Lincoln's market is barely bigger than the Columbia/Jefferson City, MO, corridor, and that obviously ignores the impact in St. Louis and Kansas City if one certain Big 12 North team were chosen over another; and d) it's not like they are actually going to deliver more "national eyeballs." No cable company in other areas of the country is going to think to themselves, "Well, we didn't add the Big Ten Network while they had national names like Ohio State, Michigan, Penn State, et cetera, but now that they've added Nebraska, I'm in!"
Outside of the football writer community, NU does not appear to carry any more heft than other Big Ten expansion candidates (Holiday Bowl ratings this year were down 5%, though some of that might have obviously had something to do with the game's utter lack of competitiveness), but watching the simultaneous reactions of "They would be a homerun!!!" and "Yawn" from different corners of the Internet has certainly been entertaining.
(And yes, the entire two paragraphs above were written by a Missouri fan. Take it for what it is worth. But facts are facts. Nebraska has a case, but it's far from a no-brainer.)
But despite the entertainment, we are still only two months into As The Big Ten Expansion Turns, and it's hard to imagine the other rumors that may pop up by the time the Big Ten makes a decision. (And ironically, no matter what rumors pop up in the meantime, they could end up deciding not to expand at all.) Consider it one more way for college football to stay in the spotlight during the long offseason, especially now that the Pac-10 has thrown its hat into the expansion ring as well. This time around, conference expansion and realignment seem much more realistic than it did in the past, but ... you're still going to get awfully tired of hearing about it.
33 comments, Last at 22 Feb 2010, 7:15pm by Jeff Fogle