Varsity Numbers: When Coaches Leave

by Bill Connelly

Each offseason, about 15-25 athletic directors deal with it, and another 15-25 deal with rumors of it. There is no drug for it, and there is no sure solution. You do your research, and you make an educated guess that makes the most sense for you, and you pray that it works out. It is coaching change, and it is the scourge of statistical projections.

Looking through the college football projections found in the Football Outsiders Almanac, a common thread ties together many of the more surprising picks -- new coaches. No. 16 Texas Tech, No. 20 Tennessee, even No. 49 Kansas ... FOA 2010 projects these teams all higher than conventional wisdom, and the primary reason for the differential is the drop-off they are expected to incur in the first year under a new leader. (Of course, there are other factors involved here too -- Tech and Tennessee have major inexperience issues on the offensive line, and Kansas has the same in skill positions and at linebacker.) Is that fair? Are we humans assuming regression to our own peril?

In general, it is difficult to get a read on how a coaching change might impact a team. Sometimes the change leads to immediate improvement, and sometimes a team falls off a cliff. Sometimes a team takes a temporary step backwards while learning a new system, and sometimes it takes an unexpected surge, quickly proving that a new system was desperately needed. This week, we'll use this space to see if there are any conclusions available regarding these changes. Obviously every hire has different limitations or requirements, but are there some statistical generalizations we can make? In other words, is there any way to incorporate coaching changes into the F/+ projections?

Where You Are vs. Where You Have Been

Thinking of coaching changes from a projections standpoint, two approaches initially come to mind: 1) looking at program potential, and 2) type of hire. We'll first tackle the issue of program potential. (If you have other suggestions, let us hear them in the comments section.)

Every so often, a historical power struggles under one coach, dumps him for a new one, and rebounds. Oklahoma went five straight years without a winning record from 1994-98, hired Bob Stoops, and won the national title two years later. Florida lost five games per year under Ron Zook, then lost four combined in Urban Meyer's first two seasons. They also won a title in year No. 2. Nick Saban, Mack Brown, and Pete Carroll almost immediately reversed the fortunes of Alabama, Texas and USC, respectively. Hiring a new coach doesn't make an old power good again, of course -- it has to be the right hire. I mentioned in the 2009 Football Outsiders Almanac that the only reason any of these great hires were made was because the last hire was poor. But if or when the right hire is made, the rebound is so much easier when you've got historical clout. Historical programs have more money (with which they can draw the big-time coaches) and get more benefit-of-the-doubt from recruits. God bless the oligarchy known as college football.

To put numbers to this, we're going to compare how a team did the previous season to how they did in the previous 10. We will call this a team's Ratio to Recent History. We could shrink that down to an acronym, but a) it might not ever get used again after this column, b) "RTRH" doesn't exactly roll off the tongue, and c) quite frankly, there are already too many acronyms in this world. So Ratio to Recent History it is. If the ratio is above 1.0, that means the team's performance last season was higher than its 10-year baseline. Below 1.0, and it was worse. (This was attempted with five, 10 and 20 years as the baseline, and 10 was the most statistically significant.)

Do BCS conference teams who underachieved compared to recent history tend to rebound a bit faster? Yes, to an extent. Below is data collected from between the 2003-09 seasons. Where it is available (2005-09), S&P+ data is used. Before 2005, Est. S&P+ is substituted.

Teams who finished above or below their 10-year baseline the year before a coach was hired tended to regress to the mean in the coach's first season. If you were below the baseline (within reason), you will likely improve a bit. The first-year change of plus-2.1 percent represents approximately 4-6 spots in the S&P+ rankings.

Meanwhile, if you overachieved before losing your coach, you tend to regress by about 5.9 percent, which could mean a fall of 10-14 spots in the rankings. In theory, this makes sense -- if you overachieved, the chances are greater that your coach was hired away to a bigger job. For the five 2010 teams that fall into this quadrant, however, only one had their coach hired away (Cincinnati). Two (Kentucky, Vanderbilt) are replacing retiring coaches, and, strangely enough, two are replacing coaches who were fired for player abuse. There is not yet enough of a sample size to add "reasons for departure" into this study, but regardless, fans of these schools may be in for an unpleasant autumn.

Then there is the "

What do we see when we look at the same data for non-BCS automatic qualifying conferences?

The same dynamic is at play, but there are differences. Teams indeed improve by a decent amount if they were below their 10-year baseline the previous season, but teams who slightly overachieved tend to hold steady instead of regressing definitively. Whether this is because of the talent the departing coach left behind (there is parity in terms of talent at the non-BCS level, so a medium-sized batch of it could work wonders) or other factors, it is so far unclear.

Meanwhile, the teams who overachieved significantly (Ball State 2008, New Mexico State 2004, Akron 2003) took sharp tumbles after losing their leader. This is bad news for Buffalo and UL-Monroe.

So what happens if we were to add this "Ratio to Recent History + New Coach" factor into the F/+ projections? How much weight would this factor carry? At the BCS conference level, the correlation between this factor and the next season's performance is 0.189. At the non-BCS level, it is 0.288. This is actually relatively strong for a change factor. For perspective, here are some other change factors and their correlations:

• Last year's turnover margin: 0.273
• Total starters returning: 0.262
• Picks in last year's NFL Draft: 0.193
• Last year's fumble recovery percentage: 0.152

So "Ratio to Recent History + New Coach" factor is as strong a predictor of many factors we weight heavily in our own judgment (turnover margin, talent lost, etc.). That appears strong enough to work into the projections. But here is something even more interesting: the "Ratio to Recent History + Second-Year Coach" factor is even stronger, at least at the BCS level. A team's recent performance (as compared to history) and its predictive ability for teams with second-year coaches is 0.275, almost exactly the same as turnover margin. Is it possible that the first year under a new coach is a bit of a crap shoot, but by the second season, the historically strong programs will be able rather predictably make their move (if they've got a move to make)?

Let's look at the same data as above, only with second-year change.

As a whole, teams tend to improve in their second seasons under a new head coach. It is uncertain at this point whether he is likely to succeed on a large level yet, but with a few more of his own players in the program, and with a year for players from the previous regime to learn a new system, teams take steps forward. But if things clicked a little too well in the coach's first season (think Baylor 2008, N.C. State 2007), then they might regress at least a little bit in Year Two. These are not earth-shattering conclusions here, but it is nice to see some statistical heft behind the seemingly easy-to-understand concept.

Are we told the same story with the non-BCS data? Yes.

In general, no matter where a team is in its coach's regime, they are likely to regress (or ascend) back toward their historic power level. But the correlation is stronger in a coach's first couple of seasons.

Does the Type of Coach Matter?

We've taken a look at things from a program standpoint. Is there anything we can learn from the type of coach that a school hires? Using the same pool of data (2003-09 performance), I separated all recent coaching hires into one of seven categories: college assistants, BCS conference head coaches, non-BCS head coaches, recently fired BCS conference coaches, FCS (formerly 1-AA) head coaches, coaches from the NFL level, and high school coaches. Only one high school coach has been hired since 2003 (Todd Dodge at North Texas), so that category is eliminated from the below data.

Right away you are probably finding a problem -- most coaches could be placed in multiple categories. This is not the most statistically sound way of looking into the issue. For the most part, I labeled an incoming coach by his most notable or applicable position. When Oregon State's Mike Riley returned to Oregon State in 2003, he was both a BCS coach and an NFL coach -- for these purposes, he was a BCS coach, since that experience was quite applicable in his hiring. Nebraska's Bo Pelini was a successful college assistant and former NFL assistant, and he was the interim head coach in Nebraska's 2003 bowl win. He was categorized as a college assistant because of his most recent work as defensive coordinator at LSU, Oklahoma, and Nebraska. There are hundreds of different ways to categorize a given coach. Does that gray area prevent us from decent findings? Not necessarily. Even with the expected shakiness of the data, tying a coach "type" to first-year performance gives us a 0.212 correlation, which, again, falls right in the middle of most of the change factors used for the F/+ projections.

Here is how the data shakes down over a coach's first two seasons:

First things first. If you hire an NFL guy, there will almost certainly be a breaking-in period. Notre Dame's sudden improvement under Charlie Weis is the exception to the rule. (Plus, he somewhat violently regressed to the mean over time.) Bill Callahan led Nebraska to a 16.7-percent decrease in Est. S&P+ in his first year. Texas A&M saw a 15.5-percent regression in Mike Sherman's first season (2008). Tim Brewster oversaw a 16.8-percent regression at Minnesota in 2007, while career NFL assistant Mike Shula led Alabama to a 7.5-percent regression in his first year there. Only Weis (+10.5 percent) and Pittsburgh's Dave Wannstedt (+0.9 percent) oversaw any sort of improvement, and Wannstedt's was negligible.

The first year appears to be a rough one for FCS-level coaches as well, but the sample size is small, and Washington State's complete collapse under Paul Wulff (Wazzu plummeted 26.2 percent in their first season under the former Eastern Washington coach) has skewed the data rather significantly. Purdue improved by 8.9 percent under Danny Hope last season, while Mississippi State improved slightly (+2.3 percent) under Sylvester Croom in 2004. Hope is not lost for Virginia and first-year coach Mike London, as long as he does not turn out to be Wulff II. (There is a "Werewulffs of London" joke to be made in there, but I will abstain because I am not Chris Berman.)

It should be no surprise that BCS-level coaches tend to succeed a bit more than others. They likely would not go from one BCS school to another (without being fired) if they weren't very good at their jobs. It doesn't always work out (see: Rodriguez, Rich), but it is, by a small margin, the safest bet of the bunch. Of course, only certain schools are able (i.e. rich enough) to pull a coach away from another BCS job, and those teams likely fall into the "historically successful" category, so of course there is noise throughout this data. It was inevitable.

At the non-BCS level, teams are picking from a different pool of candidates. How does the data shape up there?

One thing is certain: If you can stumble across a major-conference coach at this level, snatch him up. Unfortunately, it takes pretty crazy circumstances for that to happen. The only two coaches who went from a BCS conference to a non-BCS conference without getting fired for performance-related issues were George O'Leary at Central Florida (successful at Georgia Tech, hired by Notre Dame before it was revealed that he fudged his résumé) and Mike Price at UTEP (successful at Washington State, hired by Alabama, fired before his first game for questionable behavior). Because neither was removed from his job for matters pertaining to their coaching ability, they both were labeled BCS Conf. Coaches.

Have we stumbled across a goldmine of amazing data here? Probably not, but we have certainly come up with a way to quantify coaching changes with correlations strong enough to work into the projection formulas. Depending on who is doing the hiring, and what kind of coach is being hired, we might be able to make some approximate judgments about whether they will charge out of the gates, or whether they will take a while to get momentum.

USC benefits from hiring a coach with BCS-level experience (albeit just one year), but last year's performance suggests they might be in "starting from scratch" mode to some degree. Notre Dame made as close to a slam dunk hire as you can make, bringing in a successful BCS-level coach at a time when their program is likely to bounce back a bit. Kansas appears likely to regress temporarily, having brought in a mid-major coach at a time when the school has been overachieving a bit (though telling a Kansas fan that his/her team overachieved last year while losing seven in a row at the end of the 2009 season might draw a guffaw ... then again, 35 percent of Kansas fans think their football coach's name is Bill Self, so they've got that going for them).

Incorporating coaching changes into the projections (which will be updated two weeks from today) likely will not change a lot, but they could have an impact on a small handful of teams.

Random Golf Clap

To TCU, for aiming high. On Monday, the TCU athletic program announced a ridiculous $105 million plan to renovate Amon G. Carter Stadium. Expanding by a few thousand seats? Yawn. They're actually decreasing the permanent seating from 44,358 to 40,000 (last season's average attendance: 38,186) and adding luxury boxes and ... well, beauty. Not content to update their ancient stadium, they are aiming to turn it into "the Camden Yards of collegiate football stadiums."

To the Mountain West, for aiming higher. BYU starts to walk out of the door, hoping to move its football team to independent status and its non-football teams to the WAC, so the Mountain West kneecaps the WAC by inviting Fresno State, Nevada and evidently Utah State to join the Mountain West. Fresno State and Nevada accept, leaving the WAC with just six programs and leaving BYU's decision in flux. You can't move to another conference if the conference no longer exists, right?

And to BYU, for aiming highest. Whether you think BYU's attempted move toward independence (not yet official) is smart or foolish, it is definitely high in testicular fortitude. It is a high-risk effort with potentially outstanding rewards. Execution matters, but they get an "A+" for bravery. Until they decide not to leave the MWC after all, anyway.

Random Mini-Rant

You turned them down, Utah State? You could have joined the Mountain West, and you said no?!

(One can see how, when they turned down the offer, USU officials might have thought that the WAC might be at least a semi-stable long-term choice. Plus, in the cannibalistic world of college sports, loyalty is something upon which we should not frown. But ... damn.)

Random Reasons to Love College Football

I cede this section to Sports Illustrated's Austin Murphy this week. This line was particularly outstanding: "However the landscape of this sport changes during the next several seasons, college football will still be more beautiful, more interesting and more fun than the dome-sheltered, corporate, conformist game that's played in the NFL."

Random Playlist

In honor of all the new coaches out there.

"Brand New Day" by Van Morrison
"New Pony" by Bob Dylan
"The New Style" by Beastie Boys
"Trouble No More" by Muddy Waters
"Welcome" by The Who
"Welcome to Heartbreak" by Kanye West
"Welcome to Paradise" by Green Day
"Welcome to the Cruel World" by Ben Harper & The Innocent Criminals
"Welcome to the Jungle" by Guns N' Roses
"Welcome to the Show" by J Dilla

Closing Thoughts

Thirteen days until Presbyterian and Wake Forest kick off to begin the 2010 college football season. Yes, yes, yes.