Writers of Pro Football Prospectus 2008

Most Recent FO Features

BradyTom00-10.jpg

» Week 4 DVOA Ratings

Five different teams from last year's DVOA top eight rank in the bottom half of the league through four weeks of 2014. What can we learn from other teams with similar starts in the past?

27 Aug 2010

Varsity Numbers Catches Fire

by Bill Connelly

No matter how much some of us like computers and spreadsheets, we are all human. Our most recent impressions of teams have an impact on us, whether we intend it or not. If Team A starts slow but looks great in its bowl game, and if Team B starts like a house afire but fades down the stretch, we are going to think more highly of Team A, even if both teams finished with the same record. It may be natural, but is it accurate?

Last year around this time (almost to the day, in fact), I took a look at late-season momentum and whether it truly had an impact on the next season's performance. I also promised to revisit the subject when there was a healthier amount of data in the bank. It is time.

Late in the Season

The question is a simple one: Can you derive anything from late-season hot streaks? If a team gets hot at the end of one season, does it carry over eight to nine months later, when the next season begins?

To answer this, I am going to create a measure (or a series of measures) and run correlations between said measures and the next season's performance. This simple approach is the basis for our current college projections. The amount of weight a factor is given depends on the strength of its correlations.

So how predictive is late-season success? Let's break out the chart we used during February's Recruiting Spectacular, updating it with the following late-season factors:

  • Late-Season S&P+: A team's S&P+ for games taking place on November 1 or later.
  • Late-Plus-Starters: A team's Late-Season S&P+ multiplied by the percentage of starters they return for the coming season. Obviously getting hot late in the year doesn't matter as much if the players responsible for the good momentum depart.
  • My personal favorite, the ABC Ratio: A team's Late-Plus-Starters figure divided by the teams' full-season S&P+. In this way, we can differentiate between teams who got hot late and teams who were simply great all season long. Why ABC? Always Be Closing (NSFW), that's why. Second prize doesn't get a set of steak knives, but rather a trip to the SteakKnives.com Bowl.
Correlation between F/+ performance and common projection factors
Category Factor/Correlation BCS Conf. Non-BCS Conf.
Last Year's Performance Last Year's F/+
Last Year's S&P+
Last Year's FEI
Last Year's Point Differential
Last Year's Win %
0.615
0.597
0.593
0.611
0.586
0.616
0.689
0.593
0.630
0.602
Historical Performance 2-Year History
3-Year History
4-Year History
5-Year History
0.515
0.542
0.552
0.546
0.725
0.723
0.679
0.634
Recruiting Performance 4-Year ESPN Grade Avg.
5-Year Rivals Points Avg.
4-Year Rivals Points Avg.
5-Year Rivals Rank Avg.
5-Year Weighted Rivals Rank Avg.
0.600
0.547
0.545
0.476
0.468
N/A
0.138
0.423
0.346
0.343
Late-Season Performance Late-Season S&P+
Late-Plus-Starters
0.584
0.430
0.518
0.385
Correlation between change in F/+ performance and common projection factors
Category Factor/Correlation BCS Conf. Non-BCS Conf.
Year-to-Year Change Factors Last Year's Turnover Margin
Last Year's Pct. of Fum. Rec.
Offensive Starters Returning
Offensive Pct. of Lettermen Returning
Ret. Starting Quarterback
Defensive Starters Returning
Defensive Pct. of Lettermen Returning
Draft Points Lost
Ratio To Recent History + Starters
ABC Index
0.301
0.226
0.240
0.135
0.116
0.155
0.239
0.257
0.189
0.211
0.074
0.036
0.364
0.088
0.184
0.390
0.479
0.164
0.288
0.133

If the question is "Can you derive anything from late-season success?" then the answer is "Yes," as long as you know what you are looking for. Both Late-Season S&P+ and the starters-adjusted version tell us something, but full-season performance still tells us more. What is most interesting to note, however, is the ABC Index. The correlations between ABC Index and next year's success are comparable to those of other change factors we tend to take seriously -- turnover margin, starters returning, talent lost to draft, etc. But what happens on the far ends of the spectrum is most interesting.

If we broke all the teams from the past five seasons into four quadrants based on their ABC Index, here's what we would find: Teams in the top quadrant, the ones who experienced the highest level of truly worthwhile improvement, very often see their overall S&P+ improve the next season. In fact, a resounding 66.3 percent of them improved, as opposed to just 33.7 percent who regressed. Meanwhile, those in the bottom quadrant, the teams who both regressed significantly and lost a good portion of talent, saw their S&P+ drop the next season 66 percent of the time. The teams in the middle 50 percent do not show any major trends, which hurts the overall correlations, but it does appear that there is something to the combination of late-season play and experience. Coffee is for closers.

Top 20 BCS Conference (plus Boise State and TCU)
Teams According to ABC Index, 2005-08
Team Year ABC
Index
S&P+ Chg
(Y+1)
Arizona 2005 1.10 +2.0%
Arkansas 2005 1.08 +5.5%
Florida 2008 0.98 -1.0%
Ole Miss 2008 0.97 -3.2%
Ohio State 2007 0.94 -1.4%
UCLA 2006 0.94 +6.1%
Georgia 2007 0.93 +4.2%
Virginia Tech 2006 0.92 +2.8%
Purdue 2006 0.90 +11.4%
TCU 2007 0.89 +24.9%
Stanford 2006 0.89 +12.6%
Texas Tech 2007 0.89 +5.5%
Virginia 2006 0.88 +7.1%
Boise State 2005 0.87 +10.6%
Georgia Tech 2008 0.86 +8.4%
South Florida 2006 0.86 +12.3%
Clemson 2008 0.85 +6.9%
Wisconsin 2006 0.85 -6.8%
Vanderbilt 2006 0.84 +2.6%
Minnesota 2006 0.83 -16.8%

Of the 20 major conference programs on the list above, 15 improved the next season. Meanwhile, 13 of the 20 worst teams according to ABC index (all of whom measured at 0.43 or worse in ABC Index) regressed. That qualifies as something of a trend.

Here are the six major conference teams whose 2009 ABC Index was 0.83 or better: North Carolina (1.05), Boise State (0.98), Missouri (0.86), Oregon (0.83), Washington (0.83). Obviously North Carolina's current uncertain status means their ratio could change soon, but at the moment, at least three of these five teams are likely to improve by a solid amount in 2010. This offseason's darlings -- Nebraska, Texas A&M, Arkansas -- finished in what constitutes the second quadrant. Nebraska and Texas A&M improved down the stretch, but not nearly to the extent of the listed teams.

On the flip side, here are the major conference teams whose ABC Index was the worst in 2009: Arizona State (0.34), Oklahoma State (0.37), LSU (0.43), Ole Miss (0.44), Tennessee (0.44), and Iowa State (0.47). The three SEC teams on this list all made the cut in our FO Top 25 a couple of months ago. When the updated projections come out this time next week, that might not be the case anymore, at least for two of them.

Bowl Magic

Does a great bowl performance lead to great next season? It seems that most "offseason darlings" (Georgia in 2008, Ole Miss in 2009, Nebraska in 2010) dominated a bowl foe and receive a bit of extra hype because of it. As mentioned above, it does appear that there is value in late-season momentum, as long as you know how to find it. What about the power of a single game in late December or early January?

The verdict is, predictably, not so favorable. The correlation between a team's performance in a bowl and their performance the next season is 0.08. Only 39 percent of the teams in the top quadrant of bowl performers actually improved the next season.

Top 20 Bowl Performances According to Single-Game "+" Score*, 2005-08
Team Year Bowl Result Game "+"
Score
S&P+ Chg
(Y+1)
Tulsa 2007 GMAC def. Bowling Green, 63-7 426.0 +1.8%
Georgia 2007 Sugar def. No. 10 Hawaii, 41-10 363.5 +4.2%
Oklahoma State 2007 Insight def. Indiana, 49-33 343.0 +6.4%
LSU 2005 Peach def. No. 9 Miami, 40-3 341.6 +5.9%
Missouri 2007 Cotton def. Arkansas, 38-7 335.3 -0.1%
Wisconsin 2005 Capital One def. No. 7 Auburn, 24-10 325.4 +4.7%
Rutgers 2007 International def. Ball State, 52-30 318.9 -1.4%
Connecticut 2008 International def. Buffalo, 38-20 310.0 -3.4%
LSU 2008 Peach def. No. 14 Georgia Tech, 38-3 306.7 +7.5%
Mississippi State 2007 Liberty def. Central Florida, 10-3 305.1 -9.5%
Team Year Bowl Result Game "+"
Score
S&P+ Chg
(Y+1)
Oregon 2007 Sun def. No. 21 South Florida, 56-21 303.0 -2.8%
USC 2007 Rose def. No. 13 Illinois, 49-17 301.0 +12.1%
USC 2008 Rose def. No. 8 Penn State, 38-24 297.5 -23.7%
TCU 2008 Poinsettia def. No. 9 Boise State, 17-16 292.8 +2.9%
West Virginia 2007 Fiesta def. No. 4 Oklahoma, 48-28 291.7 -24.2%
Tulsa 2008 GMAC def. No. 22 Ball State, 45-13 291.0 -5.8%
New Mexico 2007 New Mexico def. Nevada, 23-0 291.0 +0.2%
Wake Forest 2007 Meineke Car Care def. No. 25 UConn, 24-10 290.9 -0.4%
Texas 2007 Holiday def. No. 11 Arizona State, 52-34 290.0 +16.0%
Notre Dame 2008 Hawaii def. Hawaii, 49-21 286.6 +7.2%
* While the S&P+ measure is recorded by taking every game of the season into account, each game a team plays brings with it a single-game "+" score derived using the exact same formulas as full-season S&P+.

Of the 20 best bowl performers from 2005-08, 10 improved the next season and 10 regressed. Of the 20 worst bowl performers, it was the same story. For every positive story, there is an equal and opposite story.

Here is the bottom line. There is value in determining who gained worthwhile momentum during the last 3-5 games of the season, but you should not spend even five seconds attempting to derive value in bowl performances.

Random Golf Clap

To the Notre Dame blog Rakes of Mallow, for burning the image of Will Forte as Dayne Crist into my head.

Another small round of applause to all the players and programs out there who haven't gotten into trouble this offseason. Dwight Dasher, Derrick Washington, the entire North Carolina defense ... the recent week has just continued what has quite plainly appeared to be one of the most trouble-plagued, stressful offseasons in memory. If your team has, for the most part, escaped injury, suspension, dismissal, or NCAA punishment, consider yourself lucky. College football fans deserve a great 2010 season after the less-than-enjoyable last 12 months.

Random Mini-Rant

Note to the Big Ten: As I mentioned on Twitter on Tuesday (and SI.com's Stewart Mandel did around the same time) -- stop overthinking this. I try not to get worked up about things I probably won't really care about two years from now, and therefore I'll resist spending too many words on the likely move of Michigan-Ohio State away from the end of the season. Dr. Saturday already did a great job of that anyway (as did Smart Football's Chris Brown in response). I don't like it, but I probably won't care too much in the future. It's like when Nebraska and Oklahoma stopped playing every year. In time, I can stomach the loss of one great thing when it comes with the addition of two great things.

But I can't let the divisions conversation go without commenting. Quick: List the members of both ACC divisions in 10 seconds or less. If you can do that, you're either a) a fan of an ACC school, or b) an even bigger nerd than I am. I have always been in favor of geography-based divisions instead of the scattershot approach. Here's where you say, "You're an alum of a Big 12 North school. Of course you are going to prefer geographic divisions." While I have selfish reasons for preferring geography, it has nothing to do with Missouri's fortunes in the Big 12. It has to do with simple memory. I don't have to think to tell you that Ole Miss is in the SEC West. It is a reflex. I don't want to have to use process of elimination to remember something, like I do with the ACC. "Let's see, Wisconsin is in the same division as Minnesota, but not Iowa ... so that means they're also in a division with Nebraska and, I think, Michigan ... carry the one ... so I think that means they're in the National Division. Or was it the American Division? I can't remember." Geography is intuitive. I'd be willing to send Penn State to the Big Ten West if it meant keeping all the other geographic integrity intact. Two years from now, I probably won't care that Michigan and Ohio State are playing in October. But I'll damn sure be getting annoyed every time I realize I can't remember -- off the top of my head -- the division in which Purdue resides.

But hey, it's not like the Big Ten is deciding to move to an 18-game schedule. So they've got that going for them.

Random Playlist

In honor of those who catch fire late in the year...

"Closing Time," by Semisonic (too easy not to include)
"Elevate Me Later," by Pavement
"Fire," by the Ohio Players
"Getting Better," by the Beatles
"Have You Seen Me Lately," by Counting Crows
"I'm on Fire," by Bruce Springsteen
"The Late Greats," by Wilco
"Late in the Evening," by Paul Simon
"Not Too Late," by Norah Jones
"Too Little Too Late," by Barenaked Ladies

Closing Thoughts

Always. Be. Closing.

Posted by: Bill Connelly on 27 Aug 2010

2 comments, Last at 30 Aug 2010, 7:24pm by BGNoMore

Comments

1
by zlionsfan :: Fri, 08/27/2010 - 2:43pm

I'm not sure I understand the comparison between the reactions of Doc Saturday and Chris Brown ... Doc Saturday described in detail why he felt it was a stupid thing to do and Brown basically tweeted "nuh-uh." I enjoy reading Smart Football, but Brown's reply came across to me as thoughtless.

But maybe you react the way you do (apparently more strongly about the divisions than moving The Game) because of your background. Do you think you might see it differently if you'd grown up as an Auburn fan or a Texas fan or a USC fan? Missouri has rivals, sure (hey, one of my NCAA 10 characters signed with Missouri, I know a little), but it doesn't strike me as being the same.

To me, as an Ann Arbor native, the UM-OSU game belongs on the last week of the Big Ten schedule, end of discussion. I actually feel I'm in the minority among UM fans because I can discuss it somewhat rationally. (Check out mgoblog, for example, if you want to see what I mean.) You're accustomed to playing your biggest rival at the end of the season (conference championship notwithstanding). Even at Purdue, in 2001 when scheduling anomalies put two games after the Bucket (Hawaii, regularly-scheduled, and Notre Dame, rescheduled because of 9/11), the games after IU just weren't the same.

I would also prefer geographic divisions. For one thing, it makes things easier for other sports, even if there is extra TV money around to help defray those costs. For another, attempts to "balance" a conference simply can't work across sports. Few schools are strong at both football and men's basketball, and what might work temporarily in one sport (or not at all; how's that FSU-Miami championship game, ACC?) might not work at all in another. The current layout lends itself well to a geographic split that preserves most rivalries anyway (UM, MSU, OSU, PSU, Indiana and Purdue in the East).

2
by BGNoMore (not verified) :: Mon, 08/30/2010 - 7:24pm

Put that coffee down, Bill.