Instant replay review is one of the cornerstones of the modern NFL. The process and its myriad special rules have been internalized and constantly debated. Mike Kurtz wonders: is it worth it?
27 Aug 2010
by Bill Connelly
No matter how much some of us like computers and spreadsheets, we are all human. Our most recent impressions of teams have an impact on us, whether we intend it or not. If Team A starts slow but looks great in its bowl game, and if Team B starts like a house afire but fades down the stretch, we are going to think more highly of Team A, even if both teams finished with the same record. It may be natural, but is it accurate?
Last year around this time (almost to the day, in fact), I took a look at late-season momentum and whether it truly had an impact on the next season's performance. I also promised to revisit the subject when there was a healthier amount of data in the bank. It is time.
The question is a simple one: Can you derive anything from late-season hot streaks? If a team gets hot at the end of one season, does it carry over eight to nine months later, when the next season begins?
To answer this, I am going to create a measure (or a series of measures) and run correlations between said measures and the next season's performance. This simple approach is the basis for our current college projections. The amount of weight a factor is given depends on the strength of its correlations.
So how predictive is late-season success? Let's break out the chart we used during February's Recruiting Spectacular, updating it with the following late-season factors:
|Correlation between F/+ performance and common projection factors|
|Category||Factor/Correlation||BCS Conf.||Non-BCS Conf.|
|Last Year's Performance||Last Year's F/+
Last Year's S&P+
Last Year's FEI
Last Year's Point Differential
Last Year's Win %
|Historical Performance||2-Year History
|Recruiting Performance||4-Year ESPN Grade Avg.
5-Year Rivals Points Avg.
4-Year Rivals Points Avg.
5-Year Rivals Rank Avg.
5-Year Weighted Rivals Rank Avg.
|Late-Season Performance||Late-Season S&P+
|Correlation between change in F/+ performance and common projection factors|
|Category||Factor/Correlation||BCS Conf.||Non-BCS Conf.|
|Year-to-Year Change Factors||Last Year's Turnover Margin
Last Year's Pct. of Fum. Rec.
Offensive Starters Returning
Offensive Pct. of Lettermen Returning
Ret. Starting Quarterback
Defensive Starters Returning
Defensive Pct. of Lettermen Returning
Draft Points Lost
Ratio To Recent History + Starters
If the question is "Can you derive anything from late-season success?" then the answer is "Yes," as long as you know what you are looking for. Both Late-Season S&P+ and the starters-adjusted version tell us something, but full-season performance still tells us more. What is most interesting to note, however, is the ABC Index. The correlations between ABC Index and next year's success are comparable to those of other change factors we tend to take seriously -- turnover margin, starters returning, talent lost to draft, etc. But what happens on the far ends of the spectrum is most interesting.
If we broke all the teams from the past five seasons into four quadrants based on their ABC Index, here's what we would find: Teams in the top quadrant, the ones who experienced the highest level of truly worthwhile improvement, very often see their overall S&P+ improve the next season. In fact, a resounding 66.3 percent of them improved, as opposed to just 33.7 percent who regressed. Meanwhile, those in the bottom quadrant, the teams who both regressed significantly and lost a good portion of talent, saw their S&P+ drop the next season 66 percent of the time. The teams in the middle 50 percent do not show any major trends, which hurts the overall correlations, but it does appear that there is something to the combination of late-season play and experience. Coffee is for closers.
|Top 20 BCS Conference (plus Boise State and TCU)
Teams According to ABC Index, 2005-08
Of the 20 major conference programs on the list above, 15 improved the next season. Meanwhile, 13 of the 20 worst teams according to ABC index (all of whom measured at 0.43 or worse in ABC Index) regressed. That qualifies as something of a trend.
Here are the six major conference teams whose 2009 ABC Index was 0.83 or better: North Carolina (1.05), Boise State (0.98), Missouri (0.86), Oregon (0.83), Washington (0.83). Obviously North Carolina's current uncertain status means their ratio could change soon, but at the moment, at least three of these five teams are likely to improve by a solid amount in 2010. This offseason's darlings -- Nebraska, Texas A&M, Arkansas -- finished in what constitutes the second quadrant. Nebraska and Texas A&M improved down the stretch, but not nearly to the extent of the listed teams.
On the flip side, here are the major conference teams whose ABC Index was the worst in 2009: Arizona State (0.34), Oklahoma State (0.37), LSU (0.43), Ole Miss (0.44), Tennessee (0.44), and Iowa State (0.47). The three SEC teams on this list all made the cut in our FO Top 25 a couple of months ago. When the updated projections come out this time next week, that might not be the case anymore, at least for two of them.
Does a great bowl performance lead to great next season? It seems that most "offseason darlings" (Georgia in 2008, Ole Miss in 2009, Nebraska in 2010) dominated a bowl foe and receive a bit of extra hype because of it. As mentioned above, it does appear that there is value in late-season momentum, as long as you know how to find it. What about the power of a single game in late December or early January?
The verdict is, predictably, not so favorable. The correlation between a team's performance in a bowl and their performance the next season is 0.08. Only 39 percent of the teams in the top quadrant of bowl performers actually improved the next season.
|Top 20 Bowl Performances According to Single-Game "+" Score*, 2005-08|
|Tulsa||2007||GMAC||def. Bowling Green, 63-7||426.0||+1.8%|
|Georgia||2007||Sugar||def. No. 10 Hawaii, 41-10||363.5||+4.2%|
|Oklahoma State||2007||Insight||def. Indiana, 49-33||343.0||+6.4%|
|LSU||2005||Peach||def. No. 9 Miami, 40-3||341.6||+5.9%|
|Missouri||2007||Cotton||def. Arkansas, 38-7||335.3||-0.1%|
|Wisconsin||2005||Capital One||def. No. 7 Auburn, 24-10||325.4||+4.7%|
|Rutgers||2007||International||def. Ball State, 52-30||318.9||-1.4%|
|Connecticut||2008||International||def. Buffalo, 38-20||310.0||-3.4%|
|LSU||2008||Peach||def. No. 14 Georgia Tech, 38-3||306.7||+7.5%|
|Mississippi State||2007||Liberty||def. Central Florida, 10-3||305.1||-9.5%|
|Oregon||2007||Sun||def. No. 21 South Florida, 56-21||303.0||-2.8%|
|USC||2007||Rose||def. No. 13 Illinois, 49-17||301.0||+12.1%|
|USC||2008||Rose||def. No. 8 Penn State, 38-24||297.5||-23.7%|
|TCU||2008||Poinsettia||def. No. 9 Boise State, 17-16||292.8||+2.9%|
|West Virginia||2007||Fiesta||def. No. 4 Oklahoma, 48-28||291.7||-24.2%|
|Tulsa||2008||GMAC||def. No. 22 Ball State, 45-13||291.0||-5.8%|
|New Mexico||2007||New Mexico||def. Nevada, 23-0||291.0||+0.2%|
|Wake Forest||2007||Meineke Car Care||def. No. 25 UConn, 24-10||290.9||-0.4%|
|Texas||2007||Holiday||def. No. 11 Arizona State, 52-34||290.0||+16.0%|
|Notre Dame||2008||Hawaii||def. Hawaii, 49-21||286.6||+7.2%|
|* While the S&P+ measure is recorded by taking every game of the season into account, each game a team plays brings with it a single-game "+" score derived using the exact same formulas as full-season S&P+.|
Of the 20 best bowl performers from 2005-08, 10 improved the next season and 10 regressed. Of the 20 worst bowl performers, it was the same story. For every positive story, there is an equal and opposite story.
Here is the bottom line. There is value in determining who gained worthwhile momentum during the last 3-5 games of the season, but you should not spend even five seconds attempting to derive value in bowl performances.
To the Notre Dame blog Rakes of Mallow, for burning the image of Will Forte as Dayne Crist into my head.
Another small round of applause to all the players and programs out there who haven't gotten into trouble this offseason. Dwight Dasher, Derrick Washington, the entire North Carolina defense ... the recent week has just continued what has quite plainly appeared to be one of the most trouble-plagued, stressful offseasons in memory. If your team has, for the most part, escaped injury, suspension, dismissal, or NCAA punishment, consider yourself lucky. College football fans deserve a great 2010 season after the less-than-enjoyable last 12 months.
Note to the Big Ten: As I mentioned on Twitter on Tuesday (and SI.com's Stewart Mandel did around the same time) -- stop overthinking this. I try not to get worked up about things I probably won't really care about two years from now, and therefore I'll resist spending too many words on the likely move of Michigan-Ohio State away from the end of the season. Dr. Saturday already did a great job of that anyway (as did Smart Football's Chris Brown in response). I don't like it, but I probably won't care too much in the future. It's like when Nebraska and Oklahoma stopped playing every year. In time, I can stomach the loss of one great thing when it comes with the addition of two great things.
But I can't let the divisions conversation go without commenting. Quick: List the members of both ACC divisions in 10 seconds or less. If you can do that, you're either a) a fan of an ACC school, or b) an even bigger nerd than I am. I have always been in favor of geography-based divisions instead of the scattershot approach. Here's where you say, "You're an alum of a Big 12 North school. Of course you are going to prefer geographic divisions." While I have selfish reasons for preferring geography, it has nothing to do with Missouri's fortunes in the Big 12. It has to do with simple memory. I don't have to think to tell you that Ole Miss is in the SEC West. It is a reflex. I don't want to have to use process of elimination to remember something, like I do with the ACC. "Let's see, Wisconsin is in the same division as Minnesota, but not Iowa ... so that means they're also in a division with Nebraska and, I think, Michigan ... carry the one ... so I think that means they're in the National Division. Or was it the American Division? I can't remember." Geography is intuitive. I'd be willing to send Penn State to the Big Ten West if it meant keeping all the other geographic integrity intact. Two years from now, I probably won't care that Michigan and Ohio State are playing in October. But I'll damn sure be getting annoyed every time I realize I can't remember -- off the top of my head -- the division in which Purdue resides.
But hey, it's not like the Big Ten is deciding to move to an 18-game schedule. So they've got that going for them.
In honor of those who catch fire late in the year...
"Closing Time," by Semisonic (too easy not to include)
"Elevate Me Later," by Pavement
"Fire," by the Ohio Players
"Getting Better," by the Beatles
"Have You Seen Me Lately," by Counting Crows
"I'm on Fire," by Bruce Springsteen
"The Late Greats," by Wilco
"Late in the Evening," by Paul Simon
"Not Too Late," by Norah Jones
"Too Little Too Late," by Barenaked Ladies
Always. Be. Closing.
2 comments, Last at 30 Aug 2010, 7:24pm by BGNoMore