VN: Impressions From House-Cleaning

December 23, 8:04 pm ET

Photo: USA Today Sports Images

by Bill Connelly

As each season progresses, I phase out preseason projections and raw data in favor of the opponent-adjusted S&P+ you've come to know and ~~despise~~ love. This year, however, I noticed something: even seven or eight weeks into the season, leaving a little more of the raw data in, and diluting the opponent adjustments a bit, seemed to improve both the correlations between S&P+ and win percentage, and with it, the general predictive success of S&P+ as a whole.

The former improvement makes sense, of course -- the raw data is likely going to correlate well with win percentage when major conference teams are playing major conference teams and mid-majors are playing mid-majors. The whole point of using opponent adjustments is to figure out how to see through the ridiculous raw statistics a team may post while playing nobody in particular. But it was the latter that surprised me. (Honestly, I would have assumed more adjustment was needed.) There will be more observation in this regard in the offseason, but it appears that diluting the opponent adjustment a bit improves S&P+ overall. And if it improves S&P+, then obviously it improves F/+ as well.

So with time between the regular season and the chaos of next week's bowl previews (catch up on my bowl work for SB Nation here), I set about updating previous years of data to see if the same improvement took shape. Again, there will be a more in-depth investigation into general predictive abilities this offseason, but for now, the changes have absolutely improved the general correlations of S&P+ and F/+ to wins and losses.

When combined with Brian Fremeau's recent 2007-10 FEI changes, which allow us to make the same special teams adjustments to F/+ that we are using in 2011, that gives us a whole new set of better, more accurate rankings.

2005
New F/+ Rankings
New S&P+ Rankings

(National champions shaded in yellow below.)

2005 F/+ Top Ten

F/+ Rk	Team	Record	F/+	S&P+	S&P+ Rk	FEI	FEI Rk
1	Texas	13-0	+35.9%	279.9	1	0.342	1
2	USC	12-1	+33.3%	272.4	2	0.326	2
3	Ohio State	10-2	+29.2%	264.8	3	0.283	3
4	Penn State	11-1	+28.6%	264.4	4	0.272	4
5	Virginia Tech	11-2	+27.0%	259.7	5	0.262	5
6	Miami	9-3	+22.2%	253.8	6	0.196	12
7	LSU	11-2	+21.7%	242.8	8	0.241	6
8	Alabama	10-2	+20.7%	247.2	7	0.199	11
9	Auburn	9-3	+19.6%	239.6	9	0.214	10
10	Notre Dame	9-3	+19.5%	237.2	11	0.224	8

S&P+ and FEI mostly agreed on the top teams this season. And really, as long as Texas is No. 1 and USC is No. 2, then everything else is secondary.

2006
New F/+ Rankings
New S&P+ Rankings

2006 F/+ Top Ten

F/+ Rk	Team	Record	F/+	S&P+	S&P+ Rk	FEI	FEI Rk
1	Florida	13-1	+29.1%	257.2	4	0.320	1
2	LSU	11-2	+27.9%	262.1	2	0.273	3
3	Ohio State	12-1	+26.7%	263.6	1	0.241	5
4	USC	11-2	+26.1%	247.0	10	0.311	2
5	Louisville	12-1	+25.7%	257.6	3	0.250	4
6	West Virginia	11-2	+23.0%	251.6	7	0.226	6
7	BYU	11-2	+22.3%	251.7	6	0.213	9
8	Michigan	11-2	+22.2%	252.9	5	0.203	12
9	Boise State	13-0	+20.2%	245.1	13	0.201	13
10	Virginia Tech	10-3	+19.9%	249.5	8	0.175	19

Ohio State built enough of a lead in S&P+ that they maintained their No. 1 ranking despite the dreadful BCS Championship result, but FEI pushed Florida over the top overall.

2007
New F/+ Rankings
New S&P+ Rankings

2007 F/+ Top Ten

F/+ Rk	Team	Record	F/+	Off F/+	Rk	Def F/+	Rk	Spec. Tms F/+	Rk
1	USC	11-2	+26.2%	1.6%	41	23.9%	1	0.7%	14
2	West Virginia	11-2	+25.3%	6.7%	21	17.8%	9	0.8%	10
3	LSU	12-2	+24.1%	6.2%	24	17.6%	10	0.3%	36
4	Florida	9-4	+23.9%	25.7%	1	-2.9%	72	1.1%	2
5	Ohio State	11-2	+23.2%	2.0%	39	20.8%	5	0.4%	29
6	Oregon	9-4	+21.6%	7.3%	17	14.1%	13	0.2%	47
7	Oklahoma	11-3	+19.3%	5.7%	27	12.8%	16	0.9%	8
8	Virginia Tech	11-3	+19.1%	-3.9%	73	21.7%	4	1.3%	1
9	South Florida	9-4	+16.9%	-2.7%	65	19.4%	6	0.2%	49
10	Georgia	11-2	+16.7%	2.8%	34	13.2%	14	0.7%	13

This season has the weakest overall correlations, which makes sense, really, as this season was one of the craziest on record. Basically every team with a good offense had a porous defense, and every team with a good defense couldn't move the ball very well. That's how you get a two-loss national champion (and a four-loss team in the top four).

2008
New F/+ Rankings
New S&P+ Rankings

2008 F/+ Top Ten

F/+ Rk	Team	Record	F/+	Off F/+	Rk	Def F/+	Rk	Spec. Tms F/+	Rk
1	Florida	13-1	+35.4%	11.5%	9	23.4%	3	0.5%	2
2	USC	12-1	+33.8%	7.8%	16	25.8%	2	0.2%	25
3	Oklahoma	12-2	+29.4%	18.3%	1	11.3%	21	-0.2%	89
4	Texas	12-1	+26.3%	11.1%	11	15.0%	12	0.2%	23
5	Penn State	11-2	+25.8%	15.8%	3	9.7%	25	0.3%	15
6	TCU	11-2	+23.7%	-5.0%	75	28.5%	1	0.1%	29
7	Alabama	12-2	+20.0%	5.0%	29	15.0%	13	0.0%	56
8	Boise State	12-1	+20.0%	1.4%	50	18.3%	6	0.3%	8
9	Ohio State	10-3	+17.7%	3.7%	37	13.5%	16	0.5%	3
10	Utah	13-0	+16.5%	1.0%	53	15.3%	11	0.3%	9

Florida was easily the most well-rounded, balanced team in the country, but USC almost overtook them on the power of a ridiculous defense, and TCU squeezed into the top six because of the same.

2009
New F/+ Rankings
New S&P+ Rankings

2009 F/+ Top Ten

F/+ Rk	Team	Record	F/+	Off F/+	Rk	Def F/+	Rk	Spec. Tms F/+	Rk
1	Alabama	14-0	+33.9%	3.1%	36	30.6%	1	0.3%	49
2	Florida	13-1	+32.8%	9.8%	10	21.4%	9	1.6%	11
3	TCU	12-1	+30.7%	5.1%	26	23.1%	7	2.4%	3
4	Virginia Tech	10-3	+25.5%	5.7%	21	18.1%	13	1.7%	8
5	Texas	13-1	+24.9%	2.8%	40	20.1%	12	2.1%	5
6	Ohio State	11-2	+23.2%	-4.3%	75	26.7%	5	0.8%	29
7	Boise State	14-0	+21.4%	-0.3%	58	20.2%	10	1.6%	10
8	Penn State	11-2	+21.4%	-0.8%	61	23.2%	6	-1.0%	97
9	Oregon	10-3	+19.8%	10.0%	9	9.5%	25	0.2%	52
10	Iowa	11-2	+19.6%	-10.2%	98	28.6%	2	1.2%	17

The interesting team here is Texas, which actually graded out worse than they had in 2008, but they capitalized on a weaker Big 12 (and some luck in the Big 12 title game) to make the BCS Championship. All of their luck was apparently used up in the process, as they watched their quarterback get hurt on their opening drive of the title game.

2010
New F/+ Rankings
New S&P+ Rankings

2010 F/+ Top Ten

F/+ Rk	Team	Record	F/+	Off F/+	Rk	Def F/+	Rk	Spec. Tms F/+	Rk
1	Boise State	12-1	+30.2%	10.6%	10	19.5%	5	0.2%	56
2	Auburn	14-0	+29.2%	18.1%	1	11.0%	23	0.1%	60
3	Alabama	10-3	+29.0%	14.3%	5	13.5%	15	1.3%	5
4	Stanford	12-1	+27.6%	10.7%	9	16.4%	11	0.6%	37
5	TCU	13-0	+27.2%	6.3%	24	19.8%	4	1.1%	13
6	Ohio State	12-1	+25.4%	0.9%	47	24.0%	2	0.6%	36
7	Oregon	12-1	+21.8%	3.5%	34	17.7%	7	0.6%	34
8	Oklahoma	12-2	+21.6%	3.1%	38	17.9%	6	0.7%	27
9	Virginia Tech	11-3	+21.4%	10.2%	11	9.6%	29	1.6%	1
10	Arkansas	10-3	+21.2%	11.3%	8	9.6%	30	0.3%	47

This almost certainly produces the most odd result at the No. 1 spot. Auburn, which skated through close game after close game but still finished undefeated in the SEC, falls one percent behind Boise State thanks to the S&P+ adjustments. Honestly, though? I can see it. Boise State's one loss (on the road to No. 24 Nevada) was almost as fluky as a couple of Auburn's wins. Auburn earned the crown, but I'd have loved to see the Broncos get a shot (and they wouldn't have, even if they had beaten Nevada).

Some general conclusions to this process:

The new S&P+ adjustment helps mid-major teams, which is somewhat counter-intuitive to a lot of us. But if it results in both stronger correlations to wins and better predictive potential, then that's the way it's going to be. And let's be honest: the two most consistently great mid-major teams in recent years -- Boise State and TCU -- did little to disprove the notion that they were both very good. From 2008 to 2010, Boise State went 4-0 versus major conference teams, and TCU went 6-1 (the lone loss: to 2008 Oklahoma). We can talk about a major conference "grind" if we want, and there's no way to prove or disprove the grind's existence, but facts are facts, and the few results we have favor Boise State and TCU tremendously.

One of the major projects moving forward will be the predictive side of a measure like this. It is going to be very difficult for the same measure to both evaluate and predict at a high level, and it is quite possible that there will be two different F/+ or S&P+ types of measures -- one for reflecting on the year to date, and one for making predictions. And honestly, F/+'s recent success (knock on wood) in bowl projections might be justification for this. Once you remove week-to-week momentum from the equation, the predictive ability of F/+ seems to improve. Last year, it was awful in the bowls; this year, it has started out 5-1.

Just for fun, here are the top ten teams of the last seven years (including 2011 teams and their pre-bowl ratings):

1. 2011 LSU (+38.1%)
2. 2005 Texas (+35.9%)
3. 2008 Florida (+35.4%)
4. 2009 Alabama (+33.9%)
5. 2008 USC (+33.8%)
6. 2005 USC (+33.3%)
7. 2009 Florida (+32.8%)
8. 2009 TCU (+30.7%)
9. 2011 Alabama (+30.5%)
10. 2010 Boise State (+30.2%)