VN: Your Best Against the Best

November 15, 2012, 3:04 pm ET

Photo: USA Today Sports Images

by Bill Connelly

Each year I become a little bit more enamored with what I call the "personality stats." We tend to put stats in one of two categories -- evaluative or predictive -- but there is another category. Personality stats are the ones that don't really tell us anything good or bad about a team; instead, they give us the "how" and "why" of the good and bad. Pace is a personality stat. An offense's run-pass ratio. The bend-but-don't-break-ability of a defense. The degree to which defenses go for strips and attack the ball. Even though S&P+ breaks out into any number of categories (run, pass, standard downs, passing downs, red zone, per-quarter, per-down, etc.), there are a lot of ways to rank 10th in S&P+, or 20th in Rushing S&P+, or anything else. The "how" is the interesting part.

I thought we would revisit one of these personality stats today, the measure simply known as Covariance. In math terms, covariance basically measures the strength of the correlation between variables. In football terms, covariance can tell you which teams play their best games against their best opponents, and which play their best against their worse opponents. At least, that's how we will use it here. I discussed it in Varsity Numbers just over a year ago, and I utilized it in each of this summer's statistical profiles at SB Nation.

We tend to look more favorably on the teams that play the best against the best, but that doesn't necessarily signify a better team. Plenty of teams go in the opposite direction and win a lot of games. The trick, of course, is balance. If you play your best against the best teams, then you run the risk of both losing a lot of close games to good teams and suffering letdown losses against lesser teams.

This year, Washington State is a lovely example of this. The Cougars fared as well as just about anybody against Oregon (third in the F/+ rankings), lost by 13 to Oregon State (11th), stayed within seven points of Stanford (12th) and almost completed a ferocious comeback in an eight-point loss to UCLA (30th). They were perfectly respectable against those four teams but went 0-4. Meanwhile, they collapsed and lost to lowly Colorado (123rd), got their doors blown off by Utah (56th), and tried really hard to lose to both Eastern Washington (FCS) and UNLV (86th). As you'll see below, Wazzu is on the extreme "best against best" pole of the Covariance scale, with a 2-8 record to show for it. Play well against the worse opponents and take blowout beatings against the best, and they're probably 4-6 right now.

At the same time, you could end up like Louisville, playing just well enough to beat most of the better teams on their (rather weak) schedule (34-31 over Cincinnati, 45-35 over Pittsburgh, 39-34 over North Carolina) and barely beating some of the lesser opponents (28-21 over FIU, 21-17 over Southern Miss in a downpour). Until last week's blowout loss to Syracuse, the Cardinals were actually undefeated through nine games despite what wasn't even a top 40-level performance.

Meanwhile, playing your best against the worse opponents could mean you stay afloat against a tough schedule (Michigan is 7-3 versus the what is the No. 3 toughest schedule according to Brian Fremeau), and it could mean you crash and burn (Illinois is 2-8 versus the No. 36 schedule).

Below you will find a table with the following information in it:

Covariance. A team's covariance rating and ranking. IMPORTANT NOTE: Positive numbers mean you play better against worse opponents; negative numbers mean you play better against the best. The ranking may confuse, but a higher or lower ranking doesn't mean a team is better or worse in this regard.
Record, F/+ rating and F/+ ranking.
FEI SOS Ranking. This is the "SOS Pvs" number from Fremeau's FEI page. As defined there, it is the "strength of schedule to date, based on the likelihood of an elite team going undefeated against the given team's schedule to date."
Standard Deviation. As described last year, it is "simply the standard deviation of each team's week-to-week Adj. Scoring Margin. The higher the standard deviation (and rank), the more a team's performance varies from week to week. This allows us to differentiate between one good/bad team and another."

Team	Covariance	Rk	Record	F/+	Rk	FEI SOS Rk	Std Dev	Rk
West Virginia	414.9	1	5-4	-1.4%	67	27	15.5	12
Bowling Green	379.4	2	7-3	-12.4%	82	67	20.0	3
Illinois	373.4	3	2-8	-24.5%	110	36	13.8	27
Vanderbilt	360.5	4	6-4	-1.2%	66	41	17.2	6
Michigan	323.4	5	7-3	+23.4%	22	3	12.1	44
Purdue	307.7	6	4-6	-4.5%	71	13	14.1	21
Arizona State	284.4	7	5-5	+4.6%	52	23	12.0	46
Arkansas	268.2	8	4-6	+1.0%	60	21	13.7	28
Connecticut	257.3	9	4-6	-3.2%	69	90	13.2	32
Colorado	248.0	10	1-9	-45.1%	123	12	10.2	77
Texas Tech	244.7	11	7-3	+11.2%	35	10	10.6	66
Florida State	226.9	12	9-1	+34.6%	8	98	16.1	11
Utah	226.6	13	4-6	+3.5%	56	24	11.3	60
Stanford	208.9	14	8-2	+28.0%	12	29	21.2	1
Alabama	201.0	15	9-1	+54.4%	1	45	20.1	2
SMU	197.2	16	5-5	-16.2%	94	86	13.9	26
Iowa State	197.1	17	5-5	+4.5%	53	6	9.8	86
UAB	196.1	18	3-7	-13.0%	84	64	12.6	36
BYU	192.3	19	6-4	+20.6%	25	16	14.8	16
North Carolina	173.7	20	6-4	+9.5%	39	110	16.4	10
Team	Covariance	Rk	Record	F/+	Rk	FEI SOS Rk	Std Dev	Rk
Ole Miss	159.1	21	5-5	+1.1%	59	17	12.6	37
Minnesota	156.2	22	6-4	-4.5%	70	71	8.7	99
Georgia Tech	155.5	23	5-5	+5.9%	46	77	16.4	9
Navy	154.3	24	6-4	-13.2%	85	58	10.6	65
Auburn	152.3	25	2-8	-15.7%	93	49	12.4	40
Arkansas State	146.8	26	7-3	-9.1%	79	38	12.0	47
Georgia	135.3	27	9-1	+26.3%	17	57	16.7	8
UL-Monroe	128.4	28	6-4	-7.1%	75	121	14.2	20
Tennessee	123.1	29	4-6	+10.7%	37	14	6.0	121
South Carolina	122.1	30	8-2	+23.9%	18	37	14.5	17
Central Florida	118.3	31	8-2	+12.0%	34	92	13.2	33
Hawaii	114.5	32	1-8	-36.8%	120	52	7.9	109
Michigan State	110.6	33	5-5	+26.5%	16	11	11.5	56
San Diego State	105.3	34	8-3	+11.1%	36	83	8.7	98
Arizona	102.7	35	6-4	+15.8%	28	5	12.3	42
Houston	100.3	36	4-6	-17.0%	97	104	10.6	67
Boston College	98.2	37	2-8	-8.8%	78	26	7.3	112
Utah State	96.0	38	8-2	+23.8%	20	68	11.4	58
Texas	92.2	39	8-2	+23.3%	23	51	12.7	35
TCU	91.7	40	6-4	+5.2%	50	48	11.2	61
Team	Covariance	Rk	Record	F/+	Rk	FEI SOS Rk	Std Dev	Rk
Western Kentucky	90.3	41	6-4	-6.1%	72	63	13.2	34
Florida Atlantic	87.2	42	3-7	-22.6%	106	35	9.9	83
Wake Forest	85.7	43	5-5	-15.5%	92	69	13.4	31
Oklahoma State	84.0	44	6-3	+27.4%	13	31	11.8	50
Buffalo	80.6	45	3-7	-22.2%	104	75	6.1	120
Northwestern	78.9	46	7-3	+21.5%	24	55	6.3	119
Tulane	78.8	47	2-8	-28.7%	117	117	13.9	25
Idaho	78.6	48	1-9	-29.1%	118	53	11.4	59
Massachusetts	78.4	49	1-9	-42.2%	122	87	10.5	70
Fresno State	74.1	50	8-3	+12.0%	33	40	8.8	97
LSU	68.9	51	8-2	+32.3%	9	8	15.3	14
UNLV	68.4	52	2-9	-13.4%	86	65	11.6	54
Akron	66.6	53	1-10	-27.1%	114	107	10.1	78
Northern Illinois	62.0	54	9-1	+12.4%	32	118	11.2	63
Miami	57.0	55	5-5	+0.3%	62	2	8.8	96
Wisconsin	55.9	56	7-3	+23.5%	21	39	14.0	23
Marshall	53.2	57	4-6	-22.3%	105	124	11.4	57
Tulsa	49.8	58	8-2	+0.2%	63	115	10.5	71
California	47.9	59	3-8	-1.0%	64	15	13.7	29
San Jose State	47.2	60	8-2	+8.9%	41	76	9.2	92
Team	Covariance	Rk	Record	F/+	Rk	FEI SOS Rk	Std Dev	Rk
Nebraska	47.1	61	8-2	+29.7%	10	18	12.4	39
USC	44.1	62	7-3	+27.1%	14	34	10.2	74
Missouri	43.5	63	5-5	+5.6%	47	9	10.2	75
Washington	25.7	64	6-4	+8.1%	42	4	14.3	19
Baylor	23.2	65	4-5	+0.9%	61	42	9.0	93
Duke	23.1	66	6-4	-11.6%	80	46	11.6	53
Penn State	20.9	67	6-4	+15.4%	29	56	8.9	95
South Alabama	20.4	68	2-8	-27.9%	116	112	4.3	124
Virginia Tech	11.2	69	4-6	+10.1%	38	44	17.4	5
Rice	10.4	70	4-6	-19.2%	100	105	9.6	88
Syracuse	0.6	71	5-5	+13.6%	31	54	8.4	102
Pittsburgh	0.3	72	4-6	+4.8%	51	33	8.2	106
East Carolina	-0.9	73	6-4	-23.3%	107	84	9.3	90
Kansas	-4.5	74	1-9	-17.9%	98	1	7.3	113
Maryland	-14.1	75	4-6	-11.9%	81	101	6.7	115
Iowa	-16.6	76	4-6	+3.1%	57	60	11.9	49
Texas State	-20.0	77	3-6	-14.7%	89	96	11.1	64
New Mexico	-22.7	78	4-7	-29.9%	119	79	11.7	52
Boise State	-24.5	79	8-2	+18.7%	26	89	12.2	43
Oklahoma	-26.9	80	7-2	+40.5%	5	20	7.9	108
Team	Covariance	Rk	Record	F/+	Rk	FEI SOS Rk	Std Dev	Rk
Oregon State	-29.5	81	7-2	+29.6%	11	32	11.2	62
South Florida	-32.7	82	3-6	+4.3%	54	78	8.9	94
Troy	-32.8	83	5-5	-7.4%	77	120	8.4	103
Kentucky	-32.9	84	1-9	-21.2%	103	43	16.8	7
Toledo	-40.6	85	8-2	+7.0%	44	108	11.7	51
Air Force	-41.8	86	5-5	-13.7%	87	100	9.9	84
Temple	-43.0	87	3-6	-12.9%	83	74	6.6	117
Miami (Ohio)	-46.4	88	4-6	-23.8%	108	50	8.5	100
Mississippi State	-47.3	89	7-3	+2.1%	58	19	12.0	45
Florida International	-49.1	90	2-8	-14.9%	90	119	7.5	110
New Mexico State	-54.5	91	1-9	-45.7%	124	106	9.3	91
UTEP	-56.7	92	2-8	-16.3%	95	61	8.3	104
Central Michigan	-60.5	93	4-6	-19.5%	101	81	7.5	111
North Texas	-61.8	94	4-6	-27.1%	113	22	10.0	79
Wyoming	-62.8	95	3-7	-20.5%	102	80	5.5	122
Louisiana Tech	-70.0	96	9-1	+5.6%	49	82	6.6	116
Clemson	-72.4	97	9-1	+23.8%	19	85	9.9	82
Rutgers	-73.3	98	8-1	+8.1%	43	116	8.5	101
Eastern Michigan	-74.1	99	1-9	-23.8%	109	95	10.2	73
Nevada	-78.4	100	6-4	-7.1%	76	122	6.9	114
Team	Covariance	Rk	Record	F/+	Rk	FEI SOS Rk	Std Dev	Rk
Memphis	-78.8	101	2-8	-26.6%	112	123	10.6	68
Western Michigan	-80.6	102	4-7	-14.1%	88	102	6.4	118
Notre Dame	-92.0	103	10-0	+40.0%	6	30	8.1	107
Texas A&M	-93.3	104	8-2	+34.7%	7	25	11.5	55
Army	-98.9	105	2-8	-16.3%	96	93	10.2	76
Kent State	-99.0	106	9-1	+9.0%	40	113	10.3	72
Ball State	-99.1	107	7-3	+6.6%	45	72	4.8	123
Middle Tennessee	-99.4	108	7-2	-6.2%	73	111	13.9	24
Florida	-103.2	109	9-1	+43.7%	4	28	13.4	30
UL-Lafayette	-108.9	110	5-4	-2.1%	68	59	8.3	105
Virginia	-118.0	111	4-6	-15.0%	91	91	14.3	18
N.C. State	-146.1	112	6-4	+3.6%	55	99	14.9	15
Colorado State	-156.6	113	3-7	-25.8%	111	97	9.4	89
Ohio State	-158.4	114	10-0	+26.9%	15	62	12.0	48
Ohio	-173.1	115	8-2	-7.1%	74	109	10.5	69
UTSA	-180.1	116	4-4	-37.5%	121	114	9.9	81
Kansas State	-181.2	117	10-0	+45.2%	2	47	14.0	22
Indiana	-188.2	118	4-6	-1.1%	65	66	9.7	87
Southern Miss	-233.9	119	0-10	-27.7%	115	70	12.4	38
Cincinnati	-235.6	120	7-2	+15.9%	27	103	10.0	80
Team	Covariance	Rk	Record	F/+	Rk	FEI SOS Rk	Std Dev	Rk
Oregon	-272.2	121	10-0	+44.8%	3	88	9.8	85
UCLA	-279.0	122	8-2	+15.4%	30	73	18.4	4
Washington State	-283.9	123	2-8	-18.2%	99	7	15.4	13
Louisville	-341.5	124	9-1	+5.6%	48	94	12.4	41

A few thoughts:

[ad placeholder 3]

I tinkered for quite a while with correlations between Covariance and any number of other measures, from F/+ ranking to returning starters, and everything in between. There is basically no strong correlation between Covariance and anything else, which makes it a true "personality stat" devoid of quality evaluation. It also makes it a little bit maddening, but that's another story altogether.
It might be worth noting, though, that of the top eight teams according to F/+, five are 103rd or lower (best against best) in Covariance, while two are 15th or higher (best against worse). The two in the "best against worse" pile (Florida State and Alabama) have each suffered losses to double-digit underdogs. The five in the "best against best" stack (Kansas State, Oregon, Florida, Notre Dame and Texas A&M) are a combined 47-3 with losses just to Georgia (Florida), LSU (Texas A&M) and Florida (Texas A&M). Kansas State, Oregon and Notre Dame probably prove that the absolute best place to be is both a) elite and b) on the "best against best" side, which would probably mean both that you play at a high enough level to avoid upsets, and that you have proven capable of raising your game an extra level if necessary. Ohio State, also 10-0 (but banned from the postseason) is 114th in Covariance as well.
So far this year, UCLA has crushed Arizona (28th in F/+) by 56 points, lost to California (64th) by 26, and crept by both Utah (56th) and Nebraska (10th). Without looking, I would have put a lot of money on the Bruins being in the top 5 in standard deviation. They are fourth.
This data shines at least a little bit of light on BYU's strangely high S&P+ ranking (12th). The Cougars have basically played like a top-5 or top-10 team in dominating lesser opponents like Idaho, Hawaii, and Washington State and have looked just good enough in some losses (7-6 to Boise State, 17-14 to Notre Dame) to grade out pretty well (too well, really). (And no, these measures don't help to explain Utah State. What can I say? Play-by-play measures clearly have a pro-Utah bias.)
One interesting tidbit: the teams in the top 15 of Covariance have an average win percentage of 0.550. The teams in the bottom 15 have an average win percentage of 0.616. The middle 30: 0.478. This might be purely coincidental, but it also might suggest that, whatever you are (best against best, best against worse), it is better to be an extreme version of that, at least this year.

This Week at SB Nation

Football Outisders+ Perks & Features image

Comments

6 comments, Last at 16 Nov 2012, 8:56pm

#1 by Adam H (not verified) // Nov 15, 2012 - 4:35pm

Would you expect covariance to be predictive of future performance at all?

#2 by Bill Connelly // Nov 16, 2012 - 6:23am

It would certainly seem like there might be potential, but I haven't figured out the balance yet (if there is one).

#3 by Quigley (not verified) // Nov 16, 2012 - 1:37pm

Maybe it matters only when good goes against good?

When predicting matchups vs. top teams, this may be important, eg, ND vs OU, OU vs UT. ND is elite in prep for top teams and though OU is high, not at that level. ND had the edge and won. UT romps over the weaklings, but can't achieve vs top teams where their talent differential doesn't overwhelm.