Writers of Pro Football Prospectus 2008

printable version

Varsity Numbers

For students majoring in breakdown of college football play-by-play

Need Help with KUBIAK or Premium?

blog advertising is good for you


15 Nov 2012

VN: Your Best Against the Best

by Bill Connelly

Each year I become a little bit more enamored with what I call the "personality stats." We tend to put stats in one of two categories -- evaluative or predictive -- but there is another category. Personality stats are the ones that don't really tell us anything good or bad about a team; instead, they give us the "how" and "why" of the good and bad. Pace is a personality stat. An offense's run-pass ratio. The bend-but-don't-break-ability of a defense. The degree to which defenses go for strips and attack the ball. Even though S&P+ breaks out into any number of categories (run, pass, standard downs, passing downs, red zone, per-quarter, per-down, etc.), there are a lot of ways to rank 10th in S&P+, or 20th in Rushing S&P+, or anything else. The "how" is the interesting part.

I thought we would revisit one of these personality stats today, the measure simply known as Covariance. In math terms, covariance basically measures the strength of the correlation between variables. In football terms, covariance can tell you which teams play their best games against their best opponents, and which play their best against their worse opponents. At least, that's how we will use it here. I discussed it in Varsity Numbers just over a year ago, and I utilized it in each of this summer's statistical profiles at SB Nation.

We tend to look more favorably on the teams that play the best against the best, but that doesn't necessarily signify a better team. Plenty of teams go in the opposite direction and win a lot of games. The trick, of course, is balance. If you play your best against the best teams, then you run the risk of both losing a lot of close games to good teams and suffering letdown losses against lesser teams.

This year, Washington State is a lovely example of this. The Cougars fared as well as just about anybody against Oregon (third in the F/+ rankings), lost by 13 to Oregon State (11th), stayed within seven points of Stanford (12th) and almost completed a ferocious comeback in an eight-point loss to UCLA (30th). They were perfectly respectable against those four teams but went 0-4. Meanwhile, they collapsed and lost to lowly Colorado (123rd), got their doors blown off by Utah (56th), and tried really hard to lose to both Eastern Washington (FCS) and UNLV (86th). As you'll see below, Wazzu is on the extreme "best against best" pole of the Covariance scale, with a 2-8 record to show for it. Play well against the worse opponents and take blowout beatings against the best, and they're probably 4-6 right now.

At the same time, you could end up like Louisville, playing just well enough to beat most of the better teams on their (rather weak) schedule (34-31 over Cincinnati, 45-35 over Pittsburgh, 39-34 over North Carolina) and barely beating some of the lesser opponents (28-21 over FIU, 21-17 over Southern Miss in a downpour). Until last week's blowout loss to Syracuse, the Cardinals were actually undefeated through nine games despite what wasn't even a top 40-level performance.

Meanwhile, playing your best against the worse opponents could mean you stay afloat against a tough schedule (Michigan is 7-3 versus the what is the No. 3 toughest schedule according to Brian Fremeau), and it could mean you crash and burn (Illinois is 2-8 versus the No. 36 schedule).

Below you will find a table with the following information in it:

  • Covariance. A team's covariance rating and ranking. IMPORTANT NOTE: Positive numbers mean you play better against worse opponents; negative numbers mean you play better against the best. The ranking may confuse, but a higher or lower ranking doesn't mean a team is better or worse in this regard.
  • Record, F/+ rating and F/+ ranking.
  • FEI SOS Ranking. This is the "SOS Pvs" number from Fremeau's FEI page. As defined there, it is the "strength of schedule to date, based on the likelihood of an elite team going undefeated against the given team's schedule to date."
  • Standard Deviation. As described last year, it is "simply the standard deviation of each team's week-to-week Adj. Scoring Margin. The higher the standard deviation (and rank), the more a team's performance varies from week to week. This allows us to differentiate between one good/bad team and another."

Team Covariance Rk Record F/+ Rk FEI SOS Rk Std Dev Rk
West Virginia 414.9 1 5-4 -1.4% 67 27 15.5 12
Bowling Green 379.4 2 7-3 -12.4% 82 67 20.0 3
Illinois 373.4 3 2-8 -24.5% 110 36 13.8 27
Vanderbilt 360.5 4 6-4 -1.2% 66 41 17.2 6
Michigan 323.4 5 7-3 +23.4% 22 3 12.1 44
Purdue 307.7 6 4-6 -4.5% 71 13 14.1 21
Arizona State 284.4 7 5-5 +4.6% 52 23 12.0 46
Arkansas 268.2 8 4-6 +1.0% 60 21 13.7 28
Connecticut 257.3 9 4-6 -3.2% 69 90 13.2 32
Colorado 248.0 10 1-9 -45.1% 123 12 10.2 77
Texas Tech 244.7 11 7-3 +11.2% 35 10 10.6 66
Florida State 226.9 12 9-1 +34.6% 8 98 16.1 11
Utah 226.6 13 4-6 +3.5% 56 24 11.3 60
Stanford 208.9 14 8-2 +28.0% 12 29 21.2 1
Alabama 201.0 15 9-1 +54.4% 1 45 20.1 2
SMU 197.2 16 5-5 -16.2% 94 86 13.9 26
Iowa State 197.1 17 5-5 +4.5% 53 6 9.8 86
UAB 196.1 18 3-7 -13.0% 84 64 12.6 36
BYU 192.3 19 6-4 +20.6% 25 16 14.8 16
North Carolina 173.7 20 6-4 +9.5% 39 110 16.4 10
Team Covariance Rk Record F/+ Rk FEI SOS Rk Std Dev Rk
Ole Miss 159.1 21 5-5 +1.1% 59 17 12.6 37
Minnesota 156.2 22 6-4 -4.5% 70 71 8.7 99
Georgia Tech 155.5 23 5-5 +5.9% 46 77 16.4 9
Navy 154.3 24 6-4 -13.2% 85 58 10.6 65
Auburn 152.3 25 2-8 -15.7% 93 49 12.4 40
Arkansas State 146.8 26 7-3 -9.1% 79 38 12.0 47
Georgia 135.3 27 9-1 +26.3% 17 57 16.7 8
UL-Monroe 128.4 28 6-4 -7.1% 75 121 14.2 20
Tennessee 123.1 29 4-6 +10.7% 37 14 6.0 121
South Carolina 122.1 30 8-2 +23.9% 18 37 14.5 17
Central Florida 118.3 31 8-2 +12.0% 34 92 13.2 33
Hawaii 114.5 32 1-8 -36.8% 120 52 7.9 109
Michigan State 110.6 33 5-5 +26.5% 16 11 11.5 56
San Diego State 105.3 34 8-3 +11.1% 36 83 8.7 98
Arizona 102.7 35 6-4 +15.8% 28 5 12.3 42
Houston 100.3 36 4-6 -17.0% 97 104 10.6 67
Boston College 98.2 37 2-8 -8.8% 78 26 7.3 112
Utah State 96.0 38 8-2 +23.8% 20 68 11.4 58
Texas 92.2 39 8-2 +23.3% 23 51 12.7 35
TCU 91.7 40 6-4 +5.2% 50 48 11.2 61
Team Covariance Rk Record F/+ Rk FEI SOS Rk Std Dev Rk
Western Kentucky 90.3 41 6-4 -6.1% 72 63 13.2 34
Florida Atlantic 87.2 42 3-7 -22.6% 106 35 9.9 83
Wake Forest 85.7 43 5-5 -15.5% 92 69 13.4 31
Oklahoma State 84.0 44 6-3 +27.4% 13 31 11.8 50
Buffalo 80.6 45 3-7 -22.2% 104 75 6.1 120
Northwestern 78.9 46 7-3 +21.5% 24 55 6.3 119
Tulane 78.8 47 2-8 -28.7% 117 117 13.9 25
Idaho 78.6 48 1-9 -29.1% 118 53 11.4 59
Massachusetts 78.4 49 1-9 -42.2% 122 87 10.5 70
Fresno State 74.1 50 8-3 +12.0% 33 40 8.8 97
LSU 68.9 51 8-2 +32.3% 9 8 15.3 14
UNLV 68.4 52 2-9 -13.4% 86 65 11.6 54
Akron 66.6 53 1-10 -27.1% 114 107 10.1 78
Northern Illinois 62.0 54 9-1 +12.4% 32 118 11.2 63
Miami 57.0 55 5-5 +0.3% 62 2 8.8 96
Wisconsin 55.9 56 7-3 +23.5% 21 39 14.0 23
Marshall 53.2 57 4-6 -22.3% 105 124 11.4 57
Tulsa 49.8 58 8-2 +0.2% 63 115 10.5 71
California 47.9 59 3-8 -1.0% 64 15 13.7 29
San Jose State 47.2 60 8-2 +8.9% 41 76 9.2 92
Team Covariance Rk Record F/+ Rk FEI SOS Rk Std Dev Rk
Nebraska 47.1 61 8-2 +29.7% 10 18 12.4 39
USC 44.1 62 7-3 +27.1% 14 34 10.2 74
Missouri 43.5 63 5-5 +5.6% 47 9 10.2 75
Washington 25.7 64 6-4 +8.1% 42 4 14.3 19
Baylor 23.2 65 4-5 +0.9% 61 42 9.0 93
Duke 23.1 66 6-4 -11.6% 80 46 11.6 53
Penn State 20.9 67 6-4 +15.4% 29 56 8.9 95
South Alabama 20.4 68 2-8 -27.9% 116 112 4.3 124
Virginia Tech 11.2 69 4-6 +10.1% 38 44 17.4 5
Rice 10.4 70 4-6 -19.2% 100 105 9.6 88
Syracuse 0.6 71 5-5 +13.6% 31 54 8.4 102
Pittsburgh 0.3 72 4-6 +4.8% 51 33 8.2 106
East Carolina -0.9 73 6-4 -23.3% 107 84 9.3 90
Kansas -4.5 74 1-9 -17.9% 98 1 7.3 113
Maryland -14.1 75 4-6 -11.9% 81 101 6.7 115
Iowa -16.6 76 4-6 +3.1% 57 60 11.9 49
Texas State -20.0 77 3-6 -14.7% 89 96 11.1 64
New Mexico -22.7 78 4-7 -29.9% 119 79 11.7 52
Boise State -24.5 79 8-2 +18.7% 26 89 12.2 43
Oklahoma -26.9 80 7-2 +40.5% 5 20 7.9 108
Team Covariance Rk Record F/+ Rk FEI SOS Rk Std Dev Rk
Oregon State -29.5 81 7-2 +29.6% 11 32 11.2 62
South Florida -32.7 82 3-6 +4.3% 54 78 8.9 94
Troy -32.8 83 5-5 -7.4% 77 120 8.4 103
Kentucky -32.9 84 1-9 -21.2% 103 43 16.8 7
Toledo -40.6 85 8-2 +7.0% 44 108 11.7 51
Air Force -41.8 86 5-5 -13.7% 87 100 9.9 84
Temple -43.0 87 3-6 -12.9% 83 74 6.6 117
Miami (Ohio) -46.4 88 4-6 -23.8% 108 50 8.5 100
Mississippi State -47.3 89 7-3 +2.1% 58 19 12.0 45
Florida International -49.1 90 2-8 -14.9% 90 119 7.5 110
New Mexico State -54.5 91 1-9 -45.7% 124 106 9.3 91
UTEP -56.7 92 2-8 -16.3% 95 61 8.3 104
Central Michigan -60.5 93 4-6 -19.5% 101 81 7.5 111
North Texas -61.8 94 4-6 -27.1% 113 22 10.0 79
Wyoming -62.8 95 3-7 -20.5% 102 80 5.5 122
Louisiana Tech -70.0 96 9-1 +5.6% 49 82 6.6 116
Clemson -72.4 97 9-1 +23.8% 19 85 9.9 82
Rutgers -73.3 98 8-1 +8.1% 43 116 8.5 101
Eastern Michigan -74.1 99 1-9 -23.8% 109 95 10.2 73
Nevada -78.4 100 6-4 -7.1% 76 122 6.9 114
Team Covariance Rk Record F/+ Rk FEI SOS Rk Std Dev Rk
Memphis -78.8 101 2-8 -26.6% 112 123 10.6 68
Western Michigan -80.6 102 4-7 -14.1% 88 102 6.4 118
Notre Dame -92.0 103 10-0 +40.0% 6 30 8.1 107
Texas A&M -93.3 104 8-2 +34.7% 7 25 11.5 55
Army -98.9 105 2-8 -16.3% 96 93 10.2 76
Kent State -99.0 106 9-1 +9.0% 40 113 10.3 72
Ball State -99.1 107 7-3 +6.6% 45 72 4.8 123
Middle Tennessee -99.4 108 7-2 -6.2% 73 111 13.9 24
Florida -103.2 109 9-1 +43.7% 4 28 13.4 30
UL-Lafayette -108.9 110 5-4 -2.1% 68 59 8.3 105
Virginia -118.0 111 4-6 -15.0% 91 91 14.3 18
N.C. State -146.1 112 6-4 +3.6% 55 99 14.9 15
Colorado State -156.6 113 3-7 -25.8% 111 97 9.4 89
Ohio State -158.4 114 10-0 +26.9% 15 62 12.0 48
Ohio -173.1 115 8-2 -7.1% 74 109 10.5 69
UTSA -180.1 116 4-4 -37.5% 121 114 9.9 81
Kansas State -181.2 117 10-0 +45.2% 2 47 14.0 22
Indiana -188.2 118 4-6 -1.1% 65 66 9.7 87
Southern Miss -233.9 119 0-10 -27.7% 115 70 12.4 38
Cincinnati -235.6 120 7-2 +15.9% 27 103 10.0 80
Team Covariance Rk Record F/+ Rk FEI SOS Rk Std Dev Rk
Oregon -272.2 121 10-0 +44.8% 3 88 9.8 85
UCLA -279.0 122 8-2 +15.4% 30 73 18.4 4
Washington State -283.9 123 2-8 -18.2% 99 7 15.4 13
Louisville -341.5 124 9-1 +5.6% 48 94 12.4 41


A few thoughts:

  • I tinkered for quite a while with correlations between Covariance and any number of other measures, from F/+ ranking to returning starters, and everything in between. There is basically no strong correlation between Covariance and anything else, which makes it a true "personality stat" devoid of quality evaluation. It also makes it a little bit maddening, but that's another story altogether.
  • It might be worth noting, though, that of the top eight teams according to F/+, five are 103rd or lower (best against best) in Covariance, while two are 15th or higher (best against worse). The two in the "best against worse" pile (Florida State and Alabama) have each suffered losses to double-digit underdogs. The five in the "best against best" stack (Kansas State, Oregon, Florida, Notre Dame and Texas A&M) are a combined 47-3 with losses just to Georgia (Florida), LSU (Texas A&M) and Florida (Texas A&M). Kansas State, Oregon and Notre Dame probably prove that the absolute best place to be is both a) elite and b) on the "best against best" side, which would probably mean both that you play at a high enough level to avoid upsets, and that you have proven capable of raising your game an extra level if necessary. Ohio State, also 10-0 (but banned from the postseason) is 114th in Covariance as well.
  • So far this year, UCLA has crushed Arizona (28th in F/+) by 56 points, lost to California (64th) by 26, and crept by both Utah (56th) and Nebraska (10th). Without looking, I would have put a lot of money on the Bruins being in the top 5 in standard deviation. They are fourth.
  • This data shines at least a little bit of light on BYU's strangely high S&P+ ranking (12th). The Cougars have basically played like a top-5 or top-10 team in dominating lesser opponents like Idaho, Hawaii, and Washington State and have looked just good enough in some losses (7-6 to Boise State, 17-14 to Notre Dame) to grade out pretty well (too well, really). (And no, these measures don't help to explain Utah State. What can I say? Play-by-play measures clearly have a pro-Utah bias.)
  • One interesting tidbit: the teams in the top 15 of Covariance have an average win percentage of 0.550. The teams in the bottom 15 have an average win percentage of 0.616. The middle 30: 0.478. This might be purely coincidental, but it also might suggest that, whatever you are (best against best, best against worse), it is better to be an extreme version of that, at least this year.

This Week at SB Nation

Posted by: Bill Connelly on 15 Nov 2012

6 comments, Last at 16 Nov 2012, 9:56pm by Bill Connelly

Comments

1
by Adam H (not verified) :: Thu, 11/15/2012 - 5:35pm

Would you expect covariance to be predictive of future performance at all?

2
by Bill Connelly :: Fri, 11/16/2012 - 7:23am

It would certainly seem like there might be potential, but I haven't figured out the balance yet (if there is one).

3
by Quigley (not verified) :: Fri, 11/16/2012 - 2:37pm

Maybe it matters only when good goes against good?

When predicting matchups vs. top teams, this may be important, eg, ND vs OU, OU vs UT. ND is elite in prep for top teams and though OU is high, not at that level. ND had the edge and won. UT romps over the weaklings, but can't achieve vs top teams where their talent differential doesn't overwhelm.

4
by Quigley (not verified) :: Fri, 11/16/2012 - 2:37pm

Maybe it matters only when good goes against good?

When predicting matchups vs. top teams, this may be important, eg, ND vs OU, OU vs UT. ND is elite in prep for top teams and though OU is high, not at that level. ND had the edge and won. UT romps over the weaklings, but can't achieve vs top teams where their talent differential doesn't overwhelm.

6
by Epim (not verified) :: Fri, 11/16/2012 - 5:17pm

Apologies if I missed this in the article, but I'm not clear on what measure (F/+?) you're computing the covariance of?

7
by Bill Connelly :: Fri, 11/16/2012 - 9:56pm

Yes! F/+! Thought I had mentioned that, but I had not. Comparing a team's Adj. Scoring Margin with the F/+ ranking of each opponent.

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

More information about formatting options