VN: The Numbers Are on to You (or Not)

November 20, 3:05 pm ET

Photo: USA Today Sports Images

by Bill Connelly

If you look at one of the advanced statistical profiles I've designed at Football Study Hall (We'll use Clemson as an example), you can find both a team's results (complete with something called Win Expectancy) and remaining schedule (with something called Win Probability). My lack of creativity in coming up with names and headlines has caught up to me a bit here, and I've had to explain what I'm describing in those two fields quite a bit.

The latter, Win Probability, is pretty easy to explain. Clemson, the No. 1 team according to both S&P+ and the College Football Playoff committee, is given a 98 percent chance of beating No. 97 Wake Forest and a 92 percent chance of beating No. 82 South Carolina.

The former, however, has required more than a few explanations. The point of Win Expectancy is this: taking the stats from a given game -- efficiency, explosiveness, and all the things that end up going into S&P+ -- and tossing them up in the air, how often could a team have expected to win said game? You can win the statistical battle but, depending on the degree, still lose sometimes given the bounce of the pointy ball. (This is similar to what the "Estimated Wins" stat in each week's DVOA article does for NFL teams, but looking at individual games rather than the entire season.)

Regarding Clemson: the Tigers were so dominant against Wofford, Appalachian State, Georgia Tech, and Miami that no arrangement of each game's stats could have possibly created a loss. They had a 100 percent chance of winning those games, no matter the bounces. They were comprehensive wins for Dabo Swinney's squad.

For other games, however, the result was more in doubt. The Notre Dame result (a 24-22 win decided on a late two-point conversion stop), for instance, produced only a 54 percent win expectancy. That was a tossup game, and Clemson happened to call heads. Meanwhile, the Tigers were the better team against both Florida State (79 percent win expectancy) and Syracuse (76 percent) but left some room for doubt.

Win Expectancy is a backwards look at domination. There are some random, funky results -- because this is designed to look at full-game numbers (and doesn't filter out garbage time like the S&P+ ratings do), you'll occasionally see something like Texas A&M-Ball State, where A&M put the 56-23 game away quickly, but garbage-time issues led to a win expectancy of just 67 percent. (I've tinkered with the thought of using non-garbage stats for this number too, but I can't decide what makes the most sense for this one.)

While Win Expectacncy is indeed backward-facing, it turns out it might be able to see into the future a bit, too.

S&P+ is having a pretty decent season in terms of performance. My newly redesigned win probabilities are performing well, and while the performance against the spread has been pretty mediocre overall, most of that comes from an awkward transition from preseason projections to in-season data. (I've been tinkering with different methods to use for that transition in the future.)

S&P+ vs. the spread

Weeks 1-4: 54%
Weeks 5-7: 45%
Weeks 8-11: 54%

Even if the midseason results were shaky, S&P+ seems to have had a pretty good lay of the land in recent weeks.

But that obviously doesn't mean it's got a read on every team. Some of the current rankings -- North Carolina at No. 30, for instance -- still have me a little bit queasy, and it appears there might be reason for that.

If we compare projected win probabilities to actual results, we can get a decent read for who S&P+ has figured out and who it doesn't. But results limit everything to either 0 percent or 100 percent. Sometimes a metric like S&P+ doesn't really get things wrong when it projects the wrong team to win -- sometimes that team simply got lucky. (SB Nation's Northwestern site wrote a piece about a certain team that might fall into that category.)

By comparing pre-game win probability to in-game win expectancy, though, we can get an interesting picture of who is and isn't doing better than S&P+ thought.

Team	S&P+ Rk	Record	Pregame Win Probability	In-Game Win Expectancy	Difference	Rk
Clemson	1	10-0	84.6%	89.8%	5.3%	32
Alabama	2	9-1	79.3%	86.3%	7.0%	26
Ohio State	3	10-0	84.9%	97.4%	12.6%	15
Oklahoma	4	9-1	75.6%	88.2%	12.6%	14
Michigan	5	8-2	81.2%	86.1%	4.9%	35
Notre Dame	6	9-1	66.8%	85.6%	18.8%	4
Baylor	7	8-1	82.9%	91.0%	8.1%	23
Florida State	8	8-2	75.6%	85.9%	10.2%	21
Ole Miss	9	7-3	77.7%	79.0%	1.3%	56
Florida	10	9-1	67.9%	82.8%	14.9%	9
USC	11	7-3	74.6%	67.1%	-7.5%	107
Navy	12	8-1	70.3%	84.7%	14.4%	11
LSU	13	7-2	69.3%	73.6%	4.3%	38
West Virginia	14	5-4	71.2%	55.1%	-16.1%	124
Stanford	15	8-2	74.1%	78.3%	4.2%	39
Bowling Green	16	8-2	54.7%	79.7%	25.0%	2
UCLA	17	7-3	76.4%	75.2%	-1.2%	75
Wisconsin	18	8-2	74.8%	73.4%	-1.4%	78
NC State	19	6-4	64.2%	62.8%	-1.3%	77
Oklahoma State	20	10-0	65.0%	84.6%	19.6%	3
Tennessee	21	6-4	67.0%	62.5%	-4.5%	94
Western Kentucky	22	8-2	73.7%	76.2%	2.5%	49
Arkansas	23	6-4	57.8%	62.5%	4.7%	36
Mississippi State	24	7-3	63.3%	67.4%	4.1%	40
Washington	25	4-6	54.6%	48.0%	-6.5%	100
Team	S&P+ Rk	Record	Pregame Win Probability	In-Game Win Expectancy	Difference	Rk
Toledo	26	8-1	69.3%	77.0%	7.7%	24
Michigan State	27	9-1	72.4%	74.3%	1.8%	53
Iowa	28	10-0	64.2%	79.1%	15.0%	8
TCU	29	9-1	75.4%	82.1%	6.7%	27
North Carolina	30	9-1	63.0%	88.9%	26.0%	1
Utah	31	8-2	61.3%	65.1%	3.9%	41
Penn State	32	7-3	71.4%	67.7%	-3.7%	91
Louisville	33	6-4	61.3%	61.5%	0.1%	66
Memphis	34	8-2	65.4%	66.2%	0.9%	59
Louisiana Tech	35	7-3	71.6%	74.8%	3.2%	44
Appalachian State	36	8-2	72.8%	76.1%	3.3%	43
Houston	37	10-0	65.3%	78.2%	12.9%	13
BYU	38	7-3	62.9%	65.3%	2.4%	51
Georgia	39	7-3	69.0%	69.0%	0.0%	68
Georgia Southern	40	7-2	66.2%	77.7%	11.4%	18
Minnesota	41	4-6	47.6%	45.6%	-2.1%	81
Pittsburgh	42	7-3	55.4%	59.9%	4.5%	37
Oregon	43	7-3	61.0%	63.6%	2.6%	47
Illinois	44	5-5	57.6%	46.1%	-11.4%	116
Marshall	45	9-2	79.2%	74.0%	-5.2%	95
California	46	6-4	57.7%	59.6%	1.9%	52
Utah State	47	5-5	59.4%	56.9%	-2.5%	83
Duke	48	6-4	70.9%	62.0%	-8.9%	110
San Diego State	49	7-3	55.4%	71.6%	16.2%	6
Air Force	50	7-3	64.9%	69.9%	5.1%	34
Team	S&P+ Rk	Record	Pregame Win Probability	In-Game Win Expectancy	Difference	Rk
Temple	51	8-2	69.3%	72.0%	2.7%	46
Texas Tech	52	6-5	49.4%	56.0%	6.7%	28
Northern Illinois	53	7-3	58.9%	69.6%	10.7%	20
Nebraska	54	5-6	58.6%	54.6%	-4.0%	92
Virginia Tech	55	5-5	54.1%	53.5%	-0.6%	71
Boise State	56	7-3	79.0%	70.5%	-8.4%	109
Western Michigan	57	6-4	48.7%	58.0%	9.4%	22
Northwestern	58	8-2	53.7%	54.0%	0.3%	64
Cincinnati	59	6-4	55.8%	67.5%	11.6%	17
Miami-FL	60	6-4	53.0%	54.5%	1.5%	54
Southern Miss	61	7-3	53.7%	70.9%	17.2%	5
Texas A&M	62	7-3	66.2%	62.5%	-3.7%	90
South Florida	63	6-4	42.5%	57.1%	14.6%	10
Boston College	64	3-7	54.8%	49.4%	-5.4%	96
Indiana	65	4-6	42.0%	42.5%	0.5%	61
Iowa State	66	3-7	39.8%	41.2%	1.5%	55
Central Michigan	67	5-5	46.9%	49.9%	3.0%	45
Auburn	68	5-5	48.4%	46.8%	-1.6%	79
Arizona State	69	5-5	55.6%	49.0%	-6.6%	101
Georgia Tech	70	3-7	59.1%	31.7%	-27.4%	128
Washington State	71	7-3	46.3%	62.4%	16.1%	7
Colorado State	72	5-5	59.1%	56.3%	-2.8%	85
Ohio	73	6-4	59.2%	59.5%	0.3%	63
Arkansas State	74	7-3	54.1%	67.8%	13.7%	12
Maryland	75	2-8	34.5%	31.1%	-3.4%	88
Team	S&P+ Rk	Record	Pregame Win Probability	In-Game Win Expectancy	Difference	Rk
Kansas State	76	3-6	58.2%	33.4%	-24.7%	127
Middle Tennessee	77	5-5	44.4%	51.4%	7.0%	25
East Carolina	78	4-6	46.6%	42.5%	-4.1%	93
Arizona	79	6-5	49.9%	53.2%	3.3%	42
Vanderbilt	80	4-6	42.1%	40.9%	-1.2%	74
Akron	81	5-5	43.1%	54.0%	10.9%	19
South Carolina	82	3-7	41.7%	39.0%	-2.7%	84
Missouri	83	5-5	49.5%	42.5%	-7.0%	103
Syracuse	84	3-7	41.1%	39.8%	-1.3%	76
Buffalo	85	5-5	40.5%	46.9%	6.4%	29
Texas	86	4-6	42.9%	42.4%	-0.5%	69
Virginia	87	3-7	34.7%	27.7%	-7.0%	104
Purdue	88	2-8	32.3%	33.4%	1.0%	57
Tulsa	89	5-5	53.4%	41.8%	-11.6%	117
Troy	90	3-7	40.3%	42.7%	2.4%	50
Florida Atlantic	91	2-8	52.8%	43.8%	-9.0%	111
Massachusetts	92	2-8	42.0%	32.1%	-9.9%	113
Connecticut	93	5-5	41.5%	47.3%	5.9%	30
San Jose State	94	4-6	46.7%	44.7%	-2.0%	80
Colorado	95	4-7	37.7%	38.0%	0.3%	65
Kentucky	96	4-6	38.7%	39.4%	0.8%	60
Wake Forest	97	3-7	34.9%	27.2%	-7.7%	108
UNLV	98	3-7	39.2%	27.4%	-11.8%	118
UL-Lafayette	99	4-5	53.1%	41.0%	-12.1%	119
Georgia State	100	3-6	35.0%	46.6%	11.6%	16
Team	S&P+ Rk	Record	Pregame Win Probability	In-Game Win Expectancy	Difference	Rk
Nevada	101	6-4	53.2%	49.7%	-3.5%	89
Florida International	102	5-6	46.3%	43.0%	-3.2%	87
New Mexico	103	6-4	42.0%	41.5%	-0.5%	70
Fresno State	104	3-7	32.6%	35.2%	2.6%	48
South Alabama	105	5-4	36.7%	41.8%	5.1%	33
UTSA	106	2-8	41.4%	35.5%	-5.9%	97
Rutgers	107	3-7	29.0%	26.9%	-2.1%	82
Ball State	108	3-7	45.6%	33.0%	-12.5%	120
SMU	109	1-9	26.3%	16.1%	-10.2%	115
Idaho	110	3-7	35.4%	29.5%	-5.9%	98
Oregon State	111	2-8	34.5%	21.6%	-12.9%	121
Texas State	112	2-7	49.5%	28.0%	-21.4%	126
Kent State	113	3-7	36.3%	35.6%	-0.7%	72
Old Dominion	114	5-5	47.9%	34.8%	-13.1%	122
Tulane	115	3-7	30.0%	30.9%	0.9%	58
Hawaii	117	2-9	35.0%	28.2%	-6.8%	102
Miami-OH	118	2-9	24.3%	24.4%	0.0%	67
Wyoming	119	1-10	26.5%	12.6%	-13.9%	123
Rice	120	4-6	32.9%	38.3%	5.5%	31
New Mexico State	121	2-7	32.2%	32.7%	0.4%	62
Eastern Michigan	122	1-10	29.3%	19.3%	-9.9%	114
Army	123	2-8	38.8%	35.7%	-3.2%	86
UL-Monroe	124	1-9	25.7%	18.4%	-7.3%	106
Charlotte	125	2-8	33.3%	24.2%	-9.1%	112
Kansas	127	0-10	13.2%	7.0%	-6.1%	99
Central Florida	128	0-10	30.3%	11.1%	-19.2%	125
UTEP	129	4-6	26.8%	25.8%	-1.0%	73
North Texas	130	1-9	14.2%	7.0%	-7.2%	105

[ad placeholder 3]

This pretty quickly starts to point out some of what the naked eye would have considered odd ratings. USC is lingering near the top 10 despite three losses (two of which were by double digits). North Carolina is barely in the top 30 despite not only going 9-1 but frequently looking dominant. West Virginia has mastered some weird strength-of-schedule trick in lingering in the teens.

Of course, this is comparing full-season probabilities to full-season results, which means differences could have occurred because of faulty preseason projections or that awkward Week 5-to-Week 7 transition where schedule adjustments are iffy. If I'm saying that S&P+ has a pretty good read on the lay of the land today, then let's focus on more recent differences between S&P+ and reality. Here are the teams that have most drastically over- or underachieved their projections over the last four weeks:

Team	S&P+ Rk	Record	Wk 8-11 Difference	Rk
North Carolina	30	9-1	36.3%	1
San Diego State	49	7-3	30.7%	2
Northern Illinois	53	7-3	29.9%	3
Arkansas State	74	7-3	29.8%	4
Cincinnati	59	6-4	29.3%	5
Minnesota	41	4-6	29.0%	6
New Mexico	103	6-4	28.3%	7
Buffalo	85	5-5	28.2%	8
Oregon	43	7-3	27.1%	9
Bowling Green	16	8-2	25.9%	10
Virginia Tech	55	5-5	23.7%	11
South Florida	63	6-4	22.5%	12
Washington State	71	7-3	22.5%	13
Troy	90	3-7	21.9%	14
Air Force	50	7-3	20.8%	15
Alabama	2	9-1	20.3%	16
Arkansas	23	6-4	18.7%	17
Florida	10	9-1	18.2%	18
Southern Miss	61	7-3	17.6%	19
Middle Tennessee	77	5-5	17.4%	20
Notre Dame	6	9-1	16.0%	21
Purdue	88	2-8	15.8%	22
Georgia State	100	3-6	15.4%	23
Ohio State	3	10-0	15.4%	24
Ole Miss	9	7-3	14.3%	25
Team	S&P+ Rk	Record	Wk 8-11 Difference	Rk
Louisiana Tech	35	7-3	13.7%	26
Oklahoma State	20	10-0	13.6%	27
Connecticut	93	5-5	13.5%	28
Colorado	95	4-7	13.2%	29
Iowa State	66	3-7	13.2%	30
Navy	12	8-1	13.1%	31
Oklahoma	4	9-1	13.1%	32
Florida State	8	8-2	13.0%	33
Auburn	68	5-5	11.1%	34
Louisville	33	6-4	10.8%	35
Akron	81	5-5	9.9%	36
North Texas	130	1-9	9.3%	37
Maryland	75	2-8	8.6%	38
Penn State	32	7-3	8.5%	39
Western Michigan	57	6-4	8.4%	40
Central Michigan	67	5-5	8.2%	41
Colorado State	72	5-5	8.0%	42
UTEP	129	4-6	7.1%	43
Arizona State	69	5-5	6.2%	44
Arizona	79	6-5	6.1%	45
Virginia	87	3-7	6.0%	46
Texas Tech	52	6-5	6.0%	47
Georgia Southern	40	7-2	5.9%	48
Tennessee	21	6-4	5.7%	49
Texas	86	4-6	5.7%	50
Team	S&P+ Rk	Record	Wk 8-11 Difference	Rk
NC State	19	6-4	5.3%	51
Northwestern	58	8-2	4.1%	52
Fresno State	104	3-7	4.1%	53
Clemson	1	10-0	3.9%	54
Tulane	115	3-7	3.7%	55
Mississippi State	24	7-3	3.1%	56
Kansas	127	0-10	3.0%	57
Miami-FL	60	6-4	3.0%	58
Idaho	110	3-7	1.3%	59
Miami-OH	118	2-9	1.2%	60
Syracuse	84	3-7	1.1%	61
New Mexico State	121	2-7	0.5%	62
Western Kentucky	22	8-2	-0.6%	63
Stanford	15	8-2	-1.2%	64
Georgia	39	7-3	-1.2%	65
Michigan State	27	9-1	-1.8%	66
Marshall	45	9-2	-2.1%	67
Indiana	65	4-6	-2.6%	68
Nevada	101	6-4	-2.7%	69
Baylor	7	8-1	-3.4%	70
Wisconsin	18	8-2	-3.5%	71
San Jose State	94	4-6	-3.6%	72
Toledo	26	8-1	-3.8%	73
Rice	120	4-6	-3.8%	74
Memphis	34	8-2	-4.0%	75
Team	S&P+ Rk	Record	Wk 8-11 Difference	Rk
UCLA	17	7-3	-4.6%	76
Central Florida	128	0-10	-5.2%	77
South Carolina	82	3-7	-5.3%	78
Vanderbilt	80	4-6	-5.4%	79
UNLV	98	3-7	-5.6%	80
Utah	31	8-2	-5.9%	81
Florida International	102	5-6	-6.0%	82
Old Dominion	114	5-5	-6.1%	83
Missouri	83	5-5	-6.3%	84
Pittsburgh	42	7-3	-6.6%	85
Massachusetts	92	2-8	-6.8%	86
SMU	109	1-9	-6.9%	87
Kentucky	96	4-6	-8.1%	88
Wake Forest	97	3-7	-8.3%	89
BYU	38	7-3	-8.5%	90
USC	11	7-3	-9.3%	91
Florida Atlantic	91	2-8	-9.4%	92
Ball State	108	3-7	-10.0%	93
Houston	37	10-0	-10.2%	94
Rutgers	107	3-7	-10.2%	95
Oregon State	111	2-8	-11.0%	96
UL-Monroe	124	1-9	-11.5%	97
South Alabama	105	5-4	-11.7%	98
Temple	51	8-2	-11.9%	99
Tulsa	89	5-5	-12.0%	100
Team	S&P+ Rk	Record	Wk 8-11 Difference	Rk
Charlotte	125	2-8	-12.2%	101
Michigan	5	8-2	-12.2%	102
Wyoming	119	1-10	-12.7%	103
Nebraska	54	5-6	-12.9%	104
Iowa	28	10-0	-13.7%	105
Illinois	44	5-5	-14.7%	106
UL-Lafayette	99	4-5	-15.0%	107
TCU	29	9-1	-15.5%	108
Army	123	2-8	-16.9%	109
Washington	25	4-6	-16.9%	110
Appalachian State	36	8-2	-16.9%	111
Texas State	112	2-7	-17.1%	112
Utah State	47	5-5	-17.4%	113
California	46	6-4	-18.0%	114
UTSA	106	2-8	-18.2%	115
Hawaii	117	2-9	-19.5%	116
West Virginia	14	5-4	-22.2%	117
Boise State	56	7-3	-22.6%	118
Ohio	73	6-4	-22.6%	119
Texas A&M	62	7-3	-24.2%	120
LSU	13	7-2	-25.0%	121
Kent State	113	3-7	-30.3%	122
Kansas State	76	3-6	-31.7%	123
Georgia Tech	70	3-7	-32.1%	124
Duke	48	6-4	-34.9%	125
Eastern Michigan	122	1-10	-36.3%	126
East Carolina	78	4-6	-37.5%	127
Boston College	64	3-7	-39.3%	128

[ad placeholder 4]

This is a pretty good look at the teams that are either picking up steam or losing oil quickly. North Carolina has looked incredible, particularly in the past two or three games. San Diego State has been strangely fantastic, too, and NIU is performing its normal "Well, I guess it's time to go win the MAC" surge. Minnesota, iffy through most of the first half of the season, has been between competitive and downright good in losses to Michigan, Ohio State, and Iowa. And yes, Alabama is looking more and more like the killing machine we've come to expect.

On the other end of the spectrum, there are few surprises. Duke has fallen apart. Boston College's horrific offense has led to its awesome defense taking more chances and springing more leaks. Texas A&M is fading again. LSU has gone from Playoff contender to "Is Les Miles on the hot seat?" in record time.

We never really know until the past tense whether something is a brief blip or a sign of momentum, but this looks like a pretty good way of comparing where a team is to where S&P+ thinks it is.