Writers of Pro Football Prospectus 2008

Most Recent FO Features

BellLe'13.jpg

» 2016 Slot vs. Wide: RBs and TEs

Part II of our look at slot vs. wide targets focuses on running backs and tight ends. Was Le'Veon Bell used more diversely than David Johnson? Also: is there actually a stat where Rob Gronkowski ranked last among his peers?

20 Nov 2015

VN: The Numbers Are on to You (or Not)

by Bill Connelly

If you look at one of the advanced statistical profiles I've designed at Football Study Hall (We'll use Clemson as an example), you can find both a team's results (complete with something called Win Expectancy) and remaining schedule (with something called Win Probability). My lack of creativity in coming up with names and headlines has caught up to me a bit here, and I've had to explain what I'm describing in those two fields quite a bit.

The latter, Win Probability, is pretty easy to explain. Clemson, the No. 1 team according to both S&P+ and the College Football Playoff committee, is given a 98 percent chance of beating No. 97 Wake Forest and a 92 percent chance of beating No. 82 South Carolina.

The former, however, has required more than a few explanations. The point of Win Expectancy is this: taking the stats from a given game -- efficiency, explosiveness, and all the things that end up going into S&P+ -- and tossing them up in the air, how often could a team have expected to win said game? You can win the statistical battle but, depending on the degree, still lose sometimes given the bounce of the pointy ball. (This is similar to what the "Estimated Wins" stat in each week's DVOA article does for NFL teams, but looking at individual games rather than the entire season.)

Regarding Clemson: the Tigers were so dominant against Wofford, Appalachian State, Georgia Tech, and Miami that no arrangement of each game's stats could have possibly created a loss. They had a 100 percent chance of winning those games, no matter the bounces. They were comprehensive wins for Dabo Swinney's squad.

For other games, however, the result was more in doubt. The Notre Dame result (a 24-22 win decided on a late two-point conversion stop), for instance, produced only a 54 percent win expectancy. That was a tossup game, and Clemson happened to call heads. Meanwhile, the Tigers were the better team against both Florida State (79 percent win expectancy) and Syracuse (76 percent) but left some room for doubt.

Win Expectancy is a backwards look at domination. There are some random, funky results -- because this is designed to look at full-game numbers (and doesn't filter out garbage time like the S&P+ ratings do), you'll occasionally see something like Texas A&M-Ball State, where A&M put the 56-23 game away quickly, but garbage-time issues led to a win expectancy of just 67 percent. (I've tinkered with the thought of using non-garbage stats for this number too, but I can't decide what makes the most sense for this one.)

While Win Expectacncy is indeed backward-facing, it turns out it might be able to see into the future a bit, too.

S&P+ is having a pretty decent season in terms of performance. My newly redesigned win probabilities are performing well, and while the performance against the spread has been pretty mediocre overall, most of that comes from an awkward transition from preseason projections to in-season data. (I've been tinkering with different methods to use for that transition in the future.)

S&P+ vs. the spread

  • Weeks 1-4: 54%
  • Weeks 5-7: 45%
  • Weeks 8-11: 54%

Even if the midseason results were shaky, S&P+ seems to have had a pretty good lay of the land in recent weeks.

But that obviously doesn't mean it's got a read on every team. Some of the current rankings -- North Carolina at No. 30, for instance -- still have me a little bit queasy, and it appears there might be reason for that.

If we compare projected win probabilities to actual results, we can get a decent read for who S&P+ has figured out and who it doesn't. But results limit everything to either 0 percent or 100 percent. Sometimes a metric like S&P+ doesn't really get things wrong when it projects the wrong team to win -- sometimes that team simply got lucky. (SB Nation's Northwestern site wrote a piece about a certain team that might fall into that category.)

By comparing pre-game win probability to in-game win expectancy, though, we can get an interesting picture of who is and isn't doing better than S&P+ thought.

Team S&P+ Rk Record Pregame Win
Probability
In-Game Win
Expectancy
Difference Rk
Clemson 1 10-0 84.6% 89.8% 5.3% 32
Alabama 2 9-1 79.3% 86.3% 7.0% 26
Ohio State 3 10-0 84.9% 97.4% 12.6% 15
Oklahoma 4 9-1 75.6% 88.2% 12.6% 14
Michigan 5 8-2 81.2% 86.1% 4.9% 35
Notre Dame 6 9-1 66.8% 85.6% 18.8% 4
Baylor 7 8-1 82.9% 91.0% 8.1% 23
Florida State 8 8-2 75.6% 85.9% 10.2% 21
Ole Miss 9 7-3 77.7% 79.0% 1.3% 56
Florida 10 9-1 67.9% 82.8% 14.9% 9
USC 11 7-3 74.6% 67.1% -7.5% 107
Navy 12 8-1 70.3% 84.7% 14.4% 11
LSU 13 7-2 69.3% 73.6% 4.3% 38
West Virginia 14 5-4 71.2% 55.1% -16.1% 124
Stanford 15 8-2 74.1% 78.3% 4.2% 39
Bowling Green 16 8-2 54.7% 79.7% 25.0% 2
UCLA 17 7-3 76.4% 75.2% -1.2% 75
Wisconsin 18 8-2 74.8% 73.4% -1.4% 78
NC State 19 6-4 64.2% 62.8% -1.3% 77
Oklahoma State 20 10-0 65.0% 84.6% 19.6% 3
Tennessee 21 6-4 67.0% 62.5% -4.5% 94
Western Kentucky 22 8-2 73.7% 76.2% 2.5% 49
Arkansas 23 6-4 57.8% 62.5% 4.7% 36
Mississippi State 24 7-3 63.3% 67.4% 4.1% 40
Washington 25 4-6 54.6% 48.0% -6.5% 100
Team S&P+ Rk Record Pregame Win
Probability
In-Game Win
Expectancy
Difference Rk
Toledo 26 8-1 69.3% 77.0% 7.7% 24
Michigan State 27 9-1 72.4% 74.3% 1.8% 53
Iowa 28 10-0 64.2% 79.1% 15.0% 8
TCU 29 9-1 75.4% 82.1% 6.7% 27
North Carolina 30 9-1 63.0% 88.9% 26.0% 1
Utah 31 8-2 61.3% 65.1% 3.9% 41
Penn State 32 7-3 71.4% 67.7% -3.7% 91
Louisville 33 6-4 61.3% 61.5% 0.1% 66
Memphis 34 8-2 65.4% 66.2% 0.9% 59
Louisiana Tech 35 7-3 71.6% 74.8% 3.2% 44
Appalachian State 36 8-2 72.8% 76.1% 3.3% 43
Houston 37 10-0 65.3% 78.2% 12.9% 13
BYU 38 7-3 62.9% 65.3% 2.4% 51
Georgia 39 7-3 69.0% 69.0% 0.0% 68
Georgia Southern 40 7-2 66.2% 77.7% 11.4% 18
Minnesota 41 4-6 47.6% 45.6% -2.1% 81
Pittsburgh 42 7-3 55.4% 59.9% 4.5% 37
Oregon 43 7-3 61.0% 63.6% 2.6% 47
Illinois 44 5-5 57.6% 46.1% -11.4% 116
Marshall 45 9-2 79.2% 74.0% -5.2% 95
California 46 6-4 57.7% 59.6% 1.9% 52
Utah State 47 5-5 59.4% 56.9% -2.5% 83
Duke 48 6-4 70.9% 62.0% -8.9% 110
San Diego State 49 7-3 55.4% 71.6% 16.2% 6
Air Force 50 7-3 64.9% 69.9% 5.1% 34
Team S&P+ Rk Record Pregame Win
Probability
In-Game Win
Expectancy
Difference Rk
Temple 51 8-2 69.3% 72.0% 2.7% 46
Texas Tech 52 6-5 49.4% 56.0% 6.7% 28
Northern Illinois 53 7-3 58.9% 69.6% 10.7% 20
Nebraska 54 5-6 58.6% 54.6% -4.0% 92
Virginia Tech 55 5-5 54.1% 53.5% -0.6% 71
Boise State 56 7-3 79.0% 70.5% -8.4% 109
Western Michigan 57 6-4 48.7% 58.0% 9.4% 22
Northwestern 58 8-2 53.7% 54.0% 0.3% 64
Cincinnati 59 6-4 55.8% 67.5% 11.6% 17
Miami-FL 60 6-4 53.0% 54.5% 1.5% 54
Southern Miss 61 7-3 53.7% 70.9% 17.2% 5
Texas A&M 62 7-3 66.2% 62.5% -3.7% 90
South Florida 63 6-4 42.5% 57.1% 14.6% 10
Boston College 64 3-7 54.8% 49.4% -5.4% 96
Indiana 65 4-6 42.0% 42.5% 0.5% 61
Iowa State 66 3-7 39.8% 41.2% 1.5% 55
Central Michigan 67 5-5 46.9% 49.9% 3.0% 45
Auburn 68 5-5 48.4% 46.8% -1.6% 79
Arizona State 69 5-5 55.6% 49.0% -6.6% 101
Georgia Tech 70 3-7 59.1% 31.7% -27.4% 128
Washington State 71 7-3 46.3% 62.4% 16.1% 7
Colorado State 72 5-5 59.1% 56.3% -2.8% 85
Ohio 73 6-4 59.2% 59.5% 0.3% 63
Arkansas State 74 7-3 54.1% 67.8% 13.7% 12
Maryland 75 2-8 34.5% 31.1% -3.4% 88
Team S&P+ Rk Record Pregame Win
Probability
In-Game Win
Expectancy
Difference Rk
Kansas State 76 3-6 58.2% 33.4% -24.7% 127
Middle Tennessee 77 5-5 44.4% 51.4% 7.0% 25
East Carolina 78 4-6 46.6% 42.5% -4.1% 93
Arizona 79 6-5 49.9% 53.2% 3.3% 42
Vanderbilt 80 4-6 42.1% 40.9% -1.2% 74
Akron 81 5-5 43.1% 54.0% 10.9% 19
South Carolina 82 3-7 41.7% 39.0% -2.7% 84
Missouri 83 5-5 49.5% 42.5% -7.0% 103
Syracuse 84 3-7 41.1% 39.8% -1.3% 76
Buffalo 85 5-5 40.5% 46.9% 6.4% 29
Texas 86 4-6 42.9% 42.4% -0.5% 69
Virginia 87 3-7 34.7% 27.7% -7.0% 104
Purdue 88 2-8 32.3% 33.4% 1.0% 57
Tulsa 89 5-5 53.4% 41.8% -11.6% 117
Troy 90 3-7 40.3% 42.7% 2.4% 50
Florida Atlantic 91 2-8 52.8% 43.8% -9.0% 111
Massachusetts 92 2-8 42.0% 32.1% -9.9% 113
Connecticut 93 5-5 41.5% 47.3% 5.9% 30
San Jose State 94 4-6 46.7% 44.7% -2.0% 80
Colorado 95 4-7 37.7% 38.0% 0.3% 65
Kentucky 96 4-6 38.7% 39.4% 0.8% 60
Wake Forest 97 3-7 34.9% 27.2% -7.7% 108
UNLV 98 3-7 39.2% 27.4% -11.8% 118
UL-Lafayette 99 4-5 53.1% 41.0% -12.1% 119
Georgia State 100 3-6 35.0% 46.6% 11.6% 16
Team S&P+ Rk Record Pregame Win
Probability
In-Game Win
Expectancy
Difference Rk
Nevada 101 6-4 53.2% 49.7% -3.5% 89
Florida International 102 5-6 46.3% 43.0% -3.2% 87
New Mexico 103 6-4 42.0% 41.5% -0.5% 70
Fresno State 104 3-7 32.6% 35.2% 2.6% 48
South Alabama 105 5-4 36.7% 41.8% 5.1% 33
UTSA 106 2-8 41.4% 35.5% -5.9% 97
Rutgers 107 3-7 29.0% 26.9% -2.1% 82
Ball State 108 3-7 45.6% 33.0% -12.5% 120
SMU 109 1-9 26.3% 16.1% -10.2% 115
Idaho 110 3-7 35.4% 29.5% -5.9% 98
Oregon State 111 2-8 34.5% 21.6% -12.9% 121
Texas State 112 2-7 49.5% 28.0% -21.4% 126
Kent State 113 3-7 36.3% 35.6% -0.7% 72
Old Dominion 114 5-5 47.9% 34.8% -13.1% 122
Tulane 115 3-7 30.0% 30.9% 0.9% 58
Hawaii 117 2-9 35.0% 28.2% -6.8% 102
Miami-OH 118 2-9 24.3% 24.4% 0.0% 67
Wyoming 119 1-10 26.5% 12.6% -13.9% 123
Rice 120 4-6 32.9% 38.3% 5.5% 31
New Mexico State 121 2-7 32.2% 32.7% 0.4% 62
Eastern Michigan 122 1-10 29.3% 19.3% -9.9% 114
Army 123 2-8 38.8% 35.7% -3.2% 86
UL-Monroe 124 1-9 25.7% 18.4% -7.3% 106
Charlotte 125 2-8 33.3% 24.2% -9.1% 112
Kansas 127 0-10 13.2% 7.0% -6.1% 99
Central Florida 128 0-10 30.3% 11.1% -19.2% 125
UTEP 129 4-6 26.8% 25.8% -1.0% 73
North Texas 130 1-9 14.2% 7.0% -7.2% 105

This pretty quickly starts to point out some of what the naked eye would have considered odd ratings. USC is lingering near the top 10 despite three losses (two of which were by double digits). North Carolina is barely in the top 30 despite not only going 9-1 but frequently looking dominant. West Virginia has mastered some weird strength-of-schedule trick in lingering in the teens.

Of course, this is comparing full-season probabilities to full-season results, which means differences could have occurred because of faulty preseason projections or that awkward Week 5-to-Week 7 transition where schedule adjustments are iffy. If I'm saying that S&P+ has a pretty good read on the lay of the land today, then let's focus on more recent differences between S&P+ and reality. Here are the teams that have most drastically over- or underachieved their projections over the last four weeks:

Team S&P+ Rk Record Wk 8-11 Difference
Rk
North Carolina 30 9-1 36.3% 1
San Diego State 49 7-3 30.7% 2
Northern Illinois 53 7-3 29.9% 3
Arkansas State 74 7-3 29.8% 4
Cincinnati 59 6-4 29.3% 5
Minnesota 41 4-6 29.0% 6
New Mexico 103 6-4 28.3% 7
Buffalo 85 5-5 28.2% 8
Oregon 43 7-3 27.1% 9
Bowling Green 16 8-2 25.9% 10
Virginia Tech 55 5-5 23.7% 11
South Florida 63 6-4 22.5% 12
Washington State 71 7-3 22.5% 13
Troy 90 3-7 21.9% 14
Air Force 50 7-3 20.8% 15
Alabama 2 9-1 20.3% 16
Arkansas 23 6-4 18.7% 17
Florida 10 9-1 18.2% 18
Southern Miss 61 7-3 17.6% 19
Middle Tennessee 77 5-5 17.4% 20
Notre Dame 6 9-1 16.0% 21
Purdue 88 2-8 15.8% 22
Georgia State 100 3-6 15.4% 23
Ohio State 3 10-0 15.4% 24
Ole Miss 9 7-3 14.3% 25
Team S&P+ Rk Record Wk 8-11 Difference
Rk
Louisiana Tech 35 7-3 13.7% 26
Oklahoma State 20 10-0 13.6% 27
Connecticut 93 5-5 13.5% 28
Colorado 95 4-7 13.2% 29
Iowa State 66 3-7 13.2% 30
Navy 12 8-1 13.1% 31
Oklahoma 4 9-1 13.1% 32
Florida State 8 8-2 13.0% 33
Auburn 68 5-5 11.1% 34
Louisville 33 6-4 10.8% 35
Akron 81 5-5 9.9% 36
North Texas 130 1-9 9.3% 37
Maryland 75 2-8 8.6% 38
Penn State 32 7-3 8.5% 39
Western Michigan 57 6-4 8.4% 40
Central Michigan 67 5-5 8.2% 41
Colorado State 72 5-5 8.0% 42
UTEP 129 4-6 7.1% 43
Arizona State 69 5-5 6.2% 44
Arizona 79 6-5 6.1% 45
Virginia 87 3-7 6.0% 46
Texas Tech 52 6-5 6.0% 47
Georgia Southern 40 7-2 5.9% 48
Tennessee 21 6-4 5.7% 49
Texas 86 4-6 5.7% 50
Team S&P+ Rk Record Wk 8-11 Difference
Rk
NC State 19 6-4 5.3% 51
Northwestern 58 8-2 4.1% 52
Fresno State 104 3-7 4.1% 53
Clemson 1 10-0 3.9% 54
Tulane 115 3-7 3.7% 55
Mississippi State 24 7-3 3.1% 56
Kansas 127 0-10 3.0% 57
Miami-FL 60 6-4 3.0% 58
Idaho 110 3-7 1.3% 59
Miami-OH 118 2-9 1.2% 60
Syracuse 84 3-7 1.1% 61
New Mexico State 121 2-7 0.5% 62
Western Kentucky 22 8-2 -0.6% 63
Stanford 15 8-2 -1.2% 64
Georgia 39 7-3 -1.2% 65
Michigan State 27 9-1 -1.8% 66
Marshall 45 9-2 -2.1% 67
Indiana 65 4-6 -2.6% 68
Nevada 101 6-4 -2.7% 69
Baylor 7 8-1 -3.4% 70
Wisconsin 18 8-2 -3.5% 71
San Jose State 94 4-6 -3.6% 72
Toledo 26 8-1 -3.8% 73
Rice 120 4-6 -3.8% 74
Memphis 34 8-2 -4.0% 75
Team S&P+ Rk Record Wk 8-11 Difference
Rk
UCLA 17 7-3 -4.6% 76
Central Florida 128 0-10 -5.2% 77
South Carolina 82 3-7 -5.3% 78
Vanderbilt 80 4-6 -5.4% 79
UNLV 98 3-7 -5.6% 80
Utah 31 8-2 -5.9% 81
Florida International 102 5-6 -6.0% 82
Old Dominion 114 5-5 -6.1% 83
Missouri 83 5-5 -6.3% 84
Pittsburgh 42 7-3 -6.6% 85
Massachusetts 92 2-8 -6.8% 86
SMU 109 1-9 -6.9% 87
Kentucky 96 4-6 -8.1% 88
Wake Forest 97 3-7 -8.3% 89
BYU 38 7-3 -8.5% 90
USC 11 7-3 -9.3% 91
Florida Atlantic 91 2-8 -9.4% 92
Ball State 108 3-7 -10.0% 93
Houston 37 10-0 -10.2% 94
Rutgers 107 3-7 -10.2% 95
Oregon State 111 2-8 -11.0% 96
UL-Monroe 124 1-9 -11.5% 97
South Alabama 105 5-4 -11.7% 98
Temple 51 8-2 -11.9% 99
Tulsa 89 5-5 -12.0% 100
Team S&P+ Rk Record Wk 8-11 Difference
Rk
Charlotte 125 2-8 -12.2% 101
Michigan 5 8-2 -12.2% 102
Wyoming 119 1-10 -12.7% 103
Nebraska 54 5-6 -12.9% 104
Iowa 28 10-0 -13.7% 105
Illinois 44 5-5 -14.7% 106
UL-Lafayette 99 4-5 -15.0% 107
TCU 29 9-1 -15.5% 108
Army 123 2-8 -16.9% 109
Washington 25 4-6 -16.9% 110
Appalachian State 36 8-2 -16.9% 111
Texas State 112 2-7 -17.1% 112
Utah State 47 5-5 -17.4% 113
California 46 6-4 -18.0% 114
UTSA 106 2-8 -18.2% 115
Hawaii 117 2-9 -19.5% 116
West Virginia 14 5-4 -22.2% 117
Boise State 56 7-3 -22.6% 118
Ohio 73 6-4 -22.6% 119
Texas A&M 62 7-3 -24.2% 120
LSU 13 7-2 -25.0% 121
Kent State 113 3-7 -30.3% 122
Kansas State 76 3-6 -31.7% 123
Georgia Tech 70 3-7 -32.1% 124
Duke 48 6-4 -34.9% 125
Eastern Michigan 122 1-10 -36.3% 126
East Carolina 78 4-6 -37.5% 127
Boston College 64 3-7 -39.3% 128

This is a pretty good look at the teams that are either picking up steam or losing oil quickly. North Carolina has looked incredible, particularly in the past two or three games. San Diego State has been strangely fantastic, too, and NIU is performing its normal "Well, I guess it's time to go win the MAC" surge. Minnesota, iffy through most of the first half of the season, has been between competitive and downright good in losses to Michigan, Ohio State, and Iowa. And yes, Alabama is looking more and more like the killing machine we've come to expect.

On the other end of the spectrum, there are few surprises. Duke has fallen apart. Boston College's horrific offense has led to its awesome defense taking more chances and springing more leaks. Texas A&M is fading again. LSU has gone from Playoff contender to "Is Les Miles on the hot seat?" in record time.

We never really know until the past tense whether something is a brief blip or a sign of momentum, but this looks like a pretty good way of comparing where a team is to where S&P+ thinks it is.

Posted by: Bill Connelly on 20 Nov 2015

0 comments