Writers of Pro Football Prospectus 2008

Most Recent FO Features

HarvinPer09.jpg

» Impact of the NFL's Kickoff Rule Change

After three NFL seasons of kicking off from the 35-yard line, what has been the impact on touchbacks, returns, field position, scoring and injuries? Also, is this rule responsible for a record number of big comebacks?

02 Oct 2008

Varsity Numbers: Leverage, Leverage, Leverage

by Bill Connelly

Missouri head coach Gary Pinkel likes to say that playing defense is all about leverage. In a nutshell, if two guys are pursuing a ball-carrier who is running toward the sideline, the job of the first guy isn't necessarily to tackle him, but to make him cut back toward the middle of the field to be tackled. A runner who breaks a tackle near the sidelines will often find a clear path to the end zone. If he breaks a tackle in the middle of the field, however, there will usually be about four other guys in pursuit to bring him down. It's not necessarily about making the big play yourself, it's about making it harder for the runner to make the big play. Or something like that. Pinkel describes it much better than I do.

Why am I mentioning this? Because today’s Varsity Numbers column is about win correlations, and the numbers suggest that it is not necessarily how many big defensive plays you make that determines how well you do; it is more about leveraging the offense into uncomfortable situations -- in other words, Passing Downs.

First, let us define what constitutes a Passing Down. Once I had enough data to analyze, I began to look at sack rates and success rates for different down yardages. I determined that the following situations are tipping points between successful and unsuccessful drives in college football:

  • Second-and-8 or more;

  • Third-and-5 or more;
  • Fourth-and-5 or more.

There is a tremendous difference in sacks and successes for plays above and below those yardages. And I'd say the numbers back that up.

(And once again, Success Rate = FO’s Success Rate, adjusted for college data; PPP = EqPts Per Play; S&P = Success Rate + Points Per Play.)

NCAA Offense by Situation, 2007
Play Type Passing Downs Non-Passing Downs
Rushing 26.7% success rate
0.21 PPP
0.477 S&P
47.4% success rate
0.36 PPP
0.833 S&P
Passing 31.7% success rate
0.17 PPP
0.486 S&P
47.4% success rate
0.39 PPP
0.864 S&P
All Plays 30.1% success rate
0.18 PPP
0.483 S&P
47.4% success rate
0.37 PPP
0.845 S&P

You would naturally expect a pretty strong difference in levels of success between those two categories, but that is still pretty staggering.

With that background information out of the way, let’s move to today’s topic.

Win Correlations

Win Correlations, or WinCorr for short, is the correlation between any given statistical category and wins/losses. As you’ll see, they can serve a couple of different purposes: We can use them to determine which statistical categories are truly the most important on a national level, and we can look at team-specific WinCorr’s to develop a unique footprint for each team. I will cover the former this week and the latter next week.

National WinCorr

There are two ways to look at WinCorr on a national level: determining which statistical categories are most tied to winning a specific game, and determining which categories are most tied to winning seasons, i.e., being a good team. We'll look at both.

As I said above, we compare each statistical category with overall wins and losses, but how do we come up with a number for wins and losses when we're talking about a single game? We have a couple options:

  • Black and White: We give wins a 1 and losses a 0 and run correlations off of that.
  • Gray Areas: We compare the stats from each game with the percentage of points a team scored in that game. So if a team wins 20-10, instead of giving the winning team a 1, we'd give them a 0.667, as they scored 66.7 percent of the game's points.

The former is cleaner (and leads to lower correlations, obviously), but the latter is probably a bit more telling. There's a difference between winning 24-23 and winning 41-3.

A couple other things to note: First, I ran Spearman correlations for these numbers. Second, the list below shows the strongest correlations, so there is the possibility of a negative correlation on the list with positive correlations.

Also, I'm not listing the most obvious correlations. You don't need lots of stats to figure out that things like "percentage of points" and "total points" are going to be highly correlated to wins. And as you will quickly notice, for now I am sticking with my own stats and using EqPts instead of yards. Once I have pulled more of that information together, I can begin to look at both conventional stats and unconventional stats.

Single-game WinCorr, based on percentage of points
Stat Scenario WinCorr
PPP Close game* 0.682
S&P Close game* 0.678
PPP Overall 0.642
S&P Overall 0.634
Total EqPts Overall 0.617
Total EqPts Non-Passing Downs 0.597
Total Rushing EqPts Overall 0.587
Passing S&P Close game* 0.583
Total Rushing EqPts Non-Passing Downs 0.582
Total Rushes Fourth quarter 0.579
Success Rate Close game* 0.578
PPP Non-Passing Downs 0.575
Passing S&P Overall 0.573
Passing PPP Close game* 0.565
S&P Non-Passing Downs 0.565
Passing PPP Overall 0.563
Success Rate Overall 0.540
Total Rushes First down 0.534
Total Rushing EqPts First down 0.529
Rushing PPP Close game* 0.529
Rushing S&P Close game* 0.523
Total EqPts First down 0.521
Total Line Yards Non-Passing Downs 0.517
Total Passes Fourth quarter -0.516
Total Rushes Overall 0.510
*Close game = Scoring margin of 16 points (two possessions) or less

Correlations for those 25 categories were all over 0.500. This tells a few really interesting stories:

  • While the broadest stats see the top of the list, it's clear that success on Non-Passing Downs is crucial. Here's where the "leverage" idea comes into play. Stopping a first-and-10 rush for 3 yards instead of 5 creates a much less comfortable situation for the offense. Little things like that could be seen as just as important as big hits and singular big defensive plays over time.
  • What strikes me as most interesting here is that PPP is worth a smidge more than S&P. The idea of S&P (Success Rates + Points Per Play) is to combine the efficiency of Success Rates and the explosiveness of PPP. However, PPP's WinCorr is 0.642 (in close games, 0.682), while the Success Rate WinCorr is only 0.540 (in close games, 0.578). It drags down the overall applicability of S&P. I may have to think about retooling the idea of S&P, giving PPP more weight.
  • It's also interesting that PPP (the ability to make big plays) and S&P are more important to the passing game, while total (not per-play) EqPts is more important to the rushing game. Draw your own conclusions there, but to me it seems that keeping the defense honest with the threat of a good passing game is as important as actually performing well all day in the passing game.
  • One of my initial suspicions rings true: In the end, the "close game" numbers are a bit more important than the "overall" numbers. This is what I expected -- and it's why I created a "close game" measure in the first place -- but affirmation is always nice.
  • Obviously the presence of "total fourth quarter rushing attempts" (as a decent positive correlation) and "total fourth quarter passing attempts" (as a negative one) is a bit of "correlation vs. causation" here. It's not that you win more because you're rushing in the fourth quarter, it's that you rush more in the fourth quarter because you're winning. This does, however, verify that bit of conventional wisdom.

Now we’ll look at WinCorr over the course of a season, i.e. comparing season totals to a season “% of pts” total. See if you can pick out trends.

Season WinCorr, based on percentage of points
Stat Scenario WinCorr
EqPts Per Game Overall 0.752
PPP Overall 0.749
PPP Non-Passing Downs 0.748
S&P Non-Passing Downs 0.745
S&P Close games 0.733
PPP Close games 0.725
Rushing PPP Non-Passing Downs 0.722
S&P First downs 0.711
Success Rate Overall 0.708
PPP First downs 0.706
Rushing PPP Overall 0.702
Rushing PPP Close games 0.693
Passing S&P Overall 0.692
Success Rate Close games 0.690
Rushing S&P Overall 0.687
Rushing PPP First downs 0.687
Rushing S&P Non-Passing Downs 0.686
Rushing S&P Close games 0.682
S&P Third downs 0.681
S&P First quarter 0.675
Success Rate Non-Passing Downs 0.675
Success Rate Third downs 0.674
Passing PPP Overall 0.663
PPP First quarter 0.654
Rushing S&P First downs 0.653

  • Of the 25 stats on the list, eight were either related to Non-Passing Downs or first downs. Leverage, leverage, leverage.
  • EqPts and PPP continue to exceed S&P and Success Rates in importance. Ten of the 25 most important stats were variations of PPP, 11 if you include EqPts Per Game.
  • Interesting, though, is the fact that over the course of a season, overall PPP becomes more important than close-game PPP. I have no immediate explanation for that.
  • Nine rushing statistics on the list, two passing statistics. Interesting. And these categories have nothing to do with pure quantity of rushes (which means the "correlation vs. causation" argument really doesn't come into play).

'+' Number Correlations

Since I spent all that time developing the "+" Number concept, you had to know I was going to look at that too. What's funny, though, is that for this one, the correlations with win percentage were significantly stronger than the correlations with percentage of points. Correlations in the 0.9 range? That's quite significant. So that's what we're going to use.

"+"-number WinCorr, based on win percentage
Offense Defense
Stat Scenario WinCorr Stat Scenario WinCorr
S&P+ Close games 0.910 EqPts+ Overall 0.919
EqPts+ Overall 0.899 S&P+ Close games 0.898
S&P+ Overall 0.896 S&P+ Overall 0.889
Passing S&P+ Overall 0.801 Passing S&P+ Overall 0.787
Rushing S&P+ Overall 0.765 Rushing S&P+ Overall 0.775
Rushing EqPts+ Overall 0.763 Rushing S&P+ Close games 0.734
Rushing S&P+ Overall 0.745 S&P+ Non-Passing Downs 0.733
S&P+ Second downs 0.680 S&P+ First downs 0.730
Rushing S&P+ Close games 0.674 Rushing S&P+ Non-Passing Downs 0.699
Passing S&P+ Close games 0.669 Rushing S&P+ First downs 0.696

  • If you're a "Defense wins championships" kind of person, then here you go: Defensive EqPts+ is more tied to winning percentage than Offensive EqPts+. Granted, we're talking 0.919 vs 0.899, but it is indeed a difference!
  • While EqPts+ are more tied to winning on the defensive side, S&P+ numbers are more tied to winning on the offensive side.
  • Four more "leverage stats" show up on the defensive side (S&P+, Non-Passing Downs; S&P+, first downs; Rushing S&P+, Non-Passing Downs; Rushing S&P+, first downs), however they're conspicuously absent from the offensive list. Granted, they're still important (garnering correlations of 0.663, 0.573, 0.557, and 0.482 respectively), but they're significantly less important. It seems consistency is most important on defense, while pure big-play potential and the ability to score at any moment -- in Passing Downs or Non-Passing Downs -- makes the most difference on offense.

So we can reach some pretty interesting conclusions from this data, and most of it comes back to the idea of leverage. I have data broken out for all quarters, all downs, the red zone, etc., and by far the most significant category is how teams perform in Non-Passing Downs.

Passing Downs = Turnaround?

This brings me to an interesting question: If Passing Downs are equivalent to death, on average, then would the teams with the best numbers on Passing Downs be privy to a possible turnaround in luck the next year? In other words, are Passing Downs a lot like turnovers? Is success in the category somewhat arbitrary, and does it even out over time? I only have one full year of play-by-play data, so all I can do is take a look at the best (and worst) teams in the category, speculate, and see what happens at the end of the year. When I have multi-year data, it's going to be fun to tie all these season stats to success the next season, so I can see which stats are the best predictors of future success.

A list of the top 10 offenses, based on S&P+ on Passing Downs, looks like this:

1. Nebraska
2. Florida
3. Oregon
4. Texas Tech
5. Tulsa
6. Washington State
7. Kentucky
8. Hawaii
9. Louisville
10. West Virginia

Now, Florida, Texas Tech, Tulsa, Kentucky, Hawaii, and Louisville were six of the best passing teams in the country, so their presence on the list should surprise no one. Oregon and West Virginia had great all-around offenses as well. But Nebraska? Washington State?

What if I looked at the teams with the most disproportionate success on Passing Downs? Would that give me an indication of who might be due a turnaround in 2008? Here's a list of the top 10 teams, based on the ratio of success on Passing Downs to success overall.

1. Houston (0.974)
2. Tulsa (0.965)
3. Indiana (0.921)
4. Memphis (0.906)
5. Nebraska (0.889)
6. Nevada (0.886)
7. Texas Tech (0.882)
8. Minnesota (0.876)
9. Kentucky (0.862)
10. Toledo (0.860)
11. Washington State (0.859)
12. Boise State (0.851)
13. Hawaii (0.836)
14. Bowling Green (0.836)
15. Wisconsin (0.822)

Now, A) I only have BCS games entered to date, B) it's early in the season -- some BCS teams on that list haven't played the toughest of schedules, and C) some of those teams have changed quarterbacks or even coaches (have I given enough disclaimers yet?), but let's see what a comparison of 2007 and early-2008 numbers tells us about disproportionate Passing Downs success.

Disproportionate Passing Down success, 2007 to 2008, BCS teams only
2007 2008
Team S&P PD S&P Ratio S&P PD S&P Ratio Change
Indiana 0.717 0.661 0.921 0.828 0.560 0.676 -0.245
Nebraska 0.885 0.787 0.889 0.937 0.605 0.646 -0.243
Texas Tech 1.020 0.899 0.882 1.049 0.884 0.842 -0.040
Minnesota 0.707 0.619 0.876 0.949 0.966 1.018 0.142
Kentucky 0.879 0.758 0.862 0.770 0.555 0.721 -0.141
Washington State 0.728 0.626 0.859 0.615 0.514 0.836 -0.023
Wisconsin 0.808 0.664 0.822 0.817 0.730 0.894 0.072

So it's quite early, and I'm pretty sure further schedule difficulty will help bump down Minnesota's numbers a bit, but half the teams on this list have seen quite a decent change of proportion. However, only two teams have seen their overall S&P drop so far. I'll be checking on these numbers at the end of the year.

Summary

One exciting thing (I hope) about getting on with Football Outsiders at this time is that, as I said in my first column, we're just on the ground floor here. Running correlations of stats to wins is something I've wanted to do for a long time -- not just for these stats, but for the standard box score stats as well -- and something like this is just the start.

I was intrigued by the fact that explosiveness (Points Per Play) is worth more than efficiency/consistency (success rates); I was also intrigued by the staggering numbers in what I've been calling "leverage" figures. It certainly seems to spell out the surest blueprint for winning: 1) Do whatever you can to stay out of Passing Downs and awkward situations that lead to turnovers and easy scores for your opponent, and 2) Have explosive players who can score at any point from anywhere. It definitely shows why there's such a premium on those top-shelf, explosive recruits, but it also shows that there's a way to win by playing smart and using leverage to your advantage.

Posted by: Bill Connelly on 02 Oct 2008

5 comments, Last at 05 Oct 2008, 11:53am by Bowl Game Anomaly

Comments

1
by pawnking (not verified) :: Fri, 10/03/2008 - 4:00pm

Interesting, I'll be considering this over the weekend.

2
by Pat (filler) (not verified) :: Fri, 10/03/2008 - 4:01pm

I may have to think about retooling the idea of S&P, giving PPP more weight.

Why don't you just run a 2D correlation with success rate and points per play? That'll tell you whether success rate is adding anything to the discussion at all. There's a slight worry in that there's probably a correlation between success rate and points per play, but you can check that, too.

I also really, really think that you should run win correlations on games just between teams in the Top 25, too. It is a major, major assumption that games between teams in the Top 25 are in any way similar to games between teams in the Top 25 and teams at the bottom of the league.

It's entirely conceivable that a team can be easily built to dominate inferior opponents, but poorly constructed to match up with teams of equivalent strength.

How you delineate "teams in the top 25" is a reasonable point, but to be honest, most measures will probably have the majority of the teams the same, and the few outliers (the teams where maybe their true strength is between 20-30 or so) won't make a difference, as realistically, you're trying to see if there's a bias due to the fact that the league is gigantic.

3
by Tom Gower :: Fri, 10/03/2008 - 5:12pm

explosiveness (Points Per Play) is worth more than efficiency/consistency (success rates)
Is this true for all levels of Success Rate? My supposition is that, in the college game, you see less capability for the sort of consistent success Success Rate values, particularly in the passing game, which leads in turn to a greater emphasis on those explosive plays. If you can complete 50% of your short passes and 20% of your long passes, it makes sense if you throw deep more than if you complete 65% of your short passes and 22% of your long passes. See, e.g., Kirby Freeman's 1/14, 84, 1/? for the Hurricanes, and compare, say, Todd Reesing.

4
by Joseph :: Sat, 10/04/2008 - 2:30pm

I'll say that the difference in "explosive points" for O vs. "success rate" for D is their natural purpose. In other words, as an offense, a big play can lead directly to points, or be the boost you need coupled with a couple of successful but shorter-gain plays to score. On D, steady defense will lead to stops, even if you allow a few first downs. Eventually the offense will put together an imcompletion or two together with a short gain or two and have to punt/kick a FG/have a turnover (tradional or on downs).

5
by Bowl Game Anomaly :: Sun, 10/05/2008 - 11:53am

Several issues:

1. I wouldn't worry about PPP being more important than S&P. The difference is almost definitely not significant. In fact, I wouldn't worry about almost any stat being higher than any other stat in your first 2 win correlation tables (single game and season), because the differences are relatively small. Maybe when you get another few seasons of data you can say more definitively that performance in one stat leads to a higher probability of winning, but at this point I'd stick with "any stat with a correlation of .60 to .70 is a good one, and any stat higher than .70 is a very good one," and so on. (BTW, instead of cutting the stats off arbitrarily at 25 categories, it would have made more sense to cut them off at a certain minimum correlation, like .55 off the top of my head.) The fact that S&P looks better when you incorporate the + aspect should alleviate any concerns anyway.

2. I really don't like how you switch standards from percentage of points to pure wins on the + stats just because you got higher correlations that way, for 2 reasons. Higher correlations are good, but by changing the standard you have made your results incomparable to each other. There is no way to evaluate how your + stats compare to your non + stats. Even more importantly, those pretty correlations on your + stats are not so impressive if you honestly believe that percentage of points is a better measure of winning than pure wins. Essentially you just sacrificed good methodology for the sake of better numbers.

3. This is tied to point 1, but your first 2 conclusions from the + data table are not justified by the numbers. The differences are too small. I think the 3rd conclusion is legitimate, though.

4. IF YOU ARE TRYING TO FIND THE SINGLE BEST STAT FOR TEAM RANKINGS: One problem with win correlations though is that you don't want the correlation to be too high because you're looking for something better than wins, not exactly the same as wins. This is why Aaron cares that one year's DVOA correlates with the following year's wins more than the first year's wins correlates with the following year's wins (rather than trying to get DVOA to correlate as highly as it can with the same year's wins). If team quality is relatively more stable year-to-year that its wins are (is that one of your assumptions?), then you need to make sure your stat is also more stable year-to-year than wins. That means looking for correlations over multiple years of data. If you don't assume team quality is stable, then you should be looking for the stat with the highest correlation to wins (over a full season, I think, not a single game) which is based on a good theoretical foundation, like your + stats.

5. Building on point 4: While I think you should re-calculate the + stats using the same standard as before (percentage of points rather than pure wins), based on the numbers you have now it seems apparent that S&P+ and EqPts+ are the 2 best team ranking stats, but not which is better. If you want a ranking system for teams, I would use both of those side-by-side until you can demonstrate than one is definitively better than the other.

(Formerly "The McNabb Bowl Game Anomaly")