Caponomics Part III: Planning by Position

Guest Column by Bruce Stram

With some groundwork laid in Caponomics Part I [1] and Part II [2], I can take a logical next step and attempt to make the simplifying assumptions in my model more realistic. The trick is to avoid making things so complicated that it becomes nearly impossible to deduce results. I also will outline how some of these concepts might be translated into real world analysis.

The most glaring fallacy of my simple model is the fiction that a player can be rated with a skill-level number and that those numbers can be simply added to get an overall team skill level. A number of people commented on this after parts I and II of this series. One obvious correction is to recognize that football has player positions that require different skill sets. There's every reason to suppose that skill levels don't conveniently add up. Changing this assumption to something more realistic is going to take a bit of care.

I'm going to try to accomplish two very distinct things in the rest of this article. First, I'm going to outline a theoretical method of assessing player quality at the level of fundamental physical skills in order to test how the addition of the concept of position players affects the conclusions already drawn. I'm by no means sure these concepts can be applied in the real world: they're a test of the theoretical model. Then, I'm going to outline a very practical method of assessing the relative value of player performance along the lines of Football Outsiders' DVOA methodology [3].

First, definitions are in order. A player position is defined in terms of function, and results are measured in terms of performance of that function. Performance is determined by a player skill set. At the end of the day, the object is to add up the projected performance levels determined by available skill sets at each position into a total number that reflects team quality.

Bear with me as I define positions: QB, RB, FB, WR, TE, C/G, and OT on offense, DE, DT, LB, CB and S on defense. Surely one might quibble about whether these categories ought to be further refined, but clearly a significant level of complexity has been added.

I've tried to define these positions so that they differ according to the productivity of different elements of player skills or attributes. Thus speed afoot is highly productive for RB and WR, but not nearly so important for FB and QB.

Now a big problem: how can something like QB skills be added to DE skills? The short answer is "not easily," so I need to be careful to set this up right. First the measurement of skill sets needs to be anchored in a firm reference point. How can one player's skill set be measured relative to another at a given position? One possibility is to compare player's skills to the league average at their position. Then performance at each position can be assigned an "importance" factor defining its relative value. Thus a QB 150% better than average at a position with a importance factor of 2 counts 3, and an average OT whose performance rates as .75 counts .75.

But the notion of league average isn't a very satisfactory reference point. First, the average player quality at a given position is rather changeable over time, a bad feature for a reference point. Second, it's hard to define, particularly in terms of skill sets, what is meant by the relative measurement of quality: does twice average mean anything?

Both issues are substantially resolved by employing the concept of "replacement level player" borrowed from sabermetrics.

The most critical feature of this concept is the notion that in sports (and in fact in most fields of skilled endeavor), the population of players does not form the familiar "bell curve" that is applied so successfully to various studies of general population characteristics. In fact the population of players that has attained a high level in any sport is taken from the upper tail of the general population distribution of skills related to that sport. As one moves out on the tail of that distribution, the number of persons with requisite skills thins out rapidly. But while this number may be very small relative to the total population, it is very large relative to the very few number of roster slots available in the NFL (or any professional sport.)

Thus there are, from the sports roster point of view, a relatively large number of skilled people (skilled that is from a general population point of view) that can be recruited to fill in team positions on a default basis. By definition, these players are "a dime a dozen" and command only the league minimum salary. Because the general population distribution is pretty unchanging, this replacement level of skill is likely to remain constant (unlike league average skill). Further, the bell curve distribution of population and the spot that replacement players hold on that distribution provide a means to relatively index skill quality. Comparison to league average is not as good an index because the player population in the league does not constitute a well-defined statistical distribution.

I'll try to catch everyone up with an example. Suppose in terms of the physical skills that determine performance at one of the given football positions, replacement level players are known to have skills 3 sigmas above the average of the general population (This represents the top .1% of the population in terms of that skill set, but still constitutes an ample pool of 200,000 males in the U.S. Sigma is a relative measure of how far a given attribute, say speed, varies from the population average.) Thus, an average player might be 3.5 sigma, and a very good player 4 sigma and so on. I'm assuming for the sake of argument that the replacement level player is 3 sigmas.

With this in hand, we may readily compare the value of a 3.5 sigma QB to a 4 sigma OT. First we can say that the tackle has a better set of skills relative to his position than does the QB. But the QB may still be more valuable because his position is more important. The importance factor applied is simply a measure of how much increasing skill levels at each position (in terms of sigmas) contribute to winning.

The replacement player concept explains why an average player commands salary above the league minimum. An average player is simply better than a readily available replacement, and thus commands a greater salary.

Another concept needs to be added as well: I think it is reasonable to assume that at a given position, the contribution of increased skill has declining value in terms of winning. This concept of declining returns to a given factor is very common in economic analysis, and it has certain very desirable mathematical properties, but it doesn't really have to be true. But in the real world, economists have found it to be true almost all the time. Per the NFL, I think it's obviously true, at least in the extreme. I don't know that one can imagine an OT so good that he could lead a team that was filled entirely with replacement players at every other position to the playoffs, with a replacement level RB following that tackle OT for a 5 yard gain on almost every play, and that offense scoring so many points to make up for the points given up by a replacement level defense.

I think this assumption also applies to every other position, even QB, though I'd be eager to hear about counterexamples.

How do these changes affect the results previously inferred? First and foremost, the degree of complexity associated with function of team management has increased dramatically. Coaches have to develop player skills across a great range of functions associated with the multiplicity of positions. They must be able to assess skills relative to those positions in conjunction with the GM. Teams need to balance spending among a number of positions, since acquiring the highest level skill at a given position may be too costly to pay off in terms of winning compared to upgrading other positions. And they have to assess the value of position or skill sets relative to one another. (In fact folks like us can join in: The NFLPA publishes stats on starter and average salaries by position [4].)

However, in terms of all of the basic conclusions I drew initially, nothing has changed. If each team spent exactly the salary cap, and player skills were well known and unchanging, the expected outcome for each team would be 8-8, and there would be a balance between salary level and skill level for each position for each team. Teams would become winners based on strategy, player motivation, and luck.

If player skill is assumed to have an uncertain value, then teams compete as to assessing and developing those skills, though in a more complex fashion than had been previously envisioned, and so on.

Theoreticians like this sort of outcome because it tends to show they were on the right track in the first place with the simple model.

The franchise player concept introduced in Caponomics Part I [1] can be further illuminated in this more complex framework. The definition of a franchise player used here has been a player who makes some other players perform above their skill level. (Economists would call these effects "externalities."

Each position can be examined to find its potential for creating "external" effects. Start with offensive tackle. Sure, football is a team sport, so a tackle executing properly on a complicated running play is essential for the pulling guard's contribution to matter. But that generally doesn't mean the guard's job is easier or can be performed by a less skilled player.

A defensive end, on the other hand, has some potential for external effects. A great pass rusher disrupts the passing game of the opponent, and makes pass coverage easier for DB's and LB's. If the DE is sufficiently disruptive, he may command a double-team, which makes the jobs of other defensive linemen easier. If he's also stout against the run, he may get double teamed on almost all plays. (By the way, I think I've just described Reggie White.) The DT, on the other hand, stuffs the run up the middle -- that's his job and if he's good at it, opponents simply don't run up the middle effectively.

Obviously there's room for dispute here, but I'd put LB's and CB's in pretty much the same category.

On offense, it's obvious that a great RB can make the line's job a lot easier by, among other things, effectively setting up defensive players for easy blocks. A truly disruptive RB (think Barry Sanders) can have defensive players so focused on him that everything else the offense does is easier. (An aside I can't help but make: the Lions of a few years ago had not only Sanders, but Herman Moore, an elite receiver, and wasted both their careers with a series of unproven quarterbacks. If any situation ever called for getting a good vet like Drew Bledsoe and going for it now, it was that. One of the great management failures of all time, in my opinion.)

WR's are similar. There are a few so good that they get open almost at will (Randy Moss), or require such distortion of the defense that everything else is easier, or catch the ball whether they're open or not (Michael Irvin, Herman Moore, Sterling Sharpe).

But QB is obviously the position that has the greatest impact for making other players look better. First of all, as the "on field" offensive commander, the QB's ability to change plays or otherwise have the offense react to what the defense is doing can make things much easier for everyone on a given play. In addition, certain specific physical skills can have great leverage, i.e. various aspects of his throwing ability. For example, Brett Favre's most extreme skill is his ability to throw the ball very hard and very accurately over medium distances. At a minimum, this means a receiver can get the ball in tighter circumstances than with other quarterbacks, so that the offense can make do with somewhat less speedy, quick and/or strong receivers. His skill makes whatever receivers look better than they are. Even more importantly, in the right passing scheme Favre can get the ball to open receivers exactly in stride more quickly than other QB's, giving the defense fractions of a second less time to close while the ball's in the air. This leads not only to nice completions, but great yards after the catch. This skill was the primary driver, in my opinion, that helped Mike Holmgren create a truly great offense from 1995-1997 with only slightly better than average receivers and an average offensive line.

The reader who has worked through all this theorizing might at this point ask whether such concepts can be put into practice. I think the answer is clearly yes, though with great difficulty. These ideas move us in the direction of studying the performance of every player on every play, grading that performance, and then relating that set of performances over a large number of plays to team success. Absent such analysis, one simply relies on the judgment of coaches and general managers. Such opinions obviously benefit from experience, but can be problematic because they are subjective (for example, one or two particularly good or bad plays will stand out in the memory rather than a player's overall performance).

I'll outline specifically how a systematic analysis of the relative value of player performance might be undertaken for football, but first I want to review the history of statistical analysis of player performance and management strategy in baseball.

I suspect Football Outsiders readers are familiar with Bill James and the basic ideas of sabermetrics. James, and then many others, questioned the received wisdom about the relative importance and productivity of certain player skills. Very quickly they learned a number of important lessons that seriously contradicted conventional wisdom. There are two points to be made here: first, don't always trust conventional wisdom in sports, no matter how crusty and blustery the source; and second, bring analysis down to the most elemental level of performance possible.

Baseball analysts had the distinct advantage that baseball already kept many stats relating to the outcome of fundamental play in baseball: a batter's plate appearance. Football analysts have no such luxury.

I find Aaron Schatz's stats to be very compelling because he does analyze the outcome of every play (even though I quibble with his statistical methods; we economists just chew through mind-bending statistical concepts.) He's shown that a team's capability of achieving what he defines as success on a given play relates strongly to winning.

But he's only partway there to evaluating the importance of individual player performance and the value of position play. Player performance for each position must be defined by an absolute grading. That grading must then be applied to each play or a large sample of plays for each player whose evaluation is desired. To the extent possible this concept must be applied both to NFL veterans and prospective college players. Obviously this is a lot of work and constitutes a huge body of data, which is critical to making an objective estimate of each position's contribution to winning. Then team success or team performance can be statistically related to player performance, and the relative contribution to success can be measured for each position or player. Relative position value factors can be estimated as follows: perform a statistical multiple regression analysis that relates team success (winning in terms of number of games, or percentage of wins} to the graded level of performance at each position. The coefficients estimated for the variables defined by the performance grades at each position indicate the relative importance of performance at each position.

If offenses and defenses have classes of "style" which one suspects can alter the relative value of position players, then the performance data should be grouped into such classes and separate analyses performed for each.

Similar methods applied to different levels of college play could be used to translate player performance grades at that level to the pros. That is, how do players at certain grade levels in college tend to grade out in the pros?

I know this sounds like a heroic effort, and it's not something Aaron's going to be able to do in his spare time, even if he had some. (Ed. note: I find this concept of "spare time" to be fascinating, but I'm just a caveman.) I do think most if not all NFL teams are devoting significant resources toward such efforts, as they should be. Unfortunately, I think this means that with much less in the way of resources, we amateur analysts are just going to be amusing ourselves rather than creating any breakthroughs.

I would also readily agree with the obvious statement that any such statistical analysis is a supplement to, and not a substitute for, human judgment. (Ed note: So would the San Francisco 49ers, despite what you may read in Bay Area newspapers.) Coaches and GM's should use such analysis to test and perhaps modify their preconceived ideas. If they're lucky, maybe they'll find some good new ideas in the statistical results that make sense after further consideration. Maybe they'll find some "diamond in the rough" players whom they might have otherwise overlooked.

Finally, I wanted to conclude this series on Caponomics with a comment on the long-term effect of strategy. I've implicitly considered team management issues from that point of view. That's who makes the decisions, and that's the position we fans mentally tend to play

But there's another point of view: that of the players. Management may prefer an "Even Keel" strategy, but it is the players who are the replaceable part under that strategy. While team spending over the very long term may be the same whether the team pursues the "Even Keel" strategy or the "Boom and Bust" strategy (both described in Caponomics Part II [2]), player careers occur in the very short run. Players surely prefer a strategy that maximizes their own personal income, career stability, and personal happiness. "Even Keel" is not that strategy.

Will players tolerate it when a coach like Bill Belichick tosses them aside without a negative reaction? What if Damien Woody had been hurt playing out of position "for the good of the team," or Troy Brown? What if they received poor contracts as free agents after playing a year out of position? One can cogently argue their chances of getting hurt or salary offers were unaffected by the change in position, but if things turn out badly, the person affected, are likely to look for someone else to blame regardless of the merits.

Ultimately, is "Even Keel," whose virtue is stability, stable?

Bruce Stram has a PhD in economics from the University of Maryland, was an executive for a major U.S. corporation for a number of years, and now works in business development. He's been a Packer fan since birth in Green Bay, where his mother rocked his cradle while whispering of the long ago glory days of Vince Lombardi. Bruce can be reached at Bruce_Stram@sbcglobal.net [5]. If you have an idea for a guest column, something that analyzes the NFL from a distinctive point of view, please email us at info@footballoutsiders.com [6]. In particular, we welcome submission of guest columns that either further develop or provide counter-arguments to the ideas from this three-part series on Caponomics.