Similarity scores were first introduced by Bill James to compare baseball players to other baseball players from the past. The general idea was to start at 1000 points and subtract for the various differences between two players; the players closest to 1000 were the most similar. The method is all over the great Baseball Reference [1] website and, just as UNIVAC eventually led to your Palm Pilot, can be seen as the ancient predecessor to advanced baseball projection methods like Nate Silver's PECOTA [2].
It was only natural that the idea would spread to other sports as statistical analysis spread to other sports. NBA analyst John Hollinger has created his own version to compare basketball players, and we have created our own version to compare football players.
Similarity scores have a lot of possible uses, and we aren't the only football analysts who use them. Doug Drinen of the website footballguys.com [3] has his own system that is specific to comparing fantasy football performances. The major goal of our similarity scores, however, is to compare career progressions and try to determine when players have a higher chance of a breakout, a decline, or -- due to age or usage -- an injury. Therefore we not only compare not only numbers like attempts, yards, and touchdowns, but also age and experience. We often are looking not for players who had similar seasons, but for players who had similar two- or three-year spans in their careers.
Similarity scores have some important weaknesses. The method compares standard statistics like yards and attempts, which are of course subject to all kinds of biases from strength of schedule to quality of receiver corps. The database for player comparison begins in 1978, the year the 16-game season began and passing rules were liberalized (a reasonable starting point to measure the "modern" NFL). We also project statistics for 1982 and 1987 as if the strikes did not happen, although we cannot correct for players who crossed the 1987 picket line to play more than 12 games.
The method is subject to change in the future; we want to tweak it and perfect it. But here is how it works right now:
The defensive similarity scores system is a bit too complicated to explain fully. The coefficients are different for each position. The similarity scores measure a number of stats, basically split into three categories:
When measuring a two-year or three-year span, we use a mathematical method called the "harmonic mean [4]." The harmonic mean is higher when the items being compared are closer together. We measure the most recent year twice, then add either the previous year (for a two-year span) or the previous two years (for a three-year span). For example, over a two-year span, Player B with similarity scores of 900 and 900 will come out as more similar than Player A than Player C with similarity scores of 950 and 850.
Offensive players similarity scores are based on stats back to 1978. Defensive players similarity scores are based on stats back to 1998.
Links:
[1] http://www.baseball-reference.com/about/similarity.shtml
[2] http://www.baseballprospectus.com/glossary/index.php?search=pecota
[3] http://www.footballguys.com
[4] http://mathworld.wolfram.com/HarmonicMean.html