Quarterback Similarity Ratings

Guest Column by Dan Morse

Jameis Winston in 2019 became the first quarterback to throw 30 interceptions in a season since Vinny Testaverde tossed 35 in 1988. That includes seven pick-sixes; of the five players to throw 30 interceptions since the AFL-NFL merger in 1970, only Winston was unlucky enough to have more than two of those picks returned for touchdowns. But Winston is also the only quarterback on that short list to surpass 30 touchdown passes (he threw 33), or to throw for more than 3,500 yards (he finished with 5,106). And he did it in the final year of his contract.

What does this all mean? It means Tampa Bay is in a much more confounding spot in regards to their quarterback situation than Winston's 30 interceptions might indicate.

Do a quick search of "Jameis Peyton" on twitter and you'll find an inordinate amount of Tweets comparing Winston to soon-to-be Hall of Famer Peyton Manning, like this one:

WOW

WINSTON '19* PEYTON '01
25 AGE 25
26 PASS TD 26
4,115 PASS YDS 4,131
23 INT 23
5 PICK-6’s 6
6 WINS 6
84.9 RATING 84.1
61.3 COMP PCT 62.7

*thru 13 games pic.twitter.com/PliPyftUpp

— CBS Sports HQ (@CBSSportsHQ) December 9, 2019

Even looking at Manning's first five years compared to Winston's first five yield some fun results:

Winston thru 5 years 88 ints
P Manning thru 5 years 100 ints

Make of it what u want

— Booger (@ESPNBooger) December 29, 2019

Thanks Boog.

With Winston set to become a free agent, and with the quarterback market larger than ever, the idea of how much money a team should pay him becomes one of the most interesting topics of this offseason. Granted, if you say "I don't want a guy who turns the ball over that much" I totally get it, but Winston has added enough value to make some teams think about it.

With that in mind, along with the varying comparisons to the great quarterbacks of yesteryear, I sought to create a tool where we could compare some advanced quarterback metrics visually to help us get a better understanding of how these players actually match up beyond your basic volume stats typically displayed on SportsCenter.

Remember that note about Winston having seven interceptions returned for touchdowns this year? That actually broke the old record of six, set by Peyton Manning in 2001. The similarities between the two quarterbacks on the outside are striking, but looking at these deeper numbers adds another layer to the argument. Winston topped Manning in EPA/play and EPA/dropback, and likely matched or exceeded Manning in Average Depth of Target (ADoT) and Completion Percentage Over Expected (CPOE) as well, though ADoT and CPOE have to be estimated for seasons prior to 2006 (more on this later).

Using these metrics, I attempted to mathematically classify just how similar Winston's 2019 was to Manning's 2001, and took it further to compare every quarterback-season from 1999 to 2019 using a similarity score.

There have been various approaches to this in the past, such as this Football Outsiders attempt based on the Bill James baseball similarity score. I approached similarity in a fashion similar to a k-nearest neighbors analysis -- that is, I selected n metrics I wanted to use in the comparisons and then calculated the Euclidean distance between each point if they were plotted in n-dimensional space.

That was a mouthful, and I apologize. To simplify, let's say we wanted to perform this analysis and compare quarterbacks with just two variables: EPA/dropback, and ADoT. We'll use Winston and Manning again as an example. We can plot them together and use the Pythagorean Theorem (remember that one?) to find the distance between the two points as so:

Quarterbacks with seasons more similar to each other (in terms of our chosen statistics) will be close together, while quarterbacks with far different seasons will show up farther apart.

As it turns out, the Pythagorean Theorem works with even more dimensions added, so while we can't visualize, say, a seven-dimensional plot in our minds, we can still use this equation to find the distance between two points in that space. That's the formula I utilized for this similarity score.

The metrics I selected to compare were as follows:

• EPA/dropback
• ADoT
• CPOE
• First down rate
• Turnover rate
• Sack rate
• Total rushing EPA

This gives us a pretty good variety of quarterback styles while hitting most of the major qualities we look for in quarterback evaluation.

As I mentioned above, we can't get exact values for ADoT and CPOE for years prior to 2006 because air yards were not publicly tracked back then. In order to include them in this analysis, we need to find a way to estimate them. A linear model using the inputs of yards per completion and completion percentage gives us a pretty decent estimate of both ADoT and CPOE.

Using the 2006-2019 data to train this model, our CPOE estimate is within ± 0.72 and the ADoT estimate is within ± 1.95 yards. Keep in mind that any matches prior to 2006 will include these estimated values.

Before running the distance formula, we need to normalize these metrics so they are all on the same scale. Otherwise things like ADoT, which ranges from about 6 to 12, will have a much bigger impact on the distance than sack rate, which only has a range of 0 to 1. Once we normalize the data and get each metric on a scale of 0 to 1, the final distance formula becomes:

When we're using that formula, the player-seasons with the lowest distance between them are the most similar. An identical player-season would have a distance of 0, while the complete opposite seasons would have a distance of √7. To make things more intuitive, I adjusted the distance to a scale of 0 to 100 using the formula:

Two identical seasons will now give you a similarity rating of 100, while completely opposite seasons will result in a 0.

There were 611 quarterback-seasons all compared to each other in this manner. Using this calculation, we can plot how often each season scored each similarity rating and get an idea of how unusual a given season was.

The curves on the left have, on average, lower similarity scores than the curves on the right, indicating that those quarterback-seasons were more unusual than the others. On the right, we find quarterback-seasons that frequently match with a majority of others. And in the middle there is one significant spike.

Cam Newton's 2011 is that peak. That implies that his statistics that year represent the centermost season in our dataset. Mark Brunell's 2005 had the highest average similarity score, indicating he was at the origin of the highest density cluster of quarterback-seasons. On the left we have Andrew Walter's 2006 season, the most unique season since 1999, but not in a good way.

Lamar Jackson's 2019 is considered to be something we have never really seen before, and this data somewhat backs that up. Excluding the seasons that were unique in how bad they were (Walter 2006, Jimmy Clausen 2010, David Carr 2002, to name a few) Jackson's 2019 was more singular than any season since early Michael Vick seasons. But per this methodology, it wasn't really more or less unusual than Vick's 2004-2006 run.

If we classify each player-season by its mean similarity rating (and ignore the uniquely bad quarterback-seasons) Lamar Jackson's 2019 falls in at the fourth-most unusual season, trailing Peyton Manning's 2004, Michael Vick's 2005, and Drew Brees' 2018. Brees' 2018 stands out in large part due to his EPA/dropback and CPOE both landing in the top 10% of our samples despite having a bottom-25% ADoT.

Moving back to where this all started, let's take a look at the best comparisons for Jameis Winston's 2019 season.

Peyton Manning's 2001 is indeed a decent comparison, but not as good as the Bay Area debut of fellow former first-overall draft pick Carson Palmer. Palmer threw 16 picks in just nine starts with the Oakland Raiders in 2011 while also setting a career-high in yards per attempt, very similar to what Winston did this season.

What does this mean for the Bucs, the team that has to decide what to do with their soon-to-be free agent quarterback? Palmer signed a 4-year, $43-million ($10.75M APY) deal in 2011. At that time, the highest APY among quarterbacks was $18 million. That very roughly translates to a $21-million APY deal today, much less than the $27-million franchise tag that will likely be the only way Tampa Bay keeps Winston around in 2020.

On the other hand, Palmer went on to have an MVP-caliber season just four years later, and 2001 Peyton put up one of the best quarterback seasons of all time just three years later. Ben Roethlisberger, another comp for Winston, was just in the beginning of his 16-year tenure as the unquestioned starter in Pittsburgh. There are a lot of reasons on this list that point to the idea that perhaps Winston really does have his best years ahead of him, and it might not be a bad idea for the Bucs to bet on it.

What's Next?

Taking a look at the other potential free-agent quarterbacks, such as Dak Prescott and Ryan Tannehill, and their comparables could help give insight into what those players are really worth and help us better understand just what kinds of seasons they had in 2019.

As far as the similarity rating goes, hopefully we can find a way to era-adjust some of these numbers, because football in 2004 was different than football in 2019. We spent a lot of time comparing Winston to that timeframe, but what we lacked there was the fact that the rules have changed over time to benefit quarterbacks, and we should expect more recent seasons to look better than seasons from 15 or 20 years ago.

Another future project is to create similarity ratings for other positions, such as wide receiver. There are issues with dividing credit between the receiver and the quarterback, but incorporating Football Outsiders' DYAR along with other metrics like yards per route run could yield some interesting results.

As for now, feel free to check out every quarterback's highest rated comparison here, and if you've got any requests or suggestions, leave them in the comments. New ideas are always welcome.

Dan Morse spends most of his free time studying football and hockey from a statistical perspective. You can find his other work at CowboysWire.com and BeastPode.com or on Twitter @danmorse_ ("danmorse" then underscore).