Writers of Pro Football Prospectus 2008

07 Aug 2012

Quarterback Development Patterns

When ESPN asked us to write a piece on the development of the Jets' passing game this season (which you will find here), it got me thinking of the question of when quarterbacks develop the most.

A few years ago, when he first came up with the Lewin Career Forecast, David Lewin look at our numbers for quarterbacks and found that quarterbacks improved the most between years 1-2 and years 3-4. I've quoted that research a few times over the last couple of years, and I wanted to give Sean McCormick those numbers for the ESPN piece. The problem is that David Lewin did that research back when we only had DVOA numbers for six or seven years of quarterbacks. Now we have 21 years of quarterbacks, so I went to go try to duplicate the research in order to make sure I got the same results.

I didn't.

First, I looked only at quarterbacks who threw at least 100 passes in two straight seasons. I grouped them by experience, then averaged their rise and fall in DVOA between the two seasons. The results were pretty dramatic. The average quarterback who has at least 100 passes in both his first and second seasons will improve by 12.8% DVOA in the second year. But every other year, the average change varies between around -5.5% and +3.5%, and variation looks like it is just noise. although it might be interesting that two of the bigger year-to-year drops happen in years 9 and 11, which is roughly around ages 30-32.

Of course, looking at change by looking at seasons with at least 100 passes will leave out a lot of important players -- the ones who decline so much that they can't get jobs anymore. So next, I looked at DYAR instead of DVOA, and looked at the change for every quarterback from one year to the next. Any time a quarterback played in one year but didn't play the next season, he got 0 DYAR for that second season. However, I did not count quarterbacks who were below replacement level in one year and then didn't play the next year. Otherwise, we would end up assuming that terrible quarterbacks get better when they don't play. That's different from assuming that a 30-year-old quarterback who drops out of the league (or becomes a backup who never sees the field) probably doesn't throw passes because he's dropped to replacement level (or below).

Once again, looking at things in this fashion made the transition from year one to year two stand out. Quarterbacks gained an average of 136 DYAR between those two seasons. The changes between years three and seven were close to zero. Around year eight, or roughly age 30, quarterbacks began to lose more and more DYAR each season, a lot of them from dropping out of the league altogether rather than playing both seasons but declining from one year to the next.

Overall, I would say these two views of quarterback development suggest that the only year when quarterback improvement is significantly likely is year two. Then you have a few years where guys might improve or decline unpredictibly. There isn't really anything special about the fourth season. And then around age 30 or 31, as we've seen in previous studies, quarterbacks begin to decline.

Posted by: Aaron Schatz on 07 Aug 2012

32 comments, Last at 10 Aug 2012, 7:02pm by johonny

Comments

1
by tuluse :: Tue, 08/07/2012 - 4:50pm

I'd really like to see a follow up for QBs who were good enough to last for a while.

Like what if you only look at QBs who have thrown at least 100 passes in 4 or more different seasons?

2
by Danish Denver-Fan :: Tue, 08/07/2012 - 5:45pm

This. Players where talent just wasn't there (Leaf, Russel etc.) may be pulluting the sample.

9
by Paddy Pat :: Wed, 08/08/2012 - 3:26am

You mean something more like dedication/intelligence? Leaf and Russell had plenty of talent...

27
by socccer101 :: Wed, 08/08/2012 - 2:30pm

This fails the eye test to a certain extent - quarterbacks (at least successful ones) clearly do not flatline in development after their second season. Perhaps the better question to ask is when do SUCCESSFUL quarterbacks develop, not when do quarterbacks with 100 min att develop. Isolate quarterbacks who have "successful" careers by some statistical measure (I'd suggest 3 seasons with a certain minimum DVOA) and then chart their development arc. After all, I imagine the most interesting/useful analysis to most is not the development of the Ryan Leafs (horrifyingly bad, to ugly, to find another career please) but rather the development of the Peyton Mannings (tough rookie season to hall of famer).

Put differently, I'd like to know when the quality QBs should emerge (and therefore when to give up on a prospect) more than when the draft bust will reach his (unfortunately low) ceiling.

30
by mavmoses07 :: Fri, 08/10/2012 - 1:00pm

Completely agree. Last year 44 QB's had over 100 pass attempts. Most of them are backups, journeymen, borderline starters, or average QB's. No one cares too much about their career path. Maybe non-elite QB's don't improve much after Year 2.

The real question should be how much do the elite QB's improve year by year, not every QB with min. 100 pass attempts. Maybe what separates the elite QB's from the rest is an improvement in the years beyond Year 2.

32
by johonny (not verified) :: Fri, 08/10/2012 - 7:02pm

I think you want to look at both sets. From the data you could speculate almost all QBs get better with a little NFL exposure. However the up tick in years 3-4 could be more dramatic for elite QBs while general population QBs basically declines or stays the same after a small exposure time. It would be helpful to look at different subsets to tell you when a team should cut bait on a QB or at least stop pretending they will turn into superman. Looking at this the answer seems to be 2 years, however it is possible years 3-4 is when elite QBs truly separate. One study can't tell you everything, although once again most QBs won't be elite QBs anyways:)

3
by Waverly :: Tue, 08/07/2012 - 5:56pm

Why does it appear that for the most part quarterbacks on average get worse every year after year 2, by both DVOA and DYAR?

It's not quite negative around years 5-6, but still, overall, I was struck by how both lines were mostly below zero.

8
by Aaron Brooks Go... :: Wed, 08/08/2012 - 12:59am

The sampling method for DYAR is almost destined to be negative in slope. It only includes positive DYAR QBs into the next year, includes them when they get hurt/benched, but then doesn't include them if they reappear -- because it requires 100 passes/+DYAR to be included. It's unclear if everyone is also getting the retirement hit -- that might be Favre there in the 18+ bin.

As an example, Stafford didn't get counted for 2011 (and his likely regression to non-record numbers will be perceived here as a drop-off -- indeed, his career will never have a bump because of how his passes per year worked), and Manning won't for 2012.

16
by Aaron Schatz :: Wed, 08/08/2012 - 10:08am

Actually, I didn't do the 100 passes minimum for DYAR. That's everybody. However, you are correct that I didn't include players who went from not playing to playing the next year (whether they were positive or negative).

19
by Aaron Brooks Go... :: Wed, 08/08/2012 - 10:50am

Just to clarify, I wasn't stating DYAR had the 100 pass minimum. '100 min/+DYAR' was referring to the DVOA and DYAR methods independently. That probably wasn't clear.

4
by Led :: Tue, 08/07/2012 - 6:04pm

I think there's too many variables for this to reveal much. Did the QB start as a rookie or sit on the bench for one or more years, how much experience did the QB have in college, were there personnel changes that made the team worse/better from year to year, what type of offense did the team run, were there changes to the system/coaching from year to year, etc.

The other thing is the possibility that QBs who show signs of improvement have a much better change of getting a second chance in year 2, while "established QBs" with 2 years starting experience or more get the benefit of the doubt and stay on the field more often even if they're not getting better. If true, that would skew the results up in year 2.

When it comes to predicting QB development, I'm with William Goldman: Nobody knows anything. I'd add "yet." This is a good start!

5
by BywaterBrat :: Tue, 08/07/2012 - 6:21pm

Wow- that is pretty striking.

I've got 2 questions off the top of my head (and I apologize for my lack of memory at this moment of the stat terminology I knew at some point):

- Since DYAR is a "gross" statistic these results would be even more striking given my assumed overall increase in DYAR (and even frequency of young QBs having DYAR) in the recent passing boom

- I'd also be curious to know the distribution within the noisy years- that is, whether all those players are fluctuating between the averages, some are decling steadily and others are increasing steadily, or those are really just averages and there are plenty of individual spikes in both directions

6
by Dred :: Tue, 08/07/2012 - 9:59pm

Does years started in college affect this at all?

7
by tally :: Tue, 08/07/2012 - 11:42pm

This chart really isn't indicative of when QBs improve--it's more when they get pulled for poor performance. With few exceptions, every QB is going to end his career on a downturn in production/efficiency--that's why they retire, get cut, etc.

Most of them are going to be pretty early in their careers, while a few manage to hang on for a lot longer. The former is going to significantly outweigh the latter, thus the general negative trend each year.

The exception is likely between year 1 and 2--a QB who performs poorly his first year is likely going to have a chance to redeem himself or develop a year or two before he gets cut. Thus, the first to second year improvement.

A more informative set of graphs would divide QBs into subsets--those whose careers lasted 18 years, those lasting 17 years, 16 years, etc. You'd more likely get some homogeneity as well as see growth and decline that isn't masked by those QBs with sharper growths and declines.

10
by Paddy Pat :: Wed, 08/08/2012 - 3:30am

I want to heartily second this suggestion. Grouping around similar career duration might be a much better means of demonstrating trends.

12
by Podge (not verified) :: Wed, 08/08/2012 - 5:09am

But then how useful is that as a predictor?

17
by tuluse :: Wed, 08/08/2012 - 10:22am

You can see how a QB compares to other QBs who played for 16+ years, etc.

18
by tuluse :: Wed, 08/08/2012 - 10:27am

Me too, this is a better idea that mine above.

25
by unverified (not verified) :: Wed, 08/08/2012 - 1:09pm

I agree that this version would be more informative than the data for all QB's, which is still interesting, shown here. Would subdividing by improving/worsening give some useful/predictive trends? So do QB's who improve in year 2 also improve in year 3? Do QB's who improve in years 2 and 3 then regress have any chance of recovering their previous form?

26
by Paddy Pat :: Wed, 08/08/2012 - 1:34pm

This is also an interesting idea. Obviously, this kind of data is very complex because it is influenced by many other factors. For example, Tom Brady's jump in 2007 (7th year) would be hard to understand outside of receiver context. Nevertheless, I think that grouping players with similar drops or improvements in year 3 might yield some interesting correlations for year 4, probably not predictive, but interesting nonetheless.

11
by Podge (not verified) :: Wed, 08/08/2012 - 5:08am

Sam Bradford looks at the graphs and weeps.

13
by Karl Cuba :: Wed, 08/08/2012 - 5:36am

I'd love to know what the variance is for the results after the second year because the results seem to suggest that quarterbacks are who they are after that.

14
by fyo :: Wed, 08/08/2012 - 7:01am

This. Well, the request for variance, anyway.

All the stats say is that the average improvement is close to zero after year 2 (after being "dragged down" by those who were benched, anyway).

If we want to predict a quarterback's improvement (or regression) after year 2, it might be interesting to see what the year-to-year correlation of the change is. That is, do quarterbacks who continue to get better in year 3 have a higher probability of also increasing in year 4? Or do most quarterbacks simply oscillate somewhere around the mean, meaning that any year 3 improvement is likely to be random / noise?

15
by Ryan D. :: Wed, 08/08/2012 - 8:24am

So, what are we expecting from Cam Newton this year?

20
by Aaron Brooks Go... :: Wed, 08/08/2012 - 10:52am

Based on his previous history, probably a transfer to a different school.

22
by Ryan D. :: Wed, 08/08/2012 - 11:57am

Zing!

21
by Eddo :: Wed, 08/08/2012 - 11:15am

Any chance you could publish graphs that show the actual DVOA and DYAR year-by-year, instead of changes? I feel like they'd be visually more effective (you could see the peak years).

23
by AnonymousA (not verified) :: Wed, 08/08/2012 - 12:24pm

Would love to see these charts broken down into category lines, e.g. a line for each age (QB started at 20, 21, 22, 23...), a line for each total team quality over the two years (27+ wins, 18-26 wins, 9-17 wins, 0-8 wins, or measure by some FO stat), lines broken out by CHANGE in team quality (i.e. how much is "QB improvement" related to "team improvement" -- this could be "QB gets better and powers team" or "defense gets better, QB required to do less")...

It seems pretty clear that year 2 is better, there's some room for variance in the the first 5-6 years, and then you're waiting for the decline, but it'd be awesome if we could see any major clues to who will improve and who won't.

24
by akn :: Wed, 08/08/2012 - 1:03pm

What you appear to be doing is calculating a quasi-autocorrelation function of QB performance. If that's the case, your comment about noise past the 2nd year is most likely correct. There are actually ways to estimate what the "true" autocorrelation length is, such as this one (see section on the calculation of m-hat).

I've actually used this approach in my own research. If I had the raw data, it would be fairly trivial to estimate the scale of the "autocorrelation."

28
by countertorque :: Wed, 08/08/2012 - 3:17pm

Thanks for the post Aaron. I appreciate your effort to dig into your data and post results.

29
by Dan :: Wed, 08/08/2012 - 7:02pm

Good work, Aaron. It's good to see you revisiting questions that you've covered before, and providing updates on the latest results whether or not they match your earlier results.

I think it would also be a good idea to share your data with people like akn who can use fancier statistics to tease even more information out of the data that you have.

31
by V (not verified) :: Fri, 08/10/2012 - 3:01pm

In case this thread hasn't pointed enough alternative analysis criteria, I would like to limit the data to only QB's who have been starters for at least 4-5 years (regardless of age or #yrs in NFL) and focus only on their years as the top dog. As pointed out, those are the guys of interest.

The superstars shown individually would be of interest also.

Another thought for sometime: Top players returning from season-ending injuries (offseason major surgery?). I suspect Brady's return in 2009 was abnormally good. Carson Palmer has never looked the same. Might be fun review while looking forward to this year with Vick and Peyton returning, Stafford ...