Brian Fremeau is back for his 10th season of college football advanced stats at Football Outsiders, with a lesson on average teams (like Arkansas State) and updated FEI projections for 2016.
05 Mar 2014
by Matt Waldman
After spending an insane amount of time during the last decade studying players, talking with scouts, and paying attention to history, I have learned three things about evaluating football talent:
These are my personal lessons. No one shared these three points as teachable nuggets from the book of scouting. The last two insights are unintended consequences of professionals making opposite statements.
After 10 years of studying football games, I have gained enough experience to see that I’m not an expert. As the great poet Philip Levine wrote, I’ve "begun to separate the dark from the dark."
Today, I’m sharing these degrees of darkness about scouting quarterbacks. The hope is that separating the dark from the dark may one day provide a process that is a more reliable way to find the light.
During one of our frequent phone conversations, Footballguys.com co-owner Sigmund Bloom and I concluded that the simplest way to describe good quarterbacking is to compare it to another job. Cooks and musicians offer good parallels, but the best is that of a skilled craftsman.
I used to build sets at a theater. I learned how to use a wide variety of tools. I even gained some welding experience.
Give me directions and materials and a garage full of tools and I can assemble something bought at a store after I’ve taken it apart at least once. But I’m not the guy you want to help you with a home improvement project or a repair. Unless it’s the simplest of tasks, I’d be pulled from the job within an hour.
On the other hand, give my wife Alicia a small toolbox with half the tools and she'll not only have the job completed with time to spare, she'll also have spotted and addressed two other problems around your house that you didn't know about. She didn’t start working on houses until her early 30s, but within three years she owned her own remodeling company and did everything but electric and plumbing.
You need tools to do a job, but nuance to do the job well. I had all the tools, but none of the nuance. Alicia had half the tools and a ton of nuance.
Good quarterbacking is craftsmanship. There are a basic minimum of tools (details) to complete the job: height, weight, speed, arm strength, accuracy, etc. However the craftsman integrates the tools, his knowledge, and his experience to execute at the highest level of performance.
Skills integration is nuance. Pocket presence is a collection of details integrated in variety of ways to achieve a desired result:
Each of these details can differ from one play to the next. It’s not intangible; it’s often too multi-layered and complex to capture with data. Touch, placement, and reading defenses are also examples of nuance.
A quarterback can have a collection of tools to rival the best to ever set foot in a stadium, but lack the nuance to execute at a high level. Blaine Gabbert, JaMarcus Russell, Kyle Boller, and J.P. Losman all have the tools, but lack the nuance.
Joe Montana, Rich Gannon, and Brad Johnson all lacked specific tools NFL decision-makers thought were necessary for a franchise quarterback. But these three players possessed the craftsmanship to apply what tools that they to create successful careers.
No hammer? Give me a 5/16 wrench and carabiner...
No flathead screwdriver? A pen with a metal barrel will do the trick...
Need a tension wrench? A paper clip will do...
For all his flaws, Tim Tebow is an extreme example of a player with a small toolbox of skills who possessed enough nuance to lead his team to the playoffs.
In contrast, the negative fixation on Russell Wilson’s height caused almost everyone to undervalue the nuances of the Super Bowl-winning quarterback’s game. The same fixation can happen with positive details while glossing over a lack of nuance.
He's tall, he has a big arm, he has great athleticism, and he comports himself well in public. We can coach him up. Give him some experience and he'll develop.
This leads us to experience.
Experience is not what happens to you; it's what you do with what happens to you.
It's as easy to gloss over a negative in favor of multiple positives as it is to discount craftsmanship because of a limited toolbox. I passed the Leinart Test, but failed the Gabbert Exam. When it came to the Jaguars’ quarterback, I fixated on a set of positives that I saw in one game and glossed over the serious lack of nuance that plagued him in others.
In hindsight, Gabbert is an example of a player with an identifiable lack of nuance that should have been regarded as a fatal flaw and knockout factor on an evaluation (more on this later). However, many NFL decision-makers –- owners, general managers, coaches, and scouts –- also lack nuance when it comes to scouting.
NFL decision-makers may have experience, but it doesn’t mean they possess the right kind of experience. Quality trumps quantity every time. The only thing good about duration of experience is it provides a greater chance to earn richness of experience.
Even then, there are no guarantees.
Spend 20 years with any organization –- a garage, an accounting firm, a university, or an NFL team –- and it’s possible to learn more about playing politics than how to perform the intended function of the job at the highest possible level.
These organizations fear experience. Their message is to keep your head down and your mouth shut. Don’t question the process. Don’t take risks if you want to feed your family.
Cover your ass.
Spend half that time in an organization that embraces experience and it’s a different world. Question everything. Think like an individual, but learn to work as a team. Take ownership.
The best organizations don’t paralyze their employees with the fear of mistakes. They know how to work through them. They spend more time teaching and less time punishing.
The experience-phobic organizations spend more resources cycling through employees and continue to repeat the errors they could have avoided. Even when they succeed, they often misidentify where the credit should go and cannot replicate the result.
I’m sharing this from the perspective of an evaluator who has butchered evaluations and a manager who has hired the wrong people and missed opportunities for development. But as the Hindu proverb states, "No physician is really good before he has killed one or two patients."
I empathize with the NFL’s plight. The current decision-making process teams use to pick a first-round quarterback is a resounding failure.
When Mike Tirico asked Bill Parcells on ESPN’s first NFL Draft Confidential special why teams continue to take quarterbacks in the first round despite the low probability for success, the former coach and general manager said that if a team doesn’t get one in the first round then it’s even harder to find one later.
From a data standpoint this is true, but examine the context of how teams apply the draft to its roster. On most teams, early-round picks automatically earn more practice reps and have higher expectations to play.
Some teams even force players into action who are not ready for the transition. In contrast, late-round quarterbacks earn few, if any, reps in practice. They are emergency players.
It's big news when a quarterback drafted outside the first round earns even a split of camp reps. The end result is that teams reinforce this flat earth theory that NFL management espouses - despite the fact there’s growing evidence to the contrary.
Super Bowl quarterbacks Tom Brady, Drew Brees, Kurt Warner, Joe Montana, Brad Johnson, Brett Favre, and Russell Wilson have earned 10 championship rings in the past 25 years over the likes of opposing passers Boomer Esiason, Rich Gannon, Matt Hasselbeck, Neil O’Donnell, Stan Humphries, and Jake Delhomme.
None of these championship game quarterbacks were first-round picks. However, six of those players earned the MVP award 10 times in a 25-year period. Add Esaison’s MVP year in 1988 and the count rises to 11 times in 26 years.
When the probability of success for a first-round quarterback is less than a coin-flip, it is time to change the process behind the decisions and focus solely on the final outcome. Brian Billick’s first meaningful statement in his "How to Draft a QB" article is that "nobody knows anything."
Listen to former general managers and scouts talk about draft day and it’s clear that too many decisions to select quarterbacks are rooted in fear, pressure, and an admitted lack of knowledge. I may not possess the light on how to fix this, but I can begin to separate the dark from the dark.
Teams should not only have an idea of what makes a quarterback good, but what prevents a quarterback from ever achieving his potential and stick to those guidelines. They should show more restraint in the first round when it comes to overrating quarterbacks with numerous tools and discernible lack of nuance.
At the same time, they need to formulate processes that identify and value the types of nuance that elevate players beyond their details. As I said before, there are certain details that are "must haves" but if a team is tallying good details on a player without examining the nuance he displays with them then they will continually pick quarterbacks where the whole never equals the sum of their parts.
If a team selects Derek Carr or Jimmy Garoppolo within the first 50 picks of this draft, it’s a good sign they are in this rut. This is especially true if the pick is Garoppolo.
The Eastern Illinois quarterback possesses a lot of the details that NFL scouts and general managers like, but he lacks the nuance to succeed long-term as a starter.
Details to like:
List these individual components on a scouting report and Garoppolo looks like a strong prospect on paper. Even more impressive is the quarterback putting these skills on display in an all-star game practice, the NFL Combine, and a pro day workout.
Garoppolo has been on my short list as a player whose stock will rise as workouts get underway. The anonymous scouts and general managers who liked Garoppolo and Carr before these workouts in another controlled environment will gush over the details.
Controlled environments downplay the importance of nuance. The pocket is not a controlled environment; quarterbacking-by-numbers doesn't win there.
Garoppolo’s pocket presence is similar to Gabbert’s. The Eastern Illinois starter has a bad habit of altering his body position during his release as if an opponent is within a step of delivering a hit that will alter the accuracy of the throw when there isn't a defender within 3-4 yards of him.
Moreover, Garoppolo is more likely to duck his shoulders and drop his eyes from his receivers to brace himself for a hit in situations were top prospects climb the pocket, keep their eyes on their receivers, and maintain a good throwing position. When a quarterback's first inclination is to freeze, brace for a hit, or turn tail from pressure and run well before the pressure arrives, it's a red flag.
The problem inherent with having a high grade on Garoppolo is that teams are rewarding quarterbacks for the wrong things. Displaying execution that the average NFL quarterback does successfully during 35-40 pass attempts in a game is nothing special, but when teams add all the details together like bean counters it appears far more impressive in their reports. As my buddy Bloom says, "It's like the SAT awarding 200 points for writing your name."
Teams make the mistake of using first-round picks on quarterbacks who display a large sum of details, but lack the high concentration of nuance. It's like awarding a middle manager's salary and status to a candidate who has proven he can do the job of entry-level employee on a consistent basis.
Many will argue that's how it's supposed to be: Do a good job and get promoted. This is false.
A good decision to promote an employee is based on that candidate not only excelling in his or her current job, but also displaying the skills and nuance to perform the new job. An organization that makes a decision without seeing how these skills project to the new role are relying too much on luck.
Some believe they do this, but their track record proves otherwise. Others have done it well, but they questioned the right process yielding the wrong result and instead of refining the process, they toss it aside.
The NFL hasn't collectively learned how to identify what makes a good NFL starting quarterback. I'm not speaking from a position of authority, but I am speaking from experience -- especially as the term is defined in the quotes above.
Base on this experience, I believe that the majority of NFL teams only know how to find quarterbacks who hit the target. When it comes to finding the player who hits the bull's eye on those 3-5 snaps in every game that decide the outcome, they’re lost.
The unintended consequence is that the NFL awards bull's eye money, status, and expectations to players who they know hit the target, but don't recognize how to spot what makes a quarterback special unless he fits one highly specific prototype.
Thus far, drafting that prototype in the first round works less than 50 percent of the time.
Matt Waldman authors the Rookie Scouting Portfolio. Pre-order the RSP now for it's publication on April 1. If you’re a fantasy owner the 56-page Post-Draft Add-on comes with the 2012 – 2014 RSPs at no additional charge and available for download within a week after the NFL Draft. Best of all, 10 percent of every sale is donated to Darkness to Light to combat sexual abuse. You can purchase past editions of the Rookie Scouting Portfolio (2006-2013) for just $9.95 apiece.
26 comments, Last at 08 Mar 2014, 7:58pm by Theo