FO Basics: Scouting

by Mike Tanier and Doug Farrar

Over the next couple weeks, we're going to run a series of articles we're calling FO Basics. We get a lot of questions about our work, but there are also a lot of readers who don't ask questions. We hope this series will help answer some questions and clarify some confusing things for even those readers who don't respond on the message boards.

Please click here for the other articles in the "FO Basics" series:

• August 30: Where our stats come from, and the difference between charting stats and play-by-play stats.
• August 31: A summary of research from our first seven years.
• September 1: Our college stats, how they differ from our NFL stats and from each other.
• September 6: The importance (and limitations) of watching games on tape.
• September 7: Regression towards the mean -- what it means, and how we use it.

* * * * * * *

Mike Tanier: Scouting and statistics go hand-in-hand. While some fans perceive a great gulf between statistical analysis and scouting, the fact is that good scouting informs good statistics, and vice versa. Everyone involved in serious player or team evaluation uses both.

Our Football Outsiders Game Charting Project contains data culled from watching every game of every season, but it's important to note (as we have a few times already in this series) that our game charters are not scouts, and we try to eliminate the subjective elements from the data they compile. Game charters track objective phenomena: formations, types of plays, and so on. There's some subjective stuff -- two people may disagree on what constitutes a "dropped pass," and figuring out when coverage should be marked "Hole in Zone" can be difficult -- but we do our best to minimize value judgments. Game Charters are encouraged to write "awesome block by Jahri Evans" at the end of the spreadsheet if they see one, but we don't tally the results and grade Evans on his awesome blocks, because that's far too subjective and arbitrary to be a "statistic."

Each of the FO writers watches dozens of games per year, and we all "scout" in the general sense that we make subjective judgments about various players. Doug Farrar scouts very intensively for his Cover-3 column, and I break down play diagrams in my Walkthrough column. Again, none of us are strictly "scouts," but we are informed, experienced observers who know what we're looking for. We've talked to players, coaches, and other writers, we've studied playbooks and coaching manuals, and we watch games in slow-mo and on freeze frame to learn as much as we can.

We all have one major limitation when scouting -- we use television tape, not game film. In game film, every play is shown from two angles: a sideline view and a behind-the-center view. You know what television tape looks like: you get a cheerleader shot, then a close-up of Eli Manning's face just until the snap, then you switch to the sideline camera, which usually doesn't show all 22 players. If Manning throws a 40-yard touchdown, you get replays from 10 angles, meaning you can really scout the play. If the play is a three-yard run, you're stuck with the one angle.

The reality is that the television angle only tells you so much. There are things you can see, things you can sort of see, and things you just cannot see. Most dangerous of all are the things you can talk yourself into seeing. Think of the Jim Mora, "You think you know, but you don't" rant. It's easy to misunderstand something you are watching and make some off-the-wall judgment about a play or player. And the more you study the game, the more aware you become of how wrong you can be.

Quarterbacks and receivers

Tanier: Doug Farrar and I are here to discuss what can and cannot be scouted from television tape, starting with the quarterbacks.

Doug Farrar: Before we get into this, I'd like to thank whoever invented the concept of keeping the shot on the quarterback's eyes until just before the snap on a disturbingly high number of plays, so that motion and snap formations are sometimes missed altogether. Nice going, you putz.

That said, I don't generally have any problem seeing what quarterbacks are doing. If anything, I'd love to see a wider angle before the quarterbacks release the ball on pass plays, because you frequently have to guess on parts of routes and route combinations when the camera doesn't catch up. I want to see a receiver coming back to the ball on the breakout bootleg. I want to catch all those crazy combos Sean Payton calls. Generally speaking, when receivers run downfield about 10 yards, they get lost from the shot, and you're left with, "Well, how did he round that off? Was that a hard cut? Did he zig when he should have zagged?" It should be fairly easy to zoom out and catch those routes. Even if the NFL doesn't want the general public to see the all-22 film, just being able to see those routes and combos would help our analysis immeasurably. You'll get that stuff on replay at times, but other times, you're just left with a guess. And we don't like to guess.

Mike, I'd be interested on your thoughts on this since the Eagles are one of the more interesting teams when it comes to crossing combos (at least, I think they are).

Tanier: The real frustration with scouting the Eagles is all of the deep routes they run. Like you said: Once the player is about 10 yards downfield, he's invisible. We have no idea what a receiver is doing unless he's targeted for a pass. If the quarterback scrambles, gets sacked, or checks down to the fullback, we don't know how good the deep coverage really was unless we get one of those "everybody was covered" montages you sometimes see after obvious coverage sacks. So you can't really decide if a dump-off was a wise decision.

It's important to point out here that no matter what film you look at, you still don't know the play call or the progression of reads. Sometimes you can guess, because some plays are so old and well known that they are similar in every NFL system (and common at the college and prep level). But even when we have 20 replay angles, you have to be careful to recognize when the speculation sets in.

Running backs and run blocking

Farrar: What are your thoughts on running backs? Are you able to see rushing plays develop the way you'd like, and are the blocking schemes shown in a way that doesn't hose up your writing?

Tanier: Running backs are easy to scout, and I think even the casual fan gets a good idea of a back's capabilities from watching game tape. Yet blocking schemes are almost impossible to evaluate using television tape.

Here's an experiment for readers to try. Tape a game during Week 1, load up an ordinary three-yard run, and watch the interior linemen. It's just a mass of bodies, right? You might be able to spot a double team. Maybe you see one guy blown backward by a defender. But mostly, it's a jumble of fat guys.

Now, you can glean some information from the jumble. You can put a frowny face next to the guy blown backwards and a smiley next to someone who shoves his defender back into the tunnel. But you have to be careful, because you don't know the assignments, and you can't see any subtleties. Take that double-team, for example. Maybe the guard was supposed to peel off and block the linebacker who made the tackle, but couldn't get off the block. He made a major mistake we don't know about. Maybe the backside tackle had a tough assignment like a reach block in which he steps inside and hooks the three-technique defender. Maybe he does a great job. It's invisible. And even if you can see it, you have to know what a reach block is and be able to tell a good one from a bad one. Soon, you push beyond the Farrar-Tanier comfort level, which is a bad place to be unless you are a coach, scout, or one of the NFL Matchup guys.

Pass blocking and pass rushing

Tanier: Pass blocking is a little easier, as is evaluating pass rushers. Do you agree?

Farrar: Well, sometimes. As you said, not knowing assignments can lead to confusion. Doug Free of the Cowboys gave me a very good example of that when he gave up a strip sack to Jared Allen soon after he replaced Flozell Adams in the second quarter of Dallas' division round loss to the Vikings. On the play, he down-blocked tackle Jimmy Kennedy in conjunction with left guard Kyle Kosier. Now, obviously, no coach in his right mind is going to draw up a line call that has a double-team on a reserve tackle while Jared Allen basically roams free to kill the quarterback. And since Free was new in the game, we can assume that he screwed up. But we don't really know.

As you intimated, blown blocks are the easiest to spot if they're executed in space and assignment issues aren't a factor. A guy sets you back and blows you up, you were simply beaten on that play. But inline pass blocking can be difficult. We sometimes blame a left tackle when the left guard wasn't in place for slide protection or the tight end failed to check the blitz before he ran his seam route.

I write about offensive line play a lot, and that involves slow-mo, as it should. I want to get to know the linemen I'm writing about. I want to understand the schemes as much as I can. Example: The term "zone blocking" is like the term "West Coast Offense," it's a catchall term for a concept with more than 31 flavors. The Jets implement a lot of run action and have powerful linemen, so the general assumption is that they're man-blocking all the time because they don't have little guys running around at the second level. But Leon Washington recently told me that 70 percent of the team's successful run plays in 2009 came from zone blocking. Alex Gibbs sat down with Bill Callahan before the season and gave him a few ideas about how to put that together. Other teams like the Falcons and Titans will run some "power zone" stuff. So I want to see whether the left tackle is a good fit for his scheme. Is the center light in the pants, and does he require guard chips against 3-4 defenses? Do the guards pull or chip to head upfield? I think you have to have a thread of schematic identity before you can judge any offensive lineman -- there are cases where a guy is asked to beef up and subsequently loses the edge that made him special (hello, Eric Steinbach). Some players don't fit their schemes and are unfairly criticized for it. Then, you have your Alex Barron and Levi Brown-types, guys whose sub-par play is scheme-transcendent.

Pass rushers are easy to watch. It's a flashy position and they're often tied to the quarterback, so no problem there. You might miss a neat stunt or loop once in a while, but the front four (or three, or five) are usually in the shot when they need to be. You can see inline power, hand movement, and how low a guy gets coming off the edge.

Defensive backs

Farrar: Let's move to what will undoubtedly be the most controversial part of this piece -- defensive backs in general, and safeties in particular. There are those who believe that you can see what safeties are doing on TV tape. Others simply accept a much larger margin for error instead of saying that they can't see what they can't see. I would love to write about safeties in deep coverage, but I usually can't. I will write about cornerbacks once in a while, but I have to throw caveats around like crazy. Thoughts?

Tanier: Let's pretend we could see the safeties on screen on every play. As it stands, we can only see both safeties at the snap about 50 percent of the time, but let's pretend. So we see a free safety in the deep middle of the field. What's his assignment? Is it Cover-1? Cover-3? Does he have to cover a deep quarter, third, or half? Heck, he could be in man coverage or double coverage. We have no way of knowing. The safety could be "burned deep" when in fact he's racing over to a zone that wasn't his responsibility. At least with blocking assignments there's only a few yards of space to worry about, and the camera is on the player. With safeties, you are trying to guess their responsibilities, and often you're doing it based on the one or two seconds when he's clearly on the screen.

You know, sometimes in my play diagrams I just wind up throwing the safety in, on a guess. On a running play, I may not see him until he creeps onto the edge of the screen after the tackle. I only do that on plays where his position isn't really relevant to what I'm talking about: If it's an Adrian Peterson handoff, getting the safety's position wrong doesn't make much of a difference. That just shows how hard it is to scout them.

Farrar: One thing I noticed through the 2010 preseason after going back and watching something from 2009 for another project is that the networks seem to be going to a wider view on a lot of overhead shots -- maybe you see 20 players and slightly longer route developments on plays as opposed to 18 and shorter cutoffs. FOX is definitely doing this on its HD telecasts (Joe Buck mentioned it last week in a rare moment when I was actually paying attention to what he was saying before falling asleep). They're still wasting about five yards of view behind where any offensive play would develop, but maybe they can move that a bit and actually get the All-22 in there. It seems that we will get better views in HD telecasts this season, which is at least a step in the right direction.

Sometimes with safeties, I base it on team tendencies. Often, I have to guess just as you mentioned. Some teams run their safeties parallel on Cover-2 stuff, others move one or the other a little bit. When there's a wider shot on a developing play, I have to assume that the deep safety or safeties I now see rolling back started at point X when I really have no clue. And anyone who tells you that they're getting a good, even semi-accurate read on safety positioning from any kind of TV shots -- even the new stuff -- is absolutely full of it. Let's just get that out of the way right now.

I recently did a Cover-3 on Packers rookie safety Morgan Burnett, and ESPN did do a really nice job of showing full-field replays, which allowed me to detail how Burnett cheated up from two-deep on a Charles Woodson corner blitz, handed Indy's Anthony Gonzalez off to the zone, and jumped a quick Peyton Manning pass to Pierre Garcon for an interception. Again, these are steps in the right direction, but there's still a way to go.

Marks of quality scouting

Farrar: So, where are we with this? As football nerds, what do we need to impart about what we see and about what cannot be seen in a way that doesn't have "B.S." written all over it?

Tanier: There are two marks of quality that fans and readers should look for when they are judging the value of scouting information, assuming isn't coming right from the mouth of Mike Mayock or some other unassailable source.

1. Careful amateur scouts work collegially. I've been to NFL Films enough times to see Ron Jaworski work in tandem with Greg Cosell, his spotters, and a network of coaches and experts he can call upon to get his information as accurate as possible. If he does it, we have to do it. That's why we talk to coaches and players when possible, not to mention lots of other writers, and why we spend as much time studying as we do watching tape and reporting. I inherently distrust the work of somebody working out of a bunker somewhere in the world, no matter how diligent they may be. It's too easy to fool yourself into seeing things you can't, or to gloss over your own lack of real knowledge in the rush to get something to the marketplace.

2. Be wary of those who try to objectify the subjective. If a guy tells me he watched 16 Jets games and saw Nick Mangold make a bunch of great pull blocks, I will listen. If he says he saw him make 22 great pull blocks, well, counting is nice, though I would question whether the guy knows a pull block from a fold block and what "great" means. If he tells me Mangold has a Pull Block Index of 3.2 or something, I smile politely and leave. It's not that the guy isn't trying or doesn't know a lot about football, it's that he's buried too many subjective opinions in a "statistic" that then pretends to be objective to the tenth decimal. I usually take that as a hint that the guy doesn't know enough about football, scouting, or statistics, and he is either unaware of his ignorance or deliberately trying to hide it.

That's not what we do. Our objective stats are objective, our subjective opinions are not processed into loopy "ratings," and while we use one to inform the other, we are careful every time we dip the chocolate in the peanut butter.

Farrar: Another important thing to understand is that no matter who you're listening to, we all have our own biases. I couldn't possibly have more respect for Mike Mayock, but I've noticed that he favors big-armed quarterbacks to the exclusion of other types who play the position and have the tools for NFL success despite the fact that their 9-route ball can't crack an engine block. That said, I'd rather listen to him talk football than just about anybody. Cosell has been very generous with his time with me, and you're absolutely correct about the importance of interaction. I have a couple of former scouts I talk to, and other friends who could probably be scouts under the right circumstances. I really appreciate having people who know a hell of a lot more than I do who will talk to me about the game and don't mind my more ridiculous questions and observations.

More than anything, the increased opportunity to talk to coaches and players has been a godsend. If you can talk to Louis Murphy about his first NFL touchdowns and have him tell you exactly why two members of the Chargers' defensive backfield got crossed up, you realize just how much you'll never know watching the game from afar.

We're veering into another lane to a point with the player value stuff, but I'm not of the opinion that Win Shares for NFL players are possible -- or necessary. I don't want Nick Mangold's "value" reduced to one decimal point. Not just because I don't believe it can be done within the dynamics of this particular sport, but because of the extent to which such a method precludes the kind of dialogue we're talking about. And this is not what football analysis is about to me. To me, the ideal analysis of the game mixes scouting and stats in a holistic manner. "Here's what I saw, or here's what a trusted source saw. Now let's see how whether the numbers confirm or refute what we think happened". (Or, "Here's an interesting stat, I wonder what is happening with the play calling to create that stat.")

Step 1 is getting better and more comprehensive angles to see these things. Step 2 is sharpening our own knives -- reading and learning and better understanding what we see and how to explain it. Putting it all together in a way that makes sense is the hard part. There is a growing cottage industry of guys who watch film in conjunction with stat lines and post their observations, and I've seen everything from the ridiculous to the sublime. Watching X hours of film every week isn't impressive in and of itself. The whole point is to learn what you're missing and misinterpreting as quickly as possible so you're not wasting time.