Study Hall Book Excerpt: Coaches vs. Stats

Bill Connelly

Is Football Outsiders Almanac 2013 not enough for you? Then I have good news: We've got a second book for college football fans. My first solo book is called Study Hall: College Football, Its Stats and Its Stories. It is available through Createspace and Amazon. You can so buy it as a PDF through the FO store for $8.99. Learn more about it here.

Study Hall is, above all else, a conversation. I didn't want to write a text book, and I didn't want to simply rehash the Varsity Numbers archive; my aim was to talk to as many smart football people as possible -- coaches, writers, nerds, a couple of former players -- about the topics I most enjoy about college football. The way things take shape, the first third of the book is about current events (why people love the sport, what threats it faces moving forward), the middle third is about stats (how coaches and fans use them and should use them, how we should re-imagine the box score), and the final third is full of random topics and tactics (the importance of finishing drives, underdog tactics, a couple of charting pieces).

Basically, if you are a newbie to the football stat world, Study Hall can serve as your freshman initiation seminar. But if you have read everything I have ever written (and lord help you if you have), you will still find value in the interviews and in the wealth of original content.

In the end, Study Hall turned out how I wanted it to, and it made me enjoy my job more than I already did. I couldn't be happier with it. Here's an excerpt from Chapter 6, "Coaches and Stats." Enjoy, and I hope you purchase a copy of the whole thing.

* * * * *

It is yellowing a bit at this stage, but the ink is still perfectly clear. Its lines were clearly marked using a ruler or, more likely, a yardstick. It is both enormous and enormously detailed.

It is called the “1992 Defensive Quality Control” chart. Alabama head coach Gene Stallings used it to track his team’s progress (of which there was plenty) over the course of his national title season with the Crimson Tide. He learned it from Dallas Cowboys coach Tom Landry, for whom he worked as defensive backs coach for 14 seasons between head coaching jobs. “Coach Landry was an engineer by schooling, and he had such an analytical mind,” Stalling says. “He was always big into quality control. When the game is over, Coach Landry knows exactly what happened just by looking at the board.” (His use of present tense is endearing, conclusive proof that once you become a coach, you are always a coach, long after your actual coaching days end, even long after you pass away.)

My only exposure to this chart was through a random picture passed along to me on the Internet. I cannot see all of the categories, which drives me crazy. It was no longer on display at the Bryant Museum in Tuscaloosa when I visited in September 2012. But with a single picture, I see enough. There are three groups: Goals, Philosophy, and Quality. For Goals, Stallings listed things like Points Allowed/Game (goal: 17.0), Turnovers (36), Pressure QB on Passes (35 percent), Cause Fumbles (two per game), and Yards Per Attempt—Run (3.0). (“Nobody runs against my team,” he says. And he’s right; they didn’t.)

For Philosophy, things go micro. Pass Situation Big Pass Play (allowing big plays on passing downs; goal: 1 for every 9). Run Situation Big Pass Play (goal: 1 for every 11). Average Per Pass Attempt, 2nd & 10 (goal: 5.0). Average Per Pass Attempt, 3rd & 13-17 (goal: 7.5). Rifle Force (when a strong safety comes up and forces a run outside; goal: 3.0). Pistol Force (containing an end run with a cornerback; goal: 3.5). Box Force (containing an end run with an outside linebacker; the chart cuts off, so I can’t see the goal for that one, nor can I see any of the Quality guidelines below it).

Microsoft Excel hadn’t quite caught on by 1992, so this was all done painstakingly by hand. For each week, there was a game column and a season-to-date column. If Alabama failed to meet a goal in a given game, or fell behind for the season, the box was shaded in red. “I can look at that board, and I can tell that something’s getting off-kilter rather than waiting till the end of the year. If you see just a whole lot of red, and you’ve got present game and total games, you can see problems developing as it’s happening.”

The 1992 Alabama defense was one of the best in semi-recent memory. The Tide allowed just 122 points and 660 rushing yards in 13 games, forced an absurd number of turnovers, and scored almost as many touchdowns via return (special teams and turnovers) as opponents did via pass. All-American bookend ends John Copeland and Eric Curry combined for over 20 sacks and pressured the quarterback nearly 50 other times. Corners George Teague and Antonio Langham combined for over 20 passes defensed (intercepted or broken up), and two blocked punts. Linebackers Lemanski Hall, Michael Rogers, Derrick Oden and Antonio London combined for nearly 30 tackles for loss, more than 10 sacks, and more than 10 passes defensed.

This unit didn’t exactly have to worry too much about red boxes, in other words. The Tide ‘only’ forced two turnovers and picked off one pass in the second game against Southern Miss (a meager 17-10 win), and when they ‘only’ forced three combined turnovers against Tennessee and Ole Miss, they briefly fell behind schedule for their 36 overall takeaways. But then, after what were probably some contentious, “go get the damned ball” practices, they forced 11 against LSU, Mississippi State and Auburn. That catches you up in a hurry. While they occasionally struggled (relatively speaking) in allowing more yards than preferred on first downs (5.1 per attempt against Arkansas, 6.8 versus Ole Miss, a whopping 9.0 versus Auburn), they allowed just 4.8 for the season.

Tracking stats for such a dominant team might not seem as worthwhile as doing it for a more borderline unit, but the goals for this unit were rather extreme. Stallings knew he was going to have a great defense, and he set the bar high. “Your goals come from what you did the year before,” he says. The 1991 defense had been excellent – 11.9 points per game, 300.3 yards per game – and returned a ton of talent, so Stallings set expectations accordingly.

We tend to use statistics for answers: Where does Team A rank? How many yards per game is Team B allowing? That’s fine for crafting narratives, but it doesn’t help coaches too much. For coaches, it’s not about the answers; it’s about finding the right questions to ask. Stallings’ pursuit of questions, and Landry’s before that, led to the Quality Control Chart, something every coach in the country should be creating. (Why don’t more coaches do it? “We coach the ways we know,” says Stallings. If you weren’t taught to do it by others, you might not think to do it yourself.)

The head coach of a college football team is basically a CEO. He has his hand in every aspect of the game, but his responsibilities also extend to fundraising, amateur psychology, et cetera. Stats are a useful tool, one it would be at least somewhat irresponsible not to pursue. If they can teach you more about yourself and your opponent, and if your opponent is using them in great detail to get a read on you, you should probably keep up in that regard. If other programs are subscribing to STATS, LLC’s detailed charting service or employing their own charters/stat analysis people, shouldn’t you?

“Sometimes measurement can substitute for a higher level of thinking,” says The Hidden Game of Football’s John Thorn. “You come up with an answer that you can deal with, that is rational, explicable, maybe even repeatable – you hope so – but there is a higher level of thinking in coaching, beyond measurement, that is in the realm of philosophy. Your best coaches, your Vince Lombardis, weren’t doing a lot of measurement.”

For Stallings, the goal was not to figure out how good your run or pass defense was from a 20,000-foot view. It was to keep minute track of cracks, breakdowns and trends. If opponents are gaining a few too many yards on 2nd-and-10, you can focus your film study on a small selection of plays to figure out potential problems. If your pass defense is strong in passing situations but iffy in run situations, does that mean you’re biting too much on the run? If your Rifle Force is not sufficient, then what is your strong safety doing wrong? He pared this list down to the categories that meant the most to his own defensive philosophy, and he tracked his stats religiously.

There are others. Back in his days as a receivers coach for the Baltimore Ravens and the University of San Diego Toreros, Stanford head coach David Shaw created a points system for his players. “I tried to track what most clearly affects wins and losses, and then I created a points system: Drops were negative, 15-yard catches were positive, blocks that led to touchdowns were positive. I wanted to inspire guys to do the things that lead to wins. I’m going to reward you individually for doing these things.” He did the same a few years later as Stanford’s running backs coach, rewarding things like yards after contact.

Even if coaches don’t track stats with this amount of zeal, almost every football coach in the country, at almost every level, is coming up with some way to track the tendencies of both opponents and, in most cases, his own team. That always makes for a fascinating exchange when a coach is presented stats by the media or fans.

**********

Following the departures of record-setting quarterback Brandon Weeden and two-time Biletnikoff Award winner (for the best receiver in college football) Justin Blackmon the previous year, Oklahoma State still found itself averaging a gaudy 56 points and 659 yards per game early in the 2012 season, numbers that actually exceeded the pace the Cowboys had set a year earlier (49 points, 546 yards). But that came mostly from wins over Savannah State and Louisiana-Lafayette (combined score: OSU 149, opponent 24). The offense shot itself in the foot repeatedly in losses to Arizona and Texas, and at 2-2, offensive coordinator Todd Monken wasn’t too interested in discussing the “stats” being presented to him.

“You can’t carry over points, you can’t carry over yards,” Monken told the Daily Oklahoman. “Stats are, really, for losers. You don’t want to be 600 (yards) one week and 200 the next, because you’re going to lose that game. … Most of the time, statistics and numbers are all there just to make yourself feel better.”

It was a rather harsh (and humorous) way to put it, but it’s not like he was alone in that sentiment. “‘Stats are for losers’ is one of those old coaching adages,” says Wake Forest head coach Jim Grobe. “It’s almost a coaching axiom.” Coaches have been defensive about stats for decades; one gets the impressions that younger coaches, who are perhaps a little bit more stat-inclined than their older peers, are hesitant to admit such a thing in public. 

“I just don’t believe that [coaches] are as dumb as they make themselves out to be,” says USA Today’s Paul Myerberg. And they aren’t. But in the week-to-week grind of the season, the per-game stats we lean on for our narratives don’t do much for them.

Of course, “you can say stats aren’t very important,” says Grobe, “but by the end of the year you’ll see a lot of correlation between the numbers and whether you’re good or bad. We get so caught up in all this stuff that we can’t see the forest from the trees.”

As John Thorn reminds us, legendary Baltimore Orioles manager Earl Weaver “decried the stat heads but kept rigorous index cards [with notes on matchups and stats].” Coaches probably know how they’re doing, but a) they don’t care about the same stats we do, and b) they don’t care to talk to us about it. The latter point is key. In the end, stats are one of the many things that creates a divide between coaches and fans. When things are going poorly, fans have the tendency of complaining about the wrong thing.

(Other reasons for the coach-fan divide: salaries, hurt feelings from losses, and coaches’ refusal to call the “touchdown play” more often.)

And make no mistake, coaches field a ton of complaints.

“There are not a lot of jobs in the country where you are questioned on what you do every day by the media,” says Steven Godfrey, writer for SB Nation. “Politicians, coaches, and that’s about it. They don’t admit it out loud, but coaches’ biggest asset is to be vague. The one thing they hate is having something thrown back at them. That’s why these mantras are so vanilla. It’s a cat-and-mouse game as to what they actually disclose.” When you’re winning, you can get away with saying bland things like “We do what we do.” It’s confident and a bit badass. But if you’re losing, it is proof of stubbornness and/or just being an ass.

**********

Okay, so let’s just assume that, behind closed doors, with the shades drawn and the lights turned down, coaches tinker with stats of some sort. What are they doing?

“On Sunday, one of the first things I’m going to do is look at the overall picture of the opponent: strengths, weaknesses, et cetera,” says Ball State head coach Pete Lembo. “It puts you in a better state of mind when you look at the video. You have a background heading into the video.”

Count Stanford’s David Shaw, meanwhile, as a member of the “film first” camp. “I always start with the film. It’s your résumé, it’s who you are. After I watch a couple of games or cut-ups, I can look at the stats to see if they’re backing up what I’m seeing. Stats have become a significant part of our game-planning, but we’ll make subtle changes based on what we’re seeing.”

This is something Mitch Tanney of STATS, LLC, confirms. “Teams see something on film, and they can use the data to bounce it against their hunches. And sometimes you can use data to drive your video analysis.”

It is a circular relationship, but that is okay. As Gene Stallings would say, you coach the ways you know. And you probably go about things in a similar way that you did as a no-name assistant.

“We’re not any different,” Wake Forest’s Grobe says. “We look at every stat known to man. Break down game film ad nauseum. We know what you’re doing on downs, field zones, hashes. For just about every situation you can possibly imagine, we’ve got it on a stat somewhere. But how do you manage all of those stats? How do you pull it together in a game plan that makes sense?”

“The approach hasn’t changed through the years,” says New Mexico head coach (and former Notre Dame chief) Bob Davie. “We look at it first from a defensive perspective. Every team, every offense is unique. That’s different from the pros. Every week you’re starting on Sunday, and the first thing you do is figure out what that offense does in frequency. What do they do the most of? What do they do the best? You don’t formulate a defensive plan until you figure that out.”

UL-Monroe head coach Todd Berry, meanwhile, almost goes in the opposite direction. “We tend to use stats more as an affirmation rather than trying to discover things. We’re using them to ask, ‘Is what I’m seeing accurate?’”

Regardless of the approach, film study is everything in football – you just aren’t going to get the information you need about formations, tendencies, et cetera, without it – but stats are wonderful for setting the table. And, of course, film study itself has shifted dramatically through the years.

“What you do hasn’t changed,” says Davie. “What has changed is the volume of things people do. The resources you have to do more things from a breakdown standpoint – it was so elementary, and now it’s so advanced.” 

In his first job as a graduate assistant under Johnny Majors at Pittsburgh, “the first prerequisite was taking the 16-millimeter film [of opponents], tearing it, hanging it by category, and splicing it. We used to have hangers all across our meeting room.” That isn’t so much a necessity in an age where graduate assistants can cut, splice and organize video clips in minimal time on a computer.

Sonny Dykes, head coach at California, places heavy scouting emphasis on stats. But you have to know what you’re looking for. “What we try to do is look at all the types of variables that impact a football game. What’s important? What impacts our style of play?” And for different schools, different stats are of variable importance. For Dykes, a disciple of pass-heavy Air Raid master Mike Leach (among others), time of possession means very little. For Davie, who operates a run-heavy system and is still in the process of building his team’s talent base, it is vital.

[ad placeholder 3]

“Statistics can – I’ll never say they lie, but they can give you a version of the truth that isn’t quite right,” says Shaw. You have to understand the context involved, and you have to know what’s important to you. For instance, “a team that rushes for a ton of yards might just be playing with the lead because of their great defense,” he says. At the higher level, Shaw says the following stats are most important to his team’s philosophy: first downs per game, first-down efficiency and the resulting third-down conversion rate, and red zone touchdown percentage.

For Air Force’s Troy Calhoun, the list has a few differences, but not too many: turnover margin, third-down conversion rate, yards per carry, explosive plays (he defines this as runs of 10 or more yards and passes of 15 or more yards), negative-yardage plays (sacks, tackles for loss), penalties , opponent’s starting field position, red zone touchdown percentage, fourth-down conversions. To maximize opportunities he goes for it on fourth down more than anybody else in the country.

If you surveyed every coach in the country, however, you would probably sense one major theme: “The number one thing is always turnovers,” Dykes says. “Very seldom did we win the turnover battle and lose the game.” 

There is quite a bit of luck involved in turnovers, which has to be maddening to coaches who try to control every aspect of preparation and execution. But considering the impact turnovers can have – on average, I have found that turnovers are worth about five points when you take into account the field position the offense lost and the field position the defense gained – they are very clearly important. You can’t control the way the ball bounces, but coaches very much try to control the number of fumbles they force (and commit) and, in some cases, the number of passes they bat down.

“A year ago [in 2011], we only batted down two balls at the line,” says Colorado head coach Mike MacIntyre, who came to Boulder after three years at San Jose State (2010-12). “Last year [in 2012], it must have been 35. We practiced it every day. We told the defensive line that if they get held up, get those hands in the air and tip it. There’s luck in where it gets tipped, but there’s also skill. Learning where the quarterback moves and where to rush so you’re in the passing lane is something you can do.”

**********

Depending on your system of choice, scouting yourself might be as or more important than scouting your opponent. Bob Stitt, head coach at the Colorado School of Mines, runs his own unconventional version of the spread offense, and since teams play his school differently than everybody else, “film study doesn’t necessarily produce a lot of value. We’ll prepare for the top two or three defensive fronts and their major tendencies, of course, but for the most part we focus on our own team.”

Self-awareness is vital. “Sometimes my defensive coordinator will walk in and say, ‘I think I have a really good bead on these guys,’” says Lembo. If a team is not sufficiently aware of its own tendencies, it’s like playing Tecmo Bowl on Nintendo: the defense simply calls Run 1, floods the play with defenders, and stuffs you for a loss. (You might get away with this if you have superior talent, and in college football, some teams are vastly superior in this regard.)

Coaches like UTSA’s Larry Coker go to great lengths to get the same read of themselves that their opponents will derive, then adjust the game plan accordingly. “This is what opponents are going to have on us, but here are our tendency breakers.”
This is true at every level, of course. STATS, LLC’s Mitch Tanney points out that, in 2012, from just about every formation, the San Francisco 49ers’ run-pass tendencies were the same. “That means they have probably self-scouted quite a bit.” 

Meanwhile, at the high school level, offensive coordinators also have to reflect from time to time. Rob Paschall, offensive coordinator at Carrollton Creekview High School in Texas and an enlightening presence on Twitter, says that he and his staff are often picking up on cues they are giving opponents. “We’ll find that, in this certain situation, we do a certain thing 90 percent of the time.”

Of course, being aware of your own tendencies does not necessarily mean changing them, and knowing your opponent’s tendencies doesn’t dissuade you from going with your gut. “If you know what you want to do, and it’s a good system, you don’t care so much about what the defense knows or is aligned in,” says Berry. 

Berry gives an example from one of ULM’s biggest wins of 2012. On the road against a hot Western Kentucky team, his Warhawks fought back from a 14-point, second-half deficit to send the game to overtime with a touchdown in the final seconds of regulation. In the first overtime period, WKU got the ball first and scored in two plays. It took ULM four plays to respond with its own score, but Berry decided to go for two points and the win. On the two-point conversion, “We knew what they were going to do,” he says. “When it came down to the play-call, I changed the play. The design was not as effective as the other play, but I had more confidence in the quarterback and our players executing that one play. You’re going to use stats in some degree, but you’ll still end up playing your hunches a little bit.”

(You will occasionally hear of coaches even trying to plan their spontaneity, a la Diane Chambers in Cheers. In-game situations are stressful, man.)

As was frequently the case in 2012, Berry’s hunches were correct. After executing a draw play left for a touchdown on the previous play, lefty quarterback Kolton Browning spun around a blitzing WKU defender, rolled right, and finessed a ball to receiver Rashon Ceaser in the end zone. ULM 43, WKU 42.

Self-scouting is interesting when it comes to teams that use a lot of different formations. In 2012, West Virginia ran about seven percent of its charted plays from a no-back, spread formation, 53 percent with one back in the backfield, 35 percent with two backs in the backfield, and four percent with three backs in what is typically a variation of a diamond formation (one back behind the quarterback, two either even with the quarterback or ahead of him in something like a pistol/flexbone combination). With no backs in the backfield, the Mountaineers, like most teams out of the no-back, almost always threw the ball (86 percent of the time). With one back, they ran 41 percent of the time, with two, they ran 51 percent of the time, and with three, they ran 54 percent. 

The success of WVU’s run game was inversely correlated to how predictable the run was. Out of the no-back, with everybody expecting pass, the Mountaineers averaged 7.2 yards per rush. With one back, 7.1. With two, 5.1. With three, mostly in power situations, 2.1. Some unpredictability might have helped, but this might have just been the nature of the beast. WVU was potentially just built to succeed with more players lined up wide and not well-equipped to handle short-yardage situations.

In the end, calling a football game becomes a giant game of chess, albeit one in which the pieces have minds of their own (and don’t always take directions well), and a third party called luck (bounces, officials, et cetera) can lay waste to even the best-laid plans. As Grobe puts it, “We’ve got what you do down to a science, but you know we’ve got it. It’s a cat-and-mouse game. It probably boils down to how good your players are.” I know what you’re going to do, but you know that I know what you’re going to do, and I know that you know that I know what you’re going to do…