Writers of Pro Football Prospectus 2008

Most Recent FO Features

JohnsonAnd03.jpg

» Factors: Andre Johnson

One of the NFL's best receivers notched a -2.3% DVOA last year. Does a target-by-target breakdown show he was better than that?

11 Aug 2005

Book Review: Scientific Football 2005

Scientific Football 2005
by K.C. Joyner
Self-published, available at www.thefootballscientist.com
Reviewed by Guest Columnist Jim Armstrong

Like many football fans, I had never heard of KC Joyner until Sports Illustrated's Paul Zimmerman reviewed his book Scientific Football 2005 in a recent SI.com column. Joyner, who bills himself as The Football Scientist, probably got more publicity than he ever could have hoped for thanks to Dr. Z. But at a fairly hefty $50 price tag, I would have preferred to have seen a more detailed synopsis of Scientific Football 2005 before shelling out that much money for a book sight unseen and author unheard of. My goal for this review is to provide a more detailed and critical analysis of the book so as to be helpful to those who are still wondering whether to purchase it. Perhaps it will also encourage some public dialogue regarding the book's strengths and weaknesses, providing valuable feedback the author can use to enhance future editions.

Scientific Football 2005 is the culmination of the author's quest to review film of every regular season and playoff NFL game during the 2004 season, each game recorded using a complex satellite and video recorder setup. He then charts and grades most aspects of the passing game using his own home-grown metrics.

This is a very well-written book, especially for a first edition by an author with few previous publication credentials. The prose flows very nicely, the discourse is lively, and there are relatively few typos for a book of its size with no hint of an editor. The author is obviously a very knowledgeable fan of pro football and demonstrates an excellent understanding of passing routes and coverage schemes. The book is very easy to read and feels more like a scouting report backed with facts than an arcane academic textbook you might expect after seeing the title. The writing style is quite accessible to the mainstream, albeit hardcore, football fan and should even appeal to the number-phobes, with frequent discussions of such intangibles as emotions, toughness, momentum, and by speculating on players' motivations. There's also a touch of humor here and there and numerous pop culture references as metaphors to spice up the commentary.

The author's web site preaches a "no-hype" approach, but there are a few clever marketing techniques. Besides the title, another is the repeated self-comparisons to Bill James, including one on the book's cover. James and most traditional sabermetricians concentrate on advanced statistical analysis of the objective data present in published box scores or play by play logs. Joyner uses television broadcasts to chart occurrences that are fairly subjective, but does relatively little analysis of his data other than reporting simple averages. What I find most ironic about the comparison to James is that the tired old argument in the stats vs. scouts "Moneyball" debate from old-school baseball fans is that Bill James and his brethren don't actually watch the games. I can't imagine anyone saying that about Joyner.

As a self-published book, Scientific Football 2005 is 467 8.5 by 11 inch pages held together by spiral binding, probably done at the local copy shop. Perhaps predictably, my binding broke after about 30 minutes of reading. (Ed. note: Joyner tells me he's improved the binding since the Dr. Z article led to an increase in book orders.) Navigating this book can be a bit difficult. There is no index of players or teams, and the page headers don't denote which chapter or section you're in. The stat charts are located throughout the book, but the key to the obscure abbreviations of his metrics are only found near the front of the book.

The book is organized into four sections. The first section serves as an introduction to the author, his motivations, and his methods. Joyner then dives right into a description of his unique scouting system and the stats he assembles, or at least those that are printed in the book. Joyner describes in great detail exactly what he looks for on each pass play and why. Basically it boils down to whom the ball was thrown to, how open the intended receiver was, which defenders were covering the receiver, how accurate the pass was, and whether it was completed. Then the passer, receiver and defender are all credited with stats using a variety of metrics describing the result of the play. The classification of the depth of each pass as short, medium, or deep is one of the foundations of all his metrics. It allows Joyner to examine how each player was used in the passing game and whether he was successful in that role.

In judging how open a receiver was, Joyner uses five degrees of openness: tight coverage, good coverage, open by 1 step, open by 1-2 steps, and open by 2+ steps. Although he insists that he can distinguish between all these, he doesn't really explain why it is important to do so. One stat I found particularly enlightening is the soft coverage percentage, which measures how often a cornerback plays off the line of scrimmage as opposed to tight on the line in bump-and-run coverage. Joyner also classifies quarterback sacks at the team level by eight distinctions, including blown blocks, coverage, individual effort, and scheme sacks. Blitzes are considered scheme sacks, but much to the dismay of Tuesday Morning Quarterback readers there's no discussion of when the blitz is most effective and how often it should be used.

Probably the most debatable stat Joyner uses is the bad decision metric. Using his best judgment, he counts every bad decision a QB makes and rates its severity on a scale from 1 to 5 depending the impact it has on the play and the game. He then compiles both raw bad decisions as an overall percentage of pass plays and also weighted bad decision percentages that give greater weight to the most severe bad decisions. Examples of bad decisions are scant, for example throwing into coverage or throwing the ball up for grabs instead of taking a sack. As with the degrees of openness, he doesn't really explain why five degrees of severity are necessary. He also rates defenses by the bad decisions they "forced" with little discussion of how much the quality of opposing quarterbacks faced had to do with it. I'm also reminded of Phil Simms describing in his book how what often appears to be a mental mistake is simply a physical error. For example, it may look like the QB forced a ball into double coverage, but perhaps if the pass were more accurate and arrived quicker it would have been complete. In many cases, bad decisions are hindsight; if the QB executes properly, that same decision may result in a big play for the offense. In any event, Joyner feels strongly that bad decisions is a very important metric, particularly for rating quarterbacks, but wasn't too convincing that it is indeed so.

Joyner acknowledges that there is some subjectivity in his measurements, and his response is simply that he's developed a consistency in approach over his years of tape review. Although it would seem difficult to avoid biases and sloppiness while reviewing 50 hours of video each week, I do believe he's making an honest attempt to be fair and accurate. But having personally attempted to review NFL game tapes play-by-play, I do wish he would have mentioned how much difficulty he encountered due to poor camera angles on the network broadcasts. My guess is that experience can alleviate some of these issues.

The next section contains nearly 100 pages of stats, ranking every team and every QB, WR, TE, CB, SS, and FS to see significant playing time in 2004. The players at each position are ranked by over ten different metrics with further rankings broken down by each passing depth grouping. These pages probably would have been better located in the back of the book as an appendix. The tables are a bit difficult to read because so many numbers are jammed onto each page. And unfortunately, this requires him to use cryptic abbreviations for the column headings with no key, meaning you have to flip back to the beginning of the book to remind yourself what, say, "ttl msd ps" means (total missed passes). Still, these pages are a gold mine of football statistics, unlike anything ever before made publicly available.

With all the different stats measured for each position, Joyner does not attempt to combine them into one true number by which he can rank each player from best to worst overall. Rather, he provides interpretation in the player commentary as to how good a player is overall, based on specific strengths and weaknesses shown by the stats. I found this somewhat refreshing in a world that craves absolute rankings, but at the same time I'm also left wondering which metrics are most important, which metrics tend to be more dependent on coaching philosophies, strength of opposition, or luck, and which metrics offer the most future predictive power. Certainly there exists great opportunity for further analysis to help answer these questions.

Following the stat pages is an analysis section for each team. This is really where Joyner shines. He writes several paragraphs reviewing the 2004 performance of each player to see significant playing time at the positions mentioned above. Although Joyner does not try to hide his interest in fantasy football, his analysis in this book is decidedly geared toward what the players do to help their NFL teams win. Joyner is careful to look for alternate explanations for a player's stats, for example noting whether the opposition was unusually strong or weak or whether the team's schemes tended to favor certain types of passes.

Joyner isn't at all shy about telling us when his judgment of a player's performance doesn't mesh with the common perception, particularly if they receive post-season awards. He acknowledges Ed Reed's playmaking abilities, but also notes that he was actually targeted often and ranked near the bottom of the NFL strong safety rankings in yards per attempt, touchdowns allowed, and percentage of receivers open by 2 or more steps. Tory James ranked very lowly in cornerback completion percentage and yards allowed primarily because he plays the deep pass well but allows a lot of short and medium passes. And Lito Sheppard ranked among the worst in most cornerback categories at every passing depth.

On the other hand, Joyner sings praises of those players who turned in outstanding seasons without the mainstream recognition. David Carr, whom Joyner calls "possibly the next great NFL QB," ranked among the top five QBs by many of his metrics including #1 in deep completion percentage. Ronald Curry had a good year primarily because he was able to get wide open so frequently and "could have a dominant season in 2005." Will Peterson had one of the best seasons of any cornerback in 2004, ranking highly at all depth levels. And Ken Hamlin is "one of the best free safeties in all of football," putting up great stats despite being surrounded by inferior talent in the Seattle secondary.

Although Joyner's main interest appears to be evaluating the performance of individual players in the passing game, one of the things I liked best is his discussion of the observed strategies and philosophies of various teams and coaches. For instance, he mentions Mike Shanahan, Mike Martz, and Brian Billick as the coaches that are the most ruthless in targeting the defense's weakest DB. He also noticed that the Chiefs defense tended to blitz frequently and play their cornerbacks tight on the line of scrimmage in man coverage. This backfired on them frequently because their corners weren't very good in man coverage and left them susceptible to getting beaten deep, yet the Chiefs continued using this scheme even when it wasn't working. However, in some cases what he writes doesn't quite match the stat charts. For example, he notes repeatedly that Ted Cotrell seems to be so afraid of getting burned on the deep ball that he has his Viking DBs playing off the line of scrimmage frequently. This renders them helpless in stopping short and medium passes. But Joyner's numbers show that Minnesota ranked 10th lowest in percentage of soft coverage, not what I would have expected after reading the team chapter. Still, it's these kinds of observations that you don't really see in the mainstream media and what makes this book so special.

Each team section is followed by stat pages showing how the team and its qualifying players ranked. This is intended to be a convenient presentation of the same numbers from the earlier stats section, but I think it makes the book more difficult to maneuver.

The final section contains a bunch of short essays, mostly dealing with television broadcasters. Perhaps most relevant was one describing how his approach to video review differs from that of Ron Jaworski and Merrill Hoge, two analysts who use extensive use of film on their ESPN shows. The TV analysts, as Joyner sees it, focus more on how coaches use formations and playcalling to gain an advantage during the game. Joyner, in contrast, focuses more on individual player matchups, particularly WR vs. DB, and this allows him to concentrate more fully on the performance of specific players.

One of the underlying themes about the passing game that Joyner emphasizes is that a team's defense is only as good as its weakest link in the secondary, which the best offenses will target. This is probably the most important observation in the book, and plenty of specific examples of matchups are provided to demonstrate this. He also touches on how defensive coordinators can deploy schemes to help their weaker secondary players, such as zones or double-coverages, but this topic is somewhat under-discussed.

I do wonder whether Joyner generalizes this "weakest link" motif to other aspects of the game beyond what is warranted. He often repeats a quote attributed to Tony Dungy that "85% of NFL games are lost rather than won," though I didn't find his argument particularly persuasive. He also frequently discusses the importance of QBs avoiding bad decisions and DBs avoiding blown coverages or getting burned while rarely mentioning the risk/reward element of going for a big play. And in a discussion of coaches he states that "the greatest coaches are motivated by a fear of losing". A more cynical reader would be tempted to think Joyner's fundamental prescribed modus operandi is "playing not to lose," though I'm not sure that's really what he intended to convey.

All these criticisms of Scientific Football 2005 are relatively minor. This is a remarkably thorough treatise on the NFL passing game. Joyner's analysis of individual defensive secondary play is probably the most comprehensive ever published. He weaves his original, esoteric metrics into a very readable book. Hardcore NFL fans should only hope that this book becomes a series that broadens in scope with subsequent editions. Either way, I'll be keeping this book handy as I watch this fall's games.

(Ed. note: Just to make clear, Jim is not an "official" staff member of Football Outsiders but did write a few essays for our book, Pro Football Prospectus 2005.)

Posted by: Guest on 11 Aug 2005

24 comments, Last at 14 Sep 2005, 5:38pm by J byler

Comments

1
by Sean (not verified) :: Thu, 08/11/2005 - 4:13pm

In short, it's a terrific book, and one I plan on making frequent reference to throughout the season. That said, you have to be a little careful about the stats, which are simply not 100% accurate. What's notable is that despite the fact that Joyner is missing gametapes (primarily Jaguars games) and is only reporting on games that he has personally scouted, his numbers tend to run a little high. For instance, most of his quarterbacks are charted as having between 20 and 30 more attempts than they actually threw last year according to the official NFL statistics, which suggests that some data is getting entered in the database more than once. Depending on which plays are getting counted more than once, twenty or thirty additional plays can have a definite impact on the overall numbers. But even with that said, the numbers are accurate enough to give you a very good picture of how every team's passing game and secondary worked, and it's really an invaluable book.

2
by B (not verified) :: Thu, 08/11/2005 - 4:16pm

I think I'll be picking up the book. I was a little leery about plunking down 50 bones for a self-published title, but it sounds like it's worth it.

3
by Reinhard (not verified) :: Thu, 08/11/2005 - 5:05pm

Maybe he counts some plays, sacks, etc, or something, but not spikes, etc. different than NFL.

4
by Eagles Fan in Maine (not verified) :: Thu, 08/11/2005 - 5:16pm

Jim... Great meeting you (and Aaron) at the Boston area book tour stop. I have been looking forward to this review since, and it gave me exactly what I was looking for as to what the content provided in a more detailed fashion. Many thanks.

Here's hoping future editions expand upon this work, and that FO may be able to help codify some of the weaknesses you portray. And that Lito Sheppard can perfom better than a pro bowl year.

5
by Mike (not verified) :: Thu, 08/11/2005 - 5:45pm

I have the book and I love it. It has info you cannot get anywhere else, and any hardcore fan will love it.

My biggest concern with it was one mentioned in this review, about TV angles being insufficient for this kind of analysis, and I e-mailed him about it. He said that you can see more than you think, and there is usually a replay that will show you anything you missed. I still say you're missing some stuff, but ok.

My other major gripes are that he never differentiates in his stats between whether the CB was in zone or man, which seems like a pretty big deal. Instead, all the stats are just bunched together based on depth (which really means type) of route. And my other gripe is that he talks about his sack numbers but they are not listed in the book, he just mentions a few here and there. I'd love to see those. Maybe I'll e-mail him, he was very nice when I did the other time.

Overall though it's a great book with some excellent analysis that you'll never find on ESPN, and it goes to show you how rarely the mainstream sports writers actually know what they're talking about.

6
by Larry (not verified) :: Thu, 08/11/2005 - 6:27pm

My biggest issue is that the analysis of receivers and openess is limited to the guy the ball is thrown to. There's a lot of discussion to the tune of WR X was open by steps 40% of the time, which indicates such and such. But, that's much more when the QB chose to throw it to WR X, rather than WR X's ability to get open. That's a big selection bias. I'd rather know those numbers on all plays, not just ones where the WR has the ball thrown to him. Of course, then TV coverage is a big problem to figure that out.

I'd say I'm still trying to figure out how to interpret all these numbers for myself.

It is still a compendium of info you can't get anywhere else, and that's great.

7
by MDS (not verified) :: Thu, 08/11/2005 - 7:35pm

Count me as a big fan of KC Joyner's. This is the type of stuff I like:

"Al Harris is much more effective when he plays close to the line than when he plays off....Don't expect Jim Bates to play Harris off the line very often,
as the Dolphins defense had the lowest soft coverage percentage in the league last year."

That's just the kind of insight that only someone like Joyner, who logs every game, can come up with. I love reading stuff like that because it's the stuff you never read anywhere else.

8
by Mike (not verified) :: Thu, 08/11/2005 - 10:42pm

Larry's point was one I forgot about, that's another problem with the book, there is a major selection bias because the percentage of times a player is open when he is thrown to is obviously going to be higher than his overall %, so what would be valuable to know as well was how many times he was covered, forcing the QB to go elsewhere. You do get to see the number of attempts a guy got, so that kind of tells you, but it's not adjusted for how many plays he was in on. A more useful number than times targeted would be % of plays on which the WR was in the game that he was actually thrown to. that would be a great stat.

9
by SteelerBill (not verified) :: Thu, 08/11/2005 - 10:49pm

Great book! I strongly suggest that KC's book and Football Prospectus not leave your side throughout the season - Oh by the way, KC was part of the Fantasy Football roundtable on the NFL Network last week as well...

10
by MDS (not verified) :: Fri, 08/12/2005 - 11:15am

Of the three guys on NFL Network last week, I thought KC was by far the most insightful. But they should have some of us FO guys on.

11
by Jersey (not verified) :: Fri, 08/12/2005 - 12:24pm

There's something strange that a CB renowned for being terrible, Will Peterson, by Giants fans and a player renowned for being great, Lito
Sheppard, by his fans yet KC tells us the opposite. I imagine the truth lies somewhere in the middle. I find it hard to believe Peterson is one of the best CBs in the league, seriously. Sounds like an excellent book, but I unfortunately do not have the time to plug through it to warrant the price tag.

12
by elhondo (not verified) :: Fri, 08/12/2005 - 2:22pm

So, ... after reading the book...

Who's better; Manning or Brady?

13
by Sid (not verified) :: Fri, 08/12/2005 - 4:21pm

I'm finding it hard to believe KC Joyner had to spend $25,000 to publish it.
Sounds like a great book for a hard-core stats fan, but I think I'd bore myself thoroughly looking at hundreds of pages of stats.

14
by Sid (not verified) :: Fri, 08/12/2005 - 4:41pm

One of the underlying themes about the passing game that Joyner emphasizes is that a team’s defense is only as good as its weakest link in the secondary, which the best offenses will target.

I don't totally agree. I'd rather have one great CB and one poor one than two poor ones. If I know one of my CBs is a weak player, there are things to do to compensate for that. You can try to match him up against the opposition's weaker WRs and roll coverage to his side.
I see that's mentioned in the same paragraph there, but shouldn't that prove a defense is not quite as weak as its weakest player? If he's right, he should argue that the Eagles have a bad defense.

A more cynical reader would be tempted to think Joyner’s fundamental prescribed modus operandi is “playing not to lose,� though I’m not sure that’s really what he intended to convey.

That's exactly what it sounds like.

15
by MDS (not verified) :: Fri, 08/12/2005 - 5:47pm

When did KC say it cost $25,000 to publish it?

16
by ABW (not verified) :: Fri, 08/12/2005 - 6:20pm

I dunno Sid. I think that a good offensive coordinator should be able to pick on the weak link in your secondary regardless of what you do to cover it up. If you're playing an offense with any depth, I'm sure that gets exacerbated. I'm sure Belichick was trying to cover up Dexter Reid in the Super Bowl, but the Eagles still scored a TD on him. I bet Mike Shanahan was doing his best to give Roc Alexander some help in the playoffs vs. Indy, but he still got torched for like 200 yards. There are a lot of offensive sets and plays designed around isolating one player on the defense and forcing him to make a play or decision - one of those is going to pick on your weak guy.

You can cover up a weak link in your secondary to some extent, but I think you are still better off having good depth in across the board than one great player where the other team will simply not throw to his side.

17
by Sid (not verified) :: Fri, 08/12/2005 - 7:59pm

RE: 15

Check out Dr. Z's article linked at the top of the article...

18
by Sid (not verified) :: Fri, 08/12/2005 - 8:01pm

RE: 16

Yes, I see your point, and obviously it's preferable to be strong all around, but I can't agree that "a defense is only as good as its weakest link in the secondary." It's simply not true.

19
by Mike (not verified) :: Sat, 08/13/2005 - 12:51pm

#1, the reason some guys have more attempts is that he counts playoff games, whereas the official stats do not.

20
by Sean (not verified) :: Sat, 08/13/2005 - 4:38pm

No, the inclusion of playoff games doesn't explain it, either, as his numbers are still off even when the playoff attempts are included. He includes a disclaimer in his introduction where he points out that he will get the odd duplication. Whatever the reason, there is no question that his numbers are off, and what's strange about it is that they are often high (I would expect him to have lower numbers as a result of his not reviewing 100% of the games, but it doesn't work out that way. Look at some attempts by quarterbacks who didn't make the postseason (NFL official/Joyner:

T. Green: 556/535
V. Testaverde: 495/517
A. Brooks: 542/573
D. Bledsoe: 450/458
B. Leftwich: 441/369 * (You'd expect Leftwich to be the biggest disparity, as Joyner mentions that Jaguars games are blacked out for him.)

I haven't seen a single instance where his QB attempt numbers are on the dot. That's not to say that you should therefore throw the book out the window, not at all. But don't treat any numbers in it as gospel.

21
by Frank (not verified) :: Sat, 08/13/2005 - 6:31pm

Re: 15 Actually Z said KC quit his $50K annual salary job to produce this work. Farther down, he says if 500 people pay $49.95 then the money KC spent to produce it will be covered. That's roughly $25K. I hope he sells more than 500 copies so 1) he can feed his family without having to go back to the wasteland of telecom, and 2) so he can produce more work like this. I'm buying it. Maybe the ESPN Sunday Night announcers should too.

22
by Sid (not verified) :: Sun, 08/14/2005 - 1:41am

I know it said he quit his job, but it said it would require $25 K to cover what it cost him to produce it. That's talking about the salary he missed out on?
IOW, he makes a profit regardless, but he'd need to sell 500 copies to actually do as well as he would've without the book. Then again, I'm sure watching football is a heck of a lot more fun than his real job was...

23
by Jim A (not verified) :: Mon, 08/15/2005 - 1:13am

Sid (#14, 18): the "weakest link" line is simply my general interpretation after reading the entire book. Joyner didn't actually write that specifically. Assuming that's what he meant, I think he's mostly correct except that 1) the opposing offense still has to have a good enough #3 and/or #4 WR to take advantage of a defensive liability, 2) a defensive coordinator can use schemes to help his weakest coverage DB, and 3) a particularly strong coverage DB can make it easier for his teammates in the secondary.

Re: #6, 8, I don't think the selection bias is that big of a deal and I think trying to measure openness of all receivers (even if we had the proper camera angles) would introduce a lot of random noise. For example, do you really care if a WR is wide open deep down the left sideline on a screen pass to the right flat? I don't think it would be common for a receiver to be frequently open, yet not get his fair share of balls thrown his way over the course of the season. In some ways, Joyner's openness stats describe the confidence a QB has in his receivers. I think the other stats, as well as just plain number of receptions, are already a pretty good proxy for measuring a receiver's ability to get open.

Mike, I agree that knowing the number of plays a player was in the game would be very useful for a lot of reasons, including the one you suggested.

24
by J byler (not verified) :: Wed, 09/14/2005 - 5:38pm

What about taking into account the pressure that the defensive line puts on a quarterback. If a quarterback is getting hurried and sacked, the secondary is always going to look better. Conversely, if a QB has all day to throw the best of cornerbacks are going to struggle in coverage for long periods of time.