Baltimore Ravens QB Lamar Jackson

Pigskin Principia

Guest column by Sean Clement

There is a great debate in the football world over which manner of analysis best captures performance and allows teams to exploit key edges in their pursuit of victory. At the heart of the issue are strong disagreements between those analyzing football traditionally via tape and those using modern analytical methods. These disagreements are especially present in the public sphere as there are fewer incentives for cooperation. They stem mostly from a failure to communicate, the source of which is not immediately obvious.

Discussions on the nature of football, besides the particular disagreements based on opinion, generally fall within two camps. Whether you're a philosopher of tape or an acolyte of the empirical, it's easy to see there is a communications gulf between the two camps. Arguments about the value of running backs in the NFL, establishing the run, or any other topic debated ad nauseum are not addressed here, though they are borrowed from to illustrate a larger point. There are many ways to maximize football performance, but the specific problem of resolving disagreements between our two camps is a bit abstract. First consider football through the eyes of its primary practitioners.

The Coaches' View

Head coaches in football are much like philosopher kings: each rules over his team's scheme and personnel usage with a level of authority seldom seen in the modern world. Their jobs demand not just an intense, personal, and deep love of football, but also the leadership skills required to mold their players into the best versions of themselves that fit within the coach's vision. This is no easy task, so head coaches tend to develop a philosopher's worldview with respect to football. They are seekers of truth and the deep causal roots of all aspects of football itself. To many coaches, the "why" holds a deep philosophical meaning. To know that something works is not sufficient, because it's the "why" that unlocks the type of creative genius needed to innovate at the play level.

Consider all of the aspects that go into the design or implementation of a particular play: the skill set of the current roster, the likely opponent actions, the desired movement of all 11 players on the team. Even body position and foot placement can disguise intent. Coaches do not simply accept that something works. They attempt to provide a deep and powerful explanatory framework from which they derive inspiration, meaning, and vision.

Why a play works, and the context in which it works, can help shape a coach's worldview and intuition, which are both key to his success. In some cases, the vision itself is powerful, but the ability to generate buy-in and direct a team towards that vision is something both rather difficult and difficult to measure. Proponents of coaches such as Pete Carroll would argue this pursuit of a team vision is part of what makes him a great coach. Scouts view and evaluate players similarly, as their work molds the view of coaches and decision-makers on everything from future opponents to prospects in upcoming drafts who fit the current team schema.

When viewing the conduct of play in football through a philosopher's lens, the obsession with the deep causal roots of the performance of a particular player, a play type, or a position requires an incredible amount of in-depth knowledge. Modern coaches are no less methodical than quantitative analysts, and they certainly offer a depth and breadth of knowledge of football far beyond what someone from the analytics community can offer.

The Analyst's View

While coaches are indeed the foremost experts of their craft, it is also true that some transformational changes in football are driven not by coaches, but by "outsiders." For example, fourth-down decision-making is just now reaching a point where risk aversion is the exception and acceptance the norm even though David Romer's seminal paper on fourth downs was published 15 years ago. Some have argued that football analytics has a tone problem, but the conflict over tone is simply a symptom of the true disconnect, the perception of evidence itself.

Those who work with sports data tend to have a much different view on the nature of football, which things are important, and why. Instead of a search for universal football truth, analysts are generally motivated more in line with how Michael Strevens describes scientists in his 2020 book The Knowledge Machine: How Irrationality Created Modern Science. In his work, Strevens argues that what makes science work is what he calls "the iron rule of explanation." The iron rule boils down to two simple precepts. First, that scientists must uncover and generate evidence to argue with; second, that the only thing that "counts" in a scientific debate is that which we have evidence for.

This stands in sharp contrast to how a coach or scout approaches the evaluation of football. Shallow causal inference certainly produces results, but only from things we know how to measure, and it doesn't necessarily provide us with a deep understanding of what we're observing. Isaac Newton, while harangued by his contemporaries on this lack of deep understanding about gravity, had this to say: "It is enough that gravity really exists and acts according to the laws that we have set forth and is sufficient to explain all the motions of the heavenly bodies and our sea."

The key distinction opens much more of a gulf than it would first appear. For a coach seeking to maximize his player's performance or design a new play, accounting for only what we can directly measure seems absurd. Likewise, an analyst may feel as though a scout is focusing on tendencies, patterns, and behaviors in players with no discernible benefit. Which of these two viewpoints is correct? The answer is unfortunately both.

Evidence is a matter of perspective, and our very perception of what is and is not evidence is what holds back a fuller, deeper understanding of the beautiful and violent game of football. One of the difficulties with an analytical approach is that, unlike the motion of heavenly bodies, the rules and "laws" of football are subject to the whims and machinations of man.

Historical Examples

Consider the play-action pass, which has a history going back all the way to the 1930s. Play-action is not a new concept. Yet, if you analyze play-action from when it became a staple in the 1960s to today you might end up with vastly different estimations of its usefulness even if its effectiveness has remained largely unchanged in the modern era. This provides a look at both sides of our evidence argument. Before the proliferation of play-action, there were numerous theories about why play-action worked, many of them wrong. So while play-action was born out of coaching creativity, the theories about rushing production or attempts driving play-action success turned out to not be supported by evidence. Alternatively, while analytics has shown light on various parts of play-action, it would be difficult for the analytics community to conceive a new approach to play-action on its own. Even more compelling is the possibility of some adaptation in play calling or design that fundamentally changes the balance of power with regards to play-action or some other aspect of football. If the linebackers' first intuition through sufficient training were to key in on stopping the pass first rather than defaulting towards their run fits, would we not then see a fundamental change in the effectiveness of not just play-action but the running game itself?

As Lamar Jackson took the league by storm in 2019, the rules and preconceptions about reliance on quarterback rushing were turned on their head. As more spread offense concepts work their way from college to the NFL, and mobile quarterbacks such as Jackson and Josh Allen find success, the rules shift under our feet a bit. It's not that data couldn't tell us quarterback rushing wasn't valuable before (going all the way back to Fran Tarkenton), but extrapolation from one style or archetype to another is dangerous, and anyone comparing Jackson's skill set to Cam Newton's would rightly be strongly questioned if not flatly ignored. Paradigm shifts within the game are often the result of coaching vision in the face of opposition, yet that same coaching vision can lead a team to languish behind antiquated ideas and disproven theories. Additionally, there is no global optimum to solve for. One cannot simply draft a Patrick Mahomes every year. The trends are mutable, the extrapolation is perilous, and conventional football wisdom can produce explanations which fail simple scrutiny.

Working Together

This is not meant to be football nihilism, but it's important to note that neither analytical prowess nor coaching genius alone is likely to dominate the NFL. This makes communication between the camps driven by shallow causal explanation and deep philosophical understanding all the more important. The edges may be small, but they are exploitable and they accumulate.

Clearly there is a benefit to closing the communications gap. There are certainly organizational designs and schemes that help bring the two worlds together through pairing analysts with scouts, rapid feedback between sides, keeping theories falsifiable when possible, and starting analysts off with measuring things normally measured by hand. Setting expectations for realistic feedback is the only means for improving our collective football knowledge.

It would be unreasonable to expect a proprietor of analytics to be able to identify, let alone conceptually explain, Drop-8 Inverted-Cover-1 Double-Rat, just as it would be unreasonable to ask an offensive coordinator to describe how a 2D convolutional neural network can be used to process Next Gen Stats data and produce predictions of future performance. They are radically different ways of looking at the same problems and it's very likely impossible to be an expert in both fields. People spend their entire lives studying football or data science never to truly master them. But most important is that your film or data contemporary likely thinks in a way structurally different from you, and that difference alters the very nature of what you and they consider valid evidence, feedback, and criticism.

Shared understanding will allow the most forward-thinking coaches, scouts, general managers, and organizations to leverage analysts from outside their field to discover unexploited edges not yet conceived. While in some cases analysts may be measuring the wrong thing, it's also possible that we can enhance our coach or general manager's deep causal framework of football by breaking down these relationships into smaller, more testable pieces with a focus on explainability. In doing so, we can marry our football philosophy with specific testing through a systems-based approach.

For coaches, scouts, and other practitioners relying on subject matter expertise, an openness to unlearn can be critical. The realization that conventional wisdom can be wrong or sometimes correlations are much weaker than we believe is often harder to accept than the opposite when ties are stronger than expected. Specifically, things which work at the high school and college levels don't necessarily work in the NFL. Coaching and player talent parity plays a significant role in what does and doesn't work or hold true.

Intellectual openness is the only path forward to exploiting the full strength of analytics while at the same time empowering the creativity of coaches. Openness sounds simple enough until a disagreement occurs and when the ties of teamwork are tested. It should be enough to know that performance edges are possible as any sufficiently competitive person would do whatever it takes to win.

As we do not live in such a world, several edges remain for those who are willing to have their beliefs challenged and are brave enough to change their minds when presented with new evidence. It can be hard work to translate narrow findings to an overall football framework, just as it's difficult to change preconceptions rooted in decades of conventional wisdom. Real change in public discourse will necessarily lag behind the league in this area as analysts and scouts are teammates rather than adversaries, but strong leadership is still required at the team level to drive change. Real leadership demands the humility of accepting the possibility that what we think we know is wrong, and the boldness to act on it. That work is hard but worth it for those who want to win.

Sean Clement is a former analyst for the Baltimore Ravens. He can be contacted on Twitter @SeanfromSeabeck.

Comments

25 comments, Last at 17 Jul 2021, 1:35pm

6 Thanks. It looks like the…

Thanks. It looks like the MIKE is aligned as a stand up rusher and takes the back out, while a safety lines up at MLB at races to the deep middle. The Rams were squatting on those intermediate routes but Brady was well protected. I wonder if he could have waited for the square-in to clear.

3 I think Science isn't…

I think Science isn't strictly in search of evidence. 

To me, you need a theory about how and why something exists and then you need evidence to back it up. Evidence without theory can be just as misleading as theory without evidence.

 

14 You're confusing theories…

You're confusing theories with hypotheses.  A theory only happens after a hypothesis has been tested, and therefore evidence of some sort has been generated.  I believe a hypothesis doesn't need evidence to be a hypothesis, but I'm not sure on that.

15 My background is not in…

My background is not in classic science so I don't know what the formal definition of theory stands for.

Coming from an economics background, when I do work on a particular problem, I don't just look at data and let it speak. You have to have a proper concept of cause and effect before you can start applying data to it. 

You also need to be aware of all of the assumptions you are making with the data when you do your analysis. 

16  I don't come from a…

 I don't come from a classic science background either, I just remember what I was taught in high school.  It's entirely possible my recollection is off; also, Economics is forced to use different methods than physics or biology to study reality, and unfortunately I did not have as much an education in that science as some of the others.  It's hard to have anything resembling the Michelson Morley experiment in economics, I would think.

Mainly, I was just reacting because I've heard way too many people say it's just a theory when referring to evolution, for example, when the amount of evidence needed for a hypothesis to become a valid theory is rather large.  I realize I was just nitpicking and you weren't actually suggesting something so foolish.

19 No, you're right. A…

No, you're right. A hypothesis is technically the correct term in science, although informally it's common to use "theory" as a synonym. What you mentioned in your first post is important because of the plague of correlation mining. It's easy to find "evidence" for absurd ideas, that's why you need to have a hypothesis before looking at the data.

20 My favorite example is a…

My favorite example is a fable about a Russian Czar posed with a conundrum. He's told that a particular area of Russia has high morbidity. Wouldn't you know it, that area also has a high number of doctors around. Conclusion...kill all the doctors and the problem will go away!

A historically accurate example happened in the great depression. Those poor farmers are poor because the price of pigs is too low. Solution - let's slaughter the pigs and dump their carcasses into the storm drain. This will cause the price of pork to rise and the farmers will get rich! Yes this was actually something the government actually did. 

22 ... and a more current example ...

"... Mr. Trump said, "When you do testing to that extent you're going to find more people, you're going to find cases. So I said to my people, 'Slow the testing down, please.' ..."

https://www.cbsnews.com/news/trump-slow-down-testing-coronavirus-i-dont-kid/

If posting this violates a site rule about injecting politics, sorry; I realize it might, and I'm ok if it needs to be removed. But it is just a literal quote, no opinion, other than implied.

 

24 As an actual scientist, I…

As an actual scientist, I see this as a chicken-or-egg question. The relationship between hypothesis, theory, and evidence is complex. The more basic relation is observations and questions.

4 A controversial thought…

A controversial thought. Just how much has Lamar truly revolutionized the game versus being the modern manifestation of past players?

Randal Cunningham, Steve McNair, Mike Vick and others were successful, high profile scramblers. 

In that sense this may just be less about Lamar and more about the era

 

5 I doubt anyone would dispute…

I doubt anyone would dispute that Cunningham, McNair, and Vick would each be much more successful in today's NFL then when they played, possibly to the point that each would have had a HoF career (Cunningham and McNair were pretty close as it is). Lamar really seems a lot like Vick, just with better accuracy and better dedication to his craft (Vick had an ugly play-action and his footwork was sloppy).

On the other hand, guys like Drew Bledsoe and Vinny Testaverde (just to pick some examples) might have been less successful if they had to play in today's game.

10 Why do I feel that…

Why do I feel that Cunningham, McNair, or Vick would be better now, or why Bledsoe and Vinny would be worse? I think the former is obvious. As for the latter, I was just trying to think of some traditional pocket passers who were fairly immobile and prone to sacks, and didn't really have great accuracy either (granted Vinny's numbers are dragged down by the Tampa years). Maybe add Brad Johnson to that list?

On the other hand, Dan Marino was so good at avoiding sacks (and accurate, relative to his era) that I'm sure he would have been just as good now as he was then.

11 It was the latter comment…

It was the latter comment. Statues who aren't good at avoiding sacks. I also wonder how true that narrative is. Vinny's sack percentage is around 0.063 and Drew's is 0.069 while Big Ben for example is right around 0.065.

I also think those sack numbers are inflated because of how different the game was. More teams have gone to spreading the field and throwing lots of short passes, including screens. That's going to make a modern QB's sack percentage much lower relatively just by those changes alone. 

In fact, I think those passers of yesteryere look a lot like Big Ben and Matt Ryan today. 

13 Not just the sack percentage.

  The other consequence of immobility is a QB hit.  It may not cause a loss of yards, but it can cause incomplete passes and WILL cause physical punishment that adds up over time.  Ben is a good example of that as was McNair, BTW.  Unless of course you have as quick a release as Marino, and very few QBs do.

17 Big Ben isn't really much…

Big Ben isn't really much like Bledsoe or Vinny. He's much bigger and stronger than they were, and his sacks result from his confidence in being able to take hits or power through smaller defenders, hence holding the ball longer to try to make something happen. Big Ben could get away with prolonging the play because he was so tough. He's very similar to Culpepper that way, who was also a big strong QB who took a lot of sacks not in spite of the fact that he was tough to bring down but because of it.

Also, it's true that sack% has gone down, due to shorter passing concepts. But modern pass rushers are also just plain faster than they were in the 90s. I think being a statue makes you more of a target now than it did then, despite shorter/quicker pass plays, unless you have a Marino-like release. Especially so because pocket passer statues are less common now than they were then.

18 I guess what you are really…

I guess what you are really saying is sacks are more detrimental in today's game than they were in the past, tilting those players who were net positives into net neutral or slightly net negative. I tend to take the opposite view in that schemes and pass concepts have made it harder to sack players. So one could make the argument that they would be even more viable in today's game.

Look at the so called statues that are out there now. Matt Ryan, Jimmy G, Kirk Cousins etc etc. Bledsoe would probably fall into that category today which is what he probably was for his career( I didn't watch prime Drew to confirm).

 

 

 

21 The difference between…

The difference between Bledsoe and Testaverde on the one hand, and Ryan and Cousins on the other, is accuracy. Sure, some of that is due to shorter routes, but Ryan was top-10 in completion% 8x, Cousins 6x. Bledsoe and Testaverde only managed to make the top-10 2x each, despite the fact that the league was smaller and it was slightly easier to make the top-10, and despite the fact that they were each starters for many more years than Cousins. (Brad Johnson actually was pretty accurate for his era, and Garoppolo's accuracy is middling for his, so I guess they don't fit this hypothesis.)

I am positing that immobile QBs who are poor at sack avoidance and also not very accurate who were pretty successful in the 90s would be less successful today. That's partially because of sacks, but also because not being able to scramble removes a lot of potential for creative playcalling, and because accuracy is more important that ever and if you have it, makes up for a lot of other ills.

Basically, if some QBs would be better now than they were 20ish years ago, it stands to reason that some other QBs would be worse. I am just trying to hypothesize about whom those worse QBs would be.

9 It would be fascinating to…

It would be fascinating to read some of the author's specific experiences of working as an analyst for an NFL team - although entirely understandable that he very probably aspires to work in the league again, and would not want to tread on any toes!

I'm sure (as the author suggests) that there are further edges ready to be exploited through analytics, although we may soon be at that stage where the low hanging fruit has been picked, the best stuff has been copied league wide, and further analysis yields increasingly diminishing returns. 

 

25 Newton and Gravity...

Newton was not "harangued by his contemporaries on this lack of deep understanding about gravity."  At the time, there was no way of "understanding" invisible forces.  They were the stuff of "magical thinking."  Newton's workaround was to substitute a mathematical explanation for a mechanical one.