by Mike Tanier
Over the summer, the Football Outsiders research staff discovered a statistical trend so revolutionary that it will change the way you think about football.
We stumbled upon this trend by accident, like paleontologists tripping over a stone that turns out to be the skull of some new species of dinosaur. Once unearthed, this trend changed our entire football zeitgeist. It's a powerful, unarguable fact that may alter your viewing or wagering habits. It may even change the way the game is played.
Here's what we discovered: teams that execute two or more "quarterback kneel" plays win 90 percent of the time. Teams that execute one or fewer kneels only win 46 percent of the time. And while winning teams knelt 1.65 times per game from 2000-06, losing teams have averaged just 0.25 kneels per game.
The data is clear and irrefutable: teams that kneel on the ball win far more often than teams that don't. What's more, the margin of victory correlates closely with the number of kneels: teams that kneel more win far more blowouts, while teams that kneel infrequently are involved in far more tight victories.
There's only one conclusion to be drawn from the data. If you want to win, you have call more kneel plays.
Our Kneel to Win theory had the potential to turn the football world on its ear, but at first it was just a rough diamond that needed to be cut and polished. We spent the summer applying state-of-the-art scientific methods to the theory. The results were fascinating.
In games in which the quarterback did not kneel at all, teams average 18.4 points per game. When the quarterback knelt once in a game, teams averaged 20.2 points. In games with two kneels, teams averaged 22.1 points per game. By simple linear regression (a method that relates one variable to another and allows statisticians to extrapolate predictions), it follows that if a team calls 40 quarterback kneels in a game, they will score 92.4 points per game.
Clearly, offensive coordinators are missing the boat with all of their rushing and passing and West Coast Jet Smoke gobbledygook. The data indicates that the easiest way to score in the NFL is not to move the ball at all. Further research proved that the same trend is prevalent at the college and high school levels.
How can teams score more points while kneeling more often? After studying the statistics, we developed several theories. A team that kneels more often is clearly more rested than a team that doesn't, and you can't overestimate the effect of fatigue on football players. It's almost impossible to injure a kneeling quarterback, so teams with a kneel-first philosophy won't have to rely on a backup in a key game (unless the starter gets tendonitis in his kneeling knee). And constant kneeling would allow a team to dictate the tempo of the game while projecting an air of confidence and a winning attitude. Great teams always play with confidence and a winning attitude.
We approached several NFL offensive coordinators about our Kneel to Win theory. We even had some interns devise a "kneel chart," that would show coaches what game situations call for a quarterback kneel. The coordinators were polite, but skeptical. They think of their phonebook-thick playbooks as children, and they are reluctant to part with them. They just couldn't comprehend a truly scientific, visionary approach to their sport. It's a shame, because according to the data, we could be watching football games in which both teams snap the ball and genuflect for 60 minutes, with scores that rival the NBA in the 1980s.
The Kneel to Win theory is pretty stupid, isn't it? Obviously, anyone who thinks that kneel plays lead to victories doesn't realize the difference between correlation (the relationship between two variables) and causation (one variable directly affecting the other).
In the example above, the relationship between the variables is exactly backwards. Kneeling does not cause winning. Winning causes kneeling. Every 10-year-old knows this, and anyone who thinks otherwise needs serious psychological attention.
So we can all chuckle at the Kneel to Win theory. We're all smarter than that. But then, we pick up the local newspaper and read pre-game analysis like this: "The Home Team has to run the ball more often this week. When they run the ball 30 times, they win 90 percent of the time."
Here we go again, folks.
The relationship between running the ball and winning is more complicated than the relationship between kneeling and winning. A good running game certainly contributes to the winning effort. But teams don't win because they run 30 times. Teams run the ball 30 times because they are protecting the lead on their way to a win. It's a fact every thoughtful fan appreciates, and the Run to Win myth was debunked decades before Football Outsiders came into existence. Yet sportstalk hosts and local columnists still trot out the Run to Win theory as if they think that 30 straight handoffs to start the game will guarantee a victory by statistical fiat. That's as silly as thinking that kneeling on the ball is the key to success.
Aaron Schatz founded Football Outsiders three years ago to combat just this sort of lazy analysis. Boston media pundits consistently hammered the Patriots in 2002 for not running the ball often enough; the team ran far more often in their Super Bowl season than they did during that disappointing 9-7 campaign. Why couldn't they see that they were throwing the ball too often? Aaron ran the numbers [1] and proved that the Patriots run-pass ratio early in games changed little from 2001 to 2002, but in 2001 they ran to protect leads in the second half while in 2002 they passed to catch up. The experts had it backwards. While conducting his research, Aaron laid the groundwork for the statistics that would become DVOA and DPAR, paving the way for him to launch the top independent football research site on the Internet.
Still, the Run to Win theory survives and thrives. It has corollaries, too. There's the He Throws Too Much corollary: Brett Favre isn't that great, because the Packers are 7-15 (or whatever) when he throws over 40 passes per game. Of course, Favre only throws 40 passes when the Packers are already trailing, but never mind. And there's the Good Teams Run and Stop the Run postulate, which extends Run to Win to defense. And there are other nuggets of conventional wisdom, like Good Teams Find a Way to Win Close Games, that have little factual basis but live on in the minds of fans and sportscasters.
Of course, if everybody took a careful, thoughtful, analytical approach to football, Football Outsiders would have a hard time standing apart from the crowd.
Kneel to Win is a snarky parody of the misuse of statistics. But it's also a cautionary tale. Even the most careful, well-meaning statisticians can screw up. At FO, we like to think that we provide the best analysis on the web. But how do we know that we aren't just advancing pseudoscientific claptrap, Kneel-to-Win silliness embellished with a veneer of mathematical precision?
Numbers can lie, they can mislead, they can obscure, and they can send you on mathematical wild goose chases. Sometimes, there's no way to measure the phenomenon you're trying to study. Sometimes, the statistics you want to use to rate a player's performance – like tackles for cornerbacks – are the wrong tool for the job. Because of this, some analysts throw all football stats into the circular file, except the ones that "prove" whatever point they want to make at any given moment. Other observers do the best they can with the numbers that are available, massaging them every which way in a search for underlying truth. That's more or less what we do at Football Outsiders, and we've come up with some mathematical models that rate player and team performance well and, more importantly, do a great job of predicting future performance.
There are other models and systems available, many of them very enlightening and useful, most of them very complex. But just because a statistical model is complicated and scientific-looking doesn't mean it's correct. Just because a writer has a degree in economics or quantum physics doesn't make him right. And just because a conclusion jibes with our gut instincts doesn't mean it's the right conclusion.
DVOA is a great statistical model, but not because it's built on thousands of calculations, or because Aaron is really smart, or because we all just "know" that the best teams must move the ball consistently and convert third downs. It's not a great model because we have a popular website or a deal with FOXSports.com or because Pro Football Prospectus is our ball and we'll take it home if you don't let us play quarterback. DVOA works because it makes excellent predictions, week in and year out, exposing pretender teams with gaudy records and often recognizing rising stars just before they start their ascent. And last year, when the Colts played their Sun Belt Conference schedule and the flaws in the DVOA strength-of-schedule adjustments were exposed, we got out the wrenches and fixed the system, making it more accurate not only for the Colts or the 2005 season, but for every team in the past decade.
Any complex statistical model like DVOA is only as sound as the ideas underlying it. And those ideas are only worthwhile if they are grounded in reality. That sounds simple, but it's a fact often forgotten by analysts who make the Run to Win or Kneel to Win mistake: they start believing their tables and graphs and ignoring what really happens when young men walk onto a gridiron.
Accurate statistical analysis of football requires three things. One is a knowledge of statistics. The second is a knowledge of football. You don't need a BA in mathematics and a decade of coaching experience to do stat analysis; if that were the case, Bill Belichick would be the only person in the field. But you need to know what regression analysis is, and you should be able to spot zone coverage or a trap block when watching a replay.
But the third necessity is crucial: the drive to constantly expand your knowledge of the first two factors. A good analyst strives to make himself a better mathematician and a better football guy. He watches more games, reads more books, talks with more coaches and other experts, all the while experimenting with new calculations and testing new hypotheses. Without the drive to improve, an analyst might develop a "good enough" system, then spend the rest of his energy defending it instead of proving it. In effect, he makes a modest gain, then kneels on the ball.
That's something we will never do. The watchword at Football Outsiders is always "improvement:" adding relevant variables to projection systems, revising and retesting DVOA and KUBIAK and our other stats, accumulating more data, interpreting that data, achieving better results and more accurate conclusions, eliminating spelling errors (that last one's the toughest). We strive to improve so our content is better, but we also do it to learn. As long as we keep learning, our analysis, while never perfect, will keep moving forward, steering wide of the Run to Win/Kneel to Win ditch of faulty logic.
The Football Outsiders are sometimes called the "stat geeks." In Pro Football Prospectus 2006, Aaron counters that we are really "philosophy geeks," guys who look for true team qualities that transcend wins and losses. I would shed the geek label altogether (if only) and suggest that we're simply the guys who triple check. We don't take anything at face value, even our own conclusions. We put conventional wisdom to the test. We record hundreds of games, scrutinize every play, verify every statistic. We learn as much as possible, so when we post a new statistic or power ranking or conclusion, you can be sure it's the result of keen observation, the critical starting point for scientific thought.
So as you watch football this season, be on the lookout for bad stats, backwards stats, stats that are too good to be true, stats that are too complex to be deciphered, stats that miss the forest for the trees, stats built on a foundation of ignorance or wishful thinking, good stats used badly, and bad stats used indiscriminately. Get ready to wince when the announcer says that some lousy team has the third best pass defense in the league (could it be because no one has to throw the ball against them?), that Team X is great because they have great roster stability (maybe there's little turnover because the team wants to keep quality players) or that the home team is 16-0 when Joe Workhorse gets 30 carries. Greet everything with a skeptic's eye, even what you read in Too Deep Zone or at Football Outsiders. We're all learning about football together, and the more we learn, the faster we can replace "conventional wisdom" with actual knowledge.
Links:
[1] http://www.footballoutsiders.com/2003/07/14/ramblings/stat-analysis/3/