Guest column by Zach Drapkin
Ten and a half minutes into the first game of the 2020 XFL season, Seattle Dragons quarterback Brandon Silvers connected with wideout Austin Proehl for a 14-yard score and the first-ever touchdown in the alternative league's highly anticipated reboot.
Next, it was time for Seattle coach Jim Zorn to decide how to approach the point after touchdown. Under the new XFL rule changes, teams can choose between three options after scoring a touchdown: go for one point from the 2-yard line, two points from the 5-yard line, or three points from the 10-yard line. No matter what, teams have to run a play; there's no option to kick an extra point.
Zorn, a long-time NFL player turned coach, chose to go for one. That was the wrong move.
One week in, Zorn and the rest of the XFL coaches clearly seem to think the one-point try is a smart call, having lined up at the 2-yard line following 11 of the 19 touchdowns scored in the league's inaugural set of games. That's understandable, given that going for one is a "safe" play which closely mirrors the NFL's traditional two-point conversion.
But the data tells another story. In fact, teams should almost never be attempting a one-point conversion.
So, what is the "correct" or "optimal" call?
It would be impossible and unwise to answer this question using existing XFL data, given that there are just four games and 19 conversion attempts to work with. However, if we assume that the relative strength of NFL offenses and defenses mirror those of XFL teams, there are decades of NFL data which can serve as a valuable proxy for solving this scenario.
Inspired by a recent discussion on the Wharton Moneyball radio show, I decided to give it a shot.
In order to best capture the conditions of an XFL conversion, I gathered NFL regular season play-by-play data from 2010 to 2019 (thanks to nflscrapR) and looked at just the runs and passes that came on third and fourth down with goal-to-go inside the 10-yard line. While I could have simply calculated the rate at which teams scored on these plays from the 2-, 5-, and 10-yard lines, that approach misses out on a large selection of data; all the other similar plays from the 1-yard line, for example, would just be thrown away.
Instead, I built a logistic generalized additive model (GAM) to predict the probability of a successful conversion from each yard line inside the 10, including all 3,592 relevant plays from inside the 10 in order to construct a stronger, smoother model. By resampling the entire data set 1,000 times and refitting the model each time with a fixed smoothing parameter, I constructed 95% confidence intervals for the estimate at each yard line.
Unsurprisingly, the expected chance of scoring is highest at the 1-yard line (56.14%) and lowest at the 10 (21.15%), but the difference between each of the estimates is what gives us the more valuable insight. Moving further away from the end zone, there are diminishing marginal losses in conversion probability, remarkable enough that the predicted chance of scoring from the 5-yard line is statistically inseparable from the predicted chance of scoring from the 7.
To evaluate the relative value of different XFL conversion attempts, I converted each relevant probability estimate into an expected point value. To do this, I simply multiplied the probability estimate at each specified yard line by the number of points for its respective attempt -- i.e., for the two-point conversion, I doubled the estimated conversion chance from the 5-yard line, and for the three-point conversion, I tripled that of the 10-yard line.
The results are resoundingly against the one-point conversion. At 0.473 expected points, going for one is considerably worse than either a two-point or three-point conversion. Between those other two options, however, there isn't a clear winner.
The expected points estimate is slightly higher for the two-point attempt, but due to the high variance of both estimates, and especially that of the three-point conversion, the difference between the two values is not statistically significant. This can be seen with the yellow bars in the above figure, which represent the 95% confidence interval for each estimate.
It's worth noting that this procedure doesn't take into account the possibility of the defense creating a turnover and scoring a defensive conversion. Given that there were only eight defensive scores in the 3,592 plays analyzed and no more than two of those scores occurred from the same yard-line, the potential effect from including defensive scores in the model would be unlikely to cause a meaningful change in results.
If the two-point and three-point conversions are indistinguishable, how should coaches proceed?
In some situations, the choice is obvious. If a team is trailing by three points near the end of a game, it should clearly try to tie the score. Even a one-point conversion can be preferable in some high-leverage spots; if a touchdown ties the game with little time remaining, an attempt from the 2-yard line is the surest shot at winning.
Without an available win probability model for the XFL, the decision as to when a team should go for two or three rides on the variance associated with each choice. The default option should be to go for two, given that it produced the highest expected point value and is the less volatile choice. However, if the team with possession is a clear underdog or is behind by a large margin, then that team should go for three; the more variance introduced, the higher the chance of an unlikely outcome.
To summarize, XFL coaches have got it all wrong. As tempting as it is to play it safe and choose the option with the highest conversion probability, doing so leaves points on the table. Thus, a one-point conversion should be reserved for rare situations, and coaches should go for more two- and three-point attempts. Between those two options, the three-point attempt should be used by teams with lower chances of winning; otherwise, going for two is likely the most desirable choice.
It has taken years for NFL coaches to learn when to attempt a two-point conversion. In the new XFL, full of progressive rule changes and procedures, we're about to see whether coaches can escape falling victim to the same conservatism that dominates traditional football conventions.
Zach Drapkin is an undergraduate at the Wharton School studying Statistics and Decision Processes. He works as an assistant producer for the Wharton Moneyball podcast/radio show and is pursuing a career in sports analytics. Follow him on Twitter @ZachDrapkin.