Stat Analysis
Advanced analytics on player and team performance

XFL Conversions: As Easy as 1, 2, 3?

Seattle Dragons at D.C. Defenders
Photo: USA Today Sports Images

Guest column by Zach Drapkin

Ten and a half minutes into the first game of the 2020 XFL season, Seattle Dragons quarterback Brandon Silvers connected with wideout Austin Proehl for a 14-yard score and the first-ever touchdown in the alternative league's highly anticipated reboot.

Next, it was time for Seattle coach Jim Zorn to decide how to approach the point after touchdown. Under the new XFL rule changes, teams can choose between three options after scoring a touchdown: go for one point from the 2-yard line, two points from the 5-yard line, or three points from the 10-yard line. No matter what, teams have to run a play; there's no option to kick an extra point.

Zorn, a long-time NFL player turned coach, chose to go for one. That was the wrong move.

One week in, Zorn and the rest of the XFL coaches clearly seem to think the one-point try is a smart call, having lined up at the 2-yard line following 11 of the 19 touchdowns scored in the league's inaugural set of games. That's understandable, given that going for one is a "safe" play which closely mirrors the NFL's traditional two-point conversion.

But the data tells another story. In fact, teams should almost never be attempting a one-point conversion.

So, what is the "correct" or "optimal" call?

It would be impossible and unwise to answer this question using existing XFL data, given that there are just four games and 19 conversion attempts to work with. However, if we assume that the relative strength of NFL offenses and defenses mirror those of XFL teams, there are decades of NFL data which can serve as a valuable proxy for solving this scenario.

Inspired by a recent discussion on the Wharton Moneyball radio show, I decided to give it a shot.

In order to best capture the conditions of an XFL conversion, I gathered NFL regular season play-by-play data from 2010 to 2019 (thanks to nflscrapR) and looked at just the runs and passes that came on third and fourth down with goal-to-go inside the 10-yard line. While I could have simply calculated the rate at which teams scored on these plays from the 2-, 5-, and 10-yard lines, that approach misses out on a large selection of data; all the other similar plays from the 1-yard line, for example, would just be thrown away.

Instead, I built a logistic generalized additive model (GAM) to predict the probability of a successful conversion from each yard line inside the 10, including all 3,592 relevant plays from inside the 10 in order to construct a stronger, smoother model. By resampling the entire data set 1,000 times and refitting the model each time with a fixed smoothing parameter, I constructed 95% confidence intervals for the estimate at each yard line.

Unsurprisingly, the expected chance of scoring is highest at the 1-yard line (56.14%) and lowest at the 10 (21.15%), but the difference between each of the estimates is what gives us the more valuable insight. Moving further away from the end zone, there are diminishing marginal losses in conversion probability, remarkable enough that the predicted chance of scoring from the 5-yard line is statistically inseparable from the predicted chance of scoring from the 7.

To evaluate the relative value of different XFL conversion attempts, I converted each relevant probability estimate into an expected point value. To do this, I simply multiplied the probability estimate at each specified yard line by the number of points for its respective attempt -- i.e., for the two-point conversion, I doubled the estimated conversion chance from the 5-yard line, and for the three-point conversion, I tripled that of the 10-yard line.

The results are resoundingly against the one-point conversion. At 0.473 expected points, going for one is considerably worse than either a two-point or three-point conversion. Between those other two options, however, there isn't a clear winner.

The expected points estimate is slightly higher for the two-point attempt, but due to the high variance of both estimates, and especially that of the three-point conversion, the difference between the two values is not statistically significant. This can be seen with the yellow bars in the above figure, which represent the 95% confidence interval for each estimate.

It's worth noting that this procedure doesn't take into account the possibility of the defense creating a turnover and scoring a defensive conversion. Given that there were only eight defensive scores in the 3,592 plays analyzed and no more than two of those scores occurred from the same yard-line, the potential effect from including defensive scores in the model would be unlikely to cause a meaningful change in results.

If the two-point and three-point conversions are indistinguishable, how should coaches proceed?

In some situations, the choice is obvious. If a team is trailing by three points near the end of a game, it should clearly try to tie the score. Even a one-point conversion can be preferable in some high-leverage spots; if a touchdown ties the game with little time remaining, an attempt from the 2-yard line is the surest shot at winning.

Without an available win probability model for the XFL, the decision as to when a team should go for two or three rides on the variance associated with each choice. The default option should be to go for two, given that it produced the highest expected point value and is the less volatile choice. However, if the team with possession is a clear underdog or is behind by a large margin, then that team should go for three; the more variance introduced, the higher the chance of an unlikely outcome.

To summarize, XFL coaches have got it all wrong. As tempting as it is to play it safe and choose the option with the highest conversion probability, doing so leaves points on the table. Thus, a one-point conversion should be reserved for rare situations, and coaches should go for more two- and three-point attempts. Between those two options, the three-point attempt should be used by teams with lower chances of winning; otherwise, going for two is likely the most desirable choice.

It has taken years for NFL coaches to learn when to attempt a two-point conversion. In the new XFL, full of progressive rule changes and procedures, we're about to see whether coaches can escape falling victim to the same conservatism that dominates traditional football conventions.

Zach Drapkin is an undergraduate at the Wharton School studying Statistics and Decision Processes. He works as an assistant producer for the Wharton Moneyball podcast/radio show and is pursuing a career in sports analytics. Follow him on Twitter @ZachDrapkin.

Comments

19 comments, Last at 19 Feb 2020, 12:41pm

1 momentum counts

This analysis is interesting, but it neglects the emotional importance of the binary difference between succeeding and failing on the attempt. In the NFL a two-point try failure is a pretty big momentum swing in favor of the defense, whereas a successful two-point try is only a moderate boost to the offense after having already scored a touchdown (unless, say, the 2 points ties the game). So the coach's decision should not be based purely on the expected points gained on the try. They have to consider the momentum effect on the rest of the game as well.

7 +1. "Momentum" as most…

+1. "Momentum" as most people perceive it changes VERY easily. Anytime anyone uses it in an argument, I tend to tune out. 

Emotion can be important I think... but there are always highs and lows and I don't think it's predictive in any way.

2 It seems like a mistake to…

It seems like a mistake to include third downs. The availability of the FG affects strategy for both offense and defense.

10 Maybe not

Excluding 3rd downs would probably drop sample size by 80% or more.  Teams almost always go for the TD on 3rd and goal from 10 or less, but on 4th the not-kicking proportion might range from 50% at the 1 down to 5% at the 10.

11 Can confirm sample size…

In reply to by SandyRiver

Can confirm sample size drops to 442 observations by only including 4th down, and even then results don't really change. 

3 > if we assume that the…

> if we assume that the relative strength of NFL offenses and defenses mirror those of XFL teams That's a pretty big one. I haven't watched any XFL yet, but my impression of the AAF was that the relative strengths were comparable to the NFL, except at QB. QBs were relatively worse than other positions

5 I watched all four XFL games…

I watched all four XFL games (I know, I need to get a life) and, you're right MT, QBs were relatively worse than other positions compared to the NFL though some of that could be due to the minimal practice time they've had with their offense.

12 Have watched half of two…

Have watched half of two different games now and the play is pretty terrible all around, but, yes, especially at QB, which is putrid, though the offensive line play is also very noticeably "extra worse". 

14 No stats here, but from many…

No stats here, but from many years observing the CFL, I'd say it typically takes 2 - 3 years as a back up followed by 8 to 16 games as a starter before a QB can be counted on to be anything more than a "game manager".

The CFL game differs from the college game much more than the XFL rules do, and at this point I expect the average XFL defender would be a little slower than the average CFL defender.  So maybe the learning curve will be a littler quicker in the XFL, but it shouldn't be surprising if it's not until the end of this season or next season at the earliest before some of the QB-play picks up, and season three before most of the teams are able to field dangerous QBs.

That's assuming the guys filling those roles right now have the ability to grow into the role, and just need the reps and game time to do so.

 

6 In the XFL Season Preview I…

In the XFL Season Preview I commented that I like the 1, 2 and 3 points after a TD, I don't like the way they are doing it. By making 6 points the most likely TD outcome, I think it will create more overtime games which is counter to their "keep the game under 3 hours" goal. (7 points beats two field goals)

I would stick with the NFL system but add go for 3 from the 10 (or some such).

Or make TDs worth 7 points and do the point(s) after 1, 2 or 3 from the 2, 5 and 10 yd. lines.

9 Field Goals are more valuable

I agree with Sixknots but for a different reason. Since the points after is so often failed, making a touchdown often worth only 6 points, all of a sudden field goals are more valuable in the XFL than they are in the NFL. I don't think this was their intention. But I'm guessing that the effect will be less going-for-it inside the 40 in the XFL vs. the NFL. 

Assuming the XFL survives for a few seasons, I wonder if they'll end up pulling the 1-pt conversion into the 1 or 1.5 yard line.

15 Point(s) after

1. I remember hearing or reading right before the season started that the XFL had already done the analysis (I assume very similar to this), and it was stated that the 2 point conversion was the best mathematically.
2. I have only watched the highlight videos of all 8 games--but it seems like their conversion rates are not good.
3. Considering the other comments on this thread about QB play, and observing this on the highlights and stats, I would guess that some coaches are thinking that a running/option play from the 2 is their best chance to get an additional point.
4. Anecdotally, I have heard that O-line play is pretty sub-par, and this tracks with what NFL fans say too. Also, we know that good blocking requires at least 4, if not all 5, linemen doing their job in conjunction with each other--where one D lineman can make his teammates look better than they are individually. We also know that in general, run blocking is easier than pass blocking; and that runs overall average about 4 yards. However, this also decreases inside the 10, where defenses do not have to play their safeties deep. Thus, a running play from the 5 or 10 is almost guaranteed to fail--whereas one from the 2 has a reasonable chance of succeeding--especially if it involves misdirection, an option, etc.
5. See comment 9 by Seattle-Brian--any extra point(s) makes them worth more than 2 FG's.

Now, I am not disputing the analysis given here--in fact, I would have guessed the overall conclusion, based on similar analyses of the NFL's point after rules. I am just trying to offer some logical reasons why the XFL coaches are acting the way they are, considering that I remember hearing or reading that the coaches were informed of similar analysis.

16 Your intuition is correct…

In reply to by Joseph

Your intuition is correct. Based on my tracking, there have been 39 attempts so far, with 57 potential points. 11 have been successful (28.2%), for 17 total points (0.298 PPA - points per attempt). Here's the breakdown by try:

1 pts: 6/23, 26.1%, 0.261 PPA

2 pts: 4/14, 28.6%, 0.571 PPA

3 pts: 1/2, 50%, 1.5 PPA

Moral of the story: Go for 3 everytime!!

More seriously, hard to draw conclusions with these tiny sample sizes, but 1-pointers ain't lookin too good. Better off going for 2.

17 I still think the most fun…

I still think the most fun way to do conversions is to require the guy who scores the TD to be the only one who can KICK a PAT.