Extra Points
News and commentary from around the Web

EdjSports Coach Rankings

John Harbaugh
Photo: USA Today Sports Images

The debut of the new EdjSports coach rankings comes with a video talking about some of the best and worst coaching decisions of the 2019 season so far.

View Full Article


21 comments, Last at 23 Nov 2019, 9:54pm

1 It looks like this new…

It looks like this new rating engine is going to need a whole lot of work...

According to the engine, I note that the Indianapolis Colts' Frank Reich's best decision of the year happened in the Carolina Panthers game against the Los Angeles Rams, when the panthers punted from their own 43 on 4 and 2 in the first quarter of a 0-0 game. It was also apparently Ron Rivera's worst call of the year....

4 email fail

The routing for the "questions or comments" email at the bottom of the page is set up wrong. It looks like it redirects to an address that only accepts internal emails.

edit: info[at]... routes to inquiries[at]... which sends the original sender some fun info.

5 I love the graphic by…

I love the graphic by Washington, good job whoever did that.

I haven't watched the video, so apologies if this info is in there, but how much gap is there from best to worst, or best to average?

6 Risk/Reward

Forth and 2 from their own 28, up 5 with 4:14 to go in the 4th quarter, and the Packers should not have punted? Advanced analytics, meet the sniff test.

11 That sounds perfectly…

In reply to by alljack

That sounds perfectly reasonable. Two yards isn't much. You have the chance to salt away the game if you get a good drive going. On the opposite side, while failing on the fourth down would be bad, it does have an upside. Even if the opposing team scores, they likely won't be able to burn up all of the clock. The Packers would still have time to drive for winning or game tying FG.

16 I saw that too and I agree…

In reply to by alljack

I saw that too and I agree. I've seen MLF make a couple dumb decisions so far, but that wasn't one of them. The defense was playing good. There was no reason to give them a shot at great field position. Plus, the offense hadn't shown much after the first quarter. There was no good reason to go for it. They had a fourth and short at about their opponents 40 recently (Carolina?) and didn't go for it. I thought that was a far worse punt. 

20 If you go for it on fourth…

In reply to by alljack

If you go for it on fourth-and-2, the most likely scenario is you make it. Hooray! If you don't get it, and your opponent is down five, the most likely scenario is you give up a go-ahead touchdown ... but thanks to the short field, you would probably still get the ball back with enough time to drive for a game-winning field goal. And there's always the chance you'll keep them out of the end zone in the first place.

If you punt, now your opponent has the ball, and the whole field AND four downs to work with, and the odds they eat up all that clock and score the winning touchdown without ever giving you the ball back are much higher. 

I admit that I am on a plane right now and can't actually check all this math, but the theory seems sound to me.

21 Moderate Risk / Uncertain Reward

In 2018, GB converted three of four two-point conversion attempts, also 2 yards. The suggested high probability of success if the Packers go for it does not overcome the instinctive aversion to risk among fans and coaches. The position on the 28 yard line and the relative strengths of offense and defense as noted above argue for the cautious move.

Also, the difference between four feet to go and six feet to go is significant, although both would be called 4th and 2.

7 I'm conflicted about the use…

I'm conflicted about the use of analytics to evaluate SINGLE decisions. Knowing what the average outcome of some event is and how that changes the average probability of winning a game would appear insufficient to make the ideal decision in any single situation. Statistical analysis works because a ton of "noise" is averaged out, but what is "noise" in the overall scheme of things can be an incredibly important data point in the individual case. Important player on either team injured or even just gassed? Snowing? Windy? Opponents unable to stop the run all game?

It seems much more reasonable to apply these statistics to the overall decision making in certain situations, although there are serious statistical issues with situations that do not occur often! For instance, I'd love to see the statistical background used to evaluate the decision to go (or not) for it on 4th down and 2 on your own 28 up 5 with 4:14 remaining.

In particular, I would love to see the data underpinning the punt-on-long-to-go-while-behind-deep-in-own-territory-late-in-game. How many times have teams really gone for it in those, or sufficiently similar, situations? Odds are, if you are behind, you aren't converting 4th-and-long at league average rates under normal circumstances.

Additionally, the argument that coaches are generally not aggressive enough in their play-calling is a constant refrain (and correct, IMHO), but it should be noted that this lack of aggressiveness directly influences opponent expectations, increasing the positive outcomes. One would, naively at least, expect some regression once a given coach (or all coaches) get a reputation for more aggressive play-calling.

9 I don't think they're using…

I don't think they're using league average rates for their decisions. Their probability graphs don't start near 50%, so there has to be at least some weighting. Of course, that says nothing about the other confounding factors like weather, injuries, exact matchups, info from that game, or even if the weighting is done right.

I think going for long 4th downs in your own territory while down late in the game is probably an advantage. Punting and failing both are likely to lead to running out the clock, or putting you in a significantly worse position. The extra field position the other side gets is pretty unnecessary. Are they going to make 4th and long often? No, but are they gonna win often if they get the ball back with 30 seconds left and no time outs? Bigger no, if they even get it back.

Your complaint about lack of data seems off. We don't need to know how often someone down big goes for the long 4th downs, just how often the long 4th downs convert. After that, it's just 1st down to whatever side. Some numbers thrown out of my ass. Say 25% chance to make the 4th down. 10% chance to win if you make it 1% if you don't, and 2% if you punt. Going for it is 3.25% chance to win. Punting is 2% chance to win. Go for it and miss, there's no hope. Punting is a delaying tactic to keep the definite loss later. It's like delaying going for 2 late in the game when you know you'll need one. If you go and miss, it's nearly insurmountable, but you have a chance. If you wait, your chance stays higher than the miss case, but then if you miss, you're completely done.

I have more issues with going for 4th down in your own 30 while up 1, 2 or 3 late. Missing is pretty close to ending in a loss. Maybe I'm wrong, and the conversion rates do say it's right, but that one's hard for me.

The last paragraph is generally true, but fails on fourth down decisions, which is where the coach rankings come from. For fakes, it works, but once a team sends their offense out, there's no surprise.

10 I think the point is maybe…

I think the point is maybe all 4th and longs convert 25% of the time, but a specific 4th and long might have widely different odds depending on a number of factors, including, for example, the fact that you're, say, 1/10 on 3rd downs for the day (if you're getting your ass handed to you). I think it's a good point.

12 Equally if you are already a…

Equally if you are already a heavy favorite in a game, you should generally be seeking to reduce variance, even at the expense of tiny theoretical reductions in win %. For example, I noted some criticism of Bill Belichick for punting on 4th & short earlier early in one of the games where the Patriots were overwhelming favorites earlier this season. Clearly Belichick isn't stupid: he knew that avoiding such high leverage situations was the surest path to victory. 

13 That's already agreed to in…

That's already agreed to in my first paragraph.

Still, though, nobody goes for it in these situations. There have to be some cases where a team is playing to their average 4th down shot, or at least close enough that it's the right decision.

18 On the subject of datapoints…

On the subject of datapoints... Field position matters. How many fourth and long from e.g. own 20-30 have there been and can those teams be considered in any way representative of 1) the league or 2) the actual team in question? Anecdotally, it sure seems that defenses run a very different defense when they have opponents "on their back foot" late in games. What might sometimes be a "prevent" style defense will other times be an all-out blitz. (If you've been watching the Dolphins this season, there have been several examples of either option.)

On the subject of "surprise"... How many times have you seen a defense have to take a timeout or simply live with having the wrong personnel on the field because they were surprised by the offense's decision to go for it? Or maybe the offense wasn't going to go for it, but simply try and get an offside penalty on the D? Or maybe the D *thought* that's what the O was going to do, but in actuality they intended to go for it all along? These things happen often and while it may just be a bit of noise in the grand scheme of things, when you are looking at a SINGLE coaching decision, then that "noise" could very well be the signal.

15 But I thought the whole…

But I thought the whole point of this was to apply it to critical plays (4th and whatever, kick FG  or go for it,  4th and goal, go for PAT or 2ptr). 

Other than that, I have no idea how these rankings are compiled, based on the info provided.

8 Maybe include a coaching consistency variable?

Nagy’s CCI is not surprising (low 20s) due to poor tactical in game decision making.

But if he is even worse when you layer in consistency as a measure of - “do they have an analytical framework?”. Because he’s all gut and his instincts are terrible.

Matt Nagy goes for it on 4th and 9 one week in the dead zone, punts on 4-2 the next possession in the dead zone, kneels down for a long FG the next week and punts 4-2 on his own 30 in middle of 4Q when trailing and then never gets the ball back. Just a few examples of horrible decision making.

19 it's a problem

That all thirty-two "best calls" are to "go for it" and all thirty-two "worst calls" are punt/FG/kicking related.

Makes me believe that some bias is baked into this approach.

There is a fundamental problem trying to predict success rates for 4th down attempts by looking at historical data: the data is already self-selecting. The coaches are not randomly selecting strategies, but rather they already are considering the relative strength of their offense vs the opponent's defense. And if they think the odds of success are low, they will avoid the attempt.

That fact means that success rates will be artificially inflated.

Can this bias be accounted for? Perhaps. But I won't believe that it has been accounted for until I see explicitly that this has been done.