Bill Belichick Fourth-and-2 Calculator

Thanks to Mike Harris, creator of the playoff odds simulator, for creating this Bill Belichick fourth-and-2 simulator. You can enter your own estimate of the probabilities of the Colts scoring either after a failed fourth down or a punt, and then find out if Belichick made the right decision based on your own opinions.

View Full Article


46 comments, Last at 18 Nov 2009, 1:56am

1 Re: Bill Belichick Fourth-and-2 Calculator

/ceases own work on an excel spreadsheet
/plays around with calculator
/goes back to doing something that I actually get paid for

Thanks, Mike!

3 Re: Bill Belichick Fourth-and-2 Calculator

I think this starts to miss the larger point: Belichick's decision about whether it was "right" essentially comes down to each persons' subjective assessment of the circumstances affecting the general odds for the expected payoff. To me, this means that Belichick's decision was, at a minimum, understandable. Even if it was technically wrong, the call was clearly close enough that I don't think he should get much heat for it. As many others have stated, if we're going to second-guess the late-game decisions by Belichick, the lack of foresight on third down and the wasted timeouts were the real miscues.

ETA: Also, those verdicts are pretty funny.

31 Re: Bill Belichick Fourth-and-2 Calculator

I absolutely agree. You can argue for or against the decision he made, it all comes down to subjective probabilities and a risk analysis. No one can argue that they knew exactly what those probabilities were. However, those timeouts, or at least the last one, was inexcusable. If he had the ability to challenge the spot, there's probably a 50/50 it gets moved in his favor. Also, as you hinted, you ought to know on first down if you are going to go for it on fourth if you are close, and plan accordingly. Those were the bad calls, not so much the going for it on fourth.

4 Re: Bill Belichick Fourth-and-2 Calculator

For anyone that read Peter King this morning, if you punch in his estimates and assume the Colts have a 90% chance of scoring from the 30 the numbers come out 68 to 64 in favor of going for it. Not exactly Grady Little...

7 Re: Bill Belichick Fourth-and-2 Calculator

It was a good call, because Manning was scoring the touchdown no matter what happened.

2 minutes and 70 yards...with timeouts. Manning scores on 2 minute drives when he ISN'T in the hurry-up.

9 Re: Bill Belichick Fourth-and-2 Calculator

I still think the biggest problem was the play call. If you're going for it on 4th down you should run it on 3 and 2. 2 sneaks/runs will get you 2 yards against the Colts.

11 Re: Bill Belichick Fourth-and-2 Calculator

Having had ESPN and then Colin Cowherd on in the background all morning, I have to say it was a lousy decision just because it gave these self-righteous a-holes something to talk about non-stop. God, I'm an idiot for not changing the channel hours ago, and this is the last time I watch ESPN for months except during actual sporting events.

12 Re: Bill Belichick Fourth-and-2 Calculator

That's a product of their obsession with "stories" and the game itself rather than the decision. If BB had called for a punt and it worked, it would have been "ZOMG why didn't Colts run X?" If the Colts returned it for a TD, it would have been "ZOMG why didn't he go for it?"

and so on ... micro"analysis" of this game will continue until they meet next season.

27 Re: Bill Belichick Fourth-and-2 Calculator

"Having had ESPN and then Colin Cowherd on in the background all morning,"

That was your first mistake.

The next time Colin Cowherd provides intelligent analysis on any subject will be
the first time.

If you want to know what the right opinion is to have on a topic, find out what Cowherd thinks, and assume the opposite position.

13 Re: Bill Belichick Fourth-and-2 Calculator

The thing is, the probabilities of scoring from the 30 and from a long field are inherently related - there's no reason to let people choose them in a vacuum. If you treat the field as independent chunks of yards, it should be roughly exponential: whatever probability you assign them of scoring from the 30, raise this to the power of 2.5 and you get the probability that they score from their own 25 (which makes a generous assumption of a 47-yard net on a punt.) Obviously, this method would break down if time were a significant factor, but with two minutes left, I think it should do a good job of tying the probabilities together.

25 Re: Bill Belichick Fourth-and-2 Calculator

Just read the Chase Stuart article, in which he pretty much says straight out that this doesn't work. Not sure I'm entirely convinced, however, as if the last yards really were disproportionately difficult, the linear approximation of expected points relative to field position wouldn't fit as closely as it does (I know there is slight non-linearity near the goal line, but not pronounced.) Intuitively, it seems that the fact that the defense only has to defend a short field near the goal line should be counterbalanced by the fact that they may be tired at the tail end of a drive. So I don't see why treating the field as independent sections fails to generate at least a good approximation.

29 Re: Bill Belichick Fourth-and-2 Calculator

"Intuitively, it seems that the fact that the defense only has to defend a short field near the goal line should be counterbalanced by the fact that they may be tired at the tail end of a drive."

But that wouldn't be true of a drive on a short field. I'm not convinced that it's true anyway, but I imagine Aaron et al have pretty accurate numbers on that sort of thing, as an input to the DVOA formula.

Personally, I have a really, really hard time coming up with a set of probabilities that I don't find outlandish which don't say go for it.

Note, though, that this finding is game-specific. If I'm Belichick, facing the Colts, I go for it. Don't even have to think about it. If I'm Marvin Lewis, in exactly the same situation in some parallel version of Sunday's game at Heinz Field, I punt, punt, punt, punt, punt. Somewhere out there is a matchup where I would consider it a wash - and in that universe it's fake punt time. Who am I kidding? It's always fake punt time! Why can only Rex Ryan see this?

37 Re: Bill Belichick Fourth-and-2 Calculator

There are several reasons why it might not be exponential. There is the one mentioned above of the last yards being harder to come by. But that is only if you start a play close to the endzone. If you run a go route and catch the ball 30 yards from the endzone with no one in front of you, those last 30 yards are quite easy to come by. How these combine in expectation is uncertain without some data analysis, but I would suspect it is closer to linear than exponential. For example, the probability of getting a TD starting 1 yard away is probably about 95%, maybe even less. If you use exponential reasoning, the probability of getting a td 80 yards away is 1.6%, whereas linear gives you 16%. The 1.6% is obviously too low.

46 Re: Bill Belichick Fourth-and-2 Calculator

Well, scoring from the 1 yard line is a qualitatively different phenomenon - the situation of having four chances to go one yard does not arise elsewhere on the field, so there is no iteration of the same basic "drive success" probability that provides the intuition for the exponential idea. This idea would only compare scoring probabilities from >= 10 yards to go.

If you look at the two-minute drill table (the column with two minutes and 3 TO remaining, as this minimizes interference from clock factors) you'll see that the data actually do fit the exponential hypothesis quite well, though to be fair, there are a couple of pairwise comparisons where the linear model fits better.

Perhaps I should modify my suggestion slightly to say that probability of scoring is roughly exponential in terms of the ratio of average number of first downs required in order to score from a given point on the field. I had assumed that the correlation between this and the ratio of yards to go would be pretty tight, but maybe this is not the case. Obviously, the greater the number yards required at the outset, the greater the opportunity to pick up "residual yards" that reduce the number of first down successes required. But in any kind of strategic equilibrium, surely these extra yards would have to be paid for by a lower probability of a success having occurred. But perhaps this does not come out in the wash as neatly as I had thought (hoped.)

Finally, I think we're in agreement that deviations from the exponential model are in the direction of greater probability of scoring from the longer distance. (This would also be the result if Chase Stuart's point about the last yards being the most difficult is indeed a significant factor.) So as it applies to the Belichick decision, the exponential method of relating probabilities of scoring from a short or a long field, if anything, underestimates the value of the fourth-down conversion attempt.

16 Re: Bill Belichick Fourth-and-2 Calculator

probabilty of large punt return or punt return touchdown needs to be added in, plus smaller probabilty of muffed punt or botched punt/block/fumble/shank.

I luv the calculator anyway. I am guessing NFL average on 4 n 2 is around 65% conversion and Colts high 80's (85 for example)from 30 yrds and 30% from own 25.....its exactly 70% a wash.

I hate Trent "Clown" Dilfer and his know-it-all ESPN bullshyt....(one of your FO guys tell him he is a giant corporate tool that should try and be honest and not tow the party line.)

Stats say this is a closer call than the stupid "experts" kept saying. BB took an intentional safety from his own end-zone, and somehow still won the game years ago. Big Balls call then, big balls call now. That is why he is BB coaching in the NFL and we are playing numbers games in Excel.


17 Re: Bill Belichick Fourth-and-2 Calculator

No matter what percentages you punch in this very handy calculator, this still greatly understates the chances for victory by going for it. In the instance where the 4th and 2 fails, since the Colts have such a short field, there is a greater chance they score quickly and the Pats get the ball back. Just think about the 2nd play where Addai nearly scored with over a minute to play. Had he got in the endzone, Brady would've had over a minute to get a FG.

18 Re: Bill Belichick Fourth-and-2 Calculator

Exactly, I've done this in Madden many times, you let the other team score a touchdown (do not tackle anyone) then you have more than a minute to get a FG for the win. Of course the counter is if the Colts ahd downed it on the one, and ate up the clock to about 30 ticks or so and at least three downs to get it in.

22 Re: Bill Belichick Fourth-and-2 Calculator

But that is because Madden's algorithm doesn't consider clock management for his RBs and WRs. Take the case of Brian Westbrook in 2007 against the Cowboys, or MJD this week. Addai could have just kneeled down close to the goal line and let the clock keep ticking. They still had a timeout and don't tell me Manning can't score from 1 yard out because he did.

This is one of the reasons BB's decition doesn't make much sense: did he actually consider the possibility that the Colts might actually ran out the clock a bit, since they had to cover just 28 yards (and I'm not saying that's a simple task)?

19 Re: Bill Belichick Fourth-and-2 Calculator

I think the decision to go for it was one of the greatest coaching moments the nfl will ever see. In the 4th quarter, Manning had shown a ridiculous ease to score in under 2 minutes and starting from his own 20. Why give him another chance to do what he had just done twice already?
We all saw how quickly Manning drove for a td from 30 yards out. The Pats defense doesn't suddenly turn into a different, and effective, unit if Peyton gets the ball from 80 or 200 yards out.
The Pats offense has a 100% chance of victory if they convert a 4th and 2. And they aren't the Lions or Rams offense, this is the Pats. They are going to convert that 4th and 2 almost every single time. We just happened to witness a rare event. No need to be fooled by randomness.

It's a shame that a coach will never get fired for punting the ball.

41 Re: Bill Belichick Fourth-and-2 Calculator

No, the Patriots offense is not going to convert 'almost every time'. When the game hinges on two points, it is difficult for any team to get the extra yards. The real mistake 'Belichick' (in reality the OC) made was going empty backfield- leaving a back in to block means you don't have to run the 2 yard 'no mistakes' route that ultimately failed (if you notice, Freeney was almost on top of Brady when the throw was made). Burning a timeout right before a play where there's a significant chance of a disputed spot isn't exactly rocket science either.

20 Re: Bill Belichick Fourth-and-2 Calculator

Here's how I calculate it:

If you go for it, you win if:
a) You succeed
b) You fail and the Colts don't score a TD
c) You fail, the Colts score a TD, and you make a game-winning drive

If you punt, you win if:
a) The Colts don't score a TD
b) The Colts score a TD and you make a game-winning drive

Let's (fairly arbitrary #s, but taken from of variety of articles written about this [or just made up]) say you have a 60% chance of succeeding on 4th down, the Colts score the TD 90% of the time after a failure and 40% of the time after a punt, and you make a game-winning drive 20% of the time after a failure and TD and 10% after a punt and TD (less time on clock--I'm calculating this as Belichick would have had to, not with benefit of hindsight on how much time actually ended up there).

Then p(Win|GFI)=.6+.4*.1+.4*.9*.2=.712
and p(Win|Punt)=.6+.4*.1=.64

So as someone above points out, this increases the odds in both cases (and increases the spread between) from the calculator's .63 and .6 (even if you give the same 10% probability of a game-winning drive, it's .676 to .64), but to mind the really interesting thing is that p(Win|GFI)-p(Win|Punt) should be increasing in some underlying latent variable measuring offense quality (not just because of the 4th-down success, but also the likelihood of a game-winning drive) and decreasing in an underlying latent variable measuring defense quality, which means the pundit-esque "it shows he didn't believe in his defense" would actually be borne out by this model.

23 Re: Bill Belichick Fourth-and-2 Calculator

his ego was so big, he thought he could pull it off.

maybe he won't wear that smirk on his face so much now.

32 Re: Bill Belichick Fourth-and-2 Calculator

Everyone seems to assume that if they convert, the game is automatically over. But the Colts still had 2 timeouts, so they still had a decent chance of getting the ball back anyway, albeit with considerably less time on the clock.

Normally, I'm a big fan of BB's willingness to think outside the box, but this time I think he took it too far. It would have been different if it were 4th and inches, where a QB sneak would pick it up like 95% of the time.

33 Re: Bill Belichick Fourth-and-2 Calculator

"Considerably less time" is right - it's the difference between a drive with 2 minutes and a timeout (they only had one), and a drive with 30 seconds and no timeouts. Games with 30 seconds, no timeouts, and the length of the field to drive are over, I would guess over 99% of the time.

38 Re: Bill Belichick Fourth-and-2 Calculator

I was pretty sure they had 2 timeouts left (remember, it was the Patriots who called the timeout prior to the 4th down play), but even if you're right, the clock still would have stopped after the conversion (because of the 2 minute warning), so if the Colts could have forced a 3-and-out, they would have gotten the ball with about 40 seconds to go, and barring a spectacular punt, they would need only about 80 yards. I would figure Manning and the Colts have a considerably better than 1% chance of going 80 yards in 40 seconds.

36 Re: Bill Belichick Fourth-and-2 Calculator

It was the appropriate call. But only a coach with job security would dare make it.

Here's an interesting article on the odds of going for it, featuring mathematical analysis by a UC Berkeley professor that indicates that the extreme conservativism of coaches is inconsistent with optimizing victory.

39 Re: Bill Belichick Fourth-and-2 Calculator

I've watched the re-run of the game yesterday. The bad call was not the 4th and 2. That one was actually pretty smart, looking at the opposition.
The bad call was the 3rd and 2. If you call a pass play on 3rd and 2, you got to know what you're call is going to be if it's incomplete.
BB didn't, hence the time out. Then he went for it on 4th, because he knew that Peyton was going to win the game if he got the chance - even from his own 30 (minus return).
BB (or someone on the NE staff) should've realised all this one down earlier and should've have called a run on 3rd and 2. And then go for it on 4th and x.

43 Re: Bill Belichick Fourth-and-2 Calculator

The timeout wasn't because of not knowing the playcall. Even worse, the timeout was because the decision to go for it apparently wasn't communicated to the punt team, because the punt team started coming on the field and they had to burn the timeout to get that all settled.

45 Re: Bill Belichick Fourth-and-2 Calculator

There are a couple probabilities missing - Odds that there will be enough time left in each situation for Brady to counter the TD with a successful FG drive. Odds the patriots make the 1st down but leave enough clock for Manning to still score a touchdown on them.

There was a pretty good chance for Addai to score with time left and then have Brady go score. It would have been less likely for the Colts to leave time on the clock after a 70 yard drive.