Extra Points
News and commentary from around the Web

Further Thoughts on Fourth-and-2

Further Thoughts on Fourth-and-2
Photo: USA Today Sports Images

by Bill Barnwell

(Ed. Note: This was the introduction to yesterday's Quick Reads on ESPN.com. We're running it as separate commentary on FO in order to prevent the fourth-and-2 discussion from blocking out discussion of the other 14 games in the Quick Reads discussion thread. The rest of Quick Reads for Week 10 can be found here.)

Too close to call.

It's not the satisfying answer to the Great Belichick Debate, which has seen him declared to be either an infallible genius or an overzealous tinkerer by most observers, but it's the most accurate one.

No matter how we tinker with or adjust the probabilities to account for the game situation and the quality of the two offenses and defenses, it's difficult to find a dramatic difference between the choices of going for it or punting.

Take the first option, the one Bill Belichick chose. Teams attempting to convert a fourth-and-2 have successfully done so at a 48.6 percent rate this year, well down from 62.3 percent the year before. The "true" likelihood of such a play being converted by an average team in an average situation is closer to that latter figure, based on historical data.

Of course, the phrase "…average team in an average situation" simply does not apply. The Patriots don't have an average offense, they have the league's best by DVOA; the Colts' defense ranks sixth. Although the Patriots had failed to convert in two of their three previous third-and-2 situations (including the one directly preceding the decision to go for it), their chances of making it in this situation are greater than the average team's.

On the other hand, the probability has to be adjusted for the situation. Robert Mathis had been abusing right tackle Nick Kaczur all night, meaning that Brady wouldn't have time for the combination of Randy Moss and Wes Welker to run anything resembling an intricate route pattern. Dwight Freeney lurked on the other side. Furthermore, the probability of a team going for it on fourth down might very well be artificially high because of selection bias -- teams are far more inclined to go for it on fourth down against the Lions or the Rams than they are against the Vikings or Ravens.

Throw all these numbers into a big soup, spin them however you'd like, and you'll end up with an expected conversion rate of about 60 percent. It might be 63, it might be 57; truthfully, it's not going to be enough to change our analysis.

The Colts aren't going to score every time they get the ball on the opposition's 29-yard line, but they will score most of the time. Toss in momentum and the quality of the Colts' offense versus the Patriots' secondary, and you can estimate, say, an 85 percent chance of the Colts scoring in that situation. That makes Belichick's decision to go for it a little stronger, upping the Patriots' chances of winning by going for it to 66 percent.

Then, it comes down to punting and where Manning gets the ball, which requires even more theoretical assumptions. Chris Hanson has a 39.6-yard net average, but it was in a dome, and the Colts don't have great return units. If we just assume 40 yards, the Colts get the ball on their own 32-yard line with two minutes to go and one timeout. If you believe that the Colts had a 34 percent chance or better of scoring a touchdown in that situation (100 percent minus the 66 percent chance we mentioned a moment ago), Belichick was right. If you think the odds are below 34 percent, Belichick was wrong.

If you disagree with the expected percentages of conversion above, Mike Harris (creator of the playoff odds simulator) has developed a nifty calculator that lets you plug in your own averages and figure out whether Belichick made the right call by those figures. You can find that calculator here.

Of course, we haven't even considered the possibility of running the ball as opposed to passing it on fourth down, turnovers, onside kicks, or the Patriots scoring on a game-winning drive. The bottom line is that a mathematical analysis of the decision boils down into too many assumptions and inapplicable historical expectations to say very much about one decision on one drive in a very unique situation, and when we make the broadest assumptions possible about the decision, the decision isn't close to clear-cut.

The important factor that the cacophony of responses seems to be missing is that you can't judge Belichick's decision by the fact that it didn't work. As we've mentioned more than once in these pages, you cannot judge decisions by their outcome. You have to consider the process that goes into them, and then decide whether they're right or wrong at the moment they're made.

Think back to another controversial Belichick decision made in the heat of a prime-time game, his decision to take a safety on purpose down one point during the fourth quarter of a Monday night game against the Broncos. Of course, the Patriots ended up getting the ball back and won the game. Belichick took virtually no flak after the game for his unconventional choice, and was instead hailed as an aggressive, brilliant game manager.

If Kevin Faulk stumbles two feet forward, Belichick is being spoken about in those glowing tones today by virtually everyone lining up to criticize him. That doesn't make his decision correct or incorrect, any more so than Faulk coming up short does. If Belichick's decision was wrong, it was wrong from the moment the playcall went to Tom Brady. And with everything we know about the situation, it's impossible to say whether that was truly the case.

(Another Ed. Note: If you would like to read a similar analysis of a completely different coaching decision, find a copy of the 1986 Bill James Baseball Abstract and read his commentary on Tommy Lasorda's decision to pitch to Jack Clark with a base open late in Game 6 of the 1985 NLCS. Some truths are universal.)

Comments

214 comments, Last at 18 Nov 2009, 5:55pm

1 Re: Further Thoughts on Fourth-and-2

" If you believe that the Colts had a 34 percent chance or better of scoring a touchdown in that situation (100 percent minus the 66 percent chance we mentioned a moment ago), Belichick was wrong. If you think the odds are below 34 percent, Belichick was right."

Am I reading this wrong, or is this backwards?

19 Re: Further Thoughts on Fourth-and-2

No, I think this is right.

Going for it means there's a 66% chance to win.
Punting means there's a 34% (or more/less) chance to *lose*.

So if the Colts' odds of scoring after a punt are 30%, then that's a 70% chance for the Patriots to win and punting is the right decision. If the Colts' odds of scoring are 40%, then that's 60% for a Patriot victory and then punting would be the wrong decision.

3 Re: Further Thoughts on Fourth-and-2

If you believe that the Colts had a 34 percent chance or better of scoring a touchdown in that situation (100 percent minus the 66 percent chance we mentioned a moment ago), Belichick was wrong. If you think the odds are below 34 percent, Belichick was right.

Other way around, right?

Nice post overall though - I'm a Pats fan, but all of the hyperbole and hand-wringing and general lack of understanding of basic probability by so many so-called experts is bothering much more than the loss itself.

6 Re: Further Thoughts on Fourth-and-2

All the analysis I have seen indicates what I thought immediately with my gut - this was either the correct call beyond doubt or at least a push. When a decision is too close to call, it is the job of the coach to make the decision, and the fans job to second guess him.

Had the Pats converted, BB would be called a genius again and again (and again!). As they failed to convert, his thinking is being scrutinized.

As for me, I don't much like Bill as a man, but give the devil his due. He's on heck of a coach, and that was a call which is both gutsy and defensible. It was a good call and I cannot second guess him on it.

7 Re: Further Thoughts on Fourth-and-2

They way I've been thinking about it, the decision to go for it on 4th down was the right one. It was a couple things leading up to it that were wrong: wasted timeouts, the play run on the 3rd down. Even with those things, it was still the right decision. And if it had been a more bog-standard NFL coach you could rationalize the imperfection of it all. But given that it was the vaunted Belichick, sure fire hall of famer and in the conversation as best coach of all time, this tarnishes his rep, especially if the Pats go on to lose a road game in the playoffs (and especiallly if that road game is in Indy)

27 Re: Further Thoughts on Fourth-and-2

You are absolutely right. If he was going to go for it on fourth down, he should have known this before the third down play. He could have directed his offensive coordinator to call a running play that would have made it a shorter fourth down.

Another mistake by Belicheck I'd add in was the seemingly non aggressive play calling on the drive that led to the Pats final field goal. I was saying at the time that there was too much time to be running clock and that the Pats should aggresively try and make it a 17 point game.

8 Re: Further Thoughts on Fourth-and-2

I can't think of a better situation in which to fake the punt than the one in which the Pats were in Sunday night.

23 Re: Further Thoughts on Fourth-and-2

Now you're really mucking things up.

I don't know what the data on fake punts is, and it probably suffers from small sample size, but I'd assume the trickeration would not carry a lot of weight--especially after a timeout. (Now if they had Randall Cunningham at QB, a shotgun snap and QB-punt might have landed Indy on their own 5.)

I sided with BB in his choice--as a Colts fan I was a lot less worried about a punt than a potential conversion. But taking the ball out of the hands of your highly regarded QB and trusting it to a punter who played QB in HS, or the equivalent, or relying on a direct snap to a protecton back that your team practiced all of six times last week, does not sound like a great idea to me.

Of course a 2-yard pass pattern when you need exactly 2 yards doesn't leave a lot of room for error.

10 Re: Further Thoughts on Fourth-and-2

Thanks for splitting it off from Quick Reads. Actually comments to Audibles was unreadable as all commenters ignored 15 games (14 others + 99% of this game). I think the subject has been blown way out of proportion.

11 Re: Further Thoughts on Fourth-and-2

"you can't judge Belichick's decision by the fact that it didn't work."

Not to take anything away from the article which I like, but this is just incorrect.

First let me be clear that I understand the argument that leads to this statement it just doesn't apply to football, and I think this call makes that clear.

It only applies in situations were you can clearly see whether the process was good or not. No one can actually tell you what the odds were of them converting nor can anyone actually tell you what the odds were of Peyton scoring after a punt or turn over on downs.

In poker when you look at the cards in your hand and the cards on the table you can calculate the odds of winning, but this isn't poker, and we can't apply the same logic.

You are very correct in saying, "The bottom line is that a mathematical analysis of the decision boils down into too many assumptions and inapplicable historical expectations to say very much about one decision on one drive in a very unique situation, and when we make the broadest assumptions possible about the decision, the decision isn't close to clear-cut."

Since we can't judge whether the process was good or not we either have to judge the result or not judge at all.

The reality in football is that we have to judge the result because by and large it's all we can do. If someone wins that's great if someone losses they get fired.

I admit this a an oversimplification because there are sometimes when we know the process was wrong, but when a team isn't drafting Darius Heyward-Bey we have to go by results.

So as much as it pains me I have to say that BB has won three Super Bowls, I'm not going to second guess him.

32 Re: Further Thoughts on Fourth-and-2

"In poker when you look at the cards in your hand and the cards on the table you can calculate the odds of winning, but this isn't poker, and we can't apply the same logic."

actually your odds in (hold 'em) poker are approximations as well, since there at least 12 (five other players, two burned) cards and as many as 20 cards that are out of the deck, that you have to assume are in play when you make your calculations.

"Since we can't judge whether the process was good or not we either have to judge the result or not judge at all."

Isn't this the same as all the talking heads saying, "it didn't work, so it was wrong?"

42 Re: Further Thoughts on Fourth-and-2

In poker you don't have all the information but you can still determine the odds, which means that if you have a good process over the course of your career things should even out.

It's not quite the same at least what I've seen has been a lot of judging the process by saying you always punt in that situation. I would also take a slightly broader view that it's not one individual play but their body of work that you judge people on so BB gets a pass.

41 Re: Further Thoughts on Fourth-and-2

Whether or not the odds can be calculated shouldn't factor in at all. What they are getting at is whether or not the decision made sense given what was known at the time. In other words, as long as going for it can't be shown to be the wrong decision, you can't claim it's wrong simply based on result.

Look at this at the extremes. Say a team A is leading team B, and team A has the ball with five seconds left in the game. Pretend team A's QB takes the snap and starts to kneel and for some reason throws the ball over his head, at which point team B scores and wins.

In that case, just because the outcome was bad, doesn't mean kneeling was the wrong decision. Given the facts at the time (lead with 5 seconds left) the kneel play is justifiable. The same thing applies to Belichick. Given what he knew at the time for his team and the opponent, the decision made sense. You can't clearly say that the odds were in favor the other way, so regardless of outcome it was a good decision. (Or at least an OK one.)

110 Re: Further Thoughts on Fourth-and-2

"It only applies in situations were you can clearly see whether the process was good or not. No one can actually tell you what the odds were of them converting nor can anyone actually tell you what the odds were of Peyton scoring after a punt or turn over on downs."

I'm arguing the the "process" in this case is simply taking all available information, properly weighing it, and making an informed choice. The fact that no one can actually tell you the odds of those things doesn't matter. I don't find the poker analogy to be valid because of the example I gave above taking a football situation to extremes.

48 Re: Further Thoughts on Fourth-and-2

Not only is it amazing that someone believes that Barnwell's statement doesn't apply to "football," but it is absolutely astonishing that you used half a page to defend this belief. Ummm, no.

It's better to be lucky than good. I will grant you that.

We can never know the "real" odds of anything (other than on theoretical questions). I will also grant you that.

We can use results to try to determine what the odds might have been in any given situation. That's what this site does. It compiles statistics, analyzes them, and often tries to make predictions about what such analysis says about the risk profile of any past, present or future decision.

That said...

If someone is fairly flipping a fair coin and says to me, "I will give you $10,000 if you pick heads or tails correctly, and you will pay me $1,000 if you choose incorrectly," I will take that bet any time.

If it comes up tails and I said heads, I may *hate* the result, and I may really, really wish the result was different, but I absolutely and totally made the right decision before the result was known.

As for Belichick, he has won consistently like no other coach/team builder in recent history. It's getting to the point that we cannot reasonably attribute this to luck. He is "smarter" than his competitors. The yin/yang of this is that his competitors are "stupider" than he is. How is this possible? Well, I would posit that all his competitors are subject to cognitive biases in evaluating players and in-game decisions, most notably the CYA bias of being deathly afraid of looking stupid to the public. Of course, their desperate desire to avoid looking stupid *condemns* them to looking stupid when someone like Belichick (who is clearly not burdened by the same fear) absolutely schools them.

At what point is everyone going to realize that this is why Belichick wins? If he did the conventional thing every time, drafted according to Kiper's big board, designed every fourth down play the way Norv Turner would, *he would not have been so consistently successful.* It is literally impossible (save extreme luck) to so consistently and overwhelmingly defeat your opponents unless *you are doing something different than your opponents, all of them.* He wins because *you* are stupid.

For my own two cents, this was a close call. It may not have been "right," but I have always thought more coaches should go for it on fourth down. Had the game been tied or the Patriots up by three or less, then it would be different. Clearly, you must punt there. But if I'm the coach in this situation, I want to win it on my own terms. I have one of the best and most experienced offenses in the NFL. This is not a playoff game. I've proven I can outplay the Colts on their home field. I don't even need to win this game. But I will put it on my offense to make these two yards. I will take the risk that *even if it fails,* my defense can still prevent a TD from 30 yards away. I am trying to keep this in my own hands, not in Manning's. I don't know if I would have had the stones to make this decision, but I would *know* I made the right decision the next day after listening the hordes of the ignoramus hoi polloi condemning it. I would *know* that their idiocy is why I win. I would never be able to win without it.

55 Re: Further Thoughts on Fourth-and-2

Wow, I hope you can unpucker your lips for BB's ass long enough to read this. You are vastly overrating BB's edge over his competitors. Off the top of my head I can cite several examples of "coaching" by him.

* 2006 AFC championship game. Pats 1st and 10 ready to run out the clock and seal the game. Opps, 12 men in the huddle. Five yard penalty which eventually leads to a punt and the rest is history. A total lack of organization on the sidelines leads to this crucial mistake.

* Same game. BB the GM has foolishly let all of his best WR's go via free agency and now is left with a wide-eyed scrub (can't remember his name) dropping crucial passes that hit him in the hands.

* Fast forward to 2009. The Pats have the wrong people on the field on 1st and 10 AFTER a TV timeout. Seriously, how does this happen after a change of possession and tv timeout? Burn another crucial time out to fix.

* Same game. Indecision before the fourth down play leads to another blown time out.

* Same game. He foolishly uses a challenge on a Reggie Wayne catch that is clearly a catch. I can only assume he threw the challenge flag on a wing and prayer or out of total frustration.

I think we can all agree that BB is a great coach. Probably one of the all-time best. But remember there are other great coaches in the business too. So your depiction that he has a limitless advantage over everyone else is vastly overstated.

12 Re: Further Thoughts on Fourth-and-2

I still have no idea how anybody can rail on about how the choice was incorrect when, had the refs given Faulk credit for forward progress, the Pats would have made the first down!

Rather than rail on about BB's arrogance, perhaps we should discuss the logic behind not giving Faulk credit for forward progress at all.

20 Re: Further Thoughts on Fourth-and-2

This comment is patently false. If you look at the tape you see he lands at the 28. If they had not given him forward progress they would have spotted it at the 28, not the 29.5. The announcer was incorrect.

26 Re: Further Thoughts on Fourth-and-2

By that logic, if he had the ball in his hands but was bobbling it while standing on the 30, then dove backwards (still bobbling it) and landed on his back on the 25 with the ball firmly in his grasp, are you saying the placement should be the 30? I think the NFL rule book would disagree.

Fwd progress starts with possession of the ball. With RBs it's a little more obvious that they have it rammed into their gut--they have the ball, so it's simpler to eyeball where they cease moving forward.

Possession is the first element of a catch, and then position. Think of all the out of bounds "catches" you've seen where the guy does a great job grabbing the ball but it's not an official catch because he's OB. Same thing here--when he came down to the field of play with full possession, it was the 29.5 not the 30.

If the pass had not been rushed it might have been more accurate and catchable--his feet were beyond the 30 before he went up for it. Had he had firm control rather than bobbling it, you'd be correct. But the bobble was the deal killer here.

40 Re: Further Thoughts on Fourth-and-2

the official's judgement was that the bobble ended at the 29.5, so that's the end of the arguement. but to my (fan's) eye the bobble sure looked to have ended on the far side of the 30. the ball is pinned to his chest when bullit lifts him off his feet. it sure would've been nice to have a timeout in that spot, either way

44 Re: Further Thoughts on Fourth-and-2

Faulk stopped bobbling it before being tackled. (screencap screencap) NFL rules say
Rule 2-15-2: "When an airborne player makes a catch, forward progress is the furthest point of advancement after he possesses the ball if contacted by a defender.

(Casebook 2.15.1): It is first and 10 for A at B's 12-yard line. A1 sprints near the end line and then buttonhooks. He jumps and possesses a forward pass while in the air above the end zone. (a) A1's momentum carries him back into the field of play and he lands and is downed at the 1 yard line; or (b) while in the air in the end zone, he is contacted by B1 and he then lands and is downed on B's 2-yard line. Ruling: In (a), it is A's ball first and goal at B's 1-yard line. In (b), it is a touchdown if the covering official judges the contact by B1 is the cause of A1 coming down at the 2-yard line, instead of in the end zone.
and from "Ask Jerry Markbreit" who answers rule interpretation questions (former ref)

    Q

:I have seen plays where the receiver jumps to catch the ball in the playing field and is drilled while he is still in the air, so that his feet come down with possession and control of the ball several yards back. Doesn't the official give him forward progress where the original contact was made while he was in the air? If so, doesn't the rule on forward progress refer to a "runner?" He cannot be a "runner" until his feet come to the ground. Could you explain this? --James Buldas, Chicago

When a receiver of a forward pass is knocked backwards in the field of play and he comes down in possession and in control of the ball, he is given the foremost point of the catch, even though it was in the air. Forward progress does not only refer to a runner. If a runner dives in the air for the goal line and breaks the plane of the goal line with the ball in his possession, it is a touchdown.

Forward progress doesn't require the catch to already be complete. He was contacted over the 30 with the ball in his possession. It was a blown call. BB gets a small part of that blame because he didn't have a TO, but the onus is on the officials to get it right.

52 Re: Further Thoughts on Fourth-and-2

I realize this is picking nits, but I think the end of your post is misleading. You say:

Forward progress doesn't require the catch to already be complete.

But above, the quoted rule says:

"When an airborne player makes a catch, forward progress is the furthest point of advancement after he possesses the ball if contacted by a defender. [sic]

I interpret the rule to mean that the receiver has to have completed the catch, i.e. not bobbling the ball ("after he possesses") before forward progress is determined.

This in no way changes the fact that it is quite possible Faulk completed the catch at the 30. I just thought that last bit was off.

57 Re: Further Thoughts on Fourth-and-2

two different definitions of completing the catch. One of you is using: 2 feet down with control of ball. The other is using 2 feet down.

You both actually agree though that control of ball is all that matters.

The difference then is where he got control of the ball. Was it past the 30, or short of the 30? The official said short, and the replays I've seen are not conclusive to me either way. Maybe I just don't have as much 3-d extrapolation ability as the poster, or maybe he's letting his wishes impact his perceptions.

63 Re: Further Thoughts on Fourth-and-2

I am most definitely extrapolating, since you are correct in saying there is no single conclusive replay. but. looking at the endzone angle you can see the ball take a single short bounce before he pins it and gets hit, and then adding that to the (admittedly off angle) sideline camera, you see bullitt make contact with faulk what appears to be past the 30 yard line. so you're right, it's probably not conclusive enough to be overturned, but that doesn't mean I can't say BAD SPOT, REF!

113 Re: Further Thoughts on Fourth-and-2

That second screen shot is pretty damned convincing (confirms what I thought anyway, but more convincingly than the moving video). Because the camera is at around the 20-yard-line the angle isn't great and Faulk's upper body position is uncertain-- except for the fact that his extended left foot doesn't lie, and that foot is on the ground but has barely crossed back over the 30-yard-line.

Let's put it this way: that's more convincing video evidence than with last year's forward-progress decision on Santonio Holmes' winning TD catch in Baltimore, which was awarded (rightly I believe) on a replay-booth overturn. So you never know what the replay decision would have been. It would have met my personal standard for "indisputable video evidence" for an overturn though.

184 Re: Further Thoughts on Fourth-and-2

Agree with the first poster (screencaps). The key is that the bobble ended BEFORE Faulk gets hit. Faulk was past the 30 before he got hit. Logic dictates that he had possession past the 30.

Furthermore, when I watched the game live, I remember the ref that was closest to the play start to line up past the 30, where the spot should have been, only to be overruled by the sideline ref. Only later did I realize that that judge could not have seen the bobble. But he seemed pretty damned adamant that the ball be placed at the 29.5.

Add to that the phantom PI call earlier in the half, and the fact that this particular crew has the largest bias towards home teams (http://refchat.blogspot.com/2009/11/nfl-referee-statistics-before-week-10.html), and I guess you see it is what it is.

13 Re: Further Thoughts on Fourth-and-2

Bill Barnwell - it's intelligent articles like this that separate FO from almost all the other sites.

14 Re: Further Thoughts on Fourth-and-2

The economist in me likes the stats but this call was nothing to do with stats. You can't remove the future implications of the decision and they are huge. This was the wrong call regardless of the outcome. Unless your punter broke his leg earlier in the game you punt the ball 100% of the time in that situation.

25 Re: Further Thoughts on Fourth-and-2

I don't understand why, if we accept Barnwell's numbers (which by the way I think are good) we would think this is a close decision. Don't you think Manning has a better than 1-in-3 chance of scoring of TD with 70 yards to go, 2 minutes left, and a TO? (Especially given the fact that we just allocated an 85% chance to Manning scoring a TD with 30 yards to go in the same scenario).

45 Re: Further Thoughts on Fourth-and-2

Plus that has to be an 85% chance Indy runs the clock down and then scores, not just scores. The chance that Indy scores too fast there is pretty significant. Brady would have loved to have 60 seconds to go 40-50 yards for a FG try.

85 Re: Further Thoughts on Fourth-and-2

This is a very, very important consideration that favors BB's call, and not just a negligible throw-in. In fact, the Pats would have had over a minute left had their understandably conscientious DB not tackled Addai at the 1, or about 45 seconds had they not stopped the run on the ensuing play. If we think the colts had a 33% chance of scoring a TD from their own 30, perhaps the Pats would have had a 15-20% chance of driving to a makeable field goal under these combined (ex ante) scenarios?

177 Re: Further Thoughts on Fourth-and-2

4/13 may be misleading. Maybe it would be more relevant to say that on the past three drives - after Manning had a chance to adjust to the defensive game plan and when the Patriots defenders were gassed - the Colts had twice scored from over 70 yards out, and took an average of less than two minutes to do so. They hadn't even had to use a time out. That'd put the odds between 50%-67%. Small sample size fun for everyone!

185 Re: Further Thoughts on Fourth-and-2

Defense tired, game plan clearer. Perhaps you saw how easily Manning moved the ball down the field in his prior possession. Perhaps you saw how easily Manning moved the ball down the field in his final possession. In fact, if you think about it, Manning moved the ball 29 yards in about 30 seconds, with the defenders having the advantage of having the end zone as a backstop. That's about 2/3 the distance of the punt the Pats might have gotten in the same situation. So basically, having the ball at the Pats 30 with 2 minutes to go vs. having the ball at the Pats 40 with 1:30 to go is the difference between 85% and 33%? Nonsense.

Also, let's not forget that the refs were calling some blatantly pro-Colts calls in the 4th quarter. Whether or not you think the refs were exhibiting a pro-Colts bias (this link would suggest they were: http://refchat.blogspot.com/2009/11/nfl-referee-statistics-before-week-10.html), that has to be a factor in thinking ex ante about how likely it is that Peyton can move the ball 65 yards down the field or not.

53 Re: Further Thoughts on Fourth-and-2

Economist understand statistics = fail

Sad sad day for our economy, if you are in a position of decision making power or teaching the youth of our nation.

The future implications is absolutely modeled into the formula.

65% complete fourth down (estimated NFL average for 4 n 2)
85% chance indy scores TD from the 29 (Indy great offense)
35% chance indy score TD from own 25 (Indy again great offense)

expected outcome:

70% chance of winning if you go for it
64% chance of winning if you punt

see awesome calculator:

http://belichick-decision.heroku.com/

Don't hate the model, hate the game

15 Re: Further Thoughts on Fourth-and-2

Hmm.. being that the numbers were sooo close I think that it reinforces punting was the slightly "better" decision. By punting you spread your personal risk and put more things in play that could go wrong for Indy. Tipped ball for an int, sack, fumble snap, holding penalty, drops,etc. By going for it he put all his eggs in the "Kevin Faulk running a 2 and a half yard route/defending only 30 yards" basket.

Quick question about 4th down conversion rates. Are garbage time numbers factored? For example I'm assuming that these numbers also include say Tony Romo completing a 4th and 4 when down 17 points late in the fourth quarter right?

29 Re: Further Thoughts on Fourth-and-2

But by punting, he puts all his eggs in the "defend for ~65 yards" basket. All of the things you mention that could go wrong on a 65-yard drive could still go wrong on a 30-yard drive, but if the offense can advance the ball six feet, they don't have to go wrong.
The fact that the numbers were close reinforces your opinion that punting was better, but I'll bet if you originally thought going for it was the better option, you'd see them as backing that up. What they say to me is that neither decision is definitively "better," and while I'm happy to sit here on Tuesday morning and say yeah, they should have gone for it, if I happened to be calling plays for the Patriots that night, I'd probably have sent out the punt team.

54 Re: Further Thoughts on Fourth-and-2

I believe 4th and 1 NFL completion average is in the 90% range

and the NFL completion average on 4th and 2 would be conservatively in the 65% range, also I would say even 75% of the time would be acceptable number w/ Brady and Pats Offense short pass ability advantage

120 Re: Further Thoughts on Fourth-and-2

This isn't the right way to think about it: that's the aggregate of all 4th and 1 plays, for example. You don't get to run an "average 4th and 2" play. You get to run one play. Belichick's decision to go for it rather than punt is absolutely wedded to his play choice, because what he actually chose is running that play over punting.

And I really doubt that play had a 65% chance of working.

127 Re: Further Thoughts on Fourth-and-2

So if Bill's going by numbers that say he has a 90% chance of converting 4th and 1 why on earth does he pass on 3rd and 2? This is the part of the analysis that is so suspect in my view. It isn't fair to argue based on the results but a lot of people seem to think it's just fine to assume Hoodie was making a detached, stat-based decision when the evidence indicates he was acting emotionally. The stat stuff exists in a vacuum; after a 3rd-down pass and 2 blown timeouts on that drive I think we can agree that Belichick's call didn't.

186 Re: Further Thoughts on Fourth-and-2

Game theory.

On 3rd and 2, wouldn't you presume that the Colts would be trying to stop the run? I'd rather get the 1st down on 3rd and 2 than 4th and 1 myself.

That being said, I think the NE offense has an excessive reliance on the pass, but I don't think the logic is that flawed, nor does it show that the "stat stuff exists in a vacuum". Fail.

17 Re: Further Thoughts on Fourth-and-2

Belichick was playing to maximize his chances of winning, not to minimize his chances of losing.

To this end, he made the correct decision, but not the correct play call(s). If you start down the path of, "Well, he should have considered what could have happened had he not made it...", then you also need to consider that Indy COULD have returned the punt for a TD or put themselves in great field position, or his punter COULD have shanked the punt or had it blocked, etc.

You have to believe that you can gain two yards at a time when your offense is averaging about 7 yards per play.

As several others have stated, the events leading up to 4th and 2 were the problem, not the 4th and 2 decision. Heck, the execution on 4th and 2 was more of a problem than the decision or the play call.

-------------
Sports talk radio and sports message boards are the killing fields of intellectual discourse.

31 Re: Further Thoughts on Fourth-and-2

my thoughts exactly (maximizing winning).

In fact, I'd say the mistake in the second half was Welker calling the timeout on first and 10. I think timeouts are largely undervalued (not nearly as much as in college basketball, though), and I would guess the Patriots in 1st and 15 near midfield aren't in that bad of a position - of course that assumes that instead of Welker calling the time out, he'd have done nothing and the Pats would have taken a delay-of-game penalty. I guess the only catch is that time outs are valuable primarily when you are losing; coaching aggressively might imply that you should be less concerned with using time outs strategically late in the game and more concerned with using them as the need arises.

I can understand the time out before first and fourth down, if only because at that point the goal was to convert and close out the game, and a 1st and 15 in that situation is probably harder to accept than a 1st and 10. (Certainly a 4th and 7 is much worse than a 4th and 2; the last time out had to be used.)

39 Re: Further Thoughts on Fourth-and-2

maximizing your chance of winning vs minimizing your chance of losing is pretty meaningless. It's a zero-sum game. If you minimize your chances of losing, your chances of winning goes is affected inversely (yes I know there are ties). Either variable you choose to look at (% to win or % to lose), it's really the same thing.

now, I'm thinking you probably meant something like Belichick was trying to play the aggressor which is inherently advantageous. I think there was no way to do that in this case. Even though the Pats had the ball, I think at that point in the game, the Colts defense was clearly the aggressive unit. The Pats were just trying to hold on regardless of the personnel on the field.

129 Re: Further Thoughts on Fourth-and-2

I'm not buying this explanation. The potential for botched execution and the choice of play are all inherent in the decision to go for it. As someone else mentioned, Belichick didn't decide to call an "average 4th-and-2 play." He decided to call *that* play, in *that* situation, after giving Indy a cue that something was up by mucking around with the timeout, and with the knowledge that he had just removed his own ability to challenge any near misses even though he was calling routes that gave no room for error. When people say it was a bad decision they aren't talking about going for it in some random 4th-and-2 situation that exists in a statistical vacuum, they're specifically referring to 4th-and-2 from the -28 against the Colts in a game with large playoff implications where Peyton Manning has been rather inconsistent.

Maybe I'm just biased against Hoodie (who I will nonetheless say is obviously an exceptional coach) but to argue that you have to split the playcall out from the decision to go for it is ludicrous. Deciding to go for it without doing anything to set up for 4th-and-2 was a bad decision and using that play made it worse. People are tying themselves in knots to avoid that simple acknowledgement.

18 Re: Further Thoughts on Fourth-and-2

I liked the aggressive play call, but I didn't care for the sequence. All three second half timeouts seemed to relate to the Patriots not being organized. When Indy calls time out before the third down play, the Patriots have to be deciding whether they are in 3-down territory or 4-down territory. If you are in 4-down territory, that third down play has to be a run. That gets you either another Indy time out or a two-minute warning to plan the fourth down play.

Also, earlier, Brady threw the ball away on third down with just over four minutes left in a 10 point game. The field position wasn't as important as the clock. He needed to take the sack and keep the clock moving. Who knows what changes if that New England possession happened on the other side of the two-minute warning.

21 Re: Further Thoughts on Fourth-and-2

BB's mistake was made earlier, on first down. The Pats had a 17 point lead at one point in the 4th Q and lost it. Bill Cowher, who never gets any mention as a situational genius, won 99% of the games where the Steelers got an 11 point lead. He didn't win those games by throwing the ball 4 times in a row after getting the ball 1st and 10 on his 20 with 2:23 to go. HE RAN THE BALL! He had a Bus and he rode it. Before he had a Bus, he had other backs who could get 3 yards and cloud of dust. If the Pats had run on first down, with whatever personnel grouping on the field (assuming the wrong group was on the field-was it a running play group?), hell a QB sneak would have taken it to the 2 min warning or force Indy to take a time out.

On 3rd down, again, how do you not run it? If you know you are going for it on 4th, get half on 3rd half on 4th. BB said he hadn't thought about it before missing it on 3rd!!! WTF??? Are you kidding me? He's making it up as he goes?

NOw if you want to argue that a healthy Fred Taylor would have allowed the outcome, OK. But why sign an older bruiser back when you know they have trouble playing 16 games. BB needs to straighten out the guy who "buys the groceries."

Until this series, I had a (grudging) respect for BB. Now, not so much. Making a bad series of decisions which put you in a position to take a big chance which can be justified if the proper assumptions are made doesn't do it for me when the alternative path is more certain.

(CAPTCHA for tara fits with the theme of getting ground!)

81 Re: Further Thoughts on Fourth-and-2

Temo: Yes well part of the reason that Belichick is a better coach and has won more games than Cowher is because he managed to amass more games with 11+ point leads than Cowher did.

Actual data from Wikipedia: (assuming that Wikipedia has "actual" data!)

Bill Cowher 149-90-1
Bill Belichek 144-89-0

Temo, please explain to me again how 144 is "more than" 149.

91 Re: Further Thoughts on Fourth-and-2

I'm not attempting to "judge." I don't "judge" quarterbacks by who has the most rings. I don't even "judge" coaches by who has the most wins--for instance (a case that might be pertinent to the current discussion) Bill Belichek amassed very few of his wins and a ton of his losses when he was coaching an inferior team, and has done considerably better coaching a good one. Temo stated "Belichek is a better coach and has won more games than Cowher." The first part of his statement can be argued (and, even though I am a Steelers fan, I concede that it will probably be argued successfully.) The second part of his statement, as of the time he made it, is demonstrably false. The entire force of my comment was that his factual assertion was incorrect. Anticipating future victories by Belichek-coached teams doesn't change that.

24 Re: Further Thoughts on Fourth-and-2

As has been noted, the largest error was not considering a four down strategy until fourth down arrived. One of the primary advantages to reducing punting frequency lies in how offensive strategy changes, given a commitment to a four down strategy.

Also, even if the third down pass had been completed, it was a lousy play to run, since their was a decent chance the received would have been driven out of bounds, thus stopping the clock prior to the two minute warning, leaving the Colts with one time out. This means that even a new set of down could easily have resulted in the Colts getting the ball with more than a minute left. If you want to run out the clock, or come close to it, with one first down, then call plays that will likely do so.

28 Re: Further Thoughts on Fourth-and-2

Interesting perspective--in my mind, until yesterday, I thought BB always had a 4-down strategy in mind. With a 12 our of 16 success rate vs Indy on 4th down attempts since 2001, I just assumed he always considered it.

I guess he didn't, by his own admission. That really surprised me, as I thought he was a guy who plotted out every plausible scenario ahead of time.

12 of 16 includes a 4th and 15 last year with Matt Cassell getting picked off. If you include just Brady's makeable 4th down attempts, that's 11/15 or 80%. I like those odds.

36 Re: Further Thoughts on Fourth-and-2

I haven't checked the Colts front 7 alignment on third down, but I suspect even a qb sneak on that snap would have been a far superior play to call, compared to a pass to the sideline. Hell, if the fourth down play had been called on third, the Pats likely win the game, even with that spot. I just can't figure out why they would call a sideline pass in that situation, other than they obviously liked (incorrectly, it turned out) the match up out on the edge.

30 Re: Further Thoughts on Fourth-and-2

I was under the impression that they had considered it prior to fourth down? Although the quote below could be taken either way, I guess.

From ESPN.com:

"We had a little miscommunication on that. Once some of the guys on the punt team started out, then the guys on offense they started to come off," Belichick said, noting the decision had been made on third down that they wouldn't punt. "That wasn't cleanly handled. I'll take responsibility for that."

75 Re: Further Thoughts on Fourth-and-2

Just to restate a bit: we need to judge on process, not on outcome. Some commenters have suggested that process is beyond what we know and, therefore, we cannot judge it. Not true. Process does not just mean 4th down. Process means the whole sequence of downs. In other words, the 3rd down call is important to our understanding of whether or not going for it on 4th is “correct”.

Many of us here agree that the 3rd down call was problematic. Some even question the 1st down call (see #21).

We would have preferred a run on 3rd down. That has the major advantage of keeping the clock moving (either forcing the Colts to call timeout or running the clock down to the two-minute warning). It has other possible advantages including Brady (and his OL!) not being subjected to Indy’s pass rush. As the Patriots appeared to be running well against Indy, that might also be something to take into account.

We also think the call on 3rd down was not a good one (see #24), though we may be on shakier ground.

The process suggested by the 3rd down call is one that many of us think was faulty. It follows therefore that the 4th down call was the result of a bad process, not a good one.

187 Re: Further Thoughts on Fourth-and-2

One screenshot I thought was VERY interesting during the game. Following a Colts TD, Belichick was seen talking with his defensive unit, even while the Pats offense was on the field. It made me think that he doesn't actually call most of his offensive plays, but rather delegates that process to someone he trusts.

In the instance of the controversial 4th down play, Brady has been quoted as saying that he had pushed for that particular play, as it was something he and Faulk had practiced a lot.

All of which makes me think that the particular play calls were not initiated by BB. Obviously, as the head coach, he has to take responsibility for the playcalling as well as the processes that led to the playcalling. But it did seem striking to me (as opposed to someone like Norv or Jim Zorn, who would call their own plays).

188 Re: Further Thoughts on Fourth-and-2

he doesn't call plays. that's been true since the days of charlie weis. in the 2007 super bowl I don't think he even watched an offensive snap, he spent so much time exhorting his defense to do their jobs.

he was responsible for the decision to go for it, and also responible for not making that decision until the last minute, and burning his last time out to do it

34 Re: Further Thoughts on Fourth-and-2

I agree that there is an intolerable fickleness to the post-game analysis. Every talking head is using this decision as a posturing tool, and yes, it's sickening.

However, there remains a bit of a straw man at the center of Mr. Barnwell's argument. Most of the people whose analysis I respect AND who criticize BB for the call -aren't- bashing him based on the outcome of the play. And nor do they rely on using P-F-R's little percentage tool . . . because stats aren't the only legitimate tool in the box for analyzing what happened.

There's a wider angle to all of this: probabilistic arguments ("in my view they had a 65% chance of winning") aren't the only rational ones. Stat-head claims notwithstanding, things like momentum, player psychology, even crowd involvement, and other unmeasurable factors need to play a part in making — and assessing — the call. It takes, in addition to guestimates and percentages, judgment of personnel, character, and the individual personalities that constitute one's team, etc.

Narrowing in on what would happen if we ran the simulation 10,000 times, to the exclusion of other factors, can be misleading.

56 Re: Further Thoughts on Fourth-and-2

great post!!!!

A few thoughts to consider:

Models are built like road maps. You can add details, but it will never be an actual road. K.I.S.S is one of the most important things to remember when modeling. Keep it simple, stupid!

reason: once you factor in too much qualitative data, you need to quantify it to get numerical results. This quanitification of non-numerical data is ripe w/ assumptions and biases and changing variables. You can "manipulate" these numbers for any outcome you want. In fact it is easy to do, once the model is set up. Many times it is done unintentionally.

I like the simplitic approach...or road map making approach, you can see the direction and get from point A to point B, but you wouldn't want to buy your dream home from looking at a road map (limitations to the model)

37 Re: Further Thoughts on Fourth-and-2

Any time the defense sells out to stop a play, they've got alot better chance than 40% of stopping it. Probably most '4th and twos' are situations where the offense has to have it, while the defense still has a lot more scoreboard/turf to play with.

I really, really want to see in detail the data cited on behalf of 60%.

46 Re: Further Thoughts on Fourth-and-2

Here's your explanation: http://www.advancednflstats.com/2009/09/4th-down-study-part-3.html

-------------
Sports talk radio and sports message boards are the killing fields of intellectual discourse.

51 Re: Further Thoughts on Fourth-and-2

There are no details there, aside from mentioning that the conversion rate depends on field position. In any case, it would make more sense to use the "inside 10" rate (~55%) since the field for that play was effectively 2 yards long.

I'm also curious what effect excessive crowd noise would have, since it makes calling an audible almost impossible. That would certainly further reduce the conversion rate.

The total disrespect for the Pats' running game--a point conceded when the Pats lined up 5 wide--allowing the Colts to sell out on the short pass didn't help either. Bad spot or not, the play was very well defended by the Colts.

61 Re: Further Thoughts on Fourth-and-2

I don't understand why "inside 10" would be superior, the point of which being that out-of-bounds line at the back of the end zone serves as an extra defender. The 11 players on defense only need to protect a maximum 20 yards deep box for "inside 10", while they need to defend a max 80 yard box for this situation.

49 Re: Further Thoughts on Fourth-and-2

this was terrible when you clumsily shoehorned it into QR yesterday, and it's terrible today too. kill yourself, bill barnwell.

58 Re: Further Thoughts on Fourth-and-2

Once again, an analysis that misses the key point.

Besides the cited reasons that 4th and 2 data doesn't apply, the chance of success takes a huge dependency on the payoff matrix, because there are usually more possible outcomes than simply success and failure. Most of the time when a team goes for 4th and 2, it is either because they are outside desirable field goal range or because they are desparately trying to keep a drive alive at the end of a game. In either of these cases, there is a big difference between making the first down and breaking off a big play. This means that the D usually can't sellout to the same extent to stop the first down as they could in this circumstance, which means that the NE chance of making the first down was considerably less.

64 Re: Further Thoughts on Fourth-and-2

I don't think they could sell out and ignore the big play here, I certainly wouldn't have as D coordinator.

I haven't seen one good reason posted so far as to why NE would have a lower than average chance of making it?

IND has a good DEF, NE has an even better OFF...
IND was defending NE well. really? 34 pts in a game is doing well? NE was having a good day on OFF.

And so on and so forth.

I also wouldn't be second guessing Belicheck's play calls. I think there is a lot more subtly in how much people are lining up against the pass vs the run than people realize. I am sure BB knows that a run is more likely than a pass to get at least 1 yrd on 3rd down. I am also sure he had his reasons for choosing a pass.

200 Re: Further Thoughts on Fourth-and-2

I don't think it's correct to assume that a team plays consistantly the same throughout an entire game. New England put up 34 points, yes, but 24 of those points came during about a quarter's worth of play; from between the first score in the middle of the first quarter through to the Edelman touchdown in the middle of the second. Over the next 37 minutes of play, the Patriots scored only ten points, (and on driving of 38 yards combined to get those points, thanks to excellent starting field position). The Patriots did move the ball on other drives, notching 12 first downs, but the defense had also forced turnovers on both of the Patriots longest second-half drives.

65 Re: Further Thoughts on Fourth-and-2

The advancednflstats.com argument is just wrong.

It is. There's no other way to look at it. The simplest argument is that the "judging coaches' decisions" websites (advancednflstats, footballcommentary) are making naive judgments much like the Monty Hall problem, assuming that the "naive estimator" for win probability should factor into decisions *ignoring the fact that one path allows you to make more decisions than another one.*

In that situation, you punt. Period. Why? Because going for it allows you to make *one* decision - you get the first down, you win. You miss the first down, you lose. Punting allows you to make *many* decisions and to *adapt* based on the result.

The only time when going for it makes sense is if you believe you're a worse decision maker than the other coach. Otherwise, you punt. (Put another way, if Belichick's so damn smart, why didn't he have his team punt so he could have several chances to prove his defensive brilliance?)

96 Re: Further Thoughts on Fourth-and-2

The Colts scored every single time they were inside the Patriots' 30. Belichick's best estimate would have been that a failed 4th down conversion is a 100% lose rate. Maybe if he was really smart, he could say that it's "most likely 85% or higher." (85% is the roughly the 4th root of 50% - it's the point at which a 4/4 outcome is more likely than any less than 4).

72 Re: Further Thoughts on Fourth-and-2

Wrong, Wrong, Wrong

How is this so hard to understand? I mean know wonder my quant finance program was 99% Chinese. They would get this, model it in Excel, run the monte carlo simulation 5 different ways within 10 minutes and report to you on the outcomes

and that is with no understanding of American Football or who this Ball Billichuck is.

Seriously, WTF, all this discussion and you missed the whole point.

it is not a zero-sum decision.

Wow, saddened by the American public. Unless of course you are from elsewhere, then I will be a bit happier w/ us Yankees not being so stupid

82 Re: Further Thoughts on Fourth-and-2

You can't Monte Carlo this without modeling Belichick and Caldwell, and good luck doing that. Just because an "average" team does something doesn't mean that either one of them would.

92 Re: Further Thoughts on Fourth-and-2

By replacing Belichick and Caldwell (and Manning) with NFL historical averages.

This is why I hate ZEUS and arguments like this. Football's a decision tree, and the historical averages do not equally populate that decision tree. If you applied ZEUS to chess, for instance, the behavior of tons of Grand Masters would frequently look moronic - the Hedgehog Defence, for instance, would've looked incredibly stupid prior to the 1970s.

(Note... to be fair, it's entirely possible that Belichick made this decision because Manning had been countering everything Belichick threw at him all day, and so he figured his chances were better with his offense. I find this logic dubious given the fact that Manning scored on only 4/14 previous drives. It's also possible that Belichick thought that his 4th and 2 play was very unlikely to be stopped by the Colts, but I also find this logic dubious considering his 3rd and 2 play was stopped.)

73 Re: Further Thoughts on Fourth-and-2

Put another way, if Belichick's so damn smart, why didn't he have his team punt so he could have several chances to prove his defensive brilliance?

Somehow, I doubt that proving his defensive brilliance was at the top of his mind.

Who takes seriously the dismissal of a complex argument with the sentence "Period."? Who says, "Oh, now I'm conviced. You just said 'Period.' Now I see the folly of my ways."

In this case it was particularly funny, because after you say "Period." you go on to elaborate at great length.

83 Re: Further Thoughts on Fourth-and-2

No, the "period" was meant to indicate that it has little to do with average outcomes of the game at that point, and if you overthink which decision "on average" wins out, you'll miss the point. It has everything to do with which option gives you more future decisions. Punting does, so you take it.

189 Re: Further Thoughts on Fourth-and-2

No, that's not what "period" means in English.

I think you should look it up. Here's the definition I think makes the most sense for your context: the end or completion of something. In any case, there's no denotation or connotation of "period" that includes average outcomes.

74 Re: Further Thoughts on Fourth-and-2

I would have said the exact opposite. Going for it gives you more opportunities to win. You can win by getting the first down. If you fail to get the first down, you can win by preventing the Colts from moving 30 yards and scoring. Punting, however, removes the opportunity of winning by getting the first down -- the only opportunity you have to "make decisions" to win is on defense. And with the way the Colts were playing on offense, you were likely to have about as many "decision points" defending a 30-yard field as you would have defending a 70-yard field.

88 Re: Further Thoughts on Fourth-and-2

I would have said the exact opposite. Going for it gives you more opportunities to win.

No, you're thinking about "paths to win," not "remaining decisions." Just think about the "expected number of plays remaining in the game."

If you go for it, you've got the 1 4th down play (which is much more important than the others) and then if you fail, the ~3-4 plays they will take to score. If you punt, you've got at least twice that number.

Plus, if you punt, you've put the Colts into a situation where their possible decisions are reduced - they've got 1 timeout, and need to go 80 yards. That reduces the number of possible plays they can call, which increases the chance that you'll be able to make a decision on defense that's better than their decision on offense.

The only way the decision to go for it makes sense is if Belichick thought that Peyton Manning was smarter than him (which, granted, might not be insane...).

And with the way the Colts were playing on offense, you were likely to have about as many "decision points" defending a 30-yard field as you would have defending a 70-yard field.

The Colts scored every time they were within 30 yards of the end zone. They scored only 4/11 times when they had ~80 yards to go. Based on every drive in that game, I'd say that you're wrong.

94 Re: Further Thoughts on Fourth-and-2

the colts two touchdown drives prior were 6 plays 79 yards and 5 plays 79 yards. so they gain maybe two more opportunities? and in the second half (the more situationally significant) drives, the colts were 3 for 5, so...

102 Re: Further Thoughts on Fourth-and-2

It's not "entirely" about number of decisions - you want the situations to be neutral (equal gain/loss) or biased in your favor. If you gain a bunch of crappy decisions, it doesn't help you. Going for it on 4th down gives you one favorable decision situation, and if you fail, maybe 3-4 crappy decision situations. Punting gives you a lot of favorable decision situations.

and in the second half (the more situationally significant) drives, the colts were 3 for 5, so...

With only 1 of those 5 short enough to conceivably happen within 2 minutes. Granted, it was the most recent one - it is possible that Belichick thought he was out of tricks, and Manning was going to score no matter what. I just don't find that entirely likely, but it is possible.

119 Re: Further Thoughts on Fourth-and-2

I've heard reports that Belichick consulted with his defensive coaches prior to making his decision (and like I said I think he made his decision too late), and if this is true, I think it likely that he was told that his pass rushers were gassed. His depth in this area was depleted, what with one of his best pass rushers having been hurt earlier in the game. His defensive backs are not outstanding. I think he simply decided that 40 extra yards of field position were not nearly as valuable as they would normally be, given the physical state of his pass defense.

I wish there was an in depth study on the effectiveness of pass defense, when facing a good offense, factoring the number of previous pass rushes. It would be hard to structure, I'm sure, but it might give some insight regarding the chances of success in such situations.

123 Re: Further Thoughts on Fourth-and-2

If that's true, then he should've simply let Indy score as soon as possible. If his decision is predicated on the fact that Indy has a strong chance of scoring from their own 20, they've got a crazy chance of scoring from their opponent's 30.

(and like I said I think he made his decision too late),

I couldn't agree more. The problem is that people are yanking that decision out and trying to judge it by itself, which is crazy. You can't, because football requires decisions before and after a play. It'd be like looking at a gambit play in chess and saying "oh, that's terrible, because losing a piece is bad" - in general, the comment is true, but if the player sets it up correctly and takes advantage, it can be a good decision (or a neutral one).

A 4-down strategy might've been a good idea in that situation, but it doesn't matter because Belichick didn't commit to the idea, either before (the 3rd down play + timeout) or after (attempting to stop the Colts after the decision was predicated upon the inability to do so).

Failing to follow a consistent strategy is a great way to allow an opponent to make better decisions than you, and this just showed it.

124 Re: Further Thoughts on Fourth-and-2

Oh, I pretty much agree. The time to decide that your pass rushers are dangerously gassed is before the Colts kick the ball to you, not after you have run three plays. If this attitude is adopted prior to your offense taking the field, then everything changes. I think Belichick was like a chess player, who, under great emotional pressure, failed to plan enough future moves. Nobody is immune to that sort of thing.

135 Re: Further Thoughts on Fourth-and-2

No, what's important is the difference between their likelihood of scoring from their 20 and their likelihood of scoring at the Pats' 30. But this is still a terrible, terrible way of thinking about a strategic game.

138 Re: Further Thoughts on Fourth-and-2

well, if your position is that the colts are 100% to score from the 30, while at the same time saying it's less than 34% from their own 30, I think I'd have to respectfully disagree.

eta. you're still assuming a 100% failure rate on the fourth down play, as well

141 Re: Further Thoughts on Fourth-and-2

well, if your position is that the colts are 100% to score from the 30, while at the same time saying it's less than 34% from their own 30, I think I'd have to respectfully disagree.

Really? Because, uh, that's pretty much what actually happened in the game.

I mean, argue against the selection all you want, but the Colts did score 100% of the time inside the 30, and less than 34% of the time from their own 24 (which is the rough location of where the punt would've ended based on Hanson's actual punting for the day).

eta. you're still assuming a 100% failure rate on the fourth down play, as well

No, the success rate on the fourth down play is actually immaterial to the decision making process. You don't choose between going for it and punting. You choose between punting and another play (But still: I'm not assuming a 100% failure rate, as that would imply you better freaking punt the ball).

178 Re: Further Thoughts on Fourth-and-2

Pat, doesn't your entire argument start with the premise of the Patriots' offensive failure?

I don't think, as a Head Coach, you should be putting an inordinate amount of thought thinking of "What's our best opportunity to win after we fail to convert?"

155 Re: Further Thoughts on Fourth-and-2

Offense and defense are not the same concerning "decisions" or choice. Choice resides with the offense in football. Offense gets to determine defensive personnel, fer gawd's sake (this one facet is exploited a lot more than we really pay attention to). The best a defense can do is to limit offensive playcalling options by being great at something (yes, sometimes this "best" is phenomenal).

Defense is not about choice, but reaction (based, admittedly, on a lot of study regarding tendencies - but still basically reacting to what the offense does). Defensive players have to keep several things in mind and be ready to react (ladies and gentlemen and Patriots fans, I give you Andy Katzenmoyer!). Pass or run? Screen or draw or downfield throw? If the offense can't stop a pass rusher what does their protection look like and what plays can they call out of that formation? Yes, sometimes down and distance - and time left - dictate the choices the offense is likely to make, but the Patriots have a very, very good offense. Their choices on any one play are not so limited. But their choices on defense are, in fact, constrained.

A team does not maximize their choices/decisions by punting. They give over decision making power to the other team. A team maximizes their control by MAKING A CHOICE on offense.

I happen to think the whole set of downs process was flawed (I don't think the Pats were in four down mode), so going for it on 4th was not the optimal decision - in my opinion. But I also think that the flaw being criticized in #65 is not a real one. (Not that anyone's mind is being changed by all this foofaraw! Great conversation - mostly. I feel like I'm in a really intelligent bar after three drinks.)

210 Re: Further Thoughts on Fourth-and-2

The best a defense can do is to limit offensive playcalling options by being great at something

Robber coverage is specifically designed to anticipate a quarterback going to his hot read. That alone is better than your "best." But it's a far cry from what teams like the Steelers and the Eagles do in disguising coverage and specifically weakening themselves in an area anticipating that the offense will not take advantage.

Defense is not about choice, but reaction

Sorry, I don't agree. Playcalling in college tends to be more about reaction, but in the NFL you see so many defensive alignments and defensive plays that a lot of it is about choice. You'll see college teams play Cover-2/Cover-3 an entire game. You'll almost never see that in the NFL.

In the situation in question, the Patriots definitely had a choice: play a soft cover zone, force Manning to work the clock and hope you can force an incompletion or two, or rush him hard and hope to force a play on defense. And that's just vanilla thought - it's perfectly possible that, with 80 yards to go, Belichick could offer an underneath zone a few times, then show it and take it away suddenly. The Colts had one timeout, so really any big play seriously reduces their chance at winning.

(It should be noted that other people have suggested that Belichick went for it because he believed his defense was basically gassed and would never be able to hold Manning. If so, that just makes the entire drive a strategic disaster, and the 4th down decision doesn't matter as it becomes a desperation ploy rather than a thought-out strategy.)

69 Re: Further Thoughts on Fourth-and-2

First let me say I think they should have punted. I don't think it's the worst call ever, but I flat out do not agree with it.

"The important factor that the cacophony of responses seems to be missing is that you can't judge Belichick's decision by the fact that it didn't work. As we've mentioned more than once in these pages, you cannot judge decisions by their outcome."

Maybe not by the outcome, but you can judge it by what Belichick was risking by going for it there just to gain a few percentage points in win probability. Is that marginal gain worth giving Manning the ball with less than 30 yards to go? That gives the psychological edge to Indy's offense, now knowing that their job is much easier than what it would have been had the Pats punted. And a loss means almost no chance at HFA over Indy, a defense left wondering why they weren't allowed to win the game, and ex-Patriots ripping Belichick in the media. I wouldn't want to deal with all of that for such a marginal gain by going for it.

I don't see anyone taking notice that they had a better chance of winning the game than losing it just by punting (>50%). No matter how great Manning is, there's a greater chance for mistakes when you have a field that's twice as long. One sack or penalty can kill a drive.

77 Re: Further Thoughts on Fourth-and-2

"I don't see anyone taking notice that they had a better chance of winning the game than losing it just by punting (>50%)."

if we're only using generalizations, then I would say that I would rather have the 99% chance at victory once we convert than the 50% chance after a punt. these numbers are completely wrong, by the way, and there are a number of posts already explaining why, but even using your logic the call to go for it is still defensible.

80 Re: Further Thoughts on Fourth-and-2

What he was risking was losing the game if he didn't convert. He also risks losing the game if he punts. The argument that going is it was incorrect because you're more likely to lose if you fail to convert is flawed, because it presupposes that you'll fail to convert.
A loss does mean HFA is almost impossible, but it means that no matter how you lose, so that's irrelevant. The other two statements boil down to punting to avoid criticism. If that's your argument, well, it's not necessarily wrong - as you point out, a team that punts is still more likely to win than lose - but I hope my team has a coach who makes decisions without worrying what they'll say in the papers the next day.*

*I should note that, because "my" team is the Lions, I will likely never know if they have such a coach.

90 Re: Further Thoughts on Fourth-and-2

Belichick was risking by going for it there just to gain a few percentage points in win probability.

A *few* percentage points? Dude, if they converted it in bounds, they won. That's a lot of percentage points.

70 Re: Further Thoughts on Fourth-and-2

How many Colts fans at Lucas Oil thought, after celebrating the unsuccesful 3rd-down, that Belichik was crazy to give IND the ball at NE-28?!

How many Colts fans at Lucas Oil were VERY worried about the fact that NE were going for it on 4th-down instead of punting?

It seems that every Colts fan -or, at least, a lot of them- thought they had a great chance of winning the game IF the Patriots punt the ball, and that they were in serious trouble with the 4th-down attempt.

Well, also did Belichick...

As far I'm concern, the decision wasn't crazy or wrong. And if there is something Bilichick learned about this is NOT that they should have punted, but that thay should have thought about going for it BEFORE the THIRD down.

He will do it again.

97 Re: Further Thoughts on Fourth-and-2

After the Pats didn't make the 3rd down, every Colts fan worth their salt KNEW Belichick was going for it. I mean, we've watched this guy every year, and sometimes twice a year, for nearly a decade. We KNEW that he was going for it, because that's what he does.

As for how nervous we were, I bit off three fingernails between the end of the 3rd down play and the end of the 4th down play.

As for after the play, we really weren't thinking "We got this." It was more a "Let's see if Manning can earn that ridiculous contract he's about to get."

It wasn't over until Reggie scored. And even then, we were all sitting there going "Oh hell, it's going to be a Hail Mary with Randy Moss streaking toward the end zone, and if the Pats protection holds up for five, six seconds, he's going to drop a bomb right into his hands and we're screwed."

Basically, it wasn't until the clock hit 00:00 that we finally went "Holy crap we won".

98 Re: Further Thoughts on Fourth-and-2

For what its worth, the Colts radio play-by-play announcer and the colorman (former all-pro lineman Will Wolford) were downright GIDDY when the Pats lined up to run the fourth down play. Of course, they were even happier after the play failed, but prior to the play they were figuratively "licking their chops" for the chance to stuff the play in BB's face.

112 Re: Further Thoughts on Fourth-and-2

Your comment brings up an interesting point regarding the "emotional" aspects of the decision. I've heard several of the Colts defensive players say things like "that was total disrespect towards us" and "we needed to prove that we could stop them". Granted, this could be just idle chatter. However, BB's decision to go for it was so unusual I think it had an unusally high emotional impact on the Colts players. Of course, we will never know how (or if) these emotions impacted the play, but it is interesting to consider.

76 Re: Further Thoughts on Fourth-and-2

A few quick arguments to a lot of what I've seen people say in favor of going for it. (Note-I'm a Giants fan so a mostly impartial observer)

1. @barnwell - There was no point where I thought it was a good idea to go for it. As soon as Belichick didn't put out the punt unit on 4th down I was screaming at the TV that it was the wrong decision. I have witnesses. I'm not judging based on the outcome. At no point was that a good decision.

2. @Statguys - You say that going for it favors winning based on percentages and probability? Thats cool... I'm in... BUTTTTTT if that's the case and Belichick was playing percentages then he should have been playing them on 3rd down as well. If he wanted to get the first there and was going to go on 4th down then it should have been two down territory and he should have run on first down to milk the clock and work on getting closer/getting the first. Belichick admits that he didn't decide that he was going for it on fourth down until after third down. That is what I'm arguing against.

Also.. In what world does that percentage of drives end in a touchdown? You can't claim all of these "hard" stats and then claim Peyton scores TDs on more than 50 percent of his drives. Thats just plain wrong.

3. @Others - Oh... so the New England offense is a dominating offense? That why they didn't score more in the fourth I guess. Their offense had been just as bad as their defense in the fourth if not worse. I mean... They just got stopped to what would be a 3 and out. Is that not a sign already?

And another example of Belichick being outcoached in the game? The Indy CBs knew that they were going to run that play on fourth down. Appparently the coaches told em all week in meetings that if it came down to a fourth down like that the patriots would call that play.

78 Re: Further Thoughts on Fourth-and-2

nobody's saying he scores on more than 50% of his drives after a punt; they're saying that if he scores on more than 34%, that's one time in three, then going for it provided a better chance to win.

as to the third down/being outcoached...I don't think you'll get too many arguments, even from bill

93 Re: Further Thoughts on Fourth-and-2

they're saying that if he scores on more than 34%, that's one time in three, then going for it provided a better chance to win.

Colts vs NE, prior to that drive: scored on 4/13 drives. That's less than 1 in 3. Therefore, the best evidence that Belichick had at the time should've told him to punt. And that completely ignores the time situation, which only tilts it more in favor of punting (a terrible guess would say that the Colts only had 1 scoring drive of less than 2 minutes - 1/13 makes it a terrible decision - note that this is an awful, awful guess).

But making decisions based on expected winning percentage alone is a very poor way to play a strategic game.

103 Re: Further Thoughts on Fourth-and-2

Your assessment is the same one Peter King makes...if that is the company you want to be in then go ahead. Minimizing losing is the same as maximizing winning in football, unless you are playing for a tie, which was a highly unlikely <1% chance under the scenario that game was in. It's super easy to do a more complex evaluation of the decision, without using numbers at all.

You have 9 viable variables IMO and they are

Conversion chance
Chance Colts Score if not converted
Chance ball turned over on downs after successful conversion
Chance Colts TD after successful conversion
Chance of Colts TD on punt play
Chance of Colts TD after successful punt
Chance of Pats scoring after colts TD on punt play
Chance of pats scoring after colts TD after failed conversion
Chance of pats scoring after colts td after successful punt

109 Re: Further Thoughts on Fourth-and-2

It's not about minimizing losing. It's about maximizing the number of opportunities for your players to win the game/you to have a situation to choose the right decision.

You have 9 viable variables IMO and they are

... completely unknowable because the 2009 Patriots and 2009 Colts have never played each other in the regular season.

Analyzing football like that is incredibly silly, because you can't possibly know any of those variables, to any precision. You can't base your entire decision on what the outcome of other games has been. You've got to base them on what you think you can actually do.

And if Belichick thought it was worth essentially gambling the entire game on a 2-yard passing route... yeah, that's not a smart decision. And the playcall does matter, because the decision to go for it is based on how favorable that 4th and 2 situation is for you. And if that's the best call he could come up with... it wasn't a favorable situation.

87 Re: Further Thoughts on Fourth-and-2

If the odds of picking up these two crucial yards against a defense in full sudden-death sellout mode were 60% or more (as I keep seeing), well, then the Colts beat fairly long odds by making the stop on both 3rd and 4th downs (at about a 15% chance to stop both plays given the 60% success assumption). That's the part of the analysis I most question; that going for it in that situation was even a straight-up 50% winning proposition. Maybe we do need to take a look at the actual on-the-field results-- at least partially-- in making an assessment. The 3rd-down play had no chance as executed and the 4th-down play was well-defended with the ultimate result coming down to a virtual coinflip (even insofar as to how a replay decision would have been ruled). So I don't think it's outrageous to believe that the Patriots' odds of converting on 4th down were even as low as 40%.

In summary, in football as opposed to dice rolls, I think it's okay to at least consider the result of a play in the final opinion. No situation or scenario is ever the same, and the actuarial data aren't perfect, as the authors have acknowledged. So as such on this particular decision I agree: too close to call. Anyone claiming to have "proved" that this was a good decision or bad decision is implicitly relying on imperfect data and assumptions.

105 Re: Further Thoughts on Fourth-and-2

Here's the issue. One can logically hold only one of two positions. Either the 60% conversion rate on every potential 4th and 2 is correct in which case each and every coach in professional, college and high school football is wrong. Or, the 60% conversion rate on every potential 4th and 2 is wildly overstated.

The subset of 4th downs that coaches go for is apparently around 50% this year. Presumably, some of these are in desperation in which case the defense plays it safe conceding an increased chance for a conversion as opposed to a long, quick TD, or they are done because the offense thinks they have a matchup that they can take advantage of, once again increasing the chances of conversion.

Whatever one thinks that the chances are, and I've seen amazing conclusions drawn from data that can't support the results on this issue, it is apparent that coaches treat the conversion chances in that situation as being much lower than 60%.

As an interesting and much more useful exercise, do these calculations backwards. Calculate the number of times coaches punt in that situation and how many times they go for it and from this data determine what coaches perceive the precentage to be.

Without going through the numbers, I'm going to guess that professionals act as if they calculate the chance to be significantly less than 25%.

Whatever the number is, compare that to the 60% being thrown around here and then ask why is it that professionals who do this every day act, put their careers on the line, using an estimate much lower than the data from 4th down conversions which are actually attempted.

A decent statistician would instantly recognize that the data set of all 4th down conversions attempted and the data set of all possible 4th downs are two different data sets and there is no possible mechanism besides conjecture for making them equivelent.

Since the data is not conclusive one might think there would be a little more humility on the part of statisticians who lack the data to support their conclusions.

Luckily, we live in a free market system and any of these statistician hobbyists are more than welcome and able to get themselves at least high school football coaching positions and show the world that they are correct because the data certainly doesn't conclusively prove that they are.

130 Re: Further Thoughts on Fourth-and-2

Most high schools probably shouldn't punt. Most have very poor punting games to start with. If you're going to net 20 yards a punt, what's the point? Particularly in a game that is generally based on a big play. There aren't many high school drives that go 10+ plays leading to TDs. Too many young kids making too many mistakes.

A game (college and above) that prevents bigger plays and has better kickers warrants a punting game.

116 Re: Further Thoughts on Fourth-and-2

I think "put their careers on the line" explains what you're describing, though. If the Patriots had punted and Manning had driven the Colts down the field to score the TD and win the game, nobody would have criticized Belichick for punting on 4th-and-2. (They might have criticized him for other things, of course.)

A coach who isn't 100% sure that the ownership/management is completely behind him might quite rationally (for his own self-interest) pick a non-optimal team strategy for any individual game, on the thinking that for his own career, it is better to lose conventionally than unconventionally, even if it lowers his overall winning percentage. The coach's self-interest and the team's interests are not always aligned, especially during an individual game.

111 Re: Further Thoughts on Fourth-and-2

Great article Bill. I just have a problem with this part:

"The Colts aren't going to score every time they get the ball on the opposition's 29-yard line, but they will score most of the time. Toss in momentum and the quality of the Colts' offense versus the Patriots' secondary, and you can estimate, say, an 85 percent chance of the Colts scoring in that situation."

While the likelihood of the Colts scoring on the opposition's 29 may be 85%, the Colts specifically needed a touchdown in this situation- not just any score. I'd imagine that removing the possibility of a field goal would lower that 85% scoring figure quite a bit. Advanced NFL Stats figured the average NFL team to get a touchdown in 53% of the time in that same situation. I don't believe that even Peyton Manning exceeds average to the extent of bumping that probability all the way up to getting a touchdown on 85% of any drives from any position (let alone with only 2:00 left to play).

117 Re: Further Thoughts on Fourth-and-2

The Colts scored a touchdown 4/4 times within the 30 that game. 85% is roughly the point at which the likelihood of getting 4/4 exceeds 50%. 85% is probably the best guess that Belichick could've made.

54% seems wildly unsupported by the evidence from that game.

142 Re: Further Thoughts on Fourth-and-2

It's a bit immaterial. The problem was that Belichick didn't think that there was a significant difference between their TD% from the Pats 30 to the Colts 20-25. You only punt if you think it's going to make a meaningful difference for the defense.

Based on how thoroughly the Colts had hosed the Pats' D on the previous drive, it seems eminently justifiable for BB to lack faith in its ability to stop the Colts from either their own 30 or from the Colts' 20.

Belichick has a functioning memory. He knew that the Colts' offense was red hot and that his defense was gassed. Why is he obliged to send it back out on to the field?

To let the Colts get their 3rd 80-yard <2 minute TD drive of the quarter?

144 Re: Further Thoughts on Fourth-and-2

Assume that logic. In that case, the 3rd and 2 call was horrible, the timeout taken was really awful, as was actually preventing the Colts from scoring after that, because you've already conceded that you can't prevent them from scoring a TD. All you do is shift the "bad decision" from 4th and 2 to nearby decisions.