Writers of Pro Football Prospectus 2008

17 Nov 2009

Further Thoughts on Fourth-and-2

by Bill Barnwell

(Ed. Note: This was the introduction to yesterday's Quick Reads on ESPN.com. We're running it as separate commentary on FO in order to prevent the fourth-and-2 discussion from blocking out discussion of the other 14 games in the Quick Reads discussion thread. The rest of Quick Reads for Week 10 can be found here.)

Too close to call.

It's not the satisfying answer to the Great Belichick Debate, which has seen him declared to be either an infallible genius or an overzealous tinkerer by most observers, but it's the most accurate one.

No matter how we tinker with or adjust the probabilities to account for the game situation and the quality of the two offenses and defenses, it's difficult to find a dramatic difference between the choices of going for it or punting.

Take the first option, the one Bill Belichick chose. Teams attempting to convert a fourth-and-2 have successfully done so at a 48.6 percent rate this year, well down from 62.3 percent the year before. The "true" likelihood of such a play being converted by an average team in an average situation is closer to that latter figure, based on historical data.

Of course, the phrase "…average team in an average situation" simply does not apply. The Patriots don't have an average offense, they have the league's best by DVOA; the Colts' defense ranks sixth. Although the Patriots had failed to convert in two of their three previous third-and-2 situations (including the one directly preceding the decision to go for it), their chances of making it in this situation are greater than the average team's.

On the other hand, the probability has to be adjusted for the situation. Robert Mathis had been abusing right tackle Nick Kaczur all night, meaning that Brady wouldn't have time for the combination of Randy Moss and Wes Welker to run anything resembling an intricate route pattern. Dwight Freeney lurked on the other side. Furthermore, the probability of a team going for it on fourth down might very well be artificially high because of selection bias -- teams are far more inclined to go for it on fourth down against the Lions or the Rams than they are against the Vikings or Ravens.

Throw all these numbers into a big soup, spin them however you'd like, and you'll end up with an expected conversion rate of about 60 percent. It might be 63, it might be 57; truthfully, it's not going to be enough to change our analysis.

The Colts aren't going to score every time they get the ball on the opposition's 29-yard line, but they will score most of the time. Toss in momentum and the quality of the Colts' offense versus the Patriots' secondary, and you can estimate, say, an 85 percent chance of the Colts scoring in that situation. That makes Belichick's decision to go for it a little stronger, upping the Patriots' chances of winning by going for it to 66 percent.

Then, it comes down to punting and where Manning gets the ball, which requires even more theoretical assumptions. Chris Hanson has a 39.6-yard net average, but it was in a dome, and the Colts don't have great return units. If we just assume 40 yards, the Colts get the ball on their own 32-yard line with two minutes to go and one timeout. If you believe that the Colts had a 34 percent chance or better of scoring a touchdown in that situation (100 percent minus the 66 percent chance we mentioned a moment ago), Belichick was right. If you think the odds are below 34 percent, Belichick was wrong.

If you disagree with the expected percentages of conversion above, Mike Harris (creator of the playoff odds simulator) has developed a nifty calculator that lets you plug in your own averages and figure out whether Belichick made the right call by those figures. You can find that calculator here.

Of course, we haven't even considered the possibility of running the ball as opposed to passing it on fourth down, turnovers, onside kicks, or the Patriots scoring on a game-winning drive. The bottom line is that a mathematical analysis of the decision boils down into too many assumptions and inapplicable historical expectations to say very much about one decision on one drive in a very unique situation, and when we make the broadest assumptions possible about the decision, the decision isn't close to clear-cut.

The important factor that the cacophony of responses seems to be missing is that you can't judge Belichick's decision by the fact that it didn't work. As we've mentioned more than once in these pages, you cannot judge decisions by their outcome. You have to consider the process that goes into them, and then decide whether they're right or wrong at the moment they're made.

Think back to another controversial Belichick decision made in the heat of a prime-time game, his decision to take a safety on purpose down one point during the fourth quarter of a Monday night game against the Broncos. Of course, the Patriots ended up getting the ball back and won the game. Belichick took virtually no flak after the game for his unconventional choice, and was instead hailed as an aggressive, brilliant game manager.

If Kevin Faulk stumbles two feet forward, Belichick is being spoken about in those glowing tones today by virtually everyone lining up to criticize him. That doesn't make his decision correct or incorrect, any more so than Faulk coming up short does. If Belichick's decision was wrong, it was wrong from the moment the playcall went to Tom Brady. And with everything we know about the situation, it's impossible to say whether that was truly the case.

(Another Ed. Note: If you would like to read a similar analysis of a completely different coaching decision, find a copy of the 1986 Bill James Baseball Abstract and read his commentary on Tommy Lasorda's decision to pitch to Jack Clark with a base open late in Game 6 of the 1985 NLCS. Some truths are universal.)

Posted by: Bill Barnwell on 17 Nov 2009

214 comments, Last at 18 Nov 2009, 6:55pm by sethburn

Comments

1
by Dales :: Tue, 11/17/2009 - 12:31pm

" If you believe that the Colts had a 34 percent chance or better of scoring a touchdown in that situation (100 percent minus the 66 percent chance we mentioned a moment ago), Belichick was wrong. If you think the odds are below 34 percent, Belichick was right."

Am I reading this wrong, or is this backwards?

2
by John (not verified) :: Tue, 11/17/2009 - 12:32pm

Pretty sure it's backwards. Had the same thought.

19
by peterplaysbass (not verified) :: Tue, 11/17/2009 - 1:14pm

No, I think this is right.

Going for it means there's a 66% chance to win.
Punting means there's a 34% (or more/less) chance to *lose*.

So if the Colts' odds of scoring after a punt are 30%, then that's a 70% chance for the Patriots to win and punting is the right decision. If the Colts' odds of scoring are 40%, then that's 60% for a Patriot victory and then punting would be the wrong decision.

22
by peterplaysbass (not verified) :: Tue, 11/17/2009 - 1:15pm

Wait- I read it again and it looks like it was backwards.

I should read twice before writing once.

3
by stevemoy (not verified) :: Tue, 11/17/2009 - 12:33pm

If you believe that the Colts had a 34 percent chance or better of scoring a touchdown in that situation (100 percent minus the 66 percent chance we mentioned a moment ago), Belichick was wrong. If you think the odds are below 34 percent, Belichick was right.

Other way around, right?

Nice post overall though - I'm a Pats fan, but all of the hyperbole and hand-wringing and general lack of understanding of basic probability by so many so-called experts is bothering much more than the loss itself.

4
by Merr (not verified) :: Tue, 11/17/2009 - 12:34pm

Thank the gods, some stats nerds who finally see this for what it is.

60
by Eddo :: Tue, 11/17/2009 - 3:17pm

Every "stat nerd" I've read over the last two days has had nearly the *exact* same opinion as Barnwell on this matter; it was a very close call, but Belichick probably made the correct decision.

5
by tunesmith (not verified) :: Tue, 11/17/2009 - 12:37pm

Now I want to see a monte carlo simulation!! Hop to it, men!!!

50
by Phil O'sopher (not verified) :: Tue, 11/17/2009 - 2:28pm

Easily could plug simple model into Excel and run the Crystal Ball (or @risk) simulators to see results.

Would be interested to see that.

6
by DaninPhilly (not verified) :: Tue, 11/17/2009 - 12:41pm

All the analysis I have seen indicates what I thought immediately with my gut - this was either the correct call beyond doubt or at least a push. When a decision is too close to call, it is the job of the coach to make the decision, and the fans job to second guess him.

Had the Pats converted, BB would be called a genius again and again (and again!). As they failed to convert, his thinking is being scrutinized.

As for me, I don't much like Bill as a man, but give the devil his due. He's on heck of a coach, and that was a call which is both gutsy and defensible. It was a good call and I cannot second guess him on it.

7
by Keasley (not verified) :: Tue, 11/17/2009 - 12:41pm

They way I've been thinking about it, the decision to go for it on 4th down was the right one. It was a couple things leading up to it that were wrong: wasted timeouts, the play run on the 3rd down. Even with those things, it was still the right decision. And if it had been a more bog-standard NFL coach you could rationalize the imperfection of it all. But given that it was the vaunted Belichick, sure fire hall of famer and in the conversation as best coach of all time, this tarnishes his rep, especially if the Pats go on to lose a road game in the playoffs (and especiallly if that road game is in Indy)

27
by rdy4thefiesta :: Tue, 11/17/2009 - 1:25pm

You are absolutely right. If he was going to go for it on fourth down, he should have known this before the third down play. He could have directed his offensive coordinator to call a running play that would have made it a shorter fourth down.

Another mistake by Belicheck I'd add in was the seemingly non aggressive play calling on the drive that led to the Pats final field goal. I was saying at the time that there was too much time to be running clock and that the Pats should aggresively try and make it a 17 point game.

8
by Joe T. :: Tue, 11/17/2009 - 12:41pm

I can't think of a better situation in which to fake the punt than the one in which the Pats were in Sunday night.

16
by Ryan Mc (not verified) :: Tue, 11/17/2009 - 1:02pm

That's what I've been thinking too. I have no problem with the decision to go for it, but why not do it from the punt formation when you've got the element of surprise in your favor?

23
by Bobman :: Tue, 11/17/2009 - 1:15pm

Now you're really mucking things up.

I don't know what the data on fake punts is, and it probably suffers from small sample size, but I'd assume the trickeration would not carry a lot of weight--especially after a timeout. (Now if they had Randall Cunningham at QB, a shotgun snap and QB-punt might have landed Indy on their own 5.)

I sided with BB in his choice--as a Colts fan I was a lot less worried about a punt than a potential conversion. But taking the ball out of the hands of your highly regarded QB and trusting it to a punter who played QB in HS, or the equivalent, or relying on a direct snap to a protecton back that your team practiced all of six times last week, does not sound like a great idea to me.

Of course a 2-yard pass pattern when you need exactly 2 yards doesn't leave a lot of room for error.

35
by Danish Denver-Fan :: Tue, 11/17/2009 - 1:40pm

Oh hear hear - that would've been pure genius! Try to pull a guy offside, playclock at one, drop back and kick it away... Didn't Cassel do it last year? I think it was a snowy game...

38
by Danish Denver-Fan :: Tue, 11/17/2009 - 1:48pm

Oh I thought the poster meant lining up as if to go for it, and then have the QB punt it. Now that, would've been priceless... And not total lunacy, am I right?

114
by DomM (not verified) :: Tue, 11/17/2009 - 6:28pm

Yeah, against Buffalo in the season finale (although I don't think it was from fourth down). As a result Matt Cassell has 57 yard net punt average.

9
by Bill Barnwell :: Tue, 11/17/2009 - 12:41pm

Typo corrected (I think)...

10
by Levente from Hungary :: Tue, 11/17/2009 - 12:46pm

Thanks for splitting it off from Quick Reads. Actually comments to Audibles was unreadable as all commenters ignored 15 games (14 others + 99% of this game). I think the subject has been blown way out of proportion.

11
by Fan in Exile :: Tue, 11/17/2009 - 12:48pm

"you can't judge Belichick's decision by the fact that it didn't work."

Not to take anything away from the article which I like, but this is just incorrect.

First let me be clear that I understand the argument that leads to this statement it just doesn't apply to football, and I think this call makes that clear.

It only applies in situations were you can clearly see whether the process was good or not. No one can actually tell you what the odds were of them converting nor can anyone actually tell you what the odds were of Peyton scoring after a punt or turn over on downs.

In poker when you look at the cards in your hand and the cards on the table you can calculate the odds of winning, but this isn't poker, and we can't apply the same logic.

You are very correct in saying, "The bottom line is that a mathematical analysis of the decision boils down into too many assumptions and inapplicable historical expectations to say very much about one decision on one drive in a very unique situation, and when we make the broadest assumptions possible about the decision, the decision isn't close to clear-cut."

Since we can't judge whether the process was good or not we either have to judge the result or not judge at all.

The reality in football is that we have to judge the result because by and large it's all we can do. If someone wins that's great if someone losses they get fired.

I admit this a an oversimplification because there are sometimes when we know the process was wrong, but when a team isn't drafting Darius Heyward-Bey we have to go by results.

So as much as it pains me I have to say that BB has won three Super Bowls, I'm not going to second guess him.

32
by jackgibbs :: Tue, 11/17/2009 - 1:37pm

"In poker when you look at the cards in your hand and the cards on the table you can calculate the odds of winning, but this isn't poker, and we can't apply the same logic."

actually your odds in (hold 'em) poker are approximations as well, since there at least 12 (five other players, two burned) cards and as many as 20 cards that are out of the deck, that you have to assume are in play when you make your calculations.

"Since we can't judge whether the process was good or not we either have to judge the result or not judge at all."

Isn't this the same as all the talking heads saying, "it didn't work, so it was wrong?"

42
by Fan in Exile :: Tue, 11/17/2009 - 1:54pm

In poker you don't have all the information but you can still determine the odds, which means that if you have a good process over the course of your career things should even out.

It's not quite the same at least what I've seen has been a lot of judging the process by saying you always punt in that situation. I would also take a slightly broader view that it's not one individual play but their body of work that you judge people on so BB gets a pass.

41
by dbostedo :: Tue, 11/17/2009 - 1:53pm

Whether or not the odds can be calculated shouldn't factor in at all. What they are getting at is whether or not the decision made sense given what was known at the time. In other words, as long as going for it can't be shown to be the wrong decision, you can't claim it's wrong simply based on result.

Look at this at the extremes. Say a team A is leading team B, and team A has the ball with five seconds left in the game. Pretend team A's QB takes the snap and starts to kneel and for some reason throws the ball over his head, at which point team B scores and wins.

In that case, just because the outcome was bad, doesn't mean kneeling was the wrong decision. Given the facts at the time (lead with 5 seconds left) the kneel play is justifiable. The same thing applies to Belichick. Given what he knew at the time for his team and the opponent, the decision made sense. You can't clearly say that the odds were in favor the other way, so regardless of outcome it was a good decision. (Or at least an OK one.)

47
by Fan in Exile :: Tue, 11/17/2009 - 2:08pm

All you've done is restate the argument which I've already replied to.

110
by dbostedo :: Tue, 11/17/2009 - 6:16pm

"It only applies in situations were you can clearly see whether the process was good or not. No one can actually tell you what the odds were of them converting nor can anyone actually tell you what the odds were of Peyton scoring after a punt or turn over on downs."

I'm arguing the the "process" in this case is simply taking all available information, properly weighing it, and making an informed choice. The fact that no one can actually tell you the odds of those things doesn't matter. I don't find the poker analogy to be valid because of the example I gave above taking a football situation to extremes.

48
by Anonymous Jones :: Tue, 11/17/2009 - 2:15pm

Not only is it amazing that someone believes that Barnwell's statement doesn't apply to "football," but it is absolutely astonishing that you used half a page to defend this belief. Ummm, no.

It's better to be lucky than good. I will grant you that.

We can never know the "real" odds of anything (other than on theoretical questions). I will also grant you that.

We can use results to try to determine what the odds might have been in any given situation. That's what this site does. It compiles statistics, analyzes them, and often tries to make predictions about what such analysis says about the risk profile of any past, present or future decision.

That said...

If someone is fairly flipping a fair coin and says to me, "I will give you $10,000 if you pick heads or tails correctly, and you will pay me $1,000 if you choose incorrectly," I will take that bet any time.

If it comes up tails and I said heads, I may *hate* the result, and I may really, really wish the result was different, but I absolutely and totally made the right decision before the result was known.

As for Belichick, he has won consistently like no other coach/team builder in recent history. It's getting to the point that we cannot reasonably attribute this to luck. He is "smarter" than his competitors. The yin/yang of this is that his competitors are "stupider" than he is. How is this possible? Well, I would posit that all his competitors are subject to cognitive biases in evaluating players and in-game decisions, most notably the CYA bias of being deathly afraid of looking stupid to the public. Of course, their desperate desire to avoid looking stupid *condemns* them to looking stupid when someone like Belichick (who is clearly not burdened by the same fear) absolutely schools them.

At what point is everyone going to realize that this is why Belichick wins? If he did the conventional thing every time, drafted according to Kiper's big board, designed every fourth down play the way Norv Turner would, *he would not have been so consistently successful.* It is literally impossible (save extreme luck) to so consistently and overwhelmingly defeat your opponents unless *you are doing something different than your opponents, all of them.* He wins because *you* are stupid.

For my own two cents, this was a close call. It may not have been "right," but I have always thought more coaches should go for it on fourth down. Had the game been tied or the Patriots up by three or less, then it would be different. Clearly, you must punt there. But if I'm the coach in this situation, I want to win it on my own terms. I have one of the best and most experienced offenses in the NFL. This is not a playoff game. I've proven I can outplay the Colts on their home field. I don't even need to win this game. But I will put it on my offense to make these two yards. I will take the risk that *even if it fails,* my defense can still prevent a TD from 30 yards away. I am trying to keep this in my own hands, not in Manning's. I don't know if I would have had the stones to make this decision, but I would *know* I made the right decision the next day after listening the hordes of the ignoramus hoi polloi condemning it. I would *know* that their idiocy is why I win. I would never be able to win without it.

55
by Johnny Socko (not verified) :: Tue, 11/17/2009 - 2:54pm

Wow, I hope you can unpucker your lips for BB's ass long enough to read this. You are vastly overrating BB's edge over his competitors. Off the top of my head I can cite several examples of "coaching" by him.

* 2006 AFC championship game. Pats 1st and 10 ready to run out the clock and seal the game. Opps, 12 men in the huddle. Five yard penalty which eventually leads to a punt and the rest is history. A total lack of organization on the sidelines leads to this crucial mistake.

* Same game. BB the GM has foolishly let all of his best WR's go via free agency and now is left with a wide-eyed scrub (can't remember his name) dropping crucial passes that hit him in the hands.

* Fast forward to 2009. The Pats have the wrong people on the field on 1st and 10 AFTER a TV timeout. Seriously, how does this happen after a change of possession and tv timeout? Burn another crucial time out to fix.

* Same game. Indecision before the fourth down play leads to another blown time out.

* Same game. He foolishly uses a challenge on a Reggie Wayne catch that is clearly a catch. I can only assume he threw the challenge flag on a wing and prayer or out of total frustration.

I think we can all agree that BB is a great coach. Probably one of the all-time best. But remember there are other great coaches in the business too. So your depiction that he has a limitless advantage over everyone else is vastly overstated.

71
by BigDerf :: Tue, 11/17/2009 - 4:22pm

OOOH OOOH... I know the name of the wide-eyed scrub. Reche Caldwell.... His eyes were always that wide though.

12
by RickD :: Tue, 11/17/2009 - 12:49pm

I still have no idea how anybody can rail on about how the choice was incorrect when, had the refs given Faulk credit for forward progress, the Pats would have made the first down!

Rather than rail on about BB's arrogance, perhaps we should discuss the logic behind not giving Faulk credit for forward progress at all.

20
by turbohappy (not verified) :: Tue, 11/17/2009 - 1:14pm

This comment is patently false. If you look at the tape you see he lands at the 28. If they had not given him forward progress they would have spotted it at the 28, not the 29.5. The announcer was incorrect.

26
by Bobman :: Tue, 11/17/2009 - 1:22pm

By that logic, if he had the ball in his hands but was bobbling it while standing on the 30, then dove backwards (still bobbling it) and landed on his back on the 25 with the ball firmly in his grasp, are you saying the placement should be the 30? I think the NFL rule book would disagree.

Fwd progress starts with possession of the ball. With RBs it's a little more obvious that they have it rammed into their gut--they have the ball, so it's simpler to eyeball where they cease moving forward.

Possession is the first element of a catch, and then position. Think of all the out of bounds "catches" you've seen where the guy does a great job grabbing the ball but it's not an official catch because he's OB. Same thing here--when he came down to the field of play with full possession, it was the 29.5 not the 30.

If the pass had not been rushed it might have been more accurate and catchable--his feet were beyond the 30 before he went up for it. Had he had firm control rather than bobbling it, you'd be correct. But the bobble was the deal killer here.

40
by jackgibbs :: Tue, 11/17/2009 - 1:52pm

the official's judgement was that the bobble ended at the 29.5, so that's the end of the arguement. but to my (fan's) eye the bobble sure looked to have ended on the far side of the 30. the ball is pinned to his chest when bullit lifts him off his feet. it sure would've been nice to have a timeout in that spot, either way

44
by PantsB (not verified) :: Tue, 11/17/2009 - 1:57pm

Faulk stopped bobbling it before being tackled. (screencap screencap) NFL rules say
Rule 2-15-2: "When an airborne player makes a catch, forward progress is the furthest point of advancement after he possesses the ball if contacted by a defender.

(Casebook 2.15.1): It is first and 10 for A at B's 12-yard line. A1 sprints near the end line and then buttonhooks. He jumps and possesses a forward pass while in the air above the end zone. (a) A1's momentum carries him back into the field of play and he lands and is downed at the 1 yard line; or (b) while in the air in the end zone, he is contacted by B1 and he then lands and is downed on B's 2-yard line. Ruling: In (a), it is A's ball first and goal at B's 1-yard line. In (b), it is a touchdown if the covering official judges the contact by B1 is the cause of A1 coming down at the 2-yard line, instead of in the end zone.
and from "Ask Jerry Markbreit" who answers rule interpretation questions (former ref)

    Q

:I have seen plays where the receiver jumps to catch the ball in the playing field and is drilled while he is still in the air, so that his feet come down with possession and control of the ball several yards back. Doesn't the official give him forward progress where the original contact was made while he was in the air? If so, doesn't the rule on forward progress refer to a "runner?" He cannot be a "runner" until his feet come to the ground. Could you explain this? --James Buldas, Chicago

When a receiver of a forward pass is knocked backwards in the field of play and he comes down in possession and in control of the ball, he is given the foremost point of the catch, even though it was in the air. Forward progress does not only refer to a runner. If a runner dives in the air for the goal line and breaks the plane of the goal line with the ball in his possession, it is a touchdown.

Forward progress doesn't require the catch to already be complete. He was contacted over the 30 with the ball in his possession. It was a blown call. BB gets a small part of that blame because he didn't have a TO, but the onus is on the officials to get it right.

52
by Todd S. :: Tue, 11/17/2009 - 2:40pm

I realize this is picking nits, but I think the end of your post is misleading. You say:

Forward progress doesn't require the catch to already be complete.

But above, the quoted rule says:

"When an airborne player makes a catch, forward progress is the furthest point of advancement after he possesses the ball if contacted by a defender. [sic]

I interpret the rule to mean that the receiver has to have completed the catch, i.e. not bobbling the ball ("after he possesses") before forward progress is determined.

This in no way changes the fact that it is quite possible Faulk completed the catch at the 30. I just thought that last bit was off.

57
by TGT2 (not verified) :: Tue, 11/17/2009 - 3:01pm

two different definitions of completing the catch. One of you is using: 2 feet down with control of ball. The other is using 2 feet down.

You both actually agree though that control of ball is all that matters.

The difference then is where he got control of the ball. Was it past the 30, or short of the 30? The official said short, and the replays I've seen are not conclusive to me either way. Maybe I just don't have as much 3-d extrapolation ability as the poster, or maybe he's letting his wishes impact his perceptions.

63
by jackgibbs :: Tue, 11/17/2009 - 3:29pm

I am most definitely extrapolating, since you are correct in saying there is no single conclusive replay. but. looking at the endzone angle you can see the ball take a single short bounce before he pins it and gets hit, and then adding that to the (admittedly off angle) sideline camera, you see bullitt make contact with faulk what appears to be past the 30 yard line. so you're right, it's probably not conclusive enough to be overturned, but that doesn't mean I can't say BAD SPOT, REF!

113
by GlennW :: Tue, 11/17/2009 - 6:22pm

That second screen shot is pretty damned convincing (confirms what I thought anyway, but more convincingly than the moving video). Because the camera is at around the 20-yard-line the angle isn't great and Faulk's upper body position is uncertain-- except for the fact that his extended left foot doesn't lie, and that foot is on the ground but has barely crossed back over the 30-yard-line.

Let's put it this way: that's more convincing video evidence than with last year's forward-progress decision on Santonio Holmes' winning TD catch in Baltimore, which was awarded (rightly I believe) on a replay-booth overturn. So you never know what the replay decision would have been. It would have met my personal standard for "indisputable video evidence" for an overturn though.

184
by Boo-urns (not verified) :: Wed, 11/18/2009 - 12:34pm

Agree with the first poster (screencaps). The key is that the bobble ended BEFORE Faulk gets hit. Faulk was past the 30 before he got hit. Logic dictates that he had possession past the 30.

Furthermore, when I watched the game live, I remember the ref that was closest to the play start to line up past the 30, where the spot should have been, only to be overruled by the sideline ref. Only later did I realize that that judge could not have seen the bobble. But he seemed pretty damned adamant that the ball be placed at the 29.5.

Add to that the phantom PI call earlier in the half, and the fact that this particular crew has the largest bias towards home teams (http://refchat.blogspot.com/2009/11/nfl-referee-statistics-before-week-1...), and I guess you see it is what it is.

197
by Spoon :: Wed, 11/18/2009 - 2:20pm

A screen shot can't prove possession, and certainly not those screenshots. It's a bit of an NFL Rorschach, everyone is going to see exactly what their perspective leads them to see.

13
by jimm (not verified) :: Tue, 11/17/2009 - 12:51pm

Bill Barnwell - it's intelligent articles like this that separate FO from almost all the other sites.

14
by Rob H. (not verified) :: Tue, 11/17/2009 - 12:52pm

The economist in me likes the stats but this call was nothing to do with stats. You can't remove the future implications of the decision and they are huge. This was the wrong call regardless of the outcome. Unless your punter broke his leg earlier in the game you punt the ball 100% of the time in that situation.

25
by Ghost of Chris T Jones (not verified) :: Tue, 11/17/2009 - 1:17pm

I don't understand why, if we accept Barnwell's numbers (which by the way I think are good) we would think this is a close decision. Don't you think Manning has a better than 1-in-3 chance of scoring of TD with 70 yards to go, 2 minutes left, and a TO? (Especially given the fact that we just allocated an 85% chance to Manning scoring a TD with 30 yards to go in the same scenario).

45
by CowWithBeef :: Tue, 11/17/2009 - 2:03pm

Plus that has to be an 85% chance Indy runs the clock down and then scores, not just scores. The chance that Indy scores too fast there is pretty significant. Brady would have loved to have 60 seconds to go 40-50 yards for a FG try.

85
by Bjorn Nittmo (not verified) :: Tue, 11/17/2009 - 5:10pm

This is a very, very important consideration that favors BB's call, and not just a negligible throw-in. In fact, the Pats would have had over a minute left had their understandably conscientious DB not tackled Addai at the 1, or about 45 seconds had they not stopped the run on the ensuing play. If we think the colts had a 33% chance of scoring a TD from their own 30, perhaps the Pats would have had a 15-20% chance of driving to a makeable field goal under these combined (ex ante) scenarios?

126
by Pat (filler) (not verified) :: Tue, 11/17/2009 - 7:55pm

Why? He scored 4/13 times prior to that, with no time pressure. 1-in-3 seems an overestimate if anything.

177
by Spoon :: Wed, 11/18/2009 - 9:32am

4/13 may be misleading. Maybe it would be more relevant to say that on the past three drives - after Manning had a chance to adjust to the defensive game plan and when the Patriots defenders were gassed - the Colts had twice scored from over 70 yards out, and took an average of less than two minutes to do so. They hadn't even had to use a time out. That'd put the odds between 50%-67%. Small sample size fun for everyone!

185
by Capitan (not verified) :: Wed, 11/18/2009 - 12:39pm

Defense tired, game plan clearer. Perhaps you saw how easily Manning moved the ball down the field in his prior possession. Perhaps you saw how easily Manning moved the ball down the field in his final possession. In fact, if you think about it, Manning moved the ball 29 yards in about 30 seconds, with the defenders having the advantage of having the end zone as a backstop. That's about 2/3 the distance of the punt the Pats might have gotten in the same situation. So basically, having the ball at the Pats 30 with 2 minutes to go vs. having the ball at the Pats 40 with 1:30 to go is the difference between 85% and 33%? Nonsense.

Also, let's not forget that the refs were calling some blatantly pro-Colts calls in the 4th quarter. Whether or not you think the refs were exhibiting a pro-Colts bias (this link would suggest they were: http://refchat.blogspot.com/2009/11/nfl-referee-statistics-before-week-1...), that has to be a factor in thinking ex ante about how likely it is that Peyton can move the ball 65 yards down the field or not.

43
by jackgibbs :: Tue, 11/17/2009 - 1:55pm

I keep thinking of the last two meaningful times the defense had to stop one drive in the fourth quarter, 2006 and 2007, and I was really glad to see brady with the ball

53
by Phil O'sopher (not verified) :: Tue, 11/17/2009 - 2:42pm

Economist understand statistics = fail

Sad sad day for our economy, if you are in a position of decision making power or teaching the youth of our nation.

The future implications is absolutely modeled into the formula.

65% complete fourth down (estimated NFL average for 4 n 2)
85% chance indy scores TD from the 29 (Indy great offense)
35% chance indy score TD from own 25 (Indy again great offense)

expected outcome:

70% chance of winning if you go for it
64% chance of winning if you punt

see awesome calculator:

http://belichick-decision.heroku.com/

Don't hate the model, hate the game

15
by kwameF (not verified) :: Tue, 11/17/2009 - 12:56pm

Hmm.. being that the numbers were sooo close I think that it reinforces punting was the slightly "better" decision. By punting you spread your personal risk and put more things in play that could go wrong for Indy. Tipped ball for an int, sack, fumble snap, holding penalty, drops,etc. By going for it he put all his eggs in the "Kevin Faulk running a 2 and a half yard route/defending only 30 yards" basket.

Quick question about 4th down conversion rates. Are garbage time numbers factored? For example I'm assuming that these numbers also include say Tony Romo completing a 4th and 4 when down 17 points late in the fourth quarter right?

29
by Fontes of Wayne :: Tue, 11/17/2009 - 1:31pm

But by punting, he puts all his eggs in the "defend for ~65 yards" basket. All of the things you mention that could go wrong on a 65-yard drive could still go wrong on a 30-yard drive, but if the offense can advance the ball six feet, they don't have to go wrong.
The fact that the numbers were close reinforces your opinion that punting was better, but I'll bet if you originally thought going for it was the better option, you'd see them as backing that up. What they say to me is that neither decision is definitively "better," and while I'm happy to sit here on Tuesday morning and say yeah, they should have gone for it, if I happened to be calling plays for the Patriots that night, I'd probably have sent out the punt team.

33
by Richard Loppnow (not verified) :: Tue, 11/17/2009 - 1:39pm

Exactly what he says here. That 60% estimate sounds WAY!, WAY! high.

54
by Phil O'sopher (not verified) :: Tue, 11/17/2009 - 2:47pm

I believe 4th and 1 NFL completion average is in the 90% range

and the NFL completion average on 4th and 2 would be conservatively in the 65% range, also I would say even 75% of the time would be acceptable number w/ Brady and Pats Offense short pass ability advantage

120
by Pat (filler) (not verified) :: Tue, 11/17/2009 - 6:46pm

This isn't the right way to think about it: that's the aggregate of all 4th and 1 plays, for example. You don't get to run an "average 4th and 2" play. You get to run one play. Belichick's decision to go for it rather than punt is absolutely wedded to his play choice, because what he actually chose is running that play over punting.

And I really doubt that play had a 65% chance of working.

127
by funbob (not verified) :: Tue, 11/17/2009 - 8:08pm

So if Bill's going by numbers that say he has a 90% chance of converting 4th and 1 why on earth does he pass on 3rd and 2? This is the part of the analysis that is so suspect in my view. It isn't fair to argue based on the results but a lot of people seem to think it's just fine to assume Hoodie was making a detached, stat-based decision when the evidence indicates he was acting emotionally. The stat stuff exists in a vacuum; after a 3rd-down pass and 2 blown timeouts on that drive I think we can agree that Belichick's call didn't.

186
by Capitan (not verified) :: Wed, 11/18/2009 - 12:43pm

Game theory.

On 3rd and 2, wouldn't you presume that the Colts would be trying to stop the run? I'd rather get the 1st down on 3rd and 2 than 4th and 1 myself.

That being said, I think the NE offense has an excessive reliance on the pass, but I don't think the logic is that flawed, nor does it show that the "stat stuff exists in a vacuum". Fail.

17
by Sophandros :: Tue, 11/17/2009 - 1:02pm

Belichick was playing to maximize his chances of winning, not to minimize his chances of losing.

To this end, he made the correct decision, but not the correct play call(s). If you start down the path of, "Well, he should have considered what could have happened had he not made it...", then you also need to consider that Indy COULD have returned the punt for a TD or put themselves in great field position, or his punter COULD have shanked the punt or had it blocked, etc.

You have to believe that you can gain two yards at a time when your offense is averaging about 7 yards per play.

As several others have stated, the events leading up to 4th and 2 were the problem, not the 4th and 2 decision. Heck, the execution on 4th and 2 was more of a problem than the decision or the play call.

-------------
Sports talk radio and sports message boards are the killing fields of intellectual discourse.

31
by zlionsfan :: Tue, 11/17/2009 - 1:36pm

my thoughts exactly (maximizing winning).

In fact, I'd say the mistake in the second half was Welker calling the timeout on first and 10. I think timeouts are largely undervalued (not nearly as much as in college basketball, though), and I would guess the Patriots in 1st and 15 near midfield aren't in that bad of a position - of course that assumes that instead of Welker calling the time out, he'd have done nothing and the Pats would have taken a delay-of-game penalty. I guess the only catch is that time outs are valuable primarily when you are losing; coaching aggressively might imply that you should be less concerned with using time outs strategically late in the game and more concerned with using them as the need arises.

I can understand the time out before first and fourth down, if only because at that point the goal was to convert and close out the game, and a 1st and 15 in that situation is probably harder to accept than a 1st and 10. (Certainly a 4th and 7 is much worse than a 4th and 2; the last time out had to be used.)

39
by whatyousay :: Tue, 11/17/2009 - 1:51pm

maximizing your chance of winning vs minimizing your chance of losing is pretty meaningless. It's a zero-sum game. If you minimize your chances of losing, your chances of winning goes is affected inversely (yes I know there are ties). Either variable you choose to look at (% to win or % to lose), it's really the same thing.

now, I'm thinking you probably meant something like Belichick was trying to play the aggressor which is inherently advantageous. I think there was no way to do that in this case. Even though the Pats had the ball, I think at that point in the game, the Colts defense was clearly the aggressive unit. The Pats were just trying to hold on regardless of the personnel on the field.

129
by funbob (not verified) :: Tue, 11/17/2009 - 8:18pm

I'm not buying this explanation. The potential for botched execution and the choice of play are all inherent in the decision to go for it. As someone else mentioned, Belichick didn't decide to call an "average 4th-and-2 play." He decided to call *that* play, in *that* situation, after giving Indy a cue that something was up by mucking around with the timeout, and with the knowledge that he had just removed his own ability to challenge any near misses even though he was calling routes that gave no room for error. When people say it was a bad decision they aren't talking about going for it in some random 4th-and-2 situation that exists in a statistical vacuum, they're specifically referring to 4th-and-2 from the -28 against the Colts in a game with large playoff implications where Peyton Manning has been rather inconsistent.

Maybe I'm just biased against Hoodie (who I will nonetheless say is obviously an exceptional coach) but to argue that you have to split the playcall out from the decision to go for it is ludicrous. Deciding to go for it without doing anything to set up for 4th-and-2 was a bad decision and using that play made it worse. People are tying themselves in knots to avoid that simple acknowledgement.

18
by Tball (not verified) :: Tue, 11/17/2009 - 1:09pm

I liked the aggressive play call, but I didn't care for the sequence. All three second half timeouts seemed to relate to the Patriots not being organized. When Indy calls time out before the third down play, the Patriots have to be deciding whether they are in 3-down territory or 4-down territory. If you are in 4-down territory, that third down play has to be a run. That gets you either another Indy time out or a two-minute warning to plan the fourth down play.

Also, earlier, Brady threw the ball away on third down with just over four minutes left in a 10 point game. The field position wasn't as important as the clock. He needed to take the sack and keep the clock moving. Who knows what changes if that New England possession happened on the other side of the two-minute warning.

21
by JMM* (not verified) :: Tue, 11/17/2009 - 1:14pm

BB's mistake was made earlier, on first down. The Pats had a 17 point lead at one point in the 4th Q and lost it. Bill Cowher, who never gets any mention as a situational genius, won 99% of the games where the Steelers got an 11 point lead. He didn't win those games by throwing the ball 4 times in a row after getting the ball 1st and 10 on his 20 with 2:23 to go. HE RAN THE BALL! He had a Bus and he rode it. Before he had a Bus, he had other backs who could get 3 yards and cloud of dust. If the Pats had run on first down, with whatever personnel grouping on the field (assuming the wrong group was on the field-was it a running play group?), hell a QB sneak would have taken it to the 2 min warning or force Indy to take a time out.

On 3rd down, again, how do you not run it? If you know you are going for it on 4th, get half on 3rd half on 4th. BB said he hadn't thought about it before missing it on 3rd!!! WTF??? Are you kidding me? He's making it up as he goes?

NOw if you want to argue that a healthy Fred Taylor would have allowed the outcome, OK. But why sign an older bruiser back when you know they have trouble playing 16 games. BB needs to straighten out the guy who "buys the groceries."

Until this series, I had a (grudging) respect for BB. Now, not so much. Making a bad series of decisions which put you in a position to take a big chance which can be justified if the proper assumptions are made doesn't do it for me when the alternative path is more certain.

(CAPTCHA for tara fits with the theme of getting ground!)

68
by Temo :: Tue, 11/17/2009 - 3:45pm

Yes well part of the reason that Belichick is a better coach and has won more games than Cowher is because he managed to amass more games with 11+ point leads than Cowher did.

81
by young curmudgeon (not verified) :: Tue, 11/17/2009 - 5:01pm

Temo: Yes well part of the reason that Belichick is a better coach and has won more games than Cowher is because he managed to amass more games with 11+ point leads than Cowher did.

Actual data from Wikipedia: (assuming that Wikipedia has "actual" data!)

Bill Cowher 149-90-1
Bill Belichek 144-89-0

Temo, please explain to me again how 144 is "more than" 149.

84
by jackgibbs :: Tue, 11/17/2009 - 5:08pm

cowher has 7 more games. repost at the end of the season and then we can judge

89
by Alexander :: Tue, 11/17/2009 - 5:28pm

Will BB win 5 more without losing 1 is the only real comparison.

91
by young curmudgeon (not verified) :: Tue, 11/17/2009 - 5:30pm

I'm not attempting to "judge." I don't "judge" quarterbacks by who has the most rings. I don't even "judge" coaches by who has the most wins--for instance (a case that might be pertinent to the current discussion) Bill Belichek amassed very few of his wins and a ton of his losses when he was coaching an inferior team, and has done considerably better coaching a good one. Temo stated "Belichek is a better coach and has won more games than Cowher." The first part of his statement can be argued (and, even though I am a Steelers fan, I concede that it will probably be argued successfully.) The second part of his statement, as of the time he made it, is demonstrably false. The entire force of my comment was that his factual assertion was incorrect. Anticipating future victories by Belichek-coached teams doesn't change that.

95
by young curmudgeon (not verified) :: Tue, 11/17/2009 - 5:46pm

And someday, I'll learn to spell "Belichick."

99
by Temo :: Tue, 11/17/2009 - 5:51pm

True, I have a tendency to not remember Belichick's Cleveland years. It still seems silly to suggest that Belichick doesn't know how to protect leads however. Or that he does it worse than Cowher.

131
by funbob (not verified) :: Tue, 11/17/2009 - 8:36pm

Huh? Belichick didn't go winless in Cleveland. If you ignore the time with the Browns he's even further behind Cowher. You aren't making any sense dude.

176
by Boston Dan :: Wed, 11/18/2009 - 7:42am

I suppose if you want to make the case for Cowher, it's easy to overlook Cowher's playoff record of 12-9 compared to Belichick's of 15-4.

24
by Will Allen (not verified) :: Tue, 11/17/2009 - 1:17pm

As has been noted, the largest error was not considering a four down strategy until fourth down arrived. One of the primary advantages to reducing punting frequency lies in how offensive strategy changes, given a commitment to a four down strategy.

Also, even if the third down pass had been completed, it was a lousy play to run, since their was a decent chance the received would have been driven out of bounds, thus stopping the clock prior to the two minute warning, leaving the Colts with one time out. This means that even a new set of down could easily have resulted in the Colts getting the ball with more than a minute left. If you want to run out the clock, or come close to it, with one first down, then call plays that will likely do so.

28
by Bobman :: Tue, 11/17/2009 - 1:29pm

Interesting perspective--in my mind, until yesterday, I thought BB always had a 4-down strategy in mind. With a 12 our of 16 success rate vs Indy on 4th down attempts since 2001, I just assumed he always considered it.

I guess he didn't, by his own admission. That really surprised me, as I thought he was a guy who plotted out every plausible scenario ahead of time.

12 of 16 includes a 4th and 15 last year with Matt Cassell getting picked off. If you include just Brady's makeable 4th down attempts, that's 11/15 or 80%. I like those odds.

36
by Will Allen (not verified) :: Tue, 11/17/2009 - 1:42pm

I haven't checked the Colts front 7 alignment on third down, but I suspect even a qb sneak on that snap would have been a far superior play to call, compared to a pass to the sideline. Hell, if the fourth down play had been called on third, the Pats likely win the game, even with that spot. I just can't figure out why they would call a sideline pass in that situation, other than they obviously liked (incorrectly, it turned out) the match up out on the edge.

30
by zlionsfan :: Tue, 11/17/2009 - 1:31pm

I was under the impression that they had considered it prior to fourth down? Although the quote below could be taken either way, I guess.

From ESPN.com:

"We had a little miscommunication on that. Once some of the guys on the punt team started out, then the guys on offense they started to come off," Belichick said, noting the decision had been made on third down that they wouldn't punt. "That wasn't cleanly handled. I'll take responsibility for that."

75
by Martial (not verified) :: Tue, 11/17/2009 - 4:34pm

Just to restate a bit: we need to judge on process, not on outcome. Some commenters have suggested that process is beyond what we know and, therefore, we cannot judge it. Not true. Process does not just mean 4th down. Process means the whole sequence of downs. In other words, the 3rd down call is important to our understanding of whether or not going for it on 4th is “correct”.

Many of us here agree that the 3rd down call was problematic. Some even question the 1st down call (see #21).

We would have preferred a run on 3rd down. That has the major advantage of keeping the clock moving (either forcing the Colts to call timeout or running the clock down to the two-minute warning). It has other possible advantages including Brady (and his OL!) not being subjected to Indy’s pass rush. As the Patriots appeared to be running well against Indy, that might also be something to take into account.

We also think the call on 3rd down was not a good one (see #24), though we may be on shakier ground.

The process suggested by the 3rd down call is one that many of us think was faulty. It follows therefore that the 4th down call was the result of a bad process, not a good one.

187
by Capitan (not verified) :: Wed, 11/18/2009 - 12:49pm

One screenshot I thought was VERY interesting during the game. Following a Colts TD, Belichick was seen talking with his defensive unit, even while the Pats offense was on the field. It made me think that he doesn't actually call most of his offensive plays, but rather delegates that process to someone he trusts.

In the instance of the controversial 4th down play, Brady has been quoted as saying that he had pushed for that particular play, as it was something he and Faulk had practiced a lot.

All of which makes me think that the particular play calls were not initiated by BB. Obviously, as the head coach, he has to take responsibility for the playcalling as well as the processes that led to the playcalling. But it did seem striking to me (as opposed to someone like Norv or Jim Zorn, who would call their own plays).

188
by jackgibbs :: Wed, 11/18/2009 - 1:04pm

he doesn't call plays. that's been true since the days of charlie weis. in the 2007 super bowl I don't think he even watched an offensive snap, he spent so much time exhorting his defense to do their jobs.

he was responsible for the decision to go for it, and also responible for not making that decision until the last minute, and burning his last time out to do it

34
by Never Surrender (not verified) :: Tue, 11/17/2009 - 1:39pm

I agree that there is an intolerable fickleness to the post-game analysis. Every talking head is using this decision as a posturing tool, and yes, it's sickening.

However, there remains a bit of a straw man at the center of Mr. Barnwell's argument. Most of the people whose analysis I respect AND who criticize BB for the call -aren't- bashing him based on the outcome of the play. And nor do they rely on using P-F-R's little percentage tool . . . because stats aren't the only legitimate tool in the box for analyzing what happened.

There's a wider angle to all of this: probabilistic arguments ("in my view they had a 65% chance of winning") aren't the only rational ones. Stat-head claims notwithstanding, things like momentum, player psychology, even crowd involvement, and other unmeasurable factors need to play a part in making — and assessing — the call. It takes, in addition to guestimates and percentages, judgment of personnel, character, and the individual personalities that constitute one's team, etc.

Narrowing in on what would happen if we ran the simulation 10,000 times, to the exclusion of other factors, can be misleading.

56
by Phil O'sopher (not verified) :: Tue, 11/17/2009 - 2:57pm

great post!!!!

A few thoughts to consider:

Models are built like road maps. You can add details, but it will never be an actual road. K.I.S.S is one of the most important things to remember when modeling. Keep it simple, stupid!

reason: once you factor in too much qualitative data, you need to quantify it to get numerical results. This quanitification of non-numerical data is ripe w/ assumptions and biases and changing variables. You can "manipulate" these numbers for any outcome you want. In fact it is easy to do, once the model is set up. Many times it is done unintentionally.

I like the simplitic approach...or road map making approach, you can see the direction and get from point A to point B, but you wouldn't want to buy your dream home from looking at a road map (limitations to the model)

37
by Richard Loppnow (not verified) :: Tue, 11/17/2009 - 1:47pm

Any time the defense sells out to stop a play, they've got alot better chance than 40% of stopping it. Probably most '4th and twos' are situations where the offense has to have it, while the defense still has a lot more scoreboard/turf to play with.

I really, really want to see in detail the data cited on behalf of 60%.

46
by Sophandros :: Tue, 11/17/2009 - 2:03pm

Here's your explanation: http://www.advancednflstats.com/2009/09/4th-down-study-part-3.html

-------------
Sports talk radio and sports message boards are the killing fields of intellectual discourse.

51
by Brian Nelson (not verified) :: Tue, 11/17/2009 - 2:30pm

There are no details there, aside from mentioning that the conversion rate depends on field position. In any case, it would make more sense to use the "inside 10" rate (~55%) since the field for that play was effectively 2 yards long.

I'm also curious what effect excessive crowd noise would have, since it makes calling an audible almost impossible. That would certainly further reduce the conversion rate.

The total disrespect for the Pats' running game--a point conceded when the Pats lined up 5 wide--allowing the Colts to sell out on the short pass didn't help either. Bad spot or not, the play was very well defended by the Colts.

61
by WY (not verified) :: Tue, 11/17/2009 - 3:20pm

I don't understand why "inside 10" would be superior, the point of which being that out-of-bounds line at the back of the end zone serves as an extra defender. The 11 players on defense only need to protect a maximum 20 yards deep box for "inside 10", while they need to defend a max 80 yard box for this situation.

79
by Richard Loppnow (not verified) :: Tue, 11/17/2009 - 4:56pm

The absolute best comparison would be how often teams make it on 4th and goal from the 2.

49
by langsty :: Tue, 11/17/2009 - 2:27pm

this was terrible when you clumsily shoehorned it into QR yesterday, and it's terrible today too. kill yourself, bill barnwell.

62
by Still Alive (not verified) :: Tue, 11/17/2009 - 3:23pm

Can FO please start deleting posts like this, they are a waste of everybody's time.

58
by erniecohen :: Tue, 11/17/2009 - 3:11pm

Once again, an analysis that misses the key point.

Besides the cited reasons that 4th and 2 data doesn't apply, the chance of success takes a huge dependency on the payoff matrix, because there are usually more possible outcomes than simply success and failure. Most of the time when a team goes for 4th and 2, it is either because they are outside desirable field goal range or because they are desparately trying to keep a drive alive at the end of a game. In either of these cases, there is a big difference between making the first down and breaking off a big play. This means that the D usually can't sellout to the same extent to stop the first down as they could in this circumstance, which means that the NE chance of making the first down was considerably less.

64
by Still Alive (not verified) :: Tue, 11/17/2009 - 3:30pm

I don't think they could sell out and ignore the big play here, I certainly wouldn't have as D coordinator.

I haven't seen one good reason posted so far as to why NE would have a lower than average chance of making it?

IND has a good DEF, NE has an even better OFF...
IND was defending NE well. really? 34 pts in a game is doing well? NE was having a good day on OFF.

And so on and so forth.

I also wouldn't be second guessing Belicheck's play calls. I think there is a lot more subtly in how much people are lining up against the pass vs the run than people realize. I am sure BB knows that a run is more likely than a pass to get at least 1 yrd on 3rd down. I am also sure he had his reasons for choosing a pass.

200
by Spoon :: Wed, 11/18/2009 - 2:32pm

I don't think it's correct to assume that a team plays consistantly the same throughout an entire game. New England put up 34 points, yes, but 24 of those points came during about a quarter's worth of play; from between the first score in the middle of the first quarter through to the Edelman touchdown in the middle of the second. Over the next 37 minutes of play, the Patriots scored only ten points, (and on driving of 38 yards combined to get those points, thanks to excellent starting field position). The Patriots did move the ball on other drives, notching 12 first downs, but the defense had also forced turnovers on both of the Patriots longest second-half drives.

66
by jackgibbs :: Tue, 11/17/2009 - 3:37pm

even after dropping the conversion rate under 50% in the calculator, the numbers still come out extremely close, within 2% or so, depending on your other numbers

59
by The Other Ben Johnson (not verified) :: Tue, 11/17/2009 - 3:11pm

Nice try, but I'm still gonna overdiscuss this in quick reads.

65
by Pat (filler) (not verified) :: Tue, 11/17/2009 - 3:33pm

The advancednflstats.com argument is just wrong.

It is. There's no other way to look at it. The simplest argument is that the "judging coaches' decisions" websites (advancednflstats, footballcommentary) are making naive judgments much like the Monty Hall problem, assuming that the "naive estimator" for win probability should factor into decisions *ignoring the fact that one path allows you to make more decisions than another one.*

In that situation, you punt. Period. Why? Because going for it allows you to make *one* decision - you get the first down, you win. You miss the first down, you lose. Punting allows you to make *many* decisions and to *adapt* based on the result.

The only time when going for it makes sense is if you believe you're a worse decision maker than the other coach. Otherwise, you punt. (Put another way, if Belichick's so damn smart, why didn't he have his team punt so he could have several chances to prove his defensive brilliance?)

67
by jackgibbs :: Tue, 11/17/2009 - 3:40pm

a failed conversion =/= 100% lose rate

96
by Pat (filler) (not verified) :: Tue, 11/17/2009 - 5:48pm

The Colts scored every single time they were inside the Patriots' 30. Belichick's best estimate would have been that a failed 4th down conversion is a 100% lose rate. Maybe if he was really smart, he could say that it's "most likely 85% or higher." (85% is the roughly the 4th root of 50% - it's the point at which a 4/4 outcome is more likely than any less than 4).

100
by jackgibbs :: Tue, 11/17/2009 - 5:52pm

you're not accounting for the times indy scores and leaves enough time for the pats to get a field goal. so, even assuming a 100% td rate does not end the game on that call

104
by Pat (filler) (not verified) :: Tue, 11/17/2009 - 5:58pm

Yes, but if Belichick was thinking about that as seriously improving the likelihood of winning if he goes for it, WTF did he call his last timeout before the play for?

72
by Phil O'sopher (not verified) :: Tue, 11/17/2009 - 4:22pm

Wrong, Wrong, Wrong

How is this so hard to understand? I mean know wonder my quant finance program was 99% Chinese. They would get this, model it in Excel, run the monte carlo simulation 5 different ways within 10 minutes and report to you on the outcomes

and that is with no understanding of American Football or who this Ball Billichuck is.

Seriously, WTF, all this discussion and you missed the whole point.

it is not a zero-sum decision.

Wow, saddened by the American public. Unless of course you are from elsewhere, then I will be a bit happier w/ us Yankees not being so stupid

82
by Pat (filler) (not verified) :: Tue, 11/17/2009 - 5:01pm

You can't Monte Carlo this without modeling Belichick and Caldwell, and good luck doing that. Just because an "average" team does something doesn't mean that either one of them would.

86
by jackgibbs :: Tue, 11/17/2009 - 5:10pm

how does ZEUS do it?

92
by Pat (filler) (not verified) :: Tue, 11/17/2009 - 5:34pm

By replacing Belichick and Caldwell (and Manning) with NFL historical averages.

This is why I hate ZEUS and arguments like this. Football's a decision tree, and the historical averages do not equally populate that decision tree. If you applied ZEUS to chess, for instance, the behavior of tons of Grand Masters would frequently look moronic - the Hedgehog Defence, for instance, would've looked incredibly stupid prior to the 1970s.

(Note... to be fair, it's entirely possible that Belichick made this decision because Manning had been countering everything Belichick threw at him all day, and so he figured his chances were better with his offense. I find this logic dubious given the fact that Manning scored on only 4/14 previous drives. It's also possible that Belichick thought that his 4th and 2 play was very unlikely to be stopped by the Colts, but I also find this logic dubious considering his 3rd and 2 play was stopped.)

73
by bravehoptoad :: Tue, 11/17/2009 - 4:22pm

Put another way, if Belichick's so damn smart, why didn't he have his team punt so he could have several chances to prove his defensive brilliance?

Somehow, I doubt that proving his defensive brilliance was at the top of his mind.

Who takes seriously the dismissal of a complex argument with the sentence "Period."? Who says, "Oh, now I'm conviced. You just said 'Period.' Now I see the folly of my ways."

In this case it was particularly funny, because after you say "Period." you go on to elaborate at great length.

83
by Pat (filler) (not verified) :: Tue, 11/17/2009 - 5:04pm

No, the "period" was meant to indicate that it has little to do with average outcomes of the game at that point, and if you overthink which decision "on average" wins out, you'll miss the point. It has everything to do with which option gives you more future decisions. Punting does, so you take it.

189
by bravehoptoad :: Wed, 11/18/2009 - 1:17pm

No, that's not what "period" means in English.

I think you should look it up. Here's the definition I think makes the most sense for your context: the end or completion of something. In any case, there's no denotation or connotation of "period" that includes average outcomes.

207
by Pat (filler) (not verified) :: Wed, 11/18/2009 - 3:01pm

Right. As in, there's no situation where it's the right decision to go for it. The rest of that comment was to clarify why there was no situation.

74
by DGL :: Tue, 11/17/2009 - 4:25pm

I would have said the exact opposite. Going for it gives you more opportunities to win. You can win by getting the first down. If you fail to get the first down, you can win by preventing the Colts from moving 30 yards and scoring. Punting, however, removes the opportunity of winning by getting the first down -- the only opportunity you have to "make decisions" to win is on defense. And with the way the Colts were playing on offense, you were likely to have about as many "decision points" defending a 30-yard field as you would have defending a 70-yard field.

88
by Pat (filler) (not verified) :: Tue, 11/17/2009 - 5:22pm

I would have said the exact opposite. Going for it gives you more opportunities to win.

No, you're thinking about "paths to win," not "remaining decisions." Just think about the "expected number of plays remaining in the game."

If you go for it, you've got the 1 4th down play (which is much more important than the others) and then if you fail, the ~3-4 plays they will take to score. If you punt, you've got at least twice that number.

Plus, if you punt, you've put the Colts into a situation where their possible decisions are reduced - they've got 1 timeout, and need to go 80 yards. That reduces the number of possible plays they can call, which increases the chance that you'll be able to make a decision on defense that's better than their decision on offense.

The only way the decision to go for it makes sense is if Belichick thought that Peyton Manning was smarter than him (which, granted, might not be insane...).

And with the way the Colts were playing on offense, you were likely to have about as many "decision points" defending a 30-yard field as you would have defending a 70-yard field.

The Colts scored every time they were within 30 yards of the end zone. They scored only 4/11 times when they had ~80 yards to go. Based on every drive in that game, I'd say that you're wrong.

94
by jackgibbs :: Tue, 11/17/2009 - 5:43pm

the colts two touchdown drives prior were 6 plays 79 yards and 5 plays 79 yards. so they gain maybe two more opportunities? and in the second half (the more situationally significant) drives, the colts were 3 for 5, so...

102
by Pat (filler) (not verified) :: Tue, 11/17/2009 - 5:56pm

It's not "entirely" about number of decisions - you want the situations to be neutral (equal gain/loss) or biased in your favor. If you gain a bunch of crappy decisions, it doesn't help you. Going for it on 4th down gives you one favorable decision situation, and if you fail, maybe 3-4 crappy decision situations. Punting gives you a lot of favorable decision situations.

and in the second half (the more situationally significant) drives, the colts were 3 for 5, so...

With only 1 of those 5 short enough to conceivably happen within 2 minutes. Granted, it was the most recent one - it is possible that Belichick thought he was out of tricks, and Manning was going to score no matter what. I just don't find that entirely likely, but it is possible.

119
by Will Allen (not verified) :: Tue, 11/17/2009 - 6:44pm

I've heard reports that Belichick consulted with his defensive coaches prior to making his decision (and like I said I think he made his decision too late), and if this is true, I think it likely that he was told that his pass rushers were gassed. His depth in this area was depleted, what with one of his best pass rushers having been hurt earlier in the game. His defensive backs are not outstanding. I think he simply decided that 40 extra yards of field position were not nearly as valuable as they would normally be, given the physical state of his pass defense.

I wish there was an in depth study on the effectiveness of pass defense, when facing a good offense, factoring the number of previous pass rushes. It would be hard to structure, I'm sure, but it might give some insight regarding the chances of success in such situations.

123
by Pat (filler) (not verified) :: Tue, 11/17/2009 - 6:57pm

If that's true, then he should've simply let Indy score as soon as possible. If his decision is predicated on the fact that Indy has a strong chance of scoring from their own 20, they've got a crazy chance of scoring from their opponent's 30.

(and like I said I think he made his decision too late),

I couldn't agree more. The problem is that people are yanking that decision out and trying to judge it by itself, which is crazy. You can't, because football requires decisions before and after a play. It'd be like looking at a gambit play in chess and saying "oh, that's terrible, because losing a piece is bad" - in general, the comment is true, but if the player sets it up correctly and takes advantage, it can be a good decision (or a neutral one).

A 4-down strategy might've been a good idea in that situation, but it doesn't matter because Belichick didn't commit to the idea, either before (the 3rd down play + timeout) or after (attempting to stop the Colts after the decision was predicated upon the inability to do so).

Failing to follow a consistent strategy is a great way to allow an opponent to make better decisions than you, and this just showed it.

124
by Will Allen (not verified) :: Tue, 11/17/2009 - 7:09pm

Oh, I pretty much agree. The time to decide that your pass rushers are dangerously gassed is before the Colts kick the ball to you, not after you have run three plays. If this attitude is adopted prior to your offense taking the field, then everything changes. I think Belichick was like a chess player, who, under great emotional pressure, failed to plan enough future moves. Nobody is immune to that sort of thing.

128
by DoubleB :: Tue, 11/17/2009 - 8:09pm

And in between those two TD drives was a 1 play, 1 INT drive as well. The belief that he was clearly going to score given the situation is dubious.

132
by jackgibbs :: Tue, 11/17/2009 - 9:05pm

sigh. nobody's saying it was a forgone conclusion they would score. it only needs to be ~34% or better

135
by Pat (filler) (not verified) :: Tue, 11/17/2009 - 9:24pm

No, what's important is the difference between their likelihood of scoring from their 20 and their likelihood of scoring at the Pats' 30. But this is still a terrible, terrible way of thinking about a strategic game.

138
by jackgibbs :: Tue, 11/17/2009 - 9:33pm

well, if your position is that the colts are 100% to score from the 30, while at the same time saying it's less than 34% from their own 30, I think I'd have to respectfully disagree.

eta. you're still assuming a 100% failure rate on the fourth down play, as well

141
by Pat (filler) (not verified) :: Tue, 11/17/2009 - 9:45pm

well, if your position is that the colts are 100% to score from the 30, while at the same time saying it's less than 34% from their own 30, I think I'd have to respectfully disagree.

Really? Because, uh, that's pretty much what actually happened in the game.

I mean, argue against the selection all you want, but the Colts did score 100% of the time inside the 30, and less than 34% of the time from their own 24 (which is the rough location of where the punt would've ended based on Hanson's actual punting for the day).

eta. you're still assuming a 100% failure rate on the fourth down play, as well

No, the success rate on the fourth down play is actually immaterial to the decision making process. You don't choose between going for it and punting. You choose between punting and another play (But still: I'm not assuming a 100% failure rate, as that would imply you better freaking punt the ball).

153
by DGL :: Tue, 11/17/2009 - 10:41pm

Welcome to small sample size theater.

Statistics are not probabilities.

178
by dryheat :: Wed, 11/18/2009 - 10:13am

Pat, doesn't your entire argument start with the premise of the Patriots' offensive failure?

I don't think, as a Head Coach, you should be putting an inordinate amount of thought thinking of "What's our best opportunity to win after we fail to convert?"

206
by Pat (filler) (not verified) :: Wed, 11/18/2009 - 2:58pm

If you put no thought into it, you reduce your chances to win if you fail to zero. Which makes punting the right choice.

155
by Martial (not verified) :: Tue, 11/17/2009 - 10:45pm

Offense and defense are not the same concerning "decisions" or choice. Choice resides with the offense in football. Offense gets to determine defensive personnel, fer gawd's sake (this one facet is exploited a lot more than we really pay attention to). The best a defense can do is to limit offensive playcalling options by being great at something (yes, sometimes this "best" is phenomenal).

Defense is not about choice, but reaction (based, admittedly, on a lot of study regarding tendencies - but still basically reacting to what the offense does). Defensive players have to keep several things in mind and be ready to react (ladies and gentlemen and Patriots fans, I give you Andy Katzenmoyer!). Pass or run? Screen or draw or downfield throw? If the offense can't stop a pass rusher what does their protection look like and what plays can they call out of that formation? Yes, sometimes down and distance - and time left - dictate the choices the offense is likely to make, but the Patriots have a very, very good offense. Their choices on any one play are not so limited. But their choices on defense are, in fact, constrained.

A team does not maximize their choices/decisions by punting. They give over decision making power to the other team. A team maximizes their control by MAKING A CHOICE on offense.

I happen to think the whole set of downs process was flawed (I don't think the Pats were in four down mode), so going for it on 4th was not the optimal decision - in my opinion. But I also think that the flaw being criticized in #65 is not a real one. (Not that anyone's mind is being changed by all this foofaraw! Great conversation - mostly. I feel like I'm in a really intelligent bar after three drinks.)

210
by Pat (filler) (not verified) :: Wed, 11/18/2009 - 3:11pm

The best a defense can do is to limit offensive playcalling options by being great at something

Robber coverage is specifically designed to anticipate a quarterback going to his hot read. That alone is better than your "best." But it's a far cry from what teams like the Steelers and the Eagles do in disguising coverage and specifically weakening themselves in an area anticipating that the offense will not take advantage.

Defense is not about choice, but reaction

Sorry, I don't agree. Playcalling in college tends to be more about reaction, but in the NFL you see so many defensive alignments and defensive plays that a lot of it is about choice. You'll see college teams play Cover-2/Cover-3 an entire game. You'll almost never see that in the NFL.

In the situation in question, the Patriots definitely had a choice: play a soft cover zone, force Manning to work the clock and hope you can force an incompletion or two, or rush him hard and hope to force a play on defense. And that's just vanilla thought - it's perfectly possible that, with 80 yards to go, Belichick could offer an underneath zone a few times, then show it and take it away suddenly. The Colts had one timeout, so really any big play seriously reduces their chance at winning.

(It should be noted that other people have suggested that Belichick went for it because he believed his defense was basically gassed and would never be able to hold Manning. If so, that just makes the entire drive a strategic disaster, and the 4th down decision doesn't matter as it becomes a desperation ploy rather than a thought-out strategy.)

158
by Still Alive (not verified) :: Tue, 11/17/2009 - 11:04pm

Pat you say a lot of intelligent and interesting things, this was not one of them.

69
by Scott Kacsmar (not verified) :: Tue, 11/17/2009 - 4:17pm

First let me say I think they should have punted. I don't think it's the worst call ever, but I flat out do not agree with it.

"The important factor that the cacophony of responses seems to be missing is that you can't judge Belichick's decision by the fact that it didn't work. As we've mentioned more than once in these pages, you cannot judge decisions by their outcome."

Maybe not by the outcome, but you can judge it by what Belichick was risking by going for it there just to gain a few percentage points in win probability. Is that marginal gain worth giving Manning the ball with less than 30 yards to go? That gives the psychological edge to Indy's offense, now knowing that their job is much easier than what it would have been had the Pats punted. And a loss means almost no chance at HFA over Indy, a defense left wondering why they weren't allowed to win the game, and ex-Patriots ripping Belichick in the media. I wouldn't want to deal with all of that for such a marginal gain by going for it.

I don't see anyone taking notice that they had a better chance of winning the game than losing it just by punting (>50%). No matter how great Manning is, there's a greater chance for mistakes when you have a field that's twice as long. One sack or penalty can kill a drive.

77
by jackgibbs :: Tue, 11/17/2009 - 4:41pm

"I don't see anyone taking notice that they had a better chance of winning the game than losing it just by punting (>50%)."

if we're only using generalizations, then I would say that I would rather have the 99% chance at victory once we convert than the 50% chance after a punt. these numbers are completely wrong, by the way, and there are a number of posts already explaining why, but even using your logic the call to go for it is still defensible.

80
by Fontes of Wayne :: Tue, 11/17/2009 - 4:56pm

What he was risking was losing the game if he didn't convert. He also risks losing the game if he punts. The argument that going is it was incorrect because you're more likely to lose if you fail to convert is flawed, because it presupposes that you'll fail to convert.
A loss does mean HFA is almost impossible, but it means that no matter how you lose, so that's irrelevant. The other two statements boil down to punting to avoid criticism. If that's your argument, well, it's not necessarily wrong - as you point out, a team that punts is still more likely to win than lose - but I hope my team has a coach who makes decisions without worrying what they'll say in the papers the next day.*

*I should note that, because "my" team is the Lions, I will likely never know if they have such a coach.

90
by zippyx (not verified) :: Tue, 11/17/2009 - 5:29pm

Belichick was risking by going for it there just to gain a few percentage points in win probability.

A *few* percentage points? Dude, if they converted it in bounds, they won. That's a lot of percentage points.

70
by Paulo Sanchotene, RS, Brazil (not verified) :: Tue, 11/17/2009 - 4:21pm

How many Colts fans at Lucas Oil thought, after celebrating the unsuccesful 3rd-down, that Belichik was crazy to give IND the ball at NE-28?!

How many Colts fans at Lucas Oil were VERY worried about the fact that NE were going for it on 4th-down instead of punting?

It seems that every Colts fan -or, at least, a lot of them- thought they had a great chance of winning the game IF the Patriots punt the ball, and that they were in serious trouble with the 4th-down attempt.

Well, also did Belichick...

As far I'm concern, the decision wasn't crazy or wrong. And if there is something Bilichick learned about this is NOT that they should have punted, but that thay should have thought about going for it BEFORE the THIRD down.

He will do it again.

97
by Monkey Business (not verified) :: Tue, 11/17/2009 - 5:51pm

After the Pats didn't make the 3rd down, every Colts fan worth their salt KNEW Belichick was going for it. I mean, we've watched this guy every year, and sometimes twice a year, for nearly a decade. We KNEW that he was going for it, because that's what he does.

As for how nervous we were, I bit off three fingernails between the end of the 3rd down play and the end of the 4th down play.

As for after the play, we really weren't thinking "We got this." It was more a "Let's see if Manning can earn that ridiculous contract he's about to get."

It wasn't over until Reggie scored. And even then, we were all sitting there going "Oh hell, it's going to be a Hail Mary with Randy Moss streaking toward the end zone, and if the Pats protection holds up for five, six seconds, he's going to drop a bomb right into his hands and we're screwed."

Basically, it wasn't until the clock hit 00:00 that we finally went "Holy crap we won".

98
by Johnny Socko (not verified) :: Tue, 11/17/2009 - 5:51pm

For what its worth, the Colts radio play-by-play announcer and the colorman (former all-pro lineman Will Wolford) were downright GIDDY when the Pats lined up to run the fourth down play. Of course, they were even happier after the play failed, but prior to the play they were figuratively "licking their chops" for the chance to stuff the play in BB's face.

106
by Paulo Sanchotene, RS, Brazil (not verified) :: Tue, 11/17/2009 - 6:06pm

Ok. Thank you for the answers. My impression on TV was slightly different, but I was emotionally involved besides being a Giants fan. What a GAME it was!

112
by Johnny Socko (not verified) :: Tue, 11/17/2009 - 6:19pm

Your comment brings up an interesting point regarding the "emotional" aspects of the decision. I've heard several of the Colts defensive players say things like "that was total disrespect towards us" and "we needed to prove that we could stop them". Granted, this could be just idle chatter. However, BB's decision to go for it was so unusual I think it had an unusally high emotional impact on the Colts players. Of course, we will never know how (or if) these emotions impacted the play, but it is interesting to consider.

76
by BigDerf :: Tue, 11/17/2009 - 4:40pm

A few quick arguments to a lot of what I've seen people say in favor of going for it. (Note-I'm a Giants fan so a mostly impartial observer)

1. @barnwell - There was no point where I thought it was a good idea to go for it. As soon as Belichick didn't put out the punt unit on 4th down I was screaming at the TV that it was the wrong decision. I have witnesses. I'm not judging based on the outcome. At no point was that a good decision.

2. @Statguys - You say that going for it favors winning based on percentages and probability? Thats cool... I'm in... BUTTTTTT if that's the case and Belichick was playing percentages then he should have been playing them on 3rd down as well. If he wanted to get the first there and was going to go on 4th down then it should have been two down territory and he should have run on first down to milk the clock and work on getting closer/getting the first. Belichick admits that he didn't decide that he was going for it on fourth down until after third down. That is what I'm arguing against.

Also.. In what world does that percentage of drives end in a touchdown? You can't claim all of these "hard" stats and then claim Peyton scores TDs on more than 50 percent of his drives. Thats just plain wrong.

3. @Others - Oh... so the New England offense is a dominating offense? That why they didn't score more in the fourth I guess. Their offense had been just as bad as their defense in the fourth if not worse. I mean... They just got stopped to what would be a 3 and out. Is that not a sign already?

And another example of Belichick being outcoached in the game? The Indy CBs knew that they were going to run that play on fourth down. Appparently the coaches told em all week in meetings that if it came down to a fourth down like that the patriots would call that play.

78
by jackgibbs :: Tue, 11/17/2009 - 4:49pm

nobody's saying he scores on more than 50% of his drives after a punt; they're saying that if he scores on more than 34%, that's one time in three, then going for it provided a better chance to win.

as to the third down/being outcoached...I don't think you'll get too many arguments, even from bill

93
by Pat (filler) (not verified) :: Tue, 11/17/2009 - 5:42pm

they're saying that if he scores on more than 34%, that's one time in three, then going for it provided a better chance to win.

Colts vs NE, prior to that drive: scored on 4/13 drives. That's less than 1 in 3. Therefore, the best evidence that Belichick had at the time should've told him to punt. And that completely ignores the time situation, which only tilts it more in favor of punting (a terrible guess would say that the Colts only had 1 scoring drive of less than 2 minutes - 1/13 makes it a terrible decision - note that this is an awful, awful guess).

But making decisions based on expected winning percentage alone is a very poor way to play a strategic game.

103
by Alexander :: Tue, 11/17/2009 - 5:56pm

Your assessment is the same one Peter King makes...if that is the company you want to be in then go ahead. Minimizing losing is the same as maximizing winning in football, unless you are playing for a tie, which was a highly unlikely <1% chance under the scenario that game was in. It's super easy to do a more complex evaluation of the decision, without using numbers at all.

You have 9 viable variables IMO and they are

Conversion chance
Chance Colts Score if not converted
Chance ball turned over on downs after successful conversion
Chance Colts TD after successful conversion
Chance of Colts TD on punt play
Chance of Colts TD after successful punt
Chance of Pats scoring after colts TD on punt play
Chance of pats scoring after colts TD after failed conversion
Chance of pats scoring after colts td after successful punt

108
by Pat (filler) (not verified) :: Tue, 11/17/2009 - 6:14pm

It's not about minimizing losing. It's about maximizing the number of opportunities for your players to win the game/you to have a situation to choose the right decision.

You have 9 viable variables IMO and they are

... completely unknowable because the 2009 Patriots and 2009 Colts have never played each other in the regular season.

Analyzing football like that is incredibly silly, because you can't possibly know any of those variables, to any precision. You can't base your entire decision on what the outcome of other games has been. You've got to base them on what you think you can actually do.

And if Belichick thought it was worth essentially gambling the entire game on a 2-yard passing route... yeah, that's not a smart decision. And the playcall does matter, because the decision to go for it is based on how favorable that 4th and 2 situation is for you. And if that's the best call he could come up with... it wasn't a favorable situation.

87
by GlennW :: Tue, 11/17/2009 - 5:12pm

If the odds of picking up these two crucial yards against a defense in full sudden-death sellout mode were 60% or more (as I keep seeing), well, then the Colts beat fairly long odds by making the stop on both 3rd and 4th downs (at about a 15% chance to stop both plays given the 60% success assumption). That's the part of the analysis I most question; that going for it in that situation was even a straight-up 50% winning proposition. Maybe we do need to take a look at the actual on-the-field results-- at least partially-- in making an assessment. The 3rd-down play had no chance as executed and the 4th-down play was well-defended with the ultimate result coming down to a virtual coinflip (even insofar as to how a replay decision would have been ruled). So I don't think it's outrageous to believe that the Patriots' odds of converting on 4th down were even as low as 40%.

In summary, in football as opposed to dice rolls, I think it's okay to at least consider the result of a play in the final opinion. No situation or scenario is ever the same, and the actuarial data aren't perfect, as the authors have acknowledged. So as such on this particular decision I agree: too close to call. Anyone claiming to have "proved" that this was a good decision or bad decision is implicitly relying on imperfect data and assumptions.

101
by The Original Omar :: Tue, 11/17/2009 - 5:53pm

I hope I never sit next to any of you stat geeks at a BlackJack table...

105
by morganja :: Tue, 11/17/2009 - 6:04pm

Here's the issue. One can logically hold only one of two positions. Either the 60% conversion rate on every potential 4th and 2 is correct in which case each and every coach in professional, college and high school football is wrong. Or, the 60% conversion rate on every potential 4th and 2 is wildly overstated.

The subset of 4th downs that coaches go for is apparently around 50% this year. Presumably, some of these are in desperation in which case the defense plays it safe conceding an increased chance for a conversion as opposed to a long, quick TD, or they are done because the offense thinks they have a matchup that they can take advantage of, once again increasing the chances of conversion.

Whatever one thinks that the chances are, and I've seen amazing conclusions drawn from data that can't support the results on this issue, it is apparent that coaches treat the conversion chances in that situation as being much lower than 60%.

As an interesting and much more useful exercise, do these calculations backwards. Calculate the number of times coaches punt in that situation and how many times they go for it and from this data determine what coaches perceive the precentage to be.

Without going through the numbers, I'm going to guess that professionals act as if they calculate the chance to be significantly less than 25%.

Whatever the number is, compare that to the 60% being thrown around here and then ask why is it that professionals who do this every day act, put their careers on the line, using an estimate much lower than the data from 4th down conversions which are actually attempted.

A decent statistician would instantly recognize that the data set of all 4th down conversions attempted and the data set of all possible 4th downs are two different data sets and there is no possible mechanism besides conjecture for making them equivelent.

Since the data is not conclusive one might think there would be a little more humility on the part of statisticians who lack the data to support their conclusions.

Luckily, we live in a free market system and any of these statistician hobbyists are more than welcome and able to get themselves at least high school football coaching positions and show the world that they are correct because the data certainly doesn't conclusively prove that they are.

107
by Johnny Socko (not verified) :: Tue, 11/17/2009 - 6:13pm

My favorite post thus far. Well said.

108
by Alexander :: Tue, 11/17/2009 - 6:14pm

A guy won state somewhere without ever punting. Its been done in high school, and people decided his results didn't translate up to higher talent levels without trying it.

Like the spread option in the NFL

130
by DoubleB :: Tue, 11/17/2009 - 8:23pm

Most high schools probably shouldn't punt. Most have very poor punting games to start with. If you're going to net 20 yards a punt, what's the point? Particularly in a game that is generally based on a big play. There aren't many high school drives that go 10+ plays leading to TDs. Too many young kids making too many mistakes.

A game (college and above) that prevents bigger plays and has better kickers warrants a punting game.

116
by T. Diddy :: Tue, 11/17/2009 - 6:35pm

I think "put their careers on the line" explains what you're describing, though. If the Patriots had punted and Manning had driven the Colts down the field to score the TD and win the game, nobody would have criticized Belichick for punting on 4th-and-2. (They might have criticized him for other things, of course.)

A coach who isn't 100% sure that the ownership/management is completely behind him might quite rationally (for his own self-interest) pick a non-optimal team strategy for any individual game, on the thinking that for his own career, it is better to lose conventionally than unconventionally, even if it lowers his overall winning percentage. The coach's self-interest and the team's interests are not always aligned, especially during an individual game.

121
by Paul A (not verified) :: Tue, 11/17/2009 - 6:48pm

Umm, for the same reason that baseball managers sacrifice bunt and issue intentional walks?

111
by Chimneyfish (not verified) :: Tue, 11/17/2009 - 6:17pm

Great article Bill. I just have a problem with this part:

"The Colts aren't going to score every time they get the ball on the opposition's 29-yard line, but they will score most of the time. Toss in momentum and the quality of the Colts' offense versus the Patriots' secondary, and you can estimate, say, an 85 percent chance of the Colts scoring in that situation."

While the likelihood of the Colts scoring on the opposition's 29 may be 85%, the Colts specifically needed a touchdown in this situation- not just any score. I'd imagine that removing the possibility of a field goal would lower that 85% scoring figure quite a bit. Advanced NFL Stats figured the average NFL team to get a touchdown in 53% of the time in that same situation. I don't believe that even Peyton Manning exceeds average to the extent of bumping that probability all the way up to getting a touchdown on 85% of any drives from any position (let alone with only 2:00 left to play).

117
by Pat (filler) (not verified) :: Tue, 11/17/2009 - 6:36pm

The Colts scored a touchdown 4/4 times within the 30 that game. 85% is roughly the point at which the likelihood of getting 4/4 exceeds 50%. 85% is probably the best guess that Belichick could've made.

54% seems wildly unsupported by the evidence from that game.

142
by RickD :: Tue, 11/17/2009 - 9:45pm

It's a bit immaterial. The problem was that Belichick didn't think that there was a significant difference between their TD% from the Pats 30 to the Colts 20-25. You only punt if you think it's going to make a meaningful difference for the defense.

Based on how thoroughly the Colts had hosed the Pats' D on the previous drive, it seems eminently justifiable for BB to lack faith in its ability to stop the Colts from either their own 30 or from the Colts' 20.

Belichick has a functioning memory. He knew that the Colts' offense was red hot and that his defense was gassed. Why is he obliged to send it back out on to the field?

To let the Colts get their 3rd 80-yard <2 minute TD drive of the quarter?

144
by Pat (filler) (not verified) :: Tue, 11/17/2009 - 10:00pm

Assume that logic. In that case, the 3rd and 2 call was horrible, the timeout taken was really awful, as was actually preventing the Colts from scoring after that, because you've already conceded that you can't prevent them from scoring a TD. All you do is shift the "bad decision" from 4th and 2 to nearby decisions.

147
by jackgibbs :: Tue, 11/17/2009 - 10:06pm

and those decisions WERE bad, as bill himself would surely admit. the fourth down call is a lot closer

205
by Pat (filler) (not verified) :: Wed, 11/18/2009 - 2:55pm

You can't isolate them from each other like that. The 4th down decision only makes sense if 1) you've got a high-percentage, low risk play choice and 2) you have a strategy for dealing with failure. I don't believe they had either: you can make a claim that they had the former, but they certainly did not have the latter. And without that strategy, it's a bad decision. You're better off punting and giving your opponent more opportunities to make a mistake.

115
by Jason Tz (not verified) :: Tue, 11/17/2009 - 6:33pm

Seconding what a few have mentioned above...

What are the odds of the Patriots gaining 2 yards by running the ball on consecutive downs? 90%? 95%? Wasn't THAT was the relevant statistic, starting on 3rd and 2? I gotta believe Brady falling forward twice from right under center would have worked.

Instead, the Pats threw a dangerous out-route, and then had to call time-out (allowing the Colts to get set) to make the decision about whether to go for it or not. Then they drew up a play where one of the primary targets ran ONLY AS FAR as the first down marker. Which backfired. And was lame. Statistics don't even need to be consulted to know that Belicheck blew that sequence.

118
by Pat (filler) (not verified) :: Tue, 11/17/2009 - 6:39pm

It's worth noting that that was their last timeout, and it left the Colts with one. Add in the fact that the route brought the WR's momentum back towards the line... seriously, if Belichick thought that play had a 60% chance of success, he was absolutely delusional.

134
by RickD :: Tue, 11/17/2009 - 9:22pm

Actually, if Faulk had been given credit for forward progress, it would have succeeded.

Why, oh why, does that fact get ignored?

137
by Pat (filler) (not verified) :: Tue, 11/17/2009 - 9:32pm

Probably because the ball wasn't spotted where he landed, so he was given some forward progress, and the guy who spotted the ball was staring straight at him with a completely clear line of sight.

It should also be noted that if he had been awarded forward progress, it would've been because of a mistake by the Colts DB in even touching him before he was behind the line.

140
by RickD :: Tue, 11/17/2009 - 9:39pm

My complaint was not about what the placement of the ball was, but rather that consideration of that fact doesn't seem to enter anybody's minds when they tear into Bill Belichick.

A play that came within a whisker of actually succeeding, had a 60% chance of success?

I'm curious - do you think Kevin Faulk has a 40% bobble rate?

Where is the 60% coming from?

Certainly on that given day, with that offense against that defense, the average gain per snap was well over 2 yards.

143
by Pat (filler) (not verified) :: Tue, 11/17/2009 - 9:57pm

A play that came within a whisker of actually succeeding, had a 60% chance of success?

I'd say, yeah. I mean, the only reason it came within a whisker of succeeding is because the DB didn't play the ball very well. But the play design was really bad: backward movement towards the line, that short? Did he not see the Philly game last week?

Certainly on that given day, with that offense against that defense, the average gain per snap was well over 2 yards.

It's not the average gain, it's the percentage of time a play gains over 2 yards. Funny thing is that this play is actually a bad example for you for using average, considering there's no way the receiver would gain more than 2 yards, and so on average, he'll gain less than 2.

It's also not that play, the way it turned out - when the play's called you don't know their defense. It just wasn't a good play call - Moss wasn't doing anything to peel the defender away from Faulk, for instance. But in general there wasn't anything interesting going on - the Patriots went 4-wide, the Colts went with a man defense, only two receivers ran viable short routes... just really amazingly vanilla.

149
by Jerry :: Tue, 11/17/2009 - 10:20pm

The Pats have used their passing game for possession for a couple years now, so it's not that surprising that they'd throw instead of run in this situation.

122
by GlennW :: Tue, 11/17/2009 - 6:49pm

Solid post morganja. I agree that there's a bit too much certainty being purported in some of these probability analyses (but to his credit, not by Bill Barnwell).

Apparently AdvancedNFLStats' analysis is now semi-famous because it was featured in the Boston Globe and batted all over the local airwaves today. Just my gut reaction based on the game conditions (I've already acknowledged, I'm not taking a side): I think their 60% estimate for a successful Patriots' conversion is too high, and their 52% estimate for a Colts TD from the 29-yard-line (against a gassed New England defense) is easily too low. To be fair, I also think their 30% estimate for a Colts TD from their own 30 is too low also. Actually, I don't think there's any way Bill Belichick goes for if he thinks his defense has a 70% chance of a stop.

125
by funbob (not verified) :: Tue, 11/17/2009 - 7:46pm

The numbers game aside it was a bad decision because it was spur-of-the-moment. I've read Lombardi over at the NFP making the case that this was all planned out ahead of time but Belichick's actions just don't support that. If they knew they were going for it on 4th down regardless then a run is the only reasonable choice on 3rd and 2: run the clock (I believe) below 2 minutes and set yourself up better for the shot on 4th down. If they knew they were going for it on 4th down they wouldn't have sent then punt team out and then blown a timeout changing their minds.

I fully understand the thought process and the possible risks/rewards. In my mind the risk is too great; even if the entire gameplan was based around eventually taking a shot on 4th-and-2 from the -28 it's just not the kind of decision I can support. Setting perfect world arguments aside I don't believe for a second that what I saw was evidence of Belichick acting on a cunning plan plan. He was flying by the seat of his pants and he blew it so badly that he left himself without a timeout to challenge with.

And that's why it was a terrible decision.

136
by RickD :: Tue, 11/17/2009 - 9:27pm

I don't know why Lombardi thinks this was planned out. Obviously it wasn't planned out, and Belichick admits as such.

That doesn't make it a "terrible decision". Certainly the implementation could have been done better - it would certainly have been better to have the plan in place from 1st down. But...modulo a dubious spot by officials who ignored the forward progress on the play, the play actually did succeed.

How, then, can it have been a "terrible decision"?

I don't understand people who say "I fully understand the thought process and the possible risks/rewards. In my mind the risk is too great."

Clearly, you don't "fully understand" the thought process. You are still beholden to your gut feelings and are unwilling to let your decision-making process be ruled by strategy concerns alone. Every analysis that has looked at the decision-making process from a mathematical one has concluded that going for it on 4th down improved the Patriots' odds of winning. The problem is that the strategy is counter-intuitive.

All of the outrage of the past two days is based on people trusting their ingrained feelings about the subject more than any analysis.

152
by Johnny Socko (not verified) :: Tue, 11/17/2009 - 10:30pm

Dude, enough on the "bad spot" argument. There is no forward progress given when the receiver bobbles the football. That is a rule. The ref was in perfect position to make the call. He immediately makes the juggle motion with his hands and without hesitation spots the just inside the 30 yard line. In contrast, the ultra-slow-mo television reply that we have all seen is at an angle that makes it impossible to determine proper depth perception as it relates to Faulk and the first down line. How you can claim this replay proves anything beyond a reasonable doubt is crazy.

181
by GlennW :: Wed, 11/18/2009 - 11:27am

"Every" analysis? Really? I think that particular 4th-down playcall had a less than a 50% chance of succeeding (again, it's more like making a 2-point conversion with the game on the line), and that the Colts had at least a 75% chance of scoring a TD from the Patriots' 29. I'm not crazy about those odds. It's still close, so I'm not judgmental of Belichick-- he did what he thought was right-- but it's this notion that it's been "proven" that he made the right decision that is suspect. No such proof, or even a reasonably convincing argument, exists either way.

133
by t.d. :: Tue, 11/17/2009 - 9:20pm

My take is that people wouldn't be rushing to defend this move it was Norv, and Switzer was ridiculed for the same move in '95. So, even more so than 'you can't criticize this move just because it didn't work', I think the standard should be, 'if you would have criticized Norv or Barry for the move, you should rip Bellicheck'. Shannahan 'couldn't win without Elway', I guess Bellicheck can't win without Vinateri.

139
by RickD :: Tue, 11/17/2009 - 9:34pm

Yeah, but then you are in the position of accusing Belichick defenders of being engaged in a personality cult.

How do you know we wouldn't defend Norv Turner, or any other coach for doing the same thing.

The reason Barry Switzer was ripped was because called the same play! on third and fourth down in that game in question (against the Eagles). And certainly that's a reason I've ripped Norv Turner this year with his goal-line strategy. Long after it should have been obvious that LT was not scoring on the goal line dive with ease as he did 3+ years ago, Norv continued to call that same play.

He may have adjusted by now. Norv is dumb, but he's not a complete idiot.

As for Switzer...

145
by t.d. :: Tue, 11/17/2009 - 10:00pm

I think it's exactly a personality cult thing with why this is so controversial. Switzer had the best rushing attack in football and two attempts to get two yards (I think he actually had three tries, because one play was waived off). Martz was often criticized when playing the percentages. And Norv is ridiculed even by writers on this site when his gambles don't pay off, based on results, not process. The standard should be the same. (we don't even know what the probabilities for this particular situation, since it's such a high leverage, rare circumstance, i.e., not just a fourth and two, but a fourth and two, you lose the game if you don't hold, you win if you do). Bellicheck, more than any other factor from the Pats (though not from the Giants) blew the last Super Bowl. He shouldn't be above criticism.

146
by sethburn :: Tue, 11/17/2009 - 10:04pm

I'm going to do something here that I usually prefer not to do: Use the actual financial numbers from the in game trading.

when the Patriots were expected to punt they were given a 64% chance to win (-178 to American wagerers, .56 in European style odds). When the markets realized the Patriots were punting the odds jumped to 71% (-245 in American odds, .41 European odds).

People can claim whatever they want, but when people had to put their money on the line they gave the Patriots an additional 7% when they decided to go for it.

SEVEN EFFING PERCENT.

The math guys are agreeing that the decision was correct. The meatheads are panning it.

The decision was correct and it wasn't close. Anyone who who says the decision is wrong has failed an intelligence test. Anyone who says it was close is still failing basic football math.

The decision was correct and history will not look kindly on those who suggested otherwise.

148
by t.d. :: Tue, 11/17/2009 - 10:17pm

Yeah, they sure were right to place their trust in Bellicheck. Couldn't be that it's usually paid off in the past for him to make unconventional moves, and people were putting their trust in that. Nope, the wisdom of the crowds is always right, and people who second guess Bellicheck must be stoopid.

You are what you record says you are. It was a bad move because it didn't work, and if you take the praise, you are subject to the criticism. I'd still love to have Bellicheck as my team's coach, but he's not perfect.

150
by sethburn :: Tue, 11/17/2009 - 10:22pm

We are not exactly talking about the traditional wisdom of crowds. Financial markets are a little different because it roughly translates into $1 for 1 vote.

I might add you are attempting to argue against a straw man. The wisdom of crowds isn't wrong when a favorite loses. The wisdom of crowds is wrong when they have the wrong team favored.

Also, if you don't really understand betting markets you might not understand this but: when the market moves 7% it might have moved the wrong amount, but the odds of it having moved in the wrong direction are close to nil.

151
by sethburn :: Tue, 11/17/2009 - 10:24pm

Oh, and one of the tenets of Football Outsiders is that you are NOT what your record says you are in the short run. Teams that have a losing record can be considered much stronger than teams with a winning record.

That is not to say I can justify FO's rankings of the Eagles. :)

160
by johnmartinolive... :: Tue, 11/17/2009 - 11:07pm

Ok, so next time a Pat's opponent gets a TD punt return, then it was the wrong call to punt. It was a bad move because it didn't work.

154
by Johnny Socko (not verified) :: Tue, 11/17/2009 - 10:42pm

This is the funniest argument yet! I love your vision of people who gamble on football as a bunch of math geeks who pour over statistics to such an extent that they can instantly, without hesitation, compute precise probabilities on the outcome of a game as it plays out. Wow! I don't even need to watch the games anymore because these football savants can just TELL me whats going to happen next.

In contrast, my vision of people who gamble on football (from first hand experience of spending way too much time in Las Vegas) is a bunch of drunken fratboys who bet with their hearts and not there heads. They are quick to let their emotions get the best of them and more times than not will end-up passed out drunk before the last game of the night ends.

156
by jackgibbs :: Tue, 11/17/2009 - 10:51pm

the drunken fratboys PUSH the lines, they do not set them. smart people with a vested interest in making both sides seem even do that

157
by Johnny Socko (not verified) :: Tue, 11/17/2009 - 11:01pm

I understand, but that's not what the poster was talking about. He was talking about the line moving as the game was being played. This is the not the traditional Las Vegas line mind you, but the financial model of betting that has long been popular outside the US with sports like tennis. Regardless, I would NEVER put much stock in this type of analysis. Its simply too fraught with emotional decisions made with bad information.

161
by sethburn :: Tue, 11/17/2009 - 11:13pm

It is a myth that most of the money is public money and that the public moves the lines.

In terms of in game wagering it is even more of a myth.

As for in game-tennis wagering, that world is vastly different from the world of the NFL.

I feel like a guy trying to explain evolution to a fundamentalist congregation. It might take 5 years, it might take 20, but there will come a time when NOT going for a first down in this kind of situation will get you fired.

164
by Johnny Socko (not verified) :: Tue, 11/17/2009 - 11:28pm

OK, I bite. If most of the money is not public money, then what the hell is it? And what the hell makes "private" money so damn insightful that they can precisely calculate new probabilities on the outcome of a game within seconds of finding out NE is going for it on 4th and 2?

I'm not saying punting is right or wrong, I'm just calling your "the line moved 7%" theory as complete BS.

169
by sethburn :: Wed, 11/18/2009 - 12:01am

OK, two issues. Where the money comes from, and what the 7% means. I'll start with the money.

Public money can also be considered dumb money. It is money that will back the popular side, and is willing to do so at bad prices. There was a time when that money mattered a great deal, however the gambling world has evolved. Now, a much higher % of the money wagered is exceedingly price sensitive. It can be called sharp money, but that might be giving it more credit than it deserves. Someone betting on the Packers to win doesn't necessarily have to have any inside information, or be certain that the Packers are going to win. Rather, they have reason to believe that the Packers are more likely to win than the betting line implied, and that they have a positive expected value (referred to as positive EV). On way of thinking about this is someone betting on the roll of a six sided die. If they bet $100 that a 1-4 will be rolled at even money (risking $100 to win $100), then the will win 2/3's of the time and have a positive expected value (of $33.33). Of course, if a 5 or 6 is rolled, Peter King will call them a schmuck.

The sharp money is by no means perfect, but when I am talking about the in-game %'s I'm referring to the action at the high limit sportsbooks where winners are not kicked out or limited in any fashion.

As for the 7% being BS, I would literally be willing to back every side that has been the beneficiary of a 7% move at the pre-move price. Literally, every single one, sight unseen. I don't know how much stronger I can say this. After the fact some of those sides will lose, but I can guarantee that every single one was was positive EV (post move, at the pre-move price).

171
by Johnny Socko (not verified) :: Wed, 11/18/2009 - 12:25am

I can appreciate your knowledge of the industry, but must question your theory in the last paragraph. You claim to guarantee that every single one of the 7% moves was positive EV. While this is strong claim, it is impossible to actually verify. This particular situation we are talking about (NE vs. COLTS) is so rare that it has generated tons of discussion throughout the entire sport. So it is certainly possible that THIS 7% move was irrational and flat out wrong.

These are sports games we are talking about and you can't claim to KNOW anything as fact. That's the nature of the beast, these are games being played by humans and are subject to a countless number of variables. Since no two games or situations are the same it is impossible to put accurate (and factual) probabilities on their outcomes. This is not Strat-to-matic baseball being played with dice rolls.

Nevertheless, good discussion. But its time for me to call it a night.

173
by sethburn :: Wed, 11/18/2009 - 12:51am

We are on the same page.

I too am very tired and I too am enjoying this discussion.

Some notes:

1. I can actually track what a 7% move implies using the data from this and last season. That may or may not be a statistically significant sample, but if it is not significant yet, I'll simply keep adding data until it provides a result that is not random chance.

2. There are actually a lot of things I can know for a fact. While any one game can have an aberrant result, and any one line can be wildly wrong, but I can know that the Yankees are more likely to win the 2020 World Series than the Pirates. I can know that over the course of an NBA season home teams will score more points than road teams. I can know that over the course of a season more than half of the double digit favorites will win the games in question.

Let me use the NCAA tournament as an example. I don't know that all 4 #1 seeds will beat all 4 #16 seeds. I do know that all 4 #1 seeds will be favored.

If I choose a card at random from a normal 52 card deck, I don't know I won't pick a heart, but I do know that I am more likely not to pick a heart.

I can't know that the Patriots win if the go for it. I can know that if they have 1000 games where they go for it and 1000 where the punt... :)

And back to the 7% move, I can know on average they are correct. Can I prove that they are worth 7%? Not yet. I can prove that a 7% move is worth more than 0%.

179
by Johnny Socko (not verified) :: Wed, 11/18/2009 - 10:28am

I'm back and rested. I will take your assertion that a 7% move always presents a positive EV position at face value. However, you are basing this point on your data that indicates this over the longhaul. This data does NOTHING to prove that any one particular 7% move presents positive EV. In fact, logic tells us that a certain percentage of these moves will actually be negative EV. The problem is we have no way of determining which moves are positive and which are negative. We only know (based on your data) that overall a 7% move results in positive EV.

Therefore, the claim that since the line moved 7% when NE decided to go for it PROVES that is was a good decision is completely absurd.

194
by sethburn :: Wed, 11/18/2009 - 1:57pm

OK, this is actually a fair ground to debate.

Will there ever be a 7% move in the wrong direction? Actually, even that isn't quite right. Will there ever be a 7% move that is on the wrong direction or purely random such that the true value has not changed at all?

I think that is the question. And yes, I am fully prepared to take the side that 7% moves are literally all in the correct direction and that none will be without proximate cause.

It is difficult to prove a negative, but there really has to only be 1 case where there was a 7% move that was in the wrong direction for my thesis to be incorrect.

Out of curiosity, given the presumption that over the long haul 7% moves are positive EV, how confident would you be that any 1 move is correct, sight unseen?

196
by Johnny Socko (not verified) :: Wed, 11/18/2009 - 2:20pm

I have no way of determining if any 1 move is correct, so my answer is I don't know. You say there is no way to prove a negative, I'll go one step further and say there is no way to prove a positive within the contex of this discussion either. No amount of data you can produce with regard to sports betting lines is going to prove anything in relation to proper football strategy. There is simply no direct correlation between these things.

Now, if you want to use the betting line as SUPPORT for your assumption that BB made the right call - be my guest. But thats all it is, support.

The stats you cite are specific to betting lines, which are contingent upon where the money is flowing. However, just because more people (or money) thinks one side has gained an advantage during the course of a game does not make it true. You cannot speak this theory into existence.

To use some of your own words, throughout history the common perception of things is often later proved to be wrong. You cite punting strategy as one of these examples. Therefore, who's to say those who are moving the betting lines are always correct in their commonly held assumptions? You can't. Therefore, the betting lines offer no proof that punting in this situation is truly the best strategy.

214
by sethburn :: Wed, 11/18/2009 - 6:55pm

“To use some of your own words, throughout history the common perception of things is often later proved to be wrong. You cite punting strategy as one of these examples. Therefore, who's to say those who are moving the betting lines are always correct in their commonly held assumptions? You can't. Therefore, the betting lines offer no proof that punting in this situation is truly the best strategy.”

The betting lines are panning punting, as does the academic research.

I’m not certain that the decision was worth 7%. I do know that the calculus involved was fail 4th down * fail stop vs. fail post punt. I know that people can be fooled by randomness and as such have difficulty telling the difference between what is a random result and what isn’t. On a macro level 7% moves DEFINITELY signify an increased likelihood of success for the beneficiary. The question is on a micro level. What is our level of confidence that any individual move is indicative of an increased likelihood of victory? My answer is somewhere well North of 99.99%.

We’ve seen massive changes in baseball and basketball, but there is a pretty clear trend. The sports have become more efficient (more rational if you wish). We still have our flat Earth proponents (hi, I’m Joe Morgan), but the progress is obvious. Will we ever go too far and need to take revenge AGAINST the nerds? Probably, but not necessarily in either of our lifetimes.

Incentives matter. Betting markets reward being accurate. Pundit markets reward generating interest. Coaching decisions are a bit more complex. If you coach to win the game, there are times when you’ll bring the punditry on your head.
If some football team had developed some intellectual property, how could the reveal it while maintaining their edge?

Belichick knows he was right. I’d wager large sums that Kraft knows too. And I’d wager larger sums that the betting markets KNEW as well. :)

I’m currently seeing if there is an effective statistical proof regarding in game line moves. One of the things I'm finding is that 7% moves are often something along the lines of a 30 yard touchdown pass (remember, the offense has already moved to their opponent's 30 yard line).

163
by sethburn :: Tue, 11/17/2009 - 11:24pm

Oh, one other thing:

Vegas isn't where real money is bet on sports. The internet is where real money is bet on sports. Vegas is where the frat boys hang out.

183
by GlennW :: Wed, 11/18/2009 - 11:40am

That's laughable. "History" is written by the victors. Belichick and the Patriots lost. The people who bet money on the Patriots *to win* lost that money. Nobody's going to remember what odds were posted on some obscure in-game internet betting site (seriously, how many bettors had the time and inclination to respond to the 4th-down situation? How could one even be certain that the Patriots weren't punting?). And exactly who is this that's not going to look kindly on the dissenters? Will these people come to our homes to punish us?

More importantly, since when do betting lines represent probabilistic reality? Bettors are as likely to respond to conventional wisdom (or other whims) as anyone else, and maybe more so. Most gamblers I know most certainly are NOT "math guys".

193
by sethburn :: Wed, 11/18/2009 - 1:44pm

History is written, and then rewritten over and over.

No one is going to come to Peter King's house (I hope), but certainly you are aware of a multitude of historical examples where a decision was popular at the time, but later viewed as an abomination? It takes time for societal norms to change, and given the massive amount of coverage given to this decision, it will probably be considered the... I hesitate to use the phrase "turning point", but I cannot think of a better expression. Football play calling is going to become more efficient over time.There was a time long ago Where a 6 point lead late in the game almost assured victory. when Notre Dame broke Oklahoma's great winning streak, the final score was 7-0. In the modern era teams are much more effective at moving the ball, particularly at moving the ball swiftly.

Coaches that play the actual %'s will be ridiculed as gamblers, particularly when they lose, but they'll also win more than they otherwise would. We've already started to see this with high school teams.

As for the betting lines indicating probabilistic reality, they too have become more efficient in the modern era. There is always variance, but in the long run larger favorites will win more often than smaller favorites. In particular, 71% favorites will win more often than 64% favorites.

In terms of bettors believing conventional wisdom, some do. I'm sure in Vegas there were guys with money on the Patriots to win screaming at them to punt. However, $1-1 vote is a lot different than the guys screaming at the screens.

208
by Will Allen (not verified) :: Wed, 11/18/2009 - 3:04pm

Well, just as the market can stay stupid longer than the short seller can stay solvent, ridicule, if an owner is too easily swayed by such things, can result in a coach getting fired, before he can demonstrate that his strategy will win more frequently.

212
by sethburn :: Wed, 11/18/2009 - 3:46pm

Times change:

http://fifthdown.blogs.nytimes.com/2009/11/18/belichick-emboldened-the-g...

We've seen it in baseball and basketball.

159
by jackgibbs :: Tue, 11/17/2009 - 11:07pm

I can understand where he's coming from. the people taking the money felt a lot more comfortable with a new england victory once NE decided to go for it than they did when they thought NE was punting.

I also agree this is no way to prove anything, though

162
by sethburn :: Tue, 11/17/2009 - 11:22pm

I find this debate baffling.

On any given Sunday a 3-1 underdog might win but it would be insane to argue that they should have been favored.

It isn't a matter of feeling comfortable. It is a matter of probability. The market had new information and that information was processed.

Let me put it this way, if someone knew that a team was going to go for it in such a situation, and the market was assuming a punt, that person could get a lot of expected value (EV) by betting on the team going for it.

The fact that people can say whatever they want without having to put money behind their words is what allows for stupidity to flourish (I'm looking at you Peter King). If we could replay the game 1000 times with the Patriots going for it, and 1000 times with the Patriots punting, I'd gleefully bet on the Patriots to win a higher percentage of the games where they go for it than when they punt. I'd go so far as to call anyone who took the other side of such a wager a fool.

Fools can say that punting was correct and that going for it was stupid. Fools are also seperated from their money when they put their money where their mouths are.

166
by Johnny Socko (not verified) :: Tue, 11/17/2009 - 11:36pm

I'm amused by your insistence that only people who place money on the outcome of a game can accurately predict it. That is so far from reality that I question your sanity. Its as if you think people who place money on a game somehow obtain a magical power that enables them to see the future.

167
by sethburn :: Tue, 11/17/2009 - 11:45pm

And another straw man gets argued against.

The people placing money on a game during it might be action junkies. They might be math nerds. As a general rule they are not you normal casual bettor.

The people wagering on a game are the same kinds of people who buy and sell commodities, currencies, equities, bonds, etc. The point isn't that they are perfect. They aren't perfect, but they are far better than random chance.

You don't have to think of these people as all-knowing to understand that when they are willing to pay 10% more for a commodity due to a decision someone made regarding that commodity... well, that was a good decision.

170
by Johnny Socko (not verified) :: Wed, 11/18/2009 - 12:06am

We are getting way off topic, but what the hell. You insist that people who buy and sell commodities, currencies, etc. are far better than random chance. However, that is open for debate. For example, it is a fact that 69% of mutual fund managers consistently UNDER perform the generally accepted index for their particular market. In other words, an investor would have been better off simply placing an equal amount of money in all companies within a particular industry rather than trust a professional adviser to active manage his investments within that same industry.

So your insistence that the line moving 7% is somehow certifiable proof that going for it was a good decision is completely irrational. The only thing it proves is that people betting on that game were "comforted" by the fact NE was not punting. This neither proves nor disproves that it was actually a good decision. The betting line is completely independent of the actual game on the field.

172
by sethburn :: Wed, 11/18/2009 - 12:29am

If we start taking a random walk down Wall Street, we will have gotten WAY off topic. :)

For the record, you are correct as a general rule having someone attempt to invest your money in a fashion that beats the market is generally a bad call.

Having said that, a 7% move can be considered proof. in fact, I am so certain of this that I would be willing to take any side that has been the beneficiary of a 7% move at the pre-move price.

Can we agree that we live in a world where traders are imperfect (particularly when their incentives are not aligned with those of their clients), and that 7% moves are indicative that one side has gained an advantage over another?

174
by Will Allen (not verified) :: Wed, 11/18/2009 - 3:32am

Interesting stuff. Now, imagine a world in which Belichik can and does announce to the betting population, prior to first down, that he has adopted a 4 down strategy, while keeping that tidbit from the Colts. How much subsequent movement is there?

191
by sethburn :: Wed, 11/18/2009 - 1:31pm

Oddly, the information is not all that powerful if he is going to go for it on 4th and 10. Now, if he were to announce that he would go for it on 4th and short, say, short being 4 or fewer yards to go, the markets would have moved towards the Patriots, and the each subsequent stop on 1st, 2nd, and 3rd down, would be less effective for the Colts in terms of increasing their win %.

195
by Will Allen (not verified) :: Wed, 11/18/2009 - 1:59pm

Well, the term "four down strategy" is not meant to mean, "We absolutely have to score on this possession, thus we will not punt, no matter what", but rather, "We will not punt unless it is fourth and long, and we will make our play calls on first, second, and third down with that in mind". The point is that Belichick, by not adopting a four down strategy until fourth down, significantly reduced the advantages of a four down strategy, and this was illuminated by a very poor third down call. I had no real issue with, stripped of context, a decision to go for it on that spot on fourth and two. I assume Belichick knew the condition of his pass defense, and that the condition was not good. However, in the context of what led up to Belichick's decision to go for it, I conclude that Belichick fell well short of his goal of maximizing his team's chance to win, via sound strategy.

198
by Will Allen (not verified) :: Wed, 11/18/2009 - 2:24pm

To put it another way, assume three parallel universes. In the first, Belichick is able to signal to the betting population, just prior to third down, that he will go for it on fourth and short, and is running a qb sneak on third down. In the second, Belichick signals that he will go for it on fourth and short, and that the third down call is a sideline pass just past the first down marker. In the third, Belichick says nothing about willingness to punt, but signals that the third down call is a sideline pass just past the first down marker. What movement results in each universe? I suspect that the gap between two and three is significantly narrower than the gap between one and two, because the betting population would think that a coach willing to go for it on fourth and short would not be calling that pass play on third down.

201
by Will Allen (not verified) :: Wed, 11/18/2009 - 2:33pm

Geez, I think I'm lost in a parallel universe. Forget the last sentence of the post above.

204
by sethburn :: Wed, 11/18/2009 - 2:49pm

If I may simplify, the best scenario for the Patriots is if the know in advance they are not punting on 4th and short.

209
by Will Allen (not verified) :: Wed, 11/18/2009 - 3:06pm

You may, and my fingers thank you!

211
by sethburn :: Wed, 11/18/2009 - 3:25pm

I'm currently listening to the Bill Simmons podcast. There are so many logical fallacies here that it is sad.

It isn't about whether or not you trust your defense, or whether or not you trust you offense. It isn't about whether or not you are getting calls. It is not BINARY.

It does not matter that the Patriots had a 34-21 lead, or a 31-14 lead. It does potentially matter that they don't trust Laurence Maroney.

Having a plan to use all 4 downs to get a 1st down increases your likelihood of winning.

I'll note that the worse your defense is, the worse punting is. Similarly, better the opposing offense, the worse punting is.

Wow, still listening to the podcast. Belichick wasn't bored. He just wants to win. End of story.

In conclusion, Belichick made tactical errors in not planning ahead to go it on 4th down. He made the correct move disregarding the sunk costs of the lost time outs.

Simmons just referenced poker with a bad analogy. He talked about calling with 6-9. The correct analogy is that the Patriots made a mathematically correct call on the flop that looked bad after the turn that and looked awful on the River.

199
by sethburn :: Wed, 11/18/2009 - 2:26pm

Alright, presume a rational 4-down strategy. Yes, the Patriots benefit from that. I don't know that Belichick had made his decision before 3rd down. He might have made it during the Patriots final time out.

It is probable that the Patriots could have made better tactical decisions with their play calls on 3rd and 4th down. I think you are right that the Patriots both made the correct decision to go for it on 4th down, and also gave away some of that advantage with the previous play calling.

Yeah, not a perfect game by Belichick, but he is being excoriated for the wrong reasons.

202
by Will Allen (not verified) :: Wed, 11/18/2009 - 2:40pm

I'm pretty sure he said after the game that he didn't consider a four down strategy until fourth down arrived. If he is being candid, that's a bad error.

203
by sethburn :: Wed, 11/18/2009 - 2:48pm

Yeah, he would have been better off had he made the decision earlier.

165
by jackgibbs :: Tue, 11/17/2009 - 11:35pm

I don't disagree. I'm just saying that using the argument 'the books said it was correct to go for it' is not really proving anything new. They have access to the same numbers used a bunch of times above, and reached the same conclusion. The numbers are the probability with or without the books

168
by sethburn :: Tue, 11/17/2009 - 11:47pm

They have access to a lot of information, need to back that information with cash, and have to do so in real-time. An academic works under different parameters. That an academic reaches this conclusion is highly correlated with a gambling market reaching the same conclusion, and the correlation is most likely based on the underlying reality.

175
by morganja :: Wed, 11/18/2009 - 3:33am

Sethburn, you're insane, but I appreciate the cocaine-esque induced enthusiasm.

I guess you've never heard of a boom or a bust or do you think they are entirely monetery phenomena?

190
by sethburn :: Wed, 11/18/2009 - 1:29pm

I vaguely remember something about tulips...

There is this disconnect. I am not assuming that markets are 100% efficient, or that the people involved in markets are all geniuses. I am arguing that the response to the decision was so strong as to not possibly be ambiguous to its meaning. That the market moved 7% did not prove that the Patriots chance to win increased 7%, but rather that the Patriots chance of winning increased by an unknown amount. No one trader moved the market 7%, the volume was far too deep for that. As for emotion, I'm sure that effects some traders, but again, this was simply a huge move.

Furthermore, for the move to have been wrong, one of the two lines must have been massively wrong. That is, either the Colts were huge value at 29% (right before the snap), or the Patriots were huge value, regardless of their decision, during the time out.

My friend brought up another piece of information: The 64% for the Patriots might have already been pricing in a non-zero chance that the Patriots would go for it. It is possible that after an average net punt the Patriots would have been 60% to win.

180
by Fred (not verified) :: Wed, 11/18/2009 - 10:30am

Forget the percentages. Clearly the proper play call was to have Brady roll left and throw against his body 15 yards downfield.

http://www.youtube.com/watch?v=6x3DmBfQrIE

182
by tuluse :: Wed, 11/18/2009 - 11:33am

Since when is right in front of the QB considered across his body?

192
by Fred (not verified) :: Wed, 11/18/2009 - 1:38pm

Running left and throwing for a right handed QB. More difficult than rolling right and throwing straight ahead.

Why did I even respond to this?

213
by Jim D (not verified) :: Wed, 11/18/2009 - 4:45pm

Nice article and commentary...As a Pats fan, I liked the move at the time...Just from the gut feeling that they would probably be able to pick up 2 yards after having called time out and set up a good play. Obviously didn't work out that way but I was (and still am) fine with the decision.

I do think it's fair however, to base your opinion of the decision on the result.