Writers of Pro Football Prospectus 2008

21 Nov 2012

The FO Trap Game Challenge

Did the Houston Texans nearly lose to Jacksonville last weekend because it was a "trap game?"

I saw this suggested by a number of people across the Internet. It certainly was suggested by plenty of people in my Twitter feed. But what is a "trap game" anyway?

In Pro Football Prospectus 2007, we ran an article analyzing the concept of the "trap game." For that article -- which you can read here -- we defined a "trap game" as "any game against a sub-.500 opponent slotted between two games against opponents who, on the season, posted records above .500." Based on this definition, our analysis showed that winning teams from 1983 through 2006 actually had a slightly better record in these games than they did in other games against sub-.500 teams.

This essay did not do much to silence the believers in "trap games," however, primarily because the definition of "trap game" seems to change based on what surprising loss or close win someone is declaring to be a "trap game."

When people kept sending me tweets arguing that Houston's near-loss disproved my belief that there is no such thing as a "trap game," I had the same response: In what universe is it a "trap game" when your next game is against a 4-5 non-conference opponent?

Oh no, came the response. It was a trap game because the next game is on national television. Or, it was a trap game because the next game was on short rest. Or, it was a trap game solely because Houston had played Chicago the week before, and it didn't matter what team Houston played the next week. It seems to me that last one would be classified as a "letdown game," not a "trap game," but whatever.

Again, this is the problem with the "trap game" concept. If there is such a thing as a "trap game," we have to start by defining what the hell that is. Does it have to be a game between two good opponents, or just after a good opponent, or just before a good opponent? Or, in the case of the Texans, does it have nothing to do with opponent, just with the fact that the next game is on Thursday or on national television? Does a trap game have to be at home? Or on the road? Or either? Does it have anything to do with whether the opponent is a division rival, or not a division rival? Or does a trap game have nothing to do with the win-loss record of your opponent, but instead is related to the point spread? That was Bill Simmons' idea of a trap game on the B.S. Report a couple years ago.

Last week, Houston played a weak Jacksonville team at home, in a "trap" before short rest and a national game on Thanksgiving on the road against a bad Lions team. They nearly lost. New England played a weak Indianapolis team at home, in a "trap" before short rest and a national game on Thanksgiving on the road against a bad Jets team. They slaughtered the Colts. Why is one a "trap game" and the other not?

(Speaking of trap games based on short rest, here's Danny Tuccitto's XP from earlier this week about the effect of short weeks and bye weeks on team performance. A short week might hurt road teams a tiny bit.)

So I'm putting a call out to all the people bothering me about this on Twitter, and all the other FO readers besides. This is your opportunity to prove me wrong about the existence of "trap games." I will run any guest column that can statistically demonstrate that there is such a thing. However, your column needs a specific definition of what a trap game is, and you need to show that good teams do in fact lose in that situation more often than they lose to similar opponents in other situations, to a statistically significant extent. Anecdotes that ignore all the times when teams in "trap games" win anyway won't do it for me.

Do you believe there really is such a thing as a trap game? OK, prove it. We'll happily accept your guest columns on the subject at mailbag-at-footballoutsiders.com. If you would rather prove that there is such a thing as a "letdown game," we're happy to be convinced of that too.

P.S. Of course, math might not prove the "trap game" concept either. I should point out a comment once made by Danny, who has a masters in Sports Psychology: "The less time people play armchair sport psychologist, the better. Even if the numbers revealed some kind of formula for accurately identifying a trap game in advance, we would still have no basis for saying it's psychological because we know nothing about what anyone at team HQ is thinking."

Posted by: Aaron Schatz on 21 Nov 2012

47 comments, Last at 27 Nov 2012, 9:56pm by tgt2

Comments

1
by Karl Cuba :: Wed, 11/21/2012 - 2:19pm

The esteemed Dr Z had another definition of trap games.

http://sportsillustrated.cnn.com/2008/writers/dr_z/09/04/forecast/index....

13
by RickD :: Wed, 11/21/2012 - 3:55pm

That's a different issue. Dr. Z was talking about betting lines and the "traps" that Vegas sets to suck in foolish gamblers.

18
by Stats are for losers (not verified) :: Wed, 11/21/2012 - 4:17pm

I've never heard the term "trap game" in a non-gambling context prior to today.

19
by RickD :: Wed, 11/21/2012 - 4:30pm

Really?

I've been hearing that term used for years.

Try Google. I just got 1.76 million hits for "NFL trap game definition" and most of them fit the usage we're using here.

See, for example,

http://bleacherreport.com/articles/1190073-the-most-dangerous-trap-games...

25
by jds :: Wed, 11/21/2012 - 5:49pm

And Google also notes, in a few of the articles found "Football Outsiders defines a trap game as ...". So unless this hunt is successful, FO already has an acknowledged definition out there on the web.

26
by Stats are for losers (not verified) :: Wed, 11/21/2012 - 6:02pm

Adding the word "gambling" boosts it to 47 million results.

32
by RickD :: Thu, 11/22/2012 - 1:28am

Adding the word "Giants" boosts it to 46.2 million results.

Adding more search words often adds to the number of results.

37
by Podge (not verified) :: Thu, 11/22/2012 - 7:46am

Adding the words "purple monkey dishwasher" takes it down to 3060 results though. This definitely proves something.

42
by Karl Cuba :: Thu, 11/22/2012 - 12:23pm

Yes, it proves that it's hard to trap a giant, purple monkey in a dishwasher.

46
by tgt2 (not verified) :: Tue, 11/27/2012 - 9:53pm

Or the reverse. Nobody's asking how to do it. It must be easy.

2
by Joseph :: Wed, 11/21/2012 - 2:28pm

I do not have the time/energy/stats to devote to such an article. I leave that to my more single peers that comment regularly here.
However, having said that, I would state that a trap game is 1) only for teams that finish at least 9-7, and should maybe be limited to playoff teams; 2) probably an away game, but not necessarily; 3) against a team that was at least 2 games under .500 at the time of the game, and that finished at no better than .500 (this will get rid of some early week games); 4) comes before a game against a team that at least 2 games over .500 at the time, and that also finished at least 2 games over .500; 5) comes after a TWO+ GAME WINNING STREAK. I don't think you can include divisional games in the study, although I can see how that some people might include the HOU-JAX game from this past Sunday. I just can't see NFL teams overlooking a divisional opponent with whom they tend to have "bad blood", "long-term feud", etc., esp. when their divisional rival is from pre-realignment days.
Let the stats war begin!

4
by Danny Tuccitto :: Wed, 11/21/2012 - 2:47pm

(1) and half of (4) would suffer from retrospective bias. We don't know where they're going to finish at the time the game is played, something you clearly seem cognizant of in (3) and the other half of (4).

Like your prediction of this challenge being dominated by single peers, though. Sounds about right.

44
by Joseph :: Thu, 11/22/2012 - 5:41pm

Well, my point is that nobody thinks this years' Jags, Browns, Chiefs,and Panthers are worried about "trap" games. They are the OPPONENTS in those trap games. In other words, losing teams don't get "trapped" in not preparing for the next opponent. If they don't show up, they get blown out (see the Bucs last year). However, esp. in college football, I can see really good teams thinking that they can play less than their best and still win. Based on recent articles, the lack of tackling from HOU's DB's indicates that those specific players did exactly that.
However, I do agree with the premise that trap games don't exist--but I don't agree with the notion that certain players/units of football teams, esp. those BELOW NFL levels, don't tend to take it easy when they feel like it's not as necessary. I mean, don't most people think Albert Haynesworth failed because of exactly that?

3
by Jon Goldman (not verified) :: Wed, 11/21/2012 - 2:41pm

Not an article, just a thought:

I'd define a "trap" game as one where a team performs at a level below its previously demonstrated variance in performance, while possessing a top-offense/defensive unit, at least as compared to the opposing team: naturally, this means that trap games just don't appear early in the season.

In fact, if I may, I'd actually stretch this definition to include games following previous unsustainable high performances, again determined by looking at variance.

In less technical terms, a game where a team plays down to an opponent on a level below that of their previously lowest performance. It is just a term used to justify bad performances against bad teams.

Predicting them is impossible, as for a game to be a trap game, it, by my definition, must include a team that previously showed no indication that it would lose this game against an inferior team.

Using pure W/L record gives you things that Vegas calls upsets, and sportswriters only call trap games after the fact.

It's a term devoid of predicative value, while somehow simultaneously lacking any reviewable value.

It's totally meaningless except to say that a bad team beat a good team.

47
by tgt2 (not verified) :: Tue, 11/27/2012 - 9:56pm

People talk about trap games ahead of time, so your definition is akin to me defining an apple as orange with a thick rind.

5
by QCIC (not verified) :: Wed, 11/21/2012 - 3:03pm

There is no such thing as a trap game. Problem solved!

41
by jebmak :: Thu, 11/22/2012 - 10:31am

I would argue that a trap game happens when a team losses its swagger.

But other than that, I agree with you.

6
by Dean :: Wed, 11/21/2012 - 3:16pm

Fascenating challenge, Aaron.

I'd love to see "The Only Thing The Prevent Defense Does Is Prevent You From Winning" debunked once and for all as well. It seems like even here on this site, there are regular posters who seem otherwise enlightened yet still subscribe to such nonsense.

Oh, and I might take up your challenege. Not because I believe for one second that you're wrong, but that it would feed my ego to show even more evidence that you are right.

28
by Ryan D. :: Wed, 11/21/2012 - 8:00pm

Ron Rivera's prevent defense prevents him from winning.

39
by Podge (not verified) :: Thu, 11/22/2012 - 7:59am

I think you put the word "prevent" in there after "Rivera" by accident. The sentence makes sense without it.

7
by tuluse :: Wed, 11/21/2012 - 3:21pm

Obviously a trap game is any game where a team didn't perform to lofty expectations that their fans had.

8
by Nick Saikley (not verified) :: Wed, 11/21/2012 - 3:32pm

Lots of fans of any above-.500 team thinks every single game against a sub-.500 opponent is a trap game.

9
by QQ (not verified) :: Wed, 11/21/2012 - 3:34pm

Like many, I for the most do not believe in the idea that trap games exist.

It would be interesting, though, to look beyond W/L and also to margin of victory. It is possible that the idea took hold not because the favored teams were losing more often but more so because the margins of victory were lower than expected.

10
by In_Belichick_We... :: Wed, 11/21/2012 - 3:35pm

How to explicitly define a trap game is going to be difficult.
To me a trap game is a game vs. a poor opponent before a game with great meaning.
For example: If the Pats are playing Houston with a 1st round bye on the line, a weak match up the week before may be considered a trap game.
I believe the argument is that game planning for the cupcake may be cut short to allow extra game planning for the very important game.

11
by Danny Tuccitto :: Wed, 11/21/2012 - 3:45pm

Except, barring seasons where two teams in one conference separate themselves from the pack by -- say -- Week 15, how do we know a week in advance that a 1st-round bye will be on the line? Don't have the time to look through it, but doesn't seem like that two-dominant-teams situation happens very often. Could be wrong though.

Same thing goes for any other playoff positioning, too, I guess. We don't know -- or I should say the teams aren't preparing as if -- the game after next is going to have X playoff implications, what with convoluted tiebreakers and tightly bunched records.

14
by In_Belichick_We... :: Wed, 11/21/2012 - 4:00pm

I'm sure there are games that teams have circled on their schedule. Every game counts but some count more than others.
For example:
Denver is at Oakland on Dec 6 and at Baltimore on Dec 16. The Baltimore-Denver game potentially has first round bye implications. Obviously the Broncos may take themselves out of the running if they lay an egg against the Raiders but who expects the Raiders to win this game?
Would it be a comfortable gamble by the Denver coaching staff to sacrifice some Oakland game planning time for some extra Baltimore game planning time?

16
by Danny Tuccitto :: Wed, 11/21/2012 - 4:13pm

Don't necessarily disagree with you because I have no way of providing evidence to the contrary. A large part of the issue -- at least for me -- is that the theory is predicated on "the Denver coaching staff [sacrificing] some Oakland game planning..." We can never know this definitively without a mole in Denver. If Denver does lose to Oakland, or even plays a subpar game by their own standards, it being the result of half-assed game-planning or a psychological letdown or whatever is just based on our own assumptions/conjectures, not testable objective facts.

36
by Ivarsson.se :: Thu, 11/22/2012 - 4:50am

In this case, since the Oakland game is a thursday night game there isn't much @OAK planning time to sacrifice, and the 10 days up to the @BAL game should suffice anyway.

On the other hand, it could fit nicely in the "away team on short rest penalty category". Both DEN and OAK play a 4pm home game Dec 2nd, leaving almost the exact same rest period, but DEN has to travel an hour or two...

12
by RickD :: Wed, 11/21/2012 - 3:53pm

It's easy to define a trap game.

Team X has a "trap game" when it plays team Y but it's looking forward to playing team Z the following week. If team X falls into the trap, they underperform against team Y, possibly losing a game that they would be expected to win easily.

Do "trap games" exist? Of course they do! See, for example, the Patriots hosting the Cardinals back on Week 2. All the ingredients are there: home game against a lightly regarded opponent which is followed by a rematch of the previous year's AFC championship.

I don't know why you would think that any kind of statistical argument would be necessary to prove the existence of these things. If you prove that favorites do as well in trap games as they do in other games, all you've proven is that you can define the category of trap games! You're debunking a hypothetical claim made about trap games, but you're not denying that they exist.

This definition of a category doesn't depend on any fixed prediction of just how dangerous any particular game is.

In case it's not completely clear by now, you would expect the superior teams to avoid traps the vast majority of the time.

Finally, somebody's going to have to remind me how the Chiefs beat the Packers last year. No, really, did that actually happen?

15
by In_Belichick_We... :: Wed, 11/21/2012 - 4:05pm

I am a believer in trap games but I don't believe that was the problem in the Pats-Cards game.
The Pats had some offensive line injuries and the Cards at the very least have a very good defensive line.
J McD has proven in the past that he can't adjust his offensive play calling when the O-line is getting man-handled by the D-line (see 2008 super bowl).
People cite the helmet catch or the lack of holding calls or the lack of a stop by the Pats D. To me it was J McD unwilling to stop calling 5 and 7 step drops when the O-line was getting man-handled.

20
by RickD :: Wed, 11/21/2012 - 4:34pm

"J McD has proven in the past that he can't adjust his offensive play calling when the O-line is getting man-handled by the D-line (see 2008 super bowl)."

Well, I'll cut McD some slack for that. For one thing, Stephen Neal was lost to the Pats early in the game. And with him out and Kaczur at RT, the Pats were just too weak on that side. There's not much you can do with playcalling when the defensive line is in the backfield after two seconds. Or when they're not in the backfield, they've pushed the O-line back so far that it affects the QB.

Just a bad game for the Pats' O-line.

22
by In_Belichick_We... :: Wed, 11/21/2012 - 4:48pm

A few more screens may have kept the D line at least little honest. Perhaps 3 step drops instead of 5 and 7 step drops?
The Giants D-line was pass rushing the entire game. Heck, I think they even used 3 DEs on the line.

I won't cut J McD any slack. The greatest offense on the planet was held to 17 points.
With all of the complaining about the Pats D, it was the Pats O that lost both super bowls.

27
by PatsFan :: Wed, 11/21/2012 - 6:36pm

A few more screens may have kept the D line at least little honest. Perhaps 3 step drops instead of 5 and 7 step drops?

Especially since that's exactly how NE scored the go ahead touchdown before the Events That Cannot Be Named occurred.

To this day I cannot understand how BB and staff coached that game so badly.

33
by RickD :: Thu, 11/22/2012 - 1:32am

You lost me with your last complaint.

The offense clearly outperformed the defense in Super Bowl XLVI. FO went over the numbers back in February. The defense couldn't get off the field. Even when they forced a punt, the offense got the ball inside the 20 all game long.

"The Giants D-line was pass rushing the entire game."

As opposed to doing what on passing plays?

"Heck, I think they even used 3 DEs on the line." Yes, this was not new for the Super Bowl. The had Strahan and Osi at the ends and Tuck in the middle. That's a very good line.

Sometimes when the other team plays better, that's all it is. That doesn't mean that there was some hidden strategy that would have changed the outcome.

38
by In_Belichick_We... :: Thu, 11/22/2012 - 7:53am

Not "as opposed to doing what on passing plays?" We're talking about every friggin' play of the game. The D-line had no reason to worry about anything else other than pass rushing so pass rush they did, on every single play.
I agree that the Pats D played poorly but when the Pats lose 20-17 or 23-20 then the offense lost the game. I'm not so sure the defense played worse than the offense if you compare to their normal or expected performance.

17
by Clemson Matt (not verified) :: Wed, 11/21/2012 - 4:14pm

I say it should be defined as "Pulling a (pre2012) Clemson"

21
by Aloysius Mephis... :: Wed, 11/21/2012 - 4:37pm

I can't believe there's any confusion about what a trap game is. For anyone that doesn't understand, a "trap game" is simply any game involving any team that appears to have momentum, but which in fact may lack the moxie to come out loaded for bear and make a statement against a scrappy squad that's playing for pride and has something to prove and we all know there's nothing more dangerous than a wounded animal. If a team loses a trap game then it raises a lot of questions about whether the team has the character to come through in the clutch and show that it's a contender and not a pretender. A team that loses a trap game may also have to do some soul searching and ask itself whether it can develop the guttiness that is the hallmark of teams that have the mental toughness to 'just win' come playoff time. In conclusion, swagger. You're welcome.

29
by Jerry :: Wed, 11/21/2012 - 10:14pm

This post outkicked its coverage.

30
by Jon Goldman (not verified) :: Wed, 11/21/2012 - 10:51pm

Skip, get out of here.

23
by zenbitz :: Wed, 11/21/2012 - 5:33pm

A trap game is clearly a sub class of upset. It is a game you expect good team A to win but, it does not for [insert psychological reason]

Upset, obviously can be defined phenomenologically by either Vegas point spread, or mathematically by delta SRS or DVOA.

I think we have to assume that a "trapped" team can actually win a trap game; that is designating an upset a "trap game" post-facto seems useless.

That means we have to actually calculate underperformance as expected - observed. This obviously has a mean of 0ish (assuming vegas spreads and/or DVOA are accurate) and a standard deviation when calculated over ALL games.

Now we can define potential trap games as:
A subset of Upset games where favored team underperforms against the (Vegas and/or DVOA) spread.

Of course arbitrary subsets must exist - that's why these are just "potential" trap games.

One could take the full set of games with underperfoming favorites and start slicing the data based on presumed psychologically forward-looking trapping slices.

For example - if game N+1 is "important" or "tough" (for some mathematical definition).

In a similar way you could look for "letdown" games based on characteristics of game N-1 (rest time, big comeback, big upset, road win).

You will of course find "evidence" of correlation between some selected criteria X and underperformance by favorite - so you are not done. Once you have a correlation you are looking for, you check ALL games wit that criteria X and calculate the statistical P-value that criteria is significant in showing underperformance.

However, don't forget to multiply your P-values (Bonniferri correction) by the number of different hypotheses (Xis) you check.

What do I win?

24
by In_Belichick_We... :: Wed, 11/21/2012 - 5:40pm

Inner peace?

31
by Jon Goldman (not verified) :: Wed, 11/21/2012 - 10:52pm

Internet fame, and possibly the right to state that you defined a subjective term subjectively.

34
by RickD :: Thu, 11/22/2012 - 1:35am

Your entire argument fails because you misspelled one word.

I'm not going to tell you which.

Life is cruel.

35
by zenbitz :: Thu, 11/22/2012 - 2:19am

only 1? I call victory for opponent adjustment.

40
by Podge (not verified) :: Thu, 11/22/2012 - 8:06am

I'd say that the most common implied definition of a "trap game" that I've come across is would be a game against a crappy team in the week after a good win against a good team. The thought behind that definition of a trap game to me has always been that a team assumes "Oh the (lets just say Jags) are rubbish, and we were brilliant when we battered the (lets say Pats) last week, so we only really need to turn up and we'll win".

I don't really see the "looking past the Jags" argument as much. Although maybe these are two different types of traps? An upcoming game agaisnt a good opponent, and coming off a good win against a good opponent?

Maybe this could be linked to Guts, Stomps, Skates and Dominations? If you had a list of every game that's classed as a DOMINATION (win by over 14 points against a winning team), and then checked every subsequent game that was against a losing team, and see what percentage are STOMPS, SKATES or losses?

That feels like quite a quick (relatively speaking, because it would be a horror of data mining) and dirty analysis though. I think a better analysis would be on DVOA in those games, compared to either the previous game or the season average. I'm assuming that the Texans, although they won on Sunday, would have significantly lower DVOA than their season average for that game (probably below 0).

Basically, the hypothesis would be that, following Team A winning in a DOMINATION, if A's DVOA in the next game is (significantly?) lower than their season average DVOA (or maybe lower than their opponents?), it could be considered to be a trap, and if not, its not a trap.

All that being said, I've always thought a trap game was a game against a team with a win/loss record that doesn't reflect how good they actually are.

43
by Aaron Schatz :: Thu, 11/22/2012 - 3:31pm

Here is your first attempt at proving the existence of a "trap game" in response to this post, from the Harvard Sports Analytics Collective. They also decide it is a myth.

http://harvardsportsanalysis.wordpress.com/2012/11/22/debunking-the-trap...

45
by bengt (not verified) :: Fri, 11/23/2012 - 12:04pm

My proposal for a definition of 'trap game':

Any game were one can google an article/tweet/brainwave pattern calling it a trap game, demonstrably written prior to the game in question, and at least as coherent as a typical RaiderJoe comment.