PFP 2007: Trap Games
by Alex Carnevale
On this week's B.S. Report podcast with Bill Simmons, I mentioned an article from Pro Football Prospectus 2007 which showed that good teams don't play any worse in "trap games" than they do in games overall. For those curious who don't actually own a copy of PFP 2007, here's the article in its entirety. It's unedited, which means all the numbers in here only represent games through the 2006 season. -- Aaron Schatz
Last year Andy Reid was spared the indignity of facing the Minnesota Vikings in Brad Childress' debut season as Vikings coach, but he'll go head-to-head with his former offensive coordinator in 2007 when the Eagles travel to Minnesota in Week 8. Making this visit to the Minneapolis Metrodome to face a team that may be worse than its 6-10 record of last year is hardly the equalivalent of playing the Patriots in a freezing Foxboro Stadium, which the Eagles will do a month later. Still, it comes between two tougher home games: a matchup with the defending NFC champion Chicago Bears, and a divisional game against the Dallas Cowboys.
When Week 8 comes around, talking heads and odds-makers alike will identify this as a "trap game." The theory behind the trap game is that it's hard to stay on an emotional high game after game, so teams are prone to let up against lesser opponents when they're facing them between games against higher quality opponents. The trap game concept is not unique to the NFL -- in fact, the idea makes even more sense in basketball, where there isn't a week of preparation between each game -- and it makes a certain amount of sense. But does the trap game phenomenon really exist?
A quick statistical reality check should show us if NFL teams should really be worried about trap games. Giving this supposition as wide a berth as possible, we'll define a trap game as any game against a sub-.500 opponent slotted between two games against opponents who, on the season, posted records above .500. Going by these criteria, there have been 474 trap games since 1983. Since we're only interested in how good teams deal with this problem, we'll focus on how teams that finished the season over .500 performed in these games.
It turns out that good teams win trap games just as often as they win other games. Contending teams went 389-85 overall in trap games, good for a .820 winning percentage (Table 1). In all other games against sub-.500 teams, the same contenders posted a .815 winning percentage (Table 2), meaning they were actually more likely to win trap games than other games.
While these contenders were far more likely to win a supposed trap game at home than on the road, their home-road splits again largely mirrored their performance in non-trap games against sub-.500 teams. As you can see from Tables 1 and 2, the additional pressure (or lack thereof) of the trap didn't mean much. Despite the hype, the overall effect is nil. It's possible that the existence of the myth itself has lessened the impact of a trap game.
|Table 1: Winning Teams in Trap Games 1983-2006|
|All Traps||Home Traps||Road Traps|
|Table 2: Winning Teams Non-Trap Games against Sub-.500 Teams, 1983-2006|
|All Games||Home Games||Road Games|
Besides the game against the Vikings, the Eagles have a textbook trap game earlier in the season, hosting the Lions on September 23. That game falls in between two nationally televised games against division opponents; the Eagles host the Redskins on Monday Night Football in Week 2 and play the Giants at the Meadowlands on Sunday Night Football two weeks later. Conventional wisdom suggests this is a trap game of tremendous proportions. Not only are the games that surround it crucial intra-division contests, both of which will be broadcast on national television, but Philly will have a shortened week heading into the game against the Lions.
Are divisional trap games different from trap games in general? Since 1983, winning teams are 53-13 in trap games sandwiched between two divisional opponents. That's a slightly lower percentage (.803) than their record in trap games overall, but good teams are still winning 80 percent of these games.
While we found that success in trap games hasn't been more difficult to come by overall, for some coaches it has. One big name in particular hasn't been particularly sharp in these situations, including a trap game loss to the Lions late last year.
- Bill Parcells: 20-6 (.769)
- Herman Edwards: 5-2 (.714)
- Mike Tice: 2-3 (.400)
Others have performed better in trap games:
- Bill Belichick: 17-0 (1.000)
- Bill Cowher: 12-1 (.932)
- Marty Schottenheimer: 18-2 (.900)
- Andy Reid: 4-0 (1.000)
Andy Reid's 4-0 record in trap games is actually part of a larger streak. The Eagles have won 20 straight trap games since 1991.
The Eagles play four prime-time games in 2007: three against division rivals, plus a trip to New England. Their games against cupcakes such as Minnesota and Detroit won't be any harder just because they come in between.
Just to update a couple of those numbers, Belichick is now 21-0 in trap games. The loss to Cleveland in Week 9 doesn't count as a "trap game" by the standards of this article because it didn't come between two games against winning teams; the Patriots played Minnesota the week before. If you want to define that as a trap game, you now have to measure every game against a losing team that comes before any game against a winning team.
Philadelphia's streak of winning trap games ended at 23 when the Eagles and Bengals tied in Week 11 of 2008, but if we base the definition of a 2010 "trap game" on the current standings, Andy Reid is now 9-0-1 in trap games, including wins over Detroit and Washington this season. -- Aaron Schatz
12 comments, Last at 25 Nov 2010, 2:00am
#1 by ammek // Nov 23, 2010 - 2:50pm
Yikes. How will the media explain that Cleveland win now? Swagger? The curse of the non-handshake?
PFR did a series a while ago about when upsets occur. Their conclusion was that conditions (dome/outdoor) and geography (the old playing across time zones myth) were far less significant than simple familiarity. In other words, games are generally closer when the teams know each other well, and that helps the inferior team.
Familiarity, with its connotations of warmth and intimacy, seems a strange word to describe the Mangini-Belichick relationship. But I wonder if coaching against an opponent from the same tree, running a similar scheme, reduces the gap too.
Of course, a lot of the fuss is that the world outside FO doesn't make schedule adjustments and sees the word 'Cleveland' and thinks, oh that's a crappy team. It isn't.
#2 by Jeff Fogle // Nov 23, 2010 - 3:16pm
Regarding Andy Reid...so...going on the road to face a non-conference foe as a huge favorite immediately after two blowout wins ISN'T a trap game...even when it comes just before three straight divisional rivals (referring to your update of Reid's record because this was last year)?
Philly beats KC and TB 67-28
Philly travels cross country to Oakland
Philly has Washington, NYG, Dallas on deck
Philly (-14) is NOT a trap game at Oakland? This may be the single best example of a trap game in the history of the sport! Your definition of trap doesn't include it.
Philly would lose 13-9, failing to even score the spread.
The Eagles would then win two of those three games against divisional foes...but have to travel cross country AGAIN to face the Chargers. They lost 31-23. Hard to call it a "trap" game because it came after a loss and good teams don't lay down after a loss. Tough schedule spot though.
Philly would have another trap possibility near the end of the season.
5 game Win Streak
Home vs. losing Denver of the AFC
Next week at Dallas in a HUGE game
Philly was (-7) vs. Denver, and only won 30-27...hurt by three turnovers.
But, that WASN'T a trap game because the last win of the five game winning streak was against San Francisco? Yet...this year's game at Detroit WAS a trap because it came between Green Bay (a game they lost) and Jacksonville (non-conference road game)? Who gets trapped off a Week One loss? Who gets trapped before playing Jacksonville?
There are exceptions of course, but the oddsmaking field believes teams generally prioritize games this way:
Divisional games (because winning the division gets you into the playoffs)
Conference games (for Wildcard/tiebreaker reasons)
There's a home and road dynamic too obviously. Teams are more likely to blow off a road game than a home game. And, there are some rivalries that transcend divisions (like Indy/NE). Plus...big TV games vs. potential playoff opponents can loom large in prioritization as well. Wish there was an easy rule...but teams prioritize on the fly sometimes.
"Traps" as most of the world sees them generally involve the prioritization dynamic. I agree completely with Simmons that New England at Cleveland had "trap" written all over it
5 game winning streak
Road game at non-divisional foe
High profile TV games on deck
I'll grant it's borderline because Pittsburgh and Indy aren't the Jets or Dolphins. But, we're talking national games vs. potential playoff opponents on NBC and CBS.
A definition of "trap" that's based only on winning or losing records of the opponent misses the boat by quite a bit. It excludes way too many obvious trap nominees...and includes games that are OBVIOUSLY not remotely trap games. (And, if it's done at the end of the year, it may exclude teams who ended with a losing record because they failed to win their trap games...so you're selecting teams who avoided traps to suggest there weren't any traps...and let's add in not adjusting for expectations...so that Philly would get credit for a win over Denver in my example instead of being penalized for not matching expectations).
If we're going to determine the impact...or prove there's a lack of impact...the definition should be tied closer to the reality of the perceived dynamic. To come out and say there's an article that shows "there was no such thing as a trap game," and then go on a national podcast and assert the same point...is mind-bogglingly off the mark if that's the only evidence you have for the assertion. Stunning.
#3 by JonFromTX // Nov 23, 2010 - 3:44pm
Alright dude, why don't you go ahead and run those numbers for us under your definition of a trap game? It would be interesting to see how they come out in comparison to the article's definition of a trap game.
#4 by DavidL // Nov 23, 2010 - 3:59pm
This just makes me see the idea of a "trap game" as more of a fallacy. Take Philadelphia/Oakland. The Eagles blew out two bad teams and then lost to a third bad one. What would you say the odds are for a good team to beat a bad one on any given Sunday? Three in four? Four in five? Even then it's only barely better than 50-50 for a team to win three of those games in a row. But because the loss came at the end of the sequence it's a "trap game" instead of a routine off-day loss.
As for the Denver game....it's a seven-point spread. That's not exactly a heavy favorite.
It doesn't surprise me to see that despite some very memorable upsets in the "trap game" scenario, there are a lot more unmemorable wins where the superior team takes care of business. Just because we remember the upsets better doesn't mean they happen more often.
#5 by B // Nov 23, 2010 - 5:18pm
This thread needs more Admiral Ackbar pictures.
#6 by Jeff Fogle // Nov 23, 2010 - 5:55pm
"It doesn't surprise me to see that despite some very memorable upsets in the "trap game" scenario, there are a lot more unmemorable wins where the superior team takes care of business. Just because we remember the upsets better doesn't mean they happen more often."
Agree with the general sentiment here completely...that it's dangerous waiting until after games are over to call something a trap game (though, it was not uncommon at all to hear people calling Philly at Oakland a trap game before it started). My issues with the article, the conclusions drawn, and the repetition of those conclusions are:
*The definition hit partial elements of the phenomenon...but missed out on what are arguably the KEY elements of the phenomenon in terms of how NFL teams prioritize their games.
*The definition was an all or nothing "did they win" rather than a "did they play worse than normal" which could have been attempted with game DVOA or Las Vegas ATS records. A pothole that jolts you but doesn't wreck the car is still a pothole. You wouldn't be confident enough to say "There's no such thing as potholes" when you were only looking for wrecks.
*The study excluded possible counter-examples. An 8-8 team that fell into traps (if they exist) once or twice (or more) would be excluded. A 7-9 team that fell into traps twice would be excluded. If you limit the study to teams who dodged the bullets, you're likely to draw the conclusion that there weren't any bullets.
Editors should have dealt with these issues before the article was even published. That's not a research article with any merit. There's a bad definition that self selects after the fact for the teams that managed to survive their trips...but it doesn't check for damage or bleeding during the potential trouble spots as they happened. You can't say "there's no such thing as trap games" after a study like that. It's barely trying to find them or the impact they can have on teams that doesn't quite meet the threshold of a black or white win or loss.
Now, what's a better definition? There are so many potential influences that we're not likely to ever agree on a pure definition (or this would have been universally agreed upon many years ago). I like the phrase "trap scenarios" from the post above. Let's make it "potential trap scenarios" and see if there are broader characteristics in addition to the ones I outlined above regarding division/conference/non conference, home/road, and whether or not the team won or lost the prior game (few quality teams play flat after a loss...and it's not common to hear fans/handicappers/anybody talking about a potential trap spot when the team just lost the last game. It's more connected to a "letdown" off a success or a string of successes, and/or a lookahead to something big the next week).
Does anybody want to add potential scenarios that might make sense? Once those are in place, we can find somebody with a database to look back at qualifiers with an eye toward game-DVOA or ATS results rather than just pure win/loss.
Basic fundamental of science and the Bill James approach. If you're going to say something doesn't exist...you REALLY need to try and find it wherever it might be before asserting confidently that it doesn't exist. This article didn't make a good faith effort to define traps in the way football observers have traditionally labelled them (limiting it to won-lost records rather than other factors that get the juices flowing or don't), excluded logical teams who may fallen into traps (those finishing 7-9 and 8-8)and used an all-or-nothing assessment in an area where shades of gray in performance are definitely relevant.
Even if you disagree with the Eagles/Raiders example (which wasn't self-selected after the fact...it's just that we're having the discussion now so that's when I'm mentioning a game that was widely recognized as a flat spot at the time last year), does anyone want to make the case that FO is correct in asserting that there's no such thing as a trap game based on this study?
#7 by InTheBoilerRoom // Nov 23, 2010 - 6:04pm
Well, we will have another good example when the Patriots play the Lions on Thursday. They're facing the Lions on a short week. They're coming off of a big win over the Colts. They have a MNF game against the Jets the following week. Classic trap game? I haven't heard it described as one yet. But I suppose it will only be a trap game if they lose it. If they win, it's just another game. Yay for selection bias! Let's see what happens after the game.
Edit: Gah, and right after writing that I start listening to Simmons' podcast and he calls it a trap game.
#8 by Alexander // Nov 23, 2010 - 10:11pm
Captain Hindsight is an expert at spotting trap games.
#12 by TomE (not verified) // Nov 25, 2010 - 2:00am
Exactly right. "Trap games" are like teams/players/coaches that "can't win the big one": if they win what might have been a "big one," it is no longer a "big one." If they lose, it was. "Trap games" are only trap games when they are lost. It makes things much easier than actually using data.
#9 by Pat Swinnegan // Nov 24, 2010 - 12:27am
I agree with Mr. Fogel. I listened to the B.S. Report podcast on the way to work this morning, and I was then keen on seeing this article. But now I find it pretty disappointing. As Mr. Fogel points out, the definition of trap game is much too broad. By this definition, something like one in every 5 games is a trap game!
I'd be more curious about games where, say, a team is coming off a win against a team that was 0.7 or better, about to face a team that is 0.3 or worse, just before another game against another team that is 0.7 or better. That might do a bit better of a job of capturing the idea of a "trap game" -- at the very least, if there's no significant difference there, THEN I'd be cool with you going on podcasts and claiming, "We've done the research. There's no such thing."
meaning they were actually more likely to win trap games than other games.
Statements like this just make the article seem amateurish. (Back of the envelope, I get a p-value of like 0.45 for the above assertion.) I like to think Football Outsiders has matured a bit since 2007.
Anyway, I do love the juxtaposition of Bill Simmons and Aaron Schatz. Two Pats fans, with a totally different philosophy of how to view the game. They make a great complement to one another... I wish I could hear them break down the Pats game every week.
#10 by Kevin from Philly // Nov 24, 2010 - 8:15am
I wonder what would happen if you analyzed against the spread. But then, I guess you'd have to examine whether the spread was set too high by Vegas. Ah, never mind - happy Thanksgiving everybody.
BTW, I'm doing a Delaware parlay with the three favorites on Thursday, though I'm leery about the Jets.
#11 by Zack // Nov 24, 2010 - 1:03pm
Yeah, I think the theory applies against the spread, but this is because the line is usually inflated since the favorite is coming off a big win (for it to be a trap game).