Writers of Pro Football Prospectus 2008

24 Sep 2013

Any Given Sunday: Colts Over 49ers

by Tom Gower

While there was nothing pretty about San Francisco's performance in Week 2 at Seattle, very few fans thought there was much cause for immediate concern. After all, the 49ers had opened the season with a win against a Green Bay squad both our projections and just about everyone else thought would be one of the best in the NFC. The 49ers just had to fly home, lick their wounds, and throttle the Colts, a possibility that did not seem too difficult.

After all, Indianapolis had started the year with close home games against the Dolphins and the likely-woeful Raiders. Their run defense, league-worst by our numbers in 2012, came in ranked 30th and seemed like the cure for a San Francisco run game that had struggled to get going in 2013.

San Francisco's matchup edge seemed further strengthened by Indianapolis' offensive line injuries. Right guard Donald Thomas went to injured reserve last week and starting center Samson Satele missed the game as well. The actual starting interior line for Indianapolis on Sunday was rookie third-round pick Hugh Thornton next to 2012 All-Keep Chopping Wood team contenders Mike McGlynn at center and Jeff Linkenbach at right guard. This seemed like a major matchup advantage for a 49ers front seven that still seems like one of the best in the league.

The story of this game was supposed to be that San Francisco would dominate both lines of scrimmage, throttle the Colts on offense, and walk away with a comfortable win. But the Colts controlled the ball for over 36 minutes in a 27-7 victory, including a 7:01 drive in the fourth quarter that began with 11:14 to play and extended their lead to 20-7. So instead, the story of the game is instead how Colts offensive coordinator Pep Hamilton's preferred power run game helped the Colts control the line of scrimmage and walk away with the win.

By our numbers, the story of the game is somewhat more complicated than that.

By the VOA

The Colts were in fact the better team, but not for a reason anyone I have seen any national outlet discuss.

Some Very Special Teams
Team Off. VOA Def. VOA Special Teams VOA Total VOA
IND 26.5% -5.8% 9.4% 41.7%
SF -0.1% 16.8% -4.0% -20.9%

These are the VOA numbers, not adjusted for quality of opponent, but this is a non-intuitive result. The Colts ran 14 more offensive plays than the 49ers did. The 49ers punted on seven of their eight possessions until the Colts took that two-score lead with just over four minutes to play, including four three-and-outs. Heck, it was even Adam Vinatieri, not Phil Dawson, who missed a field goal in the game. Why are our numbers telling such a counter-intuitive story of what happened?

The Real MVP, and Another Story of What Happened

Here is one attempt at an explanation:

1. P.McAfee kicks 72 yards from IND 35 to SF -7. K.Hunter to SF 13 for 20 yards (M.Harvey).
2. P.McAfee punts 52 yards to SF 9, Center-M.Overton, downed by IND-M.Harvey.
3. P.McAfee punts 51 yards to SF 36, Center-M.Overton. K.Williams to 50 for 14 yards (C.Johnson).
4. P.McAfee punts 46 yards to SF 8, Center-M.Overton, fair catch by K.Williams.
5. P.McAfee kicks 65 yards from IND 35 to end zone, Touchback.
6. P.McAfee punts 43 yards to SF 8, Center-M.Overton, fair catch by K.Williams.
7. P.McAfee kicks 65 yards from IND 35 to end zone, Touchback.
8. P.McAfee kicks 65 yards from IND 35 to end zone, Touchback
9. P.McAfee kicks 63 yards from IND 35 to SF 2. K.Hunter to SF 11 for 9 yards (G.Whalen).
10. P.McAfee kicks 60 yards from IND 35 to SF 5. Q.Patton to SF 12 for 7 yards (S.Havili).

Outside of the punt where Kyle Williams got a return off and the drive (not listed above) that started at their 41 after Vinatieri's field goal miss, Pat McAfee's superb performance meant the 49ers had to go at least 80 yards to the end zone every time they had the ball on offense. Even modestly successful drives ended short of field goal territory, unlike those that start from more advanced field position. Both the Colts and 49ers had drives that gained 35 yards on offense. San Francisco's began at their own 20 and ended with a punt. Indianapolis' began at their own 45 after the 49ers punted from deep in their own territory and ended with a field goal.

One other factor that has not been disseminated in most major stories of the game is penalties. San Francisco was flagged six times, five on possessions that ended in Colts scores. They included two drive-extending defensive holding penalties on third-down incompletions, pass interference on second-and-9, and an unnecessary roughness call that led Donte Whitner to declare that any big hit is now a foul. Whitner had a point, as his hit was not as bad as it looked. The other calls appeared more legitimate. As we have written many times at Football Outsiders, though, defensive penalties like holding and pass interference are generally the result of a healthy level of aggressiveness rather than inherent signs that something is wrong. No call from Sunday's game exemplifies this more than the third-down defensive holding penalty that extended the long fourth-quarter drive: A very close call announced on no specific player. Without it, the 49ers get the ball back with over seven minutes to play down one score instead of with four minutes to play down two scores. The 49ers defense seemed unlucky more than bad on Sunday.

Offensively, the 49ers actually ran the ball effectively, posting a 14.4% VOA. Frank Gore gained 82 yards on 11 carries, seven of them successful. Kendall Hunter was less effective outside of his touchdown run, but the 49ers did manage to run the ball. It was the passing game where San Francisco struggled for the second week in a row, and where the most serious problems appear to be.

This is no surprise, as with Vernon Davis joining Michael Crabtree and Mario Manningham not in uniform, San Francisco was missing three of their top four targets in the passing game. While Colin Kaepernick has tremendous potential, Sunday was only his thirteenth NFL start and he remains a work in progress as a top-level quarterback. Without the benefit of the all-22 (not yet posted as I write this), it is impossible to tell if his apparent uncertainty in the pocket was a result of receivers like Kyle Williams (two catches on six targets for 12 yards) struggling to find the soft spots in zone coverage or Kaepernick struggling to identify them.

Whichever the cause, the result was the same: San Francisco did not make enough plays in the passing game. Davis was reportedly a true game-time decision on Sunday and should return for Thursday night's contest against St. Louis. That will help, as should facing a St. Louis secondary that has allowed Matt Ryan and Tony Romo to finish fifth in DYAR among quarterbacks each of the past two weeks. Unless and until both Manningham and, particularly, Crabtree return healthy, San Francisco's passing game seems likely to hover close to average, fluctuating between strong and weak performances. That means they need to build on the strength they showed in the run game against the Colts. The return of Davis and his excellent blocking should help here as well.

Three Other Things

1. After Andrew Luck scored on a scramble with 4:13 to play to give the Colts a 19-7 lead, head coach Chuck Pagano elected to kick the extra point rather than going for two for a 14-point lead. It is difficult to overstate just how bad a decision this was. Interpolating Football Commentary's go-for-it chart, the Colts should have gone for two if they had as little as a six percent chance of converting. Obviously the error did not end up costing the Colts anything, but bad process is still bad process.

2. A full exegesis of the Trent Richardson trade is beyond the scope of this column, particularly given his limited impact in the game (13 carries, 35 yards, a touchdown on a second-and-goal from the 1 carry). He had 2 YAR on a DVOA of -3.9% (remember 0% DVOA is average, while the YAR baseline is the lower replacement level). Ahmad Bradshaw looked like the more effective Colts back.

3. Aldon Smith obviously played on Sunday, but both his absence and the groin injury Patrick Willis sustained in the third quarter, one of currently undisclosed and perhaps unknown severity, are serious concerns going forward. My general take on Smith is that, as J.J. Cooper broke down during last year's postseason, so many of his sacks last year came off combination play that his production will be easier to replace than that of a normal 19.5-sack rusher unless Justin Smith has hit the wall at age 33 (34 come Monday). This is, however, the first time in Jim Harbaugh's tenure as head coach the 49ers have faced multiple significant injuries on defense, as they ranked in the top two in Defensive AGL the past two seasons. We will now find how just how much quality development San Francisco has managed behind the scenes. This has been a hallmark of defenses like the Steelers and Bears that have stayed above average for many years and we simply do not know how the 49ers will respond yet.

Posted by: Tom Gower on 24 Sep 2013

57 comments, Last at 28 Dec 2013, 12:07pm by Tony

Comments

1
by JMM* (not verified) :: Tue, 09/24/2013 - 1:43pm

Robo-Punter!?

9
by Ben :: Tue, 09/24/2013 - 3:18pm

Boomstick!

2
by Perfundle :: Tue, 09/24/2013 - 1:55pm

Huh, the percentages regarding going for two with a five point lead are somewhat surprising. Seattle had that decision to make when they went up by 5 with 10 minutes remaining against Carolina, with interpolates to the 2-point conversion needing only a 15% of success to go for it. The downside with a miss is the possibility of two field goals beating them, but I guess the touchdown lead opportunity must far outweigh that, even when there is half a game to be played.

12
by Kevin from Philly :: Tue, 09/24/2013 - 3:40pm

Of course, with only a few minutes left, the odds of a team down by five getting two FG drives together must be a lot smaller than getting one TD drive completed - if only because the fan base would burn down the stadium if their team settled for three when down by five.

3
by Bobman :: Tue, 09/24/2013 - 2:22pm

"After Andrew Luck scored on a scramble with 4:13 to play to give the Colts a 19-7 lead, head coach Chuck Pagano elected to kick the extra point rather than going for two for a 14-point lead. It is difficult to overstate just how bad a decision this was. Interpolating Football Commentary's go-for-it chart, the Colts should have gone for two if they had as little as a six percent chance of converting."

When and who will learn to actually THINK about this stuff in the NFL? I suppose if they are trying to return to the days of ground and pound, the Colts FO has bigger fish to fry and bigger intellectual hurdles to clear, but this mistake may come back to hurt them some day....

5
by Ryan :: Tue, 09/24/2013 - 2:42pm

Speaking as a Colts fan, I wonder if everyone on the sideline reacted to the Luck touchdown the way I did, which was "holy $#!% we are up 12 on the Niners in San Francisco what in the world is going on is the moon going to crash into the earth."

Of course, these guys get paid to be a bit more level-headed than that. But I do suspect that might have been the case. Pep Hamilton should have put a bug in Pagano's ear here.

6
by DJG (not verified) :: Tue, 09/24/2013 - 2:49pm

With the plethora of coaches teams employee, and franchises constantly looking for those little extra advantages, it's so amazing to me that the concept of a "clock coach" hasn't caught on yet. Somebody with the sole responsibility of managing the clock, determining when to challenge, when to go for two, etc.

Head coaches have shown they can't handle this responsibility, and, really, why should they have to? Turn it over to a football nerd, and watch the extra wins slowly trickle in over the course of a sufficiently large sample of games.

7
by Ryan :: Tue, 09/24/2013 - 2:51pm

Paging Jim Tressel!!

14
by sundown (not verified) :: Tue, 09/24/2013 - 3:50pm

Problem is extra wins slowly trickling in wouldn't save a guy's job with just one or two high-profile instances where the strategy backfired.

17
by BengalFaninIN :: Tue, 09/24/2013 - 5:11pm

Most NFL coaches appear to coach as if looking stupid and losing is worse than just losing. It does appear that losing gets you fired, and looking stupid and losing gets you fired even sooner. Doing anything that looks questionable can result in the fans, and then the media, and then the owner assigning blame to you, the coach, and deciding that you look "stupid" and firing you. Rightly or wrongly coaches appear to make many of their decisions based on avoiding anything that looks "stupid".

19
by Andrew Potter :: Tue, 09/24/2013 - 5:26pm

You'd hope that the kind of owner who lets a coach hire his own specialist strategy guru is the kind of owner who understands that bigger picture. There aren't many such owners, but I'm pretty sure guys like Kraft and Khan would qualify. If there's one thing Gus Bradley's unlikely to be fired for, it's being aggressive when the numbers back his decision.

My issue with the suggestion of employing a game strategy (four down, two point, etc) specialist is that to use these strategies to best effect, you aren't just looking at an isolated "do we go for it here?" question. If you're employing four-down offense consistently, that should affect your gameplanning, preparation, and playcalling throughout the down. If you're getting to fourth down -then- deciding whether you're going for it, in my opinion that's still bad process. The same is true of two-point conversions. In fact, I'd say that the ideal scenario is the math guy determining when -not- to go for it -- when the odds of converting are low enough that the field position gain from punting is worth it as happened with the Cowboys in Week 2. To achieve that sort of turnaround in thinking needs an approach that basically says "past this yard line, we're in four-down territory all the way" -- that's only going to come from the head coach, not a math guy on the sidelines making a few calls a game.

Clock management and time out use, again, should affect playcalling and therefore is difficult to leave in the hands of somebody who isn't involved in that part of the game. It's not always as simple as "we need to stop the clock ahead of the two minute warning, should we do that with a spike or use a time out?"

4
by Bobman :: Tue, 09/24/2013 - 2:33pm

No idea how Richardson will work out for Colts, but it's safe to assume that he'll do better once he has had more than two practices with the team. He was 0 for 3 on passes to him, which could well be because he was not sure they were coming, or he was in the wrong spot, or whatever. He caught 51 last year, so it's not necessarily his hands.

Bradshaw is a fine, proven runner, but he also has injury issues. If he NEVER gets injured in a Colts uniform, then the Richardson trade was not needed, but that still doesn't make it foolish. Odds are that Bradshaw will miss some time and Indy needed someone other than Dammit Donald Brown. There may have been higher value options (not costing a 1st round pick), but let the T-Rich story play out a bit first. Bradshaw might go down next week and even if T-Rich puts up "average" numbers, but a decent run game enables the Colts to go a game or two deep into the post-season, it will probably have been a good trade. A lot is up in the air for now.

8
by Revenge of the NURBS (not verified) :: Tue, 09/24/2013 - 3:06pm

Random thoughts:
1) Not sure what's non-intuitive about the table. It says the Colts played better in all phases of the game, which is exactly what happened. I gather from the context that it has to do with special teams. But having watched the game, it was very apparent after a while that SF was ALWAYS starting from deep in their own territory, even on kickoffs.

2) Boomstick!

3) Agreed on the 2-pointer. I was saying the same thing at the time. The odds that SF would score a TD and 2 FGs in 4 minutes (the only scenario where kicking the extra point makes any difference) were incredibly small.

4) Richardson joined the team on Thursday and played on Sunday. He obviously didn't know the playbook and openly stated that he was as physically tired as he'd ever been before a game. Hard to draw much of a conclusion from this performance. If anything, the fact that he was right around 0 YAR and DVOA could be taken as encouraging under the circumstances.

10
by Perfundle :: Tue, 09/24/2013 - 3:27pm

It's non-intuitive in that it wasn't SF's offense that was the biggest problem, as every single media site has it. It was their defense, which kept allowing long drives without being able to get off the field. Indy's offense quite resembled Seattle's in the first half of last season, with a heavy reliance on the run game and efficient but not spectacular play from the QB. DVOA really likes that kind of offense, despite the low yards per play it typically generates.

As for the 2-pointer, that's not the most likely scenario. Far more plausible would be two SF touchdowns sandwiched around a field goal by Indy (say, a three-and-out following a short field from a failed onside kick). Then making the extra point means that SF would need two straight two-point conversions after their touchdowns, while failing the conversion means that SF only needs one.

11
by greybeard :: Tue, 09/24/2013 - 3:32pm

"The Real MVP, and Another Story of What Happened
Here is one attempt at an explanation:"
Except for it does not explain anything.
This author's special ability of seeing hidden information that others cannot see, well, makes him look foolish. Let us see how the starting position affected the 49ers:

1. P.McAfee kicks 72 yards from IND 35 to SF -7. K.Hunter to SF 13 for 20 yards (M.Harvey).
3 and out with 9 yards gain. Would not have mattered if they had better field position.

2. P.McAfee punts 52 yards to SF 9, Center-M.Overton, downed by IND-M.Harvey.
TD. Would not have mattered if they had better field position.

3. P.McAfee punts 51 yards to SF 36, Center-M.Overton. K.Williams to 50 for 14 yards (C.Johnson).
37 net yards on punt is nothing to right home about. Especially given that it was not a short field. Regardless. 3 and out with 6 yards gain. Would not have mattered if they had better field position.

4. P.McAfee punts 46 yards to SF 8, Center-M.Overton, fair catch by K.Williams.
3 and out with 8 yards gain. Would not have mattered if they had better field position.

5. P.McAfee kicks 65 yards from IND 35 to end zone, Touchback.
This kickoff and at 20 yards. So no disadvantage here. Still 26 yards and punt.

6. P.McAfee punts 43 yards to SF 8, Center-M.Overton, fair catch by K.Williams.
End of half. Would not have mattered if they had better field position.

7. P.McAfee kicks 65 yards from IND 35 to end zone, Touchback.
This kickoff and at 20 yards. So no disadvantage here. Still 19 yards and punt.

8. P.McAfee kicks 65 yards from IND 35 to end zone, Touchback
This kickoff and at 20 yards. So no disadvantage here. Still 35 yards and punt.

HELLO, WHERE IS THE MISSED FIELD GOAL AND SF GETTING THE BALL AT 41 YARDS?
Which is followed by 3 and out for -3 yards. Would not have mattered if they had better field position.

9. P.McAfee kicks 63 yards from IND 35 to SF 2. K.Hunter to SF 11 for 9 yards (G.Whalen).
3 plays for -3 yards and fumble. Would not have mattered if they had better field position.

10. P.McAfee kicks 60 yards from IND 35 to SF 5. Q.Patton to SF 12 for 7 yards (S.Havili).
After Kickoff. If they had not returned the ball they would be at 20 yards instead of 12. Garbage time. 49ers went for 67 yards. Ended in interception. Would not have mattered if they started at 20 yards. Or give them a TD. Would not have mattered.

49ers offense was so terrible that the field position did not matter.
Indy scored 2 TDs by going 80 yards on each drive. A field goal by going 52 yards, another by going 30 yards and a missed one after going 59 yards. They did not score because of the good field position. They scored because they were able to gain yards.

13
by Perfundle :: Tue, 09/24/2013 - 3:49pm

Wow, you really have something against the authors.

"4. P.McAfee punts 46 yards to SF 8, Center-M.Overton, fair catch by K.Williams.
3 and out with 8 yards gain. Would not have mattered if they had better field position."

It is more difficult to design plays when you are backed up in your own end zone like this. Offenses are going to be a lot more cautious, and defenses know that offense are going to be a lot more cautious and will try to jump routes more, and that makes a big difference.

"A field goal by going 52 yards, another by going 30 yards and a missed one after going 59 yards. They did not score because of the good field position. They scored because they were able to gain yards."

So you don't think with a 36-yard punt, followed by the 3-and-out, followed by the 30-yard drive, might've led to a missed 53-yarder instead of a made 43-yarder?

18
by NathanO (not verified) :: Tue, 09/24/2013 - 5:12pm

Seemed like a pretty reasonable comment to me. It's an excellent counter to the point.

What doesn't seem reasonable is you arguing he has something against the author. Why are you assuming bad faith?

You might dislike the argument, but to immediately insult the guy doesn't make any point in your favor.

30
by Perfundle :: Wed, 09/25/2013 - 12:33am

I don't think claiming that he has something against the authors is exactly insulting. The only person doing any insulting is him when he says "This author's special ability of seeing hidden information that others cannot see, well, makes him look foolish."

I said what I said because he had previously vehemently argued against the mere existence of DVOA predictions not being what he wanted, despite what he wanted being completely useless as far as predictions go, going so far as to challenge the authors as to its predictive value, and now this. None of his objections seem particularly polite.

36
by greybeard :: Wed, 09/25/2013 - 2:28am

Foolish means lacking good sense or judgment; unwise. And I think bringing field position for this game as the "real MVP" was a foolish thing to do. I did't think it was an insult. We all do foolish things in our lives. I am doing one right now: having an argument with someone on the Internet.

26
by greybeard :: Tue, 09/24/2013 - 7:13pm

I actually did not check who the author was before writing that comment. There are some FO authors I like (such as Ben Muth), some I don't care (Tom is one of them) and some I really dislike (Andy Benoit). Based on their work BTW. Not personally. I am sure all are nice guys.

I watched the game and except for 2nd drive play calling was bad everywhere on the field. So I don't agree with your theory that they had conservative play calls because they were backed at 8 yards line and failed because of that.
When you are at 8 yards your playbook is open. Even a 7 step drop does not get you to the endzone. You can call all the plays that you can call at 20 yards.

Field position matters. My point was Indy had two 80 yards drives for TDs. And a 52 yard one for FG and a 59 yard driver for a missed field goal. It is not like they scored and won because their drives started at the 49ers side of the field.

Average starting position between Indy and 49ers was 8 yards. You don't get your ass kicked because there was 8 yards difference between average starting position.

27
by NathanO (not verified) :: Tue, 09/24/2013 - 9:26pm

Exactly, I've watched the game 2 1/2 times now (at half time through watch #3) and all of the ideas people have as to what happened seem to involve people who haven't watched the game.

If you watch the game it's pretty clear. The man coverage and pressure make Kaep screw up all game. They ran fine, and should have ran more.

They would attempt a pass on 1st. fail. run on 2nd, get yards, leaving a 3rd to pass and fail again.

Kaep also refused to throw often for "pressure" sacks, essentially, the man coverage screwing with him the whole time.

I don't even blame the WRs like everyone keeps doing. Watch the game, pass after pass is completely uncatchable because Kaep is so worried about the DB being right next to the wideout.

32
by Perfundle :: Wed, 09/25/2013 - 1:20am

For someone who watched the game so many times, you don't even have your facts right. Even counting all Kaepernick scrambles and sacks as failed pass attempts, not one single time in the game did three consecutive plays happen in the manner you described. In the first half that you have watched three times, they ran it on first down 7 times versus 4 attempted passes; they only started passing it on first down after they fell behind in the second half. As for second down, all of the runs either picked up a first down or got minimal yardage. The one time they tried to run it on third down they failed.

34
by Perfundle :: Wed, 09/25/2013 - 1:44am

I was not so much referring to you disliking this particular author as thinking that you have vigorously objected to entire portions of articles (the all caps line seemed particularly unnecessary), the other one being the DVOA predictions one.

The author never claimed that punting was the only difference in the game, just that it was something that the national outlets never discussed. Perhaps he should've gone more into the failures of SF's defense, but the punting insight was quite interesting, and something I didn't notice watching the game.

"When you are at 8 yards your playbook is open. Even a 7 step drop does not get you to the endzone."

A false start penalty on one of their drives that started at their own 8 pushed them to the 5.5-yard line, and Kaepernick was indeed standing in his own end zone.

Indy also had a 30-yard drive go for a field goal, while SF had a 26-yard drive and a 35-yard drive end in punts, entirely because of field position. Who knows what would've transpired had the halftime score been 10-7 against of 7-10, so I entirely disagree with your dismissive "Would not have mattered if they had better field position."

35
by greybeard :: Wed, 09/25/2013 - 2:21am

I am not sure what is wrong with vigorously objecting entire portions of articles. Are we here to nod our heads in agreement? Or we are only allowed to be not vigorous in our objection or can be vigorous if it is only a small portion of the article?

38
by Perfundle :: Wed, 09/25/2013 - 2:51am

Yes, you're right that I forgot to add that your vigorous objections also come with lack of tact and incredibly long yet dubious supporting evidence. Almost no one else's objections have those qualities.

Thanks for responding to each of my points, by the way.

41
by greybeard :: Wed, 09/25/2013 - 12:29pm

I think my evidence for both of lack of quality on the predictive powers of DVOA (I am still waiting for Aaron to write an article and prove statistically how good DVOA projections are, if it were he had written that article long ago) and in this case the role of field position not playing an important role on the outcome of the game quite strong.

I think the reason you think they were"dubious" is nothing to do with the quality of the argument but you don't like criticism of the content on this site much, especially when it is as direct as mine. A lot of people start their criticism with first how much they love the F0 and its content etc. I don't do that. It does not change the nature of the argument or its contents. I don't bother. You can call it lack of tact, bluntness or whatever you want.

42
by tuluse :: Wed, 09/25/2013 - 12:45pm

Aaron has written several times about the predictive power of DVOA. It has in the past been shown to be more predictive than current win/loss and pythagorean wins.

44
by greybeard :: Wed, 09/25/2013 - 1:16pm

That was 7-8 years ago and correlation was .31, which is nothing to write home about.

39
by Danny Tuccitto :: Wed, 09/25/2013 - 3:21am

Personally, I enjoy the healthy skepticism in your comments. You're right that no one who's serious about this stuff should simply be nodding their head in agreement. The problem is that you're prefacing each comment (at least the ones I've seen) with thinly veiled, sarcastic pot shots towards the site and its authors, which ends up undermining your otherwise potentially insightful points of contention. For instance, rather than just laying out your field-position argument, you started off with, "This author's special ability of seeing hidden information that others cannot see, well, makes him look foolish. Let us see how the starting position affected the 49ers:" You said down below that you're not intending to make things personal, but the thinly veiled sarcasm in that sentence -- again, which you chose to lead off with -- has me feeling a bit of healthy skepticism about your claim.

We all make arguments around here, and there's no air of certainty in them -- quite the opposite, actually. Agree with them or disagree with them, and make any counter-arguments you wish. My frequent appearances in these threads proves that the staff does read them, and constructive criticism is quite valuable. If the motivation for your comments is to get us to think differently about things and improve our content, then I'd suggest the author not undermine the force of his special ability for skepticism by engaging in, well, foolishness. (See what I mean? Wasn't that an unwarranted, unnecessarily sarcastic way to phrase the point I was trying to make? And doesn't it make you want to ignore everything else I said?)

43
by greybeard :: Wed, 09/25/2013 - 1:16pm

Thanks for the feedback. I disagree with your take on my comments. I have ben following the site since 2005 and posting since 2008. I don't post many congratulatory messages on how good an article is (IMO every Ben Muth article is great, ever Tanier article was great, most Farrar articles were great) because a lot of posters already do. I don't comment much on if I don't feel strongly on a topic. I am no Will Allen. When I feel strongly about a topic I do comment on it. And it comes out strong.

But here is my feedback on how to make things better:
1) DVOA: Make your formula public. So we can try modifying it and try to make it better. If you want put a very restrictive copyright on it that all modifications need to go back to FO, fine.
2) Make DVOA emphasize to be descriptive not predictive. If fails on the second miserably. And I believe the factors put in it to make it predictive makes it not as good as it can be for a descriptive statistics.
3) Make Any Given Sunday about the winner, not loser. This article does not really tell what Indy did to win. It is almost all about SF, except for the field position. I'd like to read what the unexpected winner did special to win. Choose a game not necessarily because it was the biggest upset of the week, but because it was an upset due to something winner did that no other team did. Of course you may not have that kind of game every week, but most weeks you do.
4) Drop talking about LCF and making new versions of it. It does not work and its promise is wrong. No amount of versioning will get you there. The success of a QB is as tightly tied to which team drafted that QB and what the circumstances are as the quality of the QB himself. LCF does not know about which team drafts a QB, nor the future circumstances and therefore cannot predict the success of the QB based on his college days.

Thanks

45
by Danny Tuccitto :: Wed, 09/25/2013 - 3:12pm

Each of these suggestions are very fair. Really appreciate the feedback. Let me try to tackle them one at a time:

1) This is a reasonable request, but it's not going to happen because it's better in theory than in practice. Mind you, I come from an academic background, where you lay your methods/formulas/whatnot out in the public domain for others to do further research on, so I understand you completely here. One issue is that the full-timers here are literally making their living off of the DVOA (and DYAR and projection model and so on) formula, and I don't think it's fair to begrudge these people for wanting to do so. You offer a compromise related to copyright, which again is reasonable. The problem is that it puts Aaron in the position to have to hire lawyers to go after people who violate that copyright. He is not the recording industry. He's not the motion picture industry. He's a middle class guy with a family who doesn't have the weight of millions (or even hundreds of thousands) of dollars to throw around in litigation. (And, obviously, the recording and motion picture industries are failing miserably in anti-piracy efforts even with bottomless pockets.) But aside from all that, although the exact formulas are tightly guarded, FO lays out the methodology for all our stats in extensive detail, such that anyone could reverse-engineer them if they wished. For instance, I did exactly that in the context of the original LCF here two years before coming to FO. Neil Payne arrived at a pretty close approximation of PAR/YAR here. Haven't seen someone try to do this kind of thing for DVOA, but it really wouldn't be all that difficult. It would take a lot of time, though.

2) I'd like to hear more about why you use the word "miserably." The correlation between preseason DVOA projection and end-of-season DVOA is around .70. The DVOA-based weekly picks against the spread are 54.4% over the past five seasons, which is higher than the break-even point of 52.4%. I know for a fact that certain professionals in Vegas, who have made a fortune, consult DVOA. (Not going to name names here. You're just going to have to trust me on that one.) Your point in the projections thread about least squares not being an ideal method was a valid one, and Aaron is looking into other methods that would capture some of the overweighted-regression and correlated errors issues you raised. But besides all that, football isn't medicine or chemistry. The NFL is 52.5% luck. Compared to other fields, yes, DVOA-based predictions perform "miserably." In psychology, which is my background, R^2's in the 40% range are minor miracles because of all the unmeasurable, and therefore unmodeled, variables and interactions. Explaining football variance is a very similar Sisyphean endeavor. For this very reason, some people think that psychology isn't science, so research psychologists should just give up trying to predict anything. This sounds like what you're saying about football. I respectfully disagree.

3) This is a useful suggestion.

4) First, and most importantly, see #2. Second, in the LCF v1.0 replication study I linked to earlier, I found an R^2 of 56.2%. That's not shabby at all; it's certainly not low enough to chuck the whole endeavor or even to conclude that the system "does not work." Second, I think your point about a QB's prospects being tied to circumstances LCF doesn't know is a good one, but that doesn't mean "the premise is wrong." From a strict methodological perspective, an interpretation based solely on one of the underlying assumptions of least squares regression tells you that the projection is in the context of all unmodeled variables and measurement error being at the population average. Maybe that assumption's being violated, such that these QBs, on average, are being drafted into situations that significantly deviate from the population average of coaching quality, receiver quality, offensive line quality, etc. It's certainly worth looking into, but I don't think it's obviously faulty on it's face, especially when you couch the projection in those terms. For instance, this seems reasonable to me: "The LCF projection for Mike Glennon is -379 DYAR in Years 3-5 assuming he's drafted into an average situation. If you think his particular situation significantly deviates from the average, then feel free to adjust accordingly in your own mind." This also seems reasonable: "The projection for Andrew Luck is 1,749 DYAR in Years 3-5 assuming he's drafted into an average situation. If you think his particular situation significantly deviates from the average, then feel free to adjust accordingly in your own mind." I also don't think "the premise is wrong" in terms of the larger idea that you can extrapolate a QB's NFL performance from his CFB performance. Football performance can be defined as true ability plus/minus opponent's true ability plus error. The LCF is trying to spit out a composite measure based on several indicators of true QB ability and true opponent ability. College completion percentage is an indicator of true accuracy ability. College games started is an indicator of true overall ability because coaches, on average, start their best players. Improvement/decline in a QB's senior year is an indicator of true learning ability in a football context. (Incidentally, this is the reason the Wonderlic is crap. It's not an indicator of true learning ability in a football context, despite being promoted as such.) A strength of schedule factor would certainly be a useful additional indicator of true opponent ability, but for now that's being indicated by the non-AQ conference penalty. Run-pass ratio and rushing yardage are related indicators of true running ability and true sack-avoiding ability. We can argue about how reliable each of these indicators are, and we can set out to find even more reliable indicators, but I don't think the LCF's underlying premise is patently wrong: true ability plus/minus opponent ability plus error equals NFL performance.

Thanks again for the feedback.

46
by greybeard :: Wed, 09/25/2013 - 4:42pm

Thanks for taking time to explain all of this. I appreciate it.

WRT your question on 2, I was looking at Win-Loss projections not how consistent DVOA is within itself.

16
by Sisyphus :: Tue, 09/24/2013 - 4:57pm

So your assumption is that field position does not affect what plays are called. Each of those play calls in each of those series were predestined; thus bringing Calvinism to NFL analysis.

23
by sundown (not verified) :: Tue, 09/24/2013 - 6:46pm

This post reminds me of the days when Mike Shanahan was in Denver. The big talking point back then was how he drew up the first 20 plays of the game ahead of time. The talking heads would bring that up every game and would imply he ran right down the list. But I always wondered how that could possibly work given he had no way of knowing what the plays would produce. Like play #3 would be identical whether it was 1st and 10 following a huge gain, 3rd and 9, 3rd and 1, etc.?

24
by tuluse :: Tue, 09/24/2013 - 6:49pm

I think it's more like the first 20 plays you run not in the redzone or 3rd down. You always adjust in those situations.

25
by greybeard :: Tue, 09/24/2013 - 6:53pm

I am not saying any of that. I am saying that 49ers offense sucked wherever they were on the field except for one drive. They did not lose the game because Indy punter was good. They lost the game because their offense was beyond terrible. Anyone who watched the game knows that.

28
by Pied :: Tue, 09/24/2013 - 10:56pm

If you want a game won by the punter, look at the 2008 season AFC wild card round--the Colts were on the other side of a robo-esque game on the part of the Chargers' Scifres.

29
by greybeard :: Wed, 09/25/2013 - 12:15am

Yes. I remember that game.

15
by Todd S. :: Tue, 09/24/2013 - 4:14pm

Tom, now sure if you had the chance to really look at the film, but I was wondering how that makeshift offensive line blocked. Was it McGlynn and Linkenbach making holes, or did the tackles (Costanzo and Cherilous) lead the way in the running game? I would never have pegged the Colts as a team to gain 175 yards on the ground against the 49ers.

20
by Lebo :: Tue, 09/24/2013 - 5:55pm

Having just watched the condensed game (it was blacked out for me where I live - London!) I was surprised at how often the Colts defense rushed only four players. And how often those four players got pressure.

22
by Karl Cuba :: Tue, 09/24/2013 - 6:11pm

The 'Aldon Smith only gets pressure on stunts' trope is getting overblown. JJ Cooper's numbers suggested that he got 12 sacks on games in his first two years, that leaves 21.5 sacks that he got on his own, which would be pretty decent production and not immediately replaceable as this article suggests.

Additionally, he isn't the only pass rusher that benefits from stunts and help from his linemates. I've never seen a record of the percentage of sacks that result from second efforts or line games but I'm pretty sure that it's a decent fraction for most players even if Aldon Smith has a few more. He's also very good at redirecting on those loops and recovering his balance once he breaks into the pocket. Justin Smith is in his fifth year as a 49er, it's only Aldon Smith that has had 33.5 sacks in two years. Parys Haralson had 13 in the previous two.

I don't think any of his 4.5 sacks this year have been on loops, though a couple have been the result of qb's moving away from other rushers, though I think it's been Ray McDonald both times.

He doesn't get many sacks from a speed rush, unlike many highly productive pass rushers that isn't his game. He's a technique rusher so he doesn't get many of the 'left the tackle in his stance' sacks that we all love to see but it's inane to suggest that 99 isn't causing any problems on his own.

Corey Lemonier looked really good in the preseason but he's going to have to be very impressive to stop the 49ers missing Aldon Smith.

31
by Paul R :: Wed, 09/25/2013 - 12:49am

I had to listen to the first part of the game on the car radio, so I didn't see the Niners' scoring drive. As a Colts fan, I was prepared for a long, depressing afternoon, but that drive was horrifying to hear. San Francisco rushing for 15, 18, 20 yards a pop. Running through the Colts line like a hot knife through butter.

However, the rest of the game (which I did get to see)--nothing! The Colts defense looked pretty tight by the time I got to a tv, and seemed to gain confidence as the game went on, while San Fran's offense got more and more frazzled.

As I said, I didn't see the Niners' scoring drive, so I still don't know what the hell happened. Did the Colts defense make some magical adjustment? Some little detail ("Hey, Mario, you've got your helmet on backward") that allowed them to totally shut out San Fran's offense? I find that hard to believe.

Did the Niners coaches completely abandon the set of plays that got them an easy scoring drive? I find that even harder to believe. Or, if that is what happened, they should all be fired.

I was hoping this article would explain the drastic difference between that scoring drive and the rest of the game. It's true that the Colts' kicking game gave them an excellent edge in field position, but when all the Niners had to do was hand the ball off to get 20 yards, it would hardly matter where they started from.

33
by Pied :: Wed, 09/25/2013 - 1:40am

I can't really vouch for this explanation myself but word going around the Coltsphere is that the LBs started keeping outside contain instead of crashing inside (especially Walden).
A lot of outside runs turning the corner on that scoring drive.

37
by Perfundle :: Wed, 09/25/2013 - 2:43am

Yep, that's what it looks like to me, too. Walden was just abused on that scoring drive but he recovered after that.

SF's troubles on offense seemed to be a combination of drops (Williams, Patton and Celek each had one), Kaepernick unnecessarily panicking in the pocket when there wasn't really any pressure, lack of separation by his receivers, way too many plays from the pistol (running plays failed particularly badly, with 5 carries for 8 yards) and the aforementioned adjustments against running plays from under center.

48
by Purds :: Thu, 09/26/2013 - 4:23pm

I agree. I also, watching live, thought the Colts were not positioned pre-snap in any way to stop the pulling lineman from getting to the second level. It looked like the outside contain was at the line of scrimmage, and if he was handled (or crashed in), then it was smooth sailing to the SF right.

It looked like, after that drive, that the Colts had better pre-snap positioning on the weak side.

40
by Damn Lies (not verified) :: Wed, 09/25/2013 - 4:21am

I'm with greybeard

47
by carljm :: Wed, 09/25/2013 - 6:21pm

Not complaining (I usually love FO), but FWIW I found the opening DVOA chart followed by assertions of "counter-intuitiveness" a bit baffling. Maybe my intuition needs a whack, but it was far from clear to me what was supposed to be unintuitive about those numbers. Having read the comment thread, I'm not much clearer. I guess it's supposed to be counter-intuitive that the Niners offense was only about average, rather than terrible?

I guess it just seems weird to setup the conventional wisdom as "Pep Hamilton's preferred power run game helped the Colts control the line of scrimmage and walk away with the win" and then claim that the numbers tell a more complicated story, when the numbers show the Colts dominating all phases of the game, with a 26% offensive VOA. The story the numbers tell doesn't look all that complicated, or all that different from the CW.

The field-position differential was pretty obvious during the game; the commentators even remarked on it at least once.

49
by KingLunker (not verified) :: Fri, 09/27/2013 - 5:01pm

I realize this is all water under the bridge at this point, but I've waited now and it's still a mystery. In the Colts/Niners game, in the 4th Quarter, recall the Colts had that 80 yard drive. In the middle of that drive they faced a 3rd and 4 at their own 48 yard line. Luck attempted a short pass over the middle that fell incomplete but there was a flag. The replay did not show holding by the Niners defense. The commentators were confused over who it was on, couldn't really identify anyone. The referee announces the call and say "Holding...Defense". No one identified. Still to this day I have searched nflpenalties.com (tracks all penalties) and they only identify one SF player flagged for Defensive Holding so far this year, which was Terrel Brown in the second quarter of this game. So what drives me nuts...that play was HUGE, gave Indy first down, extending a crucial drive that extended their lead to 13 points. Indy punts there, we have potentially a whole new game. Still to this day...no one has identified who the call was against, it was a totally horrific call. Something tells me, Harbaugh just isn't like amongst the officials.

50
by Jocuri Gratuite (not verified) :: Sun, 09/29/2013 - 10:54am

For someone who watched the game so many times, you don't even have your facts right. Even counting all Kaepernick scrambles and sacks as failed pass attempts, not one single time in the game did three consecutive plays happen in the manner you described. In the first half that you have watched three times, they ran it on first down 7 times versus 4 attempted passes; they only started passing it on first down after they fell behind in the second half. As for second down, all of the runs either picked up a first down or got minimal yardage. The one time they tried to run it on third down they failed.

51
by The Cash Box Blueprint (not verified) :: Wed, 10/09/2013 - 5:28am

Go back to the Clutch Encounters breakdown of GB-CIN, the Packers receivers started blocking while the ball was still in the air. I don't ever remember seeing a game where this gets called.

FB Business Finder

The Cash Box Blueprint

FB Business Finder

The Cash Box Blueprint

52
by The Cash Box Blueprint (not verified) :: Thu, 10/10/2013 - 3:24pm

saved with supporting screenshots and pictures, the PDF is associate degree SEO Bible for those wanting to be told the secrets of driving targeted organic traffic.
It covers each necessary facet of SEO and it complimented with action guides on a way to produce lead generating websites.

SpinRewriter 4
SpinRewriter 4 Review
FB Business Finder
FB Business Finder Review
The Cash Box Blueprint
The Cash Box Blueprint Review
SEO Enigma
SEO Enigma review
1 Click WP SEO 4 Wp Plugin
1 Click WP SEO 4 Wp Plugin Review
Down & Dirty SEO
Down & Dirty SEO Review
SEO ZEN
SEO ZEN Review
FB Business Finder
FB Business Finder Review
The Cash Box Blueprint
The Cash Box Blueprint Review
Seo Zen

53
by amasuite 3 (not verified) :: Tue, 10/15/2013 - 11:51pm

Fb Objective will show your customer how to get the most out of these new features to generate more sales, leads, likes and get the brand new Facebook ad features working for their business.
AmaSuite 3 Review
Down & Dirty SEO
1 Click wp Seo 4 plugin
Spin Rewriter 4 Review
Down & Dirty SEO Review
Spin Rewriter 4
Amasuite 3
Insta Portfolio Pro
FB Objective
Timeline Poll Wizard
iDate Course

54
by amasuite 3 (not verified) :: Fri, 10/18/2013 - 11:00pm

Fb Objective will show your customer how to get the most out of these new features to generate more sales, leads, likes and get the brand new Facebook ad features working for their business.
AmaSuite 3 Review
Spin Rewriter 4 Review
Spin Rewriter 4
Amasuite 3
Insta Portfolio Pro
FB Objective
Content Gorilla
Timeline Poll Wizard
FB Cash Study
iDate Course
FB Social Lead System 2
The Cash Box Blueprint
Social Bribe
PulseOptin
SEO ZEN

55
by fb cash study (not verified) :: Wed, 10/23/2013 - 11:41pm

is a fully loaded, state of the art Amazon Affiliate WordPress plugin that’s power packed with powerful features. This plugin previously sold on clickbank with a whopping
Azon Rocket 2
Six Figure Shortcut
Offline InstaGold Soft
Local Video Take Off
Wordpress Blog Updater
The List Building Prototype
The YouTube Cash Blueprints Program
PulseOptin
G+ Business In a Box
FB Cash Study
7-Figure Offline Membership System
Xmas Toy Domination
Xtreme Membership
FB Loot

56
by Tony (not verified) :: Sat, 12/28/2013 - 11:58am

i can't post comment :(

57
by Tony (not verified) :: Sat, 12/28/2013 - 12:07pm

oh