The Plexiglass Is Half Full
by Danny Tuccitto
The most controversial element of Football Outsiders Almanac 2012 is clearly the team forecast that predicts the biggest nosedive of any team this year compared to last: 7.2 mean projected wins for the San Francisco 49ers. The Bay Area media picked up on it, and we went so far as dedicating our first Sabermetrics Video Network contribution to the topic.
The main explanation, which Aaron Schatz talks about at length in that video, is the Plexiglass Principle. What we say in FOA 2012 is that, if you look at the 41 teams in DVOA history that -- like the 2011 49ers -- improved between 25 and 35 percentage points from one season to the next, such teams averaged a 12.1-percentage-point DVOA decline the following season, as well as a decrease of 2.3 wins. We have the 49ers declining by about six wins this season, so the Plexiglass Principle isn't the only thing at play in terms of the overall projection, but that's what I'm going to focus on for the purposes of this little addendum.
If you want to see the Plexiglass Principle in one neat package, here's a table:
| DVOA CHG YR 2 Groups | # of Teams | WIN CHG YR 3 | DVOA CHG YR 3 |
| -30% or Worse | 30 | +1.6 | +10.2% |
| -29% to -20% | 49 | +2.4 | +10.5% |
| -19% to -10% | 85 | +0.3 | +4.2% |
| -9% to 0% | 134 | +0.5 | +3.0% |
| 0% to -9% | 103 | +0.1 | +0.3% |
| +10% to +19% | 110 | -1.1 | -6.3% |
| +20% to +29% | 58 | -2.0 | -10.4% |
| +30% or Better | 20 | -1.1 | -9.1% |
I've grouped all teams in 10-percent increments (without rounding) according to their DVOA change from Year 1 to Year 2. For instance, the 2011 49ers would fall into that second-to-last group because of their 29.8-percentage-point improvement from 2010. The last two columns show what happened, on average, to each group in Year 3. Win changes aren't as tidy because of the randomness of wins, but a DVOA change of 20 percentage points or more in Year 2 predicts about twice as big of a DVOA change in Year 3 than a Year 2 change of -19% to +19%. Ladies and gentlemen, the Plexiglass Principle.
A popular argument for why the 49ers' bounce down from the plexiglass might not be that big this season is that their bounce up coincided with a coaching change. Obviously, from the theme of the San Francisco chapter in FOA12, I'm on board the Harbaugh train, so it seems useful to test this hypothesis more generally. Have teams with a new head coach in their bounce-up year tended to bounce down less than those that improved with an incumbent head coach?
In the book, we cited only 41 49ers-like teams. For this piece, I'm going to use the info we just learned from the table, and expand the plexiglass definition to the 78 teams that improved by 20 percentage points or more. For each team, I marked down who the head coach was in each season of the three-year period.
We're trying to isolate the effect of a head coaching change between Year 1 and Year 2 on performance in Year 3, so I threw out four teams that had the same coach in Year 1 and Year 2, but changed coaches between Year 2 and Year 3. Dick Vermeil retired after the Rams' championship-winning improvement in 1999. Bill Parcells retired for the second time went shopping for groceries after leading New England to a 31.4% DVOA improvement in 1996. Marty Schottenheimer took over the Chargers in 2002 after they improved by 30.6% DVOA under Mike Riley in 2001. Tom Cable got fired after a 2010 season in which the Raiders improved by 29.9% DVOA.
Another pesky issue I had to resolve was that 29 of the remaining 74 teams had overlapping three-year periods within the same franchise. For instance, the 1992-1994 Broncos overlap with the 1994-1996 Broncos. Year-to-year dependency in NFL team data is pretty much unavoidable, but I'd have been remiss to just ignore the problem here. What I ended up doing with the overlapping teams was to randomly choose one of them for inclusion.
[ad placeholder 4]
That left 59 teams in the study: a "new head coach in Year 2" group with 17 plexiglass teams, and an "incumbent head coach in Year 2" group with 42 plexiglass teams. Only thing left was to get each group's average change in Year 3, and see if the difference between the groups was statistically significant. Drumroll please...
| Group | # of Teams | WIN CHG YR 3 | DVOA CHG YR 3 |
| New Head Coach | 17 | -0.9 | -5.1% |
| Incumbent Head Coach | 42 | -2.1 | -12.9% |
| Difference | +1.2 | +7.8% | |
| p-value | .241 | .118 |
On average, plexiglass teams see a smaller dropoff in wins and Total DVOA if their massive improvement in Year 2 coincided with a head coaching change. However, neither difference from incumbent-coached plexiglass teams is statistically significant, even at the 90-percent confidence level (i.e., both p-values are higher than .10).
Now, a statistical analysis like this is certainly open to interpretation. For instance, some might say, "The DVOA difference may not be significant at the 90-percent confidence level, but I'm totally comfortable being 88.2-percent confident in saying a new head coach matters!" No rebuttal from me there. However, the win change difference is nowhere near significant, and that's been the main source of pushback against our projection for San Francisco. It's reasonable to argue that changing coaches insulates a plexiglass team from the average DVOA decline (-10.7%), but arguing for a smaller win decline than average (-1.8) is just conjecture at this point.
Another other obvious issue with the analysis is only having 17 members of the new head coach group. Again, no rebuttal from me there. It's simply the case that coaching changes coinciding with a huge year-to-year improvement have been exceedingly rare over the past 20 years. Maybe when we have twice as many data points 20 years from now, we'll see a clear picture of new head coaches insulating teams from the Plexiglass Principle. Right now, though, the stats just don't support it.
Bringing things full circle, I'll end with something specific to the 2012 49ers that I haven't even mentioned yet in terms of the Plexiglass Principle. If we look at the Year 1 baseline, new head coaches improved teams that averaged 4.1 wins and -23.5% DVOA the previous season, whereas incumbents improved teams that averaged 6.7 wins and a -11.4% DVOA. San Francisco's Harbaugh hiring came on the heels of a 6-10 record and -11.2% DVOA, which is almost identical to the typical plexiglass team that didn't change coaches.
So what we saw in San Francisco last year is Harbaugh guiding a mediocre team to legit Super Bowl contender, which is extremely atypical in a plexiglass context. This isn't Tony Sparano improving a 1-15 Dolphins team by 10 wins and 27.6 percentage points in 2008 or even Marvin Lewis improving a 2-14 Bengals team by six wins and 32.2 percentage points in 2003. Of the 17 plexiglass teams in the "new head coach" group, only four had six or more wins in Year 1, which makes it hard to apply my general finding to the 49ers' specific case. This is something FO alum Bill Barnwell touched on in his Grantland column a couple of weeks ago.
In sum, we're encased in a plexiglass No Man's Land. Statistically speaking, there's not enough historical evidence to suggest that a head coaching change between Year 1 and Year 2 portends a smaller DVOA decline in Year 3. At the same time, issues related to sample size and the team Harbaugh inherited make it reasonable for people to downplay historical trends in the case of this year's Niners. Perhaps that's what's subconsciously influencing both Aaron and yours truly when we express subjective skepticism about San Francisco's model-based win projection.
Nevertheless, setting the plexiglass principle aside, our San Francisco chapter of Football Outsiders Almanac 2012 shows pretty conclusively that they'll win fewer than 13 games for other reasons. The beauty of publishing this stuff in a book and on a website is that we can reconvene early next year to see just how right or wrong we were.
Comments
77 comments, Last at 15 Apr 2013, 7:14am
#1 by Travis // Aug 06, 2012 - 3:59pm
Bill Parcells retired for the second time after leading New England to a 31.4% DVOA improvement in 1996.
Parcells didn't retire from the Patriots, but quit to (eventually) become the head coach of the Jets.
Incidentally, the Parcells Jets improved by over 25 percentage points in DVOA in both 1997 and 1998.
#2 by Danny Tuccitto // Aug 06, 2012 - 4:15pm
Fixed (with humor).
#3 by Jeff M. (not verified) // Aug 06, 2012 - 4:24pm
For instance, some might say, "The DVOA difference may not be significant at the 90-percent confidence level, but I'm totally comfortable being 88.2-percent confident in saying a new head coach matters!" No rebuttal from me there.
Except, you know, for this being a complete misunderstanding of the statistical result you describe...
#6 by Danny Tuccitto // Aug 06, 2012 - 5:09pm
Astute observation. I phrased it that way because I was speaking in terms of someone else -- presumably a layperson -- making that kind of argument. You know, as a debate tactic in a football column.
For interested readers, the correct statistical interpretation of p = 0.118 is that, if there's actually no difference between the groups, then there's only an 11.8% chance of observing a 7.8-percentage-point (or greater) difference.
#25 by Danish Denver-Fan // Aug 07, 2012 - 11:09am
I'm not picking a fight, I'm interested, in an aspiring statistician kind of way: Which distribution do you end up with? For the test-statistic I mean. Do you assume a normal distribution? Then you'd end up with a t-test, yes?
#49 by Danny Tuccitto // Aug 08, 2012 - 2:24am
Yes, it's a t-test. In terms of the underlying assumptions for a t-test, we have a normal distribution in our dependent variable (i.e., Year 3 change) and equal variances between the two groups.
#4 by MatMan // Aug 06, 2012 - 4:36pm
I read the Keri article at Grantland, but I still don't understand why it's called the plexiglass principle.
#5 by Anonymous Jones // Aug 06, 2012 - 4:49pm
Minor quibble.
"Nevertheless, setting the plexiglass principle aside, our San Francisco chapter of Football Outsiders Almanac 2012 shows pretty conclusively that they'll win fewer than 13 games for other reasons."
It doesn't really show this conclusively (I guess it depends on how much you credit the modifier "pretty" as being wishy-washy enough to invalidate the use of "conclusively" immediately thereafter).
Only in hindsight will the number of SF wins be known conclusively, and the reasons therefore will always be in doubt.
I think you meant something like "our San Francisco chapter of Football Outsiders Almanac 2012 provides many other reasons to doubt that they'll win as many as 13 games" or "our San Francisco chapter of Football Outsiders Almanac 2012 contains in-depth analysis of how other historical patterns and statistics lead us to have high confidence in the prediction that they'll win fewer than 13 games."
I think you'll still get a lot of pushback from (let's say, aspirational) Niner fans, but at least they will have less ammunition against an article laid out with greater precision.
#8 by chemical burn // Aug 06, 2012 - 5:34pm
I think the "conclusively" in this case refers to the "other reasons" - that is, the FOA chapter shows conclusively that the plexiglass principle isn't the only thing at play here and that there are other legit reasons to expect them to fail to win 13 games. Or even 10. It conclusively shows that fumble luck, the low int rate, regression to the mean and injury luck are the big factors in predicting notably fewer wins. Plexiglass principle is just bonus - they're not just hanging their hat on that idea alone and covering that implausibility with a fig leaf.
#10 by Bowl Game Anomaly // Aug 06, 2012 - 9:33pm
I can state pretty conclusively that your interpretation is wrong.
#23 by chemical burn // Aug 07, 2012 - 10:57am
Well then, they pretty conclusively said something stupid.
#28 by CraigoMc (not verified) // Aug 07, 2012 - 12:19pm
Actually, they do spend a lot of team on Smith's league-low INT rate - 1.1, as compared to the league-average 2.9 - and why there's very little reason to expect that to continue.
#32 by Karl Cuba // Aug 07, 2012 - 12:55pm
Danny has addressed Smith's interception rate more clearly here:
http://www.ninersnation.com/2012/8/2/3215713/49ers-q-a-with-nn-writer-emeritus-football-outsiders-danny-tuccitto
They don't project it to be as good but still expect Smith to be better than average on interceptions.
#35 by CraigoMc (not verified) // Aug 07, 2012 - 2:09pm
I respectfully disagree with that, for two reasons. Smith did take care to throw the ball away from defenders, but that was in a very conservative passing offense. I don't think San Francisco picked up Manningham and Moss to throw them screens and dumpoffs. His career rate of 3.0 indicates that the improvement in his accuracy was more due to Harbaugh's system, and that the improvement will disappear with it.
Now it's possible that other areas of Smith's passing could cancel out an improved interception rate, but I'm skeptical. Last season's improvement was built mostly around his low INTs - his Y/A and completion percentage improved only modestly, his TD and first down percentages declined just as modestly, and his sack rate spiked. But nobody really noticed those things because he threw five picks and the Niners front season are unholy terrors.
#36 by Karl Cuba // Aug 07, 2012 - 2:27pm
Moss and Manningham were signed to ensure that the 49ers would not end up with only one veteran receiver for most of the season like last year. While the 49ers probably will expand the passing game a little I don't think that you're going to see them get far away from the Bo Schembechler approach that Harbaugh adheres to.
As for Smith's interception percentage, he actually started to cut down on his interceptions half way through the season before last year. He has also improved a little as has the talent around him, I think it's a little simplistic to simply expect a return to his career interception rate.
My personal expectation is that he will probably throw something like 10 interceptions but will also throw 20-25 touchdowns, which could help the 49ers avoid some of the close games that they won an unsustainable number of last year.
#67 by LionInAZ // Aug 12, 2012 - 9:45pm
Are you sure about that? Moss and Manningham are the kind of WR pair who could end up cancelling each other out. The rest depends on what you think of Crabtree.
#19 by Aaron Brooks G… // Aug 07, 2012 - 9:18am
Couldn't you, a priori, say that about any team, without any analysis of any kind?
Sort of like saying it will probably be sunny today in Palm Springs. You have about a 90% chance of being right, no matter the day.
#7 by Karl Cuba // Aug 06, 2012 - 5:23pm
Honey Badger don't care about no plexiglass.
#9 by theslothook // Aug 06, 2012 - 5:57pm
One thing that really complicates this is we can't isolate injury improvements from win total performance. Now, in the case of the 49ers, they were actually healthier the year before, but I was speaking of things more generally.
On another note, 2010 49ers were being coached by singletary who I felt was a really terrible head coach, so the harbaugh effect was even more pronounced because of this, in the same way hostler was a big reason why the 49ers offense was cover your eyes horrific in 2007.
#11 by Paddy Pat // Aug 06, 2012 - 10:51pm
The likely uptick in schedule difficulty alone seems capable of accounting for 3 or so more losses for the 49ers in 2012. Regression in STL and perhaps in ARI, projected improvement in BUF, (leading to 3 tough AFC games instead of 2), etc. Couple that with the fact that defensive and special teams strength are less consistent year to year than offense, and it seems very conceivable that we will have a SF team with Special Teams around 1.2, defense around 0.4, and offense still around -4, which would put SF right around 8-8. One thing interesting to attempt to account for in this plexiglass principle, is whether the improvement was largely offensive or defensive. In the case of the 2011 49ers, defensive improvement was definitely more statistically salient than offensive improvement.
#12 by Grndmstrjns (not verified) // Aug 07, 2012 - 12:09am
I'm wondering... since your calculations show that the Niners will win around eight games, which teams will be in the win/loss column?
Great job on this site by the way.
#39 by Thomas_beardown // Aug 07, 2012 - 3:14pm
The predictions don't really work like they. They run 10,000 simulations of the season, and the 7.2 wins is the mean number they get over all 10k.
#41 by jimbohead // Aug 07, 2012 - 3:54pm
I'm pretty sure they do work like that. Each simulation of a season, as I understand it, goes from week 1-16 doing a monte-carlo style simulation of each game based on relative DVOA of each team (which itself has 1k permutations to account for DVOA uncertainty). This simulation is performed 1k times per DVOA permutation, so 1m simulations in total. They take a season-based average to get the 7.2 figure, but I'm pretty sure it should be trivial in future years to take a week-by-week average and spit out a win probability for each game. I doubt it's something they could do in a couple hours, but maybe they could work on doing this next offseason?
#43 by Thomas_beardown // Aug 07, 2012 - 4:03pm
That's still slightly different than what he was asking for. A team could be predicted to go 8-8 with a 50% chance to win each game.
#46 by jimbohead // Aug 07, 2012 - 5:49pm
Possibly true! But I choose to give OP credit for a more nuanced idea than he specifically offered. I think a "win probability by week/opponent" chart would be awesome. Total wins projected is just the sum of the win probabilities for each game.
#47 by Thomas_beardown // Aug 07, 2012 - 5:55pm
It would be cool, I agree.
#72 by LionInAZ // Aug 14, 2012 - 8:09pm
I assume that it's true that they could do this, but they have neither the time nor the manpower to do all that for every team. An interested party could always volunteer, right?
#13 by tkink (not verified) // Aug 07, 2012 - 2:24am
I think football outsiders are credible when it comes to collecting and analyzing data from history. But when it comes to predicting the present, there's not much hope. I wonder what their overall record is of being right or wrong.
#14 by dbt // Aug 07, 2012 - 4:20am
As the article says, "The beauty of publishing this stuff in a book and on a website is that we can reconvene early next year to see just how right or wrong we were."
Here are the 2011 projections:
http://footballoutsiders.com/dvoa-ratings/2011/2011-dvoa-projections
And here are some actual standings from after the season:
http://sports.yahoo.com/nfl/standings?type=regular&year=season_2011
AFC division winners, projected: Pats, Steelers, Texans, Chargers.
AFC division winners, actual: Pats, Ravens, Texans, Broncos. 2-4.
AFC WC, projected: Ravens, Jets.
AFC WC, actual: Steelers, Bengals. 1-2.
So that's 3/6, or 4/6 AFC playoff teams.
Commentary: Steelers were projected much better than Ravens, who were picked as a wildcard, in reality they tied and the ravens won the tiebreaker. Chargers finished in a 3 way tie at 8-8 with the Raiders and Broncos, lost tiebreaker. DVOA certainly missed on the Bengals, Jets, and Broncos.
NFC division winners, projected: Eagles, Packers, Saints, 49ers.
NFC division winners, actual: Giants, Packers, Saints, 49ers. 3/4.
NFC WC projected: Falcons, Bears.
NFC WC actual: Falcons, Lions. 1/2.
So that's 4/6 correct.
NFC Commentary: Bears obviously affected by injury, and DVOA missed Philly's surprising slump. I'm used to that one now.
This is from a non-stats perspective, of course. The more interesting thing to me would be to look at DVOA from projections to EOY and figure out why teams exceeded or missed expectations. Even with the 49ers picked to win the sad-sack west last year, DVOA projections missed how good they really would be.
Anyway, that's probably enough feeding a troll for one night. Do some more reading yourself, let me know how you think they've fared over the years, and compare them to other prediction efforts.
#20 by Aaron Brooks G… // Aug 07, 2012 - 9:25am
DVOA projections missed the 2012 results, but basically got the final 2012 DVOA order --(SD-OAK-DEN-KC). Considering that division had three 8-8 teams and one 7-9 team, the difference may have hinged on Oakland being better than expected as much as it was about Denver being better.
#26 by Podge (not verified) // Aug 07, 2012 - 11:44am
The details of the ones they got wrong are pretty impressive, in that there's only one that they were flat out wrong on.
In the NFC, the Bears were 7-3 (I think) when Cutler went down and would have definitely been in with a shout at the Wild Card. The Lions were also 7-3 at that point, having split 1-1 with the Bears, so I think that's probably 50/50 who'd have got in (the Lions went 3-3 after that point). The Bears did manage to lose to teams who put up 10 and 13 points though.
The Eagles went 8-8, the Giants 9-7, and the Eagles finished higher in DVOA (by ~5%). The Eagles appear to have been 5-1 against the NFC East (1-1 with the Giants), so if they'd won one more game (and they lost a few very winnable ones) they'd probably have nicked the division.
In the AFC, as you've said, the Chargers went 8-8 and had ~10% better DVOA than Denver.
The Jets went 8-8 to the Bengals' 9-7, with a DVOA advantage of ~13%.
That seems pretty good.
#27 by Podge (not verified) // Aug 07, 2012 - 11:59am
Actually, their projections of performance were pretty amazing. If you did it purely by DVOA, rather than record, the playoff teams would have been:
NFC
Winners: GB, New Orleans, Philly, San Fran
Wild Cards: Atlanta, Detroit
AFC
Winners: Pittsburgh, Houston, New England, San Diego
Wild Cards: Baltimore, Jets
That is, if you accept DVOA as the measure of "which team played best overall", the 8 teams they projected to win their divisions were the "best" team in their divisions. The two projected wildcard teams in the AFC were the two teams who played the next "best". In the NFC, the projection was right on Atlanta being the next "best" team, but probably wrong on Chicago.
On Chicago, the Week 11 DVOA ratings article (I think this was the week before Cutler went down) have them way above Detroit (19% to 10% - it ended at 10% to ~1%). This is a bit odd though - the book lists their offensive DVOA at this point as -0.9%, but the books says they were ~-7% at that point. Presumably thats a factor of the new DVOA, where the DVOA on the Team Efficiency page is DVOA7, but the DVOA on the weekly articles was DVOA6.
I think its fair to say that the Bears with Cutler were at worst about as good as Chicago. So basically, the predictions were accurate insofar as they predicted the "best" team in each division to win the division, 3 of the next "best" teams to be the wildcards and for the 4th one they were, at worst, about a wash.
#51 by Danny Tuccitto // Aug 08, 2012 - 2:42am
This.
There's a reason why our win projections tend to bunch most teams around 8-8 (i.e., lots of randomness in winning a given game, regression analysis being a tool for predicting average change across a group, etc.). If you focus instead on DVOA projections or playoff odds or even something totally subjective like "this team is probably going to be better/worse than expected," you find we're pretty accurate.
If you just go by win projections, and calculate something like mean absolute error, yeah, you're not going to find the results as good. But, again, I think the issue is that people (understandably) latch onto a very specific number (i.e., the win projection) when we actually publish a variability around it (i.e., the probabilities for on the clock, mediocrity, playoff contender, etc.) because there's considerably variability in potential football outcomes.
#52 by Podge (not verified) // Aug 08, 2012 - 4:39am
Which then leads to the question of why DVOA projections aren't included in the book (unless I'm being a bit simple and can't see them)? You have projection for average opponent, so they must be there in the background, but I can't see anywhere where its explicit what the projection actually is, which I think would be informative (even if it was a range or something).
If the 49ers DVOA projection was to dip by the average of the plexiglass principle (as in the book), 12%, their DVOA would be about 6%, and somewhere around 8-8 doesn't look too shocking. But when its sort of expressed as "they'll be about 12% worse in DVOA last year, and we're projecting that means 5-6 fewer wins" it seems odd.
#59 by Danny Tuccitto // Aug 08, 2012 - 9:18pm
Because we post them for free the week before the season. Here's 2011.
#29 by Danish Denver-Fan // Aug 07, 2012 - 12:24pm
While this is true, and I'm certainly of the belief that DVOA is the best predictor around, I think you can make up these excuses for any set of predictions. "Oooph I came so close to getting that prediction right - if only that nickel corner wouldn't have gotten injured."
Essentially when you strip the game of all luck (injuries mostly) I think you end up with something close to DVOA. Honestly I think the formula is so complex and take so many factors into account that I don't think you'd be far off. In so far you believe in regressional analysis as the end all be all.
#30 by Thomas_beardown // Aug 07, 2012 - 12:30pm
Starting quarterback injuries a little bit different, than really any other position.
#53 by Podge (not verified) // Aug 08, 2012 - 4:50am
Just a little bit.
Average score in Bears first 10 games: 26-20.
Average score in Bears final 6 games: 14-22.
Record in first 10 games: 7-3
Record in final 6 games: 1-5
Average win percentage of opponents in first 10 games: .544
Average win percentage of opponents in last 6 games: .500
#56 by Thomas_beardown // Aug 08, 2012 - 10:27am
They're also pretty rare. Check out the last AGL articles, most teams have less than 4 games lost from QBs.
#15 by theslothook // Aug 07, 2012 - 5:42am
Is there any sight that is any good at predictions? At least Fo has a math model built on foundations of statistics that are meant for prediction, rather than gut feel, premonitions, or voodoo. They do the best they can, but in the end, things like injuries, fluctuations in schedule, rapid emergence/declines, and the inevitable funny stuff like fumbles, td returns, and missed field goals are all things that NO math model can accurately predict.
However, FO does a pretty good job all around and while the exact prediction of win totals is probably inaccurate, they do provide a useful guide as to who will be good and who won't. 49ers fans( and I have a ton of them as friends), even if they hate to admit it, can see the writing on the wall that their record and dvoa likely unsustainable.
#17 by Dean // Aug 07, 2012 - 8:47am
So if those things can't be predicted, why would you expect some other site to be able to predict them?
#21 by Aaron Brooks G… // Aug 07, 2012 - 9:26am
Well, if you believe in a suitably deterministic universe, with a fast enough simulacrum, you could theoretically predict extremely rare events very accurately before they happen.
#16 by tiltab (not verified) // Aug 07, 2012 - 7:32am
last part made me laugh so hard )
#18 by ClemsonMatt (not verified) // Aug 07, 2012 - 9:15am
Don't alot of the same principles apply to the Panthers?
New head coach, 30% DVOA improvement from 2010 to 2011.
Is it entirely the absurd injuries on defense regressing to the mean that leads to the Panther's being projected to continue to improve?
#22 by Karl Cuba // Aug 07, 2012 - 9:48am
The Panthers' offensive DVOA accounted for all of their improvement, shooting up more than 50% and offensive DVOA tends to be more predictive and sustainable from year to year. Their defense and special teams were awful last year, with both units plummeting from mediocrity to dead last in the NFL by DVOA, consequently FO would expect those units to regress to the mean and improve. (It doesn't take a genius to work out that if Newton gets better and Carolina puts a run D and finds a second receiver then the NFC south should watch out)
#24 by chemical burn // Aug 07, 2012 - 11:07am
Man, has FO ever had as much riding on a prediction as SF not being a powerhouse this year? I feel like their Giants foibles were pretty rough, but they weren't the lone voice saying that those 9-7 teams probably wouldn't fare well in the playoffs. With these SF predictions, they're basically on their own and it seems like botching it is something they won't be able to live down. I'm not sure all the "hey, we did say Harbaugh is great, but couldn't factor it in" disclaimers are going to help them if SF goes 12-4 and makes it to the NFCCG again...
#34 by dmstorm22 // Aug 07, 2012 - 1:14pm
Oh, I think they are far from on their own. FO might be a little too pessimistic, as they spit out a mean projected win total of 7.2, but I know a lot of people (and sites) that are predicting the 49ers in the 8-10 win range.
#31 by Passing through (not verified) // Aug 07, 2012 - 12:47pm
Pre-season predictions are worthless, see this article* by Brian Burke:
http://www.advancednflstats.com/2010/07/pre-season-predictions-are-still.html
Personally I ignore the win projections and the plexiglass principle (which I can't differentiate from standard regression to mean ideas).
I find value in how FO points to specific factors that tend to cause regression to the mean, such as injuries, turnovers, outlier performance, etc.
*The article that I linked has faced criticism for only looking at one (bad) season for FO predictions. I think that's valid. However I think the point is generally true: it's unlikely that an expert's predictions would beat simple blind prediction techniques.
#33 by Thomas_beardown // Aug 07, 2012 - 12:56pm
I think it was Pat who went into detail about how making actual predictions that achieved the same results in terms of mean error compared to just picking 8-8 still yields valuable information.
#37 by Jonadan // Aug 07, 2012 - 2:46pm
49ers Schedule (predicted result)
1. at Packers (L)
2. Lions (L)
3. at Vikings (W)
4. at Jets (W)
5. Bills (W)
6. Giants (W)
7. Seahawks (W)
8. at Cardinals (W)
9. BYE
10. Rams (W)
11. Bears (L)
12. at Saints (W)
13. at Rams (W)
14. Dolphins (W)
15. at Patriots (L)
16. at Seahawks (L)
17. Cardinals (W)
I'm predicting essentially knee-jerk here: I come up with 49ers at 11-5. Which isn't 13 wins, but it's a lot closer to 12 than 7.2. Leaving aside DVOA, for the moment:
Singletary's 49ers went 7-9, 8-8, 6-10 - the low side of mediocre and in a bad division - although how bad isn't clear, since the Cards were decent-to-good in '08 and '09 and the Rams had a decent '10.
Harbaugh took over the team and got 13-3 and the NFCCG. Midway between 7 and 13 would be 10 wins. Sure they had good luck with injuries and turnovers, but I see 9 wins as the floor, even if the Seahawks and Rams improve (the Cardinals... I have no idea at this point).
I guess we'll see who's right.
---
"When you absolutely don't know what to do any more, then it's time to panic." - Johann van der Wiel
#38 by chemical burn // Aug 07, 2012 - 3:09pm
I didn't realize their schedule was so front-loaded. I can easily see them getting out to a 2-4 start and imploding as they try to figure out what to do with a sub-par QB who costs them games and a locker room with a few too many headcases... Not saying it will happen, but the Vikings are the only team in those first six games I think stands almost no chance of beating them (Bills, of course could easily be a push-over, but also were sort of the mirror of the 49er's last year in losing a bunch of close games - 4 by less than a TD and a 5th by a single TD.)
Th 49er's problem is they have no plan at QB and if Alex Smith is a guy who costs you games, not win them. If they go to his back-up, they're screwed...
#40 by dmstorm22 // Aug 07, 2012 - 3:19pm
I wouldn't say screwed. The 49ers are a team that doesn't need excellent QB play to win a lot of their games. For some, sure, Smith needs to play well for the 49ers to have a chance, but that defense is good enough to keep them in a lot of games. We also have no idea what type of player Kaepernick would be.
Seeing the schedule, I would say about 9-10 wins is a reasonable prediction. As you said, it is front-loaded, but having games like the Lions and Giants at home make those more winnable. I could easily see a 4-2 start, with the losses being @GB and @NYJ as well.
#42 by chemical burn // Aug 07, 2012 - 3:57pm
Oh, I agree - all things considered, if the Seahawks, rams and Cards don't improve, SF would have to go out of their way to end up worse than 9-7. The problem with being a trendy favorite with a front-loaded schedule is that failure to meet expectations can wreak havoc. You just never know how a team will respond to that psychologically (or a second year coach who hasn't experienced any real losing as an HC.) They'll be expected to push the panic button at 2-4 (the media will definitely push for it morning noon and night) and benching Smith (which should have happened 4 years ago) is the most logical move. The problem is that they then enter a relatively soft portion of their schedule with a deeply unproven QB who you can expect to botch a game or two just because that's the way things are. All the worse if he starts 2-0 before they reel off another couple loses - they'll want to give the new guy his chance. That's stuff wasted seasons are made of. If their schedule was back-loaded, I'd feel much better about their chances to go 11-5 or what-have-you against the exact same slate of opponents.
#44 by Thomas_beardown // Aug 07, 2012 - 4:04pm
Smith has been benched plenty, the problem is that he's been the best QB the 49ers have had.
#45 by chemical burn // Aug 07, 2012 - 4:07pm
That's my entire point: they have no plan but Smith and they're going to die with him.
(Also, I just wrote a long post about how I can see them going 2-4, Smith getting benched, the new guy going 2-1 against weak competition, the Bears and Saints beating them... and then they're 4-7 and screwed with problem like Moss and Jacobs being a distraction, no plan at QB, a coach under fire and divisional opponents with the divisional title suddenly in their sites.)
#48 by t.d. // Aug 07, 2012 - 10:01pm
presumably they drafted kaeperneck for a reason, and harbaugh's track record developing quarterbacks before alex smith is pretty good
#60 by chemical burn // Aug 09, 2012 - 1:42pm
That's reasonable enough - I just think for this year, when expectations are high, developing him in real games could cause major setbacks. It especially depends on how bad he is straight out of the gate (and most QB's are pretty bad, bad enough to lose winnable games anyways, one or twice in their first few starts.) I think if the idea is to have him be a major contributor this year, they should go ahead and bite the bullet. But it seems pretty clear they're in "win now" mode and also haven't given up on Smith. I think not decisively moving away from Smith last year or this year is the biggest coaching test Harbaugh has failed thus far. With a decent QB last year (and Smith was bad QB who had a decent year, which is different), we're probably talking about a defending champ right now...
#61 by t.d. // Aug 09, 2012 - 2:40pm
often inexperienced quarterbacks thrive until there's film on them, however. i'm more upbeat than most about seattle, as i think between russell wilson(unlikely) and matt flynn (more likely), they have a decent chance to upgrade their quarterback play, but, especially looking at their schedule, i don't think the 49ers would necessarily be out of it at 4-7. i also think deficiencies at wide receiver more than smith's inadequacies cost them the nfc title game, and they were lucky to be hosting that game in the first place, rather than travelling to green bay, where they'd have been decided underdogs (albeit with a good shot at winning). maybe every year this is more true than we'd like to admit, but last year it seemed like injuries played an enormous role in the giants' run (chicago, houston, pittsburgh, and new england suffered devastating injuries before they were eliminated). it was luck that got the 49ers that close to a title, just as it was luck that eliminated them (two muffs by their /backup/ punt returner), but it isn't like they were the best team last year and got robbed
/also don't think there was an obvious upgrade to smith available last year
//maybe hasselback
#62 by chemical burn // Aug 09, 2012 - 4:54pm
I think last year was the time to take a QB development project. Anyway, even at 4-7, you're right that division could still be up for grabs, as you say. Thinking about the 49ers I think they'll go 9-7 unless a team in the division takes a leap forward. And if they go 9-7, they could easily win the divisional title. Anyhoo, the pressure-cooker of high expectation/bad QB/QB controversy has killed with a lot better offenses than this one. And watching my Eagles last year, I'm loathe to ignore how high expectations quickly dashed can effect a team and cause media/locker-room chaos. I'm especially curious how it will effect Harbaugh, who is proving to have a bit of problem controlling his temper and prone to being a little condescending with the media.
(also, I agree with everything in the second half of your paragraph.)
#63 by bravehoptoad // Aug 09, 2012 - 7:51pm
I think last year was the time to take a QB development project.
Um...they did. In the second round.
Unless you're saying that last year, when they were two plays from the Superbowl, they should have instead spent the year playing a developing quarterback?
#65 by jimbohead // Aug 10, 2012 - 11:03am
yeahhh CB i would take a bit of time and rewatch some niners games on rewind if you get a chance. I'd recommend the Eagles game, but that would seem like cruel and unusual punishment. Perhaps the NFC-Div game, and the DET game. You characterize Smith as really really bad, but that's not how he played last year. He's not top 5, but he's not bottom 10 either.
Finding a top QB is really really hard to do. The niners are under no illusions that Smith is that guy; his contract says that much. But beyond bringing in a 2nd rounder last year (alternative that year was reaching in the 1st rd for someone like ponder. I'll take Aldon Smith over that thank you very much), grabbing a promising guy off the waiver wire last year from SD, and bringing in a FA qb this year who has upside, what more do you want? Hell, they even chased Peyton! It seems like what you really want is for the niners to simply cut their best option at QB, and then try to muddle through with whatever they have left, which doesn't seem to be reasonable to me.
#70 by chemical burn // Aug 14, 2012 - 1:54pm
I agree with DVOA - Smith was the 13th or 14th best QB in football last year. But that was a career year and by most49er fans' account, he was able to reach that level be being managed carefully by Harbaugh to avoid mistakes and let the defense win. You're looking at ceiling last year, achieved with a ridiculously low interception rate and total aversion to risk.
And, yes, I do think they should have transitioned him out of the starting job last year. Sure, they made it to the NFCCG, but you're just exchanging a very fortunate 2011 for 2012, which could easily be less fruitful of a year (according to FO, it could easily be a losing season.) By not taking 2011 to develop the starter and having him ready to go for 2012, you're wasting a year on the Alex Smith project, putting off on dealing with his debt for a season in which they exceeded expectation. If Harbaugh comes in and installs a new QB in 2011, not only will fans and media have more patience with that new QB's development, the team is ahead of where it is right now. When Smith is only the 20th best QB in the league this year and throws a more usual rate of interceptions and costs them a crucial game or two, don't be surprised.
(Also, slicing up the Eagles defense early in the season last year is nothing to hang your hat on. Ryan Fitzpatrick also did it. It took John Beck and Rex Grossman to make them look competent.)
#71 by Thomas_beardown // Aug 14, 2012 - 2:01pm
You keep saying they're not developing a replacement, but I recall them drafting a QB they're quite high on in the 2nd round last year.
So I can surmise that you believe one of two things, you don't think Kaepernick is worth anything or you believe that a player has to be in games to develop.
I'm not sure either one is true.
#73 by LionInAZ // Aug 14, 2012 - 8:29pm
I think chemical burn has a point. If you're really high on a drafted QB, why leave him on the bench behind an at-best aveerage QB? You certainly wouldn't do it if he was a 1st-round pick. You have that luxury if the starter is Brett Favre, but I don't think anyone would compare Alex Smith with Brett Favre.
My interpretation is that he's saying that last season's performance by the 49ers raised SB expectations for 2012, and that if they regress that will be harmful. Instead, if they had made Kaepernick the starter, they would have dampened the expectations and possibly left the team in better position for 2012 and beyond.
#74 by Thomas_beardown // Aug 14, 2012 - 10:18pm
I could see an argument either way. I don't think the 9ers are making a serious mistake delaying the Kaepernick experience.
Also, I do trust Harbaugh to some degree to make the right judgement from watching them in practice. Even if Kaepernick might be great someday, he still might be worse than Smith this year.
#75 by chemical burn // Aug 15, 2012 - 2:42pm
LionInAZ got my point. And I do think QB's need to play to develop, unless they are Aaron Rodgers or some other extreme outlier. Even Brady took almost 4 full season to develop to his full potential. Peyton Manning had a rough rookie year. Donovan McNabb, Drew Brees and Eli Manning all took multiple seasons to reach their potential. QBing requires playing time - a QB can be ok when he starts out, but almost no QB starts out as good as they will be. That's what I mean by development.
The only reason I think they're making a major mistake delaying the Kaepernick experience is that Alex Smith has been given waaaaay too long a tenure as a starting QB. They definitely should have moved on 4 seasons ago, 5 or 6 if they were a competently run franchise. He has no upside and hasn't shown any. Harbuagh proved you can manage him into mediocrity, but pretending that a mediocre QB is acceptable is always foolish and wastes time. For all the talk about "the 2000 Ravens did it with defense and the running game, why can't X team with a terrible QB?" everyone seems to overlook that the Ravens horrible QB situation almost certainly cost them Superbowl appearances and they certainly would have been better off with people other than Dilfer and Boller at the helm.
Anyhoo, if they made the switch at some point last year, there's less pressure on Kaepernick and they can bring him along. Now, they're stuck in a position where underperformance will cause panic and growing pains for Kaepernick would be met with impatience and frustration. So... they'll go into 2013 with Smith as QB? Or a totally unproven Kaepernick? Or some other new guy because the fans have turned on Kaepernick and Smith? If you believe they'll regress in 2012 (which I do) you can see why it looks like they mismanaged the situation.
As for trusting Harbaugh, I find it amazing that a guy who watched Dick Jauron go 13-3 would trust a coached based on a single 13-3 season. I guess you have gotten used to watching the Rex Grossmen and Caleb Hanies of the world - don't you regret arguing that Hanie showed promise in the NFCCG 2 years ago? Don't you regret watching guys like that tank a contender you root for? That's what Smith is. Harbaugh seems to be getting a weird amount of credit for 2011 and mentioned among the best coaches in the league. I personally don't trust him fully because of how handled Alex Smith.
#76 by Thomas_beardown // Aug 15, 2012 - 3:48pm
Let me be clear, I trust Harbaugh to be able to tell which QB looks best in practice. I didn't say I trust him to be a good coach (although I do think he is/will be one).
"I guess you have gotten used to watching the Rex Grossmen and Caleb Hanies of the world - don't you regret arguing that Hanie showed promise in the NFCCG 2 years ago? Don't you regret watching guys like that tank a contender you root for? That's what Smith is."
I don't regret saying Hanie showed potential because he did, I actually thought that a couple years before from pre-season. Now if I was running the Bears, would I have trusted him as the only backup to Cutler? No, I wouldn't have, but I can still see a valid argument for him being a #2 at the time. As a small aside the biggest miss handling of Hanie was not starting him in the final game of the 2010 season so they could get a full game's worth of experience and see how he looked.
Trust me, I understand that Smith is mediocre on his good days. My major point is that finding a good QB is hard and that often sticking with the devil you know makes more sense. There is a sizable chance that Kaepernick is worth than Smith and will never be better than him. If that's the case then they should never start him.
This whole thing is a risk management situation right? They can either bet that Smith will repeat a career year and still be average at best, and the defense will remain dominant for at least one more year, or they can bet that Kaepernick will be better than Smith this year or needs playing time to become better than him. That's an awfully tough decision.
Also, none of your QB examples prove that a QB needs playing time to improve, it just proves they need to be in the NFL to improve. Tom Brady sat his rookie year and was better his first year starting than all of your examples (except maybe Peyton). Kyle Orton after starting 15 games his rookie year, sat for about 2 whole years and when he got to play again he was better without playing any more games. Personally, I think a QB can improve X by playing and Y by being coached by NFL coaches and practicing with NFL players, and X+Y=Z which is the maximum improvement a QB can attain. I'm not sure that you can't max out Y before you put a QB in regular season game, and if you can't max it out, you could probably still increase it a lot.
#64 by bravehoptoad // Aug 09, 2012 - 7:54pm
...it was luck that got the 49ers that close to a title....
I think the luckiest thing to happen to them was that they got to play the Saints at home, courtesy of some obscure tie-breaker. Hard to imagine they could have beat them on that race track in New Orleans.
Otherwise, the 9ers weren't a particularly lucky team, except for the scarcity of defensive injuries.
#66 by t.d. // Aug 10, 2012 - 6:27pm
nah, the luckiest thing that happened for the niners was that someone else took out the presumptive Super Bowl favorite (not saying they wouldn't have been able to give the packers a hell of a game, but they wouldn't have been favored)
#69 by LionInAZ // Aug 12, 2012 - 10:09pm
These are both true statements.
#50 by Danny Tuccitto // Aug 08, 2012 - 2:31am
Front-loaded especially for turnstile RT Anthony Davis, which I allude to in my Q&A over at Niners Nation (linked in a comment somewhere above and also in that SB Nation XP).
#54 by Podge (not verified) // Aug 08, 2012 - 5:03am
I don't hugely disagree with any of your predictions, but just picking a game you should win and saying you *will* win are two different things, aren't they.
Say you try to predict the chance you have of winning each game, and have that as the number of wins for each game.
1. at Packers (0.2)
2. Lions (.5)
3. at Vikings (.8)
4. at Jets (.5)
5. Bills (.6)
6. Giants (.5)
7. Seahawks (.7)
8. at Cardinals (.7)
9. BYE
10. Rams (about 10,000, but lets say .9)
11. Bears (.6)
12. at Saints (.4)
13. at Rams (.7)
14. Dolphins (.7)
15. at Patriots (.3)
16. at Seahawks (.5)
17. Cardinals (.8)
That comes to 9.4 wins. As you say, I think about 9 wins is probably the floor, but say that you are slightly worse than expected, and a couple of teams are slightly better than expected (I'm looking at the Bears and Bills here particularly), and you blow one game you should win (if you could do that in week 10 that would be lovely) then 8-8 or worse isn't a stretch.
I think the most surprising thing about FO's projection for you is that it only gives a 6% chance of 11+ wins, and a 12% chance of 0-4 wins.
I'm curious what the difference in the projection would be if you reran it with the 49ers expected DVOA working off the New Head Coach Plexiglass Principle number, rather than the average Plexiglass number.
#57 by Jonadan // Aug 08, 2012 - 11:26am
That's an interesting thought - I assume those were your own guesstimated probabilities? Not that there's anything wrong if there from somewhere else, just confirming.
I guess what I'm really bugged by is:
The 49ers improved 7 wins (from 6 to 13) last season.
DVOA predicts that they fall almost as far (6 wins) as they just improved.
It makes no sense. A regression halfway (to 9.5 wins) is a reasonable prediction. But it seems like we're seeing FO conflate "less than 13 wins" (reasonable) and "7.2 wins" (very odd) in their defense. The last line of this article gives it away. "[Well... these are reasons 7.2 might make sense...] there are other reasons they won't win 13 games again." Does anyone expect 13 wins?
---
"When you absolutely don't know what to do any more, then it's time to panic." - Johann van der Wiel
#58 by Podge (not verified) // Aug 08, 2012 - 11:49am
Yeah, just my quick guesses.
On the last paragraph, I agree. What's most shocking about the prediction is that its only a 6% chance they have 11+ wins, and a 12% chance they have 0-4 wins. I can see the 49ers being an 8-8 sort of team. I can imagine them challenging for the Superbowl. I can't imagine them being in a position to be picking one of the elite college QBs in the draft next year.
Actually, if they did fall apart this year and manage to be in position to get someone like Barkley or whoever else is supposed to be highly drafted (was Landry Jones one of them?) that might actually be a terrifying thought for the rest of the NFC.
#55 by Bill (not verified) // Aug 08, 2012 - 9:46am
The problem here is that you can't statistically measure the incompetence of Mike Singletary and Mike Nolan. This wasn't MERELY a coaching change. It was a swap of guys who didn't know what they were doing with a very experienced and competent coaching staff (not Harbaugh, but look at his coordinators).
#68 by BaronFoobarstein // Aug 12, 2012 - 9:58pm
I like that the only team that actually fired the coach who just presided over a huge improvement is the Raiders. Presumably a coach with much better 40 time was available.
#77 by johnpeterson (not verified) // Apr 15, 2013 - 7:14am
My friend mentioned to me your blog, so I thought I’d read it for myself. Very interesting insights, will be back for more!
http://www.chron.com/business/press-releases/article/Consumer-Reports-2013-Mattress-Ratings-Inspire-4343826.php