Extra Points
News and commentary from around the Web

Coaching Matters

This article at the MIT Sloan School of Management summarizes a research paper by University of Chicago public policy professors Christopher R. Berry and Anthony Fowler entitled "How Much Do Coaches Matter?" Berry and Fowler found that in all sports, coaches "explain about 20 to 30 percent of the variation in a team’s success."

Specifically germane to the NFL, Berry and Fowler found that coaches "significantly affect" a team’s fumbles and penalties. Also, "the researchers found that New England Patriots head coach Bill Belichick was 18 percent more likely to win a game (based on the quality of his opponent and home field advantage) than an average coach. While that isn’t the highest percentage of his peers, it is 'extremely unusual' given his 17 seasons [as measured in the study] with the team." Since Belichick was hired as head coach in 2000, the Patriots have the fewest turnovers in the league, and they have committed the fifth-fewest penalties.

View Full Article


12 comments, Last at 13 Mar 2019, 10:00am

1 Re: Coaching Matters

I haven't read the article in detail yet, but looking at Figure 4 makes me pretty suspicious. The spread in "coach performance" is much wide for NFL coaches who coached for only 1-2 years than it is for coaches who coached for 5-6 years, which is what I'd expect to see if their measure of coach performance was mostly picking up on noise rather than coaching ability.

2 Re: Coaching Matters

That seems to be what I'd expect for a reasonable grading system myself. The variance drops as more seasons are measured partly because of regression to mean, but also because the demonstrably poorer coaches generally gets weeded out and don't coach many seasons. Note that the figure in question has the vast majority of the reduction in variance coming from the lack of data points below the 'average' line as the number of seasons coached increases.

This is exactly what I'd hope to see in a professional sport, in amateur/college sports there might be more politics and economic pressure to keep poorer coaching in place, but in the NFL there is no reason to keep someone if they do not show themselves capable of at least hitting 'average' levels of results within a few seasons.

Given that the coach who stands out as being definitively above Belichik in the figure in question is Paul Brown it is probably not doing badly at identifying standout coaches.

9 Re: Coaching Matters

The funnel shape is there for positively rated coaches as well as for negatively rated ones, which is a strong sign that it's at least partly about variance & sample size.

It's true that it is stronger for negatively rated coaches. I see two main reasons that would happen. One is that a coach who is actually bad is likely to lose his job after a year or two while a coach who is actually good is likely to keep his job, which would lead to some funnel shape on the negative side based on actual coaching ability. The other reason is that a coach who has bad luck in his first year or two is likely to lose his job while a coach who has good luck in his first year or two is likely to keep his job, which is a way in which noise (plus firing decision) would make the funnel asymmetric.

My basic take on the graph is 1) the fact that there's this funnel shape shows that it's probably picking up on a lot of noise and 2) the fact that the long-tenured coaches generally score well shows that it's probably picking up on some true ability. And I'm worried that when they make claims like "explain about 20 to 30 percent of the variation in a team’s success" those numbers are inflated by the noise.

3 Re: Coaching Matters

RE: "Since Belichick was hired as head coach in 2000, the Patriots have the fewest turnovers in the league."

TB12 might have something to do with that?

4 Re: Coaching Matters

Maybe, maybe not. We really don't know how much coaching affect players into taking care of the ball, but sometimes you get these little things that add up into one major thing.

For example, 'fumbling is luck', but it seems like you could a) coach players into securing the ball with two hands, b) enforce it by terminating the fumble-prone guys, and c) other small factors that don't necessarily show up.

There was a video a few weeks ago showing Edelman yelling at the running backs to wipe down their arms to minimize sweating from affecting ball security, and I can't help but wonder what other factors like that play into the rates that could be attributed to coaching paying attention to small details like that.

6 Re: Coaching Matters

The obvious person to look at with regard to that would be Matt Cassel.

2007: Tom Brady, 16 GS, 6 fumbles
2008: Matt Cassel, 15 GS, 7 fumbles
2009: Tom Brady, 16 GS, 4 fumbles

So a different player, in as close a situation as possible (i.e., trying to control other variables) had close to the same number of fumbles, so maybe it isn't TB12.

On the other hand, you could also inspect 2016:
Tom Brady, 12 GS, 5 fumbles, .4166/game
Jimmy Garoppolo, 2 GS, 2 fumbles, 1/game
Jacoby Brissett, 2 GS, 3 fumbles, 1.5/game

This would suggest that TB12 is better at it, although the low sample sizes for Garoppolo/Brissett might be a problem, too.

This helpful information has been brought to you by Writing The Post Before Looking Up The Data

7 Re: Coaching Matters

Or look at RBs who left the team.

Though I would submit that someone like BenJarvus Green-Ellis throws a huge, huge outlier into everything, because the guy had what, 0 fumbles through his entire career with the Patriots. Of course, he accounts for 4 years, and we're talking about 17 years of Bill Belichick, so that sort of stuff should even out with more and more years...

Anyway BJGE:

4 years, 0 fumbles with the Patriots. 2 years, 5 fumbles with the Bengals.

Stevan Ridley, who I recall being basically an issue for ball security with the Patriots:

2011: 1 fumble in 16 games.
2012: 4 fumbles in 16 games
2013: 4 fumbles in 14 games.
(no real amount of carries, no fumbles in between)
2018: 2 fumbles in 10 games (with PIttsburgh)

One other anecdote about Belichick-coached teams, I think they're coached to -not- stretch their arms forward with the balls trying to gain extra yards. I don't recall where this came up (I want to say it was James White who said this.)

11 Re: Coaching Matters

Beyond the small sample size - Brissett had basically been in an NFL camp for 2 months - so any theory with coaching having an impact really needs to disregard him.

Cassel had one year more than Garoppolo - so some advantage there. If you add INTs into the equation, Cassel goes to 18 TO/15 GS, while Garoppolo is still at 2/2. Brady is at 14/16 for 2007 and 17/16 for 2009 - which is all close enough to say no real difference.

Also - Fumbling isn't luck - it correlates highly year-to-year. Fumble recoveries are luck.

8 Re: Coaching Matters

And if he does... how does that contradict the belief that Belichick is responsible for having fewer turnovers? Who was it that was coaching TB12?

I think a lot of people seem to think that Brady fell into the Patriots as the anointed GOAT and the coaching staff had nothing to do with his play. Brady was a driven and smart but mediocre player coming out of college, with some real deficiencies in his physical toolset, drafted with a 6th round flyer pick after not beating Drew Henson out for the starting job in college. His first year as a Pro, he was fourth on the depth chart. His next few years he was a competent but not amazing starter. He didn’t have a truly great year till 2007, and arguably wasn’t playing at a GOAT level till 2011 or 2012.

Meanwhile, it was Belichick and his staff that coached Brady into what he became, and designed an offense to play to his strengths and allow him to minimize turnovers.

It’s always the players who do or don’t commit turnovers, since they’re on the field and the coaches are not. That doesn’t mean coaching doesn’t affect things.

10 Re: Coaching Matters

People seem to forget that Brady basically had one skill the first couple of years - he made decisions quickly. He wasn't hugely accurate, He wasn't turnover averse - he threw picks at about a 2.7% rate (Eli manning is at 3% career). It wasn't until 2007 - that he dropped below 2% (and stayed there - he's been like 1.5% since). His YPA went up (and stayed up) by about 1.5 yards too.

The Patriots built an entire offence - every piece - from small, agile receivers who run lots of shallow crossing routes, to an offensive line strongest in the middle - to take advantage of Brady's specific skills - the ability to make reads very quickly, make decisions quickly, be relatively accurate in the short field, and move vertically in the pocket. The idea that doing that didn't significantly advantage Brady is crazy.

You switch Tom Brady's draft position with Spergon Wynn's, or Tod Husak's, or most of the other QBs taken as late round flyers that year, and he probably never starts an NFL game - because his skillset didn't match what people thought NFL QBs should be.

12 Re: Coaching Matters

This, exactly this. Brady didn't enter the NFL playing at an all-Pro level. Indeed, his regular season stats lagged behind Manning's for several seasons. He brought to the NFL a strong arm (though not the strongest by any measure), an ability to make decisions quickly, coolness under pressure, and the willingness to learn. Also, a competitive attitude that was matched by very few. He had to learn the game of football to become an all-Pro, record-setting QB.

5 Re: Coaching Matters

Also, I imagine, if we were applying this to DVOA, I'm guessing RIFLE, if I understand the paper correctly, would show its influence more in a) how consistent the DVOA is and b) how much above-average it consistently is, over the years.

(Of course, I understand little about the internal mechanisms of either system, so I'm hoping to see some sort of analysis on RIFLE from FO as pertaining to DVOA.)