Coach Rankings: Week 12
EdjSports is widely acclaimed for its in-game risk management analysis and proprietary Game-Winning Chance (GWC) metric used by NFL teams and media. GWC is a team’s win probability at any point in the game and is generated from the proprietary EdjFootball simulation model. EdjSports is creating the industry standard for head coach rankings by allocating its GWC to coaching decisions.
About the EdjFootball model
Built on 20+ years of historical NFL play-by-play and statistical data, the EdjFootball model is a fully customizable simulation engine. It accounts for each team’s strengths and weaknesses on offense, defense, and special teams. Model inputs include game state (score, timeouts, quarter, clock, down and distance, and field position), venue characteristics (indoor, outdoor, grass, turf, elevation), second half kickoff team, key injuries and Football Outsiders DVOA. Each week the model evaluates team performances and adjusts team strengths and weakness accordingly. As a result of these analyses, over the course of a season the EdjFootball model simulates over 3 billion games to conclusion.
About the EdjSports Coach Rankings
EdjSports analyzes every coaching decision during the course of a season. The EdjFootball model enables an in-depth examination of all critical calls (4th downs, PATs, and kickoffs), in terms of the amount of GWC at stake. The coach’s play-calling choices (run, pass, field goal, punt) are assessed at the point of decision (pre-snap) and rated with respect to their impact on winning the game. As a result of this process all play calling decisions can be objectively classified as either optimal decisions (correct calls) or suboptimal decisions (errors).
The EdjSports Coach Rankings are based on this methodology and consist of two main components that result in the overall ranking.
Edj Power Indexes (EPI) Ranking
This is a cumulative ranking of the Edj Power Indexes (EPIs) including Offensive Pass, Offensive Rush, Defensive Pass and Defensive Rush. This ranking is a reflection of how teams performed in every situation during the season.
Offensive Play Calling (CCI) Ranking
This ranking is based on an analysis of offensive play calling on 4th downs and compares teams in terms of GWC lost or gained, on a normalized basis so that all teams are given the same test. Additionally, the CCI rankings are adjusted to account for the frequency and situational nature of critical calls to allow for fair comparisons of all 32 teams.
EdjSports Coach Ranking
This ranking is a weighted average of the Edj Power Indexes (EPI) and Offensive Play Calling (CCI) Rankings that provides a more comprehensive assessment of coaching strengths and weaknesses.
Presented by:
Comments
25 comments, Last at 07 Dec 2020, 12:17pm
#15 by Noahrk // Dec 04, 2020 - 12:01pm
After reading some of the comments below, I changed my mind. In principle, maybe EPI and CCI could have equal weight: Imagine a coach that nearly always makes the optimal decision. That would be worth gold. The problem is that CCI is deeply flawed in that it's only a general guidance that cannot consider the specifics of each particular game situation: weather, opponent, relative strenghts and weaknesses, etc.
#19 by burbman // Dec 04, 2020 - 12:58pm
There is more to a coach than just in game decision making. I see what the rankings are trying to show, but a head coach is primarily a manager of personalities. A great coach can take a bunch of pretty good players and make them a competitive team, a bad coach can take a bunch of superstars and lose a lot of games. Judging them on just 4th down decisions, or team DVOA without regard for the talent available at the time, seems to be a gross oversimplification of the job.
Also, since coaching skills are usually not as fleeting as athletic ability, coaches should be evaluated over a longer period to account for varying talent levels on the team over time.
#24 by Noahrk // Dec 07, 2020 - 12:10pm
That's good to know. Still, regardless of the season-long strengths and weaknesses, on any given day things may turn out quite different. The Seahawks inabilty to move the ball yesterday against the Giants is an example. Do you trust your season-long performance when making decisions or your game-day performance so far? I'm inclined more towards the former.
#18 by Aaron Schatz // Dec 04, 2020 - 12:16pm
EPI is based on how well the team has played overall this season. The Saints are No. 1 in DVOA, thus they are No. 1 in the EPI.
Payton's performance in CCI is shocking. He's historically one of the most aggressive coaches in the league on fourth down. For some reason he's been very conservative in 2020.
#4 by KaosTheory // Dec 03, 2020 - 6:38pm
Doug Marrone should have gone for it on 4&8 in a tie game from his own 25?
Andy Reid should have gone for it from his own 23 early in the third in a tie game?
I think I get the reasoning, but I fail to see how these are wrong decisions even if the numbers disagree.
#8 by Raiderjoe // Dec 04, 2020 - 4:22am
Based o n natbematjcal formulas and othdr equations og football things mixed in metaphorical boiling cauldron in m y basement, piece of paper with Never Punt listed on it would havs led Loins to more wins than M. Patricia djd and Jets would not be 0-11.
#10 by Will Allen // Dec 04, 2020 - 9:14am
Without even examining the analytical framework here (my guess would be that it fails to adequately account for the nature of the matchups on the field in that particular game), I kind of dislike the fact that it promotes viewing coaches primarily through the prism of in-game decisions. It's the least important aspect of the job.
#22 by Frank Frigo // Dec 06, 2020 - 9:55am
I would like to address some of the comments/concerns regarding our initial 2020 Coach Rankings and also explain the methodology a bit.
- The coach rankings are an assessment of performance on critical 4th down decisions (CCI) and overall team strengths and execution (EPI)
- We give considerable weight to 4th down decisions because they often impact outcomes a great deal and the Edj simulation model is particularly well suited to measure their effect.
- Having studied 4th down decisions for almost two decades, the average NFL team gives up approximately 3/4 of a game per season on suboptimal choices, and these are 100% attributable to the coaches.
- 4th downs are complex, often counterintuitive and very dependent on the game state. While a strategy of greater aggression is important, they must be viewed in their proper context and that is often very difficult.
- We do account for venue, game state (ball location, score, clock, timeouts) and comparative team strengths (EPIs based on DVOA and DAVE) in all of our custom simulations.
- We also find that weaker teams may be less likely to face highly leveraged 4th downs (fewer close games) than very strong offenses and this is accounted for by netting the value of good choices against poor ones.
- There is far more to being a good coach than 4th down decisions of course, and this is why we put so much weight on the EPI. The quality of the players and execution in key situations is a reflection of the head coach and captured in DVOA and subsequently our EPIs. Surprisingly to some, New Orleans shows up in the middle of our rankings as a very strong team that often makes poor decisions. An enigma in that they are on top of one column and the bottom of the other. The Jets on the other hand are deservedly 32nd in EPI but Adam Gase has been fairly sound on 4th down decisions compared to his peers.
- Finally, this list focuses solely on YTD data and not the historical successes of coaches. Some change their behavior over time, and if they are making poor decisions per our criteria this season if affects their ranking. These rankings will undoubtedly change between now and week 16 when we reveal our Coach of the Year.
#25 by Noahrk // Dec 07, 2020 - 12:15pm
Thank you for your comment. The biggest weakness I see in the model is not being to account for when your team is laying an egg or, at the other extreme, owning a superior team. Maybe an adjustment for in-game DVOA could help? In other words, how in-game EPI affects CCI.