Fewer Practices Does Not Mean More Injuries
by Zach Binney
The expression "no days off" is as synonymous with Bill Belichick as cut-off hoodies. So when the 2011 collective bargaining agreement (CBA) between the NFL and the NFL Players' Association (NFLPA) restricted offseason and in-season practice days, the coach was not pleased. He and others grumbled that their players were no longer getting the reps they needed to perform well in the regular season.
But Belichick did not stop there. He also blamed the practice restrictions -- which cut organized team activities (OTAs) from 14 weeks to nine; eliminated padded two-a-days during training camp; increased off days during training camp; and limited regular-season padded practices from whatever-the-coach-wants to 14 -- for increasing injuries.
The practice restrictions were designed in part to improve player safety by decreasing stress on players and giving them more opportunities to rest and recover. Belichick's dueling theory was that with fewer practices, players were physically unprepared for the rigors of an NFL season, leading to more injuries. He was not alone in his view, nor was his theory entirely without merit. There are a number of scientific studies that show that training either too little ("undertraining") or too much ("overtraining") can raise your injury risk. The idea is to find the sweet spot in the middle that minimizes injury risk. What was unknown was whether the NFL's practice regimens resulted in overtraining in the pre-CBA era, undertraining in the post-CBA era, both, or neither.
Fortunately, we don't have to rely on Belichick's theories since we have the data to examine what happened to NFL injuries before and after the 2011 CBA. To do this we counted the number of injuries by year from 2007 to 2016 and observed how they changed in the pre- and post-CBA periods. We also counted games missed due to injury each year to account for any changes in injury severity rather than just the number of injuries.
An injury was defined as any new physiologic event appearing on the NFL's weekly injury reports. We included only non-head injuries that caused a player to miss at least one regular-season game since we know reporting of minor and head injuries has skyrocketed over the last decade. We did not want those trends mixed up with any effects of the CBA. We excluded preseason injuries because the CBA also raised preseason roster sizes, which may have caused injury counts to rise just because more players were at risk. This left us with 7,425 injuries in approximately 19,000 player-seasons from 2007 to 2016.
More details about our methods are available in this pre-print paper.
All-Injury Counts Over Time
We began by looking at total injury counts and games missed before and after the CBA. Figure 1 does not show any appreciable increase in injuries after the CBA (the black vertical line between 2010 and 2011). If anything we see a decrease in injuries for the first three years under the new practice scheme (2012 and 2013) before they rebound to historical levels.
| Figure 1. Number of Regular-Season Game-Loss Injuries and Games Missed Due to Injury, 2007-2016, with 95% Confidence Intervals. |
 |
There also seem to be some modest increases in injury counts over time before the CBA ever took effect. This suggests that it could be unfair to simply compare injury counts in the pre- and post-CBA periods. Such a comparison could make the CBA look bad when an increase in injuries was due to ongoing trends that started well before it took effect.
Another way to think of this is if we were to try to draw a line through our data points, that line would be going up from 2007 through 2010. It would be a bad sign for the CBA if after 2010 there was either a sudden and persistent increase in injuries (a jump up in the line after 2010) or if the pre-CBA increase in injuries sped up (an increase in the line's slope after 2010). If the line just continues on as before for 2011-2016, we could say the CBA had no impact. Hold that thought.
Injuries Stratified by Conditioning Over Time
Looking at all injuries together may not tell the full story. Belichick's theory was that less practice time left players insufficiently conditioned for the NFL season. Perhaps under the new CBA there was an increase in injuries most dependent on an athlete's physical conditioning -- damage to soft tissues such as the calf, hamstring, and quadriceps. We can contrast this with what happened among traumatic injuries such as fractures that have little to do with an athlete's physical conditioning.
When we do this in Figure 2, we see no appreciable change in either conditioning or non-conditioning injuries when the new CBA takes effect. The closest thing we identified was a temporary spike in conditioning injuries in 2011. The 2010-2011 offseason featured a 4.5-month lockout that kept players away from their teams in addition to the CBA's new practice restrictions. This may have led to players training less than at any other time in our study, leading to poorer conditioning and a temporary one-year bump in conditioning-dependent injuries. That said, it's impossible to draw any sound conclusions from a single data point.
| Figure 2. Number of Regular-Season Game-Loss Injuries and Games Missed Due to Injury, Stratified by Conditioning Status, 2007-2016, with 95% Confidence Intervals |
 |
Statistical Models
Plotting injury counts over time does not suggest the CBA had any impact on injuries. However, it would be useful to fit a statistical model to our data in case our eyes are not picking up on some subtle trends.
We fit something called a Poisson interrupted time series model to the data. This allowed us to do a couple useful things:
- Account for the pre-CBA increase in injuries mentioned earlier.
- Control for changes in player age or other injury risk factors across the study period.
The details are available in our full paper, but suffice it to say the model found no evidence of an increase in injuries or games missed due to injury under the new CBA's practice restrictions.
[ad placeholder 3]
Sensitivity Analyses
We conducted several sensitivity analyses, which involve tweaking some analytical decisions and seeing how that changed our results. We tried including preseason injuries, including minor injuries, re-classifying ankle and knee injuries to be "non-conditioning," and looking at only hamstring injuries. None of them changed our conclusions.
Limitations
Our analysis has a major limitation: there were several other changes designed to enhance player safety and reduce injuries at the same time as the CBA and throughout the study period. Changes made simultaneously with the CBA's new practice restrictions included moving the kickoff up to encourage more touchbacks and the expansion of the "defenseless players" list to cover, among others, receivers who have not re-established themselves as runners. It is difficult to disentangle the effects of all these various changes in our data, but it is most likely these effects would have biased injuries in later years downward. This would in turn have made the CBA look better than it was.
Conclusions
There is no evidence that the 2011 CBA led to increased injuries in the NFL. It does not appear that the practice restrictions pushed the NFL, on average, from a state of optimal training that minimizes injury risk to one of undertraining. Of course restricting practice time may have other negative effects such as degrading player skill or game quality, but that is beyond the scope of this article.
Comments
40 comments, Last at 11 May 2018, 10:05am
#1 by Aaron Brooks G… // May 07, 2018 - 12:48pm
Changes made simultaneously with the CBA's new practice restrictions included moving the kickoff up to encourage more touchbacks and the expansion of the "defenseless players" list to cover, among others, receivers who have not re-established themselves as runners. It is difficult to disentangle the effects of all these various changes in our data, but it is most likely these effects would have biased injuries in later years downward.
This assumes the 'defenseless player' designation was productive. Has this been shown?
#3 by Aaron Brooks G… // May 07, 2018 - 1:21pm
I'd just like to raise the hypothetical that it may have been counter-productive.
Has "defenseless receiver" coaxed WRs and TEs into going across the middle or deep seam at a sufficiently high rate that even a decreased rate of blowing guys up has still led to a net increase in injury rate?
Granted, it also coincides with Welker making slot receivers sexy, so some of these route tendencies may just be coincidental timing.
#4 by sbond101 // May 07, 2018 - 1:52pm
Yeah;
The correct serious counter-question to the author is - could injury rates possibly provide significant evidence that the BB thesis was correct. I think there's good reason to say no. First its obvious that a lot of factors that might influence injuries outside of practice time have changed; Second, because NFL teams all practice close to the same amount there are no separate populations within the NFL to begin to perform factor-analysis to isolate practice time.
Based on the above think the conclusion is mildly irresponsible. I do think if you understood the mechanism behind injuries better than I do you could chart all the injuries and begin to do factor analysis as to how many are tied to over/under training. Based on a basic knowledge of the development of the literature on sports-medicine I suspect that it's not a straightforward issue and that the injury-type mechanic used in the above article is deeply impoverished compared to a charting approach. I assume training staffs for NFL teams do chart pathologies because they have a pretty powerful financial incentive to understand how effective their training staff is. Someone like BB who has access to that kind of charting would be well placed to offer an opinion - but I think it's pretty clear that football coaches are publicly honest about approximately 5% of what they think/know and BB is honest about 1% of what he knows about things impacting competitive balance, so taking their statements at face value doesn't seem better. The correct headline is something like new CBA, rule changes, concussion reporting, and the growth of passing, combine to make almost no statistically significant changes in injury rates/games missed - a statement which in itself is fascinating given that people in football seem to think all those things should significantly change injury rates.
N.B. based on a more thorough read of the article I have revised some of my comments on the articles statements related to factor-analysis.
#5 by Mountain Time … // May 07, 2018 - 2:56pm
Perhaps, but you're asking for a very different study than what was done here.
I assume training staffs for NFL teams do chart pathologies because they have a pretty powerful financial incentive to understand how effective their training staff is
I sure wish I could share your optimism. Surely someone here has actual first- or secondhand knowledge of a professional training staff and how their performance is measured?
#16 by sbond101 // May 08, 2018 - 10:14am
I didn't intend to ask for a different study; I intended to point out that the study done here can't possibly answer the research question identified by the headline. I proposed a charting based study because I think that's what you would have to do to start to get at the research question - but even that would be weak methodology as it would be un-controlled.
I really think what's called for hear is a less ambitious research question; The comment below to the effect of "did the various rule changes intended to lower injury rates work" seems like a less ambitious research question that can be answered (no). The trouble is that answering it doesn't indicate what the right thing to do next is.
#28 by Zach Binney // May 10, 2018 - 1:01pm
I recognize the limitations of this analysis and agree with some things that you said. Knowing more about the mechanics of each injury is definitely important. If we had that sort of information we could zero in better on the types of injuries most likely to be impacted by the BB hypothesis better than I've done here (a crude conditioning/non-conditioning split by anatomical location).
But unfortunately we're limited by the data we have publicly available. Charting data can give us some more info about what injuries happened in games or on specific plays, but the BEST source of info is going to be the NFL's own Injury Surveillance System. That would tell you information like whether a particular injury was from contact with a player, contact with the ground, non-contact, etc. That would help us get at a group of injuries likely to be impacted by poorer conditioning vs. not. Frankly I hope my analysis - based off the only data the public has available - might spur someone with the NFL or NFLPA to publish a better study with better data.
However, I would argue that you're using a rather narrow definition of control group. Here we are comparing what happens to injury rates under the new and old CBA, so we CAN in fact make a comparison between the two groups. Second, the whole point of breaking down by conditioning and non-conditioning is so that we can have groups where the BB hypothesis should have an effect and no effect, respectively. I only explained it superficially in the article, but I get into more detail in the full academic article. Please take a look if you're interested! And thanks for reading and commenting!
#34 by sbond101 // May 10, 2018 - 2:59pm
First off - really appreciate the engagement with the questions here. I went through the academic article and its interesting.
With regard to the methodology for isolating conditioning injuries and the subsequent conclusion that you could evaluate the "BB hypothesis" through segmenting the data. I caught that the first time through, and its the reason I pointed to the limits of the pathology data in question. Putting the data integrity issue aside as insoluble for the moment you still have a serious structural flaw to the study - FO has documented the extensive changes in the way that football is played on the field that are highly likely to influence conditioning injury rates (e.g. the kind of hard cutting that goes with middle of the field passing and screen game that is dominant today is very likely to be different with regard to conditioning injury rates than running the football between the tackles). moreover the number of plays per game and rest periods between them have both changed considerably over the decade of study, both of which are factors which should influence rates of conditioning injuries, as should rotational philosophies for various positions. As a result what this study can conclude with some confidence is that the NFL under the new CBA has similar rates of conditioning related injuries, but that conclusion is substantially different in its implications than the conclusion that changes in practice rules have had no significant impact on conditioning injuries.
To qualify the perspective I think its fair to view conditioning practices in sport along a similar axis to prophylactic therapeutic practices in medicine in general (to the extent that they interact with NFL rules under the auspices of influencing injury rates). A correlative uncontrolled study like this, with factors declared by the population as intentionally convoluting (like the kickoff changes), would be given very little consideration the context of a evaluating a therapeutic method. I think it's appropriate to limit conclusions in a similar way in a context like this since public opinion appears to shape how the NFL builds its policies designed to protect players from injury. - to put it bluntly, the I believe the ethical issues inherent to the topic demand a higher standard of evidence than this in order to publish the conclusion in question;
#36 by Zach Binney // May 10, 2018 - 4:08pm
Ahh, I see. You're absolutely right about the numerous other changes over time. I would counter that by saying that the changes that you list (play style, rotational philosophies, etc.) are things that have changed gradually over time - i.e. they're exactly the sort of changes our Poisson models were designed to filter out from the *snaps fingers* instantaneous change of the CBA at a particular point in time (between the 2010 and 2011 seasons). Looking for any sort of discontinuity (i.e. a sudden change) at that specific time should help separate out the CBA signal from all the other changes you listed, which accumulate over time as strategies spread across the League. Of course there were other changes simultaneous (or near-simultaneous) with the CBA's practice restrictions, and those are absolutely still a problem. But I don't think these long-term trends are as big a problem with the analysis as you suggest.
I'm also sensitive to your concerns about overreaching, but we felt the evidence was strong enough to merit our conclusions (i.e. a complete lack of evidence for any detrimental effect, though not the presence of any sort of beneficial one). Science is an art, not a...er, science, so reasonable people can come to different conclusions when looking at the same data. It's not what people want to hear, but it's the truth. So I welcome your different perspective there, even though I respectfully disagree with it!
#6 by Dales // May 07, 2018 - 3:07pm
Couldn't most of this be turned around just as easily? While there is a lack of evidence that it increased injuries, there is also a lack of evidence that it decreased injuries (which was the stated goal of the change)?
#29 by Zach Binney // May 10, 2018 - 1:05pm
Sort of. If you go the full paper and look at the Poisson time series models (and dear God, why would you?), we actually see some evidence that there might be a decline. My co-authors and I weren't convinced enough to make that our main conclusion, but it was there.
In short, there was basically zero evidence for a detrimental effect. There was SOME evidence of a beneficial effect, but there was also some evidence for no substantial effect. We felt most comfortable concluding that there was no detrimental effect rather than that was a beneficial or no effect specifically. If that makes sense.
#7 by John Walt // May 07, 2018 - 4:48pm
While I understand leaving head injuries out of the study due to their increased awareness around the league and the new openness to reporting them, but could some of the increase in head injuries be attributed to a lack of fundamentals in tackling? This could be the result of a lack of practice time. Better athletes with worse technique hitting each other? I'm just curious how many more head injuries there are.
#24 by LionInAZ // May 09, 2018 - 3:44pm
Lack of "tackling fundamentals" is a problem that predated the change in practice.
Much more likely that the increase in head injuries is due to the requirement that concussions actually be dealt with and reported.
#8 by Bright Blue Shorts // May 07, 2018 - 5:06pm
We used to talk about whether grass or astroturf caused more injuries.
There's been a few new stadia used since the CBA - Giants, Minnesota, LA, London Twickenham, Mexico off the top of my head and I don't know about the surfaces involved.
No signs of this being controlled for in the study and not sure how much potential difference it could make?
#10 by Vincent Verhei // May 07, 2018 - 6:59pm
We used to talk about whether grass or astroturf caused more injuries.
There's been a few new stadia used since the CBA - Giants, Minnesota, LA, London Twickenham, Mexico off the top of my head and I don't know about the surfaces involved.
No signs of this being controlled for in the study and not sure how much potential difference it could make?
...
Has anyone done injury rate by stadium? Even just of visiting teams?
I root for Wash and Oak, you see...
Zach looked at both of these issues last year.
https://www.footballoutsiders.com/stat-analysis/2017/turf-type-and-nfl-injuries-part-i
https://www.footballoutsiders.com/stat-analysis/2017/turf-type-and-nfl-injuries-part-ii
#12 by fyo // May 08, 2018 - 4:57am
In my opinion, the article premise is backwards...
"Is Belichick right?"
As opposed to
"Did the practice restrictions succeed in the stated goal of reducing player injury risk?"
The answer to that question would appear to be an unequivocal NO.
Despite several other initiatives aimed at reducing injuries, overall injury risk remains unchanged.
That would seem to be a strong argument in favor of scrapping the apparently arbitrary and ineffective practice restrictions.
#14 by Aaron Brooks G… // May 08, 2018 - 8:16am
That would seem to be a strong argument in favor of scrapping the apparently arbitrary and ineffective practice restrictions.
No, that's an equivocal argument in favor of scrapping the apparently arbitrary and ineffective practice restrictions.
A strong argument would have been a rise in apparent injury rate.
But the finding was no change, which means there is an equally strong argument that the practice restrictions are not detrimental -- that is, extra practices did nothing -- positive or negative.
If you want to argue that scraping the extra practices are worthless, you're adopting the argument that the players' personal time is worthless. That may be the argument of the coaches and owners, but probably is not the argument of the players (who you perceive as worthless).
#15 by nat // May 08, 2018 - 9:25am
Didn't the injury rate actually go up? If I understand the analysis as done, the post CBA injury rates were compared to a projected (rising) trend, instead of just comparing the pre- and post-CBA injury rates.
With that trend deduced from a short (four data point) time series, and with no convincing theory of why the trend would have existed in the first place (except for head/concussion injuries, which were excluded), I am not convinced that comparing to a projected trend was valid in this case.
Without the bias introduced by the use of that trend line, I suspect the change in the injury rate is significant.
#17 by sbond101 // May 08, 2018 - 10:26am
That's correct. It would be interesting to see a long term analysis of injury rates. Its my suspicion from observation that pre-cba the trendline had a positive slope and negative 2nd derivative, and that it relates to changes in training techniques, nutrition, injury treatment, and PED use that really transformed injury rates between the late 80's and early 2000's - but again it's an extremely difficult set of questions to rigorously study.
#32 by Zach Binney // May 10, 2018 - 1:22pm
There's actually quite a bit of evidence for a rise in injuries over time (e.g. https://footballoutsiders.com/stat-analysis/2017/truth-behind-rising-injury-rates).
I admit that trying to discern a linear trend from four data points is weak, but it's DEFINITELY worse to just compare before and after when we have strong reason to believe there are time trends (which we do). We didn't want to put too much weight on the assumed (counterfactual) linear time trend, though, which is why we interpreted our results as showing no evidence of a detrimental effect rather than as offering some evidence of a beneficial effect (which they actually did). Ultimately, we have 10 years of data in this study and there is nothing to suggest an appreciable increase in injuries post-CBA.
#33 by nat // May 10, 2018 - 2:01pm
Well, I'd disagree. It's always very bad form to derive a trend from four data points. It's even worse to project such a trend forward.
I'd agree that in the past injury rates have gone up. (Thanks for that link, btw. But it relies on the same 4 data points before the CBA to justify a steep trend line.)
But here's the rub: is there a plausible explanation for that trend that would justify extending it beyond 2010? Or were they simply four years that were bad enough to get the league to agree to changed practice rules in the new CBA?
Sometimes trends are due to accumulating effects (e.g. computer chip density improved by continuous R and D spending). Sometimes apparent trends have no ongoing cause, and thus give no insight into the future.
You have 10 years of data. Every single post-CBA year has a higher injury rate than the earlier years. You dismiss that by claiming that those increased injuries were caused something that existed pre-CBA, and still exists today in an every increasing amount.
Any theories?
#37 by Zach Binney // May 10, 2018 - 4:25pm
Well, first: if you don't artificially split the data into pre- and post-CBA periods, then there is a pretty clear linear increase over a full decade. So at least that's 10 data points, rather than 4. We have no particular reason to believe there would have been a linear trend before the CBA but not after the CBA unless, of course, the CBA eliminated the increase entirely. Our models account for that possibility - if you go into the full paper itself, they basically assume some linear trend at all times, but the line can be different in the pre- and post-CBA periods. It's an assumption, absolutely, but we felt it was a reasonable one. Certainly one that was better than simply comparing injury counts across two periods where we KNOW they aren't comparable.
Also, here's a full decade of data from Brian Burke showing a pretty strong linear increase in per-play injury risks before the CBA: http://archive.advancedfootballanalytics.com/2011/09/will-new-kickoff-rules-really-reduce.html. So that's 10 pre-CBA data points, which beats (but supports) my 4.
As far as theories for why injuries are increasing, there are quite a few of those. One is simply better reporting as interest in/scrutiny of the NFL continues to increase. There's a lot of evidence for this, though it wouldn't operate as strongly in the game-loss injuries that are the subject of this analysis. Another is gradually more cautious treatment of injuries as player safety becomes a stronger and stronger buzzword. Players are getting bigger/stronger/faster nearly every year, and that leads to more violent forces and thus injuries. Playing styles could be changing in a way that leads to more injuries. Whatever the reason(s), there's just overwhelming evidence that there is some sort of secular trend happening that needs to be accounted for.
I welcome other explanations in the comments!
#35 by Mountain Time … // May 10, 2018 - 3:43pm
If so, why would that be? It seems like four points where we have strong confidence in the numbers has real value, just a different kind of value than, say, eight or ten less points with less certainty. I'm wary of basing an opinion on a "seems like", but does this really have the weakness you say, and why?
#40 by nat // May 11, 2018 - 10:05am
Well, four data points means just three degrees of freedom (if I'm using the term correctly) with regards to a trend. As far as the certainty, look to the confidence interval bars.
In the actual data, the injury counts went up, then down a bit less, and then up the same amount. In the four points, the 95% confidence intervals(based on Poisson distribution, which seems a legit model choice) all overlap by at least half their ranges. That's a strong hint that we're looking at random variation, not a yet trend.
If it weren't just random variation, the up-down-up pattern is more indicative of temporary, unaccounted for factors, rather than a continuing trend. (Although four points is too little to support either idea, really.)
Anyway, in comment 37, Zach now says the trend was not determined from the four pre-CBA years. It was determined, we now know, from the full 10 years. In a way, that's even worse. The whole point of the analysis is to compare the two periods. But his 10 year analysis detected a rising trend precisely because the post-CBA years had more injuries. Such a trend calculation is only valid if your model assumes the two periods are subject to the same conditions, and that the conditions include some linearly escalating factor. The whole point of the CBA's injury-related changes was to change the factors giving rise to the pre-CBA injury rate and trend. You cannot validly assume that possibility away while at the same time claiming to test it.
Zach also mentions a separate earlier study which showed rising injury rates earlier in the century. Looking at that data, it really looks like a step increase in 2005, despite (or perhaps because of) a raft of rules changes intended to reduce injuries that year. 2000-2004 and 2005-2010 were both pretty flat. It only looks like a trend when taken together.
In a nutshell, the whole analysis is "if you assume the CBA had no effect on injury trends, then the CBA is shown to have no effect on injury rates."
#19 by fyo // May 08, 2018 - 2:00pm
No, the argument was never "the players need more free time". When the argument used to make a significant change turns out to be invalid, then that's a strong argument to reverse the change -- at least unless it brought something unexpected (positive) to the table. In this case, the reverse seems more likely. Considering the other initiatives directed at lowering injury rate, there's a real risk that these are actually masking any increase in injury risk due to the practice restrictions. Then there's the argument about the product on the field, where both players, coaches, and talking heads have argued that there is a negative effect.
#21 by Aaron Brooks G… // May 08, 2018 - 5:04pm
If the owners want it back, they'll have to give something up during the next CBA. It wasn't granted via largess -- it was a negotiated change.
Goodell's head on a pike, maybe?
\I like your appeal to subjective anecdote.
\\The argument goes that blocking and offensive line performance is degraded
\\\Passing yards, yards per pass, and net yards per attempt are at historical highs since the CBA
\\\\https://www.pro-football-reference.com/years/NFL/
#22 by fyo // May 08, 2018 - 6:28pm
It's certainly possible that the two sides cannot agree on undoing this change, but that's largely irrelevant to the arguments presented.
The "detrimental to football" semi-subjective bit is just an aside. That said, I would hope that most fans appreciate good play and not JUST high scoring.
#25 by Pat // May 09, 2018 - 4:06pm
"No, the argument was never "the players need more free time"."
The players didn't come in demanding less practices to lower injury rates. They came in demanding *less work*, period. Coaches/owners weren't about to give up games - even preseason games - but they would cut down on practices, so that's where the compromise came in.
The fact that injuries haven't really gone down isn't going to result in practice time *going up*. If anything, it'll result in a *further* decrease in practice time or possibly finally cuts in preseason games in order to satisfy player demands. It's unlikely owners would ever give up games at all, so that's probably off the board.
#39 by LionInAZ // May 10, 2018 - 5:34pm
Well... the NFL has been talking about shortening the preseason in favor of a longer regular season, so that has almost nothing to do with player demands. Hell, fans hate paying for meaningless preseason games.
My questions would be:
(1) Does the reduced practice reduce preseason injuries? I recall Mikel Leshoure's rookie season destroyed by an Achilles' tear caused by his own teammate (Cliff Avril) in summer camp. Conditioning issues are probably a bigger issue in preseason than regular season.
(2) Does reduced practice affect regular season play? Belichick's complaints notwithstanding, there's no evidence that teams perform differently than the sum of their player + coaching talent, as far as I can tell.
#31 by Zach Binney // May 10, 2018 - 1:14pm
The reason why I framed the article this way is a bit subtle, but let me try and explain.
We went into this analysis with an open mind that the CBA could have had a beneficial, detrimental, or no effect. When we plotted the data and ran our models, we saw SOME evidence for a beneficial effect, some for no effect, but nothing to suggest a detrimental effect. In the end, we were more confident concluding that there was no evidence of a detrimental effect rather than that there was no effect or a beneficial effect. We felt that was the most responsible conclusion. The full academic article has a fuller discussion of this issue, if you're curious!
I framed this (FO) article around Belichick being wrong because that part was just fun, but it matches up with what we say in the full academic version.