Writers of Pro Football Prospectus 2008

Most Recent FO Features

JohnsonAnd03.jpg

» Factors: Andre Johnson

One of the NFL's best receivers notched a -2.3% DVOA last year. Does a target-by-target breakdown show he was better than that?

23 Sep 2003

Week 3 Team Efficiency Ratings

by Aaron Schatz

Eventually, the cream rises to the top.  This week, after dismantling Atlanta's offense, the defending Super Bowl champion Bucs move into the top slot on our team efficiency rankings, measured by our proprietary VOA system that breaks down every single play and compares a team's performance to the league averaged based on situation in order to determine value over average.  (Explained further here.)

I spent some time this weekend playing with the VOA system and tweaking it to improve accuracy.  This week's ratings use what I'll call "VOA version 2.0," the first in what will hopefully be several improvements to the formula.  The simple explanation is that a first down play now needs five yards instead of four to qualify as successful, and second down success is also slightly harder to get.  The complicated explanation for those interested in the math, and correlations to show why this version is a slight improvement, look to the bottom of this page.

The system recognizes that the Carolina win over Tampa Bay was a fluke caused by special teams -- and not even normal ol' special teams plays like good punt returns, but very rare special teams plays.  VOA also recognizes that San Francisco is still one of the best teams in the league despite losing two very close games.  49er fans, however, are reminded of this chart that accompanied our article on the Pythagoran Theorem and coaches, showing that Dennis Erickson has a record of losing more games than his team's performance would otherwise indicate.  What's Dennis Green up to these days, anyway?

To get away from objective numerical analysis, what the heck is wrong with the Raiders?  I wrote in our season preview, "Every year I think they will get old, and every year they don't."  Apparently, I was wrong.  This year, they got old.  To my eyes, however, the problem was not Jerry Rice or Tim Brown. The problem was the offensive line and the secondary.  The offensive line was constantly overwhelmed by Denver even though the Broncos were rarely rushing more than four men.  Gannon was constantly pressured, and on rushing plays Charlie Garner went nowhere.  The secondary was even worse.  I'm not sure I ever saw a Raider within five yards of Ed McCaffrey or Ashley Lelie on any pass.  The only time a Raider got close to a Bronco receiver was when Rod Smith almost decked one of the Raiders, missed, and got a ref instead.

For this first time this week, I did some partial figures on opponent adjustments.  After three games, I don't feel comfortable making them "official" by printing them here, but I will note that based on very, very early numbers, Cleveland and St. Louis are have played difficult schedules and are better than their VOA numbers, while Miami is a bit overrated by VOA.  Let's be honest -- there are losses, and then there are losses to Houston.

Opponent adjustments also move Tampa Bay down, making Seattle the #1 team, but the reason here is also the reason why it is too early to give "official" opponent-adjusted numbers.  With only three games, a dominant team like Tampa Bay creates a feedback loop.  Tampa Bay gets penalized for playing three horrible passing offenses, Atlanta, Philadelphia, and Carolina.  But those teams all register as horrible passing offenses because they have played Tampa Bay.  Atlanta, in particular, was an average passing offense until they got pasted by the Bucs this week.  We need to wait for those teams to play a few more games against non-Tampa pass defenses before we can judge their true quality.

Finally, let's talk about the Jacksonville problem.  Despite being 0-3, the VOA system still thinks they are an average team.  This week, they lost to Indianapolis 23-13, but VOA says the game was very close (Colts +3.4%, Jags -3.4%).  Jacksonville also came out higher in our 2002 VOA numbers than their record should indicate, so what's the problem here?  With an offense ranked #5 and a defense ranked #25, you would probably think the VOA system is missing something wrong with the Jaguar offense.  After all, isn't Marc Brunell over the hill and close to being replaced?

But, no, the problem is not the #5 ranked offense.  The problem is the #25 ranked defense.  That ranking may be too high.  Early on in 2003, they are having major problems in one very important place and at one very important time.  First of all, they are horrid in the red zone.  Jacksonville's opponents have a +90% VOA in the red zone, but only +5% VOA on the rest of the field.  Second, they are having a harder time on third down.  Jacksonville's opponents have +7% VOA on first and second down but +30% VOA on third down.

So this brings up two questions.  First, will these tendencies stay consistent throughout the season, or will they eventually even out a bit.  I honestly can't answer that question.  Second, shouldn't the VOA formula be tweaked to give more importance to success or failure on third down and in the red zone.  Based on the tests I've done so far, yes and no.  Success on third down probably does need to be more important -- so far I haven't figured out the right way to do it, but that's probably coming in "VOA version 2.1."  On the other hand -- and I know this sounds counterintuitive -- tweaking the formula to give more weight to red zone success or failure does not seem to make VOA numbers correlate better with either wins or points.  I'll have to play with that more, but as of now I would have to say that Jacksonville's red zone defense is likely to improve so that is more in line with the team's defense on the rest of the field, and this will result in some wins, at some point.  If Arizona and Houston can win games, surely Jacksonville can win sometime. 

As always, positive numbers represent more points so DEFENSE is better when it is NEGATIVE.  Here are the ratings through Week 3:
 


TEAM
TOTAL
VOA
LAST WEEK
RANK
OFFENSE
VOA
OFFENSE
RANK
DEFENSE
VOA
DEFENSE
RANK
1 TAM 72.8% 5 15.1% 4 -57.7% 1
2 SEA 72.7% 1 27.3% 1 -45.4% 2
3 MIN 59.3% 3 26.3% 2 -33.0% 5
4 KAN 48.2% 6 13.1% 6 -35.2% 4
5 SFO 47.3% 2 11.9% 7 -35.4% 3
6 DEN 28.3% 9 10.6% 8 -17.7% 11
7 PIT 25.2% 12 -1.2% 14 -26.5% 6
8 IND 22.6% 7 8.6% 9 -14.0% 12
9 BUF 21.4% 4 -4.8% 17 -26.2% 7
10 DAL 14.5% 11 3.9% 11 -10.6% 13
11 WAS 12.8% 8 18.5% 3 5.8% 21
12 NWE 8.2% 13 -10.2% 19 -18.4% 10
13 OAK 1.5% 10 3.7% 12 2.2% 17
14 JAC 1.1% 14 13.6% 5 12.5% 25
15 GNB -1.5% 16 -4.1% 16 -2.6% 15
16 NYG -1.7% 20 -1.9% 15 -0.2% 16
TEAM
TOTAL
VOA
LAST WEEK
RANK
OFFENSE
VOA
OFFENSE
RANK
DEFENSE
VOA
DEFENSE
RANK
17 MIA -7.1% 25 -12.9% 21 -5.8% 14
18 CLE -7.4% 23 -26.0% 27 -18.6% 9
19 BAL -8.8% 18 -29.9% 29 -21.1% 8
20 NYJ -11.7% 19 7.1% 10 18.8% 28
21 TEN -13.3% 26 1.2% 13 14.5% 26
22 CIN -19.4% 17 -16.4% 22 3.0% 19
23 HOU -19.8% 15 -17.1% 23 2.7% 18
24 STL -24.8% 24 -19.4% 25 5.5% 20
25 SDG -29.3% 27 -8.7% 18 20.7% 29
26 NOR -30.3% 21 -19.1% 24 11.2% 24
27 DET -40.0% 28 -24.4% 26 15.6% 27
28 ATL -42.2% 22 -32.8% 30 9.4% 23
29 ARI -43.8% 31 -10.4% 20 33.5% 31
30 CAR -50.8% 29 -29.1% 28 21.7% 30
31 PHI -59.1% 30 -50.0% 31 9.2% 22
32 CHI -110.2% 32 -73.2% 32 37.0% 32

*Further VOA version 2.0 explanation for the numbers-hungry:

Using the old VOA formula, 48% of first down plays came out as "successful," 44% of second down plays came out as "successful," and only 39% of third down plays came out as "successful."  I decided to try tweaking the formula so that the percentage of "successful" plays was equal for all three downs.  Since 48% is about one-fifth higher than 39%, I decided to move the threshold for first down success from 40% of needed yards to 50% of needed yards.  I also moved the threshold for second down from 60% of needed yards to 65% of needed yards.

Changing this formula improves the correlation coefficients for 2002 data slightly, but it was enough that I decided to apply the changes to this season's data.  For now, the 2002 data elsewhere on the site continues to be VOA version 1.0.  Which actually isn't version 1.0, since it already includes one change, moving the value of a turnover from the -4 in Hidden Game of Football New Edition to -8.  This is actually the second change.  But who's counting?  The major changes in the 2002 numbers with the new VOA formula, by the way, include downgrading Minnesota to a position more in line with their 6-10 record and moving Philadelphia closer to Oakland and Tampa Bay at the top of the rankings.

Here is what the correlation coefficients look like with version 1.0 and version 2.0:

 


  VOA 1.0 VOA 2.0
correlation to 2002 wins .841 .846
correlation to 2002 point differential .921 .928

By the way, VOA version 2.0 also makes the good teams look better and the bad teams look worse.  So while the extremes of best and worst VOA should move towards 0% as the season goes on and we have more plays to count in our ratings, changing to version 2.0 this week means that the teams are still pretty spread out.

Posted by: Aaron Schatz on 23 Sep 2003

comments