Writers of Pro Football Prospectus 2008

24 Dec 2013

Correction: FO Playoff Odds (UPDATED)

We would like to apologize for last week's ESPN Playoff Odds Report, which contained numerous errors.

What happened? We wanted to make sure that when Danny Tuccitto wrote the article, he had the proper equation that the playoff odds report uses to figure out the odds for each game. Unfortunately, when I sent him the equation, I apparently sent the wrong one, something we had used in a different simulation in the past. Therefore, in last week's Playoff Odds Report, some of the single-game probabilities were incorrect. (They were generally slanted way too heavily towards the favorite.) The odds given for two-game scenarios (i.e. what appeared in the Playoff Odds Report here on FO) were correct.

Because the Monday game between Detroit and Baltimore was so important for playoff odds, we didn't end up writing the article until really late. Rushed for time, neither Danny nor I did our due diligence, which means neither of us noticed that the single-game probabilities did not at all match up with the playoff odds. That's on us, huge mistake, and we apologize. We're watching to make sure this does not happen in the future. This week's Playoff Odds Report at ESPN Insider was checked extra to make sure the numbers agreed with each other properly. (You can read that article here.)

We don't like to make mistakes, of course. We're human, and it happens, but we're not happy about it. In the future, if you discover that we've made a multiplication error in one of our articles, the best way to let us know is through e-mail to mailbag-at-footballoutsiders.com. Commenting in the discussion thread on Football Outsiders will sometimes get our attention, but not always. We only recommend using Twitter if you don't expect a response in return.

Speaking of Twitter, one thing I've learned these past few days while dealing with this issue is that 140 characters is simply not enough space to respond properly to criticism without seeming snarky and unpleasant. Therefore, in the future, I will not be responding to criticism over Twitter. If you just need to bring our attention to a small error, then sure, use Twitter. However, if you feel that a larger piece of analysis is subpar or want further explanation of something I have written, please ask your question over e-mail. You can also use a discussion thread, although I am less likely to respond. However, with either e-mail or a discussion thread comment, I can give a response that properly respects the reader's point of view without a character limit that forces incomplete responses that often read as obnoxious.

(OK, let's try this again... There seems to be a lot of misunderstanding about what I'm asking for here. I have no problem with being criticized on Twitter. The problem is that people criticize us on Twitter, then criticize us a second time because they feel the response is too short or too obnoxious. But that's the thing about Twitter... it's short and obnoxious. I was upset because the Twitter response to our mistake in this article was so negative, but then someone pointed out that my own response to someone's criticism of us in the past had been equally snarky and negative. I don't really want to be like that. If people have legitimate criticism that deserves a well thought out response, I want to give a well thought out response. And so please, by all means, criticize us publicly on Twitter when we make mistakes. What I'm saying here is that if you want to actually get a response to your criticism longer than just "whoops, we'll go fix that, thanks for letting us know" then Twitter is really not the way to go.)

Thanks again for understanding, everyone. As always, we're doing our best around here.

UPDATE: There is now a corrected version of the article posted at ESPN Insider.

Posted by: Aaron Schatz on 24 Dec 2013

47 comments, Last at 23 Jan 2014, 8:32am by Cheap Fitted Hats


by Treima :: Tue, 12/24/2013 - 3:09pm

The Twitterverse weeps for its loss. No longer will "@FO_ASchatz GTFO STOOPID CHEATRIOTS FAN I CALL BIAS #GEAUX #SAINTS #WHODAT" be considered worthy of a reply.

The rest of us who follow FO (and surely your sanity) are celebrating, I reckon.

by Adam2 (not verified) :: Tue, 12/24/2013 - 3:24pm

Unfortunately, Treima, the criticisms which spawned this correction aren't anything of that ilk...

This is embarrassing, Schatz.

by Treima :: Tue, 12/24/2013 - 9:49pm

Getting self-righteous over a miscalculation in playoff odds is a bit more embarrassing, in my opinion, but to each their own.

by Adam2 (not verified) :: Wed, 12/25/2013 - 1:16am

I don't think anybody but Tuccitto has been self-righteous throughout this process? Maybe he can bring up his meaningless credentials next time he writes for Insider, tho.

FO sells numerous products that are focused on predicting the performance of players and entire football teams... and they won't offer a correction to ESPN because they don't want to bring too much attention to how badly one of their writers botched an article? The charade above from Schatz is cute (multiplication error LOL), but its turn into a personal attack and the focus on discouraging public criticism of FO is quite atrocious from an accountability perspective. I guess it's more important to protect that brand than be wholly honest with their revenue streams?

This is all quite hilarious. #teamAGENDA if you call out a tour operation, apparently.

by Adam2 (not verified) :: Wed, 12/25/2013 - 1:20am

*tout operation.

I'll quickly admit to and rectify the error in my writing, since I'm not as sensitive as the FO writers.

by Duff Soviet Union :: Wed, 12/25/2013 - 6:58am

"The charade above from Schatz is cute (multiplication error LOL), but its turn into a personal attack and the focus on discouraging public criticism of FO is quite atrocious from an accountability perspective."

Um, what? Who exactly is this a personal attack on? How are they discouraging public criticism?

by Karl Cuba :: Wed, 12/25/2013 - 6:20pm

The made a mistake, they then apologised. What's your problem?

by SB_2000 (not verified) :: Tue, 12/24/2013 - 3:30pm

The real issue here is, how could a professional football writer and editor not recognize that Chiefs couldn't possibly be 19-1 favorites over another playoff team?

Also, why is this correction running here, instead of at espn.com where the original incorrect article ran? Really unprofessional behavior.

by Anonymous Jones :: Tue, 12/24/2013 - 4:32pm

I wish you a long enough life so that many times you are contemptuously asked a variant on the question: "How could you [a ________] do [what was one of the most careless mistakes of your life]?"

I mean, I'm guessing (off a small sample size, of course) that it won't take you that long to achieve this distinction (I'd bet you're living in a glass freaking mansion at this point), but however long it takes, I wish it for you. So, you've got that going for you...

by CaffeineMan :: Tue, 12/24/2013 - 10:03pm

...which is nice.

(and I agree with you, especially since the person asking that ridiculous question usually doesn't know diddly about being a [_________].

by speedegg :: Tue, 12/24/2013 - 4:48pm

Going to miss the comical replies on twitter. Not your response to criticism since it's factual, but everyone else's replies since they're getting their virtual feathers ruffled. Man, that is/was high comedy.

by Danny Tuccitto :: Tue, 12/24/2013 - 5:42pm

Since that article had my name on it, I just want to add a little to what Aaron said...

First, I'd like to reiterate the fact that I am a human being. Like everyone else, I make mistakes. Between Niners Nation, FO, Football Guys, and ESPN, I've written hundreds of columns over the past five years, and I can guarantee you there have been mistakes in many of them. I don't have to Google this to know that there have been typos, grammatical errors, brain farts in calculation, overlooked flaws in my methods, etc. I also don't have to Google any other football stats bloggers (or watchers of Las Vegas) to know that those things have appeared in their columns as well -- even after going through the editing process. And as Aaron said, this isn't to excuse mistakes or find them acceptable; they just, well, happen. To me, the biggest personal irony in this whole episode is that, behind the scenes, I'm notorious for being a perfectionist -- Aaron and Rivers just nodded in agreement while reading that -- but yet here I am being excoriated for simple mistakes in one of the few articles where I didn't have the time to be as perfectionistic as I habitually am. As everything was going down last week, my emotions weren't tied to any of the unnecessary vitriol that was being thrown my way; they were internally focused on how much I had failed myself, and how shameful I felt that a little more of something "so Danny (i.e., perfectionism)" was all it took for this to never have happened.

On that issue of unnecessary vitriol, though -- which, yeah, I've been called worse than a hack who can't do simple math or recognize outliers, so no biggie -- another really disconcerting thing to me about what happened is related to Aaron's point about the right way and the wrong way to alert us about mistakes. As any regular reader will notice, almost every comment section includes someone (or many people) pointing out mistakes, large and small. On the more stats-intensive pieces, people (often times vehemently) convey their disagreements with our methods, conclusions, what have you. Everywhere I've written, this is the normal state of affairs, and reader-identified corrections are a valuable asset for any website. I'm sure some have noticed over the past few years that I jump into the comments more often than others. When you say something to me, I'm almost sure to see it, and pretty likely to respond. Therefore, it totally escapes me why on earth a respectable site such as Vegas Watch, which I have written really nice things about in the past (http://www.ninersnation.com/2010/9/8/1676865/in-accuracy-rankings-for-nf...), would feel the need to take their grievances to Twitter, reject my cordial -- downright jovial! -- responses (https://twitter.com/FO_DTuccitto/status/414141229501149184), and make sure to alert as many people as possible with a direct line (or a potential line) to my livelihood. All of that negative effort in service of a belief that I can't do basic math or recognize outliers, and without the knowledge of what was going on behind the scenes at the time? Now, to Vegas Watch's credit, they did correctly identify a deeper issue than "Tuccitto is bad with numbers." And again, it was a great boon to us that they did, as we were able to correct the problem in time for today's column. Still, why proceed in the manner they did? If you're going to use Twitter to alert us, that's perfectly acceptable to me. This isn't about Twitter, per se. It's about the idea that you attract more bees with honey than vinegar. If the purpose of alerting us is to get the mistake corrected, why obscure that positive intent behind thinly veiled negativity? I suppose, hey, it worked; we fixed the errors. But, speaking for myself, it just seems to have been all so unnecessarily mean-spirited. I constantly struggle with figuring out the proper way to address negativity on the internet when writing about NFL stats requires me to be here on a near-daily basis. Over time, I've softened with age, leaning more towards the "ignore it" end of the spectrum; but situations like this make me wonder.

Finally, because this whole kerfuffle ended up reaching the computer screens of people I respect and who may end up doing business with me in the future, the most important thing I'd like to achieve in this commentary is to clearly and unequivocally state the following: The only reason I am where I am in the football stats community -- which isn't very high on the totem pole, to be sure -- is because I have done and continue to do good work. I've written up many research projects investigating a variety of NFL-related topics. I've invented a statistic or two (e.g., snap-weighted age, which is now a perennial feature in FOA). I was one of the technical editors (i.e., the perfectionist who triple-checks every topic-relevant fact to make sure they're correct) on the recently released book, Newton's Football. I've presented an NFL research paper at Harvard University. I got a relatively advanced statistical analysis past my graduate advisory committee, and several others past the review boards of academic journals. I contribute to the site that pioneered football stats writing on the internet. These (and others) are accomplishments that I'm proud of, and I hope there are many more to come in the future. But make no mistake: I don't bring them up here to pat myself on the back; thousands of people have accomplished far more than I have in this field. I don't bring them up to imply that I'm better than you. I don't bring them up to suggest in any way, shape, or form that my work is beyond scrutiny. (In fact, I welcome it!) No, I bring them up because I'll be damned if I'm going to allow another person -- armed only with a handful of honest mistakes, a moderate number of Twitter followers, and an overwrought sense of outrage -- to besmirch the reputation that I have worked very hard to earn over the past decade or so. I really wish I didn't have to do it, and before this week I never dreamed I would need to, but there comes a time when one needs to defend oneself.

It's Christmas Eve, so I won't be around all that much over the next couple of days, but I'll make sure to respond to as many polite comments or questions as I can. Happy holidays, everyone. Thanks for taking the time to read this, and thanks for your continued support. It's greatly appreciated.

by PeterJMoss :: Tue, 12/24/2013 - 6:47pm

Your response is appreciated and respectful, although I will point out that this site and most sports analysis websites spend tons of times ripping on coaches/management mistakes who I'm sure could point out similar rebuttals. I'd just own it - say I messed up - and let it pass.

by Danny Tuccitto :: Tue, 12/24/2013 - 6:59pm

Thanks. And I'm aware of the phenomenon you're referencing. Speaking only for myself, I'll say that I wrote about it on Twitter when this whole thing went down last week (don't know how to link to entire series of tweets, but it's on my timeline for Dec. 20). Basically, I've learned over time that ripping people you don't know isn't right because you can never be in their shoes, and know all the reasons why they did the thing you vehemently disagreed with.

And besides, getting apoplectic about things too often gives the really worthy instances less bite. Boy who cried wolf, so to speak.

by After the Crash (not verified) :: Tue, 12/24/2013 - 7:32pm

I'll also point out that at least you guys make the correction and issue an apology fast. I have help desk tickets that have been open for days (sometimes a week or more) and nothing happens.

by trex5755 (not verified) :: Tue, 12/24/2013 - 7:54pm

Thanks: Will there be a correction posted on espn.com so that people misled by the original article will be informed of the errors?

by Danny Tuccitto :: Wed, 12/25/2013 - 12:27am

I'm not sure that's necessary or practical considering that a) we're now over a week removed from it so it's doubtful anyone will be returning to the column in question, and b) there's considerable crossover between the readership of our Insider articles and the readership here. That said, if ESPN approaches us insisting on a correction post of some kind, I'd be happy to do it. And I'd also be amenable to posting in this thread what the probabilities should have been according to the correct formula. I suppose there might be people out there tracking these things (perhaps for an accuracy analysis) whose data set would benefit from not having errors in it.

by Dan k (not verified) :: Wed, 12/25/2013 - 10:23am

Danny, as a former journalist and editor for 12 years, I find it offensive that FO is not notifying ESPN that the story needs corrections. Mistakes happen. Ive made plenty in my career. Some big ones. And as embarrassing as it was to admit to the mistake, when I was called out on it, I always wrote a correction.Your arguments for reasons why the article shouldn't be corrected are not valid. One cannot base their justification for not publishing a correction on the belief that nobody will ever view the article again or that anyone who had read it will also read this page. The lack of editorial integrity being shown here, presumingly to avoid further public accountability, is laughable and shameful. I am going to notify ESPN to ask for a correction - surely it has a policy around amending inaccuracies in its stories, especial ones that cost money to read - and I urge others reading this to do the same. Thank you.

by trex5755 (not verified) :: Wed, 12/25/2013 - 11:23am

So you're just totally fine with an article riddled with basic errors being left up uncorrected on the most popular sports website in the world (by orders of magnitude) under your name?

by Theo :: Wed, 12/25/2013 - 8:01pm

So you find out that mistakes were made but you don't at least correct them?
That's... odd.

by Danny Tuccitto :: Wed, 12/25/2013 - 9:12pm

Guys, I'm going to go ahead and put the corrections together, and send them to ESPN. When they go live there, we'll notify everyone and cross-post them on FO. Sound good?

by andrew :: Tue, 12/24/2013 - 10:08pm

I note that in the article where Ninersnation points out the FO innacuracy of their 2009 projection, it singles out the FO 5-10-1 projection for the "NFC Championship game host Minnesota Vikings."


by Danny Tuccitto :: Tue, 12/24/2013 - 10:25pm

Hah! Awesome catch! I actually make that mistake to this day. For some reason, my memory has never been able to assimilate that that game was in New Orleans. Must be something about both being dome teams, Minnesota blowing an NFC championship game during my peak memory years, and the niners fan in there wanting to believe Favre's choke-job came on his home field.

by Djg (not verified) :: Tue, 12/24/2013 - 7:29pm

schatz can u show us where u said whoops sorry as u claim? All I see is u calling VW an ahole and Danny saying who cares. Thx

by Danny Tuccitto :: Tue, 12/24/2013 - 9:22pm

Serious question. Can you expand on me saying who cares? In the twitter conversation I linked to, it was VW who literally started the back and forth with #WHOCARES. When I said it back to him at the end, I was pretty transparently joking around in a third or fourth attempt to make him less confrontational. And in my post above, I think talking about my perfectionism by definition means I actually care more than most people about not making mistakes. Maybe you mean I'm coming across as thinking that the drama, outrage, and attempt to smear my reputation is undeserved considering the innocuous nature of my "crime?" If that's it, then yeah, I wholeheartedly hold that view.

by Anonymous42 (not verified) :: Tue, 12/24/2013 - 8:38pm

The bottom line you both thought, even if for only half a second, that 19-1 odds was an accurate projection. Any long-term football analyst should immediately balk at that number, even if seen out of the corner of the eye after a couple beers during a half-ass skim. If you have to be paying full attention to catch this magnitude of error, one must wonder how qualified you are to analyze football. Sounds harsh, but it is the truth.

by Theo :: Tue, 12/24/2013 - 10:55pm

Seems like some people take some things waaaaay to serious.

Merry xmas all. All the best.
Humans make mistakes, if someone doesn't get that, then that's their problem.

by RJ Bell (not verified) :: Tue, 12/24/2013 - 11:42pm

I think that Aaron does great work, and his Premium Picks have really turned a corner. Danny seems like a solid guy, as well.

All the best,
RJ Bell
Founder/CEO, PreGame.com

by Dre (not verified) :: Wed, 12/25/2013 - 5:13am

Well at least he got backing from a well-adjusted, reputable source....

by Danny Tuccitto :: Wed, 12/25/2013 - 9:18pm

Thanks, RJ.

by johnlimberakis :: Wed, 12/25/2013 - 11:22am

I'm in data analysis and I understand what can happen when a simple report gets messed up that everyone relies on. How you handle it is just as important as the mistake itself.

The important thing is to not beat yourself up about it. I'd also think hard about your public interactions with people. That's your brand. You need to protect your brand. Flippant or rude responses are not the way to endear yourselves to people. Now I didn't engage you over any of this via twitter and I am a huge fan of FO. I've emailed you before thanking you for the work you've done.

Be professional, be apologetic, and move on. In the grand scheme of things this is a minor speed-bump. Don't let minor speed-bumps derail your ascent to the top. Keep up the great work guys.


John Limberakis

by Danny Tuccitto :: Wed, 12/25/2013 - 9:18pm

Thanks, John.

by Vague (not verified) :: Thu, 12/26/2013 - 7:55am


Happy Holidays. I work in the US Military. Occasionally mistakes are made that endanger lives. 99%(or more) of the time with good oversight we catch and correct mistakes, before they actually due so. Sometimes the answer is an additional layer of oversight. Sometimes that is unnecessarily cumbersome.

If I may there appear to be two seperate issues; a bad article that a major company paid for, and some public relations problems stemming from that article. There will be sites and folks who are jealous of what those of us, who dont know, assume to be a generous relationship with ESPN. They will want that contract.

The mistake may signal its time to have a pure Quality Control staffer who insures this does not happen in the future.

As FO continues to rise in profile (as it should) there will be a time when a public relations staff because necessary. This might be a sign you are moving in that direction. I do NOT suggest you decrease your consumer interactions as they do greatly improve your product.

I just identified two hires which may not be financially viable. Perhaps in future negotiations with ESPN or other large corporate suitors you get them to provide compensation for exactly these needs.

Football Outsiders has grown a long way since its early days of loving 9-7 Andy Reid teams. This appears to signal necessary organizational growth.

Ben Pealer

by MC2 :: Thu, 12/26/2013 - 11:43am

Normally, I try not to nitpick people's comments, but the irony here is just too rich to resist.

I counted seven mistakes (spelling, punctuation, etc.) in your comment, so perhaps you are the one in need of a Quality Control staffer to edit your comments.

Might I suggest Raiderjoe?

by andrew :: Thu, 12/26/2013 - 10:27pm

The thought of having Raiderjoe do PR work for FO... heh.

by Anonymousse (not verified) :: Thu, 12/26/2013 - 11:25am

"We don't like to make mistakes, of course. We're human, and it happens, but we're not happy about it. In the future, if you discover that we've made a multiplication error in one of our articles, the best way to let us know is through e-mail to mailbag-at-footballoutsiders.com."

Maybe if you stopped doing this shit by hand, or in Excel, and then copy/pasting you results, and invested in doing it the right way, you'd stop having mistakes every week.

The constant human errors undermines ALL of your work.

by Aaron Schatz :: Thu, 12/26/2013 - 1:12pm

Unfortunately, investment takes money, and I often have to choose between investment in Football Outsiders and doctors/therapists for my daughter. It's not a hard choice.

by Cregg Schinckle (not verified) :: Fri, 01/03/2014 - 11:51am

Did you actually write this or did someone post it to shame you? #itsnotahardchoice

by Johnny Toogod (not verified) :: Thu, 12/26/2013 - 3:48pm

Wow this site comes off like a bunch of whiny bitches. You messed up. Say you are sorry, correct the article and move on. Dont get mad at commentators or other people because YOU screwed up.

Your response, coming from a reporter who has made a mistake or two, is embarrassing. I will no longer be reading here. I mean, who knows if anything is accurate anymore since you were planning on leaving an INCORRECT article up as is.

A disgrace to the football stats community. ESPN should look in another direction for stats. This display has been completely unprofessional.

by Duff Soviet Union :: Thu, 12/26/2013 - 7:06pm

You'll be missed.

by LionInAZ :: Thu, 12/26/2013 - 9:39pm


by max :: Fri, 12/27/2013 - 11:28am

I don't believe the Rams have only an 87.2% chance of getting a top 3 pick. How did you arrive at that number?


by Hurt Bones :: Fri, 12/27/2013 - 11:49am

Doesn't seem that unrealistic. The Redskins play the Giants. There are 5 teams at 4 -12. I haven't figured out all the strength of schedules, but all it takes is two of the 4-12 with weaker strength of schedules to lose and Washington to win and the Rams get the No. 4 Pick.

by max :: Fri, 12/27/2013 - 5:33pm

I agree that it doesn't seem unrealistic if you don't do any SOS analysis, but it has been done and it's been reported that the likelihood of the Rams drafting worse than 3rd overall is practically nil.

All that's really required is to run all the combinations of week 17 results to determine every possible SOS outcome. I wonder if FO has done that. I doubt it.


by Bailey Button Boots For Kids (not verified) :: Tue, 01/21/2014 - 10:00am

I just now had to suggest thanks one more time. I do not know the things I was able to possibly need created not having the solutions suggested by most people over many of these area. It's actually an awfully intimidating matter with my view, but the capacity to see a good solid skilled methodology you medicated that developed me towards leap with the help of fulfillment. I

by cheap Cleveland Cavaliers Jerseys (not verified) :: Tue, 01/21/2014 - 10:00am

I don

by Cheap Fitted Hats (not verified) :: Thu, 01/23/2014 - 8:32am

I just now had to suggest thanks one more time. I do not know the things I was able to possibly need created not having the solutions suggested by most people over many of these area. It's actually an awfully intimidating matter with my view, but the capacity to see a good solid skilled methodology you medicated that developed me towards leap with the help of fulfillment. I