Introducing “Datecdotes”, When Streaming Companies Use Data to Win the PR Wars

Here are some fun stats. What do they tell us?

– Netflix over the summer had 80 million customer accounts watch one of their Netflix Original Romantic Comedies.

– Netflix had 20 million streams for The Christmas Chronicles over the last weekend.

– Amazon Prime/Video/Studios had 14.7 million total customers watch an NFL Thursday Night football game.

– Snapchat had over 10 million viewers watch a Snapchat Original show this year.

At first blush, that’s a lot of data. And it’s big! You know, in terms of size, in that 80 million sure is a lot of people.

But let’s count the actual numbers released. One. Two. Three. Four.

Four numbers is not “big data”, in the data science sense. Data doesn’t get “big” until you reach the hundreds of thousands of data points. In fact, some data scientists would say data doesn’t really get big until you have millions of data points with many, many categorical variables.

Alas, as we ponder the bare handful of data points above, if we really pause to think on them, we understand how little we’re being told. Take the journalism “Five W’s”, who, what, when, where and why. Most data can’t tell us the why—it’s implied—but in streaming video it can tell us the other four.

When streaming video companies release single data points, they usually only give us two of the five W’s. First, they give us the “who”—customer accounts, customers or monthly active users. And they give us the “where” in the broadest sense possible in that they give us the “global numbers”. But crucially they always omit the “what”. How many minutes were viewed per person? The “when” is also usually implied, but not explicitly stated, usually so that the numbers are as large as possible. In the case of The Christmas Chronicles, they gave us the “what”, but left out the why.

As a result, usually we can learn very little as competitors, observers or investors from these nuggets. A contrarian might say, look here, Entertainment Strategy Guy, you said in this very early article that you LOVE data. At least these companies are providing us some data.

Well, I’ll dust off a great quote from statistics to counter that,

“The plural of anecdotes is not data.”

Netflix, Amazon and Snapchat—who are just the three companies I’m picking on today, Twitter, Facebook, Twitch, Hulu and Youtube do this too—aren’t providing data, they’re giving us anecdotes. Selectively curated data-based anecdotes in the hopes—that are almost always granted—that unsuspecting and unquestioning news outlets will repeat to boost their perception among customers, Wall Street and competitors.

And we always fall for it.

See, the companies above aren’t choosing between one or two data points. Or even a couple of dozen. These companies are literally choosing between millions of potential data points, which make these numbers some of the most selective anecdotes you could possibly come across.

The analogy (and yes it is in the title) is the old saw about the iceberg. 10% of the ice floats above the sea, with an even larger 90% below the water. This is how it feels when a streaming company drops their knowledge on us.

Slide1

With streaming video, the numbers are even more extreme. They have millions of customers watching tens of thousands of videos with at least a dozen or more categorical variables per interaction. We’re talking thousands of potential ways to meaningfully slice the data, and the companies pick one or two per quarter. Again, the plural of anecdotes isn’t data.

Slide2

The line is so close to the top of the iceberg, it may as well not even be touching it. That’s how much data we don’t have access to.

I have a new name for this. Even if you have a data point, that still isn’t “data”. It’s an anecdote. It’s a “datecdote”, an anecdote of data. Interesting, but not enough to base decisions off of.

Netflix, we’ve been told, isn’t an entertainment company, they’re a product company that leverages huge amounts of data to deliver us our entertainment. Maybe that’s true, for internal work. But when it comes to PR? Netflix isn’t a data company. They’re an anecdote company. They’re a datecdote company.

I’ve spent a lot of the last week polishing an article digging deep into the second most recent Netflix datecdote. My main conclusion is that at conferences or on investor calls or when choosing to publish press releases, as journalists we need to push back. We need at least the five W’s, and we need at least comparisons to put these datecdotes in context. Without those, and this is controversial, we just shouldn’t publish their number. I’m realistic enough to know this won’t happen, but we’d know a lot more if we did.

  1. […] be that no one is really lying, but that streaming video numbers can be really, really misleading. With hundreds of metrics on millions of customers that can be sliced thousands of ways, Netflix has a lot of options to present a data story that is extremely positive to them, while […]

    Like

    Reply

  2. […] of the keys to datecdotes is to limit the number of categories you release. Essentially, the most important two numbers are customers and then the hours they spend watching a […]

    Like

    Reply

  3. […] are my thoughts on Netflix raising prices? Well, my rule of thumb is if I write 2,000 words on something, it becomes its own article. So […]

    Like

    Reply

  4. […] Narratives are created by weaving together anecdotes and reasoning from first principles, sometimes using statistics as your anecdotes. Good data analysis is the antidote to bad narratives. The problem is that data analysis is hard to […]

    Like

    Reply

  5. […] Since I’ve written about their data and even coined a phrase about how selectively they pull it (read here for “datecdotes”), I naturally had the most questions for Netflix, and they convinced me to finally write an […]

    Like

    Reply

  6. […] I regularly admonish my readers to avoid overhyping single data points. I repeat this piece of guidance when things flop—Solo, The Lego Movie 2, NFL Ratings last season—or when they go skyhigh—most Disney movies, Netflix datecdotes. […]

    Like

    Reply

  7. […] While you won’t find a bigger “back of the envelope” guy than me, the problem I repeatedly run into is that streaming back of the envelope math often features a basic accounting error. Because of this, commentators routinely overestimate the value of individual shows to streaming networks. (And yeah, this is mainly aimed at Netflix and Amazon datecdotes.) […]

    Like

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: