If you were to ask anyone in the entertainment industry—or even non-Hollywood types—what it is that makes Netflix more successful than any other company, the answer is fairly simple:
They have an algorithm.
But not any algorithm. THE algorithm. The “almighty algorithm”. An algorithm that has so much power, so much knowledge, wisdom and foresight, there’s hardly anything it can’t do. It tells Netflix what movies they should greenlight, what shows they should cancel, and what talent they should hire.
It even gives notes during the development process! “The almighty algorithm knows how to compose a genre film designed to draw in the maximum amount of subscribers…”
Here’s Cary Fukunaga, speaking to GQ, in perhaps the purest distillation of belief in the almighty algorithm:
“Because Netflix is a data company, they know exactly how their viewers watch things,” Fukunaga says. “So they can look at something you’re writing and say, We know based on our data that if you do this, we will lose this many viewers. So it’s a different kind of note-giving. It’s not like, Let’s discuss this and maybe I’m gonna win. The algorithm’s argument is gonna win at the end of the day.”
And it’s evil. As Hannah Gadsby posted on Instagram last fall during the Dave Chappelle debacle, “F*** you and your amoral algorithm cult.” Perhaps the weirdest example is the fact that, somehow, the literal villain of Space Jam 2 was…an algorithm. An algorithm created by Warner Bros to make movies. This emphasis on the algorithm is…weird.
Before the Netflix’s double stock dip in the last four months, the “algorithm” was the secret to their success; now it’s the cause of their demise.
But there is no magical Netflix “algorithm”. Sure, Netflix has lots of data, probably more than any other studio in history. And they have a wonderful recommendation engine. (Which is powered by algorithms, perhaps the best in the business!) But when it comes to picking and choosing what TV shows and films to greenlight, no, they don’t have a data-based algorithm.
Instead, they have people. People making decisions. Using data. Just like everyone else in Hollywood.
Nobody Knows Anything
People are constantly complaining about the algorithm and how it’s picking and choosing which movies and TV shows Netflix should make, something which has only gotten worse in the last month after Netflix’s stock drop. Seriously, just search “Netflix algorithm” on Google News.
The most common quote about Hollywood is probably William Goldman’s classic refrain, “Nobody knows anything.” As others have pointed out, it’s overused, since it’s often used as an awkward, poorly-thought-out cudgel in favor of epistemological nihilism. Someone argues something and you can just be like, “No we don’t! No one knows anything, since a famous guy who wrote some classic films once said that.”
What Goldman actually meant is that predicting what’s going to be a hit and what’s going to fail is very, very, very difficult, if not impossible.
A studio may think they have a classic on their hands, but it doesn’t resonate with audiences. Thomas Lennon and Ben Garant wrote in their memoir about how they and everyone at Disney thought, for sure, that Herbie: Fully Loaded was a hit and the studio was even talking about a sequel before it got released. Then it bombed. Or it can work in the reverse, with box office duds turning into classics, like The Shawshank Redemption.
Nobody knows anything is just a way of saying, “Nobody can predict the future”, which is surprisingly good advice for most people most of the time, whether it’s sports, the stock market, politics, or whatever.
In entertainment, this problem is exacerbated by the distribution of hits to misses. Unlike flipping a coin, the misses-to-hits ratio is more like 90-10, but the “ten” aren’t a little better than the bombs; they’re orders of magnitude better. Take the over 800 films that had some sort of theatrical release in 2019. Then consider that one of those films singlehandedly made $2.8 billion dollars, more than the lowest 700 grossing films…combined.
Again, predicting hits and misses in this type of field is really, really hard.
This Has Been Tried (And Tried and Tried) Before
Since the days of the Oracle at Delphi getting high off lava fumes, humans have tried to predict the future. We started with tea leaves, animal bones and astrology, and now we’ve moved onto computers.
Since everyone loves movies, many, many, many statisticians have come to predict which films will succeed and which will flop. It’s not just Netflix; everyone wants a magical algorithm to predict hit films.
Now, unfortunately for those of us in the industry, we have to contend with tech buzzwords like “big data”, “machine learning” and “A.I.”. And a host of companies abusing those terms with grandiose claims that some new “A.I. machine learning algorithm” can predict what screenplays or movies will be hits and what will be misses, as the Verge described four such companies in 2019:
“Cinelytic isn’t the only company hoping to apply AI to the business of film. In recent years, a bevy of firms has sprung up promising similar insights. Belgium’s ScriptBook, founded in 2015, says its algorithms can predict a movie’s success just by analyzing its script. Israeli startup Vault, founded the same year, promises clients that it can predict which demographics will watch their films by tracking (among other things) how its trailers are received online. Another company called Pilot offers similar analyses, promising it can forecast box office revenues up to 18 months before a film’s launch with “unrivaled accuracy.”
And they didn’t mention Worldwide Motion Picture Group, which received a profile from the New York Times in 2013. Or Epagogix, which received a Malcolm Gladwell profile in the New Yorker in 2006. Or two new companies, Slated and StoryFit.
Everyone has an algorithm!
As a former strategy person at a streamer, I fielded pitches from these companies all the time. Usually, whenever we tested their models on future films, they stopped working. Even now I, the Entertainment Strategy Guy, gets approached by people asking me to use my data to predict future hits. That’s the key, they say, to really making lots of money in the streaming ratings game.
Of course it is! I know that. And if I could do it, I would! But the math says I can’t. And I won’t lie to people about that.
The Basic Statistics
As I often do, I’m going to boil down a statistics 101 class into a handful of paragraphs.
To predict the future requires two things:
1. A data set of the past.
2. A belief that past data will reflect future data.
Let’s take the second part first. Most of the time, it’s true. The way people behave in the past reflects how they behave in the future. If you’re Google, you believe that past search behavior will reflect future search desires. This is often a strong assumption, but sometimes it changes over time.
Now, do we have data for entertainment? Yes, we do. We know what was popular in the past and could assume that will be popular in the future.
But predicting the future accurately requires one more piece:
3. Lots and lots and lots of data.
The problem, for Netflix and their mythical algorithm, is that they can’t even compare to Google, Facebook, Youtube, Apple, Amazon, Microsoft and a few others in terms of data:
– Google delivers 5.6 billion search result every day. And they can track those users, knowing their location and past search history, if not more, including really, really good guesses at their ages, gender, race, nationality, and so on, especially for the 1.5 billion Gmail users or 2.5 billion Android users.
– Youtube has 1.9 billion monthly users watching 5 billion videos every day, or a billion hours of viewing, coming from 15 million content creators.
– Facebook has 2.9 billion users while Meta has 3.5 billion users of all of their social media apps. They host 250 billion photos and 350 million photos are uploaded daily. And most users just give Facebook all of their personal data.
And what about Netflix? Well, folks will say Netflix has about 200 million global users, which is a lot of users!
But users isn’t what matters here. In any equation, a statistician (or modeler) uses X to predict Y. The X—in this case—isn’t users; it’s the shows and films released to the public. The Y is whether they are popular. (Contrariwise, their recommendation algorithm does use the 200 million users’ choices to predict if they’ll like a show. That’s a lot more data! And even that is far from perfect.)
Once you understand that, in the U.S., across all broadcast, cable and streaming, there were 530 scripted TV shows released in the U.S. last year, you see the problem. That’s a small sample size. Considering that many are returning shows, and you could divide comedy from drama, then divide that out by genre, and considering budgets, talent, ratings and what not, I could come up with a hundred totally relevant variables.
What I’m describing is the process to set up a “regression” analysis, the most common tool for prediction. In a linear or non-linear analysis, every categorical variable reduces the accuracy of the equation. When you have only a few hundred data points, that makes your uncertainty incredibly wide. More complex statistical methods (say, a neural network to get super fancy) still has the same limitations: more categories means much more uncertainty.
Trying to figure out what drives success in a sample size of hundreds is basically impossible.
Now, one could say, what about Netflix’s library of titles? They have thousands of shows! Again, that’s not really the sample set we’re looking at. Those are the shows that survived the development process. To use library titles in one’s regression equation would require knowing every failed pilot made over the years at every network and every other streamer. Netflix doesn’t have that data. And even if they did, before about the year 2000, I think TV tastes were too different to compare to modern times.
On top of that, the logarithmically distributed returns—the idea that most shows are flops and a few are huge hits—makes modeling success even trickier. Again, the uncertainty is huge.
This is why, at best, the “algorithm” mostly predicts which genres and types of movies will be successful, but most of these “predictions” seem fairly obvious. You don’t need Netflix’s data to glean these “insights”.
People like silly romcoms and bad holiday films? Uh, yeah, just look at the Hallmark channel’s ratings.
“Sad-coms” are out? I wrote about that two weeks ago—but I’d had the insight months before that—before someone leaked a memo to Business Insider saying the same thing.
People like comic book films? No kidding. You needed an algorithm to tell you that?
If Netflix Has such a Great Algorithm, How Come They Can’t Predict Future Hits?
Hollywood has always used “data”, like Nielsen ratings and box office returns, which they use to try (the key word being “try”) to predict future hits. That’s nothing new.
And yes, Netflix has data and ratings and viewership. Probably more than any other entertainment company ever before. But this isn’t a revolution more than a next step forward. And that data isn’t picking and choosing what TV shows to make. People are.
Netflix’s track record backs this up. Honestly, you can see the lack of an algorithm—or better said, the difficulty in predicting hits and misses—by what works and what doesn’t on Netflix.
– Manifest came out and became the surprise show of the summer. I heard from folks in and around Netflix that this utterly shocked them. You can tell that this is true because they spent very little money marketing the series.
– Squid Game. Again, the execs liked the show. Did they have data that it would become a mega-hit? No.
– Tiger King. Before that Tiger King came out of nowhere to surprise the world’s biggest streamer. Yet the sequel utterly failed to resonate with audiences. Netflix didn’t predict either.
I’m sure someone out there will say, “If they don’t have an algorithm to make shows, then why do they have so many hits?” Well, Netflix make a lot of shows! And by “a lot”, I mean the most in the industry. If you make the most shows, but have an average hit rate, then you’ll still end up with the most hits! (But you won’t make a lot of money.)
Is there evidence of a magical, future-predicting algorithm in Netflix’s hit rate? Not really. I don’t have a fleshed out database to analyze and compare Netflix’s hit rate to other streamers yet—ironically, the data set just isn’t big enough, but should be by the end of the year—but if I had to guess, I’d predict that Netflix’s hit rate is about average. It definitely isn’t well above average, like say Kevin Feige at Marvel Studios.
And that’s why, on the flip side, while they have a lot of hits, they also have a lot of flops, from the expensive (Cowboy Bebop, Jupiter’s Legacy) to the small (Too Hot to Handle, Selling Tampa) and many more I don’t have time to list.
After Netflix’s stock plunge, it feels like a bunch of people, if not more, are now blaming the algorithm for their failures, but the data doesn’t really support that either. If anything, aside from true crime documentaries, I’d argue that Netflix hasn’t used the ratings data they do have to make shows that the broader American public wants to watch. (And when they do make those types of shows, people complain about the algorithm anyway.)
I don’t want it to seem like I’m picking on Netflix here. I could run the same exercise for every studio or entertainment conglomerate, pointing out hits and misses. Again, it’s very difficult to predict the future, so you make a bunch of movies and hope the hits out-perform the misses.
Except Netflix lets people believe that they’re doing something that no other studio has ever done before.
They aren’t.
Can An Algorithm Give Notes?
I’ve been thinking about this for a long time. And by a long time, I mean since September 2018, when I read that Cary Fukunaga profile in GQ. Fukunaga—an incredibly talented director and, by all accounts, a smart person—thinks that Netflix has an algorithm that can give specific notes on a specific TV show episode and provided an example:
“There was one episode we wrote that was just layer upon layer peeled back, and then reversed again. Which was a lot of fun to write and think of executing, but, like, halfway through the season, we’re just losing a bunch of people on that kind of binging momentum. That’s probably not a good move, you know? So it’s a decision that was made 100 percent based on audience participation.”
To prove this to yourself, remember that an algorithm (or model) only works by using past data to predict the future. So ask yourself, how many episodes or TV shows could this even apply to where an algorithm could even give this note? Even if you come up with examples, now you need to separate the sitcoms from the dramedies from the surreal sci-fi explorations.
Off the top of my head, it makes me think of Lost, one of the biggest hits of the last twenty years, or Rick and Morty. On Twitter, I got great suggestions like episode 6 of The Watchmen TV show (H/T Brandon Katz), Ted Lasso (H/T Robbie Whelan and Arlene Wszalek) and This is Us (H/T Andy L). The problem is those three shows all premiered after Maniac was in production!
So contrariwise to Fukunaga, it seems like non-linear stories with twists are actually part of some of the best shows in the genre?
Even weirder, Netflix is literally a pioneer in non-linear storytelling, experimenting with choose-your-own-adventure-style films and TV show like Bandersnatch.
Ask yourself, what’s more likely: that Netflix has an algorithm says to not do something…or some development exec lied to the filmmaker he’s working with, blaming the algorithm for his notes instead of his own creative instincts?
Fukunaga ended this section of the interview predicting, “I have no doubt the algorithm will be right.”
Based on the fact that 90% of the people reading this—including people who work at Netflix—probably have no idea what Maniac even is, we can say, confidently, that the algorithm was not right. Maniac didn’t work. It bombed. Strike 1,000 for the algorithm. (That doesn’t exist.)
It All Comes Down to People
Why write 3,000 words debunking Netflix’s algorithm? In addition to just being really tired of hearing this silly point, it’s important for any leader or any one working in entertainment to realize this very key strategic insight:
People make decisions. And people are often biased.
If you’re a pop culture writer or critic, don’t blame “the algorithm”. Blame Netflix’s development execs for greenlighting and making subpar TV shows.
If you work in entertainment and a data researcher comes to you saying they’ve proved what future TV shows or movies will be hits, be skeptical.
If a development exec claims that they “know” a film will be a hit, either due to their personal taste or using data, be skeptical. Know, ahead of time, that all of your development execs bring their own biases to what shows they want to greenlight, who they want to cast, and how they do things. We all do. That’s human nature.
The algorithm obscures this very real truth.
At the end of the day, the people running Netflix are people, not computers or data or algorithms. Sure, those people are using data and computers to make and aid their decisions—after all, that’s the whole point of my streaming ratings report each week—but it’s still people making those decisions.
And if you’re mad at Netflix, either for giving Dave Chappelle a platform or canceling your TV show or just another TV critic bored by an unimaginative, derivative TV show or film, don’t blame “the algorithm”. Blame the people who work at Netflix.
Because the algorithm doesn’t exist.
