(Welcome to the Entertainment Strategy Guy, a newsletter on the entertainment industry and business strategy. I write a weekly Streaming Ratings Report and a bi-weekly strategy column, along with occasional deep dives into other topics, like today’s article. Please subscribe.)
Today, I want to lay out my approach to data in one place, explaining why my data philosophy is both unique in the streaming ratings game and, frankly, worth paying for. This is a revised version of a similar article from three years ago, but since I have so many new readers, both free and paid, I figured I’d update it and send it out again.
It’s also, hopefully, a good breakdown on smart ways to approach data and data analysis.
Having a philosophy matters. Everyone should have core principles that guide them in anything they do, in terms of approach, aesthetics and, most importantly, ethics. Today, I want to explain how I analyze data, how it differs from everyone else, and why it matters. So let’s dig into that philosophy, since it is really how I deliver value.
But first, a brief history of the streaming wars and viewership data.
A History of Streaming Viewership Data
When I first started writing about the entertainment industry in 2018, I focused on strategy, with an emphasis on the “streaming wars” and theaters/movies. Along the way, I noticed a glaring gap in the coverage of said streaming wars. As I’ve written before, content is king, and I’d estimate that content (both making it or sitting on a giant pile of it) will determine at least 50% (if not more) of the battle to “win” the streaming wars.
But at that time, no one knew what TV shows or movies were doing well!
The operative word being “were”, since many companies rushed in to fill this gap, and starting roughly around the spring of 2020, the “streaming ratings era” began.
Starting in March of 2020, Netflix started releasing weekly top ten lists. Nielsen started releasing a weekly U.S. top ten list that same year. Then several more streaming analytics firms began releasing select data points as well. Some of those firms put out weekly updates, like Luminate, Samba TV, TV Time, Just Watch, Reelgood and Parrot Analytics. Other places like Comscore, TVision, Digital-i, and VideoAmp occasionally release their data too.
While we have more data than ever, different companies use different measurements, with different definitions, on different sample sizes. In some ways, this all leaves folks more confused than when we knew nothing.
That’s where I come in.
My Approach to Streaming Ratings
When it comes to streaming ratings, you can get them in one of three ways:
- The streamers provide some data, but mostly “datecdotes”. (The big exception, of course, is Netflix, which provides weekly global top ten lists and bi-yearly data drops. Crucially, they restrict country-level data, since, as Ted Sarandos said, it’s really valuable.)
- Streaming analytics companies measure and publish their own data, either publicly or privately.
- Reporters at the trades (and in the biz press) use either of the two above to write articles.
Each of these groups has their own set of biases to contend with:
- The streamers try to deliver the datecdotes or data that puts their shows in the best light.
- The streaming analytics companies try to put out interesting tidbits that will go viral.
- And the reporters at the trades usually repeat the data, trying to write an article as quickly as possible, given the insane output demands by most media companies.
Realistically, I’m competing with that third group of people, but I come at it in a fundamentally different way, since I have a different business model that isn’t based on clicks.
First, I start not with the story in mind, but with the data.
As I wrote above, it takes me the better part of three days to collect, analyze, visualize and explain the streaming ratings/viewership data. And I analyze multiple data sets to tell a fuller picture. Again, most articles in the trades don’t have time to compare multiple data sources. At most, they can focus on one set of data at a time.
Further, I try to put everything in context, having built up databases of shows and films by streamer and by data source for over five years now.
- I’ve collected every publicly available Nielsen data point to date.
- I’ve built a database of streaming films and TV shows, with a variety of custom, personalized metadata, which I keep expanding over time with new categories.
- I’ve collected virtually every datecdote from every streamer, especially HBO Max, Prime Video, Disney+, Paramount+ and, formerly, Netflix (who mostly releases top ten lists now).
- Same goes for Samba TV’s datecdotes and weekly top ten charts.
- I started collecting Luminate’s weekly data last year, with a full dataset of publicly available data.
- I collect IMDb ratings and reviews.
- I’ve been collecting WhipMedia’s TV Time top ten lists for years now.
- I started collecting Reelgood and Just Watch’s interest charts each week in 2024.
- I’m building out a Google Trends data set too.
- And I keep expanding my data collection as new, publicly available data becomes available.
Lastly, I’m not trying to make issues of the Streaming Ratings Report “go viral” because I’m not focused on clicks or advertising, but my relationship with my readers. I want you to trust what I’m writing, so I write a sober analysis each week, which might not have a flashy headline to go with it.
The Tenets of the Streaming Ratings Reports
I’ve honed my approach and distilled those improvements into a few key tenets that feature in every Streaming Ratings Report. Together, they make this single most informative ratings report on the market:
- Consistent. Too many articles are still driven by press releases from the streamers. With my report, you can guarantee that I will cover every week of streaming ratings data, even if I take a break for the holidays.
- Comprehensive. I track as much content as possible, not just the hit shows that drive the most clicks. If you work in entertainment, you need to know how everything performs.
- Context. Data points by themselves don’t mean anything. Data needs context. It isn’t enough to know which film “won the weekend”; was it the best of this year? All time? How does it compare to similar types of content, either genre or release style? The only way to know is to leverage a comprehensive database. And I have the most comprehensive database in the industry. Many news stories just repeat top ten lists with little additional details, besides what’s included in the press release.
- Multi-source. Right now, I collect data from six regular sources and two variable sources. I’ll keep adding other streaming analytics companies that match our criteria. This means my analysis isn’t driven by any one agenda, but a “poll of polls” that provides the most accurate look.
I’d argue that few other outlets—be they streamers or trades or reporters or the analytics firms themselves—are as upfront with their approach to data as I am.
My Data Analysis Principles
Here are the other principles that guide my data analysis; each one could be its own article!
Use Multiple Data Sources
If I were using intelligence jargon, I’d call this “multi-source intelligence”. If I were mimicking Nate Silver, I’d call it my “poll of polls” approach to streaming ratings.
The beauty of having multiple sources for ratings data is that you can use them to verify each other!
For the most part, different sources tend to agree with each other. Directionally, they tell the same story. The biggest TV shows, according to Netflix’s data, are also the biggest shows according to Nielsen. Is this relationship perfect? No, because Nielsen doesn’t capture mobile viewing, and some TV shows that do well overseas don’t do well in the U.S. and vice versa, but it generally tracks. In fact, I’d be more suspicious if Nielsen’s data perfectly tracked Netflix’s. That would be a sign someone is rigging their numbers.
This applies to other services as well. The top shows on TV Time are also usually among the top shows on IMDb, which are the top shows on Google Trends, and you get the idea. Very rarely does a show miss on the rankings and have great IMDb scores. Or high Samba TV numbers.
But when the various ratings sources do diverge, that’s when the fun begins, figuring out the signal through all the noise. Finding the nuance between the various ratings is difficult— that’s the whole point of my report—but it can be done!
Today, with Luminate, Samba TV, Nielsen, and others, we have a fuller picture of what people like to watch than ever before.
Highlight the Misses
These days, most people know what shows are hits, since hit shows get lots of media coverage, but the streamers release dozens of films and TV shows each week. And those misses go unnoticed. At worst, people think that some TV show flops were actually hits.
I want to look for and highlight the films and shows that fail so we can have a better understanding of what works and what doesn’t in streaming. This is especially crucial these days because, often, the streamers try to manipulate the industry trades into telling people that they have a hit show, even if said show wasn’t a hit.
Every week in the Streaming Ratings Report, I call out a “Miss of the Week” and list off every new show that didn’t make the charts. Now, this section is almost exclusively behind the paywall, so if you want to cut through the noise and know what didn’t work, you have to subscribe. I also highlight the losers at the end of the year.
Viewership is King
Not only are there a lot of streaming analytics companies collecting data on streaming films and TV shows, they’re also measuring different things, like viewership hours, households, interest, and more.
How do I rank all of those different types of measurements to figure out what’s a hit?
– First party viewership numbers
– Second party viewership estimates
– Customer ratings (as in reviews on IMDb or Rotten Tomatoes)
– Interest
…
…
…
[repeat a hundred times]
…
– The “social conversation”
While some folks want to believe there is some value in “loving” a show, at the end of the day, you need people to actually watch content, which is why it’s the best measurement we can get. Plus, as I’ve covered before, “love” is highly correlated with viewership. The most watched shows have more devoted fans than smaller shows.
Some folks might wonder why “social” is last. That’s because it’s the category that is the least representative of society. Some shows are buzz-worthy, but others aren’t, but those not-buzzy shows still get tons of viewership. So I downplay social metrics in my analysis.
Always Compare Things “Apples-to-Apples”
I’ve written a whole article on this concept because it’s probably the most violated rule in data journalism. By making bad comparisons (not apples-to-apples comparisons), writers can wildly overhype trends.
The absolute worst, most common example is people comparing global YouTube video views to U.S.-only linear TV ratings. Which leads to crazy statements like, “More people watched this YouTube Video than the Super Bowl.” But they didn’t, otherwise that stupid YouTube video would have sold tens of millions in advertising dollars, right? Or “This was the podcast election.” without actually breaking down what constitutes a view on YouTube.
Everyone should try to control for as many variables as possible, which I try to do every issue.
Find Actionable, Trustworthy Insights
If you can’t make decisions based off the analysis, then why do the data analysis? I try to find the relationships between what’s working and what’s not working on streaming.
But I’m far more skeptical than other people.
First, I ignore random results. I call this the “No blue uniforms” rule. During the 2018 March Madness tournament, Google touted that its AI discovered that teams with blue uniforms did better in the tournament. This statistic is meaningless and random. My goal isn’t to find connections for connection’s sake, but insights that folks can leverage.
Next, I avoid looking at one-off datapoints. Every week, I read articles in the trades that amount to “This TV Show/Film Was a Hit, and Here’s What They Did.” but that’s just one datapoint! You need a lot more to identify a robust trend.
And if I can’t find a clear conclusion, I’ll let you know that as well, since knowing the “null hypothesis” can also be useful!
Magnitude is Better Than Direction.
Lots of folks provide rankings for their data. These are useful and I use them, too, but I prefer actual numbers, which show the differences in magnitude/scale (or velocity) as opposed to simply the rankings (or direction).
Think of it like this: both Novocaine and Minecraft were the number one films at the box office their opening weekends. One made nearly twenty-one times more in box office dollars, so those “number ones” are obviously not equal. That difference is the value in magnitude over direction. In 2023, Godzilla Minus One and The Boy and the Heron were number one films their opening weekends, leading to headlines about foreign films saving theaters when…they were barely hit films; the overall box office was low. In 2024, Terrifier made waves when it did well, but MaXXXine had really similar box office numbers, but got ignored during the middle of the summer box office busy season.
So knowing the actual numbers matters much, much more than just if something topped the charts.
Provide Clear “Data 5Ws”.
You know the 5Ws from journalism, right? Who, what, when, where and why? Well, often in bad data analysis, you can’t answer all five. Because if you could, it would make the viral headline wrong. Whenever possible, I try to provide these 5Ws so you know where my data comes from, especially when I introduce new datasets, like my US sub counts:
What – Number of paid subscribers
Where – In the United States (excluding Canada)
When – As of the end of Q2 of 2023 (June 30 roughly)
Who – From each major stream (over 5 million subscribers)
What – These are my estimates, based on financial reports, leaks to journalists, other analytics firms’ estimates, and my own models.
To repeat my YouTube example from above, when you compare YouTube views to Nielsen’s average minute audience, you’re comparing a global audience to a US-only one (the where!), different metrics (the what), and different data sources (the who). That’s three great reasons not to compare those numbers!
