Last week, I threw up a quick Twitter thread on The Boys and I just turned it into a full article for Decider. (And it’s short for me, about 800 words.)
So take a read and share on social media! Appreciate it in advance.
Of course, trying to judge if a series is performing well or poorly is NOT simple. And as I found some new data sources, I had thoughts that got cut from the final article. (As always.) So here’s the rest of the story, including a broadcast comparison, how I think about managing messy data sets and the rest of Amazon Studios datecdotes.
Introduction – A BH90210 Comparison
Initially, I was going to compare The Boys to BH90210, the Beverly Hills, 90210 revival that was off to a good start last week. Here’s the Variety quote on its success:
That’s good! Or is it bad? I mean, is 3.8 million people watching good? Honestly, with broadcast we don’t know since a show like Night Court used to get 20 million viewers in the 1980s, and The Big Bang Theory—the biggest show on broadcast in 2019—didn’t even get that for its finale. (Fine, it did with DVR viewing.) No seriously, here are the ratings for Night Court:
So which was bigger, BH90210 or The Boys? To the Google Trends. Now, here’s the first look and you can say, “Well The Boys won”…
But I told you Google Trends was finicky, didn’t I? The problem with a show like BH90210 is the title is super generic and derivative off another series. So here’s with a few other variations on that title.
Add them all up, and BH9210 was more in the consciousness than The Boys. Whether that translates to more viewers, I can’t say. But it provides some “broadcast to streaming” context.
Comment on Amazon Datecdotes
One of my favorite parts about writing and researching this article was it forced me to look up all of Amazon’s “datecdotes“. Which I’d been meaning to do since their last earnings report, where they again touted Emmy success while steadfastly avoiding numbers a la Netflix.
Now, why wouldn’t Amazon tell us good news? Well, the pro-Amazon case is they have all sorts of good news but are hoarding it for some advantage. That’s frankly BS. My rule of thumb with all large organizations—from the government to any corporation—is they share good news and hide/bury the bad.
The most basic assumption is that Amazon’s overall numbers are much, much smaller than Netflix, so they avoid specifics. Because if they did, they would look bad. That’s simple logic.
Anyways, here’s my Amazon datecdotes table, a la Netflix. Notably, I left out two other sets of numbers for space in my Decider piece. First, the Reuters leak from last year had aadditional details for Transparent and Good Girls Revolt. Second, last fall Amazon touted it’s NFL viewership numbers for Thursday Night Football:
Some quick notes. For The Man In the High Castle, for example, we still don’t really know what 8 million viewers means. Is that over the lifetime, up to the point in time Reuters got the leak? Or some shorter time period? With data, that distinction is really important.
Or take The Tick as a top five series for Amazon in 2017. That would worry me, given that as the IMDb data shows that series wasn’t even that popular. And it was in their top five? And now it and Sneaky Pete are off your platform? That would make make me think the other series are much much smaller than we imagine. (Sneaky Pete was also a Sony co-production. So the co-pro curse strikes again.)
Google Trends – The Boys Pessimistic Case
To be clear, Google Trends isn’t perfect and I am just about at the point of over-relying on it for these articles. Plus, Google Trends can be messy. Nothing, for example, gets confused with Game of Thrones since that’s a unique title.
But The Boys is messy. The few times I was able to get the “topic” version of it, and then run the test, it dropped WAY down in the results. Here:
But that seems much too big of a drop. It’s not like there was some generic search term that became popular at the exact time The Boys launched, it’s just the algorithms couldn’t tell what people were searching for. So if you want the most negative case for The Boys–which I don’t believe–this is it.
Also, for space, I cut the sans-Game of Thrones Google Trends view. GoT is so big it makes everything too small to compare. Here’s my initial Google Trends look of The Boys versus Stranger Things. I like it because it lined up with the fourth week of Stranger Things, which is a way of saying an Amazon hit is equivalent to a week 4 of Stranger Things.
For season 1 of a series, that’s not bad. You expect shows to start small, and grow. The worry is when they don’t after season 2.
IMDb Thoughts
I want to defend my use of IMDb ratings for a bit, because it is definitely controversial. Mainly, after a few flame wars on Rotten Tomatoes and IMDb over Black Panther, Captain Marvel and Star Wars: The Last Jedi, many critics rightly point out that trolls can run campaigns to down vote certain types of movies. That’s fair.
However, my approach to films that have been “flamed” is to leave them out of the data set entirely. Instead, the best approach with any social platform—which is what I’d call IMDb and Rotten Tomatoes—is to understand how it skews on demographics. Every platform from Twitter to Facebook to IMDb skews towards certain demographics. Or “over-indexes”. If IMDb ove-indexes male say 75-25%, I need to account for that.
More important for our use is the psychographic representation. For a superhero movie, I want a social platform that over indexes on fanpersons. Or comics fans. Or lovers of blockbusters. IMDb offers that! The fact that The Boys is doing well on a platform it *should* do well is a credit to it.
Plus, in a larger sense, IMDb’s ratings—besides a few flamed down series and movies—has a pretty good take on the top series. Especially if you weigh total volume of ratings versus the rating itself. That accounts for some series/movies that just don’t have a lot of ratings period, so are easily skewed negative. If you look at IMDb’s top series, well that list makes sense.
IMDb Distributions
For those who are curious, “total ratings”–meaning all the people who submit a ratings–is logarithmically distributed, because performance in entertainment is logarithmically distributed.
However, the average of those ratings are NOT logarithmically distributed. Instead they are roughly normally distributed, with some tail bunch ups. (Roughly, since it may not map exactly on the normal distribution.) Most movies fall in a band between 6 and 8, and the outliers are above 8, with a few above 9. (Or 3 to 4 in a 5 point system.) This applies to critic ratings too. For critical acclaim, most movies average a 60-80% rating with a few above 90%.
(Note, that isn’t the “Tomato-meter”, which is just a binary good/bad, but the average star rating by critics, which is much more useful for data analysis. The tomato meter tends to have a lot of movies bunch in the 90s, so a film that was “universally acclaimed” gets a 95 and a “strong acclaim” gets a 93. Which doesn’t tell us much.)
Other Sources
If I were king of the world, I’d have asked for a few more data sources to test this data. Off the top of my head, social sites like Reddit, Facebook or Twitter would be pretty interesting. How was the conversation on those platforms? Did it match what we’re seeing in other huge hits?
But the best is other people estimating demand using their own tools. For example, Parrot Analytics, whose data I’ve long admired from afar. Here’s a Linked-In post they referred me to, where they draw similar conclusions to mine.
