The Most Important Shape in Entertainment Part II: Logarithmically Distributed Returns

My dad didn’t like the ending of Empire Strikes Back. His felt that it didn’t finish the story, it left off with a, “See you next movie!” conclusion. That irritated him. He hasn’t seen Avengers: Infinity War yet, so you know he won’t like that.

My article yesterday probably did sort of the same thing to the audience. I come up with this big conclusion—the logarithmic distribution—but then barely touch on it.

Well, since we’re already talking about the movies, we might as use that as the ur-example of my magic trick, “Logarithmically distributed returns”. I first learned this law by analyzing movie performance, and it’s my best tool for teaching it to others. But I’m not just going to show you this phenomena, I’m going to show you it multiple ways, in multiple categories. Then I’ll explain the biggest statistical mistake I’ve seen when forecasting box office performance.

Logarithmically Distributed Returns…What is it?

Let’s start with the last word. What I’m describing today is the “output” of most entertainment or media processes. So my examples are about the “result” or the “y-value” or the “dependent variable”, to describe it in three different statistical terms.

In other words, performance. This means how well something does. Box office for movies. Ratings for TV. Sales for music. Attendance for theme parks. No matter what the format, the success (or very frequent failure) is logarithmically distributed.

What does logarithmically distributed mean? Essentially, orders of magnitude. The returns don’t grow on a geometric scale, they grow on an exponential scale. This means that the highest example can be in the billions while the smallest can be in the dollars. That’s a difference in magnitude of 9 zeroes.

The most common summation of this is the “Pareto principle”, who coined the term about “power law” distribution. Roughly speaking, Pareto is summarized by the 80-20 rule, or 20 percent of the inputs deliver 80% of the returns. And like any mathematics/statistics topic, there are obviously a ton of variations on this law and specifics that I’m not going to get into.

(For those who are curious, inputs have their own distributions, but aren’t as reliably distributed as outputs. A topic for the future.)

Logarithmically Distributed Returns Visualized: Feature Films in 2017

Enough talk about what it is, let’s use an example. I went to Box Office Mojo and pulled all the films from 2017 that grossed greater than $0 in theaters. I didn’t adjust for year and pulled everything, no matter how small. The result was 740 movies released. Oh, and I only pulled domestic gross.

I’m going to show you the data two ways to help you visualize it. First, is the less accurate way, but I love it because it shows scale. This is all 740 movies plotted from lowest to highest, with the y-value as the domestic gross in dollars.

slide021.jpg

Source: Box Office Mojo.

I love how smooth the curve looks. But the true measure of the data is the “histogram”, where you count the number of examples per category. I set up the categories myself at $25 million dollar in intervals, starting from zero.

slide031-e1536791810207.jpg

Source: Box Office Mojo.

Most people don’t realize how many films are written, produced and even released every year. Like I said, last year was over 700. So let’s add a threshold of $1 million dollars at the box office to our list. If I had production budget estimates, I’d sort by that, but the result gets you to the same place. (The reason for using production budget is that when you scan that “almost grossed $1 million threshold”, you see some legitimate films such as Patti Cake$ and Last Flag Flying, from Fox Searchlight and Lionsgate/Amazon Studios respectively. Those films cost a lot more than $1 million to make.)

slide041.jpg

Source: Box Office Mojo.

All the charts show the same story in different ways: there are hundreds of films that made less than $1 million at the box office, around 150 that did less than $25 million (many of which probably lost money), a range of movies in the middle and then a few monsters (Star Wars: The Last Jedi, Wonder Woman, Jumanji and Beauty and the Beast).

I think I can hear some of you insisting that I give you the “counting statistics”. You still want to know the average, right? Well here they are, for all 740 films. I mainly did this because I’m going to use them in the next section.

slide051.jpg

How Logarithmic Distributions Differ from Other Distributions

Perhaps the best way to describe the logarithmic distribution is to show how it isn’t other distributions. In other words, to show how inadequately the normal distribution and uniform distribution capture the performance of feature films.

Let’s start with the uniform distribution. The idea that, “Hey, a movie can gross anywhere between $600 million dollars (Star Wars) and $0, and every where in between.” What if we had an equally likely chance of that? In decision-making, the human brain often defaults to uniform distributions when assessing possibilities, so this isn’t completely academic. Here’s how that would look:slide061.jpg

If only this were how to finance movies! The industry would green light a lot more movies. But it isn’t, only a few films hit that rarefied air of $200 million plus dollars.

What about the normal distribution? I tried to chart this, using our mean of $15 million and standard deviation of $50 million. Unfortunately, that gives us a lot of “sub-zero” grosses, which I just cut off at zero. The problem with the normal distribution is it makes misses as rare as hits. That just isn’t the case. Also, the odds of a giant hit become astronomical in a normal distribution. In this case, a hit like Star Wars: The Last Jedi would be 10+ standard deviations form the mean, meaning it has a 1 in a million chance. Obviously, hits like that happen every year, so more like 1 in 200.slide071-e1536792385695.jpg

Let’s put them all on the same chart, to really show how logarithmic distribution of returns just looks different.slide081.jpg

Source: Box Office Mojo

This chart shows how quickly the results drop off in reality compared to other hypothetical distributions. If someone tells you Hollywood isn’t normal, show them this chart and say, “You’re sure right!”

Variations on the Initial Theme

I might still have skeptics in the crowd.

Maybe, they’d say, I just got lucky. That distributed returns happen to be power-law-based for the year 2017, but this lesson doesn’t really apply to other parts of film. Well, that would be wrong.

Spoiler alert: no matter how you slice the inputs, you get the same result.

First, I could expand the number of years I’m using. I happen to have box office gross from a project I did that covers 2012-2014. Here’s that chart.Slide09

Source: SNL Kagan

Here’s the next fun trick: the distribution of returns still applies for sub-categories. Take horror, which I looked at a couple of months back. Here are all the horror movies going back to the Exorcist, according to Box Office MoJo. Specifically, “Horror-R-rated”, which is 504 films:

Slide10

Source: Box Office Mojo

The rule still holds! In this case, there has been one monster horror film—It—then some other smaller ones. Of course, I could hold all the box office and adjust them for into 2018 grosses. Does that change the picture? No, if anything it amplifies it. In this case, The Exorcist did $1 billion in adjusted US gross, and The Amityville Horror did $319 million. But for those increases, a lot of other smaller films drop down even more, especially recent films.

slide111.jpgI’ve done this for a ton of different genres. Superhero movies. Foreign films. And it always holds. The only caution is that sometimes the “ceiling” of the range gets compacted.

What about sorting by something else? Say, rating? Do R-movies have more hits versus PG-13 or PG? Fortunately, my 2012-2014 data set has ratings. First, know that G, NC-17 and Not Rater just don’t have a lot of examples (only 45) so I deleted them from this analysis. Here are the other three, in line chart form:slide121.jpg

Source: SNL Kagan

As we can see, for R, it holds. For PG-13, it holds. For PG, it looks like it holds, but honestly since we only have 39 examples, it doesn’t show as clearly. Increase sample size and we’re going to see this.

You could do this analysis setting for production budget and studio and even types of studios. As long as the input is independent, it holds.

Two Examples Where This Works Less Well

Listen, I believe in being up front with my data analysis. Even though this is a magic trick, I’m not trying to hide or obscure data that doesn’t make my case as well. That’s why I left PG rated movies in above, even though it’s the least logarithmic looking line in my analysis.

So in my experience, have I come across sub-sets of movies where my rule/law/observation doesn’t hold? Absolutely, so I’ll share those with you next. To clarify, it’s not that my magic trick fails, it is that the floor disappears. So look at this chart, from my series on Lucasfilm:

slide131.jpg

Source: Box Office Mojo

These is my data set of “franchises” that included Star Wars, Marvel, DC, X-Men, Harry Potter, Lord of the Rings, Indiana Jones and Transformers. As you can see, those films just don’t have flops. The “floor” is about 200 million in domestic box office, with only 14% of all films dropping below that. So it isn’t logarithmic on one end. I actually think my timeline of films by box office, with their names, shows this floor pretty clearly over time:

slide151.jpg

Source: Box Office Mojo

My rule doesn’t hold—this is important—when I sort by another output, not by an input. In other words, I’m sorting by the result.

A franchise is a series of films made off a successful first film. In other words, it is sorting by “success” of the first franchise film. Many aspiring franchises therefore didn’t make my data set. Four examples off the top of my head that I did not include, from three different genres: The Golden Compass, Battleship, The Lone Ranger and John Carter from Mars. If I included all aspiring franchises, the list would have looked more exponential Also, this data set is small, only 50 movies.

What about that huge data set I just pulled to look at Oscar grosses? Well, I haven’t even histogrammed that yet, so I don’t know what it looks like. So we’ll see. Again, though, this is in a way a “success” metric in that these are all “good” films. Obviously, a lot of films at the bottom of our list—meaning getting sub $1, $10 and $25 million grosses—were just bad, so no one saw them. With the Academy Awards, we’ve deliberately sorted that out.

slide161.jpg

Source: Box Office Mojo

The rule holds! Mostly. Now, with adjusted gross we do see a bit of a floor. Historically, a best picture film tended to get more than $50 million in domestic box office. But with both Oscars and Franchise Films, we can see that “super-hits” are still rare, but present.

Final Lesson: This is Why Linear Regression Doesn’t Work in Entertainment.

I have one final lesson for the data heads in the crowd.

Let’s say you’re an aspiring business school student who hopes to go into entertainment. Or you’re a junior financial analyst. Or a statistician diving into entertainment. (Three real world examples I’ve encountered.) You’re given a mess of data on the performance of feature films at the box office. And you want to draw some conclusions.

Well now that we know how our data is distributed—logarithmically—we should come to one clear conclusion: linear regression WILL NOT WORK!

It’s really just right there in the name. Linear regression works on things that have linear growth, and our things have exponential growth, which throws off all conclusions. The work around is that you can convert our data points to logarithms, and then have a “log-normal” distribution, which gets you closer to accuracy. (Though, as I wrote here, you still have a sample size problem.) In general, as well, since you have so few examples of success—the long tail at the right—you just can’t draw statistically meaningful conclusions.

Conclusion – What’s Next?

Well, I didn’t say this was a law of media and entertainment because it applies to feature films. I said it applies to everything. And it does.

But that’s for our next installment and another dozen or so tables and charts!

  1. […] we’d see that Moonves/CBS shows would take an out sized portion of the top 20% of shows. Given the logarithmic distribution of returns (so excited to use that already!), that means he had an outsized impact in creating hugely popular […]

    Like

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: