If you want to know why I started this website, just take a look at the furor unleashed on the Academy of Motion Pictures Arts and Sciences when it announced (via Twitter!) that it would add a new category called “Achievement in Popular Film”.
First came the questions: “How would this even work? By box office? By user reviews? By top 25 films at the box office?”
Then came the pondering: “Hey, what film would have won in the last ten years? What will happen to Black Panther?!?”
Then came the criticism: “Hey, this won’t work. This won’t solve the problem.” Or summarizing Rob Lowe on Twitter, this will just plain suck.
Throughout all those takes, the data was largely missing from equation. In data’s place lived assumptions. Assumptions from which the rest of the arguments derived. Consider what a flawed world that is: how can we fix something if we don’t know what the problem is? Or worse, when we don’t know what caused the problem?
Well, no more. Let me step into the void with as much data as I can muster to challenge the assumptions permeating the Oscars debate. Let’s separate fact from fiction. Call out our assumptions. Review what we know from what we can only guess. Let’s do this.
But first, my usual warning on data when it comes to media & entertainment.
Warning: We’re in small sample size.
I’m not going to go into as much detail as I did for my series on Mergers & Acquisitions in media and entertainment on my data, but the same admonition that drove that series drives this one: we’re firmly in the realm of small sample size.
Box Office Mojo tracked “Oscar bumps” going back to 1982, so that’s the sample of Best Picture nominees I used. So that’s our starting sample size: 216 films. However, drawing conclusions from 1982 data just seems wrong. Too much has changed since then. From DVDs to going from 200 or so films per year to over 500. As a result, we’ll leverage the last 20 years of data, which is only 136 films, 81 since 2009 and 55 from 1998-2008.
Assumption 1: The Oscars feature fewer and fewer “popular” movies.
Rating: True.
The ostensible reason the Academy needs to make a “popular film” category is because popular films aren’t being included in the nominees for Best Picture. This statement seems obvious which is why so many people said it on podcasts or in articles summarizing the issue. Narratively, this is an easy case to make: In 2017, only two films grossed over $100 million dollars, Dunkirk and Get Out. Worse, the winners in 2016 and 2017 grossed under a $100 million dollars combined, and had a combined box office of $45 million when they were nominated. No films have been nominated since 2014 that we could call a “blockbuster” meaning it did over $250 million at the box office.
(I defined movies as either “popular” with greater than $100 million in domestic box office or “blockbuster” with greater than $250 million. “Popular” and “blockbuster” are my definitions, but they work pretty well.)
The problem with easy narratives is they can often be countered with an equally compelling counter-narrative. If I squint at 2015, it’s hard not to call The Martian a near blockbuster since it did $228 million in domestic box office and more at the international market. I could also point out that La La Land, which was so close to winning it was even announced as the winner, did $400 million in total box office. Or I could just play with the timing: The Oscars don’t have a problem with nominating blockbusters since Star Wars was nominated in 1997, ET in 1982, Beauty and the Beast in 1992, Titanic in 1997 or even Avatar as recently as 2009.
So let’s go to the data. I plotted this a few ways, and they all tell roughly the same story. First, the raw counts:
Chart 1: Count of “Popular Films” and “Blockbuster Films” in Best Picture Nominees. Data: Box Office Mojo
Even that doesn’t really tell the story right, since the number of films eligible doubled in 2009, as the Academy expanded from 5 films per year to “up to 10”. So here is the percentage of films defined as popular or blockbuster for all films nominated.
Chart 2: Percentage of “Popular Films” and “Blockbuster Films” in Best Picture Nominees. Data: Box Office Mojo
But even that doesn’t tell the whole story. That’s about one or two movies passing a certain threshold. Arguably, the more important fact is the average box office performance of the nominees. How has that trended?
Chart 3: Average Domestic Box Office per Best Picture Nominee. Data: Box Office Mojo
Does even that metric tell a misleading story? See, a dollar in box office in 1998 isn’t equal to a dollar in box office in 2017. According to Box Office Mojo, a 1998 ticket only cost $4.69 whereas on average in 2018 that price has jumped to $9.27. So we need to make the three tables above, but adjust for the price of a ticket in a given year. Fortunately, Box Office Mojo does this for us.
Chart 4: Count of “Popular Films” and “Blockbuster Films” in Best Picture Nominees, in 2018 adjusted dollars. Data: Box Office Mojo
Chart 5: Percentage of “Popular Films” and “Blockbuster Films” in Best Picture Nominees, in 2018 adjusted dollars. Data: Box Office Mojo
Chart 6: Average Domestic Box Office per Best Picture Nominee in 2018 adjusted dollars. Data: Box Office Mojo
Accounting for ticket price inflation, everything looks even worse for the Academy. The worst measure is the average box office per film. It was on a downward slide that was only arrested for a two year period, then it has gone back downward. The number of blockbusters per year looks equally bad, as the period from 1998 to 2004 regularly featured blockbusters, then again besides the period from 2009-2014, they haven’t featured any.
Taking those six charts together, we see a narrative forming that in our time period, we’ve seen two slides away from popular films and towards smaller films. Starting in the 2000s, the popularity of the films started dropping, bottoming out in 2005, and staying in that low period through 2008. So in 2009, the Academy expanded the field to 10 films to hopefully get more “popular” films. It worked, and the number of popular films, blockbusters and average box office jumped right back up.
But after this initial surge of popular films and blockbusters, the voters returned to form and the number of popular films, blockbusters and average box office per film plummeted again. To show this, I combined the data of films through these three time periods:
Of course, the key question for the Academy is how a lack of popular films reflects in TV ratings. (I’d personally argue that ignoring blockbuster films means the Best Picture category isn’t truly representative of the quality of films in a given year, but I can’t quantify that.) So let’s test that next.
Assumption 2: Featuring more “popular” movies will drive TV ratings for the Oscar telecast.
Rating: Maybe, leaning towards true
Again, narratively this is a really seductive argument. Basically, if you feature really popular films, people will tune in to see those films rewarded with nominations and wins at the telecast. Of course, the counter-narrative is also persuasive and I heard it on two different, influential podcasts (The Ringer’s Press Box? and KCRW’s The Business with Kim Masters): the types of people who watch the Academy Awards don’t watch/like popular movies anyways.
So here’s the one the chart that implicitly everyone referenced but I never saw: TV ratings plotted against average box office per film. (I also did the percentage of popular films, but it was even noisier than this line chart.)
Chart 8: Average Domestic Box Office per Best Picture Nominee in 2018 adjusted dollars versus TV Ratings, in Millions. Data: Box Office Mojo, Nielsen data from Wikipedia.
So what can we draw from this? Honestly, not much. Technically, I should plot this as a regression model using a time series analysis, but I can tell you ahead of time it won’t be statistically significant so I’m not going to introduce that bad data to the world. Instead, the strongest conclusion we can draw is the Oscar telecast peaked in 1998 at 55 million people and have been sliding down ever since.
As for whether popular movies have slowed this decline, you can cherry pick data either way. First, let’s make the “popular movies matter” case. In 2008, ratings hit their nadir at 32 million, just above 2017’s 32.9 million. Each of those years represented near low points in average box office per film. Then in 2009 and 2010 saw increases in the TV ratings, and those two years were the highest average box office since 1998.
I can also make the case that “popular movies don’t matter” pretty easily. 2014 had the highest ratings since 2004—remember this for a minute, I’ll get back to it—and it only featured 1 movie with a box office over $100 million dollars. 2015 saw an increase in the number of popular films and average box office, but ratings still fell from 2014. 2003 had a huge average box office per film, but TV ratings ticked down that year. Moreover, even blockbusters like Titanic weren’t enough to bump up TV ratings, if you look back that far.
How do we draw a conclusion from this? Well, first we admit that one variable like “popularity of films nominated for Best Picture” is just one variable among hundreds. Other variables like the host(s), the date of the telecast, the length of the telecast, the quality of the broadcast, major a-list celebrities nominated and more could all impact TV ratings. Focusing on one variable to explain all our conclusions is a fraught enterprise.
Assumption 3: The Oscars haven’t featured diverse films in the recent past.
Rating: False.
Okay, so I didn’t actually read anyone who wrote this specifically. Or relating it to the move to make “Achievement in Popular Film” a category. But you can’t talk about the Oscars since 2015 without addressing the #OscarsSoWhite controversy.
Arguably, no groups has benefited more from the move from 5 films per year than diverse films and filmmakers. (Except maybe science fiction films, which I’ll get to.) Take a look at the number of films featuring African-American characters or themes measured before and after the expansion in number of films:
The difference is stark.
In the 9 years since expanding the field, a film featuring African-American characters or themes has been nominated every year except 2015 and 2010, and 2009 featured two, if you count The Blind Side. I debated it since arguably Sandra Bullock is the main character and it is her story, but if we exclude The Blind Side, I’d rule out three of the films from the 2000s for the same reason, which would only make my case stronger. Of the four films in the 11 years before the change that I counted, three are ensembles that don’t really focus on African-American themes. I could easily say that only Ray really qualifies. Before the Academy expanded the field it really ignored African-American characters.
(And obviously I’m only dealing with Best Picture here, not the acting categories, which in a lot of ways was the key driver of the #OscarsSoWhite movement.)
Of course, I could define “diversity” in a variety of ways. Take “global diversity”. Has the expansion of eligible Best Picture nominees helped foreign language films? Not really. From 1998 to 2008, four foreign language films were nominated for Best Picture (Life is Beautiful, Crouching Tiger, Hidden Dragon, Letters from Iwo Jima and Babel) and since then only one film has been nominated (Amour), which is even worse when you consider how many more films are nominated since the expansion.
(I’d also say you could look into Latino-American-themed films, but you’d basically find zero examples of any films in any time period. The Oscars may be so white, but they’re even less Latino-representative than African-American representative. Asian-Americans fair similarly poorly.)
Assumption 4: The Academy wasn’t nominating enough different types of films.
Rating: False, but trending towards true.
One of the biggest topics around this year’s nominees was the bugaboo about Get Out. Specifically, I heard people claiming it was unique because it was a horror movie, and those types of films never get nominated.
Well…
To test this, I wanted to see how many “genre” films were nominated before and after this change. I defined “genre” films as any film in the following genres: science fiction, fantasy, war, musical, comedy or animation. And if a film in Box Office Mojo was more a drama than a fantasy (say, The Curious Case of Benjamin Button) or more a drama then a comedy (Juno was a tough edge case here), then I excluded it. Again, we’re looking for films known as genre films, not films that happen to have genre elements.
In this case, nothing much has changed. From 1998-2008, roughly 69% of the films were “drama” films. From 2009 to present, 62% were “drama” films. So it’s gone up by 7%, but that’s within the margin of error. Even the last two years which “felt” like not a lot of genre films were featured still had one-third genre movies.
Of course, for certain categories, the change has really helped. As I mentioned above, since the change 9 clear “science fiction” movies have been nominated: Avatar, District 9, Her, Inception, Gravity, Her, Mad Max, The Martian, Arrival and, last year’s winner, The Shape of Water. Some of those films were also blockbusters or popular. Though the three biggest science fiction films since Avatar (Star Wars) haven’t been nominated. War films have also done very well, but have been generally less popular than the science fiction films, including American Sniper, Hacksaw Ridge and Dunkirk. The other categories all had smaller changes which are more likely noise than genuine signal.
The last category I’d call out is animation. During the first two years when the Academy expanded it’s number of films, two Pixar films achieved Best Picture nominations, Up and Toy Story 3. Arguably, Pixar’s quality the last few years has been at its highest with Inside Out and Coco being frankly, masterpieces (Both were “universal acclaim” on Metacritic.) but it hasn’t seen the respect from the Academy. I would argue that if the Academy really wanted to train a new generation to love movies, putting films like Coco and Inside Out (and even Frozen) would help a lot.
Assumption 6: Politics hurts the Oscars.
Rating: True, but not for the reason you think.
Well, maybe for the reason you think.
The one narrative that was hinted at throughout the #OscarsSoWhite campaign was that featuring non-diverse filmmakers would keep a diversifying America from watching the campaign. As we saw above, though, diverse films have regularly been featured as Best Picture nominees. Instead, the arguably bigger lack of diversity comes from the lack of political diversity in the films.
To explain this, I go back to the biggest discrepancy in the data comparing TV ratings to average box office: why were 2014’s ratings SO high? Again, this year only featured one movie grossing over $100 million dollars, which happened to be its lone blockbuster. So it was an unpopular set of movies that also lacked any major A-list talent. What happened?
American Sniper was the one film.
American Sniper was a major topic on Fox News and other right wing new sites. That’s right, if the Academy is looking honestly at its whole slate of Best Picture nominees, this is pretty much the only film that could be labeled “right leaning” that has been nominated since 2010. (Maybe Zero Dark Thirty too.) The conclusion here is that far from “popular” films, the Academy needs popular films that also appeal to the political-cultural right. (I’m not necessarily recommending that, just acknowledging that this data says.)
Conclusion: A Summary
So here’s my short line summary of the history of the Academy Awards as it relates to popular films and TV ratings.
– By the end of the 1990s, the Oscars featured generally popular films such as Forrest Gump, Titanic, Gladiator and The Sixth Sense, while also featuring critically acclaimed but unpopular films such as Chocolat or The Cider House Rules.
- Then, from 2000 to 2008, the Oscars featured increasingly fewer popular films, as shown by multiple metrics. The nadir was 2005 to 2008.
– In 2009, the Academy expanded the number of films from 5 to 10 and changed the voting system. As a result, from 2009-2012-ish, the were more popular than the previous five year period.
– Since 2014, the Oscar films have trended downward in popularity, especially among “blockbusters”—films grossing over $250 million—which the Academy hasn’t nominated since 2014.
– The TV ratings have been on a general downward trajectory, though limited evidence (2009, 2010, 2012 and 2014) indicate that popular films can help increase TV ratings.
– The expansion in number of Best Picture nominees helped African-American films/film-makers more than any other category.
– While the films have decreased in popularity, “genre” films have been represented at roughly the same rate. In other words, “dramas”, which tend not to be “popular” have earned about 62-67% of nominations.
– Finally, the biggest discrepancy in the “TV ratings to film popularity” came in 2014, when American Sniper arguably drove the biggest TV ratings in the decade, a film that was the personal favorite of Fox News and its viewers.
I still have a ton of questions to answer on this (some fun, some business) but I think that’s enough for this post.
