Category: Management Advice

Applying “VORP” to the Entertainment Industry: A Step by Step Guide

Think about your team right now. Either the people reporting to you or your peers. The people sitting around you in your cubicle or open office desk farm. The ones who should be working, but are probably reading the internet, like you are right now.

How many of them could you “replace” and see your team improve?

How many are just average?

How many are delivering LeBron James-esque over-performance?

Yesterday, I explained “value over replacement player”, a concept from sports that compares all all players to the average to determine how much value they add to the organization. And how rare they are, hence how much compensation they can demand. Unfortunately, as I also wrote yesterday, VORP isn’t commonly used in business. Most managers don’t look around at their peers and direct reports and judge them in “value over replacement” terms. They especially don’t measure performance that concretely.

That should change!

Today I’ll explain why. I’ll even provide some principles that lay the out basics for how to apply this to your team. And then I’ll provide some examples. From the entertainment business.

Before we get to the good stuff, we have to explain the difficulty with this whole enterprise.

VORP: Why not business?

Let’s start with one very simple answer, and then dig into the details:

Value Over Replacement is hard!

That’s it. It takes a lot of work, requires a lot of numbers to do it well, which requires a lot of thinking and analyzing, and then a lot of feelings could get hurt. But let’s get into some specifics.

First, there is a goal or “value” problem.

Teams applying advance metrics start with “value”. Ideally every part of every organization would know right off the top of their heads how they create value. In value based terms. But while most people could speak generally about value, very few could define it in concrete terms. Even fewer could tell you how well they did last year. To put this in sports terms, most teams (and even companies) couldn’t tell you their record from last year.

Each team’s goals should be aligned with the company’s mission. Since most businesses are pretty awful with goal setting (and accountability) you have a problem from the start. At its core, it’s really easy to account for “value over replacement” when you have objective numbers like runs scored and assists generated like in sports; it’s much harder when it is about running a team with an undefined goal.

(This is why most companies and analysts default to stock price, since it is arguably the goal, but mainly it is the easiest thing to track. Even profits/cash flow require digging into financial statements.)

I could pick on numerous traditional teams in movie studios. I’ll try to stick with just one for these examples: business affairs (BA). (For those who don’t know, the BA folks are responsible for negotiating with agents and managers for the deals that make up TV shows or films. All the deals too, including actors, writers, directors, producers and sometimes production staff. Sometimes they also do “Legal Affairs” making them BALAs!) I don’t mean to pick on BA, but they make a great example, because they usually have a clear mission and they have a defined skill set/experience (law school).

A BA team adds value in one of two ways: they either negotiate cost effective deals, or they close a lot of deals quickly. The balance between these two can change per company. Some companies—like Netflix and other streamers that aren’t constrained by budgets—care more about getting all the deals they want or the speed to close a deal. Other studios could pinch pennies, like say Viacom or Lionsgate.

But many (most?) studios never clearly define the goals for this team. Sometimes they want deals closed quickly, sometimes they want to save money, sometimes they want to get projects. The heads of BA usually don’t push on this either, since not having goals makes it easier to succeed. Overall, they definitely don’t define how the BA teams add value. In the gap, the BA teams just work really hard.

Second there is a data problem.

Or problems. Take the fact that even if teams do have clear goals, they have way fewer metrics measuring their progress towards these goals.

Read More

The Most Important Thing Business Can Learn from Sports: Value Over Replacement Player Explained!

Do sports provide good examples or case studies for the oft discussed topic of “leadership”? If we want to excel at leadership, should we study coaches?

On the surface, it would seem so. We cite coaches in particular for leadership all the time in the press. Having gone to UCLA, I was indoctrinated early on to believe in Coach John Wooden, and his pyramid of success, as the touchstones for good leadership. You can find books doing this. So many books. (At least five by my count.)

You can even hire current coaches to speak to your executive team or corporate board about leadership. Nick Saban charges $50-100K to provide this service.

Honestly, isn’t that all kind of nonsense?

Coaches have one of the easiest leadership roles of all leadership roles. All management/leadership is tough, including coaching, but there is a scale, and coaches have it a lot easier than others. They have players who rely—literally are utterly dependent—on their coach for their very future. Sure, a lot of coaches fail—most in fact; sometimes spectacularly—but that’s because sports is a zero sum game: the number of wins equals the number of losses each year.

Yet, I love studying sports for lessons. Love it. There are tons of principles and tactics ane best practics that apply to business in general. It’s just that the most beneficial parts are probably the hardest to figure out and apply.

To take just one example, I think many, many knowledge industries could benefit from the type of discipline embodied by the concept of “practice”. Consider this: college football coaches “only” have their players for 20 hours every week. The goal is to not waste a minute of that time to maximize their training, and hence output on the field on Saturdays. Do you as a manager have the same discipline with your team? Or do you have almost no idea how your team spends their time? (Except when they’re in a meetings with you, of course.)

If this sounds dictatorial, I get it. Imagine your manager tracking every minute of every day you spend in the office. If you chafe at the idea of planning your schedule down to the minute, then don’t bring Nick Saban to your board meeting to talk leadership. Read this article and you understand this is EXACTLY what he does.

So if you won’t use Saban’s single best tool to manage his players, why do you care what he has to say about “leadership”? John Wooden practiced a similar level of control. Coaches are great leaders, but they also control their players with a level of tyranny unheard of in most of corporate America.

(The article in Fortune I quote above cites “leadership” in the title, while again never noting that no CEO would manage his executives with this level of control. I’d add, most knowledge workers would loathe this type of control over their own schedules, but recommend it often for minimum wage factory, retail or warehouse workers, who don’t have a say. Sigh.)

Today, isn’t about complaining about misapplied lessons in sports, though I loved getting that rant off my chest. Instead, it’s celebrating something from sports that would be tremendous if we applied it to office work and corporations broadly. The single biggest insight, in my opinion, from sports statistics (I hate the term “advanced analytics”, since I don’t know what simple analytics are) that most managers should use is:

Value Over Replacement Player

VORP and its cousin WAR (wins above replacement)—from basketball and baseball respectively—are two holistic measurements that take multiple statistical variables and combine them into one measurement. The goal is to get a single measurement that is best correlated with predicting future performance. The value comes from applying that single number to evaluate all potential players against each other; hence, it directly compares everyone to the average.

Today, I want to explain what “value over replacement” is, how it works, and the principles behind it. Tomorrow, I’ll try to apply it to everyone’s favorite industry, entertainment.

The Origins of “Value Over”

In Hollywood, there are really two camps when it comes to sports: either you follow sports super closely, or you hate it, with little in between. For those who hate it, here’s a brief summary on how analytics started in sports, which birthed us WAR/VORP.

Analytics started in baseball in the 1980s by many people, but was popularized by arguably one man, Bill James. He wrote an influential publication that created new statistics for baseball that he believed better captured the influence of various players and strategies. For instance, they tried to downplay the role of batting average. He called this sabermetrics after the organization he founded. James was followed by some other people, notably Billy Beane, general manager of the Oakland As (popularized by Michael Lewis in Moneyball) and Nate Silver, now of the election website FiveThirtyEight.com.

This deeper dive into statistics started in baseball for arguably two reasons. First, baseball is the most “statistical” sport. It has a huge amount of data and a lot of people poring over that data to compare current players to the past. By playing 180 games and tracking most everything that happens during every play, your sample size goes way up. Second, a lot of people play fantasy baseball, which is taking all that data and trying to apply it to fake sports. This was how Silver entered the analytics movement.

The critical challenge for these sabermetricians was to move beyond simple counting statistics and try determine, holistically, how much each stat at the end of they contributed to wins. This led to statistics in baseball like “win shares”—from Bill James—and the one I mentioned above “wins above replacement player”—author unknown, as far as I can find.

As I wrote in my post on “Has Hollywood been Moneyballed?”, baseball is a team sport that cares about winning. Like all team sports. The best way to do this is to have the best players, in general. With more and more people scouring data and drawing conclusions, eventually the baseball teams realized these people could help them win more games. This arguably started with the Oakland A’s and their manager Billy Beane, which was immortalized in the early 2000s, by Michael Lewis in Moneyball, and the idea of using data in sports took off like a rocket. (And people like me wrote articles about it then and now.)

Oh, I guess there is a third reason that explains the explosion of “analytics” in sports. Computing power and storage has increased exponentially from the 1980s to 2010s, making statistics easier for everyone. This has caused the amount of statistics in general just to increase.

The next most data heavy sport is basketball, so the next “holistic” metric popped up there, Value Over Replacement Player, or VORP. (The timing of this little history may be slightly off, but that’s fine. It’s close enough for the internet.) That said, the “democratization” of sports data—and the fact that a lot of geeks/nerds also love sports and fantasy sports—meant that basketball doesn’t just have one “holistic measurement” but several, all of which get to the same point, including PER, VORP, Win Shares, Real Plus/Minus and Box Plus/Minus. All are attempts to summarize a player’s value in one number that can be compared to every other player.

So that’s where it comes from—a single statistic in multiple sports to define value—what is it?

Value Over: How it Works

Let use an example to show how the concept works. I’m going to stick to basketball, because it’s the sport I know best.

Let’s say Player X scores 8 points per game. The key question for a team looking to acquire him is, “Well, how valuable is that?” Let’s go to the numbers. In the NBA last season, 540 players played at least a minute of NBA game time. (I’m using Basketball Reference for my statistics here.) Here’s how many points they scored per game:

Slide1You’ll notice it isn’t quite logarithmically distributed, and as I’ll hopefully finish next week, everything in media and entertainment is logarithmically distributed. That includes sports. The trouble here is that “per game” statistics technically combine two metrics: games played with points scored. If you just focus on points scored…
https://www.basketball-reference.com/leagues/NBA_2018_totals.html

Slide2

That’s better. Logarithmic distribution rules the day again!

Let’s try to understand these two charts. A player scoring 8 points per game is roughly right in the middle of the NBA. The mean average of points per game is 6.6 and the median is 8. In this case, the median gets closer to what we mean by “average” when it comes to points per game.

But you know I hate averages, which is why I put the distribution charts up first. The distributions are arguably the most useful way to look at this since we can quickly see how many players score how many points in various buckets. Combining players who score 6-8 points per game and 8-10 points, we see that about 150 players are in this range, or roughly a quarter of all players who played in the NBA last year.

So now we ask: is scoring 8 points per game valuable? Not really. Or in other words, it is about exactly average, which is how we should compensate Player X. Basically, scoring 8 points per game is very common. Even if a player played all 82 games averaging 8 points, he’d only move up to the 65 percentile in scoring, meaning 35% of players scored more than him. In other words we can find a replacement for that player easily.

What about Player Y—who I’ll call LeBron for short—who scores 27.5 points per game? Well, only two other players score more than that per game. In other words, this hypothetical LeBron fellow is extremely rare and hence extremely valuable. (Welcome to Los Angeles!)

LeBron is a good example, because he doesn’t just score points: he passes and rebounds too. If we’re trying to capture value, we need to value those activities too. In basketball you can count rebounds, assists, steals and the shooting percentage on a variety of shots. Add all this up—with a lot of other calculations and adjustments—and get closer to calculating the “value over replacement” for any individual player.

I’d add, even if you aren’t using a specific metric like WAR or VORP, Moneyball or analytics-minded or sabermetrics-minded general managers like Daryl Morey or Billy Beane think about players in these terms. You either take points, runs, wins or “value” and think, “how much higher is this player’s performance than the average player, who I can find easily?”

Value over Replacement: Complications

That’s the concept, but it quickly gets complicated.

The first part is the challenge to gather all the data. This seems easy (grab all the box score) but even the box score only captures so much. In recent years, teams have begun collecting “movement data” on the basketball court, baseball diamond and football field. This means tracking the movement of the ball and players, which is a lot of data, but allows you to track speed of players, yards run, where shots occurred and how far baseballs traveled. Lots and lots and lots of data.

The downside with this new data is the sample size is limited in years. Take blocks in basketball. We didn’t track blocks in the 1970s, so we don’t know how many times Kareem Abdul-Jabbar blocked his opponents. Same with sacks in the NFL. This means our data sets are limited by the years we collected the data. Moreover, in basketball, for example, a lot of the box score statistics are weighted to the offensive end (points, rebounds, assists, shooting percentages) versus the defense (mainly steals, blocks). This applies to baseball too. This is an example of how what we measure—which is sometimes what is easiest to collect—could could skew our perspective.

Then, once we have the data we have to judge how to weight it. I really like “box plus/minus” as a tool to judge players, and Basketball Reference has a great explainer for how they developed it. Read that explainer and you’ll discover it’s really complicated. It involved a lot of regression analysis and a large sample size. Then using that analysis to weight each statistic. Then testing its predictive power on another half of the data set. That’s a process that requires a lot of art and a lot of science.

Finally, one piece of data that is particularly tough to assess is the context of the data. In sports, this can mean the performance of the entire team and teammates. Going back to my player named “LeBron”, being on a team with a guy who can score 27 points a game and dish out 8 assists is extremely beneficial. It could be the case in basketball—and it is—that playing with fellow all star players makes your numbers go up as a result. This is just one example of how “situation” can improve your position.

If being on a good team is valuable with good players, this could apply to what “league” you play in. It’s easier to be great in college football than professional football. It’s easier to be great in the triple A baseball than the major leagues. So applying the statistics of one level to the next can be difficult. Context matters, and you have to account for that.

Applying to the Business of Entertainment

That seems like a simple proposal and concept. So why hasn’t this genius concept made it to business? Well, that will take another article to explain. Tune in tomorrow.