Think about your team right now. Either the people reporting to you or your peers. The people sitting around you in your cubicle or open office desk farm. The ones who should be working, but are probably reading the internet, like you are right now.
How many of them could you “replace” and see your team improve?
How many are just average?
How many are delivering LeBron James-esque over-performance?
Yesterday, I explained “value over replacement player”, a concept from sports that compares all all players to the average to determine how much value they add to the organization. And how rare they are, hence how much compensation they can demand. Unfortunately, as I also wrote yesterday, VORP isn’t commonly used in business. Most managers don’t look around at their peers and direct reports and judge them in “value over replacement” terms. They especially don’t measure performance that concretely.
That should change!
Today I’ll explain why. I’ll even provide some principles that lay the out basics for how to apply this to your team. And then I’ll provide some examples. From the entertainment business.
Before we get to the good stuff, we have to explain the difficulty with this whole enterprise.
VORP: Why not business?
Let’s start with one very simple answer, and then dig into the details:
Value Over Replacement is hard!
That’s it. It takes a lot of work, requires a lot of numbers to do it well, which requires a lot of thinking and analyzing, and then a lot of feelings could get hurt. But let’s get into some specifics.
First, there is a goal or “value” problem.
Teams applying advance metrics start with “value”. Ideally every part of every organization would know right off the top of their heads how they create value. In value based terms. But while most people could speak generally about value, very few could define it in concrete terms. Even fewer could tell you how well they did last year. To put this in sports terms, most teams (and even companies) couldn’t tell you their record from last year.
Each team’s goals should be aligned with the company’s mission. Since most businesses are pretty awful with goal setting (and accountability) you have a problem from the start. At its core, it’s really easy to account for “value over replacement” when you have objective numbers like runs scored and assists generated like in sports; it’s much harder when it is about running a team with an undefined goal.
(This is why most companies and analysts default to stock price, since it is arguably the goal, but mainly it is the easiest thing to track. Even profits/cash flow require digging into financial statements.)
I could pick on numerous traditional teams in movie studios. I’ll try to stick with just one for these examples: business affairs (BA). (For those who don’t know, the BA folks are responsible for negotiating with agents and managers for the deals that make up TV shows or films. All the deals too, including actors, writers, directors, producers and sometimes production staff. Sometimes they also do “Legal Affairs” making them BALAs!) I don’t mean to pick on BA, but they make a great example, because they usually have a clear mission and they have a defined skill set/experience (law school).
A BA team adds value in one of two ways: they either negotiate cost effective deals, or they close a lot of deals quickly. The balance between these two can change per company. Some companies—like Netflix and other streamers that aren’t constrained by budgets—care more about getting all the deals they want or the speed to close a deal. Other studios could pinch pennies, like say Viacom or Lionsgate.
But many (most?) studios never clearly define the goals for this team. Sometimes they want deals closed quickly, sometimes they want to save money, sometimes they want to get projects. The heads of BA usually don’t push on this either, since not having goals makes it easier to succeed. Overall, they definitely don’t define how the BA teams add value. In the gap, the BA teams just work really hard.
Second there is a data problem.
Or problems. Take the fact that even if teams do have clear goals, they have way fewer metrics measuring their progress towards these goals.
At a high level, the problem stems from a lack of data. How do you measure the amount of time each employee spends on a task? Do you measure the quality fo each task? Do you measure any of those things? Probably not. Broadly speaking, hardly any companies track “statistics” for “knowledge” workers. We could—it really wouldn’t be that hard in a digital age—we just don’t.
Let’s go back to the BA team example, to show this in specifics. Even if the BA team had a clearly defined goal—say negotiations should close as quick as possible—that would mean keeping track of when negotiations start and end. It would also probably require keeping track of the financials to make sure they didn’t blow up budgets in the process.
Yet, those higher level metrics are really the “wins”. The BA team would then need to track emails, phone calls and meetings devoted to each project to see how they can speed up the process of each piece. That means a ton of data on employee time management, which they don’t track. (And probably don’t know how.)
Even if you read this and decide, “Dammit, I’m doing it Entertainment Strategy Guy”, your data will only start from the moment you start tracking…which means you have a small data set. The data set starts the moment you start tracking, and then you need to wait for results to see what is correlated with success. That takes time.
Third, working in teams makes all this harder.
Call it the “situation” problem. I mentioned this yesterday in my “complications” section when I described how some sports “value over replacement” metrics try to take this into account.
The logic is it is easier to get good stats playing next to LeBron James or on the Golden State Warriors—a great player and a historically great basketball team, for the non-NBA crowd—then it is on the Sacramento Kings—one of the league’s worst. Basically, you don’t want to ascribe to individual talent what may actually be team or company success.
This problem is heightened in a business context. It’s easier to generate leads for McKinsey than Deloitte, and easier for both then a boutique consulting firm. So it isn’t just about your value in the “industry-wide” setting, but accounting for the trajectory of your firm and many other situational factors.
In our BA example, it may be easier to be a Business Affairs exec who is crushing deals for Netflix—with the seemingly unending pocket book—or Disney—with their track record of hits—then a minor production company. It definitely is. If someday we compare BA execs across industries, we’d need to account for “team” and “situation”.
Value over Replacement: The Principles
Writing out those challenges above, I noticed a trend emerging. A pattern. Those challenges provide us a roadmap for how to implement this in our teams. Overall, I want to make this as actionable as possible. (I’m a believer in the Manager Tools philosophy that don’t tell me what something is, tell me how to use it.) So here it goes: here’s how to create a “Value Over Replacement” Tool for your company.
(I’d also reread my guide to “What is Moneyball?” because the “value over replacement’ metric is a tool that fits under that general guide.)
First, identify how your team creates value.
Notice that I keep bringing this back to “wins” or “value”. If you read my article on Moneyball, you understand the importance of knowing what the ultimate goal is. In sports, it’s usually wins (which leads to championships). I focused on the term “value” over the term “win” in these articles for a reason. Winning games is the currency of the NBA or MLB or NFL, but you don’t get wins in business or the entertainment business. Instead, you generate “value”, which can either be profit or consumer surplus, which I’ve been writing about a lot recently.
Individual teams inside a corporation or company are a lot like individual businesses in that they can either create or destroy value, depending on how well run they are. So before you can judge your people, you need to identify how your team creates value. Do you make things better? Go faster? Save money? Determine that first. Then go to step 2.
(Also, when it comes to costs, replace it with “resources” to better expand the scope. An employee has a set amount of time, which is a resource. The more done in less time the better.)
Second, identify the metrics that go into creating value.
For baseball, this means the hitting and fielding statistics, except for pitchers who it is about the balls, strikes and outs they get. For basketball, these are all those stats above that help a team win. For both sports, “efficiency” is also important.
Once you know how your team creates value, then you need to identify the individual metrics that add up to success. These are the inputs and their success rates. This will take time and in a lot of cases means you’re starting from scratch. Still, better late than never.
Third, find which metrics correlate the most with creating value.
In any given field, there are a lot of statistics, but not all are created equal. In basketball, steals aren’t as valuable as their prominence in the box score. Even things that you wouldn’t think are correlated to wins—like minutes played—can end up contributing. Regression and other statistical analyses will help you determine this.
This step is key because it isn’t just about having metrics, but understanding how well correlated they are with creating or destroying value. Also, back to the moneyball ideas, it isn’t about what you assume creates value, but what you can prove creates value. For decades, batting average was the key measurement for hitters in baseball. It isn’t correlated with winning as much as on base percentage in baseball. Same with points scored in the NBA; it turns out shooting percentage is more important than points scored, in many cases.
So do this analysis for your team. It will take time. And lots of thinking/analysis. But it’s key.
Fourth, account for situation.
Is your company or school in the top tier or in the middle? We have to account for how all of this can skew performance. It’s easier to market Disney films, right now, than Paramount. (No offense intended, that’s just the truth.) At first, steps 1 through 3 will take all your time, but be prepared to address this as you grow your data.
VORP in Action – Some Examples
Honestly, I wrote the preceding 4,000 words (I check word counts a lot, metrics!) to get to this section. The fun. I desperately wish I had the numbers to show which teams are above and which are below average across every company. To cover entertainment like sports.
Instead, these are hypotheticals. I’m going to go through various teams and departments in a typical film or TV studio or production company and explain how its “value over replacement X” statistic could look like.
(Oh, and one particular job will become its own article. And it will be gloriously controversial.)
I took out the term “player” and replaced it with “X”. I was tempted to use “employee”—making it VORE—but that would mean every employee is judged with the same metrics, even if they have a specific skill set or technical role. I also think this makes it more flexible.
VORBA – Value over Replacement BA
They were our example up above, so it’s the easiest to start with. The key metrics here are related right back to the role of the business affairs executive, which is closing deals at a good price. This metric would likely be…
VORBA = Number of deals closed * time required to close / total value of deals above average.
A replacement level BA does an average number of deals, many of which take a long time to close. Many of these deals cost the company more than they would like. A great BA closes a lot of deals, with good financial terms and does it quickly. The inputs will take time to figure out, but shouldn’t be too hard to track.
I’d also add, I could see a role for “legal affairs” in ensuring that contracts don’t have hidden or bad clauses that hurt the company. Overall, though, legal affairs would be judged mostly by time: can you close deals quickly? (My experience? No.)
VORPE – Value over Replacement Production Exec
In the studio sense not the people acting as production managers on set. Production executives are the people who oversee all the productions. Ostensibly, their job is to keep the train rolling on time and on budget. With those two metrics, it could be pretty easy to see if they are above or below replacement. The ones with a lot of shows missing budgets aren’t. The ones who repeatedly bring things in under-budget are.
I could see VORPE having two key inputs: the timing measurement and the cost overruns. So you take the number of shows and the days late and that gets to your “delivery skill” metric, and you take the cost overruns and you get your “financial metrics”. An above average executive could deliver lots of shows on time and on budget. In a dream scenario, the production executive could make this a competitive advantage for their company.
VORPPE – Value Over Replacement Post-Production Exec
I mean, this is like production executives, but focused even more on time and budget, if that’s possible.
VORDE – Value Over Replacement Development Exec
On the one hand, the “wins” are really easy here: make hit TV shows or movies. The good ones make great shows. The bad ones make average shows. The ones who do this the most, hypothetically, should run networks and studios.
It’s all the inputs that make it complicated. You could evaluate how good each development exec is at each stage of development, from taking pitches to reading scripts to giving notes on scenes. Add it all up with a dash of programming, and you’re on the way to a VORDE.
In reality, none of that is tracked. Not really. There are so many metrics to measure here and evaluate development execs, none of which are. You could evaluate how well development execs judge scripts, predict which pilots will succeed, predict successful casting, bring in pitches. This is a data rich environment that is mostly uncollected.
More on this to come.
VORM – Value Over Replacement Marketing
This one is tough, mainly because I have trouble finding a marketing team that isn’t at least a dozen people strong. How do you determine who does what? In my experience, at multiple companies, no marketing executive travels alone: if there is a meeting, 20 people show up.
There’s also the “goal” problem. Bad shows or films are hard to market and vice versa. So marketing and development execs constantly battle over sharing the blame. Also, VORM sounds like “worm”, which isn’t a good start.
I’ll say, just because it is tough, doesn’t mean it is impossible, and the first step is to start collecting metrics. Marketing delivers openings good or bad. And more marketing is measured and tracked then ever before.
VORCEO – Value Over Replacement CEO
To conclude my examples, I’ll drop my most socialist take: most chief executive officers BOMB the VORCEO.
Let’s say that again, most CEOs have extremely little value about replacement level CEO. For one, it’s a definitional thing, at least half are below average. But more importantly, most CEOs take credit for stock increases that are inline with S&P and industry average. If the CEO was merely going to keep the firm on average—and if most CEOs are average anyways—then you could drastically cut their pay and achieve the same results. Actually, you’d do better, since you saved costs.
Instead, CEO pay has been rising across corporate America, even though performance hasn’t actually increased. (Again, profits are up in general, but a tax cut had more to do with that than individual firm performance.)
The Alternative to VORP? Perception of Work
I’ll leave you with a depressing thought about the state of work. Another reason why “Value Over Replacement” doesn’t happen. Or more precisely, what replaces it when you don’t have goals based on creating value and measurements towards those goals:
Managers replace value creation with “perception of value creation”.
And there is a difference.
Basically, if no one knows how to evaluate your work, the best way to evaluate your work is the perception that you are doing a lot of work, which must be valuable. This means the employees who are the most visible and seem to be working the hardest will generate the most praise.
This is why I would add a caveat to all goals and metrics that “internal communication” is rarely the key metric. In none of the value creation metrics above do I mention email, meetings or phone calls. Emailing, meetings and powerpoint are tools to deliver value, not the value in and of themselves.
Even reports and presentations can be an end, but they aren’t the end state. It isn’t the goal. And yes, relationships are vital to great companies. So is great communication. But the people communicating the most may not be delivering the most value…and in fact could be inhibiting it.
