Most organizations–from corporations to sports teams to armies to universities to government agencies to non-profits–think they use a sound process to make important decisions.
But they aren’t. They are arriving at the easiest decisions.
I’ll use a media example to show the worst process. Basically, political pundits opine and opine and opine for months about an upcoming election. Then when it happens, they go on air and, if they were wrong, ignore what they said before, or, if they were right, embrace it wholeheartedly. Either way, they develop a narrative to suit whatever the data says. That’s an easy process.
The opposite process is much harder. In my opinion, the gold standard of openness is the journalism of FiveThirtyEight. Nate Silver and his team go out of their way to be open with their process, even explaining how their election model works (or doesn’t). They also write articles owning up to mistakes when they make them. The highlight was when Silver admitted that he didn’t acknowledge the data that showed Trump’s underlying strength during the primaries. He corrected and didn’t make that mistake in the 2016 general election.
I wish corporate America had such an open process.
I want to emulate Nate Silver’s approach to tough problems. He has politics and sports covered (along with some other great journalists), so I’m going to focus on entertainment. I want to ask big, interesting questions, and use a sound decision-making approach to find unique insights and answers.
Honestly, I want to get better at analyzing problems and hopefully making better decisions overall. I want to be more accurate. To do so, I am going to use the decision-making process I wish I could have used in my last job. Yeah, I stole Nate Silver’s philosophy, but I’m tailoring it to my style.
That’s the process I want to explain here, once, so I can reference it in all future “analysis” articles. (I’ll explain what those are in a separate post.) I want to be very open with my process. This way, when people criticize my conclusions, I can point to my process and ask for recommendations for how to improve that. Hopefully, it will also make other people better at understanding their processes.
Part 1: Blink Analysis
I start the process with my “blink” analysis. Yes, I stole the name from Malcolm Gladwell’s book. Like any long Gladwell topic, you never quite know how well the initial ideas have stood up to longer-term academic scrutiny.
But the core idea is fantastic. As soon as you hear something, you get an instant reaction. And I try to capture that as soon as I get an idea for a question. I try not to think about it, but just put my thoughts on paper. Consider this my “looking at a statue and calling it fake” analysis for those who have read Gladwell’s book.
My working theory is that most of us do this ten times a day in conversation. You hear some news, “Did you hear that so-and-so was hired at such-and-such company?” and draw a conclusion, instantly. We just don’t acknowledge it. I’m trying to acknowledge it up front.
Part 2: The Gut
My next step is the “gut”. Here, I try to write down what I think about a question, but my goal is to reflect a bit deeper on it. Whereas the blink is a paragraph, this could run to a page or more.
The best analogy is the “case question” in an interview. In my analysis articles, I’ve asked a challenging business question (to myself) and now I need to see if I can answer it. And like a case study question, I won’t have any data to go from. I can divine some data I think I know or know for sure, but I can’t use the internet to confirm it. So my “gut” analysis is deeper than blink, but without the numbers that come from part 3 of the process, my analysis.
Honestly, when I set up this process, I realized how often this is essentially what we do in a business meeting. Sure we get initial blink answers, but as we have a conversation about an important issue or the future of entertainment (say at a conference talk or in a group or around a table in a meeting), we’re having a “gut” conversation. We can’t pull up numbers, so we do our best guesses.
The reference book here is Thinking Fast and Slow, or The Undoing Project, which are stand-ins for the work of professors Kahneman and Tversky. Two of the founders of behavioral economics, they basically show that “blink” goes wrong when it relies on misleading heuristics. In other words, your blink can often mislead you. The gut stage is supposed to short circuit that quick analysis to allow a moment of reflection.
That said, the gut may be the most dangerous or seductively appealing of the three stages. My “gut analysis” is NOT my final answer. The gut can be tremendously misled by assumptions we haven’t fully explored because we don’t have enough time to do it. Or the data doesn’t support it. It gives us unearned confidence.
This is why I think Hollywood overall makes most of its bad decisions, it stops after the gut decision-making. So how do we make our gut a little more accurate? We do the analysis.
Part 3: The Analysis
This is really the meat of the process. This is where we make better decisions. And it’s the toughest part.
So what is it? It is about deciding on a framework, and putting numbers to that framework to derive an answer. It won’t be the same process each time—that’s asking too much from any framework—but every process should have some numbers. Those numbers are then analyzed. Throw in more quantitative and qualitative data along with history and experience. But it is ultimately boiling down to some numbers, whose judgement I have to accept.
A key to making this work is to do the analysis myself. In most cases that will mean asking the right questions, finding the data, building the models and thinking about all that I’ve analyzed. The key, though, is doing the work myself. I specifically want to avoid the thoughts of others as long as possible in this part.
Why? Because I want to be part of the experts in the crowd, not the crowd-source.
Let’s explain that. Take any one expert. Usually, they are pretty inaccurate. Sometimes they have hot streaks, but overall they can be wrong often. A real world example is NFL experts picking games. Any one expert gets a lot wrong. But if you poll a dozen experts, they’re more accurate than the majority of experts. In entertainment, the gold standard is Gold Derby, whose poll of Oscar predictors is more accurate than most other predictors.
Another example is Nate Silver taking a poll of polls. He doesn’t run his own poll—a new set of data that can be inaccurate—but he polls the polls to make them more accurate. In this analogy, I actually want my analysis to be like the NFL expert or individual poll or the individual Oscar predictor. I’ll be inaccurate often, but if you combine my analysis with others, you’ll have a more accurate picture of the world.
By doing the analysis myself, I’ll come up with unique insights. This means I’m adding analysis into the world, not simply parroting opinions I’ve read elsewhere. I don’t want to be the person collecting the crowd-source, but adding data and analysis into the world you can’t find anywhere else.
My caution is I’m doing this because I can. I don’t do this exhaustive analytical process for fields I’m weak in. Take fantasy football. I can’t watch every football game in the NFL, so I don’t try. Instead, I rely on experts who do and statistics that have predictive accuracy. I think I can add my own individual expertise into the analysis of the entertainment industry because I excelled at this analysis in business school and in my career, and I consider myself an expert. I wouldn’t be writing all these words otherwise.
Part 4: Research and Other Opinions
Doing analysis myself doesn’t mean I ignore other opinions and analysis. After I’ve done my opinion, I head to the crowd. See what they think. Then, I weigh whether any of it has a strong enough weight to change my mind.
In some cases, like comparing “Atom Tickets to Fandango to Movie Pass”—a future article I’m working on—there won’t be a ton of people weighing in on the issue. Or there won’t be a substantive analysis to compare my own to. In other cases, like evaluating The Walt Disney Company-21st Century Fox acquisition, lots of people analyzed that deal at the time.
There is another analogy to polling that provides the one caution at this stage. One of the things Nate Silver worries about is when he sees polls cluster around the same result. This worries him because he thinks pollsters are subtly manipulating their polls to arrive at a conclusion. The polls do this because they don’t want to seem like outliers with the negative connotation that comes from that.
I think this phenomena plays out all the time in the world of business. Doing your own analysis takes a lot of time, so it is easier to let someone else do it. So a lot of the business world (and stock market) cluster around similar opinions even if their own analysis pointed somewhere else. (Or would point to it if they took the time to do a clear process.)
So after I do my own analysis, I’ll read other people’s thoughts and update my thinking. See where I go against the grain and see where I don’t. This may be in my analysis or in later updates, I haven’t decided yet.
So that’s my ideal process. Now it’s time to see how it works.
