Intangible Returns

It’s Monday morning.

Your calendar notifies you that Monthly Planning is starting now, so you fire up Zoom, and next thing you know you’re staring down a big list of ideas in a spreadsheet with your team.

The problem is simple: prioritize the list, so you can pick the best things to do next. But, how? This is one of the most important and challenging decisions every manager faces. 

Many high-performing teams use some form of an expected value calculation to prioritize the list, which, if adhered to too strictly, could cause problems. 

Here’s how it works:

For each idea in your list, follow these steps:

1. Quantify the impact you think the project would have if it works. For example, you might have an idea for a new landing page design that you’d like to A/B test. You think it could boost conversions by 10%, and at your current scale that’s worth roughly $10k in MRR.
2. Assign a probability between 0–1 of it actually working. In this case, you know a similar design in a different landing page had a 10% conversion boost, so although it’s hard to know for sure, you feel reasonably confident in the idea. You think there’s a 50% chance you’ll get the 10% bump.
3. Multiply the impact by the probability to get “expected value”. So $10k MRR * 50% = $5k MRR of expected value.
4. Estimate the costs. You estimate it’ll take 15 hours worth of design and engineering effort to run this test, and as a rough rule of thumb you peg your team’s time at $200 per person/hour. So $200 * 15 = $3,000.
5. Arrive at a “net estimated value” figure. In this case, $5k MRR - $3k in costs = $2k “net estimated value” in the first month, and an ongoing 10% improvement every month after that (for as long as the landing page stays relevant).

Once you do that, deciding what to do is easy! All you have to do is sort your spreadsheet by “net estimated value” in descending order, and start at the top.

Easy, right!

Of course, it’s much more complicated than this in reality.

First, it’s impossible to know for sure what the impact would be and how probable that impact is. The process of thinking it all through helps, but only so much. I’d bet on an experienced leader’s gut instincts any day over an amateur’s expected value calculation.

Second, it’s worth noting that ideas are connected to each other — they’re interdependent. So even if your spreadsheet says the “Landing page A/B test” has a higher expected value calculation than “fix the bug in our analytics querying interface” you might need to do the latter before the former, for boring, logistical reasons. And this is an artificially clean example. In reality, product teams have a seemingly infinite set of options to choose from, depending on the way they look at it. For example, would it be better to just run a single A/B test on that one landing page, or should we rethink our system for shipping landing pages entirely?