Product Discovery

So you have a general vision for your product; a vivid and inspiring guiding light for what it could be and the huge impact it could have on the people using it. You’re fired up, brimming with ambition and ready to roll your sleeves up and make this dream a beautiful reality.

But how do you know exactly which problems your product needs to solve, and the features that will solve them?

Knowing what to build, and more importantly, whether what you're building will actually be valuable to both the business and the end users, is identified through a process called Product Discovery.

Now that we know what product discovery is, the next question might be when you should carry it out. The answer is constantly, all the time.

Let me explain.

Continuous Discovery

In Teresa Torres’ excellent book ‘Continuous Discovery Habits’, she outlines a practice referred to as “continuous discovery”, which she defines as follows:

At a minimum, weekly touchpoints with customers,
By the team building the product,
Where they conduct small research activities,
In pursuit of a desired outcome – Teresa Torres, ‘Continuous Discovery Habits’

The premise is that product teams make decisions every day, so the more frequently that team communicate with their customers, the more likely they will be to make those decisions with their customer’s needs in mind.

The goal is to continuously discover unmet customer needs, and the solutions that will address those needs.

Research in pursuit of an outcome

Increasing the frequency that product teams meet with their customers is a big step forward, but alone it’s not enough. The secret is hidden in the last line of the continuous discovery definition - “in pursuit of a desired outcome”. In order to understand why this part is so pertinent, we need to revisit one of the core principles of product management - outcomes over output. I wrote about this in more detail in the ‘Product vs Project Mindset’ section here.

While it’s true that product teams aren’t communicating with their users nearly enough, even those that are often aren’t talking to them in pursuit of a desired outcome; continuous discovery isn’t about doing user research to tick a box or look good to management, it’s about conducting research with the specific intention of uncovering the customer needs, pain points and desires that the product team can then address to achieve the desired business outcome.

In any product with sufficient product market fit, there are a set of activities that sit directly at the intersection of what the users want, and what the business needs to do in order to fulfil the strategy and achieve it’s goals and objectives. Good Product Managers operate at the point at which these two converge.

The prerequisite mindsets

In the book, Teresa Torres outlines some mindsets that need to be adopted to allow continuous discovery to flourish in your organisation. Upon reading them, I actually think many of them are mindsets that should be adopted to allow the entire practice of Product Management to flourish in your organisation, so I’m listing them here:

1. Outcome oritented - this comes up time and time again in so many Product Management books, and I’ve touched on it above. All too often companies become delivery focussed rather than value focussed, which typically means they start focussing on outputs rather than outcomes. That’s a slippery slope that can very easily lead to becoming blinded by vanity metrics (number of tasks delivered, speed of ticket completion), rather than whether you’re actually building something that adds value to users in a way that leads to profit for the business. Don’t be one of those companies, focus on outcomes.
2. Customer-centric - this one is an essential addition to the first; if you’re outcome-oriented, you need to pair that with a customer-centric mindset. In the midst of a high pressure working environment and loud stakeholder requests, it’s so easy to lose sight of the fact that the customer is ultimately the most important voice to listen to, by far, and when you lose sight of that you run the risk of delivering outcomes that serve the business in the short-term but are terrible for the customer and therefore cause long-term harm. Place the customer at the centre of your world, and be laser focussed on solving their problems in a way that also serves the business.
3. Collaborative - fully embracing the idea of a ‘product trio’, where a Product Manager, UX Designer and Engineer all work together on identifying and delivering value to the product, instead of each working in a silo. This means all members of the product trio should be involved in discovery too, as each bring a different perspective to the same customer insights.
4. Visual - rather than just talking through ideas or writing them down, make them visible; draw them, diagram them, map them out on a Miro board.
5. Experimental - think like a scientist; identify a hypothesis then gather evidence to prove or disprove it. I’d argue this is one of the most powerful habits you can adopt as a Product Manager.
6. Continuous - rather than thinking about discovery as something you only do at the start of the product development lifecycle, you infuse it throughout the whole process. This allows you to continuously gain customer insights and adapt your product accordingly to prevent it from ever getting out of touch with what customers want. Think about it like this - if it takes a year to build and launch your product, and you only speak to customers at the start of that process, you’re making 12 months worth of product decisions without updated customer input.

The structure of good product discovery

Like much of product management, product discovery is an ill-structured problem. I gave a more traditional definition of ill-structured problems in the philosophy section here, but a more Product Management specific way of knowing whether you’re dealing with one is whether you’re immediately hit with a crashing wave of anxiety when your manager asks you to do a certain task and you realise you have absolutely no idea how to even begin to approach it.

To help you avoid that tidal wave of workplace anxiety, let me now break into something more concrete; a series of steps that comprise the structure of good product discovery so that we can do that most Product Manager-y thing of converting the ill-structured problem into a well-structured one, so that we can know what the hell to actually do.

1. Definte the outcome - know what outcome you’re speaking to users in the pursuit of achieving. This will be a business need, because ultimately the business needs to make money or you wont be able to serve your customers at all.
2. Map out the opportunity space - to reach the desired outcome, the product trio must explore the opportunity space; the user needs, pain points or desires that, if addressed, would help you achieve the outcome.
3. Identify and validate the solutions - figuring out the specific ways your product will solve for the opportunities in order to deliver the desired outcome, and validating them to ensure they definitely will.

The magic of this model is that it ensures alignment between doing what’s right for the business, and doing what’s right for the customer; the outcome is a business need, and the opportunity space is a collection of user needs, pain points or desires that deliver the business need if you solve them. Everyone’s happy.

This three step process is a useful framework for knowing where to start, but I admit that the simplicity of it belies the wall of text I’m about to hit you with as we break down each one, so grab a coffee and buckle up. We’re going on an adventure.

The detailed breakdown

Defining the outcome

This will come from your product strategy; a business need, goal or KPI you’re aiming to hit. What it most certainly is not, is a specific feature.

Let’s take a specific example - imagine you had a KPI in your product strategy to increase daily active users by 30%. Theoretically there’s all manner of possible ways to achieve that, but which way will actually work for your specific users? That’s what you’re aiming to uncover during your weekly sessions with them.

Mapping out the opportunity space

As mentioned earlier, finding the best path to your desired outcome is an ill-structured problem, but how the product trio define and map the opportunity space is how they give structure to it.

I don’t love the term “opportunity space” because I find it ambiguous and a little too much like a corporate buzzword that I don’t want to run up the flag pole and circle back to, but it can be more easily conceptualised by thinking of it as what you can learn about your users from the research; their pain points, desires, goals.

You ask your users questions (for details on which questions to ask, see the User Research section here), then the answers to those questions generate a list of problems that would provide value to your customers and your business if you solved them. That list of problems is your “opportunity space”.

However, an unstructured list of opportunities is almost impossible to prioritise and execute on. If you’re a Product Manager at Netflix for instance, how do you prioritise between solving for “I can’t find anything to watch” and “I want to see what my friends are watching”.

To help with this, you map out the opportunity space on an ‘Opportunity Solutions Tree’ (OST).

OSTs help to visually represent the paths you can take to reach your desired outcome (point 1 in the aforementioned framework for product discovery - identifying the outcome, mapping out the opportunity space and identifying and validating the solutions).

Here's an example:

As you can see, the desired outcome sits at the root of the tree, then the opportunity space branches out from there; the customer needs, pain points and desires that can deliver the desired outcome if addressed.

Underneath the opportunity space sits the solution space, where you link the solutions you’re exploring back to the opportunity they’d solve for.

Then assumption tests are linked to each solution. These tests are how each solution is validated to give you the best possible chance of going after solutions that will actually deliver the opportunity.

The beauty of OSTs is twofold; they nail point 4 of the prerequisite mindsets for good product discovery by making the process more visual, and they essentially provide a roadmap to prevent you from getting lost on what to do next during discovery - you work vertically down the tree to flesh out a given opportunity.

When you go about actually mapping out your tree, what you’re looking to do is collate all the opportunities (user needs, pain points and desires) that you identified during the user interviews, then arrange them on the tree by two types of connections; parent-child connections (when an opportunity is a subset of another and therefore sits underneath it in the tree) and sibling connections (a separate opportunity that helps achieve the same desired outcome or opportunity as another, but isn’t directly related to it and therefore can be addressed without having to address one of the siblings).

This means grouping together opportunities that are similar, then finding a logical parent.

Let’s look at an example - imagine we had the following customer insights that came up in user interviews:

• I can't find anything to watch.
• I'm out of episodes of my favourite shows.
• I can't figure out how to search for a specific show.

The second two bullet points directly contribute to the problem in the first bullet point of not being able to find anything to watch, so it makes sense that they could sit directly under it in a parent-child relationship.

In this example, the parent opportunity of “I can’t find anything to watch” was explicitly mentioned in interviews, but it might not always be so clean; sometimes only the second two bullet points will be mentioned and you’ll have grouped them together because they seemed similar. In this case, it’s ok to add your own parent opportunity, as that need was implied by the users from the two opportunities they did raise.

While combing through your opportunities to find similar ones to group, you’re likely to find some that are actually the exact same opportunity, just worded differently. In this case, just combine them together by putting only one on the tree. We need every opportunity on the tree to be distinct from every other, such that we can work on any one opportunity without needing to work on any others at the same time, so group similar opportunities together under one parent, but combine exact opportunities together into one.

As you start to group similar opportunities together and place them under parent opportunities, you’ll begin to end up with a set of mini trees. Now start to look for sibling relationships between the parents of the mini trees, and if needed you can add a parent opportunity to cluster similar mini trees together. Keep iterating through this until you have a joined up tree that ladders up to the overarching desired outcome.

You now have a fully mapped opportunity space.

Prioritising the opportunity space

Once you’ve done the hard work of speaking to users to gather insights, then mapping them out on the OST, it’s time to start the equally hard work of prioritising the list of opportunities so you know which to go after first.

Thankfully, the tree-like structure of the OST really helps to make this easier.

You’ll start with the top-level, parent opportunities on the OST and compare them against each other to identify which is the highest value opportunity to go after first.

Identifying the highest level opportunity can be done in a million different ways, which is probably why this question comes up allll the time in Product Management interviews. For me, I like to prioritise against the product strategy by asking “which of these opportunities moves the needle the the most” on any KPI from the strategy.

However you decide to prioritise, the important part is that you’re making relative priority judgements by comparing sibling opportunities to determine which to explore first, and embracing the messiness of it. Interestingly, Teresa Torres suggests not to use any mathematical formulas when prioritising (ie, scoring each opportunity against set criteria, then working on the one that scores highest), because it leads us to a false-confidence. Remember, product discovery is an ill-structured problem; there is no one right solution for your desired outcome, so why would there be one right feature to work on at any given time? Our jobs here, like much of product management, is to review the facts, discuss the options in our product trio, make the best decision we can with the information we have, move forward even with some doubt, then course-correct if later find out this solution isn’t going to get us to where we want to be. Keep it subjective and leave room for the doubt, that is the mechanism that allows you to be confident enough to course-correct rather than being dangerously wedded to any one solution.

Also remember that at this point we’re not committing to which solution you’re going to build and launch, only which you’re going to explore further in discovery first. There will be plenty of opportunities to switch course as we gather more information about this solution during further discovery, and if we decide we picked the wrong solution, we simply jump to another on the OST, so this is a two-way door, an easily reversible decision. Don’t spend too much time trying to make the perfect decision. We’ll learn more, faster, by taking action.

Once you’ve identified the highest value opportunity, you can ignore all other branches on the tree and focus solely on the one you picked; if you’ve decided that the parent opportunity you picked is the highest priority, then the highest impact opportunity to address lives in that branch. This is one of the ways that the OST helps maintain focus and ensure you always know what to do next during discovery.

Now carry out the same compare and contrast exercise but with the children of the parent opportunity you prioritised, to identity which of the child opportunities would be highest value to address. If that child opportunity also has children, repeat this exercise again. Keep working down the tree until you identify an opportunity that has no children (this is called a ‘leaf node opportunity’), then start working on that.

Working on the lowest level opportunity in the branch ensures that we’re always delivering iterative value over time; if we start at the bottom of the branch and work our way up then we’re solving the smallest, quickest opportunities first, and therefore getting value out to the users sooner, but also solving the lower-level opportunities in the branch contribute to solving the larger, more intractable problems at the top.

Identify and validate the solutions

Ok, so you’ve spoken to users, identified a set of needs, pain-points and desires, grouped them onto parent-child and sibling relationships until you form an OST, and selected a leaf-node opportunity (an opportunity with no children) to work on.

It’s now time to switch from the opportunity space into the solution space; actually identifying solutions for these opportunities.

How do you come up with solutions? Well, essentially it’s creative thinking. At this point you might be despairing thinking “but Andrew, I’m not creative!”. Yes you are. Creativity, like most other things in life, is actually very achievable once you break it down. Here’s a few ways to get your creative juices flowing:

1. Spend some time thinking, then stop - don’t spend hours racking your brain trying to force ideas, just timebox an initial ideation session, then go and do something completely different. Through a mystical process called ‘incubation’ our brains actually carry on problem solving in the background, even when we’re no longer consciously thinking about the problem. This is why you can completely fail to find a solution to a problem at work, go home, come back in the next day, think of a solution straight away and wonder why you didn’t see this now obvious solution in the first place.
2. Do it when works for you - maybe you feel most creative in the morning, or maybe just after lunch works better, maybe you’re a night owl and come up with your best ideas when everyone else is asleep. Creativity is a very individual process, so engage in it in whatever way works for you.
3. Look for analogous solutions - you can look at how your competitors are solving this problem, or you can also look at how completely different industries solve similar problems. Here’s an example from the Continuous Discovery Habits book - if you work for a company that has built a job board product and you’re trying to identify solutions to better help your users evaluate jobs, you can look at how eCommerce stores help users evaluate products, or how travel aggregators help users choose hotels, or how insurance companies present different policies. A lot of creativity is actually just taking inspiration from what’s out there and building on it. Why do you think sites like Dribbble exist?
4. Generate ideas alone, then share them with others - research carried out around brainstorming alone versus in groups consistently showed brainstorming alone produced more ideas, and the productivity loss increased as the group size increased. Time to bin off your colleagues and become the lone wolf solopreneur you’ve always dreamed of? Not quite. Separate research also confirmed that alternating between individual ideation and sharing ideas in a group increases the number of ideas generated in the next solo brainstorming session; exposure to others’ ideas helps inspire more of your own ideas, so come up with ideas alone until you get stuck, then share them in a group, listen to others ideas then separate into another solo ideation session.

Keep pushing to find more solutions for your chosen opportunity until you’ve generated around 15-20 ideas. You want this many because your first ideas are rarely the best or most innovative; your solutions often get more creative and novel the more you think about the problem.

Then, work with your team to whittle it down to 3 ideas. You can do this by throwing them on a miro board and using the dot-voting feature. Give everyone 3 votes that they can use as they see fit; they can put all 3 votes on one solution, or one vote on three solutions. Remind everyone that when voting, you’re selecting the option that you believe best addresses the target opportunity you selected.

Why select 3? Why not just vote on the one best solution and run with that? We want to avoid over-committing to one idea, where we put all our eggs in one basket, enter into an over-confidence about it because we’ve all decided it’s the best idea, then falling victim to a confirmation bias where we’re all looking for evidence to support this idea while discounting any contrary information. This risk is particularly high in Product Management, because we often have to propose and defend our ideas to stakeholders. If you’ve brought one idea to the people above you, who have the power to promote or fire you, and you’ve very vocally sold it as the right solution, you’re going to be extremely invested in proving you were right. We don’t want that, we want to leave room for doubt so that we can continue to critically evaluate our idea as we go.

Identifying assumptions

We now have 3 ideas that we’ve prioritised. We can’t work on 3 ideas at a time though, so we still need to whittle them down to 1. This has to be done objectively, if we could do it subjectively we would have just picked 1 at the last step rather than 3. We could build a prototype for each then A/B test them against each other, but there’s a quicker way. Instead of testing a whole prototype, we can just test the assumptions that need to be true in order for them to succeed. If the underlying assumptions fail, the whole idea can be dispensed with.

In order to test the assumptions, we first need to identify them all. They typically fall into the following categories:

• Desirability - do our users actually want it? Will they get value from it? Will they trust us enough to use it? The latter is particularly relevant in FinTech products.
• Viability - is it financially viable for us as a business to build it? Will we get enough value from it as a business to make it worth our while?
• Feasibility - can we actually build it? Do we have the technical ability to? Does it meet compliance regulations? Is it ethical?

In addition to the assumptions you can think of off the top of your head, you can identify more through a practical exercise called story mapping, which is where you map out every step a user has to take to get value out of your product. I won’t go into detail about how to story map here because it could be a full article in itself, but if you’re unfamiliar with the process you can read Jeff Patton’s book ‘User Story Mapping’.

Throughout this process of user story mapping, you’ll be making a lot of assumptions about actions a user will take, and thinking through the journey in more detail to realise what you’d need to do to build it. These assumptions can all be added to your assumption list.

As an additional method of identifying assumptions, look at your OST and work up from your proposed solution to your desired outcome, asking questions at each step like “the proposed solution will address the target opportunity because…”, and “addressing the target opportunity will drive the desired outcome because…”. Answering these questions will contain assumptions you can capture. For example, if you work for a streaming platform and your desired outcome is to increase the amount of subscribers, and your proposed solution is to add sports channels, you could start from this solution and work up to the outcome by asking questions like “adding local channels will allow our subscribers to watch live sports” because “the sports that our subscribers want to watch are on local channels” because “most of the major sports are on local channels” and “our subscribers are more likely to want to watch major sports”. Each one of those statements contains an assumption that should be captured and tested.

Prioritising assumptions

Now that you have a list of assumptions you’ve made about your solution, it’s time to prioritise them to see which to test first. As always, there’s a technique for this. David J. Bland wrote a book called ‘Testing Business Ideas’ and introduced the idea of what he calls “leap of faith” assumptions; the assumptions that carry the most risk and therefore need to be tested first. To identify these, you’ll use an assumption map.

It looks like this:

As you can see, the X axis plots how much we already know about whether this assumption is true or false. If we have a lot of evidence that it’s true, then we place it on the left side, and if we have very little evidence that it’s true then we place it on the right side.

The Y axis plots how important the assumption is to the success of our idea. Can this particular assumption we worked around if it turns out to not be true, or does the whole idea fall apart? If the assumption is very important, it’s placed at the top of the map, if not then it’s placed at the bottom.

Assess the assumptions against each other; once you’ve placed the first assumption on the grid, you can then ask “Do we have stronger or weaker evidence that this assumption is true compared to the first assumption?” or “Is this assumption more or less important to the success of the idea than the first assumption?”. Don’t agonise over each placement, do this in your product trio quickly and move on.

Testing assumptions

Now you have all your assumptions mapped, we’re going to start testing the two or three from the upper right corner; highest importance and most unknown about whether the assumption is true, and therefore most in need of testing. These are the “leap of faith” assumptions.

The goal with assumption testing is to gather data that will help us move the assumption from the right to the left on the assumption map (from unknown about whether the assumption is true or not, to one where we know if its true or not.

As with user interviews, we want our assumption tests to focus on collecting data about what people actually do in a specific situation, not what they think they do in general, so a good assumption test simulates an experience. We want the assumption test to give us an opportunity to see if users behave in accordance with the assumption or not. In other words, if our assumption is true, what would we expect the user to do. We’ll set up an assumption test like this for each of the assumptions we identified earlier.

At this point you’re likely wondering how many users to test your assumption with in order to feel confident enough to move forward with it. Your first instinct is likely to try and test with as many as possible, but this comes at a cost; it’s slow. Remember, the goal is to quickly test assumptions for various ideas so that we don’t invest too much in an idea that won’t deliver your outcome. Start small and treat this experiment as an early-stage signal; an experiment that helps move the assumption from unknown to known just enough to prove it warrants further investment in larger and more elaborate experiments. If the assumption fails at this early stage, you’ve not wasted too much time or effort on it.

If an idea passes the assumption test, keep investing in larger tests until you’ve de-risked the idea enough to warrant building it.

How to actually run these tests

Time for some practical advice around how we actually go about running assumption tests. One incredible way is unmoderated user testing, which is a technique where you provide a stimulus such as a prototype, along with a set of tasks to complete and questions to answer, then allow users to work on this in their own time. I’ve used tools like UserBrain for this in the past, and while there there was often significant inconsistency in how thoroughly the users carried out the tasks, overall I found it a very quick, effective and useful way of gathering early feedback and testing underlying assumptions.

Another quick and easy method is to simply check the data you’ve already got. If you work at a streaming service and you’re trying to test the assumption “users will want to watch sports on our platform” you could check the data for terms users have typed in to the search field to assess how many were related to sports.

Remember, in your first round of assumption testing you are looking for quick, low effort tests that help you gather early signals, because the best teams are rapidly testing as many assumptions for as many ideas as possible.

Define your success criteria upfront

Before you run your assumption tests, make sure to consider how you’ll evaluate if the assumption was true or false. How many users need to interact with the prototype in the way you expected, 7 out of 10? How long should it take them to do so?

Discuss this with your product trio and agree on the criteria as a team, you don’t want one person to view the test as a pass and another to view it as a fail.

Effective instrumentation

In order to be able to gather sufficient data about whether your solution is viable and driving the desired outcome, you’re going to eventually test in production with a working version, even if its crude. As we touched on before, your experiments will start small and increase in size and complexity as it passes each stage of the assumption tests, and once it reaches production we now need to consider what analytics will be in place to be able to capture the data you need to evaluate the experiment. This is called product instrumentation.

Naturally, it can be testing to just think “let’s measure everything, the more data the better right?”. Well, as with much of Product Management, this is another trade off; if you spend so long considering everything you can measure and building in the required events and data capture functionality, you’ll lose a lot of time and possibly invest too heavily in an idea too soon. As with the experiment itself, start small with your instrumentation and add more as you go. Measure only what you need to validate the solution and increase from there.

Conclusion

If you've made it this far in this article, fair play, you've got the sort of relentless perserverance and curiosity that will set you up nicely for a whirlwind career in Product Management.

As you've learned by now, discovery isn't a one time event, it's a continuous process, and it's not distinct from delivery, it feeds delivery and delivery feeds discovery in return.

One of the most important traits to adopt if you want to be an effective Product Manager is to stay close to your users, and put them at the front and centre of every product decision you make. Let continuous discovery be the mechanism that enables that.