Validating Your Startup Idea

How to make great decisions about your product idea

My last post covered the challenges of Zero-to-One product development and some high-level ways to deal with them. Today, we’re going to look a little closer at a concept known as validation and how to validate your startup idea.

For any product, there is typically one or several hypotheses that look something like this: 

[PRODUCT] will [DESCRIPTION OF EXPECTED OUTCOME] to [USER TYPE] because [USER MOTIVATION]. 

As an example, during “early” Uber, the hypothesis might have read something like…

Ridesharing will make urban transportation easier and more enjoyable for young professionals because urban travel is frustrating and takes time and energy away from productive work. 

These theories are intended to address any product’s risk (or feature). If you haven’t read my first piece linked above, it may be helpful to, before proceeding. If you have, you may recall that we identified three primary risks that any new product (or feature) faces at any stage of its life cycle:

1. Value Risk: Does anyone want this? 
2. Usability Risk: Can anyone use this? 
3. Resource Risk (AKA Viability Risk): Can you find traction before running out of cash?

We then engage in efforts to validate or invalidate these hypotheses and as a result, de-risk the product idea in question.

If a particular hypothesis is true, then you know you’re on the right track with your product idea in regards to this specific value proposition. If false, you can assume that the risk the hypothesis was testing for cannot be addressed in the way the hypothesis describes and move on to validating a new hypothesis.

Validation, then, is the set of activities and tools we use throughout this phase to get closer and closer to validating or invalidating our product hypotheses. 

Probability vs. Certainty

Is Your Opportunity Real?

This is the golden question. But let me start by telling you that if you’re looking for 100% certainty that the opportunity you’re solving for is real, you’ll probably be waiting forever.

The mindset of needing certainty is one of the most dangerous mindsets for early-stage products. Let me tell you why.

First, this mindset leads to what we like to call, ‘analysis paralysis’. Because that’s what it is —you keep looking at your product and thinking more research is needed, more ideas need to be tested out, before you can decide on the next step.

Second, we fall victim to ‘self (and team) deception’, in which we project a facade of certainty and make decisions based not on probability, but on our “gut” and the input from the people in our immediate vicinity, most often our team, friends, and family.

Now that we’ve established the mindset with which not to approach these activities, let’s move on to the correct goal to aim for. That is, a probabilistic sense that the opportunity you’re solving for is real and worth investing the time and money into building a solution for.

This approach refocuses our efforts on validating our product ideas by adding enough data and detail to help us make decisions we believe have a high probability of success. Since, in reality, most of our decisions, particularly those related to products, are bets.

Everyone who’s ever played poker knows that even under the best circumstances, bets can go south. However, the best players are aware of this and seek to improve their odds of success by employing techniques and understanding how to read and interpret the action taking place at any given table. This way, they can translate that into decision-making data for themselves. 

For early-stage products, validation works in a similar way. We have a series of techniques that we can practice and refine that helps us add data to our decision-making process and that will help us make sound decisions with a high probability of success.

In the next section, we’ll cover some of the qualitative and quantitative validation techniques you can employ to help make better probabilistic decisions about your product and increase your chances of success.

Validation Techniques

The following techniques are designed to gather two types of data: Qualitative and Quantitative.

• Qualitative data deals with data that isn’t quantifiable as a number. Most often, this will come from interviews or prototype demos.
• Quantitative data, on the other hand, is usually attached to a numerical metric. And can come from a variety of sources including market research, demand testing, and keyword research.

Qualitative Data 

As mentioned above, qualitative data refers to the unquantifiable. Behaviors, motivations, pain points, desires, and unmet needs are all qualitative data points that help us better understand the lived experience of our users.

Discovery Interviews

Discovery interviews are the bread and butter of modern product discovery. 

Short, carefully-crafted interviews with real or prospective users can uncover a treasure trove of opportunities for your product, in addition to validating that the opportunity is indeed something people care about.

Discovery interviews involve excavating stories from interviewees about times when they have encountered the problem, need, or desire our idea is designed to address. We want to know about their motivations to address these issues and what they’re already doing or trying to do to address these issues. 

This data helps us validate whether our hypothesis is or isn’t true, and often uncovers new opportunities we hadn’t even considered! 

Assumption Testing

Assumption testing is a fantastic way to test your risky assumptions directly. This will help you better understand if they present a serious enough risk for your product or if they are more perceived than real.

Assumption tests come in all shapes and sizes. From prototype demos to short surveys, the goal here is to simulate an experience that makes the risky assumption obvious, in addition to collecting data that’ll help you validate or invalidate your risky assumptions. These can be observations or recordings of how your users respond.

Ask yourself: What must be true or untrue in order for my proposed solution to deliver the value it promises?

The answer(s) to this question are your risky assumptions.

Once you’ve outlined your assumptions, you can design simple tests around them. Keep in mind that these tests need to replicate a decision point where the user must make a decision that either validates or invalidates the assumption.

If you get a negative result, that’s a pretty strong signal that your idea is missing some consideration that could act as a poison pill. Spend some time thinking through the assumption and how you can overcome it before proceeding.

Prototyping

Prototyping is a common method of testing. The goal is to test a fake version of your product with users to see if they understand how to navigate it and if they can grasp its value proposition. 

This method allows you to collect excellent qualitative feedback from users from as close to a real version of your product as possible, without building it. It also helps you uncover new opportunities to evolve your product and saves tons of time and money in the process. 

A prototype can mean many things. It might be hand-drawn artwork, interactive slides, or even high-fidelity drawings created with a program like Figma or Sketch. Not sure which format is best for you? Easy decision! The best way is whichever you can put together the fastest. Remember, you can always add increasing levels of detail and fidelity over time. If all you have to get started with is a hand-drawn sketch, use that.

Quantitative Data

This type of data can be obtained from a variety of sources, including market research, keyword research, and demand testing.

Market and Opportunity Sizing

Market sizing is the process of quantifying the size of the market and opportunity your product presents in hard numbers, most often currency. 

You should examine it from three perspectives: Total Addressable Market (TAM), Serviceable Addressable Market (SAM), and Serviceable Obtainable Market (SOM).

• Total Addressable Market (TAM): What is the sales volume of the total market over a given time period? (Usually one calendar year). 
• Serviceable Addressable Market (SAM): Over the SAME time period, what is the sales volume of the specific segment of the entire market that your product addresses?
• Serviceable Obtainable Market (SOM): What is your best estimate as to how much SAM your product may reasonably obtain within the SAME time period?

Taken together, TAM/SAM/SOM paints a picture of the total market size along with the opportunity size for your product.

Keyword Research

Keyword research lets you quantify the search volume for certain words and phrases. This data may be used to estimate the amount of interest and/or demand for your product’s value proposition.

With keyword research, the most important thing is knowing how to choose the right keywords to analyze. Here are a couple of techniques for doing so: 

Google offers a free tool for this called Keyword Planner as a part of their Ads product. There are other, more technical (and expensive) tools available like Ahrefs and Similar Web that can give you more robust data as well as help you analyze competitors’ websites and ad campaigns. 

Demand Testing

Demand testing is the process of running experiments to test for market demand against your idea’s value proposition. The most common form involves running an ad campaign that drives traffic to a landing page with a strong call to action (CTA). 

Well-designed demand tests provide you with several very useful data points:

1. Click-Through Rate (CTR): What percentage of people who view the ad are clicking on it? You can gauge success here by researching CTR benchmarks for your industry and trying to iterate your ad copy and creativity to meet these benchmarks. From there, continue to iterate and see how much you can improve the CTR. 
2. Conversion Rate (CVR): Once you’ve driven people from your ad to your landing page, how many of them actually take the action you’re prompting them to take on the landing page? A very common CTA is an email submission form in exchange for being added to an early notice list or for an asset like a white paper or other useful documents. 

Demand tests are instrumental in evolving your value proposition to better meet the needs of your target users. 

Making Good Decisions with Data

We’ve gathered the data! Now what? This section explains how you can evaluate that data tying it back to whether you should pursue your idea.

Opportunity Size: How big is the market and how big might it become? 

Building and scaling products is hard, expensive work, so we better be sure that the opportunity is big enough to justify the blood, sweat, and tears that will go into it. 

If the SOM is big enough and the TAM is expected to grow at a healthy pace, then it’s a decent bet that the opportunity presented by your idea is worth considering further. 

Validated Demand: Does it exist and is it enough? 

We can combine our qualitative and quantitative data to make a decision based on validated demand. 

Risky Assumptions: Can you satisfy these or will they kill your product? 

Understanding the market size and opportunity for your idea and validating that demand exists still leaves one very important (and often ignored) consideration untouched: Risky Assumptions. 

Remember, risky assumptions are things that must be true or untrue in order for your idea to deliver value to customers. 

Ideally, you’ve identified what these risky assumptions are. You’ve also run simple tests designed explicitly to conclude that, 1) the assumptions are safe, or 2) they will kill your idea’s potential to acquire customers.

Sometimes a big enough market and validated demand cause us to miss the fact that we’re asking our potential customers to change behavior in a way that they’re simply unwilling to do. 

In essence, if you’ve tested these risky assumptions and you get positive feedback, great! If you get negative feedback, time to either go back to the drawing board and figure out a way around the risky assumption or abandon the idea altogether. 

Conclusion

Building new products is expensive, risky, and really hard work. We’ve all heard the statistics that startup success rates are in the low single digits. Sadly, many of the failures implied by that statistic either didn’t have to be failures or could have been avoided early on with proper validation.

These, then, should be your goals with your validation efforts: 

• Abandon a bad idea early, or…
• Evolve your idea to a place where the probability of success is much higher.

By engaging in sound, data-driven validation practices, you can get the early signals you need to make sound decisions about your idea and the product it could become.