By now, most entrepreneurs realize (or at least know) that before spending a lot of resources building a new product or attracting partners and investors, you need to "validate" your idea to make sure you're not wasting your time and energy on something no one needs. Some archaically refer to it as a "proof-of-concept", but the sentiment is usually the same.
What is different, however, is what people think validation looks like. I've heard a lot of different opinions and even heated arguments on this subject over the years (even participated in some), until it suddenly hit me that in a way, everyone was right! There is no single method of validation, but a continuous scale or different levels of validation. That being said, the higher the level of validation, the better. Here I will list all the validation methods/principles that I can think of, ordered by their "level of validity".
Level 0: "Secret genius"
"I have a brilliant idea that no one in the world thought of and it can make me rich fast. My belief in this idea is the only 'validation' I'll ever need" OR "I have a great idea, but I don't want to tell anyone about it - it's so obviously good that they will steal it" OR "I'd love to talk about my brilliant idea, but I'm too embarrassed, because it's so undeveloped and/or I don't know how to present it, so first I'll build the entire perfect product, because I already know exactly what it should be, and let it speak for itself"...
As strangely as it sounds, these kind of thinking are not 100% wrong, but it's so close to 100% that it might as well be. It didn't use to be (or so I've heard) - in the good old days you could create cool new technology in your garage without telling anyone and then go out and sell it for millions. But nowadays it rarely works like this and people who believe that it does are almost invariably newbies and/or "wantepreneurs" who have no idea how modern entrepreneurship works and, often, no desire to learn. By the way, if you meet an "entrepreneur" who wants you to sign an NDA before saying even a word about their idea, then you can safely bet there has been zero validation done to that idea.
Level 1: Expert opinion
"I talked to experts in this field and they [mostly] say that my idea is good and the product will be a success" OR "I am an expert myself and it is my expert opinion that my product will be a success".
That's better than nothing, of course, but there are actual studies that say experts are no better predicting the future than a group of ordinary people from the street, not to mention other forms of validation. As von Braun reportedly put it: "One good test is worth a thousand expert opinions." But, at the very least, an expert can reliably say if the idea is technically feasible in the first place. They could be wrong even on that, though. One thing experts are really good at, however, is recalling past experience, if you prompt them to. So, instead of asking an expert to predict the future of your idea, ask them (or yourself, if you're the expert) if they've ever encountered or heard of something similar and what ever came out of it. You may learn more than you think.
Level 2: Customer interview, polling, focus groups
"I talked to X potential customers and Y of them said they like the idea and/or will use the product once I build it".
Many people (including some seasonal entrepreneurs, to my surprise) believe that this is an excellent form of validation. Unfortunately, practice (including my own experience) shows that it's not, for several reasons, including:
- Many people are too nice to tell you that your idea stinks (especially if they're your friends or relatives, which is often the case). So they just tell you what you want to hear in hope that this will be the last time they hear of it (and they're often right). Some do that intuitively, without even realizing it, by the way.
- You didn't describe your idea properly (which is nearly always the case) and interviewees had an entirely different picture in their heads. Once they see the complete product they will be totally disappointed.
- You asked the wrong people, not your real potential customers (this is especially true for polls, where you have less control over the audience). While they liked your idea in the abstract, they have no use for your product, so they can't judge it properly.
- They actually told you your idea sucks, but your brain ignored it and cherry-picked only the words that you wanted to hear. Happens all the time, don't feel bad.
Fortunately, there are ways of increasing the reliability of potential customer interaction. The most important one is to forego polls and focus groups, and concentrate on one-on-one in-person interviews as much as possible. You need to judge the person's reactions, body language, tone of voice, and more - not something you can achieve via polls. But even interviews have to be conducted properly so as not to fall prey to all the mistakes. Here are a few tips for improving the reliability of interviews:
- Don't ask yes/no questions - those lead to the answer that you want to hear, not the correct one
- A picture is worth a thousand words. Don't just say or write words - convey your idea in a relevant visual format. For example, if you want to make a website, at least draw demo screenshots in PowerPoint or something
- Keep doing interviews until you start hearing the same things over and over - only then you know you have a good picture of what's going on. That usually requires at least 20 interviews, if you've defined your target audience correctly and not too broadly
- Document everything, write down responses or, better yet, videotape/audiotape them, if possible, and then give the records to someone else for an objective review.
An excellent guidance on conducting customer interviews can be found in "Lean Customer Development" by Cindy Alvarez, which I highly recommend if you want to validate your product on this level.
Level 3: Non-binding action
No matter how good you are at extracting information from potential customers, talk is cheap. What really matters is action. So, if potential customers say they like your idea, ask them to prove it by doing something that demonstrates their potential (non-binding) commitment. Here are a few possible actions from potential customer that do just that even if you don't have any product yet:
- Register as a user for your future product (name, phone, email, etc.). You can whip up a registration form in minutes using various free services on the web or you can register users manually with a paper and a pen. The fact that they are willing to share personal information means a whole lot more than a promise to do so in some vague future
- Write a letter of intent. This is suitable for large business customers. A letter of intent, in this context, is a written commitment to use your product once it's ready. It's not legally binding (if carefully written), but it does demonstrate a serious interest, and even your potential investors will take it as a good sign.
Level 4: Binding action
Similar to the previous level, except that the actions that demonstrate commitment are all but irreversible, so you know these potential customers are truly committed. For example:
- Paying for a "minimal viable product" (MVP) - the simplest possible product to make that provides some kind of value to potential customers, a value which is part of the value of the intended product. Many feel awkward asking to pay for something that they whipped-up in a quick-and-dirty fashion over a weekend, but if you really want to validate your idea, you have to overcome those feelings. As Reid Hoffman, the founder of LinkedIn once said: "If you are not embarrassed by the first version of your product, you've launched too late."
- Pre-paying for a future product. If potential customers are willing to pay for product that doesn't exist yet, based only on your promise to make one, then you know they want it badly. And yes, it happens. That's basically what crowdfunding is all about (or at least it should be).
- Signing a binding contract. It's like a letter of intent, but the level of commitment is greater.
Level 5: Action + referral
Personal commitment is a great validation, but if that person starts spreading the word about your idea in an attempt to get others to commit, then that's a whole new level. Most people understand the responsibility in recommending products - they'd rather pay their own money than facing a fallout (even imaginary one) from recommending something crappy to their friends. So, they probably won't do it unless they truly believe in your idea and are very interested in its success.
Level 6: Sustained action
One-time actions, including referrals, are great, but a regularly repeated action is even better. Customers can be hyped and excited about a future product for a short period of time, even recommend it, but then lose interest in it. If that happens too often, then even if you build this product, your business model may turn out to be non-sustainable, which is only slightly better than building a useless product. So, as you're building your product, you should provide regular updates to prospective clients who signed up previously, and ask for some additional action (non-binding, no need to push it too far) every time to measure the sustainability of their interest.
Those are all the validation methods I can think of. Do you know any other?
Anyway, the main point of contention between different factions seems to be the minimal level, which could be considered a sufficient validation. Since the spectrum is continuous, any number on it would be arbitrary and having merit at least to some degree.
Personally, based on my experience and goals, I want to see level 3 and up (preferably level 4 and up) before I'm convinced that the idea has some real merit. What is your preferred threshold?