Not everyone is born with the ambition to become a start-up Founder.
But for those of you who see this as your greatest ambition, learning as much as there is to know about what it takes to get an idea off the ground is essential.
This should be the case considering that the start-up scene is highly competitive; research shows that 90% of start-ups fail and only 50% make it to the 5-year mark.
While there are many reasons for this - poor cash flow, weak management, or lack of marketing amongst others - the most detrimental factor for early-stage start-ups is creating a product no one needs or wants. This could be because:
- The solution you envisaged doesn’t solve a problem;
- The audience (for the problem) isn’t large enough;
- The problem isn’t a big enough deal to warrant paying for a solution.
To avoid these stumbling blocks, aspiring start-up Founders need to identify:
- Who your target audience is;
- The problems your audience face;
- The solution your audience needs and wants.
Since the product is the heart of every business, you must ensure that it can be used to solve a specific problem. Taking the time to develop a strategic roadmap to test the different variables of its operation, while looking at important performance metrics, will go a long way to ensuring that the product will remain useful and relevant for years to come.
Every product-focused Founder/CEO, Product Manager, or Marketer who wants to de-risk the possibility of spending a lot of time and money testing a solution should seriously consider using smoke tests to validate their new business idea or new feature.
What are smoke tests?
A smoke test, or smoke testing, is a low-risk way to test desirability for a product or service.
The term "smoke tests" stems from hardware testing. It was used to denote that a device was ready to use if it didn’t catch fire (creating smoke!) when it was turned on for the first time.
Why should I use smoke tests?
There are six great reasons to try smoke testing:
- Save time and money
- Reduce risk
- Gather customer insight
- Establish desirability
- Test your assumptions
- Validation your idea
Too many Founders make the mistake of using the product development process as a tool for validation, which costs them dearly in both time and money. Some Founders don’t do any validation at all and don’t find out until they’ve spent thousands building a prototype, or even worse: launched their Minimum Viable Product (MVP) that there’s no demand for the product or the features they’ve built.
Smoke tests allow you to test early signs of desirability for your product or service before you’ve even designed your product or service, saving you time, money, and potentially a huge amount of disappointment.
As part of the preliminary testing process of a product or a service, smoke tests will test the accuracy of your initial assumptions about your target audience and whether they are interested in your proposed solution.
A successful smoke test will help to validate your business idea, giving a definitive answer to one of the most dreaded questions of all time: ‘Will people buy my product?’
They also gather essential data on the interest of the audience towards your product, allowing you to gain a deeper insight into buyer intent and demand.
Needless to say, establishing the needs of the market and ensuring that your product will be well-received is essential. Doing so using smoke tests will help make predictions on the performance of the product and, therefore, on the revenue you can generate as a start-up company.
How do smoke tests work?
To put it briefly, smoke tests give the impression that customers can buy (or use) your product or service before it’s ready to buy, use, or even before it’s been designed.
There are many ways to run a smoke test (scroll down for our favourite ELEVEN) but one example is a landing page for your product. The landing page would provide important information about your product, with CTAs (calls-to-action) throughout to encourage visitors to sign up for early access, pre-order, or register their interest. Because you don’t need the product to do this, it’s low-risk but the value of the research is high.
Meanwhile, you can track these actions and measure them against success metrics such as, “If 100 people click the ‘Buy Now’ CTA within two weeks, we have enough validation to start production.”
How do I choose which smoke test to use?
(check the full document and download it for free today)
Taking into account your success metrics, and the time and budget you have at your disposal, you’ll want to use the smoke testing technique(s) that will bring you the most effective and relevant results.
There are a number of smoke tests that can be used in a variety of ways. To get the most out of them, you need to choose which one both works best for you and matches your specific criteria.
There are a lot of actors to consider before making your decision, including:
- Type of product or service
- Team (and skills) you have at your disposal
- Time constraints
-
Budget
Review the following smoke tests and use the PDF to compare which will be most appropriate for your product.
And remember, look at this as an experiment - you might not find the right method right away, you might need to try a few until you find which works best for you, or even run some in parallel.
The 11 smoke testing techniques you need to know
We’ve shortlisted our favourite, most effective, smoke testing techniques with examples (where we could find them) below.
#11 Pre-order form (mock sale)
Business model: e-commerce, digital goods, B2B, services
Set-up time needed: 2-4 days
Cost: <£250
Validation strength: 3x
For the pre-order form (or the ‘mock sale’) you’ll create a mock-up of the product(s) you intend to sell on a website. From there, the visitor chooses the item(s) they want to buy and provides their details using some kind of form. If the user completes and submits this form, then you have a ‘positive sign of desirability’. If you are testing pricing options, you can require them to select a pricing tier. If you are selling a physical product, ask for specifications and the quantity. Once everything has been selected and submitted, set up a notification (pop-up boxes or landing pages work well) explaining that you will notify the visitor when the product is ready. Make sure you explain that you won’t take payment until then - or you may have an inbox full of complaints!
#10 Coming soon page
Business model: all
Set-up time needed: 1 day
Cost: <£50
Validation strength: 1x
This is a one-page website - a landing page - that presents your product and its value proposition along with the benefits it comes with, to convince your target audience to buy it. The ‘coming soon’ page CTA encourages your visitors to complete an email capture form, enabling you to collect the email addresses of potential users. This smoke testing technique allows you to measure the conversion rate on email sign-ups and traffic sources, as well as bounce rate and session time.
Example
As one of the simplest, quickest, and inexpensive solutions to validate a business idea, the ‘coming soon’ page is really popular in the start-up community; chances are, you’ve come across them yourself. Robinhood, Pitch and Center are just recent examples of companies that have created high converting ‘coming soon’ landing pages. They all used the core elements that every successful landing page needs:
-
- Clear value proposition
- Hero image
- Eye-catching and simple CTA
- Social proof
- List of product benefits
Center even included a countdown to the product release date. Being creative with the colours, copy, and images you choose - as well as using strong directional cues through your CTAs - will help you achieve a powerful result.
#9 Explainer Video
Business model: All
Set-up time needed: 4-5 days
Cost: £500-1000+
Validation strength: 3x
The Explainer Video is a video that allows you to explain the full proposition of your product in an easy-to-understand video format. You can measure the number of email sign-ups, impressions (watches), and shares. This video aims to give your site visitors a detailed summary of what your product is and what it does. To help them get a sound idea of what you plan to offer, you can use recorded prototypes and mock-ups focusing on your product’s key features. For greater insight, you can create several explainer videos for different landing pages targeting unique audience segments to test which one works best.
Example
Drew Houston, Dropbox’s CEO, initially used an explainer video to encourage his target audience to view his product as a viable solution to their file-sharing and synchronisation problems. Focusing on customer experience, Drew came up with a 4-minute video demonstration that showed how people can use Dropbox to share and store files online. But he didn’t stop there. Instead, he tailored the content of the video in a way to resonate with the community of people he wanted to approach first, and by using in-jokes he spoke to them authentically. The video brought 70,000 sign-ups overnight.
#8 Invite Only/Waiting List
Business model: SaaS, Digital Product, Hardware
Set-up time needed: 3-4 days
Cost: <£500
Validation strength: 4x
The Invite Only/Waiting List is a technique that works similarly to the ‘Coming Soon’ page. The only difference to it is that it has a referral system that helps to measure the virality of the product. The way it works is easy. Once a person signs up to a waiting list, s/he gets a number that determines their position in the queue. To move up on that queue and get priority access to the product, s/he needs to refer a specific number of people to the site. This virality-testing method helps to calculate average referrals per person and bounce rate.
Example
Robinhood, which is a financial services company based in the US, used this smoke testing technique to come up with the Robinhood.io waiting list campaign. Using KickoffLabs, they came up with a sign-up page that focused on using a strong headline and copy along with a clean image with no distractions. This allowed them to focus solely on one thing - getting the visitor’s email address - using a user-focused CTA. A secondary CTA followed, on a carefully constructed ‘thank-you’ page, encouraging people to share the link on their social networks to get priority access. Meanwhile, they came up with a series of emails sent out at different times to thank users for signing up and to keep them informed of their referral progress. Inside each email, there was a link back to the ‘thank you’ page that allowed them to go back at any point to check their spot on the waiting list. The ‘thank you’ page ended up converting as well as the initial page, driving over 50% of sign-ups through social referrals.
#7 Broken Promise Test
Business model: SaaS, Digital Product, Hardware
Set-up time needed: 2-3 days
Cost: <£500
Validation strength: 3x
If you want to check the virality of your product and create some exclusivity and scarcity at the same time, go for the Broken Promise test. This technique allows you to see how many times users share your product with others (their network) while measuring referral rates. You can do this by creating a landing page with a purposeful CTA and inviting only an exclusive group of users to check it out. Upon communication with these users, who have registered for your product, you tell them they can’t share the idea with anyone else. After the trial period passes, go through the original list of users you directed to the page, and compare it with the list of total sign-ups. Any user who wasn’t in your original list counts as a referral.
#6 Pre-sale
Business model: E-Commerce, Digital Product, SaaS, Services
Set-up time needed: 1-2 weeks
Cost: £500-1000
Validation strength: 5x
With the pre-sale smoke testing technique, you set your product on offer at a discounted price (early-bird pricing) for a limited amount of time. Before you’re ready to launch, users pay in advance to either get an early version of the product. Most Founders use the former method and they offer the product at early stage for a reduced price, and for a limited amount of time. This technique allows you to check the interest of your target audience before you fully develop your product. While the up-front payments you get are a way of validating and funding the development of your product.
Example
Kettle & Fire’s Founder, Justin Mares, used the pre-sale technique to test his business idea. He wanted to learn whether people had the same need as him to buy bone broth and how much they would pay for it. What he did was craft a very basic landing page that briefly explained the benefits of using the product, accompanied by the ‘Order Now’ CTA that led users to buy the product. After using Bing ads, what he got was a 30% conversion on a product he had yet to manufacture, by offering a 50% discount. In just 2 weeks and only spending around $100, he received $500 in revenue and got his business up and running.
#5 Crowdfunding
Business model: Physical Product
Set-up time needed: 2-3 months
Cost: £5000+
Validation strength: 5x
Crowdfunding is a similar strategy to the pre-sale method, that you can use to raise money to manufacture a physical product. It allows you to set a pledge amount that you need in order to fund and start developing your product. Once you meet the pledge amount, the money is released and you can begin the manufacturing and distribution process. Compared to all the other options on this list, crowdfunding has the strongest validation strength, which is great, but it is typically the most expensive option to execute, due to the quality of the promotional materials and marketing required.
Example
Many start-ups have come to the surface through crowdfunding. Oculus - the VR headset company - pledged $250,000 on Kickstarter in 2012 in order to launch. Within only 4 hours, they exceeded their goal and raised almost $2.4 million. Oculus was then sold to Facebook for $2 billion in 2014.
#4 Pocket Prototype
Business model: App & Physical Product
Set-up time needed: 1-2 weeks
Cost: <£250
Validation strength: 3x
The Pocket Prototype smoke test can be used to perform a usability test in order to validate a business idea for a physical product, or a software product that sits within a physical (hardware) product e.g. an app for a mobile phone. This technique involves interviewing a potential customer and then showing him a physical prototype of the product you are planning to launch - which translates to pulling the prototype right out of your pocket - hence its name. By doing so, you get useful product feedback and can encourage further exploration and discussion with the prospect.
Example
On the quest to offer a better user experience, Pocket created multiple prototypes to test out their ideas. So they came up with new getting-started processes for users, using the iOS and Android Pocket apps to check user experience. With the help of Keynote, they built clickable prototypes of their proposed ideas and called in 5 participants to try them out. While users were exploring the prototypes, the Pocket team was observing their reactions and taking notes on what worked out or not, to learn what they need to do differently next time.
#3 Fake Door
Business model: E-Commerce, SaaS, Physical Product
Set-up time needed: 2-4 days
Cost: <£500
Validation strength: 3x
The Fake Door method requires creating a fake ad for a product. The ad features a mock-up of the product that you display on your website and is followed by a ‘Buy Now’ CTA, that guides users to your Coming Soon page. On the page, you ask your site visitors for their email addresses so that once you launch your product, you send them a follow-up email informing them about it. By using Fake Door, it is possible to find out how many users will click on that CTA, thus showing interest in your product. This technique is quick, simple, and cheap to use. It can offer great insight into buyer intent by allowing you to measure the number of clicks you get on your CTA. What makes it stand out, though, is that you can easily validate your business idea without the need to use any logistics or money-exchanging processes.
Example
Many entrepreneurs and product analysts are in favour of using Fake Door as a product validation tool. Amongst them is the information aggregator Tippiq.nl, which uses this technique often. To test fresh ideas, they create fake ads that lead to a unique website where they explain the product and ask users if they would like to use it. Once users click on the CTA to use it, a notification appears informing them that the product is under development, but they can leave their email address to get notified when it is ready. Thanking users and apologising to them because the product is not out yet, or even compensating them for helping you learn their needs, is common courtesy when using Fake Door. So is avoiding the use of 404 error pages, which can make people feel angry or like they’ve been tricked.
#2 Pricing Page
Business model: All
Set-up time needed: 1-2 days
Cost: <£250
Validation strength: 2x
Designing a pricing page and testing out users’ reactions is a very powerful smoke testing technique. Doing so allows you to check whether people are prepared to pay the suggested price to buy your product and to find out which pricing works best for them. You can do this by tracking the clicks on the CTAs associated with each pricing plan you present on your pricing page. Once they click on these CTAs, users visit a ‘Coming Soon’ page that asks them to leave their email address for a future follow-up informing them that the product is out. The pricing page works effectively, and it is quick and cheap to create. With it, you can measure email sign-ups, bounce rates, session time, and conversions all based on the traffic source.
Example
Buffer’s CEO and Co-Founder, Joel Gascoigne, has successfully used this method to test whether a) people wanted the product and b) whether they would pay the proposed amount of money to use it. Joel first got his website link on Twitter and gathered some feedback on the product. Then he created a pricing page that had three pricing tiers. As he saw many people clicking on the paid plans and following the guided CTAs before giving their email addresses, he found out that the pricing was successful and that people were more than willing to invest in his product.
#1 High Bar
Business model: All
Set-up time needed: 1 day
Cost: <£250
Validation strength: 2x
Our top smoke testing technique is the High Bar technique. This smoke test is another variation of the Coming Soon page, with the difference that it comes with an additional step in the user registration process. At the point of conversion where users agree to give their email address, you assign them a minor task. This task could be something like completing a form and answering a small set of questions. In the form, you may ask questions that allow you to find out who these users are to acquire demographic data, what are their expectations or difficulties or more general questions related to the use of the product. This way you gain important insight into your target audience and the types of people who are interested in your product. Not only that, but it allows you to foresee any problems or frustrations users have, which you can find the solution for. The High Bar is a powerful method to use as it is used to determine how much users want to gain access to your product and how motivated they are to get it. Agreeing to carry out a task for you means that users are genuinely interested in what you offer and aren’t just casually signing up for it.
Example
Hey - a premium email service launched by Basecamp, tried out the High Bar technique and it worked out well for them. Instead of using a feedback form, they asked people to email ‘iwant@hey.com’ a brief message that described their relationship with email. Completing this task was the only way users could get on Hey’s waiting list. Many went with it anyway and Hey got over 150,000 people on their waiting list.
Useful resources to run smoke tests
There are several tools you can use to help you run smoke tests.
Typeform is a great option for surveys, for example. If you're sending email campaigns, MailChimp is an easy alternative for segmentation and behavioural targeting. For quick and simple 1-page websites, you can use Carrd - their landing pages are effective and you can capture emails too. Don't forget to set up your Google Analytics account, of course, to keep track of your website's traffic and get valuable insights about your audience.
The book “Testing Business Ideas: A Field Guide for Rapid Experimentation” by David J. Bland, offers great guidance and techniques to help you understand and test your business and product idea.
