Why The Test & Learn Approach Does Not Work

The in-market “test & learn” approach to innovation is becoming increasingly common, because it meets the cultural needs of modern businesses to behave entrepreneurially. At The Strategy Distillery we believe (and have witnessed) this approach is life threatening to the future success of a newly launched product or service.

In this article we explain why…

The big watch out 

Without a doubt, thinking like a start-up has its benefits but the comments below raise some alarm bells because a quick analysis of the realistic outcomes from rushing a product or service to market tells us little good can come of it.

Speech bubbles containing the quotes "get it straight to market", "test in market", "test & learn in one small market", "iterate in market", "optimise in market"

Why doesn’t it work? 

The realistic outcomes from a test & learn in-market approach are:

Outcome A – A successful launch

Your launch is quite successful. The market responds well. So you relax and don’t make further iterations: why would you assign further budget to your successful project?

Problem #1: You don’t ever get around to making those improvements that could see your innovation perform not just well, but brilliantly, and you lose out on that additional revenue.

Problem #2: Your competitor seizes on the available improvements and optimisations to make a better me-too product that overtakes the success of yours.

Outcome B – An unsuccessful launch

Your launch achieves poor sales. Something is definitely not working.

Problem #1: Confidence plummets within your organisation and there’s pressure to pull the project. At worse it gets pulled; at best you have to use up valuable time and resource to get people back on-board internally.

Problem #2: You haven’t got any insight to look for clues as to where it might have gone wrong. You’re starting from scratch and the clock is ticking to put it right. This pressure leads to dangerous assumptions and quick fixes that may only compound the problem.

Outcome C – A mediocre launch

Your launch is ok but could do better. So you decide to now improve your product or service in market.

Problem #1: You are going have to work pretty hard to keep people on-board with the idea internally.

Problem #2: You still need to gather the insight to know where and how to fix the problem – so you can’t avoid spending this budget. The problem is that it’s now more expensive, more complicated and more time consuming to create and roll out the changes than if you had worked it through pre-launch.

Problem #3: Unnecessary fuss and confusion created by iterative improvements means you quickly lose credibility with sales teams, trade partners and end-users or consumers

Summary 

Whatever the outcome, you’re highly likely to lose credibility quickly with trade partners by taking this approach.

It’s expensive, time-consuming and resource draining to take a project to market (even a small one) and then improve it iteratively. Not to mention confusing and irritating for stakeholders, customers and consumers or end-users.

There are scenarios where this approach may be valid – perhaps customers aren’t bothered by regular software updates for example. If your product or service means you can push the improvements to them via technology, there maybe a strong case for going straight to market.

But the danger and frustration is that the test & learn in-market approach is being touted as best practice for innovation in general. It’s simply not ideal for all industry sectors. Even in the tech sector we suggest that you can only get away with this approach when you have already built a brand that people love, and if they are deeply engaged with the products.

Published 23rd May 2016 by Shelly Greenway @ the Strategy Distillery