Business experimentation consulting settles a strategic question with evidence from the market instead of argument in a room. We help a team name the belief that would sink an idea if it turned out to be wrong, design the smallest test that could disprove it, and agree before the test runs what result would end the argument. Vivaldi has built innovation pipelines and validated new propositions for companies including O2, Allianz, Wrigley, McGraw-Hill, Takeda and Anheuser-Busch.
- Since 1999demand, innovation and business model strategy
- 200+published case studies
- 10 officeson four continents
- 5xWirtschaftsWoche Best of Consulting, Strategy and Innovation
- The Interaction Fieldwritten by our founder
When companies bring us in
Business experimentation is usually bought by a chief strategy officer, head of innovation or growth lead who has more ideas than budget and no reliable way to separate them. The trigger is almost always one of these.
- A plan that rests on one unexamined belief. The business case works, and it works because of an assumption nobody has tested in front of a buyer.
- Research that confirmed what was already decided. A study came back positive, the launch happened anyway, and the result in market looked nothing like the deck.
- Tests that never close. The company runs plenty of activity labeled as testing, and no project has ever been stopped because of one.
- A choice between concepts that look equally good on paper. Four propositions, one budget, and the comparison is being made on conviction.
- A new model whose economics are unproven. The demand question looks answered and the question of whether anyone will pay the price the model needs does not.
Why most corporate experimentation changes nothing
Almost every large company runs tests. Few run experiments, and the difference is whether the result could have gone the other way.
The common failure is that the test is designed after the decision has effectively been made. A concept is developed, the business case is written, and research is commissioned to support it. When the result comes back ambiguous, and it usually does, the project proceeds. Nothing was at risk, so nothing was learned.
Three more specific patterns sit underneath that one.
- The wrong assumption gets tested. Teams test what is easiest to measure rather than what would end the venture if it were false. Appeal is easy to measure. Willingness to pay is harder, so appeal gets tested and the harder question travels all the way to launch.
- The test is too big to be disproven. A market pilot costing a quarter and a headcount is a soft launch wearing a lab coat. It arrives too late and costs too much for anyone to call it a failure.
- Order is ignored. Five tests run in parallel on one concept produce a muddle. Sequence matters more than volume, because the answer to the first question usually makes three of the others unnecessary.
The discipline is narrow. Identify the belief most likely to be wrong, design the smallest test that could disprove it, run that one first, and write down what the answer changed. Most of the value comes from the work a program stops early, and stopping things is what large organizations are worst at rewarding. The way large problems actually get cracked has more to do with sequencing the right questions than with the volume of analysis applied to them.
What our business experimentation consulting services include
Five pieces of work. Most engagements use all of them, in this order.
1. Finding the assumption that carries the risk
Every new proposition rests on a stack of beliefs, and only a few of them decide whether it works.
- A written list of what the idea depends on, separated into demand, willingness to pay, delivery and route to market
- A judgment on which of those are both load bearing and genuinely uncertain, a much shorter list than teams expect
- The belief ranked first because it is cheap to test and fatal if false, not because it is the one the team enjoys discussing
2. Designing the smallest test that could disprove it
A test earns its cost by the decision it can force, so we design backward from the decision.
- The instrument chosen to match the risk: a buyer conversation, a priced offer, a landing test, a single store or a limited live-market run
- A sample and a duration sized to the claim being made, rather than to the budget available
- A test that is affordable to lose, because an experiment nobody can afford to fail is a commitment with a research line item
3. Concept testing with people who would actually buy
Concept testing works when the risk sits in demand, and it fails when the concept is described as a benefit statement rather than as something for sale.
- The proposition written the way it would be sold, with the price, the trade-off and the name of the thing it replaces
- Price and value tests that ask for a commitment, because stated interest and a signed order are different kinds of data
- Non-buyers included on purpose, since the people who decline explain the ceiling better than the enthusiasts explain the opportunity
4. Deciding in advance what the result has to be
Ambiguity is the standard outcome of an unspecified test, and it is always read in favor of continuing.
- A threshold written before the test runs: the number, the behavior or the signed commitment that counts as passing
- The reading of a near miss agreed at the same time, so a marginal result cannot be renegotiated into a yes afterward
- Evidence standards made explicit, including what the test cannot tell you, which is usually the part that gets quoted later
5. Making a result count as a decision
A finding that nobody records is a finding the company will pay to produce again.
- A decision record for each test stating what was believed, what was run, what came back and what changed as a result
- Results routed to the people who allocate money, not only to the team that ran the test
- A visible register of what has been disproved, so an idea that already failed has to arrive with new evidence rather than a new sponsor
- AskWhich belief decides thisThe assumption that would end the project if it were false, rather than the one that is easiest to measureUsually skipped
- DesignA test that could come back noThe smallest instrument that could disprove the belief, sized so a negative result is affordableUsually too safe
- ReadWhat the result has to sayA passing threshold set before the test runs, so a marginal number cannot be argued into a yesUsually ambiguous
- DecideWhat changes because of itContinue, redesign or stop, written down with the evidence that settled itUsually unrecorded
Not sure which assumption to test first?
Start with a working session with one of our partners. We will take the plan you already have and find the belief it depends on most, then size the smallest test that would settle it.
How a business experimentation engagement works
- Surface the assumptions. We take the concept, plan or business case as written and pull out what it depends on, in the team’s own words, separated into demand, price, delivery and route to market.
- Rank by risk. Each assumption scored on how uncertain it genuinely is and how much of the case collapses if it is wrong. The output is one belief to test first, with the reasoning visible so the choice survives challenge.
- Design and run. The smallest instrument that could disprove it, with the passing threshold written down before fieldwork starts, run with real buyers rather than a panel standing in for them.
- Decide and record. A reading against the threshold, a decision to continue, redesign or stop, and a record your teams can run the next test from without us.
Why companies choose Vivaldi for business experimentation
Most firms that sell experimentation come from one of two places: a research house that measures reactions, or a digital optimization shop that tunes pages. Both are useful, and neither answers whether a proposition should exist.
We come at it from demand. Our work starts with what customers are trying to get done, which is what makes the difference between a concept that tests well and a concept that is bought. That means the assumptions we put at the top of the list tend to be commercial ones: whether the need is real, whether anyone will pay the price the model needs, whether the route to market exists.
We also design tests we expect to lose some of the time. A firm that never delivers a negative result is not running experiments, and a program measured by how many ideas it approves will approve them.
How to evaluate a business experimentation consultant
Worth putting to every firm on your list, including us.
- Ask what they would test first, and why that one. A firm that answers with a research plan rather than a single belief is selling reassurance. The useful answer names the thing that would end the project if it proved false.
- Ask how small they can make it. Experienced practitioners compete on how little they need to spend to learn something decisive, not on the size of the fieldwork.
- Ask whether they test willingness to pay. Appeal is easy to measure and weakly predictive. A commitment is harder to obtain and tells you considerably more.
- Ask what result would make them tell you to stop. If no answer exists before the test runs, no answer will exist after it.
- Ask who runs the second experiment. If the answer is the consultancy, you bought a study. A program leaves behind a method your own teams can run without raising a purchase order.
Business experimentation vs A/B testing vs a market pilot?
| Criterion | Business experimentation | A/B testing | Market pilot |
|---|---|---|---|
| What it answers | Whether the proposition and its economics hold at all | Which version of an existing experience performs better | How to deliver something already decided |
| When it fits | Before the thing exists, while the case is still a set of beliefs | After launch, where traffic is high and the change is small | After the decision, when implementation is the open question |
| Can it return a no | Yes, and the threshold for one is agreed in advance | Yes, within the narrow range being varied | Rarely, because the commitment is already made |
| Typical cost of being wrong | A few weeks of work and a stopped project | A reverted deployment | A quarter, a team and a launch nobody wants to cancel |
| Who runs it | Strategy, innovation and commercial together | Product, growth and analytics | The operating business |
Related work
Two connected capabilities, in the order they usually arrive, plus the group this method sits in.
- Venture design once a concept has survived its first tests and the question becomes what the thing actually is.
- Corporate venture building when the proposition is proven and the constraint is standing a new business up inside a parent company.
- Business model innovation if the uncertain part is the model itself rather than the proposition in front of the customer.
- Experimentation and ventures for the question this page does not answer: how much to spend finding out, which bets are held as a set, and when each one stops.
This work sits inside our Marketing Strategy & Acceleration practice at Vivaldi, the strategy firm behind the Demand Landscape and the Interaction Field.
Business experimentation consulting in the US and worldwide
Our experimentation work runs out of ten offices on four continents, with the US team in New York. The method travels, but the evidence does not: willingness to pay, the credibility of a claim and the route to market all move by market, and a concept validated in one country regularly fails on price or channel in the next. Where a proposition is meant to launch in several markets, we test the assumption that differs most between them first, which is usually price or distribution rather than appeal.
Start with the assumption you are least willing to test
In one working session we will take the plan you are already committed to, name the belief it depends on most, and size the smallest experiment that would settle it. You leave with the test worth running first and what its result would have to be.