Evidence-Driven Design Consulting

Evidence-driven design consulting is advisory work on a narrow question: which design decisions in a digital product are worth settling with evidence, what evidence would actually settle them, and what the company does when the evidence disagrees with the plan. Vivaldi is a strategy firm. We do not staff your design team, draw your screens or run your analytics stack. We work on the decisions those things are being asked to resolve, and on the gap between the data a product already generates and the choices anyone makes because of it.

• Since 1999brand, product and experience strategy work
• 200+published case studies
• 10 officeson four continents
• 5xWirtschaftsWoche Best of Consulting, Strategy and Innovation

Two boundaries first, because they save a wasted meeting. If your question is how the product behaves when a person acts on it, the states, defaults, feedback and automated behavior inside the interface, that is interaction design (IxD) and it is a different engagement.

If your question is the participant model, the value exchange and the governance of a system that spans several companies, that is digital ecosystem design, and this page is not the one you want.

When companies bring us in

• A redesign is funded and nobody can say, in one sentence, what result would prove it was the right call.
• The product generates more behavioral data every quarter, and the design roadmap has not changed because of any of it.
• Two senior people disagree about a flow, both cite research, and the research does not actually address the point in dispute.
• Testing happens late, after the build is committed, so it produces reassurance rather than decisions.
• An agency or internal team is producing good work against a brief that was never argued with.

Why more research does not make the decision better

Most companies in this position are not short of evidence. They are short of decisions written down clearly enough for evidence to contradict them.

Research gets commissioned against a topic rather than a question. A usability round tells you people struggled on step three, which everyone already suspected, and says nothing about the choice the team was actually arguing over. Analytics report what happened inside the design you already shipped, so they can rank variants of the current idea and cannot see the one you did not build. Meanwhile the decisions that carry the most risk, what the product is for, what it asks of a person and what it decides on their behalf, are usually made in a meeting and never written anywhere that a later finding could reach.

The failure is not rigor. It is sequence. Evidence collected before the decision is named tends to confirm whatever was already planned, because nobody agreed in advance what result would have changed their mind.

What our evidence-driven design consulting includes

1. A decision inventory

We write down the design decisions actually in play, separate the reversible ones from the ones that are expensive to undo, and establish which are already settled by strategy and which are genuinely open. Most lists come back shorter than the team expected, with two or three decisions carrying nearly all the risk. Those are the only ones worth spending evidence on.

2. Evidence that could change the answer

For each open decision, what observation would settle it, and what it costs to get. Sometimes that is a prototype in front of people who could actually buy. Sometimes it is a query against data you already hold. Sometimes the honest answer is that no affordable evidence will settle it, in which case we say so and the decision gets made on judgment, explicitly, by someone accountable for it.

3. The loop the product learns on

A design decision should leave something behind that makes the next one easier. In practice the loop breaks in the same place in most organizations.

4. Judgment on the work itself

We review design work as it comes back, against the decisions and the evidence rather than against taste, and we stay through the first release. That last part matters, because decisions get quietly renegotiated during implementation and the evidence that justified them is rarely in the room.

How an evidence-driven design engagement works

1. Frame. Interviews with the people who own the product, the people who fund it and the people who support it. A walk through the current experience with the analytics and support data alongside. The output is the decision inventory and a note on which decisions are already closed.
2. Rank. The open decisions sorted by consequence and reversibility, not by how interesting they are. We agree which two or three are worth evidence and what the evidence budget is.
3. Settle. Get the observation that resolves each one. Existing data where it answers the question, a prototype or a structured field session where it does not. Each one is agreed in advance as a test of a stated claim.
4. Hand over. The decisions, the evidence behind each, the verdict rule for the ones still running, and the loop your team keeps using once we leave. Then review the work against it through the first release.

Why companies choose Vivaldi for design decisions

We are a strategy firm sitting in design conversations, which makes us unusually willing to say that a decision is commercial rather than experiential and belongs to someone else in the building. We will not sell you a research program to postpone a judgment call, and we will tell you when the evidence you want is not worth its price.

We also do not produce the design work, which removes the obvious conflict: we have nothing to gain from recommending a larger redesign, because someone else would be doing it.

How to evaluate an evidence-driven design consultant

• Ask them to state the decision before the method. A firm that opens with a research plan rather than a question is selling a process.
• Ask what result would make them wrong. If no answer exists in advance, the study will confirm whatever you already intended.
• Ask what they would not test. Honest practice names the decisions that have to be made on judgment because no affordable evidence reaches them.
• Ask who owns the loop afterwards. If the measurement only works while the consultancy is engaged, you rented a verdict rather than building a capability.
• Ask what they would cut. A firm that cannot name research you are already paying for and should stop has not read your situation.

Evidence-driven design consultancy vs user research agency vs product analytics vendor?

Related work

• Interaction design (IxD) if the question is how the interface behaves when someone acts on it.
• Digital ecosystem design if the question spans several companies rather than one product.
• User insights and research if the evidence you need is qualitative fieldwork and you want it run properly.
• Prototyping and testing if the decision needs something built small enough to be disproved.
• User experience (UX) design if what is unsettled is what the product is for and how the flow is structured.
• User interface (UI) design if the argument is about the screen rather than the decision inside it.
• Predictive and advanced analytics if the evidence you want requires a model rather than an observation.
• Data strategy if the product cannot currently produce the evidence at all.

This work sits inside our Digital Products and Platforms practice at Vivaldi, the strategy firm behind the Demand Landscape and the Interaction Field.

Evidence-driven design consulting in the US and worldwide

Our product and experience strategy work runs out of ten offices on four continents, with the US team in New York. Most engagements of this kind run remotely, with two or three days on site at the points where decisions are actually made. Where a product launches in several markets, we keep one decision inventory and gather the evidence per market, because the behavior that settles a question in one country frequently does not settle it in another, and a verdict quietly imported from the US is the most common way a global rollout goes wrong.