Just test what really works: Why experiments make all the difference
24. Sep 2026
Making a small change to the online store is quick and easy. But the key question is: How will it affect customers' shopping behavior?
Reading time: 3 minutes
Experiments can help answer this question. In an A/B test, two groups of customers see different versions of a website. Relevant metrics are then compared to determine which alternative performs better.
This could be a newly placed button, a different product display, or a modified checkout process. Teams can test ideas, measure their impact, and choose the version that resonates better with customers. This approach helps determine what actually optimizes the shopping experience and what merely appears to be an improvement at first glance.
Experiments are becoming easier
The principle isn't new - we've been using tests like these for years. Until now, however, this involves a significant amount of effort: developers have to implement the experiments technically, the research team supports the planning and evaluation, and statistical calculations have to be performed separately.
We aim to make this process significantly simpler with our Experimentation Platform. It should help formulate hypotheses, suggest appropriate KPIs, and provide statistical results.
We combine technical e-commerce expertise with UX research and experiments. The platform should thus provide the technical foundation for tests. In addition, our research expertise supports the planning, execution, and evaluation.
For teams, this means less effort in preparation and evaluation. Experiments should be easier to set up independently and show more quickly whether a change has the desired effect. This allows teams to make more informed decisions about which adjustments they want to pursue.
What can be measured with an experiment?
Which metrics are relevant depends on the specific goal. For a new button, the focus might be on the conversion rate. When it comes to changes in the product lineup, “add to basket” or “add to wishlist” are possible KPIs. Depending on the question at hand, revenue or margin may also play a role.
These metrics raise very specific questions: Are customers finding the right product faster? Are they adding it to their cart more often? Or is the checkout process easier for them?
It’s important not to look at just a single metric. If the conversion rate rises but the return rate increases at the same time, both trends are part of the overall result and must be considered together.
What role does AI play?
AI can assist with planning and evaluating experiments. It can help formulate hypotheses, suggest appropriate KPIs, or summarize results.
Subject-matter expertise remains important. The statistics show which metrics have changed. However, the significance of these changes for the store and its customers must be evaluated within the respective context.
What happens next?
The Experimentation Platform will be expanded step by step. In the future, it will also support other types of experiments, including those in apps and for our customers’ own tests.
Our goal: To enable teams to conduct experiments more easily and independently, without having to rely on development or additional support for every test. This frees up more time for other tasks and the further development of the store.
