Skip to content
Bifröst Orbit
Sign in

Experiments in CMS Orbit

Stop guessing. Let your visitors decide.

Show some visitors another version of the hero, or of a whole page, and count who goes on to do what you want, such as booking a trip. CMS Orbit tells you in plain words which version works better and when the answer is reliable.

The winner goes into the page's draft and goes live through the page's usual review. Part of the Growth and Enterprise plans.

Try an A/B test

Northwilds tested a hero that names the Lofoten ridge walk against its usual headline, counting paid bookings.

CMS OrbitExperiment
  1. Advice in plain words
  2. Each variant as visitors see it
  3. A person chooses the winner

How it works

From an idea to the winner, in the editor

  1. 01

    Pick what to test

    Open the page, select a component such as the hero and choose Test. Or test the whole page.

  2. 02

    Choose the goal

    A click, a visit to a page, a form sent, visitors who stay, an event such as Booking paid, or a goal from Insights Orbit.

  3. 03

    Edit the variant

    Change only what you want to test, in the same editor. Check each variant on its own preview link.

  4. 04

    Start and wait

    Visitors are shared between the original and up to three variants. A week evens out weekdays and weekends.

  5. 05

    Choose the winner

    When the advice says it's safe, put the winner in the page's draft. It goes live through the usual review.

Beside the page

Started, followed and decided from the page itself

The Test panel sits in the editor's right-hand rail. It sets up the experiment, opens each variant and, once it runs, shows the advice, each variant's visitors and rate and the buttons to pause, stop or choose the winner.

  • Only what you change: components you leave alone keep following the page, so a typo fixed later is fixed in every variant.
  • Fair from the start: once it starts, variants, goals and audience are fixed and the variants open read only.
  • Preview links: anyone with the link can see a variant before it starts. Visits through them are never counted.
CMS OrbitPage editor
  1. Each variant's visitors and rate

Reading the results

How sure, how much better, and how much longer

The advice at the top says what to do: keep it running and for how long, choose a winner, keep the original, or that there's no real difference. The charts under it show why.

  • Chance to beat the original: 50% means you can't tell yet.
  • Uplift with its 95% range: a range from −5% to +20% means it could still go either way.
  • Risk if chosen: how much conversion you could lose if the variant is in fact worse.
  • Guardrails: goals that must not get worse, such as visitors who stay, turn red when a variant harms them.
  • A warning when traffic is off: if variants get a different share of visitors than planned, the results say so before you trust them.

A winner is only called once every variant has at least 100 visitors, the minimum days have passed, it's at least 95% likely to beat the original and the risk is under 1% of the original's rate. You can look every day.

CMS OrbitExperiment
  1. How sure the result is
  2. Guardrails

When a winner is called

Four checks before the advice says choose

You can look every day. The advice only says choose a winner once all four are true.

  • 100

    visitors at least, in every variant

    Small numbers swing too much to trust.

  • 7 days

    at least, to start with

    A full week evens out weekdays and weekends.

  • 95%

    likely to beat the original

    50% means you can't tell yet.

  • < 1%

    risk if chosen

    Of the original's rate, should the variant in fact be worse.

Your choice

Who takes part, and how

Audience

Computers, tablets or phones, new or returning visitors, countries, and visitors arriving from a campaign.

Share of visitors

Choose how many visitors take part at all. The rest see the original and aren't counted.

Even split or move towards the best

An even split gives the clearest answer. For short campaigns, send more visitors to the variant doing best, keeping at least 10% on each.

Schedule

Start and end it by itself, with a minimum number of days. Seven to start with, so a full week evens out.

A whole page or one component

Test the hero alone, or change, add, remove and reorder anything on the page.

With Insights Orbit

See where each variant's visitors went next, how long they stayed and the site's other goals. Insights Orbit only shows the numbers.

Privacy

Fair tests that respect your visitors

Cookie after consent

The default. Visitors who accept cookies keep their variant for 13 months, and before that take part for the day without one.

Or cookieless only

Nothing is stored in the browser. A code for the day, worked out on your server, keeps each visitor's variant until tomorrow.

Do Not Track respected

Visitors with Do Not Track or Global Privacy Control, search engines and bots always see the original and aren't counted.

Deleted after 90 days

Each visitor's record is deleted at the latest 90 days after the experiment ends. CMS Orbit stores no address, browser details or cookie.

Works with your tools

Google Tag Manager, Google Analytics 4 and VWO can see each visitor's variant through the data layer, and their events can be your goals.

AI drafts, people decide

AI agents can read results and draft an experiment for you to check. Only people start, pause, stop and choose winners.

Plans

On Growth and Enterprise

Experiments are part of the Growth and Enterprise plans of CMS Orbit. On Starter, the Test panel shows which plan includes them, and an owner can upgrade under Billing in Orbit.

Your website needs version 0.11 or later of the JavaScript SDK, which runs experiments with no extra code. One more line lets visitors take part without cookies.

Questions

About experiments

Do A/B tests in CMS Orbit need a developer?

No. On Growth and Enterprise, editors test a component or a whole page with up to three variants in the editor, choose a goal and an audience, and approve the winner like any other change. Websites on the JavaScript SDK 0.11 or later run experiments with no extra code.

How long does an experiment need?

It depends on your traffic. When you set one up, CMS Orbit estimates how long it needs to find a 10% and a 20% improvement from the page's visitors and today's success rate. A week is the minimum to start with, so weekdays and weekends even out, and the results say roughly how many more days a reliable answer takes.

Do experiments need a cookie banner?

Not necessarily. By default, visitors who accept cookies keep their variant for 13 months, and before that take part for the day without one. Or run cookieless only: nothing is stored in the browser. Which fits depends on the law where your visitors are, so ask whoever looks after data protection.

Does the winner go live straight away?

The winner goes into the page's draft and the experiment ends, so every visitor sees it until the page is published. Publishing goes through the page's usual review and approvals, so nothing goes live without a person approving it.

Find out what works. Then keep it.

Start a website in CMS Orbit on the Growth plan and run your first experiment on a real page. The trial is free for 14 days.