Nearly every offer decision — which gift, which threshold, which wording, which placement — is an empirical question that teams resolve by opinion. A split test resolves it in two weeks with your own traffic. NUO AI runs variants side by side, shifts traffic toward the winner, and reports the result in a sentence rather than a statistics table.
Pick one thing to change. Two thresholds, two gifts, two wordings. One variable per test, or the result tells you nothing.
Split the traffic. Visitors are assigned consistently, so a returning shopper sees the same variant and the comparison stays clean.
Let it run to a real sample. Offer tests need acceptances, not sessions. Stopping at the first promising day is how most tests produce wrong answers.
Shift to the winner. Traffic moves toward the better variant automatically, with a plain-language summary of what changed and by how much.
Run them in that order. Threshold and holdout tests change the economics of the offer; copy tests move it a few percent.
Stopped too early. Early results swing wildly. A variant that is 30% ahead on day two is usually noise, and calling it then is the most common error in ecommerce testing by a wide margin.
Not enough events. Offer tests are powered by acceptances, not pageviews. A store with 40 acceptances a week cannot detect a 10% difference in a fortnight, and pretending otherwise produces confident nonsense. Test bigger changes when volume is low — a different gift, not a different button colour.
Several things changed at once. A new threshold plus new copy plus a new gift tells you the combination won, not which part did, and you cannot carry the finding to the next campaign.
Measured on the wrong metric. Acceptance rate is not the goal. A lower threshold always wins on acceptance and can easily lose on margin. Judge on revenue or margin per session.
A usable rule: to detect a 10% relative difference you need roughly a few hundred conversions per variant; to detect 25% you need far fewer. That is why small stores should test large changes and large stores can afford to test wording.
Decide the duration before you start and run at least two full weeks, so both variants see the same mix of weekdays and weekends. Then resist looking for a winner daily. Checking repeatedly and stopping when the number looks good is the mechanism that turns a 5% false-positive rate into something far worse, and it is why so many "proven" offer tactics fail to replicate.
Where a test genuinely cannot reach significance, say so and decide on judgment. An honest "too small to tell" is more useful than a confident number built on 30 orders.
The compounding comes from sequence, not from any single test. Four tests a quarter, each carried forward into the next, beats one carefully designed experiment a year — and offers are unusually suited to this because the variable is usually a number you can change in seconds.
Keep a short written record of every test: what changed, what happened, what you decided. Without it, teams re-run the same threshold test annually and re-learn the same answer. With it, each campaign starts from the last one's result, and the funnel gives you the per-step detail to explain why a variant won rather than only that it did.
The threshold. Two spend levels with the same gift decides both acceptance and the size of the lift, so it moves the economics of the offer more than any copy or design change. Then run a holdout against no offer at all.
At least two full weeks, so both variants see the same weekday and weekend mix, and long enough to accumulate a few hundred conversions per variant if you want to detect a 10% difference. Decide the duration before you start.
Yes, but test large changes rather than small ones. A different gift or a holdout can be detected on modest volume; a button wording change cannot. At low volume, an honest "too small to tell" beats a confident conclusion from 30 orders.
No. A lower threshold always wins on acceptance and frequently loses on margin. Judge on revenue or margin per session, which is what you are actually trying to grow.
Early results are noise, and checking daily while waiting for a winner inflates your false-positive rate well past the nominal 5%. Set the duration up front and read the result at the end.
Once a variant shows a durable advantage, more traffic is routed to it so you stop paying for the losing variant while the test finishes. The readout explains what changed and by how much in plain language.
Yes. Automated A/B testing is included at $0 like every other NUO AI feature — no paid tier, no order cap, no revenue share.
