A/B test significance.
Enter the visitors and conversions for each variant to see the conversion rates, the uplift, and whether the result is statistically significant.
Uses a two-proportion z-test at a two-tailed 95% threshold. More data = more reliable: don’t stop the moment it crosses the line. Planning a test? Use the sample size calculator first.
Free, no sign-up. Everything is calculated in your browser.
How A/B test significance works.
An A/B test compares two versions of an ad, landing page or email to see which converts better. The catch is that small differences can appear by chance. Statistical significance tells you how unlikely your result would be if there were really no difference between the variants.
This calculator uses a two-proportion z-test. It compares the conversion rates of A and B, allowing for how many visitors each received, and checks whether the gap clears a two-tailed 95% confidence threshold. In the default example, A converts at 5.0% (250 ÷ 5,000) and B at 6.0% (300 ÷ 5,000): a 20% relative lift that clears the 95% threshold.
Rules for trustworthy tests
- Decide the sample size up front and don't stop the moment the result turns significant: repeatedly peeking inflates false positives.
- Run for full weeks so both weekday and weekend behaviour are included.
- Change one thing at a time, or you won't know what caused the difference.
- Test meaningful differences: new hooks, offers or page layouts, not tiny tweaks that need huge samples to detect.
Significant isn't the same as important
Significance tells you a difference is probably real, not that it's large or profitable. Check the winner against your CPA or ROAS targets before rolling it out. For how to structure ad tests, see our creative-testing frameworks.
Keep calculating.
A/B test sample size calculator
Find out how many visitors you need, and how long to run your test, before you start, so you don't call a winner too early.
Open tool →Testing & measurementIncrementality calculator
Compare people who saw your ads with a control group to see the lift, the extra conversions your ads caused and incremental ROAS.
Open tool →TrackingTracking checker
Enter your website and see which pixels and tags it runs, what fires before cookie consent, whether Consent Mode v2 and ad click IDs work, and if you have server-side tracking.
Open tool →A/B testing questions.
01What does statistical significance mean in an A/B test?
It’s the probability that the difference you’re seeing between two variants is real and not just random chance. 95% confidence (p < 0.05) is the common threshold for calling a winner.
02How is significance calculated here?
This tool uses a two-proportion z-test comparing the conversion rates of the two variants, then converts the z-score to a two-tailed confidence level.
03Why isn’t my test significant yet?
Usually not enough data, or the difference is genuinely small. Let the test run longer, and avoid stopping the moment it briefly crosses 95%: "peeking" inflates false positives.
A disciplined testing roadmap is part of how we work. See our consultancy and media management.