A/B test sample size calculator.
Before you run a test, find out how many visitors you need for a trustworthy result, and how long it will take.
For a two-version test (A vs B) at 95% confidence and 80% power, the standard settings. Visitors per day is your total traffic, split evenly between the two versions. When the test ends, check the result with our A/B test significance calculator.
Free, no sign-up. Everything is calculated in your browser.
How to size an A/B test.
The most common A/B testing mistake is stopping too early. A version can look like a winner after a few days purely by chance. Working out your sample size before you start tells you how long to run the test, and stops you calling a winner that isn't real.
What you need to know
- Current conversion rate: how often visitors convert today, e.g. 3%.
- Smallest improvement worth detecting: as a relative change. 20% means spotting a move from 3% to 3.6%.
- Your traffic: visitors per day, to turn the sample size into a test length.
A worked example
With a 3% conversion rate and a 20% improvement to detect, you need roughly 13,900 visitors per version: about 27,800 in total. At 1,000 visitors a day, that's around four weeks. Want to detect a 10% improvement instead? You'd need about four times as many visitors.
Rules for reliable tests
- Run for full weeks, so weekday and weekend behaviour are both included.
- Don't stop early because one version is ahead: wait for the sample size.
- Test bold changes when traffic is limited. Big differences need far fewer visitors to detect.
- Change one thing at a time, so you know what caused the result.
The same principles apply to ad creative: see our creative-testing frameworks.
Keep calculating.
A/B test significance calculator
Check whether the difference between two variants is statistically significant before you call a winner.
Open tool →Testing & measurementIncrementality calculator
Compare people who saw your ads with a control group to see the lift, the extra conversions your ads caused and incremental ROAS.
Open tool →TrackingTracking checker
Enter your website and see which pixels and tags it runs, what fires before cookie consent, whether Consent Mode v2 and ad click IDs work, and if you have server-side tracking.
Open tool →Sample size questions.
01How many visitors do I need for an A/B test?
It depends on your current conversion rate and the size of improvement you want to detect. Small improvements on low conversion rates need far more visitors than big improvements on high ones: this calculator works it out for you.
02What does “smallest improvement to detect” mean?
It is the smallest relative lift worth knowing about. A 20% improvement on a 3% conversion rate means detecting a change to 3.6%. The smaller the improvement, the more visitors you need to be confident it is real.
03What do 95% confidence and 80% power mean?
They are the standard settings for reliable tests. 95% confidence means a low chance of declaring a winner when there is no real difference. 80% power means a good chance of spotting a real improvement of the size you chose.
04Can I stop the test early if one version is winning?
It is best not to. Checking results repeatedly and stopping as soon as one variant looks ahead greatly increases the chance of a false winner. Decide the sample size up front and run the test to the end.
We design creative tests, and advise on landing page tests, that reach a reliable result, and turn the learning into better performance.