A/B Test Sample Size Calculator

Normal-approximation planning for equally sized independent groups and a binary outcome. No sequential testing or multiple-comparison adjustment.

Loading configuration…

Result

How to use this A/B test sample size calculator

Enter your current baseline conversion rate, such as 10% of visitors completing a purchase.
Enter the minimum relative lift you care about. A 20% relative lift on a 10% baseline means a 12% target conversion rate.
Choose a two-sided significance level: 0.05 corresponds to 95% confidence under the test assumptions.
Enter statistical power: 0.80 is a common planning choice.
Select Run tool to estimate required participants in each variant and in total.
Recruit enough eligible, independently assigned visitors and run the experiment for a representative business cycle.

Understanding the inputs

Baseline conversion rate (%): Expected percentage of eligible visitors who complete the measured action in the control group.
Minimum relative lift (%): The smallest proportional improvement worth detecting. Relative lift is not percentage-point lift.
Two-sided significance level (alpha): Risk of rejecting the no-difference hypothesis when it is true, assuming the test's conditions hold. A value of 0.05 corresponds to a nominal 5% false-positive rate.
Power (1 − beta): Probability of detecting the specified effect if it truly exists under the assumptions used in the sample-size calculation.

Example: baseline 10%, relative lift 20% → target 12%, an absolute increase of 2 percentage points. A 2% relative lift instead targets 10.2% and usually requires a much larger sample.

Understanding your results

Per group is the estimated minimum number of eligible observations assigned to each variation when traffic is split equally. Total combines both groups. Results are planning estimates, not a promise that an observed result will be statistically significant.

To estimate test duration, divide the total required sample by the average number of eligible experiment visitors per day. For example, 8,000 total visitors at 400 eligible visitors per day is approximately 20 days. Allow extra time for weekday/weekend variation and implementation checks.

What formula is used?

This tool describes a normal-approximation, two-sided comparison of independent binary conversion outcomes with equal group allocation. Results may differ slightly across calculators due to variance assumptions, continuity corrections and rounding. This tool does not adjust for repeated peeking, multiple comparisons or adaptive experiments.

Common A/B testing mistakes

Confusing a 20% relative lift with a 20 percentage-point lift.
Stopping as soon as a nominal p-value falls below 0.05 without an appropriate sequential design.
Including visitors who were never eligible for the experiment.
Changing the primary metric or target effect after seeing results.
Ignoring seasonality, broken tracking, cross-device duplication or interference between groups.

Frequently asked questions

Why does a smaller minimum detectable effect need more visitors?

Small conversion differences are harder to distinguish from sampling noise, so the required sample size grows.

Is 95% confidence the same as 95% probability the winning variant is better?

No. A frequentist confidence level does not directly assign a posterior probability that a particular variant is better.

Should I use relative lift or percentage points?

This calculator accepts relative lift. To convert: absolute percentage-point lift = baseline percentage × relative lift percentage ÷ 100.

Can I use the result for more than two variants?

Not without considering multiplicity and traffic allocation. A multi-arm design may require a different calculation.

Does the calculator prove that a change caused the improvement?

No. Valid randomization, reliable instrumentation and an appropriate experimental design are still required.

What happens after the experiment ends?

Use the A/B Test Significance Calculator to analyze observed conversions and group sizes.