A/B Test Sample Size
Visitors per variant to detect a lift, with confidence and power.
How to read this
You need 3,842 visitors in each of the control and treatment groups to detect a 20% relative lift over a 10% baseline, with 95% confidence and 80% power. Total traffic across both is 7,684.
Two-tailed proportion test, normal approximation. Same math as Evan Miller's classic calculator. Smaller lifts and lower baseline conversion both blow up sample size: detecting a 5% relative lift over a 2% baseline takes hundreds of thousands of visitors per variant.
About
Enter your baseline conversion rate and the smallest lift you care about detecting. Get the visitors required per variant for a two-tailed proportion test at the chosen confidence level and statistical power.
How to use
- Enter baseline conversion and target lift.
- Pick confidence and power.
- Read sample size per variant.
FAQ
Why are sample sizes huge for small effects?+
The relationship is roughly inverse-square: detecting a 1% lift takes 100× the traffic of detecting a 10% lift at the same confidence. Small lifts and small baselines compound that.