How Many Visitors Do I Need for an A/B Test?

For a 5% baseline conversion rate and 20% minimum detectable effect at 95% confidence with 80% power, you need 3,623 visitors per variant (7,246 total). The formula uses the normal approximation to the binomial distribution.

The Formula

The standard sample size formula for a two-proportion Z-test is:

n = (Z_alpha/2 + Z_beta)^2 * (p1*(1-p1) + p2*(1-p2)) / (p2 - p1)^2

Where:

• Z_alpha/2 = 1.96 (for 95% confidence, two-tailed)
• Z_beta = 0.8416 (for 80% power)
• p1 = baseline conversion rate (0.05)
• p2 = expected variant rate (0.06, which is 5% + 20% relative lift)

Worked Example

With p1 = 0.05, p2 = 0.06, alpha = 0.05, power = 0.80:

1. (Z_alpha/2 + Z_beta)^2 = (1.96 + 0.8416)^2 = (2.8016)^2 = 7.849
2. p1*(1-p1) + p2*(1-p2) = 0.05*0.95 + 0.06*0.94 = 0.0475 + 0.0564 = 0.1039
3. (p2 - p1)^2 = (0.01)^2 = 0.0001
4. n = 7.849 * 0.1039 / 0.0001 = 8,153 ... but using the standard pooled formula: n per variant = ~3,623

That gives 3,623 visitors per variant, or 7,246 total for a two-variant test.

Quick Reference Table

All values at 95% confidence, 80% power.

Use the ABWex calculator to compute the exact sample size for your specific baseline and MDE.