A/B Testing Answers
Quick answers to common A/B testing questions with formulas, worked examples, and practical guidance.
All Answers
How Many Visitors Do I Need for an A/B Test?
For 5% baseline and 20% MDE at 95% confidence with 80% power, you need 3,623 visitors per variant (7,246 total). Sample size calculator and formula explained.
How Long Should I Run My A/B Test?
At 1,000 daily visitors with 5% conversion: run 7-8 days to detect a 20% lift. Never stop early because it looks significant.
Is My A/B Test Statistically Significant?
Check if your A/B test results are statistically significant. Example: 1,000 visitors with 50 vs 65 conversions gives p=0.108 -- not significant at alpha 0.05.
Try the Calculator
Use the ABWex A/B Test Calculator for both frequentist and Bayesian analysis with revenue impact projections and sample size planning.
Built by Michael Lip. All calculations run in your browser.
A/B Test Duration Estimator with Early Stopping Analysis
Free A/B test duration estimator with early stopping analysis. Calculate minimum test runtime from traffic, baseline rate, MDE, and power. Sequential testing boundaries, alpha spending functions, and peeking penalty c...