A/B Testing Tools & Guides
Free statistical tools and reference guides for designing, analyzing, and interpreting A/B tests. All content runs client-side with no data sent to any server.
Plan the test
A/B Test Sample Size Calculator
How many visitors each variant needs before you launch, from your baseline rate, the effect you want to detect and the power you want.
Sample Size Calculator for Proportions
Sample size for a two proportion comparison, with the normal approximation stated and its small sample warning shown.
Minimum Detectable Effect Calculator
The smallest lift a given sample can reliably detect. Run it before the test, not after, because a test that cannot see your effect will not find it.
Statistical Power Analysis Calculator
Power for a planned comparison, and what happens to it when the sample or the effect changes.
Statistical Power Visualizer
The two distributions drawn side by side, so you can see where alpha, beta and the effect size actually sit.
Experiment Duration Estimator
How long a test needs to run at your traffic, once the sample size is fixed.
Multivariate Test Duration Estimator
Duration for a multivariate design, where the cell count grows faster than most people expect.
Read the result
P-Value Calculator for A/B Tests
Two proportion Z test, with the common misreadings of a p value spelled out next to the number.
Conversion Rate Calculator with Confidence Intervals
Rates and their intervals for each variant, so the comparison is a range rather than a point.
Confidence Interval Calculator
Intervals for a mean, a proportion and a difference, with the assumption behind each one stated.
Confidence Interval, Mean and Proportion and Difference
The three interval types in one place when you need to compare them directly.
T-Test Calculator
One sample, two sample and paired t tests, with the degrees of freedom shown rather than hidden.
Pooled Standard Deviation Calculator
Pool two standard deviations, then read the Student and Welch tests it feeds. Shows the variance ratio so you can see whether pooling was ever justified.
Z-Score Calculator
Standard score and the probability either side of it.
Chi-Square Test Calculator for A/B Testing
Chi square for a contingency table, with expected counts shown so you can check the approximation holds.
Chi-Square Test Calculator
The same test with a wider input form for larger tables.
Effect Size Calculator
Cohen's d, Hedges' g and Glass's delta, which is what a power calculation and a meta analysis actually consume.
Corrections and multiple metrics
False Discovery Rate Calculator
Benjamini Hochberg, Benjamini Yekutieli, Holm and Bonferroni side by side, with a first party simulation of what tracking several metrics costs. Ten independent metrics with no real effect still crowned a false winner 40.2% of the time.
Bonferroni Correction Calculator
The simplest correction, one multiplication, and the cost in power that comes with it.
Peeking Alpha Inflation Calculator
What repeatedly checking a running test does to your real false positive rate. The answer is usually much worse than people assume.
Sequential Testing Calculator, SPRT
A test designed to be looked at repeatedly, which is the honest fix for peeking.
Sequential Testing Tool
Boundaries and stopping rules for a sequential design.
Trust the data
Sample Ratio Mismatch Checker
Whether your split arrived at the ratio you asked for. A failed SRM check invalidates the result no matter how good the p value looks.
Margin of Error Calculator
The plus or minus around a survey or test estimate at a given confidence level.
Regression to the Mean Calculator
How much of an extreme early result is likely to be noise that will fade on its own.
Stratified Sampling Calculator
Proportional and Neyman allocation across strata, and the design effect each buys. Shows whether stratifying actually reduced variance or just added work.
CUPED Variance Reduction Calculator
How much variance a pre period covariate removes, which is usually the cheapest way to shorten a test.
Delta Method Ratio Metric CI Calculator
Correct intervals for a ratio metric, where the naive interval is wrong because the denominator is random too.
Bayesian
Bayesian A/B Test Calculator
Posterior distributions, probability that B beats A, and expected loss.
Bayesian A/B Calculator
The Beta Binomial model with the prior stated explicitly rather than buried.
Bayesian A/B Testing Guide
What the Bayesian numbers mean and where they differ from the frequentist ones.
Multi Armed Bandit Simulator
Thompson sampling against a fixed split, simulated so you can see the regret rather than read about it.
Money
A/B Test ROI Calculator
What a lift is worth once you account for the traffic it applies to.
Revenue Per Visitor Calculator
RPV for each variant, which catches the case where conversion rate rose and revenue did not.
Funnel Conversion Calculator
Drop off at every step, so you can see which stage the change actually moved.
Bayesian A/B Test Calculator, Posterior Distributions & Probability of Winning
Free Bayesian A/B test calculator with posterior Beta distribution visualization. Compute probability of winning, expected loss, credible intervals, and risk analysis for your experiments.
Conversion Rate Calculator with Confidence Intervals
Free conversion rate calculator with Wilson, Clopper-Pearson, and Normal confidence intervals. Compare up to 4 variants, visualize error bars, test significance between pairs, and project revenue impact.
Statistical Power Analysis Calculator for A/B Tests
Free power analysis calculator for A/B testing. Compute required sample size, visualize power curves, support one-tailed and two-tailed tests, and calculate Cohen's h effect size.
Chi-Square Test Calculator, Statistical Independence Testing
Free chi-square test of independence calculator with dynamic 2×2 to 5×5 frequency tables. Computes chi-square statistic, degrees of freedom, p-value, Cramér's V, and visualizes observed vs expected counts with critical region shading.
Sequential Testing Tool, Stop A/B Tests Early With Confidence
Free sequential testing tool using O'Brien-Fleming boundaries. Input daily visitors, baseline conversion rate, and MDE to generate a day-by-day stopping schedule. Overlay live test results to see whether you can stop your A/B test today.