Get a correct standard error and confidence interval for ratio metrics like revenue per visitor or clicks per session — the ones a naive two-sample t-test silently gets wrong.
When your randomization unit (the user, session, or cluster you flip on/off) differs from the analysis unit that appears in your ratio's denominator, treating the ratio as a simple mean of independent observations underestimates the variance. This tool applies the delta method to combine the numerator variance, denominator variance, and their covariance into a valid variance for the ratio, then reports the point estimate, standard error, z-based confidence interval, and — if you enter two arms — the two-sided p-value for the difference.
Var/Cov are the per-unit (population-style) second moments across your randomization units. In raw mode the tool derives them from your sums using Var = Σx²/n − (Σx/n)² and Cov = Σxy/n − x̄ȳ. Standard error scales as √(Var(R)/n).
A ratio metric is R = mean(numerator) / mean(denominator). Think revenue-per-visitor, where revenue is the numerator summed per user and visits (or pageviews, sessions) is the denominator. Because both the top and bottom are random and correlated, you cannot treat R as a simple sample mean. The first-order Taylor expansion (the delta method) around the means gives:
The three terms matter: Var(x) is the numerator spread, R²·Var(y) inflates variance when the denominator itself is noisy, and the −2·R·Cov(x,y) term subtracts shared movement — when heavy users generate both more revenue and more visits, that positive covariance reduces the true uncertainty. Drop the covariance term and you typically over-state the interval; treat the ratio as an i.i.d. mean of per-event rows and you under-state it, because per-event rows within a user are not independent.
clicks/impressions across all impression rows and running a t-test on that pooled rate. Impressions from the same randomized user are correlated, so the reported p-value is far too small and teams ship losers. Aggregating to the randomization unit and applying the delta method restores the correct, usually wider, interval.For two arms the tool treats the arms as independent, so Var(R_B − R_A) = Var(R_A) + Var(R_B), forms z = (R_B − R_A) / SE_diff, and returns the two-sided normal p-value. Enter Units n = 0 for Treatment to get a single-arm interval only. Everything runs locally in your browser — no data leaves the page.