Question 1

What does P(B > A) mean?

Accepted Answer

The probability that the treatment conversion rate is higher than the control, given the data and prior.

Question 2

How are credible intervals computed?

Accepted Answer

Using the Beta quantile function via Newton's method on the incomplete beta integral (regularized incomplete beta).

Question 3

How does prior choice affect results?

Accepted Answer

Skeptical priors (e.g., Beta(10,10)) pull estimates toward 0.5, reducing P(B>A). Optimistic or flat priors let data dominate. Use prior sensitivity to compare.

Variant	Posterior α	Posterior β	Conv. rate (mean)	95% CI (low)	95% CI (high)
Control (A)	-	-	-	-	-
Treatment (B)	-	-	-	-	-

Bayesian A/B Test

🅰 Control

🅱 Treatment