toolbox
Home / AI, Data & Technology / Bayesian Experiment Confidence Interval Calculator

Bayesian Experiment Confidence Interval Calculator

The Bayesian Experiment Confidence Interval Calculator estimates an approximate Bayesian credible interval for a binary experiment rate after combining observed outcomes with a Beta prior. It is useful for product experiments, model evaluations, conversion tests, or any repeated trial where each observation is classified as a success or failure and you want uncertainty expressed from a Bayesian posterior.

Enter the number of successes and total trials, then choose prior Beta parameters and a credible level. The calculator reports the posterior mean, posterior standard deviation, and lower and upper bounds using a normal approximation to the Beta posterior. The interval can help you judge how tightly the experiment rate is estimated, compare posterior uncertainty across experiments, or decide whether more observations are needed. For small samples or highly skewed posteriors, an exact Beta quantile method can produce different bounds, so this page clearly treats the result as an approximation.

Posterior interval inputs

Result
—
Approximate posterior credible interval width
Posterior mean—
Lower bound—
Upper bound—
Posterior SD—

1. Enter experiment outcomes
Provide the number of observed successes and the total number of binary trials. Successes cannot exceed total trials.

2. Set the prior
Enter positive Beta prior parameters alpha and beta. A value of 1 for each gives a uniform prior over the underlying success rate.

3. Choose a credible level
Select 90%, 95%, or 99%. Higher credible levels produce wider intervals because they cover more posterior probability under the approximation.

4. Review the interval
Use the lower and upper bounds with the posterior mean. The displayed main result is the total width between those bounds.

5. Interpret with sample size in mind
A narrow interval indicates less posterior uncertainty. With sparse or extreme data, remember that the normal approximation may be less accurate than exact Beta quantiles.

Posterior alpha = prior alpha + successes Posterior beta = prior beta + failures Posterior mean = alpha / (alpha + beta) Posterior SD = sqrt[alpha × beta / ((alpha + beta)^2 × (alpha + beta + 1))] Approx. interval = posterior mean ± z × posterior SD

Failures equal total trials minus successes. The z value is 1.645 for 90%, 1.960 for 95%, and 2.576 for 99%. Bounds are clipped to the valid probability range from 0 to 1. This calculator uses a normal approximation to the Beta posterior rather than exact Beta distribution quantiles.

What the result means

The interval gives a posterior uncertainty range around the experiment success rate under the selected prior and approximation. A smaller width means the posterior is more concentrated.

This is an analytical approximation for planning and interpretation, not a substitute for exact posterior quantiles when tails are strongly asymmetric.

Given: 70 successes in 100 trials, Beta(1, 1) prior, 95% credible level.

Calculation: Posterior alpha = 1 + 70 = 71. Posterior beta = 1 + 30 = 31. Posterior mean = 71 / 102 = 0.6961. Posterior SD ≈ 0.04535. Approximate bounds = 0.6961 ± 1.96 × 0.04535 = 0.6072 to 0.7850.

Result: Approximate interval width ≈ 17.78 percentage points.

The posterior centers near 69.61%, with most of the selected approximate credible mass spanning roughly 60.72% to 78.50%.

Why is this called a credible interval rather than a confidence interval?

In Bayesian analysis, the interval is derived from the posterior distribution after combining a prior with observed data. The interpretation is therefore posterior-probability based, although this page uses the familiar “confidence interval” wording from the tool title.

What do prior alpha and beta represent?

They set the shape and strength of the Beta prior for the unknown success rate. Larger values contribute more prior information, while Beta(1, 1) is a common weak uniform starting point.

Can I use this for very small experiments?

You can enter small samples, but the normal approximation can be poor when the posterior is strongly skewed or concentrated near 0 or 1. Exact Beta quantiles are preferable when tail accuracy is important.

Why are the bounds clipped at 0% and 100%?

A probability cannot be below 0 or above 1. The underlying normal approximation can extend beyond that range, so the displayed bounds are restricted to valid probabilities.

How should I compare two Bayesian experiments?

Compare posterior means and uncertainty ranges, but do not treat overlapping or non-overlapping intervals as a complete decision rule. A direct posterior comparison of the treatment difference is more informative when available.