toolbox
Home / AI, Data & Technology / Bayesian Experiment Sample Size Estimator

Bayesian Experiment Sample Size Estimator

Estimate a practical per-variant sample size for a Bayesian binary experiment by targeting the posterior precision of a conversion rate. The calculator combines a Beta prior with an expected conversion rate and finds the smallest sample for which a normal approximation to the posterior credible interval reaches the requested half-width. Bayesian experiment design does not have one universal sample-size definition. This page therefore uses an explicit precision-based criterion rather than claiming a fixed analogue of frequentist power. It is useful when your planning question is how much data is needed for the posterior rate estimate to become sufficiently narrow under a chosen prior.

Inputs

%
α
β
%
pp
Result
—
estimated sample per variant
Prior effective sample—
Target half-width—
Expected posterior mean—

1. Enter the expected rate
Use the conversion rate around which you want to plan posterior precision.

2. Define the Beta prior
Enter prior alpha and beta shape parameters; Beta(1,1) is a common weak uniform prior.

3. Choose credible level
Set the central posterior credible level you want to approximate.

4. Set target precision
Enter the desired half-width in percentage points.

5. Review the sample estimate
Treat the result as a precision-based planning number and consider simulation for decision-specific Bayesian designs.

Find smallest n such that z × √[m(1−m)/(α+β+n+1)] ≤ target half-width

Using expected data, posterior alpha = prior α + n×p and posterior beta = prior β + n×(1−p). The posterior mean m equals posterior alpha divided by the total posterior shape, and z is the normal quantile for the chosen central credible level.

This is a Beta-Binomial planning approximation for posterior precision, not a universal Bayesian sample-size rule. Strong priors, small samples, extreme rates, decision losses, Bayes-factor criteria, or probability-of-superiority goals may call for simulation or a different design criterion.

What the result means

Use the main result together with the supporting statistics and the stated assumptions; it is a planning estimate rather than a guarantee.

Keep units and the unit of analysis consistent. Recalculate when traffic patterns, rates, priors, sample sizes, or design assumptions change.

Given

  • Expected conversion: 10%
  • Beta(1,1) prior
  • Credible level: 95%
  • Target half-width: 1 percentage point

Calculation
The calculator increases n until the approximate posterior 95% half-width is no more than 0.01. With a weak Beta(1,1) prior and a 10% expected rate, the required n is roughly in the mid-thousands.

Result
Approximately 3,500 observations per variant

Interpretation
At about this sample size, the modeled posterior conversion-rate interval is expected to be about ±1 percentage point under the stated prior and expected rate.

Why is there no power input?

Bayesian designs can be planned around posterior precision, probability of superiority, expected loss, Bayes factors, or other criteria. This calculator specifically targets posterior credible-interval width.

What does Beta(1,1) mean?

It is a uniform prior over conversion probabilities from 0 to 1 and contributes two shape units in the posterior. Other alpha and beta values can represent stronger prior information.

Does a stronger prior always reduce required sample size?

A more concentrated prior can narrow the posterior faster, but only if that prior is appropriate for the experiment. Overly strong or misspecified priors can dominate the data.

Why is the credible interval only approximate?

The calculator uses a normal approximation to the Beta posterior for fast planning. Exact Beta quantiles can differ when the posterior is highly skewed or based on little information.

Is the result total sample or per variant?

It is a per-variant precision estimate for one binary rate. Comparing two variants may require applying the criterion to both groups or using a direct posterior-difference simulation.