toolbox
Home / Legal, Safety & Compliance / AI Governance Audit Sample Size Estimator

AI Governance Audit Sample Size Estimator

The AI Governance Audit Sample Size Estimator calculates a statistically based sample for testing a finite population of AI governance records or control instances. Examples include model inventory entries, approvals, monitoring checks, human-oversight evidence, vendor assessments, or logged decisions where the audit objective is to estimate a proportion across a larger population.

The calculation uses confidence level, margin of error, expected exception rate, and finite population size. It provides a quantitative starting point for a random-sampling plan, not a legal minimum and not a substitute for risk-based testing. High-impact systems, known control failures, unusual vendors, or sensitive use cases may warrant targeted review even when they fall outside a random sample. The audit team should match the sampling method to the conclusion it intends to draw.

Calculator inputs

items
%
%
%
Result
—
Recommended AI governance audit sample
Recommended sample—
Population—
Confidence level—
Margin of error—

1. Identify the audit population

Enter the number of governance items that are in scope for the period and control being tested.

2. Choose the confidence level

Select the statistical confidence desired for a proportion estimate.

3. Set the margin of error

Enter the precision you need in percentage points. Smaller margins require larger samples.

4. Enter an expected exception rate

Use historical results or a planning assumption. If you have no basis, 50% yields the most conservative sample for this formula.

5. Pair statistical and risk-based testing

Use the result for random selection, then separately consider targeted testing for high-risk AI systems or known issues.

Initial sample n₀ = z² × p × (1 − p) ÷ e²; Finite sample n = n₀ ÷ (1 + (n₀ − 1) ÷ N)

Where:

  • z: standard-normal score for the selected confidence level
  • p: expected exception proportion as a decimal
  • e: margin of error as a decimal
  • N: population size
  • n: finite-population sample size rounded up

Assumptions: The result applies to a simple random sample used to estimate a proportion. It does not account for correlated records, control-frequency differences, severity weighting, or legal requirements to test specific AI systems.

What the result means

The result is the rounded-up simple-random sample size for the entered finite population and proportion assumptions.

Use professional judgment to add targeted testing for high-risk systems, known issues, or control areas not represented by a random sample.

Given:

  • Population: 1,200 governance records
  • Confidence: 95%
  • Margin of error: 4%
  • Expected exception rate: 15%

Calculation:

z ≈ 1.960

n₀ = 1.960² × 0.15 × 0.85 ÷ 0.04² ≈ 306.1

Finite correction: 306.1 ÷ (1 + 305.1 ÷ 1,200) ≈ 244.0

Round up to 245 items

Result: Recommended random sample: 245 governance items.

Interpretation: Under the assumptions, the sample is designed to estimate the 15% proportion within roughly ±4 percentage points at 95% confidence.

Is this sample size required by an AI regulation?

No. It is a general statistical estimate. Any regulatory, certification, internal-audit, or assurance requirement should be checked separately.

What should count as one population item?

Use a consistent unit tied to the audit objective, such as one system review, one approval, one monitoring event, or one record. Mixing unlike units makes the statistical interpretation less clear.

Should high-risk AI systems be sampled randomly?

Random sampling can support population-level estimates, but high-risk systems may also require targeted or complete testing based on risk and applicable obligations.

How does the expected exception rate affect sample size?

Rates closer to 50% create more statistical variability and generally require larger samples. Very low or high expected rates can reduce the calculated sample for the same confidence and margin.

Can I use the result to conclude every control is effective?

No. A sample supports conclusions only about the defined population, period, attribute, and sampling method. Different controls or populations may need separate tests.