toolbox
Home / AI, Data & Technology / AI Translation Throughput Estimator

AI Translation Throughput Estimator

This estimator predicts how many successful translation tasks an existing GPU deployment can process per minute and per hour. It converts measured token throughput into task throughput after batching, utilization, and retry effects are applied. It supports capacity checks for real-time queues and offline localization batches. The calculator is model-agnostic, so the benchmark and tokens-per-task assumptions must represent the actual language mix, prompt format, and output lengths used in production.

Translation throughput inputs

GPUs
tok/s
tokens
%
%
%
tasks
Result
—
Estimated successful translation throughput per hour
Successful tasks per minute—
Usable token capacity—
Estimated backlog completion time—
Tasks per hour lost to retries—

1. Enter active GPUs

Count GPUs available to the translation service during the planning window.

2. Provide measured token speed

Use a benchmark from the same model, precision, and request profile.

3. Estimate tokens per attempt

Include source, generated output, and recurring prompt context.

4. Apply efficiency and utilization

Reflect batching behavior and sustainable operating load.

5. Include retries and backlog

Account for repeated calls and optionally estimate how long a queue will take to clear.

Usable tokens per second = GPUs × Tokens per second per GPU × Batching efficiency × Utilization Attempts per hour = Usable tokens per second × 3,600 ÷ Tokens per attempt Successful tasks per hour = Attempts per hour ÷ (1 + Retry rate ÷ 100) Backlog hours = Backlog tasks ÷ Successful tasks per hour

What the result means

The main result is sustained successful task output, not raw model calls, after retry demand is included.

If prompt processing and output generation have materially different speeds, use a blended measured rate from representative requests.

Given: four GPUs, 3,200 tokens per second per GPU, 1,000 tokens per attempt, 88% batching efficiency, 72% utilization, 4% retries, and a 100,000-task backlog.

Calculation: Usable token capacity = 4 × 3,200 × 0.88 × 0.72 = 8,110.08 tokens per second. Attempts per hour = 8,110.08 × 3,600 ÷ 1,000 = 29,196.288. Successful tasks = 29,196.288 ÷ 1.04 = 28,073.35 per hour.

Result: About 28,073.35 tasks per hour. The backlog would clear in approximately 3.56 hours.

Why is tokens per task based on an attempt?

Retries repeat some or all model processing. Defining tokens per attempt lets the retry factor convert raw attempt capacity into successful task capacity.

Should idle autoscaled GPUs be counted?

Count only GPUs expected to be active and usable during the sustained processing window.

How can mixed document sizes be modeled?

Use a weighted average tokens-per-attempt value or calculate throughput separately for size bands.

What if the backlog is zero?

Throughput still calculates, and backlog completion time is displayed as zero hours.

How does this differ from latency capacity?

Throughput estimates total output over time. Latency capacity estimates how many simultaneous slots are needed to sustain a rate at a given response time.