toolbox
Home / AI, Data & Technology / AI Translation GPU Requirements Estimator

AI Translation GPU Requirements Estimator

This estimator sizes a self-hosted translation deployment by comparing required token processing with measured tokens-per-second capacity per GPU. It adjusts for batching efficiency, sustainable utilization, retry overhead, and redundancy. The calculation is useful for early infrastructure planning and hardware comparisons. Reliable results depend on benchmarks from the exact model, quantization, context length, language mix, and GPU type intended for production.

Translation GPU sizing assumptions

tasks/hr
tokens
tok/s
%
%
%
%
Result
—
Recommended translation GPU count
Base GPUs before reserve—
Effective token demand per second—
Usable tokens per second per GPU—
Capacity with recommended GPUs—

1. Enter hourly task demand

Use successful translation tasks expected during the peak sustained hour.

2. Estimate tokens per attempt

Include source, output, and repeated instruction tokens processed by the model.

3. Use a production-like benchmark

Enter measured tokens per second for one GPU under the target model and precision.

4. Adjust batching and utilization

Apply observed batching efficiency and a sustainable utilization target.

5. Add retries and reserve

Include repeated model calls and spare capacity for failures or maintenance.

Effective token demand per second = Tasks per hour × Tokens per attempt × (1 + Retry rate ÷ 100) ÷ 3,600 Usable capacity per GPU = Measured tokens per second × Batching efficiency × Utilization Base GPUs = Effective token demand ÷ Usable capacity per GPU Recommended GPUs = Base GPUs × (1 + Reserve ÷ 100), rounded up

What the result means

The recommendation is the whole-GPU count needed to process the stated token workload with the selected operational margins.

Memory fit, context-length limits, prompt and decode speed differences, and model loading time require separate verification.

Given: 12,000 tasks per hour, 900 tokens per attempt, 3,500 tokens per second per GPU, 90% batching efficiency, 70% utilization, 5% retries, and 20% reserve.

Calculation: Demand = 12,000 × 900 × 1.05 ÷ 3,600 = 3,150 tokens per second. Usable capacity per GPU = 3,500 × 0.90 × 0.70 = 2,205. Base GPUs = 3,150 ÷ 2,205 = 1.429. With reserve = 1.714.

Result: Round up to 2 GPUs.

Should prompt and completion tokens have the same speed?

Not necessarily. Prefill and decoding often run at different rates. Use a blended benchmark from representative translation requests or model the phases separately.

Can the batching efficiency exceed 100%?

The field permits it for normalized benchmark comparisons, but most deployments should use 100% or less relative to the stated per-GPU benchmark.

How do long documents affect GPU sizing?

Long contexts may lower throughput and require more memory. Benchmark with the actual chunking and context distribution.

Does reserve capacity replace high availability design?

No. It adds numerical capacity but does not ensure replica placement, failover, networking, or state management.

When should API translation be preferred?

This calculator only estimates self-hosted GPU quantity. A decision should also compare total cost, quality, privacy, operations effort, and scaling flexibility.