Llama 3.1 8B Instruct · one GPU per deployment

Find a measured GPU result that fits your limits.

Explore how prompt length, concurrent requests and latency limits change the historical cost comparison.

Your workload

Llama 3.1 8B Instruct · bfloat16 · vLLM 0.30.0

Choose a measured provider/GPU as a baseline. We compare its historical benchmark, not measurements of your own deployment.

Optional limits

The wait before generation starts. We use the highest run-level p95 across three runs.

Generation latency limits

No prompts, files, account or provider credentials required. Comparisons run in your browser. Your workload settings are not sent to a server.

Choose a workload to begin

Compare five measured provider/GPU deployments across six workload settings.

What these results can tell you

These are synthetic random-token workloads with three runs per result: 64 requests per run at concurrency 8, or 128 at concurrency 32. A run-level p95 is not a production guarantee or a confidence interval. Aggregate output throughput differs from an individual user's generation speed.

Historical token costs exclude idle time and other operating costs. The standard protocol and separately reviewed historical evidence remain labelled. Immutable model/tokenizer revisions, regions and complete startup settings remain unverified. Check representative traffic, current prices and availability before choosing infrastructure.

Methodology and limitations · Suggest future research