Short generation
1,000 input tokens · 250 output tokens · concurrency 8
Five measured deployment configurations.
Measured GPU evidence
AI inference GPU benchmarks
Explore measured GPU performance for model-specific inference workloads.
Our initial public dataset covers Llama 3.1 8B Instruct across five provider/GPU configurations and six exact workload shapes.
Available now: historical research using synthetic prompts, with sample sizes and repeat ranges. This is not a live GPU recommendation service.
Available workload comparisons
Each linked workload compares five measured provider/GPU configurations. Compare historical output-token costs alongside throughput, latency and how each result was measured.
Short generation
1,000 input tokens · 250 output tokens · concurrency 32
Five measured deployment configurations.
Decode-heavy generation
512 input tokens · 2,048 output tokens · concurrency 8
Five measured deployment configurations.
Decode-heavy generation
512 input tokens · 2,048 output tokens · concurrency 32
Five measured deployment configurations.
Long-context prefill
8,192 input tokens · 256 output tokens · concurrency 8
Five measured deployment configurations.
Long-context prefill
8,192 input tokens · 256 output tokens · concurrency 32
Five measured deployment configurations.
A40 or H100 for Llama 3.1 8B?
Compare all six matched RunPod workloads to see when a higher hourly rate meant lower token cost, and how first-token latency differed.
Understanding inference costs
Hourly GPU rental price alone does not establish which deployment is cheapest. Effective output throughput, request mix, latency constraints and utilisation can materially change the result.