Results
Every number here was measured by Inferoptimal on the hardware named; nothing is an estimate.
Inferoptimal Recipes
Loading…
Best run per model and hardware
| Model | Hardware | $/h | Workload | Load | Output tok/s | First token | Per user | $/1M output | Date |
|---|---|---|---|---|---|---|---|---|---|
| Loading… | |||||||||
The highest output tok/s run for each model, hardware and workload kind. First token is the median, p90 on the second line. Per user is output tokens per second for one user or agent.
All runs
| Run | Load | Date | Output tok/s | TTFT | Per user | Also measured |
|---|---|---|---|---|---|---|
| Loading… | ||||||
TTFT is the median unless marked p50 or p99. Per user is output tokens per second for one user or agent, as recorded. No row on this page is an estimate.