Reference serving workloads
These optional, versioned profiles make independently published runs easier to compare without preventing custom workloads. They describe online serving—not isolated prefill or decode microbenchmarks.
Machine-readable profile catalog
| Profile | Input tokens | Output tokens | Recommended concurrency | Intent |
|---|---|---|---|---|
serve-fixed-1k-256-v1 | 1,024 | 256 | 1, 4, 8, 16, 32, 64 | Balanced serving |
serve-fixed-8k-256-v1 | 8,192 | 256 | 1, 2, 4, 8, 16, 32 | Long prompt |
serve-fixed-256-1k-v1 | 256 | 1,024 | 1, 2, 4, 8, 16, 32 | Long generation |
Copy the selected manifest_fragment, set one recommended concurrency value, and record the profile ID in workload.extensions["runpile.profile_id"]. Publish each concurrency as a separate run and include every completed measured point—not only winners or representative points. The profile fixes endpoint, streaming, token shapes, synthetic generator semantics, sampling, EOS behavior, cache/prefix state, client placement, warm-up, and failure treatment.
Harness and runtime versions, model identity, hardware, launch command, benchmark command, and material runtime parameters remain facts of the individual run. Do not claim a profile when the executed command differs from its fragment.