meta-models/Muse-Glimmer-30B-GGUF

Hardware
1× NVIDIA GeForce RTX 5090
Workload
1024 → 256
Runs
6 eligible
Efficient frontierOther eligible run
p95 TTFT (ms)Request throughput (req/s)0238.900run_xeZCvl-F7Z91PF5F: 0.27 req/s, p95 TTFT 236.6 msrun_Xq38zSjCEejpe_o9: 0.27 req/s, p95 TTFT 236.9 msrun_SkmcydlPgUKIcyOP: 0.26 req/s, p95 TTFT 236 msrun_54iivsKDUO1loCa-: 0.26 req/s, p95 TTFT 237.6 msrun_bjWHIS3zOJ1j2k0O: 0.26 req/s, p95 TTFT 238.9 msrun_KvSGyKKp7eLzit8C: 0.26 req/s, p95 TTFT 238.9 ms

Runs

All published runs with the same model build, accelerator setup, and request shape. Benchmark methodology remains attached to each run.

RuntimeMaterial parametersTest loadReq/sOutput tok/sp95 TTFTp95 TPOTp95 E2EFailuresRun
llama.cpp b10689ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=167.8236.6 ms13.88 ms3,777.1 ms0run_xeZCvl-F7Z91PF5F
llama.cpp b10689ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=167.9236.9 ms13.88 ms3,777.7 ms0run_Xq38zSjCEejpe_o9
llama.cpp b10689ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=167.7236 ms13.96 ms3,794.9 ms0run_SkmcydlPgUKIcyOP
llama.cpp b10689ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=167.7237.6 ms13.9 ms3,786.2 ms0run_54iivsKDUO1loCa-
llama.cpp b10689ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=167.8238.9 ms13.89 ms3,783.3 ms0run_bjWHIS3zOJ1j2k0O
llama.cpp b10689ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=167.6238.9 ms13.97 ms3,800.5 ms0run_KvSGyKKp7eLzit8C
Runtime
llama.cpp b10689
Material parameters
ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=1
Req/s
Output tok/s
67.8
p95 TTFT / TPOT
236.6 ms / 13.88 ms
Runtime
llama.cpp b10689
Material parameters
ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=1
Req/s
Output tok/s
67.9
p95 TTFT / TPOT
236.9 ms / 13.88 ms
Runtime
llama.cpp b10689
Material parameters
ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=1
Req/s
Output tok/s
67.7
p95 TTFT / TPOT
236 ms / 13.96 ms
Runtime
llama.cpp b10689
Material parameters
ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=1
Req/s
Output tok/s
67.7
p95 TTFT / TPOT
237.6 ms / 13.9 ms
Runtime
llama.cpp b10689
Material parameters
ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=1
Req/s
Output tok/s
67.8
p95 TTFT / TPOT
238.9 ms / 13.89 ms
Runtime
llama.cpp b10689
Material parameters
ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=1
Req/s
Output tok/s
67.6
p95 TTFT / TPOT
238.9 ms / 13.97 ms