meta-models/Muse-Glimmer-30B-GGUF

Hardware
1× NVIDIA GeForce RTX 5090
Workload
8192 → 256
Runs
8 eligible
Efficient frontierOther eligible run
p95 TTFT (ms)Request throughput (req/s)013584.400run_-AoXomjXA_6Kr_y-: 0.24 req/s, p95 TTFT 13,360.5 msrun_xhcGxbxHYf8WtTP7: 0.35 req/s, p95 TTFT 13,328.5 msrun_ahWgiU9gPnz73uld: 0.24 req/s, p95 TTFT 13,584.4 msrun_4f7EU1vdpbn8ordT: 0.18 req/s, p95 TTFT 1,949.5 msrun_JSRIEh1jrPcKuFkn: 0.24 req/s, p95 TTFT 13,465.6 msrun_7_d8G2wKpJgSGnpv: 0.35 req/s, p95 TTFT 13,423.1 msrun_R1OPtU9JPkrCQc9P: 0.25 req/s, p95 TTFT 13,382.1 msrun_WcI4LvRTOfkvJVbX: 0.18 req/s, p95 TTFT 1,969.4 ms

Runs

All published runs with the same model build, accelerator setup, and request shape. Benchmark methodology remains attached to each run.

RuntimeMaterial parametersTest loadReq/sOutput tok/sp95 TTFTp95 TPOTp95 E2EFailuresRun
llama.cpp b10689ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=862.213,360.5 ms126.8 ms36,696 ms0run_-AoXomjXA_6Kr_y-
llama.cpp b10689ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=890.313,328.5 ms87.77 ms26,694.7 ms0run_xhcGxbxHYf8WtTP7
llama.cpp b10689ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=862.113,584.4 ms126.57 ms38,272.3 ms0run_ahWgiU9gPnz73uld
llama.cpp b10689ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=146.11,949.5 ms14.2 ms5,564 ms0run_4f7EU1vdpbn8ordT
llama.cpp b10689ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=861.813,465.6 ms127.5 ms36,877.7 ms0run_JSRIEh1jrPcKuFkn
llama.cpp b10689ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=889.713,423.1 ms88.34 ms26,903.4 ms0run_7_d8G2wKpJgSGnpv
llama.cpp b10689ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=865.213,382.1 ms115.08 ms35,036.1 ms0run_R1OPtU9JPkrCQc9P
llama.cpp b10689ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=145.91,969.4 ms14.21 ms5,585.4 ms0run_WcI4LvRTOfkvJVbX
Runtime
llama.cpp b10689
Material parameters
ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=8
Req/s
Output tok/s
62.2
p95 TTFT / TPOT
13,360.5 ms / 126.8 ms
Runtime
llama.cpp b10689
Material parameters
ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=8
Req/s
Output tok/s
90.3
p95 TTFT / TPOT
13,328.5 ms / 87.77 ms
Runtime
llama.cpp b10689
Material parameters
ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=8
Req/s
Output tok/s
62.1
p95 TTFT / TPOT
13,584.4 ms / 126.57 ms
Runtime
llama.cpp b10689
Material parameters
ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=1
Req/s
Output tok/s
46.1
p95 TTFT / TPOT
1,949.5 ms / 14.2 ms
Runtime
llama.cpp b10689
Material parameters
ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=8
Req/s
Output tok/s
61.8
p95 TTFT / TPOT
13,465.6 ms / 127.5 ms
Runtime
llama.cpp b10689
Material parameters
ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=8
Req/s
Output tok/s
89.7
p95 TTFT / TPOT
13,423.1 ms / 88.34 ms
Runtime
llama.cpp b10689
Material parameters
ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=8
Req/s
Output tok/s
65.2
p95 TTFT / TPOT
13,382.1 ms / 115.08 ms
Runtime
llama.cpp b10689
Material parameters
ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=1
Req/s
Output tok/s
45.9
p95 TTFT / TPOT
1,969.4 ms / 14.21 ms