meta-models/Muse-Glimmer-30B-GGUF

Hardware
1× NVIDIA GeForce RTX 5090
Draft model
meta-models/Muse-Glimmer-30B-GGUF
Workload
1024 → 256
Runs
8 eligible
Efficient frontierOther eligible run
p95 TTFT (ms)Request throughput (req/s)02295.502run_jpAO6AqnPV8fIE6G: 2.2 req/s, p95 TTFT 2,295.5 msrun_8p6kxiiQ0v4I6779: 1.58 req/s, p95 TTFT 291.4 msrun_n1slR5fu_FGDsXy1: 1.58 req/s, p95 TTFT 284.8 msrun_Cv7M7AziqHXBXxd_: 1.59 req/s, p95 TTFT 284 msrun_KnGBYhrH8L-i8lm6: 2.22 req/s, p95 TTFT 2,266.6 msrun_Y5Axw5x4__ZQ5beL: 1.58 req/s, p95 TTFT 288.1 msrun_ZezDHb_l9UhnlgfB: 1.58 req/s, p95 TTFT 281.4 msrun_LbJk_5I94aFDjfWG: 1.58 req/s, p95 TTFT 289.2 ms

Runs

All published runs with the same model build, accelerator setup, and request shape. Benchmark methodology remains attached to each run.

RuntimeMaterial parametersTest loadReq/sOutput tok/sp95 TTFTp95 TPOTp95 E2EFailuresRun
llama.cpp b10689Speculative · Q4_K_Mspeculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=8563.32,295.5 ms13.26 ms3,982.8 ms0run_jpAO6AqnPV8fIE6G
llama.cpp b10689Speculative · Q4_K_Mspeculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=1403.2291.4 ms1.39 ms642.3 ms0run_8p6kxiiQ0v4I6779
llama.cpp b10689Speculative · Q4_K_Mspeculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=1403.8284.8 ms1.39 ms639.1 ms0run_n1slR5fu_FGDsXy1
llama.cpp b10689Speculative · Q4_K_Mspeculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=1405.8284 ms1.38 ms634.5 ms0run_Cv7M7AziqHXBXxd_
llama.cpp b10689Speculative · Q4_K_Mspeculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=8567.82,266.6 ms13.31 ms4,039 ms0run_KnGBYhrH8L-i8lm6
llama.cpp b10689Speculative · Q4_K_Mspeculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=1404.8288.1 ms1.39 ms640.6 ms0run_Y5Axw5x4__ZQ5beL
llama.cpp b10689Speculative · Q4_K_Mspeculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=1405.5281.4 ms1.39 ms635.4 ms0run_ZezDHb_l9UhnlgfB
llama.cpp b10689Speculative · Q4_K_Mspeculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=1405.1289.2 ms1.4 ms644.4 ms0run_LbJk_5I94aFDjfWG
Runtime
llama.cpp b10689Speculative · Q4_K_M draft
Material parameters
speculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=8
Req/s
Output tok/s
563.3
p95 TTFT / TPOT
2,295.5 ms / 13.26 ms
Runtime
llama.cpp b10689Speculative · Q4_K_M draft
Material parameters
speculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=1
Req/s
Output tok/s
403.2
p95 TTFT / TPOT
291.4 ms / 1.39 ms
Runtime
llama.cpp b10689Speculative · Q4_K_M draft
Material parameters
speculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=1
Req/s
Output tok/s
403.8
p95 TTFT / TPOT
284.8 ms / 1.39 ms
Runtime
llama.cpp b10689Speculative · Q4_K_M draft
Material parameters
speculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=1
Req/s
Output tok/s
405.8
p95 TTFT / TPOT
284 ms / 1.38 ms
Runtime
llama.cpp b10689Speculative · Q4_K_M draft
Material parameters
speculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=8
Req/s
Output tok/s
567.8
p95 TTFT / TPOT
2,266.6 ms / 13.31 ms
Runtime
llama.cpp b10689Speculative · Q4_K_M draft
Material parameters
speculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=1
Req/s
Output tok/s
404.8
p95 TTFT / TPOT
288.1 ms / 1.39 ms
Runtime
llama.cpp b10689Speculative · Q4_K_M draft
Material parameters
speculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=1
Req/s
Output tok/s
405.5
p95 TTFT / TPOT
281.4 ms / 1.39 ms
Runtime
llama.cpp b10689Speculative · Q4_K_M draft
Material parameters
speculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=1
Req/s
Output tok/s
405.1
p95 TTFT / TPOT
289.2 ms / 1.4 ms