meta-models/Muse-Glimmer-30B-GGUF

Hardware
1× NVIDIA GeForce RTX 5090
Draft model
meta-models/Muse-Glimmer-30B-GGUF
Workload
256 → 1024
Runs
10 eligible
Efficient frontierOther eligible run
p95 TTFT (ms)Request throughput (req/s)0865.701run_hnVplgclHiGtCpSg: 1.36 req/s, p95 TTFT 665.8 msrun_7J2w_Aw4ho6I1yWw: 1.03 req/s, p95 TTFT 671.4 msrun_ZR_RAzllpLkB8sYn: 1.38 req/s, p95 TTFT 642 msrun_vPfuBmCI0zxI4Crr: 1.13 req/s, p95 TTFT 620.9 msrun_CiulJ-cHIZuzzW_R: 0.62 req/s, p95 TTFT 88 msrun_xb2pOY5Lgv5-fgCG: 1.36 req/s, p95 TTFT 865.7 msrun_aXhViDsvAnJIxxJj: 1.09 req/s, p95 TTFT 656.3 msrun_Y576sLBflzGiCVyS: 1.37 req/s, p95 TTFT 659.5 msrun_Ox5bxidX1u7dS0UN: 1.12 req/s, p95 TTFT 673.8 msrun_p8OFEpVTMDQna7Ad: 0.62 req/s, p95 TTFT 88.5 ms

Runs

All published runs with the same model build, accelerator setup, and request shape. Benchmark methodology remains attached to each run.

RuntimeMaterial parametersTest loadReq/sOutput tok/sp95 TTFTp95 TPOTp95 E2EFailuresRun
llama.cpp b10689Speculative · Q4_K_Mspeculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=81,397.1665.8 ms5.83 ms6,430.9 ms0run_hnVplgclHiGtCpSg
llama.cpp b10689Speculative · Q4_K_Mspeculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=81,054.5671.4 ms8.15 ms8,530.4 ms0run_7J2w_Aw4ho6I1yWw
llama.cpp b10689Speculative · Q4_K_Mspeculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=81,417.9642 ms5.82 ms6,320.8 ms0run_ZR_RAzllpLkB8sYn
llama.cpp b10689Speculative · Q4_K_Mspeculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=81,156.9620.9 ms6.74 ms7,309 ms0run_vPfuBmCI0zxI4Crr
llama.cpp b10689Speculative · Q4_K_Mspeculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=1637.388 ms1.5 ms1,621.2 ms0run_CiulJ-cHIZuzzW_R
llama.cpp b10689Speculative · Q4_K_Mspeculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=81,396.8865.7 ms5.91 ms6,535.4 ms0run_xb2pOY5Lgv5-fgCG
llama.cpp b10689Speculative · Q4_K_Mspeculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=81,117.5656.3 ms7.03 ms7,554.3 ms0run_aXhViDsvAnJIxxJj
llama.cpp b10689Speculative · Q4_K_Mspeculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=81,403.9659.5 ms5.9 ms6,434 ms0run_Y576sLBflzGiCVyS
llama.cpp b10689Speculative · Q4_K_Mspeculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=81,144.5673.8 ms6.87 ms7,449.2 ms0run_Ox5bxidX1u7dS0UN
llama.cpp b10689Speculative · Q4_K_Mspeculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384c=1635.188.5 ms1.51 ms1,628.4 ms0run_p8OFEpVTMDQna7Ad
Runtime
llama.cpp b10689Speculative · Q4_K_M draft
Material parameters
speculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=8
Req/s
Output tok/s
1,397.1
p95 TTFT / TPOT
665.8 ms / 5.83 ms
Runtime
llama.cpp b10689Speculative · Q4_K_M draft
Material parameters
speculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=8
Req/s
Output tok/s
1,054.5
p95 TTFT / TPOT
671.4 ms / 8.15 ms
Runtime
llama.cpp b10689Speculative · Q4_K_M draft
Material parameters
speculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=8
Req/s
Output tok/s
1,417.9
p95 TTFT / TPOT
642 ms / 5.82 ms
Runtime
llama.cpp b10689Speculative · Q4_K_M draft
Material parameters
speculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=8
Req/s
Output tok/s
1,156.9
p95 TTFT / TPOT
620.9 ms / 6.74 ms
Runtime
llama.cpp b10689Speculative · Q4_K_M draft
Material parameters
speculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=1
Req/s
Output tok/s
637.3
p95 TTFT / TPOT
88 ms / 1.5 ms
Runtime
llama.cpp b10689Speculative · Q4_K_M draft
Material parameters
speculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=8
Req/s
Output tok/s
1,396.8
p95 TTFT / TPOT
865.7 ms / 5.91 ms
Runtime
llama.cpp b10689Speculative · Q4_K_M draft
Material parameters
speculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=8
Req/s
Output tok/s
1,117.5
p95 TTFT / TPOT
656.3 ms / 7.03 ms
Runtime
llama.cpp b10689Speculative · Q4_K_M draft
Material parameters
speculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=8
Req/s
Output tok/s
1,403.9
p95 TTFT / TPOT
659.5 ms / 5.9 ms
Runtime
llama.cpp b10689Speculative · Q4_K_M draft
Material parameters
speculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=8
Req/s
Output tok/s
1,144.5
p95 TTFT / TPOT
673.8 ms / 6.87 ms
Runtime
llama.cpp b10689Speculative · Q4_K_M draft
Material parameters
speculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Test load
c=1
Req/s
Output tok/s
635.1
p95 TTFT / TPOT
88.5 ms / 1.51 ms