meta-models/Muse-Glimmer-30B-GGUF

completed

Request throughput: 0.38 req/s
Output throughput: 97.5 tok/s
p95 TTFT: 15979.9 ms
p95 TPOT: 72.56 ms

Manifest warnings

Review these published caveats before interpreting or comparing the metrics.

Review imported facts against the local model, hardware, launch command, and benchmark command before publication.

Server launch command

CUDA_VISIBLE_DEVICES=0 <local-path> serve \
  --model meta-models/Muse-Glimmer-30B-GGUF \
  --alias meta-models/Muse-Glimmer-30B-GGUF \
  --host 127.0.0.1 \
  --port 9000 \
  --ctx-size 147456 \
  --parallel 16 \
  --batch-size 16384 \
  --ubatch-size 512 \
  --n-gpu-layers all \
  --flash-attn on \
  --jinja \
  --metrics \
  --no-webui \
  --no-cache-prompt \
  --cache-ram 0 \
  --no-cache-idle-slots \
  --ctx-checkpoints 0 \
  --model-draft meta-models/Muse-Glimmer-30B-GGUF \
  --spec-type draft-dflash \
  --spec-draft-n-max 15 \
  --n-gpu-layers-draft all

Run details

Workload: 8192 → 256
Methodology: closed loop · streaming
Test load: 8 concurrent requests
Endpoint: openai-compatible-completions
Requests: 4 unmeasured warmups per invocation after one readiness request.
Dataset: sha256=71babd85f3e4e0cd5d7ba181261c5472fed89ae35546d0fc0c820eabfbb8e595 generator=runpile-sweeps-tokenizer-verified-jsonl synthetic=true generator_revision=a3d215d3624741ac0031b7b6ec24bc4600b7dadd2dcdf9ba58f18e56ea0e3b64 shared_prefix_tokens=0
Sampling: mode=sampling top_k=64 top_p=0.95 temperature=1
Stopping: ignore_eos=true max_output_tokens=256
Prefix and cache: cache_state=warm-host cache_enabled=false shared_prefix_tokens=0
Client: name=vllm bench serve backend=vllm version=0.28.0 placement=same-host
Hardware: 1× NVIDIA GeForce RTX 5090
Runtime: llama.cpp b10689
Decoding method: Speculative · Q4_K_M draft
Draft model: meta-models/Muse-Glimmer-30B-GGUF · Q4_K_Mrevision 70bf1b61ac09f91b24d39038091b41c582bc5d7a
Runtime configuration: speculative.type=draft-dflash speculative.n_max=15 ubatch_size=512 context_size=147456 max_num_seqs=16 flash_attention=on weight_artifact=Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf max_model_len_per_slot=9216 max_num_batched_tokens=16384
Measurement: vllm bench serve 0.28.0
Timing: client-observed streaming request latency · client overhead included
Repetitions: 1
Failure handling: Failed requests counted; no client retries.
Quantization: format=KQuant Dynamic Q4_K_XL target plus Q4_K_M DFlash
Model revision: 70bf1b61ac09f91b24d39038091b41c582bc5d7a
Evidence: Raw benchmark evidence attached · eligible and ranked
Published: 8/31/2026, 12:53:59 AM by anonymous-EVjB2S

Benchmark command

vllm bench serve \
  --backend vllm \
  --base-url http://127.0.0.1:9000 \
  --endpoint /v1/completions \
  --model meta-models/Muse-Glimmer-30B-GGUF \
  --tokenizer meta-models/Muse-Glimmer-30B \
  --dataset-name custom \
  --dataset-path <EXACT_SYNTHETIC_DATASET> \
  --custom-output-len 256 \
  --skip-chat-template \
  --disable-shuffle \
  --num-prompts 64 \
  --max-concurrency 8 \
  --request-rate inf \
  --seed 0 \
  --ignore-eos \
  --temperature 1.0 \
  --top-p 0.95 \
  --top-k 64 \
  --percentile-metrics ttft,tpot,itl,e2el \
  --metric-percentiles 50,90,95,99 \
  --disable-tqdm \
  --save-result \
  --result-dir <local-path> \
  --result-filename glimmer30b-dynq4-dflash15-gpu0__baseline__serve-fixed-8k-256-v1__c8__r3__attempt1.json \
  --num-warmups 4 \
  --metadata campaign=runpile-rtx5090-glimmer-gguf-dflash15-20260831-v3 config_id=glimmer30b-dynq4-dflash15-gpu0 server_variant=baseline workload_key=serve-fixed-8k-256-v1 repetition=3 aggregate_only=true