unsloth/Qwen3.8-27B-NVFP4

completed

Request throughput: 0.41 req/s
Output throughput: 105 tok/s
p95 TTFT: 400.4 ms
p95 TPOT: 37.96 ms

Manifest warnings

Review these published caveats before interpreting or comparing the metrics.

Review imported facts against the local model, hardware, launch command, and benchmark command before publication.

Server launch command

CUDA_VISIBLE_DEVICES=0 vllm serve unsloth/Qwen3.8-27B-NVFP4 \
  --revision 7d6f8d4d72f56b92b3cdbf22f156b90e1bab0108 \
  --port 9000 \
  --host 127.0.0.1 \
  --dtype bfloat16 \
  --gpu-memory-utilization 0.92 \
  --max-model-len 9216 \
  --max-num-batched-tokens 16384 \
  --max-num-seqs 32 \
  --tensor-parallel-size 1 \
  --pipeline-parallel-size 1 \
  --kv-cache-dtype fp8 \
  --distributed-executor-backend mp \
  --no-enable-prefix-caching \
  --enable-chunked-prefill \
  --no-async-scheduling \
  --enforce-eager \
  --served-model-name unsloth/Qwen3.8-27B-NVFP4 \
  --disable-uvicorn-access-log \
  --language-model-only

Run details

Workload: 1024 → 256
Methodology: closed loop · streaming
Test load: 4 concurrent requests
Endpoint: openai-compatible-completions
Requests: 16 unmeasured warmups per invocation after one readiness request.
Dataset: sha256=74b57f701750018dce349c2ec7ae008a4de70ab161e515359e7ae76b31d90338 generator=runpile-sweeps-tokenizer-verified-jsonl synthetic=true generator_revision=d93886d7ee93eb0634b5f426ff480ce0a5798ad447d080e9c42a0d330a8bd055 shared_prefix_tokens=0
Sampling: mode=greedy temperature=0
Stopping: ignore_eos=true max_output_tokens=256
Prefix and cache: cache_state=warm-host cache_enabled=false shared_prefix_tokens=0
Client: name=vllm bench serve backend=vllm version=0.11.2.dev280+gilded.gnosis.v20.vllm4d006a4.b12xcd3ce19.fi1ac6942.cu132.20260810.r34 placement=same-host
Hardware: 1× NVIDIA GeForce RTX 5090
Runtime: vllm 0.11.2.dev280+gilded.gnosis.v20.vllm4d006a4.b12xcd3ce19.fi1ac6942.cu132.20260810.r34
Runtime configuration: dtype=bfloat16 max_num_seqs=32 enforce_eager=true max_model_len=9216 kv_cache_dtype=fp8 prefix_caching=false chunked_prefill=true expert_parallel=false async_scheduling=false gpu_memory_utilization=0.92 max_num_batched_tokens=16384
Measurement: vllm bench serve 0.11.2.dev280+gilded.gnosis.v20.vllm4d006a4.b12xcd3ce19.fi1ac6942.cu132.20260810.r34
Timing: client-observed streaming request latency · client overhead included
Repetitions: 1
Failure handling: Failed requests counted; no client retries.
Quantization: format=NVFP4 mixed precision recipe=Unsloth Dynamic V3.0 preview checkpoint=compressed-tensors
Model revision: 7d6f8d4d72f56b92b3cdbf22f156b90e1bab0108
Evidence: Raw benchmark evidence attached · eligible and ranked
Published: 8/29/2026, 8:46:23 AM by anonymous-EVjB2S

Benchmark command

vllm bench serve \
  --backend vllm \
  --base-url http://127.0.0.1:9000 \
  --endpoint /v1/completions \
  --model unsloth/Qwen3.8-27B-NVFP4 \
  --tokenizer unsloth/Qwen3.8-27B-NVFP4 \
  --dataset-name custom \
  --dataset-path <EXACT_SYNTHETIC_DATASET> \
  --custom-output-len 256 \
  --skip-chat-template \
  --disable-shuffle \
  --num-prompts 256 \
  --max-concurrency 4 \
  --request-rate inf \
  --seed 0 \
  --ignore-eos \
  --temperature 0 \
  --percentile-metrics ttft,tpot,itl,e2el \
  --metric-percentiles 50,90,95,99 \
  --disable-tqdm \
  --save-result \
  --result-dir <local-path> \
  --result-filename qwen38-27b-nvfp4-gg-r34-tp1__baseline__serve-fixed-1k-256-v1__c4__r1__attempt1.json \
  --num-warmups 16 \
  --metadata campaign=runpile-rtx5090-qwen38-exl3-pilot-20260828 config_id=qwen38-27b-nvfp4-gg-r34-tp1 server_variant=baseline workload_key=serve-fixed-1k-256-v1 repetition=1 aggregate_only=true