Dedicated GPU deployment

Qwen3.8 Flash Next FP8

Deploy Qwen3.8 Flash Next FP8 on four dedicated RTX PRO 6000 GPUs with pre-downloaded weights. This template accepts text; image and video inputs are disabled.

Deploy with template

Measured by EcoHash

RTX PRO 6000 benchmark

Results from the saved EcoHash run for this model configuration. The template may be updated independently; the methodology below states the tested settings.

Time to first token: 99 ms
Single-request output speed: 193.4 tokens/s
Aggregate throughput: 181.8 tokens/s
End-to-end latency P50: 2.15 s
End-to-end latency P95: 5.33 s
Warm start: 255.9 s
Largest tested concurrency: 1
Throughput: 23.9 outputs/min
Success rate: 100%
Failures: 0
Out-of-memory failures: 0
GPU memory · loaded / idle: 373.8 GB
GPU memory · peak workload: 373.8 GB
Host RAM · peak: 287.5 GB
Weight storage: 185.6 GB
Cache footprint (downloads + runtime): 0 GB
Output storage: 0 GB

Test conditions

4 × NVIDIA RTX PRO 6000 Blackwell Server Edition, 96 GB each; PCIeATLvLLM 0.32.0rc1.dev100+g77154ca61; PyTorch 2.13.0+cu130; TP4+EP; MTP 3Official FP8 weights; BF16 SSM state524,288 configured with YaRN factor 2; speed test used short prompts1 main; separate C=4 check2026-10-15

Main test: complete-answer C=1 run from the bastion through kubectl port-forward; public API and gateway excluded. TTFT p95 100.09 ms; TPOT p50/p95 5.17/5.99 ms/token. Output TPM 10,905.34 excludes input tokens. Cached-weight launch to Ready 255.87 s; cold-cache startup was not measured. Separate C=4 short workload: aggregate 357.83 and 363.78 tokens/s in two runs; concurrent coding-batch TTFT p50 was 1,513.87 ms then 125.04 ms, cause not identified. Separate 500K multi-turn check: cold-prefix TTFT 35.784 s; continuation TTFT 1.312/1.286 s; 3/3 retrieval checks. Coding probe: 10 samples passed Python AST syntax checks; no formal quality score. Seed not fixed.

Full benchmark report

Published template

Deployment configuration

Configuration from the published platform template. Review the template for current settings, availability, model terms and pricing.

GPU: 4 × NVIDIA RTX PRO 6000 Blackwell Server Edition
Engine: vLLM
Deployment: Dedicated model instance
Billing: GPU-hour · current price shown in the template

Deployment options

Opening a template does not start a paid instance. Review its configuration and price before launching.

Model TemplateDeploy Qwen3.8 Flash Next FP8 on 4 dedicated RTX PRO 6000 GPUs

Open the platform template to review its image, startup command, GPU configuration and current price before launching.

Deploy with template