No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from RedHatAI/DeepSeek-V4-Flash-NVFP4-FP8
All models by RedHatAI →This model was obtained by using the following branch with LLM Compressor: https://github.com/vllm-project/llm-compressor/pull/2647
This model was deployed using the following branch with vLLM: https://github.com/vllm-project/vllm/pull/41276
This model has a noticably lower accuracy recovery than the base model due to the base model being released in a quantized format and differences between mxfp4 and nvfp4. More advanced techniques such as GPTQ can be used to increase accuracy recovery beyond this model's current state.
For more details on how this model was created and run in LLM Compressor, please contact Kyle Sayers on the vLLM Slack: https://communityinviter.com/apps/vllm-dev/join-vllm-developers-slack
To run this model in vllm, install the following:
Evaluation performed on 8×B200 GPUs using vLLM with FP8 KV cache. Scores are averaged across multiple seeds (3 seeds for most benchmarks, 8 for AIME 2025). Instruct benchmarks run with reasoning OFF (nonthinking mode); Reasoning and Coding benchmarks run with reasoning ON (thinking mode).
vllm serve RedHatAI/DeepSeek-V4-Flash-NVFP4-FP8 --tensor-parallel-size 4 --port 8089 --kv_cache_dtype="fp8"python tests/evals/gsm8k/gsm8k_eval.pyResults:Accuracy: 0.910Invalid responses: 0.000Total latency: 173.006 sQuestions per second: 7.624Total output tokens: 116217Output tokens per second: 671.752python3 tests/evals/mmlu_pro/mmlu_pro_eval.py --port 8089Results:Category: allAccuracy: 0.554Invalid responses: 0.000Total latency: 112.065 sQuestions per second: 107.366Total output tokens: 24076Output tokens per second: 214.840uv pip install git+https://github.com/vllm-project/vllm.git@refs/pull/41276/head --no-cacheuv pip install tilelang==0.1.10 apache-tvm-ffi==0.1.10