No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from RedHatAI/Kimi-K3-FP8-BLOCK
All models by RedHatAI →This is a variant of moonshotai/Kimi-K3 quantized with FP8_BLOCK quantization applied to MLP and experts layers. Attention layers were not quantized due to shape mismatches, namely with self_attn.b_proj.weight
Quantized using llm-compressor:
docker run --gpus all \ --privileged --ipc=host -p 8000:8000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ -e GLOO_SOCKET_IFNAME=$IFACE_NAME \ -e NCCL_SOCKET_IFNAME=$IFACE_NAME \ -e VLLM_ENABLE_K3_LATENT_MOE_TAIL_FUSION=1 \ vllm/vllm-openai:kimi-k3 RedHatAI/Kimi-K3-FP8-BLOCK \ --trust-remote-code \ --load-format fastsafetensors \ --gpu-memory-utilization 0.95 \ --tensor-parallel-size 16 \ --nnodes 2 \ --node-rank 0 \ --master-addr $HEAD_IP \ --no-enable-flashinfer-autotune \ --disable-custom-all-reduce \ --enable-auto-tool-choice \ --tool-call-parser kimi_k3 \ --reasoning-parser kimi_k3from compressed_tensors.entrypoints.convert import CompressedTensorsDequantizer
from llmcompressor import model_free_ptq
MODEL_ID = "moonshotai/Kimi-K3"SAVE_DIR = MODEL_ID.rstrip("/").split("/")[-1] + "-FP8-BLOCK"
ignore = [ "re:.*embed_tokens.*", "re:.*self_attn.*", # no attention because (q_proj|k_proj|v_proj|b_proj|f_a_proj) are all fused, and `b_proj` has a weight non-divisible by 128 "re:.*block_sparse_moe\.gate.*", "re:.*self_attention_res_proj.*", "re:.*mlp_res_proj.*", "re:.*output_attn_res_proj.*", "re:.*lm_head.*", "re:.*vision_tower.*", "re:.*mm_projector.*",]
model_free_ptq( model_stub=MODEL_ID, save_directory=SAVE_DIR, scheme="FP8_BLOCK", ignore=ignore, converter=CompressedTensorsDequantizer( MODEL_ID, ignore=ignore, ), max_workers=7, device=[ f"cuda:{i}" for i in range(7) ],)