Qwen3.6-35B-A3B-GGUF
GGUF conversions of Qwen/Qwen3.6-35B-A3B for llama.cpp.
Sparse MoE: 35B total / 3B active (256 experts + 1 shared, top-8 routed). Hybrid Gated-DeltaNet + Gated-Attention layers (3:1), 262K native context. Includes vision projector for image/video input.
Files
Benchmarks (RTX 3090, bs=1, llama.cpp build b1-94ca829)
Q5_K_M / Q8_0 / BF16 do not fit a single 24GB GPU at usable context.
Usage
Text
Vision (image/video)
Notes
- Architecture:
Qwen3_5MoeForConditionalGeneration (qwen3_5_moe)
- Converter: llama.cpp
convert_hf_to_gguf.py (built-in support)
- At bs=1 decode, the GPU is kernel-launch bound; prefill or batched serving will show higher utilization
- For long context (>262K), see base model's YaRN config