No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from AtomicChat/Ling-3.0-flash-GGUF
All models by AtomicChat →Quantizations of inclusionAI/Ling-3.0-flash:
124B total, 5.1B active, hybrid linear attention (35 KDA blocks interleaved 5:1 with 7 gated MLA
blocks) over a 512-expert MoE.
Bits are placed by hand rather than by the default rules, and the controls that prove it is worth
something are published next to the files. At the same size, our layout sits 31 to 41 % closer to
BF16 than what llama-quantize produces on its own
These files need a TurboQuant build.
bailingmoe3is in upstream llama.cpp, but we have some important bugfixes to it. Nothing has to be compiled, see Run it.
Weights and context share your memory, so leave headroom below the number in the first column. Every rung of the ladder is in the full table.
Files without the AD- prefix are controls, published so the claim above can be checked. They
are not meant to be used: *_STOCK is what llama.cpp picks by itself, *_FLAT is our bit budget
with the differentiation switched off.
Grab the archive for your machine from release
b10269-1.5.1
or newer.
The chat template ships inside the GGUF, thinking mode and tool calling included. Sampling
recommended by the authors: temperature 0.6, top_p 0.95, top_k 20.
Intel GPUs are the one gap: there is no SYCL archive yet, that path still needs a source build.
AD meansAtomic Dynamic: the bits are placed deliberately, along three axes.
ffn_gate_inp) and the expert bias stay F32, because an error
there changes which expert runs instead of degrading its output. Attention, the KDA gates and
the shared expert stay Q8_0. output stays F16, it feeds the logits directly.down_proj gets more bits than gate/up, it is the more
sensitive half of the SwiGLU.Those rows split the win. Most of it comes from refusing to quantize the 3 % of the weights that are not experts. The per-projection and per-depth differentiation inside the experts adds the remaining 4.3 % on top.
Two builds, both 72.3 GB, ->safetensors for vLLM here<- and ->GGUF here<-.
Same format and same block layout in both. The AD build differs in one thing: the scale of each block is chosen by sweeping the neighbouring UE4M3 codes, laying the weights on the E2M1 grid for each candidate and scoring the error weighted by the importance matrix, with the same convention the k-quants use.
Worth knowing before you download: at this size a k-quant rung is much closer to BF16
(AD-Q4_K_S, 74.2 GB, KL 0.0318). NVFP4 buys native FP4 tensor cores on Blackwell, not accuracy.
All numbers are measured against the BF16 baseline on held-out text that never entered the
calibration corpus, on identical hardware (4x RTX PRO 6000 Blackwell). Raw logs and json:
AtomicChat/Ling-3.0-flash-GGUF-metrics.
Two pairs sit at the same size on purpose. At 79 GB, AD-Q4_K_M is better in the tail and
AD-IQ4_NL in the mean. At 74 GB, AD-Q4_K_S is better in the mean and smaller, AD-IQ4_XS
better in the tail and in top-1. Pick by the metric you care about.
Sizes are GB, 10^9 bytes. llama.cpp prints GiB, so AD-Q5_K_M shows up there as 83.3 GiB.
Rung names follow the community convention, not the upstream preset list: IQ4_XXS, Q5_K_L and
Q4_K_L are our mixes and you will not find them in llama-quantize.
The base is a bit-exact BF16 conversion: 877 of 917 tensors are byte-identical to the safetensors checkpoint, the remaining 40 are the MoE routers, stored as F32 instead of BF16, which is a lossless widening (max absolute difference 0.0).
The new architecture was checked layer by layer against the HuggingFace reference before any quant was produced. Over a fixed 32-token forward, the cosine similarity of the first block output is 0.99999 and the mean KL over the vocabulary is 4.8e-4, which is the noise floor between the reference GPU kernels and the llama.cpp CPU path.
The importance matrix was collected on the BF16 model, not on a quantized proxy, over 522
chunks of 4096 tokens from AtomicChat/calib-corpora.
Which tensor gets what:
4x RTX PRO 6000 Blackwell (96 GB each), full offload, llama-bench:
Consumer cards and Apple silicon will be added as those runs happen. Comparing across different GPUs is not meaningful, so every figure says which machine it came from.
GGUF runs natively on Apple silicon through Metal, MLX is not required:
A 128 GB Mac Studio fits AD-Q5_K_M (89 GB) comfortably. AD-Q6_K (107 GB) needs the wired memory
limit raised and leaves little room for context.
wget https://github.com/AtomicBot-ai/atomic-llama-cpp-turboquant/releases/download/b10269-1.5.0/llama-turboquant-linux-x64-cuda-13.3.tar.gztar xzf llama-turboquant-linux-x64-cuda-13.3.tar.gz && cd llama-turboquant-* ./llama-cli -m AD-Q5_K_M/Ling-3.0-flash-AD-Q5_K_M-00001-of-00002.gguf --jinja -ngl 99 -c 32768./llama-cli -m AD-Q5_K_M/Ling-3.0-flash-AD-Q5_K_M-00001-of-00002.gguf --jinja -ngl 99 -c 32768