No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from autotrust/GLM-5.3-Flash-GGUF-DGX-Spark
All models by autotrust →GLM 5.3 Flash, 12 % smaller — Sparse-Squared compression with AutoTrust SLIM-Q, resident on a single 128 GB DGX Spark with room to spare.
This repository is the llama.cpp GGUF build of the SLIM-Q-compressed GLM 5.3 Flash: 79.1 GiB instead of the stock 90 GiB Q2, with 32 of 288 routed experts removed per layer (256 retained — the E256 member of the SLIM-Q family) and everything else untouched. It is the single-Spark companion to GLM-5.3-GGUF-DGX-Spark, the 744B SLIM-Q E192 build whose NVFP4 edition powers Guru Turbo 2.0. Runs in llama.cpp (glm5-next), so you get the OpenAI-compatible llama-server, continuous batching, tool calling and reasoning_content on the Spark.
SLIM-Q — Selective expert pruning + Low-bit quantization for Inference of MoE — is AutoTrust's two-stage pipeline for post-training compression of frontier open-source MoE models. The idea in one line is Sparse-Squared (Sparse²) compression: a MoE is already sparse at inference (each token activates only the top-8 of 288 routed experts); SLIM-Q adds a second, structural sparsity axis — permanently removing the experts the router rarely uses — and then drives the surviving experts to low precision.
Stage 1 — Selective expert pruning (SLIM). Expert selection was profiled on a bilingual code / agent / science / maths calibration mix, and the 32 least-used of 288 routed experts per layer (11 %) were structurally removed. Attention (KDA + DSA), dense layers, shared experts, router, tokenizer and chat template are unchanged. Unlike dynamic expert-skipping, structural pruning permanently shrinks the weight footprint — on a 128 GB Spark, that is the difference between ~25 GiB and ~40 GiB of free unified memory after weights, i.e. between a 16–32 K context and a comfortable 64 K (or 4 × 16 K sessions).
Stage 2 — Low-bit quantization (Q). The pruned checkpoint is quantized aggressively on the routed experts (IQ2_XXS gate/up, Q2_K down) while attention, dense layers, shared experts, and routers stay at high precision — preserving the routing behavior that MoE quality depends on.
As on the 744B build, the per-token cost is untouched: routing still selects the top-8 of the remaining 256 experts, so compute and memory traffic per token equal the stock Flash Q2. Only the footprint drops.
The model is stored as two GGUF shards (Hugging Face's 50 GB per-file limit); llama.cpp loads them together when pointed at the first one: GLM-5.3-Flash-Q2-DGX-Spark-00001-of-00002.gguf (41.9 GiB) + -00002-of-00002.gguf (37.2 GiB), 79.1 GiB total. Checksums in GLM-5.3-Flash-Q2-DGX-Spark.sha256.
GLM-5.3-Flash support is in llama.cpp pull request #27773 (glm5-next; not merged at the time of writing — once it is, plain master works).
-c 65536 leaves ~30 GiB free; -np 4 -c 65536 (4 × 16 K) is a good multi-user setting.<think>). --reasoning-budget 0 disables it, --reasoning-budget 4096 caps it, --chat-template-kwargs '{"reasoning_effort":"low"}' selects the template's low / high / max effort. The server returns thinking separately as reasoning_content; tool calls come back as OpenAI tool_calls.cmake -B build without CUDA), discrete NVIDIA GPUs with ≥ 90 GB, or partially offloaded (-ngl N) on smaller cards.llama.cpp, CUDA, one B200, model resident (llama-bench / llama-batched-bench, 256-token prompts, 128 generated tokens, flash attention):
SLIM stage (4-bit build with the same 256-expert selection, under vLLM on one B200): HumanEval 97.6 · C-Eval 89.4 · GPQA-Diamond 77.3 (low effort) · AIME 2025 74.2 · MMMU 76.1 · BFCL Non-Live 87.7 / Live 80.5 / multi-turn 73–75 — the pruning itself costs little.
Q stage (this 2-bit file) — DwarfStar's ds4-eval harness (GPQA Diamond, SuperGPQA, AIME 2025 interleaved; thinking on, 16 000-token budget, greedy), first 40 core cases, compared with the 2-bit 744B SLIM-Q E192 build on the same cases:
An imatrix-guided build of the same layout scored 31 / 40 with twice the reasoning tokens, so this weight-energy-importance build is the one published. Qualitative checks: correct bilingual common-sense, medical and coding answers; no Chinese–English mixing.
Choosing between the two Spark builds: this Flash build closes more reasoning cases within budget and fits one Spark; the 744B E192 build keeps a clear edge on language modelling of agent/SWE trajectories and code (held-out PPL 6.2 vs 12.4 on agent traces, 3.1 vs 4.6 on SWE traces, 2.96 vs 3.17 on code) and needs two Sparks, a 180 GB GPU, or a big Mac.
45 layers (3 dense + 42 MoE) · KDA linear attention with DSA every fourth layer (k-pool indexer) · hyper-connections · top-8 of 256 routed experts + 1 shared · 154 880-token vocabulary · GLM 5.3 chat template with tool calling. Plain llama.cpp GGUF (architecture glm5-next). No MTP block; text model (no vision projector).
Total 84,929,450,144 bytes (79.1 GiB). Split with llama-gguf-split --split-max-size 45G; to get a single file: llama-gguf-split --merge GLM-5.3-Flash-Q2-DGX-Spark-00001-of-00002.gguf out.gguf.
llamacpp-pr27773-glm5next-lenient.patch (in the tooling repository): tolerate u64 metadata, and return a reply verbatim instead of HTTP 500 when max_tokens cuts it mid-UTF-8-character. Neither is required to run the model.zai-org/GLM-5.3-Flash.hf download autotrust/GLM-5.3-Flash-GGUF-DGX-Spark --local-dir ./GLM-5.3-Flash-GGUF-DGX-Sparkgit clone https://github.com/ggml-org/llama.cpp.git && cd llama.cppgit fetch origin pull/27773/head:glm5next && git checkout glm5nextcmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=121a-real # GB10cmake --build build --config Release -j# OpenAI-compatible API on :8080, 4 sessions x 16K, continuous batching./build/bin/llama-server -m GLM-5.3-Flash-Q2-DGX-Spark-00001-of-00002.gguf -ngl 99 -fa on \ -c 65536 -np 4 --cont-batching --host 0.0.0.0 --port 8080
# single long-context chat./build/bin/llama-cli -m GLM-5.3-Flash-Q2-DGX-Spark-00001-of-00002.gguf -ngl 99 -fa on -c 65536