No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from AtomicChat/Muse-Glimmer-30B-GGUF
All models by AtomicChat →GGUF builds of Meta's Muse Glimmer 30B, made from the original BF16 weights with a calibrated importance matrix and a bit layout chosen by measurement.
[!NOTE] Support was merged into llama.cpp master in #26841 (commit
62bf73d). You need a build from master — the last tagged release predates it.
[!TIP]
AD-Q6_Kat 28.3 GB is dominated by plainQ8_0at 29.6 GB, which is 1.3 GB larger and 31 % more accurate. TakeQ8_0instead unless you need exactly that size.
imatrix/ holds the importance matrix, layouts/ the per-tensor type map of every
file, experiments/ the control builds. All logs:
AtomicChat/Muse-Glimmer-30B-GGUF-metrics.
AD-IQ4_XS plus the vision projector plus the speculative drafter fits in 24.6 GB.
Full multimodal, tool calling, and 1.8x faster generation on a single consumer card.
Measured on one RTX 5090 at -c 16384 -fa on, nine prompts across code, prose and
reasoning, greedy sampling.
[!NOTE] Speculative decoding costs nothing in quality. The drafter only proposes tokens; the full model verifies every one of them, and under greedy sampling the output is identical to running without it. What it costs is 5.5 GB of VRAM and 18 % of prompt processing speed, since prefill runs through both models and gains nothing from speculation.
Build from master:
[!WARNING]
--jinjais required. Without it neither the chat template nor tool-call parsing works.
With speculative decoding:
Reasoning depth is set by a line in the system prompt, not a flag:
Reasoning strength: low|medium|high|xhigh. Default is high — at that setting short
answers can spend their entire token budget inside the reasoning block, so raise
max_tokens or drop to low for quick replies.
Reference is our own BF16 GGUF, verified bit-identical to the original weights by UUID and all 731 tensor hashes. Two corpora: neutral text in 30 languages, and agentic dialogue in the model's native chat markup with tool calls. Lower KLD is better.
[!NOTE] The same file scores an order of magnitude higher KLD on agentic dialogue than on neutral text. Agentic text is far more predictable (base PPL 2.45 vs 5.46), so the model is very confident and small probability shifts produce large divergence. A KLD number without its corpus is meaningless.
Twelve builds from another publisher were downloaded and measured against the same BF16 base, with the same binary, on the same hardware.
Below 16 GB our curve sits below theirs everywhere, by 12 % to 41 % depending on the size and corpus. Two of our files win outright — smaller and more accurate:
Above 19 GB theirs is better, by 13 % to 18 %. Their high rungs raise token_embd
and leave the FFN at the base type; ours does more or less the opposite. The layout
study below explains why that is the right call at 2–4 bits and the wrong one at 6–8.
The first release raised attn_k/v to q8_0, attn_q/output/gate one step, and both
token_embd and output, leaving the FFN at the base type. It beat the matching
llama.cpp preset at every rung — 22 % lower KLD at Q4_K_M — but below 15 GB it sat
above the preset curve once size was accounted for.
The reason is arithmetic. FFN is 20.7 of 27.9 billion parameters, 74 % of the model. The whole extra budget had gone to the other 26 %.
Seven builds from one IQ3_XXS base, each differing by exactly one override. Recovered
KL divergence divided by added size gives a price per gigabyte.
output is the worst investment in the model and the first layout raised it at every
rung. An earlier 8-bit study had suggested it mattered for top-1 agreement; at 8 bits
everything is saturated and that signal was noise.
attn_k and attn_v are the bargain. GQA 32/2 makes them 6656 x 256 — together 0.64 %
of the weights. Holding both at q8_0 costs 110 MB.
That table is measured at three bits, and it does not transfer upward. At 8 bits the
same experiment gives the opposite answer for token_embd: raising it to BF16 takes
Q8_0 from 0.000861 to 0.000838 on neutral text and from 0.019968 to 0.018397 on
agentic — a 7.9 % improvement, the largest single lever at that size.
The mechanism is straightforward. At low bit widths the FFN is coarse and every bit
spent there pays. At high bit widths it is saturated, and the residual error collects in
token_embd — the one tensor that is quantized blind, because it is a row lookup rather
than a matrix multiply and the importance collector never sees it.
So the recipe is not one rule across the ladder:
attn_k/v at
q8_0, leave everything else alone.token_embd one step.Base type and FFN type move together: IQ2_XXS/iq2_s, IQ2_M/iq3_xxs, IQ3_XXS/iq3_s, IQ3_M/iq4_xs, IQ4_XS/q4_K, Q4_K_M/q5_K + embd q6_K, Q5_K_M/q6_K + embd q8_0, Q6_K/q8_0 + embd q8_0.
Per-layer bit allocation. Importance concentration in ffn_down varies enormously
with depth — ten channels out of 19 968 hold 1.6 % of the total at layer 6 and 60 % at
layer 42. Raising only the concentrated layers 36–47 versus raising all 52 uniformly
gives builds within 1.6 % of each other, in both directions depending on corpus.
Indistinguishable. Control published as experiments/…-IQ4_XS_FLAT.gguf.
Beating the presets on layout alone. The control builds carry the same importance matrix as ours and differ only in layout. Against them our advantage is inconsistent — better at some sizes, worse at others. Our lead over the other publisher comes largely from the calibration, not from the bit layout. The controls are published so anyone can check that.
imatrix/muse-glimmer.imatrix.gguf is published so anyone can rebuild these files or
build their own on the same calibration.
An importance matrix records, for every weight matrix, the mean squared activation arriving on each of its input channels. The quantizer uses those numbers to weight reconstruction error when it searches for block scales: error on a channel that usually carries large activations is penalized more than error on a channel usually near zero.
Both flags are off by default and both matter. Without --parse-special the chat
control tokens are tokenized as literal punctuation, so 40 % of the corpus would be
calibrated on sequences the model never produces. Without --process-output,
output.weight — 1.34 B parameters sitting directly under the logit softcap — gets no
importance data at all.
Muse Glimmer has 52 layers with 8 collected matrices each, plus the output matrix, for
417 importance vectors. They reduce to five distinct profiles per layer: attn_q,
attn_k, attn_v and attn_gate all read the output of attn_norm, so their vectors
are identical to the last digit; ffn_gate and ffn_up likewise share ffn_pre_norm.
These groups cannot be told apart by importance, only by cost — which is why attn_k/v
are raised and attn_q/gate are not.
Y lists the five profiles; X is the share of a tensor's importance held by its 10
strongest input channels and by its strongest 1 %, averaged over 52 layers. Ten channels
out of 6656 would hold 0.15 % under a uniform distribution; on the attention input they
hold 38.8 %, a factor of 259. For ffn_down the ten strongest of 19 968 hold 19.8 %
against a uniform 0.05 %, a factor of 396.
X is the layer index, Y the share of ffn_down importance in its 10 strongest channels.
Layers 5–11 sit between 1.6 % and 3.6 %; layers 41 and 42 reach 59 % and 60 %.
K-quants assign one scale to a block of 32 adjacent weights along the input dimension. A single dominant channel forces that scale upward and costs precision on the other 31, so concentration makes a block harder to quantize, not easier. It marks where calibration has the most leverage — but as the per-layer control above shows, that does not translate into a usable rule for moving bits between layers.
One note on attn_gate. Gated attention is new to this architecture and adds a
6656 x 4096 matrix per layer, roughly a third of the attention parameters. No published
quantization work has had importance data for it. Its profile is identical to attn_q,
attn_k and attn_v, so it cannot be distinguished from them by importance alone.
--no-ppl is required at long context: the perplexity path allocates
n_ctx x n_vocab x 4 bytes for logits, which is 26.5 GB at 32768 context with this
model's 202 048 token vocabulary. Perplexity is not used by importance collection, so
disabling it costs nothing.
Merging is exact addition of sums and counts, so the corpus can be split across machines and recombined:
Corpus: AtomicChat/calib-corpora.
Per-tensor statistics: imatrix/imatrix-stats.txt in the metrics dataset.
Model by Meta Superintelligence Lab. llama.cpp support by @pcuenca and the ggml-org reviewers.
git clone https://github.com/ggml-org/llama.cpp && cd llama.cppcmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ONcmake --build build -j --target llama-server llama-mtmd-clillama-server -m Muse-Glimmer-30B-AD-IQ4_XS.gguf \ --mmproj mmproj-Muse-Glimmer-30B-BF16.gguf \ --jinja -ngl 99 -c 16384 -fa on \ --temp 1.0 --top-p 0.95 --top-k 64llama-server -m Muse-Glimmer-30B-AD-IQ4_XS.gguf \ -md dflash-Muse-Glimmer-30B-BF16.gguf \ --spec-type draft-dflash --spec-draft-n-max 15 \ --mmproj mmproj-Muse-Glimmer-30B-BF16.gguf \ --jinja -ngl 99 -c 16384 -fa onllama-quantize --imatrix muse-glimmer.imatrix.gguf \ --tensor-type attn_k=q8_0 --tensor-type attn_v=q8_0 \ --tensor-type ffn_gate=q4_K --tensor-type ffn_up=q4_K --tensor-type ffn_down=q4_K \ Muse-Glimmer-30B-BF16-00001-of-00002.gguf out.gguf IQ4_XS 96llama-imatrix -m Muse-Glimmer-30B-BF16-00001-of-00002.gguf \ -f calib_train.txt -o part.gguf \ -c 8192 -b 2048 -ngl 99 \ --parse-special --process-output --no-pplllama-imatrix -m any-model.gguf -c 8192 \ --in-file part-00.gguf,part-01.gguf,part-long.gguf \ -o muse-glimmer.imatrix.gguf