Cataloged from Blackfrost-AI/Muse-Glimmer-30B-Abliterated-GGUF
[!IMPORTANT]
Improvement update — August 15, 2026
This release now includes compact abliterated Q4_K_M companions: a 1.63 GB DFlash drafter and a 1.40 GB multimodal projector, matching Meta's consumer-hardware footprint while preserving this model's modified weights. The complete text quant ladder has also been refreshed with Meta's post-release Jinja correction, which normalizes
Reasoning efforttoReasoning strengthand prevents duplicate reasoning directives. Text generation, image input, and DFlash speculative decoding were validated together on current llama.cpp.
✅ All quants live
The full text quant ladder (
Q2_K→Q8_0), compact Q4_K_M and full-precision vision projectors, and compact Q4_K_M and full-precision DFlash drafters are uploaded — see the Files tab.
⚠️ EXPERIMENTAL
Same-day arch, quantized. Expect sharp edges — decode, coherence, tool-parse, serve edge cases under load. Please open a Community discussion with loader/version, quant, prompt, sampling, and failure mode. Real repros get fixed faster.
Measured on the abliterated parent (GGUF quants inherit this behavior):
| Metric | Result |
|---|---|
| True refusal (harmful, n=300) | 0 / 300 = 0.0% |
| True refusal (full 450) | 0 / 450 = 0.0% |
| Substring-harmful | 0 / 300 |
| Substring-all | 2 / 450 (XSTest false positives) |
| Errors | 0 |
The weight change removes refusals cleanly — no measured true refusals across the full 450-prompt suite.
Muse Glimmer is Meta Superintelligence Labs' 30B agentic, on-device model. This is the abliterated build — refusal behavior removed via a Blackfrost weight-change process — packaged as GGUF for llama.cpp, so it runs on a single consumer GPU or CPU, fully offline. The local footprint is the product.
| Architecture | muse-glimmer — dense, 52 layers, hidden 6656, GQA (32 q / 2 kv), sliding-window attention, + vision tower |
| Base | meta-models/Muse-Glimmer-30B — Meta, Apache-2.0 |
| Transform | Abliterated — refusal behavior removed via a Blackfrost weight-change process; multimodal capability intact |
| Formats | GGUF — Q2_K, Q3_K_S, Q3_K_M, Q4_K_S, Q4_K_M, Q5_K_S, Q5_K_M, Q6_K, Q8_0 |
| Context | 131,072 |
| Spec-decode | DFlash drafter — --spec-type draft-dflash --spec-draft-n-max 15 |
| Default persona | Ships with the "AI assistant" system template baked in |
| quant | size | recommended for |
|---|---|---|
| Q2_K | 10.0 GB | smallest, quality trade-off |
| Q3_K_S | 11.7 GB | very tight VRAM |
| Q3_K_M | 12.7 GB | tight VRAM |
| Q4_K_S | 15.0 GB | 16 GB cards |
| Q4_K_M | 15.8 GB | default — balanced, fits 24 GB |
| Q5_K_S | 18.0 GB | higher quality |
| Q5_K_M | 18.5 GB | strong quality/size balance |
| Q6_K | 21.3 GB | near-lossless |
| Q8_0 | 27.6 GB | max fidelity |
Load a text quant plus an mmproj projector for image input:
| file | size | purpose |
|---|---|---|
mmproj-Muse-Glimmer-30B-Abliterated-Q4_K_M.gguf | 1.40 GB | vision projector — compact, recommended |
mmproj-Muse-Glimmer-30B-Abliterated-F16.gguf | 3.6 GB | vision projector — full precision |
mmproj-Muse-Glimmer-30B-Abliterated-Q8_0.gguf | 1.9 GB | vision projector — compact |
dflash-Muse-Glimmer-30B-Abliterated-Q4_K_M.gguf | 1.63 GB | abliterated DFlash drafter — compact, recommended |
dflash-Muse-Glimmer-30B-Abliterated-F16.gguf | 4.8 GB | DFlash drafter — speculative decoding |
Requires llama.cpp b10353 or newer with llama-server. DFlash runs under llama-server only — it shares the target model's context, so it does not work in llama-cli.
Recommended — with DFlash speculative decoding (~1.6× faster, identical output):
llama-server \
-m Muse-Glimmer-30B-Abliterated-Q8_0.gguf \
-md dflash-Muse-Glimmer-30B-Abliterated-Q4_K_M.gguf \
--spec-type draft-dflash --spec-draft-n-max 15 \
-ngl 999 -ngld 999 -fa on --jinja \
--host 0.0.0.0 --port 8080 -c 16384 \
--temp 1.0 --top-p 0.95 --top-k 64
-md, --spec-type, --spec-draft-n-max, and -ngld.--mmproj mmproj-Muse-Glimmer-30B-Abliterated-Q4_K_M.gguf.deploy/serve.sh auto-downloads + serves; full guide in deploy/DEPLOYMENT.md.temperature 1.0, top_p 0.95, top_k 64 (Meta). Steer depth with a Reasoning strength: low/medium/high/xhigh system line.--jinja is required. The refreshed template accepts an OpenAI-style Reasoning effort: <level> line, normalizes it, and does not inject a conflicting second directive.<|eom|>. Use <|end_of_text|> and <|eot|> as stop tokens.max_tokens ≥ 1024 — heavy thinker; small budgets return empty content because the reasoning channel consumes them. Reasoning arrives in reasoning_content, the answer in content.--spec-draft-n-max 15 — DFlash block size (trained 16, clamped).-fa on for peak speed; switch to -fa off if the load hangs on a brand-new GPU paired with an older CUDA toolkit.1× NVIDIA RTX PRO 6000 (Blackwell), Q8_0, -fa off:
| config | decode tok/s | speedup |
|---|---|---|
| baseline | ~46 | 1.0× |
| + DFlash | ~73 | 1.6× |
Speedup rises with -fa on and structured/code output (Meta reports up to 3.1× on an RTX 5090).