No official torrent yet โ requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from yuxinlu1/gemma-4-12B-it-Claude-4.6-4.8-Opus-GGUF
All models by yuxinlu1 โNo matter your GPU. No matter your RAM. If you've got ~4.5 GB of VRAM or unified memory free, you can run your own private, offline AI right now. ๐ Tuned on Opus 4.6, 4.7 & 4.8 reasoning data, it delivers a major leap in reasoning power โ whether you're asking questions or writing code. ๐ง ๐ป All local, all yours, no API, no cloud.
As of June 7, 2026, mainline llama.cpp just merged Gemma 4 MTP support โ so the MTP draft model is now
live in the MTP/ folder.
Drop it next to any quant and generation gets noticeably faster with identical output (speculative decoding is
lossless) โ just add a couple of flags. ๐ See โก Speed it up with MTP below. ๐
Rough estimates ๐ค (assumes q8_0 KV cache + ~1.5 GB overhead; use q4_0 KV cache for โ2ร more context!).
Max context is 131K. "โ" = won't fit, pick a smaller quant. โ๏ธ
๐ก Apple Silicon / integrated GPUs with unified memory count too โ same numbers, just slower than a dGPU. ๐ก Low on room? Drop a quant or switch KV cache to
q4_0and your context roughly doubles.
New as of June 7, 2026! Gemma 4's Multi-Token Prediction drafter lets the model guess a few tokens ahead and verify them in one shot โ so you get more tokens/sec with byte-for-byte identical output. Pure speed, zero quality cost. ๐ช
1. Grab the tiny draft from the MTP/ folder:
๐ก The draft is tiny โ keep it Q8 or higher (over-quantizing a draft just lowers its hit rate). It pairs with any quant of the main model.
2. You need a fresh llama.cpp build โ June 7 2026 (b9553) or newer. MTP was just merged, so older builds
can't load the draft (unknown architecture: 'gemma4-assistant').
3. Run it exactly like below, just +3 flags (--model-draft, --spec-type, --n-gpu-layers-draft):
Measured on a single RTX 5090 (Q4_K_M main + Q8 draft): ~1.3ร faster at greedy and ~1.2ร at the default thinking sampling โ free, with no change to output. ๐
๐ง Heads-up: this is the stock Gemma drafter (trained on base Gemma 4), so on this fine-tune the hit rate โ and thus the speedup โ is a little lower than on vanilla Gemma 4. A re-aligned draft could push it higher (maybe a future update). Either way: free speed, no downside. ๐
โฆ-Q4_K_M.gguf) and llama-server from llama.cpp.
โ ๏ธ Needs a recent llama.cpp (this is the
gemma4_unifiedarchitecture โ older builds won't load it).
.bat shown โ tweak --port, --ctx-size to taste):http://localhost:18080 and chat. ๐ (Tip: bump --ctx-size per the table; use q4_0 KV for more.)Works in LM Studio, Jan, Ollama, etc. โ just import the GGUF, pick your quant, go. ๐พ
This model thinks in Gemma's native thought channel. Keep enable_thinking=true (the default chat template
handles it). Recommended sampling: temp 1.0, top_p 0.95, top_k 64.
google/gemma-4-12B-it.angrygiraffe/claude-opus-4.6-4.7-reasoning-8.7k,
augmented with additional Opus 4.8-generated reasoning samples I curated and mixed in.@echo offcd /d C:\llama.cppllama-server.exe ^ -m C:\models\gemma4-opus48-Q4_K_M.gguf ^ --model-draft C:\models\MTP\gemma-4-12B-it-MTP-Q8_0.gguf ^ --spec-type draft-mtp --spec-draft-n-max 4 ^ --ctx-size 16384 --n-gpu-layers 99 --n-gpu-layers-draft 99 ^ --no-mmap -fa on ^ --temp 1.0 --top-p 0.95 --top-k 64 ^ --host 0.0.0.0 --port 18080pause@echo offcd /d C:\llama.cppllama-server.exe ^ -m C:\models\gemma4-opus48-Q4_K_M.gguf ^ --ctx-size 16384 ^ --n-gpu-layers 99 ^ --no-mmap ^ -fa on ^ --cache-type-k q8_0 --cache-type-v q8_0 ^ --temp 1.0 --top-p 0.95 --top-k 64 ^ --host 0.0.0.0 --port 18080pause