No official torrent yet โ requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from yuxinlu1/gemma-4-12B-coder-fable5-composer2.5-v1-GGUF
All models by yuxinlu1 โNo matter your GPU. No matter your RAM. If you've got ~4.5 GB of VRAM or unified memory free, you can run your own private, offline coding assistant right now. ๐ This is the v1 / code edition โ distilled from real chain-of-thought so it thinks through a problem before writing the solution. ๐ง ๐ป All local, all yours, no API, no cloud.
A focused fine-tune of Gemma 4 12B on verifiable Python coding data โ every training example's reasoning leads to code that actually passed its tests. The result reasons in the open (edge cases, complexity, approach) and then emits a clean, runnable solution. ๐
๐๐ฅ IT'S HERE โ v2 is OUT NOW! v2 has shipped โ the GGUF quants are live and ready to run โ
grab v2 here. ๐
The full safetensors master (build / fine-tune on top) goes up tomorrow. v2 is agentic + coding focused โ
the piece v1 was missing.
Here's the result that got me most excited. When I saw v2's tau2-bench telecom result โ an agentic tool-use
benchmark where the model has to diagnose โ fix โ verify, exactly like real terminal/debugging work โ I literally got
launched out of my chair (โฆokay, kidding ๐). The jump in actually solving the problem is wild:
The base model tends to give up early (hands the problem off to a human); v2 keeps going and works it the way a much bigger model would. Full benchmark details are in the v2 card now. ๐ง
โ safetensors master (this v1 model) is UP. Full-precision weights are live โ yuxinlu1/gemma-4-12B-coder-fable5-composer2.5-v1 โ roll your own GGUF / MLX / AWQ quants or fine-tune straight from the master. ๐
A community member spotted that this model was reporting only a 131K context window. That turned out to be
the well-known upstream Gemma 4 metadata bug โ Google's initial config.json shipped with
max_position_embeddings: 131072 instead of the real 262144 (256K), and that value got baked into a lot of
downstream finetunes and quants (including this one) before it was fixed upstream.
The weights were always fine โ it was purely a metadata field. All GGUF quants have been re-patched to the
full 256K context (gemma4.context_length = 262144). Just re-download if you grabbed an earlier copy. ๐
This is a distillation of two complementary chain-of-thought sources, both over verifiable Python coding tasks (algorithmic / function-level problems that come with deterministic tests):
The recipe: real CoT for the bulk of solid coverage, plus synthetic "second-attempt" CoT to patch the failures โ both verified by execution before anything entered training. โ
Rough estimates ๐ค (assumes q8_0 KV cache + ~1.5 GB overhead; use q4_0 KV cache for โ2ร more context!).
Max context is 256K. "โ" = won't fit, pick a smaller quant. โ๏ธ
๐ก Apple Silicon / integrated GPUs with unified memory count too โ same numbers, just slower than a dGPU. ๐ก Low on room? Drop a quant or switch KV cache to
q4_0and your context roughly doubles.
โฆ-Q4_K_M.gguf) and llama-server from llama.cpp.
โ ๏ธ Needs a recent llama.cpp (this is the
gemma4_unifiedarchitecture โ older builds won't load it).
.bat shown โ tweak --port, --ctx-size to taste):http://localhost:18080 and chat. ๐ (Tip: bump --ctx-size per the table; use q4_0 KV for more.)Works in LM Studio, Jan, Ollama, etc. โ just import the GGUF, pick your quant, go. ๐พ
This model thinks in Gemma's native thought channel before answering โ exactly how it was trained. Keep
enable_thinking=true (the default chat template handles it). Recommended sampling: temp 1.0, top_p 0.95, top_k 64.
For coding you can also go greedy (temp 0) for more deterministic solutions.
google/gemma-4-12B-it.@echo off
cd /d C:\llama.cpp
llama-server.exe ^
-m C:\models\gemma4-coding-Q4_K_M.gguf ^
--ctx-size 16384 ^
--n-gpu-layers 99 ^
--no-mmap ^
-fa on ^
--cache-type-k q8_0 --cache-type-v q8_0 ^
--temp 1.0 --top-p 0.95 --top-k 64 ^
--host 0.0.0.0 --port 18080
pause