No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from SC117/gemma-4-31B-it-heretic-QAT-For-Edge-16G-GGUF
All models by SC117 →Heretic QAT · Q2_K + Q8_0 Embedding · 11.4 GB · Fits 16GB VRAM
Extreme quantization of coder3101/heretic-QAT using Unsloth's Q2_K_L recipe (Q2_K body + Q8_0 embedding/output). Compresses the 31B dense model to 11.4 GB, enabling it to run on 16GB VRAM consumer GPUs.
This quantization uses Q2_K (~2.9 BPW body) to compress the 31B model to 11.4 GB. This is an extreme solution for running 31B on 16GB VRAM, with known limitations:
Suitable for (low-stakes tasks):
Not recommended for: Math, precise reasoning, fact-checking, production use, or any task requiring strict output accuracy.
Base: coder3101/gemma-4-31B-it-qat-q4_0-unquantized-heretic · Heretic v1.2.0 · ARA + Row-Norm
Thinking mode (default):
Use --jinja flag with llama.cpp. Disable thinking: --chat-template-kwargs '{"enable_thinking":false}'.
Compatible with llama.cpp, LM Studio, Jan, koboldcpp, and other GGUF runtimes. Use --jinja flag and -ngl 99 for GPU offload.
Heretic Abliteration: coder3101 · Heretic v1.2.0 ARA + Row-Norm
QAT Weights: Google Gemma 4 QAT
Quantization Recipe: Unsloth Q2_K_L (Q2_K + Q8_0 embedding/output)
Quantization Tool: llama.cpp b9553 · GitHub
Original Model: Google Gemma 4 31B IT