No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from DhruvalLabs/gemma-4-12B-GGUF
All models by DhruvalLabs →Quantized GGUF versions of google/gemma-4-12B
This is a Vision-Language Model (VLM) — it can understand both text and images.
[!IMPORTANT] This VLM requires TWO files — a text backbone GGUF and the
mmprojvision encoder GGUF. Download one text backbone (e.g. Q4_K_M) and the mmproj file. Both must be in the same folder.
⚠️ You need BOTH files — one text backbone + the mmproj — to run this VLM.
Run python benchmark.py --model gemma-4-12B to generate results.
Dhptl/gemma-4-12B in LM StudioThis model uses a two-component architecture:
Why is mmproj always F16? The vision encoder maps image pixels to token embeddings. Quantizing it causes visible visual artifacts and degraded image understanding. It stays at F16 (half precision) which is already very efficient at ~1-2GB for most models.
Found an issue? Open a Discussion in the Community tab.
If useful, please:
ollama run dhptl/gemma-4-12b# Download both files to the same directory, then:./llama-llava-cli \ -m gemma-4-12B-Q4_K_M.gguf \ --mmproj gemma-4-12B-mmproj-f16.gguf \ --image /path/to/your/image.jpg \ -p "Describe this image in detail." \ -n 512./llama-cli \ -m gemma-4-12B-Q4_K_M.gguf \ -p "You are a helpful assistant." \ --conversationfrom llama_cpp import Llamafrom llama_cpp.llama_chat_format import Llava16ChatHandler
# Load VLM with mmprojchat_handler = Llava16ChatHandler(clip_model_path="./gemma-4-12B-mmproj-f16.gguf")llm = Llama( model_path="./gemma-4-12B-Q4_K_M.gguf", chat_handler=chat_handler, n_gpu_layers=-1, n_ctx=4096, logits_all=True,)
# Text + image inferenceresponse = llm.create_chat_completion( messages=[ { "role": "user", "content": [ {"type": "image_url", "image_url": {"url": "https://example.com/image.jpg"}}, {"type": "text", "text": "What do you see in this image?"} ] } ])print(response["choices"][0]["message"]["content"])