No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from DhruvalLabs/Qwen2.5-VL-7B-Instruct-GGUF
All models by DhruvalLabs →Quantized GGUF versions of Qwen/Qwen2.5-VL-7B-Instruct
This is a Vision-Language Model (VLM) — it can understand both text and images.
[!IMPORTANT] This VLM requires TWO files — a text backbone GGUF and the
mmprojvision encoder GGUF. Download one text backbone (e.g. Q4_K_M) and the mmproj file. Both must be in the same folder.
⚠️ You need BOTH files — one text backbone + the mmproj — to run this VLM.
Run python benchmark.py --model Qwen2.5-VL-7B-Instruct to generate results.
Dhptl/Qwen2.5-VL-7B-Instruct in LM StudioThis model uses a two-component architecture:
Why is mmproj always F16? The vision encoder maps image pixels to token embeddings. Quantizing it causes visible visual artifacts and degraded image understanding. It stays at F16 (half precision) which is already very efficient at ~1-2GB for most models.
Found an issue? Open a Discussion in the Community tab.
If useful, please:
ollama run dhptl/qwen2.5-vl-7b-instruct# Download both files to the same directory, then:
./llama-llava-cli \
-m Qwen2.5-VL-7B-Instruct-Q4_K_M.gguf \
--mmproj Qwen2.5-VL-7B-Instruct-mmproj-f16.gguf \
--image /path/to/your/image.jpg \
-p "Describe this image in detail." \
-n 512./llama-cli \
-m Qwen2.5-VL-7B-Instruct-Q4_K_M.gguf \
-p "You are a helpful assistant." \
--conversationfrom llama_cpp import Llama
from llama_cpp.llama_chat_format import Llava16ChatHandler
# Load VLM with mmproj
chat_handler = Llava16ChatHandler(clip_model_path="./Qwen2.5-VL-7B-Instruct-mmproj-f16.gguf")
llm = Llama(
model_path="./Qwen2.5-VL-7B-Instruct-Q4_K_M.gguf",
chat_handler=chat_handler,
n_gpu_layers=-1,
n_ctx=4096,
logits_all=True,
)
# Text + image inference
response = llm.create_chat_completion(
messages=[
{
"role": "user",
"content": [
{"type": "image_url", "image_url": {"url": "https://example.com/image.jpg"}},
{"type": "text", "text": "What do you see in this image?"}
]
}
]
)
print(response["choices"][0]["message"]["content"])