No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from AtomicChat/Qwen2.5-7B-Instruct-GGUF
All models by AtomicChat →
Qwen2.5 7B, self-quantized to GGUF by Atomic Chat. Built straight from Qwen's original weights with a per-tensor importance matrix, so this is not a repack of somebody else's files. Runs fully offline.
[!NOTE] These GGUFs are self-quantized from the original weights, not a repack. The importance matrix keeps low-bit quants closer to the full-precision model.
[!IMPORTANT] Always pass
--jinjaso the Qwen2.5 7B chat template is applied. Without it the model can emit malformed turns.
[!TIP] Pick the largest file that fits your (V)RAM with room for context.
Q4_K_MorUD-Q4_K_XLis the sweet spot for most setups;Q6_KorQ8_0for maximum fidelity.
Run Qwen2.5 7B locally with:
AtomicChat/Qwen2.5-7B-Instruct-GGUF, pick a quant, hit Use this model.llama-server -hf AtomicChat/Qwen2.5-7B-Instruct-GGUF:Q4_K_M --jinja -c 8192ollama run hf.co/AtomicChat/Qwen2.5-7B-Instruct-GGUF:Q4_K_MQwen's recommended sampling configuration for Qwen/Qwen2.5-7B-Instruct.
Qwen/Qwen2.5-7B-Instruct (original weights).--imatrix.UD-Q4_K_XL additionally pins the token-embedding and output tensors to Q8_0.Original model by Qwen, released under the Apache 2.0 license. Full terms: Apache 2.0. Quantized by Atomic Chat.
git clone https://github.com/ggml-org/llama.cppcmake llama.cpp -B llama.cpp/build -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ONcmake --build llama.cpp/build --config Release -j --target llama-cli llama-server./llama.cpp/build/bin/llama-server \ -hf AtomicChat/Qwen2.5-7B-Instruct-GGUF:Q4_K_M \ --jinja -ngl 99 -c 8192 -fa on