No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from JetBrains/Mellum2-12B-A2.5B-Instruct-GGUF-Q8_0
All models by JetBrains →This repository contains a GGUF Q8_0 quantization of
JetBrains/Mellum2-12B-A2.5B-Instruct, ready to run with
llama.cpp, Ollama, LM Studio, and
other GGUF-compatible runtimes.
This quantization (Q8_0): 8-bit. Effectively lossless — KL divergence from BF16 is ~0.016 and it keeps BF16's top token 95% of the time.
Mellum 2 Instruct is a Mixture-of-Experts assistant model (64 experts, 8 activated per token, 131,072-token context) that answers directly, without an externalized chain of thought. For the full model description, evaluation results, and architecture details, see the original model card: JetBrains/Mellum2-12B-A2.5B-Instruct.
KL divergence and top-token agreement are measured against the BF16 logits on
Wikitext-2 (n_ctx=512); lower KLD / higher agreement means closer to the
unquantized model. (Perplexity is omitted here — it is unreliable for
instruction-tuned models on Wikitext-2, which is out of distribution.)
The server exposes an OpenAI-compatible API on http://localhost:8080/v1:
Released under the Apache 2.0 license.
For the full model card, evaluation results, and architecture details, refer to the original model: JetBrains/Mellum2-12B-A2.5B-Instruct.
hf download JetBrains/Mellum2-12B-A2.5B-Instruct-GGUF-Q8_0 Mellum2-12B-A2.5B-Instruct-Q8_0.gguf --local-dir .# Pull and serve in one step (downloads the GGUF automatically)llama-server -hf JetBrains/Mellum2-12B-A2.5B-Instruct-GGUF-Q8_0 \ --ctx-size 131072 \ --temp 0.6 --top-p 0.95 --top-k 20
# Or run a one-off prompt with a local filellama-cli -m Mellum2-12B-A2.5B-Instruct-Q8_0.gguf \ --ctx-size 131072 \ --temp 0.6 --top-p 0.95 --top-k 20 \ -p "Write a Python function to reverse a string."from openai import OpenAI
client = OpenAI(base_url="http://localhost:8080/v1", api_key="llama.cpp")
chat_response = client.chat.completions.create( model="JetBrains/Mellum2-12B-A2.5B-Instruct-GGUF-Q8_0", messages=[ {"role": "user", "content": "Write a Python function to reverse a string."}, ], max_tokens=81920, temperature=0.6, top_p=0.95, extra_body={"top_k": 20},)print(chat_response.choices[0].message.content)ollama run hf.co/JetBrains/Mellum2-12B-A2.5B-Instruct-GGUF-Q8_0