[Image blocked: Mellum]
Mellum2 Thinking (SFT)
[!Note]
Research artifact: the SFT-only intermediate checkpoint of the Thinking training pipeline. Use it to study or build on the reasoning SFT stage in isolation, run post-training experiments (preference tuning, RLVR, etc.), or compare against the final RL-tuned variant. For production-quality reasoning use Thinking instead.
Mellum2 Thinking (SFT) Highlights
Mellum2 Thinking (SFT) is the supervised-fine-tuning-only intermediate checkpoint of the Mellum2 Thinking training pipeline, released by JetBrains as a research artifact.
The model uses a Mixture-of-Experts architecture with 64 experts and activates 8 experts per token. It uses a combination of sliding-window and full attention layers, with a context length of 131,072 tokens.
This checkpoint was produced from Mellum2-12B-A2.5B-Base with three epochs of SFT on a corpus of reasoning-trace-bearing data. The model emits its reasoning inside <think>...</think> blocks. It is the starting point of the RLVR stage that produces the final Mellum2-12B-A2.5B-Thinking.
Mellum2 Model Family
This repository contains one checkpoint from the Mellum2 family.
Model Overview
Mellum2 Thinking (SFT) has the following features:
- Number of Layers: 28
- Hidden Size: 2304
- Intermediate Size: 7168
- MoE Intermediate Size: 896
- Number of Experts: 64
- Number of Activated Experts: 8
- Number of Attention Heads (GQA): 32 for Q and 4 for KV
- Context Length: 131,072
- Sliding Window: 1,024
- Vocabulary Size: 98,304
- Precision: bfloat16
Serving with vLLM
Quickstart
Text-Only Input
Evaluation
Evaluation results are available in the model card. All values are self-reported by JetBrains.
For more details, see the Mellum2 Technical Report.
License
Released under the Apache 2.0 license.