No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from inclusionAI/Ling-2.6-1T-base
All models by inclusionAI →
🤗 Hugging Face | 🤖 ModelScope | Tech Report | 🐙 GitHub
Ling-2.6-1T-base is the base checkpoint behind the Ling-2.6-1T and Ring-2.6-1T. It is a trillion-parameter Mixture-of-Experts language model retrofitted from Ling-2.0-1T-base with a hybrid linear attention design, continued pre-training, and long-context mid-training.
This release is intended for research, continued pre-training, distillation, and supervised or preference-based fine-tuning. It is not a chat-aligned assistant model. If you want an out-of-the-box instruction or reasoning model, use the corresponding Ling-2.6 or Ring-2.6 post-trained checkpoints instead.
Ling-2.6-1T-base is designed to preserve the capability of the Ling-2.0 trillion-scale backbone while making long-context training and inference materially more efficient. The core upgrade is a hybrid attention retrofit that combines Lightning Attention with MLA in a 7:1 ratio, together with a smooth migration pipeline from the original GQA-based architecture.
According to the technical report, the model is trained through approximately 9.6T tokens across migration pre-training, continued pre-training, and mid-training, with staged context extension from 4K to 256K. The same base checkpoint is later specialized into:
The model starts from Ling-2.0-1T-base and is converted into the Ling-2.6-1T architecture through a multi-stage migration pipeline that includes:
This retrofit is designed to preserve pre-trained capability while reducing long-context compute cost and KV-cache pressure.
The continued pre-training and mid-training stages include:
The following numbers are selected from the technical report and reflect base-model evaluation rather than chat-aligned or instruction-tuned performance.
In the technical report, Ling-2.6-1T-base shows broad gains over Ling-2.0-1T-base, especially on factual knowledge, multilingual knowledge coverage, long-context understanding, and reasoning-oriented evaluations, while preserving or improving strong math and code capability. One notable exception in this selected subset is MMLU-Pro, where Ling-2.0-1T-base remains slightly higher.
Recommended use cases:
Not recommended as-is for:
If your goal is interactive assistant use rather than research on base checkpoints, these post-trained models are usually the better starting point.
This is a base checkpoint. One can load it for simple generation or further post-training. Notably, real deployment of a trillion-parameter model typically requires multi-node distributed infrastructure. The example below illustrates the loading pattern only.
For production inference, prefer distributed serving stacks such as SGLang or other engines that support the released architecture and model size. You can refer to our instruction version for SGLang deployment.
This checkpoint and code repository is licensed under the MIT License.
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "inclusionAI/Ling-2.6-1T-base"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)model = AutoModelForCausalLM.from_pretrained( model_name, trust_remote_code=True, device_map="auto",)
prompt = "Explain the difference between full attention and linear attention."inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate( **inputs, max_new_tokens=256, do_sample=False,)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))