No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from Jackrong/Gemopus-4-26B-A4B-it
All models by Jackrong →[!NOTE] Gemopus is an attempt at fine-tuning Gemma 4 with a core philosophy of "stability first".
While preserving the original reasoning order of Gemma 4 as much as possible, we conducted targeted refinements for answer quality, structure, clarity, and consistency.
🍎 Therefore, My fine-tuning strategy chose not to follow other teams in aggressive direct distillation from Claude. Instead, we opted for a more conservative and controllable path.
Gemopus-4-26B-A4B-it is a supervised fine-tune version based on the Gemma 4 26B Instruction model.
[!IMPORTANT] A 2026 self-distillation paper found that while self-distillation can often shorten reasoning traces and improve some in-domain performance, it may also lead to performance degradation in mathematical reasoning. This degradation is correlated with the suppression of uncertainty expressions. The authors reported that on some models and settings, out-of-domain (OOD) performance degradation can be significant. The implication of this result is that reasoning text that appears on the surface to be shorter, cleaner, and more "good at solving problems" does not mean that the student has truly acquired more robust reasoning capabilities.
When only Supervised Fine-Tuning (SFT) is available, without subsequent Reinforcement Learning (RL) or process supervision, forcefully feeding the student model with potentially non-faithful long reasoning traces together with the final answers introduces substantial hidden risks.
The core issue is that the student model may fail to reliably align the token-level rationale sequence with the latent computational process that actually supports the final decision. As a result, training can easily degenerate into imitation of the surface form of reasoning text, rather than genuine internalization of the underlying reasoning mechanism.
In this setting, such long-form reasoning data may improve stylistic resemblance, but not necessarily reasoning fidelity.
By contrast, compared with the Qwen series, Gemma 4 already exhibits a more structured, orderly, and disciplined reasoning-chat style. It is also significantly less prone to “overthinking” or producing excessively long, uncontrolled reasoning chains. For that reason, there is little value in aggressively reshaping its reasoning style during the SFT stage. Doing so risks disrupting Gemma 4’s native and already strong reasoning rhythm, while offering limited upside in the absence of later-stage alignment methods.
Based on the methodological deduction above, I chose to focus my optimization efforts on the lower-risk, more consistently rewarding levels of final answer quality and interactive experience:
⏳ Thanks to Kyle Hessling for independently running these benchmarks and sharing the results!
For the best performance, use these configurations and best practices:
Use the following standardized sampling configuration across all use cases:
temperature=1.0top_p=0.95top_k=64Compared to Gemma 3, the models use standard system, assistant, and user roles. To properly manage the thinking process, use the following control tokens:
<|think|> token at the start of the system prompt. To disable thinking, remove the token.<|channel>thought\n [Internal reasoning] <channel|><|channel>thought\n<channel|> [Final answer][!NOTE] Note that many libraries like Transformers and llama.cpp handle the complexities of the chat template for you.
🚧 The complete fine-tuning code and related notebooks for this model will be updated soon, please stay tuned!
👉 GitHub Repository: Jackrong-llm-finetuning-guide
Welcome to visit this repository to gain a deeper understanding of the codebase and reproduce the training results locally or on Colab.
🔗 Qwopus3.5-27b Complete Fine-Tuning Guide (PDF)
No one starts out as an expert, but all experts bravely took the first step.
All training and testing for this project are self-funded. If you find this model or guide helpful, giving a Star ⭐️ on GitHub is the greatest encouragement to me. 🙏
The training data specifically curates highly coherent instruction pairs with optimal structures from the open-source community, alongside natural multi-turn conversations. The goal is to guide the model to learn more mature ways of organizing and presenting conclusions, rather than mechanically imitating "fake chain of thought" without internalized logic.
Tool Calling Compatibility: The Gemma 4 series models still have known compatibility issues with tool calling functionality in local inference ecosystems like llama.cpp / LM Studio (including call failures, format mismatches, continuous loops, etc.). This has been widely reported in the community and is not unique to this model. If your workflow heavily relies on tool calling, it is recommended to thoroughly test it before official use, or temporarily consider solutions with more mature ecosystem support.
Regarding Fine-Tuning Characteristics of the Gemma Architecture: From an engineering practice perspective, the Gemma series does exhibit different training dynamics compared to the Qwen series during fine-tuning—including wider loss curve fluctuations and greater sensitivity of gradient stability to hyperparameters. This may be related to Google's model architecture design. Furthermore, the base Gemma 4 model objectively still has a gap compared to the Qwen 3.5 series in certain dimensions of its raw capabilities. We believe that truthfully stating these observations is more beneficial to the technical judgment of the community than selectively avoiding them.
Project Positioning: The core value of Gemopus-4-26B-A4B-it lies in providing an engineering exploration reference supported by methodology for SFT fine-tuning under the Gemma 4 architecture, rather than a fully production-ready solution. If you are looking for a productivity model that has undergone more iterative validation and offers more stable ecosystem compatibility, I recommend looking at the Qwopus-3.5-v3 series—its performance after fine-tuning is much more robust.
Special thanks to the developers in the open-source community for building such a thriving ecosystem. Thank you to the Unsloth team for providing excellent and highly efficient LLM fine-tuning support, and sincere respect to the Google team for open-sourcing the outstanding Gemma 4 base model. Finally, thanks to all the researchers who have contributed profound insights into CoT Faithfulness and the interpretability of LLM reasoning. It is exactly these rigorous frontier academic discussions that deeply inspired the core fine-tuning methodology of this project.
Base Model (google/gemma4-26B-it) │ ▼Targeted Supervised Fine-Tuning (SFT) (Focus on Answer Quality & Structural Alignment, Retaining Restrained CoT) │ ▼Gemopus-4-26B-A4B-it