No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from nerkyor/Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF
All models by nerkyor →[Image blocked: 44G LynnStyle GGUF overview]
LCB note: LCB is the standard abbreviation for LiveCodeBench. It selects competitive-programming problems published after the evaluated model's training-data cutoff date and judges code correctness by executing test cases, helping reduce training-data contamination. The LCB100 reported on this card is a fixed 100-problem coding evaluation under the same protocol.
This repository contains the 44G LynnStyle layer-wise mixed-precision GGUF of Qwen3.5-122B-A10B. The main GGUF already includes this training branch's step64 LoRA update, so no separate LoRA adapter is required. An optional multimodal mmproj is included.
The repository does not ship an MTP sidecar for two practical reasons: the main model, KV cache, and runtime overhead need to fit reliably on one 48GB GPU, and controlled testing found that quantizing the MTP sidecar reduced output quality. All reported scores and recommended commands therefore use NO MTP.
At a glance
- Main GGUF: 44.61 GB (41.55 GiB); main model plus mmproj: 45.23 GB (42.13 GiB).
- This 44G mixed-precision scheme keeps critical pathways at high precision while letting the largest ordinary expert layers absorb most of the compression.
- Put plainly: components that strongly affect understanding, reasoning, and output convergence are compressed conservatively; the numerous, space-heavy ordinary routed-expert weights carry the aggressive compression.
- Final NO-MTP evaluation: MMLU 91.6% (458/500), GPQA 86.4% (171/198), and LCB 82.0% (82/100).
- Verified runtime:
C4 = 4 × 32K, 131072 total context, Q4 K/V cache, full GPU offload,--fit off; peak VRAM was 45,551 MiB.
The mmproj filename keeps the Q8_0 tier label, while its actual conversion contains Q8_0 weights with F16 fallbacks and small F32 structures. The main model can be used on its own for text inference; add --mmproj only when processing visual input.
qwen3next; the main GGUF contains 807 tensors.GPT5.6Sol-SFT in the repository name identifies this model's training branch.--lora.--model-draft.The protection unit is layer × path × tensor family. Standard GGUF stores the routed experts of one layer in a stacked tensor, so this build protects entire layer-level expert tensors rather than assigning a separate precision to each expert inside one tensor.
REAP/router observations identify task pathways, while the imatrix constrains quantization error. Neither replaces final evaluation. L26 and L43–L47 were preregistered as the high-protection set during the build; this does not claim that any single layer independently determines model quality. Release acceptance comes from the complete MMLU500, GPQA198, LCB100, and real-load gates.
q4_0 K/V cache. This is runtime context storage and does not change the model-file weights.The main GGUF is 65.65% smaller than the 129,871,496,512-byte original Q8 and 82.17% smaller than the 250,196,479,366-byte BF16 source.
The fixed evaluation set contains MMLU500, GPQA198, and LCB100. Generation used temperature 0.6, top_p=0.95, and at most 32768 output tokens per slot. The 44G evaluation used C4, 32K per slot, Q4 K/V cache, and NO MTP; MMLU was 5-shot.
* The Q8 LCB baseline did not receive a symmetric 36K length-only retest. Under the same natural-32K condition, 44G scored 81.0% (81/100) versus Q8's 78.0% (78/100), a +3.0 pp difference.
This comparison is not a pure “change only the quantization bits” A/B test. The Q8 artifact is the original quality baseline, while the 44G main model also includes the step64 LoRA update. The measured differences therefore combine the training update and mixed quantization.
reasoning_content to the final answer before parsing. Rejudging the same saved 198 rows by the final standalone A/B/C/D choice gives 89.4% (177/198); all 198 rows ended with stop, and parse/error/timeout counts were zero.stop and were correct, producing 86.4% (171/198).The final corrected gate passed. Peak VRAM was 45,551 MiB, below the 49,152 MiB limit.
Or:
Use a recent llama.cpp build with qwen3next and the corresponding multimodal projector support.
Do not add --model-draft: this repository does not publish MTP, and its quality scores were not produced with MTP.
[Image blocked: 44G LynnStyle GGUF 中文总览]
LCB 说明： LCB 是 LiveCodeBench 的标准缩写。它按照被测模型训练数据的截止日期，选取该日期之后发布的竞赛编程题，并通过执行测试用例判定代码正确性，以降低训练数据污染。本卡的 LCB100 为固定 100 题同口径编码测试。
这是 Qwen3.5-122B-A10B 的 44G LynnStyle MoE 分层混合精度 GGUF。主模型已合入本训练分支的 step64 LoRA 更新，下载后不需要再挂载独立 LoRA；仓库另附可选的视觉 mmproj。为了让主模型、运行时缓存和必要开销稳定装进单张 48GB 显卡，本仓不附带 MTP sidecar；同时，受控测试发现量化后的 MTP 会降低输出质量，因此公开质量分数与推荐启动方式全部采用 NO MTP。
先看结论
- 主 GGUF：44.61 GB（41.55 GiB）；主模型加 mmproj：45.23 GB（42.13 GiB）。
- 这份44G的混合精度方案：关键通路保留高精度，体积最大的普通专家层承担主要压缩。
- 换句话说，容易影响理解、推理和输出收口的部分尽量少压；数量最多、最占空间的普通专家权重压得更狠。模型推理本身仍使用 FP16 / FP32，Q4 只用于运行时的上下文缓存。
- NO-MTP 全量终局：MMLU 91.6%（458/500）、GPQA 86.4%（171/198）、LCB 82.0%（82/100）。
- 已验证
C4 = 4 × 32K、总上下文 131072、Q4 KV、全 GPU、--fit off，峰值 45,551 MiB。- ModelScope 侧栏可能把该文件标成
unknown / IMATRIX；真实类型以本卡、manifest 和 GGUF 解析为准。
mmproj 文件名沿用 Q8_0 档名；实际转换结果为 Q8_0 + F16 fallback + F32 小结构，不是所有 tensor 都统一 Q8。主模型可独立用于文本推理；视觉输入时再添加 --mmproj。
qwen3next，主 GGUF 共 807 tensors。GPT5.6Sol-SFT 对应该模型训练分支。--lora。--model-draft。这里的保护单位是 层 × 路径 × tensor family。标准 GGUF 会把同层 routed experts 堆叠到同一张大 tensor 中，因此本档保护的是整层专家张量，而不是在一张 tensor 内给单个 expert 指定不同精度。
REAP / 路由观测用于识别任务路径，imatrix 用于约束量化误差；二者都不能替代最终测评。L26、L43–L47 是构建时预注册的高保护集合，并不等于声称某一层单独“决定”全部能力。发布结论只由完整 MMLU500、GPQA198、LCB100 和真实加载门禁给出。
q4_0 K/V cache。它只影响运行时上下文缓存，不会改变模型文件里的权重。主 GGUF 相比 129,871,496,512-byte 原始 Q8 缩小 65.65%，相比 250,196,479,366-byte BF16 缩小 82.17%。
主评测使用固定 MMLU500 / GPQA198 / LCB100 资产；生成温度 0.6、top_p=0.95、每路最大输出 32768 tokens。44G 使用 C4、每路 32K、Q4 KV、NO MTP；MMLU 为 5-shot。所有百分比均在前，绝对值附后。
* Q8 LCB 没有做对称的 36K length-only 补测。完全同为自然 32K 时，44G 是 81.0%（81/100），相对 Q8 的 78.0%（78/100） 为 +3.0pp。
这张表不是“只改量化位宽”的纯 A/B：Q8 是原始质量基线，而 44G 主模型已经包含 step64 LoRA 更新。因此差值同时反映训练更新与混合量化，不能全部归因于量化。
reasoning_content 拼在最终答案前而错误统计为 50.0%（99/198）。同一份 198 条 JSONL 采用“最后一个独立 A/B/C/D”规则重判后为 89.4%（177/198）；198/198 均为 stop，parse/error/timeout 为 0。最终 corrected gate 为 PASS；运行峰值 45,551 MiB，低于 49,152 MiB 门槛。
或：
请使用支持 qwen3next 与相应 multimodal projector 的较新 llama.cpp。不要添加 --model-draft：本仓没有发布 MTP，质量分数也不是 MTP 成绩。
mmproj 已通过 GGUF 结构与文件审计；本卡没有发布独立视觉质量基准，不应把文本分数解释成视觉分数。unknown / IMATRIX 当成损坏。from huggingface_hub import snapshot_download
model_dir = snapshot_download( repo_id="nerkyor/Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF")print(model_dir)git lfs installgit clone https://huggingface.co/nerkyor/Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUFcd Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUFsha256sum -c Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF.SHA256SUMSllama-server \ -m Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF.gguf \ -ngl all --fit off \ -c 32768 -np 1 -n 32768 \ --cache-type-k q4_0 --cache-type-v q4_0 \ --host 0.0.0.0 --port 8080 \ --metrics --no-webui --cache-ram 0llama-server \ -m Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF.gguf \ --mmproj Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF-mmproj-Q8_0.gguf \ -ngl all --fit off \ -c 131072 -np 4 -n 32768 \ --cache-type-k q4_0 --cache-type-v q4_0 \ --host 0.0.0.0 --port 8080 \ --metrics --no-webui --cache-ram 0from huggingface_hub import snapshot_download
model_dir = snapshot_download( repo_id="nerkyor/Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF")print(model_dir)git lfs installgit clone https://huggingface.co/nerkyor/Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUFcd Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUFsha256sum -c Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF.SHA256SUMSllama-server \ -m Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF.gguf \ -ngl all --fit off \ -c 32768 -np 1 -n 32768 \ --cache-type-k q4_0 --cache-type-v q4_0 \ --host 0.0.0.0 --port 8080 \ --metrics --no-webui --cache-ram 0llama-server \ -m Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF.gguf \ --mmproj Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF-mmproj-Q8_0.gguf \ -ngl all --fit off \ -c 131072 -np 4 -n 32768 \ --cache-type-k q4_0 --cache-type-v q4_0 \ --host 0.0.0.0 --port 8080 \ --metrics --no-webui --cache-ram 0