No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from nerkyor/Qwen3.8-27B-Coder390-EfficientThink-Opus5.5-GPT6Astra-Grok4.7-DSV4Pro-K3-SFT-RLOO-GGUF-NInfer
All models by nerkyor →[Image blocked: Coder390 FP8 vs. original FP8]
[Image blocked: Coder390 GGUF Q2 LynnStyle vs. original FP8]
[Image blocked: vs. the original]
A model post-trained with several rounds of SFT and RLOO on top of Qwen3.8-27B-EfficientThink-SFT-SimPO-DFlash2, which is itself the official Qwen3.8-27B post-trained with SFT and SimPO. It is built to fix the original model's habit of failing to stop: it already has an answer, yet keeps re-deriving until it hits the 94K cap with no final answer. This repository is the standalone GGUF repository with Q3 LynnStyle, Q4 LynnStyle, Q2 LynnStyle, Q8_0, and Q6_K packages under GGUF/ (each with a built-in MTP head; Q3/Q4/Q2 LynnStyle use built-in Q4 MTP, not Q8) and matching NInfer packages under GGUF-NInfer/. GGUF-NInfer/ includes the BF16 vision tower (vision-bf16.safetensors) for NInfer. For llama.cpp multimodal, use --mmproj GGUF/mmproj-Qwen3.8-27B-Q8_0.gguf (Q8_0 mmproj in GGUF/ only; not required inside NInfer packages). The BF16, static FP8, NVFP4, NInfer W4A4, INT8 W8A8, and other tiers are in the main repository Qwen3.8-27B-Coder390-EfficientThink-Opus5.5-GPT6Astra-Grok4.7-DSV4Pro-K3-SFT-RLOO-MTP-DFlash2.
Related repositories
Lineage: original Qwen3.8 27B (FP8; 177 / 444 / 83) → our SFT + SimPO110 final, i.e. the EfficientThink base (171 / 442 / 89) → K3 continuation SFT → week-2 SFT (week2dose, update-225) → first RLOO round (182 groups) → another SFT round (merge-sft-100), giving sft-base-rloo (177 / 448 / 90) → second RLOO round (172 groups) → Coder390 (178 / 445 / 90, dynamic FP8). Numbers are GPQA / MMLU / LCB, all full suites under the same 100K protocol.
Training base: this model starts from our own EfficientThink SFT + SimPO model (the SimPO110 final) and goes through several rounds of SFT and RLOO, which is itself a post-trained version of the official Qwen3.8-27B: Qwen3.8-27B-EfficientThink-SFT-SimPO-DFlash2.
The problem: after the model already has a local answer, it keeps re-running the same derivation with Wait / Actually until the context is full, and the final answer is empty. In most of these cases the model can solve the problem; it just does not stop.
RLOO data: 8 trajectories sampled per problem; only groups with both correct and wrong trajectories are kept (all-correct groups are dropped). Groups with 0 or 1 correct trajectory receive one reviewed short teacher trajectory, which replaces the shortest wrong trajectory in that group (27 groups). Final set: 172 groups, 1,376 trajectories: 859 correct and 517 wrong, including 68 empty answers.
Reward: penalties apply only to wrong trajectories; long correct reasoning still earns a positive reward, so necessary long reasoning is not suppressed.
Result: under the same protocol, 94K truncations drop from 4 to 1 on GPQA and from 13 to 3 on LCB (FP8) compared with the original FP8, and none of the three scores falls below the original FP8 (all FP8 figures; see Scores below for GGUF Q6_K).
Protocol: one RTX PRO 6000, llama.cpp (llama-server) loading GGUF/Qwen3.8-27B-Coder390-EfficientThink-Q6_K-MTP.gguf, C4, 100K context, 94,208 generation cap, client not timed, reasoning effort xhigh; full GPQA 198 / MMLU 500 / LCB 100; empty answers count as wrong, truncations that still answer correctly count as correct, and every failure stays in the denominator. GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q6_K-MTP.ninfer is converted from the same GGUF with the same backbone quantization; the full triad was run only on the GGUF.
Reasoning length is usage.reasoning_tokens; a 94K truncation means the generation cap was hit. LCB empty answers are the number of problems for which no runnable code could be extracted. Bold means better than the original Qwen3.8 27B (FP8).
The original Qwen3.8 27B (FP8) was measured on two RTX PRO 6000 GPUs at C8 per GPU with SGLang; the rest of the protocol is the same.
Note: reasoning lengths for all three Q2 LynnStyle and Q8_0 suites and for Q6_K MMLU are counted from the raw text with the BF16 tokenizer (GPQA/LCB: reasoning field; MMLU: text before the last line of response), not from the API reasoning_tokens. Q3 LynnStyle MMLU / LCB, Q4 LynnStyle MMLU / LCB, and Q6_K LCB have no saved reasoning text and cannot be counted; those cells read "—".
GPQA's 2 truncations are problems 79 and 127; both hit 94,208 with an empty pred, so they also count as the empty answers. MMLU has no truncations and no empty answers.
LCB has no 94K truncations; its 1 empty answer is problem 92, which stopped normally without producing code.
LCB is scored on llama.cpp: the LCB eval script gets replies from NInfer that are not valid JSON. This is a compatibility issue between the eval script and the API, not a model-quality issue; the same GGUF on llama.cpp passed every problem in the same spot-check batch.
BPW for GGUF Q3 LynnStyle is 17303770432 × 8 ÷ the text + MTP parameter count 27,320,697,856 = 5.07; the Q3 NInfer package was 17300551168 bytes before the vision component was added, BPW 5.07. BPW for GGUF Q4 LynnStyle is 19620406592 × 8 ÷ the text + MTP parameter count 27,320,697,856 = 5.75; the Q4 NInfer package was 19617187328 bytes before the vision component was added, BPW 5.74. BPW for GGUF Q2 LynnStyle is 13,276,009,792 × 8 ÷ the text + MTP parameter count 27,320,697,856 = 3.89; the Q2 NInfer package was 13,272,798,720 bytes before the vision component was added, BPW 3.89. BPW for GGUF Q8_0 is 29,069,202,688 × 8 ÷ 27,320,697,856 = 8.51; the Q8_0 NInfer package was 29,065,983,488 bytes before the vision component was added, BPW 8.51. BPW for the two GGUF Q6_K packages is the single file's bytes × 8 ÷ the text + MTP parameter count 27,320,697,856: the GGUF is 23,177,516,384 bytes, giving 6.79, and the NInfer package was 23,084,144,128 bytes before the vision component was added, giving 6.76. The parameter count comes from the GGUF header (866 tensors: text 26,895,998,464 + MTP 424,699,392); vision is not counted in the denominator (neither the llama.cpp mmproj nor the 295,720,448-byte vision part now inside each .ninfer). The BF16 / FP8 / NVFP4 tiers in the main repository use 27,781,427,952 (including the vision tower), so BPW should not be compared directly across the two.
Setup: one RTX PRO 6000 Blackwell; 1-minute quick benchmark, 512 generation cap, thinking on (xhigh), sampling temperature 1.0, top_p 0.95, top_k 20; each level sends only as many requests as its concurrency (1 at C1, 2 at C2, 4 at C4, 8 at C8). Numbers are aggregate tok/s (all output tokens ÷ wall-clock time). This is a quick benchmark with few requests and short outputs, so the numbers are not directly comparable with benchmarks run under other protocols. NInfer rows were measured on the GGUF-NInfer/ package with the server at --max-context 32768 --kv-capacity 32768 --kv-dtype rk8v4 --max-concurrency 8; llama.cpp rows were measured on the GGUF/ package.
--max-concurrency 4). For aggregate throughput alone, C8 is higher (388.1 tok/s); raise --max-concurrency to 8 for that.GGUF Q6_K (GGUF/, GGUF-NInfer/): Q6_K backbone calibrated with a 512-chunk imatrix, plus structure protection: the linear-attention (SSM) ssm_alpha / ssm_beta stay BF16, the token embedding and output head are Q8_0, and the MTP head (blk.64 attn q/k/v/output, ffn gate/up/down, and nextn.eh_proj) is entirely Q8_0 with its norms kept in F32. The GGUF has 866 tensors in 65 blocks (64 backbone layers + 1 MTP layer). The .ninfer is converted from the same GGUF with the same backbone quantization; its MTP attention/MLP weights are Q6_K (gguf_q6_k) and only mtp/input_projection is Q8_0; the GGUF has no vision weights (llama.cpp uses GGUF/mmproj-Qwen3.8-27B-Q8_0.gguf via --mmproj); the .ninfer carries the vision component inside the package (add --vision; see NInfer · vision below).
Load the .ninfer with NInfer-all (the master branch of iamwavecut/ninfer-all): the package keeps the GGUF quantization blocks, which other NInfer builds cannot read. llama.cpp needs an upstream build that supports --spec-type draft-mtp. NInfer's --kv-capacity is one KV pool shared by all requests; llama.cpp's -c 409600 -np 4 splits the context evenly across 4 slots, so each request gets at most 102,400 tokens, enough for a single request to reach 94K. The chat endpoint is POST /v1/chat/completions; for NInfer the request model must match --model-id. Vision: GGUF/mmproj-Qwen3.8-27B-Q8_0.gguf for llama.cpp --mmproj (629,247,008 bytes, sha256 cae9799dc9196449b0d83f716e64af89d5cf65147510462e5a629a2aa23adb32). Each .ninfer package carries the vision component; add --vision to ninfer-serve to accept images (see NInfer · vision below).
MTP and DFlash2: every GGUF / NInfer package in this repository has a built-in MTP head (Q2 / Q3 / Q4 LynnStyle use built-in Q4 MTP). To enable it, add --spec mtp --draft-tokens N for NInfer or --spec-type draft-mtp --spec-draft-n-max N for llama.cpp; leave these out to run without speculation. DFlash2 is an external draft model and is not in these packages; this repository does not ship a DFlash2 draft for GGUF / NInfer. For DFlash2, use the DFlash2-FP8/ draft that ships with the BF16 / FP8 / NVFP4 tiers in the main repository (SGLang --speculative-algorithm DFLASH) or the .ninfer packages in the NVFP4-NInfer repository (--spec dflash2). MTP and DFlash2 are mutually exclusive: a server runs one or the other, never both.
NInfer · GGUF Q2 LynnStyle · MTP (draft 4, recommended concurrency 8)
NInfer · GGUF Q3 LynnStyle · MTP (draft 4, recommended concurrency 8)
NInfer · GGUF Q4 LynnStyle · MTP (draft 4, recommended concurrency 8)
NInfer · GGUF Q6_K · MTP (draft 3, recommended concurrency 4)
NInfer · GGUF Q8_0 · MTP (draft 4, recommended concurrency 8)
Do not use --max-context 131072 --kv-capacity 131072 (the KV pool then holds only one full context and queued requests return HTTP 503). Do not use --lm-head-q6 / --embedding-q4. Requires NInfer-all (ninfer-serve); official ninfer-src cannot read gguf_blocks_v1.
NInfer · vision (image input)
Every .ninfer in GGUF-NInfer/ contains the vision component (295,720,448 bytes, added on 2026-10-09). It was quantized from the BF16 vision weights in the NInfer vision formats: patch embedding Q6, attention q/k/v and MLP fc1 Q4, other projections Q5, merger Q8, norms and biases BF16. The text and MTP weights are byte-for-byte the same as before. Add --vision to any NInfer command above; nothing else is needed (the separate vision-bf16.safetensors is not used). These packages have a built-in MTP head but no DFlash2 draft, so each package has two commands: no speculation and MTP.
Image request (OpenAI-style image_url part; url takes an http(s):// image link; a data URI of a local image also works; model must match --model-id):
Send images as OpenAI-style image_url parts to POST /v1/chat/completions. --vision keeps the vision encoder on the GPU; if VRAM is tight, use --vision-cpu instead (it runs on the CPU, and each image is capped at 256 merged tokens unless you set --vision-max-merged). Without --vision the package runs text-only, as before. GGUF-NInfer/vision-bf16.safetensors is the BF16 source of the vision part and is not needed to run NInfer. Engine test (2026-10-09, one RTX PRO 6000): the Q2 LynnStyle package with --vision answered an image prompt correctly; with MTP at 3 draft tokens the accept length was 3.00 and decode speed 207 tok/s. The other tiers carry the same vision part but were not tested one by one.
llama.cpp · GGUF Q2 LynnStyle · MTP (representative; swap the -m path for Q3 / Q4 / Q6_K / Q8_0)
To use another GGUF tier, only change -m to ...-Q3LynnStyle-MTP.gguf, ...-Q4LynnStyle-MTP.gguf, ...-Q6_K-MTP.gguf, or ...-Q8_0-MTP.gguf (and the --alias, e.g. qwen3.8-27b-coder390-q6k-mtp, qwen3.8-27b-coder390-q8-mtp). MTP is fused inside the GGUF (blk.64); do not attach an external draft; the log should show creating MTP draft context against the target model. For image input add --mmproj ./GGUF/mmproj-Qwen3.8-27B-Q8_0.gguf.
llama.cpp · vision (image input)
The GGUF files have no vision weights; --mmproj ./GGUF/mmproj-Qwen3.8-27B-Q8_0.gguf adds them and works with all five GGUF files (swap -m and --alias as above). Leave out --spec-type / --spec-draft-n-max to run without speculation. Without --mmproj llama.cpp runs text-only. Images go as OpenAI-style image_url parts to POST /v1/chat/completions (same request as the NInfer example, with model set to the --alias).
Optional: external Q8_0 MTP draft. GGUF/mtp-Qwen3.8-27B-Coder390-EfficientThink-Q8_0.gguf (3,164,006,656 bytes) is a standalone Q8_0 MTP head, the same weights as the built-in MTP head of the Q6_K and Q8_0 packages. Q2, Q3 and Q4 LynnStyle ship a smaller Q4 MTP head; add -md pointing at this file to use the Q8 head instead. Q6_K and Q8_0 already contain a Q8 head, so they gain nothing from it. With -md the log shows loading draft model rather than creating MTP draft context against the target model. Checked on one RTX PRO 6000 on 2026-10-09 (llama.cpp, Q3 LynnStyle target, --spec-draft-n-max 4): the log shows loading draft model. Greedy acceptance is 66.1% at 110.2 tok/s, against 66.0% at 112.2 tok/s for the built-in Q4 MTP head with the same command and no -md. At temperature 1.0 the two are 52.4% at 94.4 tok/s and 51.9% at 95.6 tok/s. All 12 prompts produced the same text; the Q8 file is not faster.
GGUF/ (Q3/Q4/Q2/Q8/Q6 packages + Q8 MTP draft + mmproj + SHA256SUMS)GGUF-NInfer/ (Q3/Q4/Q2/Q8/Q6 packages with vision + vision-bf16.safetensors + SHA256SUMS)imatrix/ (calibration set for the GGUF imatrix)imatrix/coder390-imatrix-calibration.txt is the calibration text for recomputing the importance matrix (imatrix) of the GGUF files: 1,368 records (171 problems × 8) rebuilt from the inputs of this model's RL training rollouts, 500,816 tokens. It contains no benchmark questions (a 13-gram overlap check against MMLU, GPQA Diamond and LiveCodeBench found 0 hits). It is functionally equivalent to, but not byte-identical with, the set used for the published GGUFs; the original imatrix data file was not kept, so none is included. Commands and checks are in imatrix/README.md.
blk.64 is Q8_0; NInfer MTP attention/MLP weights are Q6_K, input projection Q8_0), so no external draft model is needed; NInfer + C4 + MTP is recommended.GGUF/Qwen3.8-27B-Coder390-EfficientThink-Q6_K-MTP.gguf: loads directly in llama.cpp; the GGUF architecture is qwen35 with 65 blocks, the last of which (blk.64) is MTP.GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q6_K-MTP.ninfer: loadable only by NInfer-all; SGLang, vLLM, llama.cpp, and transformers cannot read it.--mmproj GGUF/mmproj-Qwen3.8-27B-Q8_0.gguf (629,247,008 bytes, sha256 cae9799dc9196449b0d83f716e64af89d5cf65147510462e5a629a2aa23adb32). NInfer: each .ninfer package carries the vision component; add --vision. GGUF-NInfer/vision-bf16.safetensors (921,497,224 bytes, sha256 d7defc90994f7bbc58a8feb8d3c4144895babb434a9be9f47e88ae3646e1df33) is the BF16 source of that part and is not needed at run time.GGUF/; image path not fully re-benched on every tier.Licensed under Apache-2.0, following the license of the base model Qwen3.8-27B.
Thanks to the Qwen team for the base model and the official FP8 scheme; to Opus5.5, GPT6Astra, Grok4.7, DSV4Pro, and K3 for problem writing, gold labels, teacher trajectories, data review, and the RLOO value review; and to the llama.cpp and NInfer communities.
[Image blocked: Coder390 FP8 对比原版 FP8]
[Image blocked: Coder390 GGUF Q2 LynnStyle 对比原版 FP8]
[Image blocked: 和原版对比]
在 Qwen3.8-27B-EfficientThink-SFT-SimPO-DFlash2(官方 Qwen3.8-27B 经 SFT + SimPO 后训练得到的模型)基础上继续做多轮 SFT 与 RLOO 后训练,专治原版「已有答案却停不下来、写满 94K 也不给最终答案」的问题。本仓库是 GGUF 档的独立仓库,目前提供 GGUF/ 下的 Q3 LynnStyle、Q4 LynnStyle、Q2 LynnStyle、Q8_0 与 Q6_K 五个 GGUF 包(均内置 MTP 头;Q3/Q4/Q2 LynnStyle 内置 Q4 MTP,不是 Q8),以及 GGUF-NInfer/ 下对应的 NInfer 包。GGUF-NInfer/ 含 NInfer 用的 BF16 视觉塔(vision-bf16.safetensors)。llama.cpp 多模态请用 --mmproj GGUF/mmproj-Qwen3.8-27B-Q8_0.gguf(mmproj 只在 GGUF/,不要求放进 NInfer 包)。BF16、静态 FP8、NVFP4、NInfer W4A4、INT8 W8A8 等其他档见主仓库 Qwen3.8-27B-Coder390-EfficientThink-Opus5.5-GPT6Astra-Grok4.7-DSV4Pro-K3-SFT-RLOO-MTP-DFlash2。
相关仓库
谱系:原版 Qwen3.8 27B(FP8;177 / 444 / 83)→ 我们的 SFT + SimPO110 终版,即 EfficientThink 基座(171 / 442 / 89)→ K3 续训 SFT → 第二周 SFT(week2dose,update-225)→ 第一轮 RLOO(182 组)→ 再一轮 SFT(merge-sft-100),得 sft-base-rloo(177 / 448 / 90)→ 第二轮 RLOO(172 组)→ Coder390(178 / 445 / 90,动态 FP8 口径)。括号内依次为 GPQA / MMLU / LCB,均为 100K 同口径全量。
训练基座:本模型以我们自己的 EfficientThink SFT + SimPO 模型(SimPO110 终版)为起点,继续做多轮 SFT 与 RLOO,该基座模型本身是官方 Qwen3.8-27B 的后训练版本,见 Qwen3.8-27B-EfficientThink-SFT-SimPO-DFlash2。
要解决的问题:模型在局部已经有答案之后,仍用 Wait / Actually 把同一套推导反复再推,直到写满上下文,最终答案为空。这类题多数不是不会,而是停不下来。
RLOO 数据:每题采样 8 条轨迹,只保留组内有对有错的题(8 条全对的不入训);组内 0 或 1 条对的,补 1 条审过的短教师轨迹,替换该组最短的一条错轨迹(共 27 组)。最终 172 组、1,376 条:正确 859 条、错误 517 条,其中空答 68 条。
奖励:惩罚只打在错的一侧,长而正确的推理仍得正奖励,以免压制必要的长推理。
结果:同口径下,GPQA 的 94K 截断从原版 FP8 的 4 降到 1,LCB 从 13 降到 3,三项分数均不低于原版 FP8(均为 FP8 档;GGUF Q6_K 见下文成绩)。
口径:单张 RTX PRO 6000,llama.cpp(llama-server)加载 GGUF/Qwen3.8-27B-Coder390-EfficientThink-Q6_K-MTP.gguf,C4,上下文 100K,生成上限 94,208,客户端不计超时,思考档位 xhigh;GPQA 198、MMLU 500、LCB 100 全量;空答记错,截断但答对记对,失败样本全部留在分母里。GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q6_K-MTP.ninfer 由同一份 GGUF 转换而来,主干量化相同,全量三联只在 GGUF 上跑。
思考长度取 usage.reasoning_tokens;94K 截断指写满生成上限。LCB 空答指没有提取到可运行代码的题数。加粗表示优于原版 Qwen3.8 27B(FP8)。
原版 Qwen3.8 27B(FP8)为两张 RTX PRO 6000、每卡 C8、SGLang 测得,其余口径相同。
注:Q2 LynnStyle、Q8_0 三项与 Q6_K 的 MMLU 思考长度从原文用 BF16 tokenizer 计数(GPQA/LCB 用 reasoning 字段,MMLU 由 response 末行前文本拆分;不以接口 reasoning_tokens 为准)。Q3 LynnStyle 的 MMLU / LCB、Q4 LynnStyle 的 MMLU / LCB 与 Q6_K 的 LCB 没有保存思考原文,无法计数,对应格记「—」。
GPQA 的 2 条截断是第 79、127 题,均写满 94,208 且 pred 为空,同时记为空答。MMLU 无截断、无空答。
LCB 无 94K 截断;空答 1 条是第 92 题,正常停止但没有给出代码。
LCB 以 llama.cpp 的结果为准:LCB 测评脚本请求 NInfer 时收到的回复不是合法 JSON,这是测评脚本与接口的兼容问题,不是模型质量问题;同一份 GGUF 用 llama.cpp 跑同一批抽测题全部通过。
GGUF Q3 LynnStyle 的 BPW 按 17303770432 × 8 ÷ 文本 + MTP 参数数 27,320,697,856 = 5.07;Q3 NInfer 加视觉前为 17300551168 字节、BPW 5.07。GGUF Q4 LynnStyle 的 BPW 按 19620406592 × 8 ÷ 文本 + MTP 参数数 27,320,697,856 = 5.75;Q4 NInfer 加视觉前为 19617187328 字节、BPW 5.74。GGUF Q8_0 的 BPW 按 29,069,202,688 × 8 ÷ 文本 + MTP 参数数 27,320,697,856 = 8.51;Q8_0 NInfer 加视觉前为 29,065,983,488 字节、BPW 8.51。GGUF Q6_K 两个包的 BPW 按单个文件字节数 × 8 ÷ 文本 + MTP 参数数 27,320,697,856 计算:GGUF 为 23,177,516,384 字节、6.79,NInfer 加视觉前为 23,084,144,128 字节、6.76。参数数按 GGUF 头部统计(866 个张量:文本 26,895,998,464 + MTP 424,699,392);视觉不计入分母:llama.cpp 的外挂 mmproj 和现在打进每个 .ninfer 的视觉部分(295,720,448 字节)都不算。主仓库 BF16 / FP8 / NVFP4 各档按 27,781,427,952(含视觉塔)计算,分母不同,不宜直接横向比较 BPW。
测试环境:单张 RTX PRO 6000 Blackwell;1 分钟快测,生成上限 512,开思考(xhigh),采样 temperature 1.0、top_p 0.95、top_k 20;每档只发与并发数相同的请求数(C1 发 1 个、C2 发 2 个、C4 发 4 个、C8 发 8 个)。数字为总吞吐 tok/s(全部输出 token ÷ 墙钟时间)。这是快测:请求少、生成短,数字不能与其他口径的测速直接比较。NInfer 行用 GGUF-NInfer/ 包测得,测速时服务参数为 --max-context 32768 --kv-capacity 32768 --kv-dtype rk8v4 --max-concurrency 8;llama.cpp 行用 GGUF/ 包测得。
GGUF Q6_K(GGUF/、GGUF-NInfer/):主干 Q6_K,用 512 个分块(chunk)的 imatrix 校准,并做结构保护:线性注意力(SSM)的 ssm_alpha / ssm_beta 保持 BF16,词嵌入与输出头为 Q8_0,MTP 头(blk.64 的 attn q/k/v/output、ffn gate/up/down 与 nextn.eh_proj)全部为 Q8_0,其 norm 保持 F32。GGUF 共 866 个张量、65 个块(64 层主干 + 1 层 MTP)。.ninfer 由同一份 GGUF 转换,主干量化相同,其 MTP 的 attention/MLP 权重为 Q6_K(gguf_q6_k),仅 mtp/input_projection 为 Q8_0;GGUF 不含视觉权重(llama.cpp 用 GGUF/mmproj-Qwen3.8-27B-Q8_0.gguf,加 --mmproj);.ninfer 包内已含视觉部分(加 --vision,见下文「NInfer · 视觉」)。
.ninfer 用 NInfer-all(iamwavecut/ninfer-all 的 master 分支)加载:包里保留的是 GGUF 量化块,其他 NInfer 构建不能读。llama.cpp 需支持 --spec-type draft-mtp 的上游版本。NInfer 的 --kv-capacity 是所有请求共享的 KV 池;llama.cpp 的 -c 409600 -np 4 会把上下文平分给 4 个槽位,每个请求最多 102,400 token,够单请求写到 94K。对话接口为 POST /v1/chat/completions;NInfer 请求里的 model 要与 --model-id 一致。视觉:llama.cpp 用 GGUF/mmproj-Qwen3.8-27B-Q8_0.gguf(--mmproj;629,247,008 字节,sha256 cae9799dc9196449b0d83f716e64af89d5cf65147510462e5a629a2aa23adb32)。每个 .ninfer 包内都含视觉部分;ninfer-serve 加 --vision 即可输入图片(见下文「NInfer · 视觉」)。
MTP 与 DFlash2:本仓库所有 GGUF / NInfer 包都内置 MTP 头(Q2 / Q3 / Q4 LynnStyle 为内置 Q4 MTP)。开启方式:NInfer 加 --spec mtp --draft-tokens N,llama.cpp 加 --spec-type draft-mtp --spec-draft-n-max N;省略这两个参数即不开投机。DFlash2 是外挂草稿模型,不在本仓库这些包里,本仓库不提供 GGUF / NInfer 用的 DFlash2 草稿;需要 DFlash2 时请用主仓 BF16 / FP8 / NVFP4 各档自带的 DFlash2-FP8/(SGLang --speculative-algorithm DFLASH)或 NVFP4-NInfer 仓库的 .ninfer 包(--spec dflash2)。MTP 与 DFlash2 互斥:同一个服务只能开其中一种,不要同时开。
NInfer · GGUF Q2 LynnStyle · MTP(草稿长度 4,推荐并发 8)
NInfer · GGUF Q3 LynnStyle · MTP(草稿长度 4,推荐并发 8)
NInfer · GGUF Q4 LynnStyle · MTP(草稿长度 4,推荐并发 8)
NInfer · GGUF Q6_K · MTP(草稿长度 3,推荐并发 4)
NInfer · GGUF Q8_0 · MTP(草稿长度 4,推荐并发 8)
不要用 --max-context 131072 --kv-capacity 131072(KV 池不够多请求排队时会 HTTP 503)。不要加 --lm-head-q6 / --embedding-q4。需 NInfer-all 的 ninfer-serve;官方 ninfer-src 读不了 gguf_blocks_v1。
NInfer · 视觉(图片输入)
GGUF-NInfer/ 里每个 .ninfer 都已打包视觉部分(295,720,448 字节,2026-10-09 加入)。视觉部分由 BF16 视觉权重按 NInfer 的视觉格式量化:patch embedding 为 Q6,attention q/k/v 与 MLP fc1 为 Q4,其余投影为 Q5,merger 为 Q8,norm 与 bias 保持 BF16。文本与 MTP 权重和之前逐字节相同。在上面任一 NInfer 命令后加 --vision 即可,不需要别的文件(单独的 vision-bf16.safetensors 用不到)。这些包内置 MTP 头,不含 DFlash2 草稿,所以每个包两条命令:不开投机、开 MTP。
图片请求(按 OpenAI 格式放 image_url;url 填图片的 http(s):// 链接,也可以传本地图片的数据 URI;model 要与 --model-id 一致):
图片按 OpenAI 格式放在 POST /v1/chat/completions 的 image_url 里。--vision 把视觉编码器放在 GPU 上;显存紧张时改用 --vision-cpu(在 CPU 上跑,单张图默认最多 256 个合并 token,可用 --vision-max-merged 调整)。不加 --vision 时和以前一样只跑文本。GGUF-NInfer/vision-bf16.safetensors 是视觉部分的 BF16 原始权重,NInfer 运行时不需要它。引擎实测(2026-10-09,单张 RTX PRO 6000):Q2 LynnStyle 包加 --vision 正确回答了看图问题;开 MTP、3 个草稿 token 时接受长度 3.00,decode 207 tok/s。其他档带的是同一份视觉部分,未逐档实测。
llama.cpp · GGUF Q2 LynnStyle · MTP(代表命令;换 -m 即可切到 Q3 / Q4 / Q6_K / Q8_0)
换其他 GGUF 档只需改 -m 为 ...-Q3LynnStyle-MTP.gguf、...-Q4LynnStyle-MTP.gguf、...-Q6_K-MTP.gguf 或 ...-Q8_0-MTP.gguf(并相应改 --alias,例如 qwen3.8-27b-coder390-q6k-mtp、qwen3.8-27b-coder390-q8-mtp)。MTP 已融合进 GGUF(blk.64),不要再外挂 draft;日志应出现 creating MTP draft context against the target model。需要图像输入时加 --mmproj ./GGUF/mmproj-Qwen3.8-27B-Q8_0.gguf。
llama.cpp · 视觉(图片输入)
GGUF 文件本身不含视觉权重,加 --mmproj ./GGUF/mmproj-Qwen3.8-27B-Q8_0.gguf 即可看图,五个 GGUF 都用这同一个 mmproj(按上文换 -m 和 --alias)。去掉 --spec-type / --spec-draft-n-max 即不开投机。不加 --mmproj 时 llama.cpp 只跑文本。图片按 OpenAI 格式放在 POST /v1/chat/completions 的 image_url 里(请求与 NInfer 示例相同,model 填 --alias)。
可选:外挂 Q8_0 MTP 草稿。 GGUF/mtp-Qwen3.8-27B-Coder390-EfficientThink-Q8_0.gguf(3,164,006,656 字节)是单独的 Q8_0 MTP 头,权重和 Q6_K、Q8_0 包里内置的 MTP 头相同。Q2、Q3、Q4 LynnStyle 自带的是更小的 Q4 MTP 头,想换成 Q8 头就加 -md 指向这个文件。Q6_K 和 Q8_0 已经内置 Q8 头,外挂没有额外收益。加了 -md 之后,日志应出现 loading draft model,而不是 creating MTP draft context against the target model。2026-10-09 在一台 RTX PRO 6000 上用 llama.cpp 实测：目标是 Q3 LynnStyle，--spec-draft-n-max 4，日志出现 loading draft model。贪心接受率 66.1%、110.2 tok/s；不加 -md、用内置 Q4 MTP 头是 66.0%、112.2 tok/s。temperature 1.0 时是 52.4%、94.4 tok/s，对照 51.9%、95.6 tok/s。12 条提示词输出一致，Q8 文件没有更快。
GGUF/(含 Q3/Q4/Q2/Q8/Q6 包 + Q8 MTP 草稿 + mmproj + SHA256SUMS)GGUF-NInfer/(含 Q3/Q4/Q2/Q8/Q6 包(含视觉) + vision-bf16.safetensors + SHA256SUMS)imatrix/(GGUF imatrix 校正集)imatrix/coder390-imatrix-calibration.txt 是重算 GGUF 重要性矩阵(imatrix)用的校正文本:用本模型 RL 训练 rollout 的输入重新整理,1,368 条(171 道题 × 8),500,816 token。不含测评题(对照 MMLU、GPQA Diamond、LiveCodeBench 做 13-gram 重叠检查,命中 0 条)。功能上与发布 GGUF 时用的校正集等价,但不是同一个文件;原 imatrix 数据文件没有保留,所以不附。命令与一致性检查见 imatrix/README.md。
blk.64 为 Q8_0;NInfer 的 MTP attention/MLP 权重为 Q6_K,输入投影为 Q8_0),不需要外挂草稿模型;推荐 NInfer + C4 + MTP。GGUF/Qwen3.8-27B-Coder390-EfficientThink-Q6_K-MTP.gguf:llama.cpp 直接加载,GGUF 架构为 qwen35,65 个块,最后一块(blk.64)为 MTP。GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q6_K-MTP.ninfer:只能用 NInfer-all 加载;SGLang、vLLM、llama.cpp、transformers 都不能读。--mmproj GGUF/mmproj-Qwen3.8-27B-Q8_0.gguf(629,247,008 字节,sha256 cae9799dc9196449b0d83f716e64af89d5cf65147510462e5a629a2aa23adb32)。NInfer:每个 .ninfer 包内已含视觉部分,加 --vision 即可;GGUF-NInfer/ 里的 vision-bf16.safetensors(921,497,224 字节,sha256 d7defc90994f7bbc58a8feb8d3c4144895babb434a9be9f47e88ae3646e1df33)是这部分的 BF16 原始权重,运行时不需要。GGUF/ 已提供 mmproj;各档图像路径未全部重测。本模型采用 Apache-2.0 许可证,遵循基座 Qwen3.8-27B 的许可。
感谢 Qwen 团队提供基座模型与官方 FP8 方案;感谢 Opus5.5、GPT6Astra、Grok4.7、DSV4Pro、K3 在出题、金标、教师轨迹、数据核对与 RLOO 价值评审中的工作;感谢 llama.cpp 与 NInfer 社区。
ninfer-serve ./GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q2LynnStyle-MTP.ninfer \ --model-id qwen3.8-27b-coder390-Q2LynnStyle-Mtp \ --max-context 102400 --kv-capacity 819200 --kv-dtype rk8v4 \ --max-concurrency 8 --spec mtp --draft-tokens 4ninfer-serve ./GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q3LynnStyle-MTP.ninfer \ --host 127.0.0.1 --port 8080 --device 0 \ --model-id qwen3.8-27b-coder390-Q3LynnStyle-Mtp \ --max-context 102400 --kv-capacity 819200 --max-concurrency 8 \ --kv-dtype rk8v4 \ --spec mtp --draft-tokens 4ninfer-serve ./GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q4LynnStyle-MTP.ninfer \ --host 127.0.0.1 --port 8080 --device 0 \ --model-id qwen3.8-27b-coder390-Q4LynnStyle-Mtp \ --max-context 102400 --kv-capacity 819200 --max-concurrency 8 \ --kv-dtype rk8v4 \ --spec mtp --draft-tokens 4ninfer-serve ./GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q6_K-MTP.ninfer \ --model-id qwen3.8-27b-coder390-q6k-mtp \ --max-context 102400 --kv-capacity 409600 --kv-dtype rk8v4 \ --max-concurrency 4 --spec mtp --draft-tokens 3ninfer-serve ./GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q8_0-MTP.ninfer \ --model-id qwen3.8-27b-coder390-q8-mtp \ --max-context 102400 --kv-capacity 819200 --kv-dtype rk8v4 \ --max-concurrency 8 --spec mtp --draft-tokens 4# Q2LynnStyle · no speculationninfer-serve ./GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q2LynnStyle-MTP.ninfer \ --host 127.0.0.1 --port 8080 --device 0 \ --model-id qwen3.8-27b-coder390-Q2LynnStyle-Mtp \ --max-context 102400 --kv-capacity 819200 --max-concurrency 8 \ --kv-dtype rk8v4 \ --vision
# Q2LynnStyle · MTP (4 draft tokens)ninfer-serve ./GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q2LynnStyle-MTP.ninfer \ --host 127.0.0.1 --port 8080 --device 0 \ --model-id qwen3.8-27b-coder390-Q2LynnStyle-Mtp \ --max-context 102400 --kv-capacity 819200 --max-concurrency 8 \ --kv-dtype rk8v4 \ --spec mtp --draft-tokens 4 \ --vision
# Q3LynnStyle · no speculationninfer-serve ./GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q3LynnStyle-MTP.ninfer \ --host 127.0.0.1 --port 8080 --device 0 \ --model-id qwen3.8-27b-coder390-Q3LynnStyle-Mtp \ --max-context 102400 --kv-capacity 819200 --max-concurrency 8 \ --kv-dtype rk8v4 \ --vision
# Q3LynnStyle · MTP (4 draft tokens)ninfer-serve ./GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q3LynnStyle-MTP.ninfer \ --host 127.0.0.1 --port 8080 --device 0 \ --model-id qwen3.8-27b-coder390-Q3LynnStyle-Mtp \ --max-context 102400 --kv-capacity 819200 --max-concurrency 8 \ --kv-dtype rk8v4 \ --spec mtp --draft-tokens 4 \ --vision
# Q4LynnStyle · no speculationninfer-serve ./GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q4LynnStyle-MTP.ninfer \ --host 127.0.0.1 --port 8080 --device 0 \ --model-id qwen3.8-27b-coder390-Q4LynnStyle-Mtp \ --max-context 102400 --kv-capacity 819200 --max-concurrency 8 \ --kv-dtype rk8v4 \ --vision
# Q4LynnStyle · MTP (4 draft tokens)ninfer-serve ./GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q4LynnStyle-MTP.ninfer \ --host 127.0.0.1 --port 8080 --device 0 \ --model-id qwen3.8-27b-coder390-Q4LynnStyle-Mtp \ --max-context 102400 --kv-capacity 819200 --max-concurrency 8 \ --kv-dtype rk8v4 \ --spec mtp --draft-tokens 4 \ --vision
# Q6_K · no speculationninfer-serve ./GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q6_K-MTP.ninfer \ --host 127.0.0.1 --port 8080 --device 0 \ --model-id qwen3.8-27b-coder390-q6k-mtp \ --max-context 102400 --kv-capacity 409600 --max-concurrency 4 \ --kv-dtype rk8v4 \ --vision
# Q6_K · MTP (3 draft tokens)ninfer-serve ./GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q6_K-MTP.ninfer \ --host 127.0.0.1 --port 8080 --device 0 \ --model-id qwen3.8-27b-coder390-q6k-mtp \ --max-context 102400 --kv-capacity 409600 --max-concurrency 4 \ --kv-dtype rk8v4 \ --spec mtp --draft-tokens 3 \ --vision
# Q8_0 · no speculationninfer-serve ./GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q8_0-MTP.ninfer \ --host 127.0.0.1 --port 8080 --device 0 \ --model-id qwen3.8-27b-coder390-q8-mtp \ --max-context 102400 --kv-capacity 819200 --max-concurrency 8 \ --kv-dtype rk8v4 \ --vision
# Q8_0 · MTP (4 draft tokens)ninfer-serve ./GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q8_0-MTP.ninfer \ --host 127.0.0.1 --port 8080 --device 0 \ --model-id qwen3.8-27b-coder390-q8-mtp \ --max-context 102400 --kv-capacity 819200 --max-concurrency 8 \ --kv-dtype rk8v4 \ --spec mtp --draft-tokens 4 \ --visioncurl http://127.0.0.1:8080/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "qwen3.8-27b-coder390-Q4LynnStyle-Mtp", "messages": [{"role": "user", "content": [ {"type": "image_url", "image_url": {"url": "https://example.com/demo.png"}}, {"type": "text", "text": "Describe this image."} ]}], "max_tokens": 2048 }'llama-server -m ./GGUF/Qwen3.8-27B-Coder390-EfficientThink-Q2LynnStyle-MTP.gguf \ -c 409600 -np 4 -ngl 99 --jinja \ --host 127.0.0.1 --port 8080 \ --alias qwen3.8-27b-coder390-Q2LynnStyle-Mtp \ --spec-type draft-mtp --spec-draft-n-max 4llama-server -m ./GGUF/Qwen3.8-27B-Coder390-EfficientThink-Q2LynnStyle-MTP.gguf \ --mmproj ./GGUF/mmproj-Qwen3.8-27B-Q8_0.gguf \ -c 409600 -np 4 -ngl 99 --jinja \ --host 127.0.0.1 --port 8080 \ --alias qwen3.8-27b-coder390-Q2LynnStyle-Mtp \ --spec-type draft-mtp --spec-draft-n-max 4llama-server -m ./GGUF/Qwen3.8-27B-Coder390-EfficientThink-Q3LynnStyle-MTP.gguf \ -md ./GGUF/mtp-Qwen3.8-27B-Coder390-EfficientThink-Q8_0.gguf \ -c 409600 -np 4 -ngl 99 --jinja \ --host 127.0.0.1 --port 8080 \ --alias qwen3.8-27b-coder390-Q3LynnStyle-Mtp \ --spec-type draft-mtp --spec-draft-n-max 4(cd GGUF && sha256sum -c SHA256SUMS)(cd GGUF-NInfer && sha256sum -c SHA256SUMS)(cd imatrix && sha256sum -c SHA256SUMS)ninfer-serve ./GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q2LynnStyle-MTP.ninfer \ --model-id qwen3.8-27b-coder390-Q2LynnStyle-Mtp \ --max-context 102400 --kv-capacity 819200 --kv-dtype rk8v4 \ --max-concurrency 8 --spec mtp --draft-tokens 4ninfer-serve ./GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q3LynnStyle-MTP.ninfer \ --host 127.0.0.1 --port 8080 --device 0 \ --model-id qwen3.8-27b-coder390-Q3LynnStyle-Mtp \ --max-context 102400 --kv-capacity 819200 --max-concurrency 8 \ --kv-dtype rk8v4 \ --spec mtp --draft-tokens 4ninfer-serve ./GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q4LynnStyle-MTP.ninfer \ --host 127.0.0.1 --port 8080 --device 0 \ --model-id qwen3.8-27b-coder390-Q4LynnStyle-Mtp \ --max-context 102400 --kv-capacity 819200 --max-concurrency 8 \ --kv-dtype rk8v4 \ --spec mtp --draft-tokens 4ninfer-serve ./GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q6_K-MTP.ninfer \ --model-id qwen3.8-27b-coder390-q6k-mtp \ --max-context 102400 --kv-capacity 409600 --kv-dtype rk8v4 \ --max-concurrency 4 --spec mtp --draft-tokens 3ninfer-serve ./GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q8_0-MTP.ninfer \ --model-id qwen3.8-27b-coder390-q8-mtp \ --max-context 102400 --kv-capacity 819200 --kv-dtype rk8v4 \ --max-concurrency 8 --spec mtp --draft-tokens 4# Q2LynnStyle · 不开投机ninfer-serve ./GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q2LynnStyle-MTP.ninfer \ --host 127.0.0.1 --port 8080 --device 0 \ --model-id qwen3.8-27b-coder390-Q2LynnStyle-Mtp \ --max-context 102400 --kv-capacity 819200 --max-concurrency 8 \ --kv-dtype rk8v4 \ --vision
# Q2LynnStyle · MTP(草稿长度 4)ninfer-serve ./GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q2LynnStyle-MTP.ninfer \ --host 127.0.0.1 --port 8080 --device 0 \ --model-id qwen3.8-27b-coder390-Q2LynnStyle-Mtp \ --max-context 102400 --kv-capacity 819200 --max-concurrency 8 \ --kv-dtype rk8v4 \ --spec mtp --draft-tokens 4 \ --vision
# Q3LynnStyle · 不开投机ninfer-serve ./GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q3LynnStyle-MTP.ninfer \ --host 127.0.0.1 --port 8080 --device 0 \ --model-id qwen3.8-27b-coder390-Q3LynnStyle-Mtp \ --max-context 102400 --kv-capacity 819200 --max-concurrency 8 \ --kv-dtype rk8v4 \ --vision
# Q3LynnStyle · MTP(草稿长度 4)ninfer-serve ./GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q3LynnStyle-MTP.ninfer \ --host 127.0.0.1 --port 8080 --device 0 \ --model-id qwen3.8-27b-coder390-Q3LynnStyle-Mtp \ --max-context 102400 --kv-capacity 819200 --max-concurrency 8 \ --kv-dtype rk8v4 \ --spec mtp --draft-tokens 4 \ --vision
# Q4LynnStyle · 不开投机ninfer-serve ./GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q4LynnStyle-MTP.ninfer \ --host 127.0.0.1 --port 8080 --device 0 \ --model-id qwen3.8-27b-coder390-Q4LynnStyle-Mtp \ --max-context 102400 --kv-capacity 819200 --max-concurrency 8 \ --kv-dtype rk8v4 \ --vision
# Q4LynnStyle · MTP(草稿长度 4)ninfer-serve ./GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q4LynnStyle-MTP.ninfer \ --host 127.0.0.1 --port 8080 --device 0 \ --model-id qwen3.8-27b-coder390-Q4LynnStyle-Mtp \ --max-context 102400 --kv-capacity 819200 --max-concurrency 8 \ --kv-dtype rk8v4 \ --spec mtp --draft-tokens 4 \ --vision
# Q6_K · 不开投机ninfer-serve ./GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q6_K-MTP.ninfer \ --host 127.0.0.1 --port 8080 --device 0 \ --model-id qwen3.8-27b-coder390-q6k-mtp \ --max-context 102400 --kv-capacity 409600 --max-concurrency 4 \ --kv-dtype rk8v4 \ --vision
# Q6_K · MTP(草稿长度 3)ninfer-serve ./GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q6_K-MTP.ninfer \ --host 127.0.0.1 --port 8080 --device 0 \ --model-id qwen3.8-27b-coder390-q6k-mtp \ --max-context 102400 --kv-capacity 409600 --max-concurrency 4 \ --kv-dtype rk8v4 \ --spec mtp --draft-tokens 3 \ --vision
# Q8_0 · 不开投机ninfer-serve ./GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q8_0-MTP.ninfer \ --host 127.0.0.1 --port 8080 --device 0 \ --model-id qwen3.8-27b-coder390-q8-mtp \ --max-context 102400 --kv-capacity 819200 --max-concurrency 8 \ --kv-dtype rk8v4 \ --vision
# Q8_0 · MTP(草稿长度 4)ninfer-serve ./GGUF-NInfer/Qwen3.8-27B-Coder390-EfficientThink-Q8_0-MTP.ninfer \ --host 127.0.0.1 --port 8080 --device 0 \ --model-id qwen3.8-27b-coder390-q8-mtp \ --max-context 102400 --kv-capacity 819200 --max-concurrency 8 \ --kv-dtype rk8v4 \ --spec mtp --draft-tokens 4 \ --visioncurl http://127.0.0.1:8080/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "qwen3.8-27b-coder390-Q4LynnStyle-Mtp", "messages": [{"role": "user", "content": [ {"type": "image_url", "image_url": {"url": "https://example.com/demo.png"}}, {"type": "text", "text": "Describe this image."} ]}], "max_tokens": 2048 }'llama-server -m ./GGUF/Qwen3.8-27B-Coder390-EfficientThink-Q2LynnStyle-MTP.gguf \ -c 409600 -np 4 -ngl 99 --jinja \ --host 127.0.0.1 --port 8080 \ --alias qwen3.8-27b-coder390-Q2LynnStyle-Mtp \ --spec-type draft-mtp --spec-draft-n-max 4llama-server -m ./GGUF/Qwen3.8-27B-Coder390-EfficientThink-Q2LynnStyle-MTP.gguf \ --mmproj ./GGUF/mmproj-Qwen3.8-27B-Q8_0.gguf \ -c 409600 -np 4 -ngl 99 --jinja \ --host 127.0.0.1 --port 8080 \ --alias qwen3.8-27b-coder390-Q2LynnStyle-Mtp \ --spec-type draft-mtp --spec-draft-n-max 4llama-server -m ./GGUF/Qwen3.8-27B-Coder390-EfficientThink-Q3LynnStyle-MTP.gguf \ -md ./GGUF/mtp-Qwen3.8-27B-Coder390-EfficientThink-Q8_0.gguf \ -c 409600 -np 4 -ngl 99 --jinja \ --host 127.0.0.1 --port 8080 \ --alias qwen3.8-27b-coder390-Q3LynnStyle-Mtp \ --spec-type draft-mtp --spec-draft-n-max 4(cd GGUF && sha256sum -c SHA256SUMS)(cd GGUF-NInfer && sha256sum -c SHA256SUMS)(cd imatrix && sha256sum -c SHA256SUMS)