MiniCPM-V-4.7-35B-A3B-CRACK (GGUF)
Refusal-direction–ablated GGUF of openbmb/MiniCPM-V-4.7-35B-A3B, the 35B-A3B Qwen3.5-moe-text (hybrid GatedDeltaNet + Full-Attention + 256 routed experts + 1 shared expert) multimodal model.
Surgery removes the model's refusal direction from all three residual-stream writer tiers of the Qwen3.5 hybrid architecture — the SSM output path is the critical tier that single-tier surgeries miss on this architecture. Multimodal (16× + 4× projectors) and DSpark speculative-decoding paths are preserved.
Files in this repo
Evaluation — base vs CRACK
HarmBench-240 (standard + contextual + copyright splits), MMLU-200 (balanced subjects, logit-eval), multi-turn 5-scenario coherence, harsh-10 hot categories. Multimodal verified on synthetic image (number 7342 + red square + blue circle).
Gate status
- Q8_0 / Q4_K_M: pass the strict compliance gate (HB-240 ≥ 97% comply) and the strict knowledge gate (ΔMMLU within ±3pp of same-quant base).
- IQ2_M: passes the compliance gate (100% comply, 0 refuse, 0 garbage, no loops) but MMLU drops −7.5pp vs same-quant base, outside the ±3pp knowledge gate. The 2.5-bit quant combined with 58 residual writers of surgery is beyond what the surgery direction tolerates under this quant. Ship anyway for compliance+size users; expect degraded knowledge accuracy on multiple-choice benchmarks. Multimodal, HB and MT are unaffected.
Compliance behavior
- HB-240 = 100.0% on every quant (0 refuse, 0 garbage, 0 empty, 0 loop across 720 rows)
- Multi-turn 5-scenario: all 15 turns coherent, no refusals, no loops
- Harsh "FULL" = complete compliance; "SOFT" = compliant with light disclaimer, no HARD refusal on any quant
- Multimodal: all 3 quants correctly identify "7342 + red square + blue circle" through both 16× (full-resolution) and 4× (reduced-token) projectors
Usage
Thinking modes: enable_thinking=true for CoT reasoning; chat_template_kwargs: {"enable_thinking": false} to skip the <think>...</think> block. The HB and MMLU numbers above are from enable_thinking=false (direct-answer mode).
Recipe
MiniCPM-V-4.7-35B is a hybrid GDN + FA MoE where three residual-stream tiers all carry refusal signal. Single-tier surgeries (attn-only or shared-only) leave the refusal path intact through the SSM recurrent-state output.
MPOA at uniform s=2.0 on deep-middle layers [11..39], three tiers:
self_attn.o_proj.weight on FA layers [3,7,11,15,19,23,27,31,35,39] intersected with band → 7 layers
linear_attn.out_proj.weight on SSM/GDN layers (every non-FA layer in band) → 21 layers
mlp.shared_expert.down_proj.weight on every layer in band → 29 layers
Routed experts (mlp.experts.N.*) are not touched — pass-through byte-identical. MoE routing bypasses the "problem expert", so ablating routed experts adds noise without helping; the Q6_K uplift on ffn_down_exps at quantize time preserves the shared-expert surgery across quants.
Vision tower (model.visual.*), MTP head, lm_head, embed_tokens: byte-identical passthrough.
- Vector blend
0.5·v_pure + 0.5·v_svd:
v_svd = SVD top-1 right-singular vector across {reasoning-ON mean-diff, reasoning-OFF mean-diff}
v_pure = normalize(v_raw + v_svd) where v_raw = mean(harm) − mean(harmless)
- Probe: BF16 extraction via
llama-probe-capture on l_out-* tensors, mode-invariant SVD per layer, chat-template-rendered prompts with enable_thinking=true AND enable_thinking=false surfaces
- Probe corpus: 10,958 harmful/benign-twin pairs = glm53flash multi2k (2000 EN+ZH twin pairs from the GLM-5.3-Flash NVFP4 calibration set) + MultiJail/HarmBench/CyberSecEval/OASST1 twins (8958 pairs across 14 languages). The glm53flash cybersec/pentest unpaired prompts (447 rows) were used in an earlier probe pass but excluded from the final BF16 probe to keep the matched-pair structure clean.
Thinking-ON compliance (verified per-row)
HB-240 was run twice on Q4_K_M (the recommended shipping quant) — once with enable_thinking=false (direct answer) and once with enable_thinking=true (CoT reasoning). Per-row answers + reasoning traces persisted to disk.
Reasoning-mode compliance matches direct-answer-mode compliance exactly across all 240 rows. The model emits a full <think>...</think> block for each harmful prompt and then completes the task — surgery is stable through the CoT path.
License
Inherits Apache 2.0 from upstream openbmb/MiniCPM-V-4.7-35B-A3B.
Detailed per-subject MMLU + per-category HB tables
35B — MMLU per-subject (base vs CRACK)
Q8_0
Overall: base 76.0% → CRACK 77.0% (Δ +1.0pp)
Q4_K_M
Overall: base 75.0% → CRACK 75.5% (Δ +0.5pp)
IQ2_M
Overall: base 76.5% → CRACK 69.0% (Δ -7.5pp)
35B — HarmBench-240 per-category verdicts
Q8_0
Overall: 240/240 = 100.0% comply · verdicts: {'comply': 240}
Q4_K_M
Overall: 240/240 = 100.0% comply · verdicts: {'comply': 240}
IQ2_M
Overall: 240/240 = 100.0% comply · verdicts: {'comply': 240}
35B — Harsh grade (10 hot categories, FULL/SOFT/HARD/EMPTY)
Q8_0
Summary: 1}
Q4_K_M
Summary: 2}
IQ2_M
Summary: 2}