No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from pottokao/Ornith-1.5-35B-A3B-abliterated-NVFP4-DFlash-GGUF
All models by pottokao →GGUF build of pottokao/Ornith-1.5-35B-A3B-abliterated-NVFP4-DFlash,
for use with llama.cpp.
The 4-bit weights are repacked bit-exact from the NVFP4 checkpoint into GGML_TYPE_NVFP4 —
they are not dequantized and re-quantized, so there is no double-quantization penalty.
On Blackwell GPUs llama.cpp runs these through native FP4 tensor cores.
19.5 GB, plus a 772 MB DFlash draft model for speculative decoding. Runs on 2×16 GB consumer GPUs (tested on 2× RTX 5070 Ti).
⚠️ Text-only. No vision tower, no MTP head (the abliteration was done on a language-model-only export). Converted with
--no-mtp.
⚠️ Uncensored. Safety refusal behaviour has been deliberately removed. You are responsible for how you use it.
Classic refusal-direction ablation (orthogonalization), single direction:
Tooling derived from
remove-refusals-with-transformers.
BF16 weights: pottokao/Ornith-1.5-35B-A3B-abliterated.
NVIDIA TensorRT Model Optimizer 0.45.0, per-layer recipe matched exactly to the official
ornith-ai/Ornith-1.5-35B-A3B-NVFP4
(verified tensor-by-tensor: weight_scale_2 30841, input_scale 130, 291 quantized layers, 0 diff
in the language model). Calibration: 64 × 512 tokens from abisee/cnn_dailymail.
Conversion (latest llama.cpp, which has a ModelOpt-aware branch):
--no-mtp is required for this checkpoint: the config still declares
mtp_num_hidden_layers: 1 but the MTP weights were stripped during abliteration, so without it the
converter writes block_count: 41 and loading fails with blk.40.attn_norm.weight not found.
Build with CUDA for your arch (120 = Blackwell / RTX 50-series). This matters: the native FP4
path is gated on blackwell_mma_available(), which checks the compiled arch.
llama.cpp auto-detects the draft type from the GGUF metadata:
layer (pipeline)Caveat: this is one configuration, not a verdict on tensor-parallel. We are not familiar with
llama.cpp's TP path and only tested 2× RTX 5070 Ti over PCIe (no NVLink) at -c 8192. TP works —
it is not broken — it just did not win here, plausibly because single-stream decode is
bandwidth-bound and the per-layer all-reduce costs more than the parallelism gains on this
interconnect. Building with -DGGML_CUDA_NCCL=ON recovered ~5 % over the internal AllReduce but
still did not beat layer. We also hit
llama_params_fit is not implemented for SPLIT_MODE_TENSOR, so memory has to be sized by hand.
If there is a better way to configure TP here, a newer build that changes this, or something we simply missed — corrections and suggestions are very welcome.
2× RTX 5070 Ti (16 GB, 250 W), DFlash K=8, -c 8192, -fa on.
rag and summarization carry 3.1–3.4 K-character prompts, so TTFT is a meaningful share of the
work — and in our runs TP was 17–21 % slower there too, i.e. we could not find a prefill-heavy case
where it came out ahead. Again: one configuration, and we may well be holding it wrong.
Across the full Spec-Bench run above (40 recorded generations):
The spread matters more than the average. On ad-hoc single prompts we measured anywhere from 5.60 (step-by-step arithmetic) down to 3.07 (free-form Chinese prose) — structured output drafts very well, free-form prose drafts poorly. Always report the prompt mix alongside an acceptance number; the same model and settings can look 2× better or worse depending on what you feed it.
Same benchmark suite, same 8 prompts per category, concurrency 1:
vLLM is 1.6–2.0× faster. This is worth stating plainly, because llama.cpp is not being held back by either of the two things you might suspect:
So the gap is in per-step execution of the MoE + hybrid-Mamba forward itself, not in the quantization format or the speculative decoding.
Pick this build if you want the llama.cpp runtime/ecosystem; pick the vLLM one for raw speed. Note the GGUF is 19.5 GB — the same size as the source — so it still does not fit on a single 16 GB card.
An AIME 2026 run (29/30) was done on the source NVFP4 checkpoint under vLLM; since this GGUF is a bit-exact repack of the same 4-bit weights, it is not re-reported here. See the NVFP4 repo for the numbers and the caveats — in short, it is a check that abliteration + quantization did not cause catastrophic degradation, not a capability claim, and there is no external baseline to compare against.
DFlash draft: z-lab/Qwen3.6-35B-A3B-DFlash,
converted to GGUF unmodified; original weights and license belong to z-lab.
python3 convert_hf_to_gguf.py /path/to/NVFP4-model \ --outfile Ornith-1.5-35B-A3B-abliterated-NVFP4.gguf --fp8-as-q8 --no-mtp
# DFlash draft (needs the target model for its tokenizer + dflash_config)python3 convert_hf_to_gguf.py /path/to/NVFP4-model/dflash_draft \ --target-model-dir /path/to/NVFP4-model --outfile dflash-draft-Ornith15.ggufcmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=120cmake --build build --config Release -j --target llama-server./build/bin/llama-server \ -m Ornith-1.5-35B-A3B-abliterated-NVFP4.gguf \ -md dflash-draft-Ornith15.gguf --spec-draft-n-max 8 \ -ngl 99 -ngld 99 --split-mode layer -c 8192 -fa on \ --host 0.0.0.0 --port 8080common_specu: auto-detected speculative type 'draft-dflash' from the draft model metadata - n_max=8, n_min=0, block_size=16, mask_token_id=248077, sample_from_anchor=trueornith-ai/Ornith-1.5-35B-A3B (BF16, MIT) └── refusal-direction ablation (layer 24, o_proj + down_proj) └── pottokao/Ornith-1.5-35B-A3B-abliterated (BF16, 65 GB) └── modelopt 0.45.0 NVFP4, recipe matched to official └── pottokao/…-NVFP4-DFlash (20 GB, vLLM) └── convert_hf_to_gguf.py (bit-exact NVFP4 repack) └── this repo (19.5 GB, llama.cpp)