No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from AtomicChat/Ornith-1.5-9B-GGUF
All models by AtomicChat →Built from Ornith's original weights with our own importance matrix. The calibration corpora and raw eval logs behind this card are public.
GGUF builds of ornith-ai/Ornith-1.5-9B, converted and measured by AtomicChat.
Sizes below already include the 0.9 GB vision projector and about 1 GB of runtime overhead at 8k context. Drop the projector if you only want text.
[!WARNING] The two smallest files are published for completeness, not because we recommend them.
AD-IQ2_XXS-IQ1_Magrees with the reference on 53.7% of tokens, which means roughly every second token is a different word than the full model would have produced. That is not a small quality drop, it is a different model. If a 9B does not fit, a smaller model at higher precision will serve you better.
Mean KL divergence against the BF16 reference, held-out eval_neutral corpus,
4096 context, single RTX 5090, llama.cpp b10505. Lower is better. Top-1 is the
share of tokens where the quant picks the same next token as BF16.
AD means a per tensor bit layout tuned for this model. Files without the
prefix are stock llama.cpp presets built with the same importance matrix, so the
comparison isolates the layout and nothing else.
Two results are worth reading off that chart directly.
AD-Q5_K-Q4_K is smaller than stock Q5_K_M and more accurate at the same
time, 5.93 GB against 6.47 GB and 0.0255 against 0.0299. No interpolation
needed to see it.
At the same size, the tuned layout cuts divergence by about a third.
AD-Q4_K-IQ4_XS is 5.61 GB at 0.0344 against stock Q4_K_M at 5.63 GB and
0.0500.
[!NOTE] The tuning only pays while the base type is coarse. Above six bits the stock presets are already close to lossless, so lifting a small part of the model buys nothing while paying for it out of a large part costs real accuracy. We tested this and lost, so above 6 GB this repo ships stock presets rather than a worse tuned file with our label on it.
Text only:
With vision:
[!IMPORTANT] Pass
--image-min-tokens 1024. Without it, dense images such as charts and screenshots get too few visual tokens, and the model answers from what it already knows instead of from what it was shown. llama.cpp prints a warning about this at load time. We hit exactly this on our first attempt: the model produced a confident, fluent, completely invented description of a chart.
[!IMPORTANT] Upstream ships no
generation_config.json, so llama.cpp falls back to its own sampling defaults, which are not the ones this model was tuned for. Set them yourself:--temp 0.6 --top-p 0.95 --top-k 20. Use--temp 1.0to reproduce the benchmark setup from the upstream model card.
Lines reading find_slot: non-consecutive token position are normal. That is
how this family numbers image patches, and they appear in successful runs too.
Single RTX 5090, llama.cpp b10505-ee4c505a4, CUDA 13.0, flash attention on.
Generation is limited by memory bandwidth, not compute: 17.9 GB read per token against 1792 GB/s puts the ceiling near 100 t/s, and BF16 reaches 94% of it. Smaller files scale close to linearly, so a 5.9 GB build lands near 280 t/s on the same card.
The model reads handwriting. Asked to describe a calligraphy piece it has not
seen before, it transcribed the full quote and picked out the small TB-09 mark
in the corner.
This image displays an elegant, calligraphic handwritten quote featuring a large blue initial capital "P" and intricate botanical illustrations in the corners. The text reads: "Peace of mind comes to me through making things with my hands. An added bonus comes if my efforts inspire others to try the creative process."
Image encoding takes 163 ms for this picture on one RTX 5090.
A short check that the thinking channel does real work rather than filling space. Prompt: A gas station is 1 kilometer away. I want to wash my car. Should I walk or drive?
The model answered "drive", then argued against itself in the open, noting that some gas stations have car washes so walking there is not obviously wrong, then closed the argument by observing that walking leaves the car at home either way:
You should drive. To wash your car, your car needs to be at the location where it is being washed.
The bit layout was not carried over from another model. We built eight candidate layouts at one size class, measured all of them against the same reference on the same box, and kept the winner. Every log below is in the metrics repo.
Three things came out of this.
The gates matter more than their size suggests. attn_gate and ssm_out
are 9% of the weights, and the importance matrix ranks attn_gate first in the
whole model by summed squared activation. Lifting those two and paying for them
out of the MLP gate and up projections, which are 36% of the weights, is the
single largest win available.
The embedding must not be squeezed. With a 248320 token vocabulary and untied weights, the embedding and the output head are 22% of this model, which makes them look like the obvious place to save. AB-T tested that and came last, worse than the stock preset. A lifted output head does not pay for a crushed embedding.
A layout does not transfer by layer position. AB-X applied a recipe that works well on a 64 layer model by scaling the layer numbers, and it lost to the flat control. What transfers is which tensor groups matter, read from the importance matrix of this model.
Read from the converted file, not from the config.
Only 8 of the 32 layers keep a KV cache, the other 24 hold a fixed recurrent state instead. Long context is unusually cheap here, roughly 32 KB per token, so 8k of context costs about 256 MB rather than the gigabytes this size of model normally implies.
[!WARNING] If you convert this model yourself, pass
--no-nextntoconvert_hf_to_gguf.py. Upstreamconfig.jsondeclaresmtp_num_hidden_layers: 1, but the checkpoint contains no such tensors. The converter believes the config, writesblock_count = 33for a 32 layer model, and reports a successful export. The file then fails to load withtensor 'blk.32.attn_norm.weight' not found. There is no speculative draft to publish for this model.
The importance matrix was computed on the BF16 weights, not on a quantized stand-in, over 4,958,490 tokens in 9,686 chunks, fanned out across 20 GPUs and merged. Vocabulary coverage of the calibration corpus is 99.5%, so almost no embedding row was quantized without evidence.
Corpus composition: agentic tool traces 24.7%, code 17.8%, reasoning 14.8%, multilingual 13.8%, long context 11.9%, vocabulary sweep 9.9%, structured data 3.9%, graphics 3.0%. The vocabulary sweep is regenerated per tokenizer, since a sweep built for another model covers a different vocabulary and calibrates nothing here.
Everything needed to check this card is public:
ornith-1.5-9b.The corpus and the context have to match ours or the numbers are not comparable.
Ornith-1.5 is trained with a loop that generates its own tasks, builds its own scaffolds, and rewards all three stages together rather than optimizing a policy against a fixed human written harness. For the method and the benchmark results, see the upstream model card and the Ornith blog. Those benchmark numbers are the authors' own and we have not independently reproduced them.
llama-server -m Ornith-1.5-9B-AD-Q5_K-Q4_K.gguf \ -ngl 99 -c 8192 -fa on --jinja \ --temp 0.6 --top-p 0.95 --top-k 20 \ --port 8080llama-mtmd-cli -m Ornith-1.5-9B-AD-Q5_K-Q4_K.gguf \ --mmproj mmproj-Ornith-1.5-9B-F16.gguf \ -ngl 99 -c 8192 --temp 0.6 --image-min-tokens 1024 \ --image your-image.jpg \ -p "Describe this image."llama-perplexity -m your-quant.gguf -f eval_neutral.txt \ --kl-divergence-base base-neutral.kld --kl-divergence -c 4096 -ngl 99