No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from AtomicChat/embeddinggemma-2-GGUF
All models by AtomicChat →Built from Google's original weights with our own importance matrix. The calibration corpora behind our builds are public.
[Image blocked: How far each file drifts from the original, Unsloth's file of the same size = 100]
Lower is better, and Unsloth's file of the same size is 100.
Q8_0 file from AtomicChat, Unsloth and ggml-org is the same file.Every number below is measured, not estimated. How we measured it is at the bottom, and the raw logs are in the metrics repo so you can check any of it yourself.
An embedding model has no next word to predict, so the usual KL divergence does
not apply. What matters is whether search still finds what the original finds.
same top result is how often a search with this file returns the same best
passage as the original weights, over 1,000 queries in 30 languages. cosine to BF16 is how close each vector is to the original's vector for the same text (1
means identical).
AD- means Atomic Dynamic: the type is chosen per tensor from measurements
rather than taken from a llama.cpp preset.
Which one to take:
Q8_0 if you can spare 310 MB. It is practically the original.AD-Q6_K for a smaller file whose vectors stay very close to the original's.AD-Q4_K_M when size matters most. Its top result differs from the
original's on about one query in eight, but it finds the right passage as
often as the original does (see Search quality below).[!IMPORTANT] Index your documents and embed your queries with the same file. Vectors from different quantizations are close, not identical, so do not mix them in one index.
[!NOTE] We do not ship F16. Google reports that the model's activations overflow float16 and produce NaN or silently degraded vectors. Use BF16 or one of the quantized files.
Images and audio go through the vision and audio encoders in a separate
projector file (--mmproj). The model also embeds video; we have not tested
that path.
This is a sanity check, not a benchmark: one image and one 17-second speech clip. In both projector files the image sits closer to its right caption than to a wrong one (0.79 against 0.56), and so does the audio clip (0.59 against 0.52).
[!NOTE] These files need llama.cpp with the
gemma-embedding2architecture, which arrived in llama.cpp PR #30054 on 6 October 2026. An older build stops withunknown model architecture: 'gemma-embedding2'. Apps built on llama.cpp, such as LM Studio and Jan, run the files once their bundled engine includes that change.
The model reads the whole input at once (bidirectional attention), so -ub has
to be at least as long as your longest input. Raise -c, -b and -ub
together for inputs up to the model's 8K context.
The vectors come back L2-normalised, with 768 dimensions.
Task prefixes. The model was trained with them, and leaving them out costs accuracy:
Shorter vectors. Keep the first 512, 256 or 128 values and normalise again.
At 256 dimensions every file keeps the same standing as in the table above (all
numbers are in results.json).
Images and audio. Start the server with the projector, then send an OpenAI-style content array:
We downloaded the other publishers' files and measured them with the same
harness, the same eval set and the same machine. 1 - cosine is the mean
distance of a file's vectors from the original's: lower is better.
[Image blocked: Quality against size: AtomicChat, Unsloth, AutoRound and ggml-org]
same top result rates differ by less than the
noise. Ours is 96.3% (95% interval 95.0-97.4%) and AutoRound's is 97.5%
(96.4-98.4%).Q8_0 from AtomicChat, Unsloth and ggml-org is the same file, tensor for
tensor.Averages can hide a few outliers, so we also compared each pair text by text. The columns are:
How to read it:
per_layer_model_proj, which ours keeps in BF16.Unsloth's and AutoRound's files use one type for almost the whole model: Q6_K everywhere, or Q4_K everywhere. Ours spend the bits by measurement instead.
[Image blocked: Where each build spends its bits]
The text model is a small Gemma 4-style encoder. A text is cut into tokens, and each token takes its 512-number row from a 262,144-row token table. The vectors then pass through 24 transformer blocks in which every token sees every other one. Five blocks with a local window alternate with one global block. At the end the model takes the mean over all tokens, and an output head turns that mean into the final 768-number vector.
The token table is half of the model: 134M of 270M parameters. The 24 blocks make up another 130M.
We quantized one part of the model at a time to Q4_K, kept everything else at Q8_0, and measured how far the vectors moved. The table shows the cost of each megabyte saved, with the token table as 1:
Two parts stand at opposite ends.
The token table is the cheapest place to save bytes. Each token takes its own row, so rounding noise in the table is different for every token. The model then averages all tokens into one vector, and independent noise mostly cancels out. A rounding error inside a transformer weight is different: every token passes through the same matrix, so the error pushes all of them the same way, and pooling cannot remove it.
The output head is the most fragile part. It works on the already-averaged vector, once, so its rounding error shifts every final vector in the same direction and nothing averages it out. Putting the head alone in Q4_K saves 0.2 MB and costs more than putting all of attention's q, k and v in Q4_K.
The same test on four bands of six blocks each:
The first and last blocks are 2-3 times more sensitive than the middle.
The four bands measured one at a time add up to what the whole stack costs at once (0.0068 against 0.0067). So within the target size we could add up measured costs, give bits to the parts where they buy the most, and then measure the few best candidates whole.
The result is the picture above:
About 30 test files went into it, each about 30 seconds to build and measure on a laptop.
per_layer_model_proj (12.6 MB). Unsloth's 4- and 5-bit files and all of
AutoRound's quantize it. Stock llama-quantize always keeps it in BF16, and
so do our files. At equal size we therefore spend about 6-8 MB less on
everything else.same top result rate: the differences there stay inside the noise.--override-kv tokenizer.ggml.add_eos_token=bool:false,
llama-imatrix stops on an assertion.--no-ppl, because there are no logits.-ub equal to -c, because attention is bidirectional.per_layer_model_proj get no importance
data at all.Quantization reorders near-ties among the candidates. It does not make search worse. How often a query finds its own passage among all passages of the set (recall@1):
The task is hard on purpose: the query is the first quarter of a chunk and the passage is the rest, so the absolute numbers are low. The point is that they do not move.
About 1.24M tokens from the calibration corpora pool: 60% Wikipedia in 30 languages and 40% source code. Everything is written the way the model sees it at work:
title: none | text: ..., or title: <file name> | text: ... for code;None of it overlaps the held-out eval set. The builder script and the corpus itself are in the metrics repo.
eval/neutral (81 Wikipedia articles in 30
languages, 1,000 query and passage pairs) and eval/code (100 source files,
400 pairs). Each chunk of text is split in two: the first quarter becomes the
query, the rest becomes the passage.llama-server --embeddings --pooling mean -fa off on an Apple M4
Max (Metal), llama.cpp commit 2690873. All files embed the 2,800 texts in
about 25 seconds, and the smaller files are not faster on this machine.AD-Q6_K uses these types:
token_embd in q5_k;inp_gate and proj in q8_0;output in q8_0.The exact rules are in the quantize logs in the metrics repo.
914f7f8, Apache 2.0.gemma-embedding2 architecture (llama.cpp PR #30054, merged on 6 October 2026). We built and tested with commit 26908739bc8a.llama-server -m embeddinggemma-2-Q8_0.gguf --embeddings --pooling mean \ -c 2048 -b 2048 -ub 2048 -ngl 99curl http://127.0.0.1:8080/v1/embeddings -H "Content-Type: application/json" -d '{ "input": ["task: search result | query: how do I reset my password?", "title: none | text: Open Settings, choose Forgot password and follow the link we email you."], "encoding_format": "float"}'llama-server -m embeddinggemma-2-Q8_0.gguf --mmproj mmproj-embeddinggemma-2-Q8_0.gguf \ --embeddings --pooling mean -c 4096 -b 4096 -ub 4096 -ngl 99{"input": [{"content": [{"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,..."}}]}, {"content": [{"type": "input_audio", "input_audio": {"data": "<base64 wav>", "format": "wav"}}]}, "task: search result | query: a newspaper front page about the moon landing"], "encoding_format": "float"}python convert_hf_to_gguf.py embeddinggemma-2 --no-lazy --outtype bf16 --model-name embeddinggemma-2 \ --outfile embeddinggemma-2-BF16.ggufpython build_calib.py --pool calib-corpora/pool --tokenizer embeddinggemma-2/tokenizer.json --tokens 1000000 -o calib.txtllama-imatrix -m embeddinggemma-2-BF16.gguf -f calib.txt -o imatrix.gguf -c 512 -b 512 -ub 512 \ --parse-special --no-ppl --override-kv tokenizer.ggml.add_eos_token=bool:falseM='(attn_(q|k|v|output)|ffn_(gate|up|down))\.weight$'llama-quantize --imatrix imatrix.gguf \ --tensor-type '^output\.weight$=q8_0' --tensor-type '^token_embd\.weight$=iq4_xs' \ --tensor-type '^blk\.\d+\.(inp_gate|proj)\.weight$=q4_k' \ --tensor-type "^blk\.([0-5])\.$M=q5_k" --tensor-type "^blk\.([6-9]|1[01])\.$M=iq4_xs" \ --tensor-type "^blk\.(1[2-7])\.$M=iq4_xs" --tensor-type "^blk\.(1[89]|2[0-3])\.$M=q4_k" \ embeddinggemma-2-BF16.gguf embeddinggemma-2-AD-Q4_K_M.gguf Q4_K_M