No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from HuggingFaceBio/Carbon-X-GGUF
All models by HuggingFaceBio →[Image blocked: No description]
Blog | Hugging Face Bio | Code
GGUF quantizations of Carbon-X, an open 14B language model for DNA understanding and generation from the Carbon family, for local inference with llama.cpp.
Carbon-X reads raw DNA inside an ordinary chat conversation and answers in English: it predicts whether mutations are benign or pathogenic, identifies regions that control gene activity, classifies sequences by organism group, predicts the next bases of a sequence and explains its reasoning. A hybrid tokenizer keeps a BPE vocabulary for text and encodes DNA as non-overlapping 6-mers. See the Carbon-X model card for details on the model and its limitations.
[!IMPORTANT] These files need llama.cpp b11742 or newer, the first release with support for the Carbon
HybridDNATokenizer(#30825). Older builds load them but tokenize DNA wrongly.
Q8_0 is the recommended file: half the size of BF16, with the same benchmark scores. Use Q4_K_M when memory is tight. In every K-quant the token embedding and output tensors, which hold the DNA tokens, stay at Q8_0.
Any llama.cpp release from b11742 on works: a prebuilt
binary from the releases page, a package manager once it ships b11742
or newer (brew install llama.cpp, winget install llama.cpp, ...), or a build from source:
The commands below assume llama-server and llama-cli are on your PATH (for a source build they are in
build/bin). With -hf, llama.cpp downloads and caches the quant on first use. While this repo is private, log in
first with hf auth login or set HF_TOKEN. To use a local file instead, download it and replace
-hf HuggingFaceBio/Carbon-X-GGUF:Q8_0 with -m carbon-x-q8_0.gguf:
llama-server provides an OpenAI-compatible API:
--jinja uses the chat template stored in the GGUF.enable_thinking: false.--temp 0.6 --top-p 0.95 --top-k 20 --min-p 0 are Carbon-X's default sampling settings, used for every reported
result. llama.cpp's own defaults differ (temperature 0.8, top-k 40, min-p 0.05).-c 40960 is the model's full context. llama-server shares it between 4 parallel request slots by default; set
-np N to change the number of slots and raise -c if each request needs the full context.-ngl 99 offloads every layer to the GPU; Q8_0 needs about 16 GB plus about 0.16 MB of KV cache per token.Query it with any OpenAI client:
Example answer (Q8_0):
This sequence is more consistent with a coding region than with non-coding DNA.
It begins with ATG, a standard start codon, and can be read in triplets from that point without encountering a stop codon within the supplied sequence. That supports a protein-coding interpretation, though it does not by itself prove that the sequence is translated or establish its biological function.
Add -st -p "your question" for a single answer instead of an interactive chat.
The tokenizer switches between BPE for text and 6-mers for DNA. Paste sequences directly into a message:
<dna>…</dna> and
tokenized as 6-mers. Runs of at least 12 bases with up to 20% IUPAC ambiguity codes (N, R, Y, …) are tagged
too, and sequences displayed in space-separated blocks are joined. Text you have already wrapped in <dna>…</dna>
is always tokenized as DNA.N, becomes a single <oov>
token. To keep full resolution, filter to canonical bases first.<dna>…</dna>, and llama.cpp keeps the tags in the
returned text. Strip them if you need the bare sequence.The llama.cpp tokenizer matches the Python HybridDNATokenizer exactly: identical token ids, detokenized text and
chat-template rendering on 1,000 benchmark prompts (1.13M tokens) and 36 DNA edge cases.
We ran two of the DNA benchmarks from the Carbon-X model card
on each GGUF file through llama-server, with thinking disabled, the default sampling settings above and one sample
per problem. The Carbon-X column is the original model served with vLLM.
Scores are single runs with sampling: the standard error is about 1.5 points on FrontierVEP and 1 point on NextBase. BF16 through Q5_K_M are within about one standard error of Carbon-X. Q4_K_M is the lowest on both benchmarks.
Apache 2.0.
Carbon-X is a joint collaboration between the research teams at Hugging Face, Zhongguancun Academy, and the Telethon Institute of Genetics and Medicine/University of Naples “Federico II”.
[Image blocked: Hugging Face, Zhongguancun Academy, Telethon Institute of Genetics and Medicine and University of Naples Federico II]
git clone https://github.com/ggml-org/llama.cppcd llama.cppcmake -B build -DGGML_CUDA=ON # drop -DGGML_CUDA=ON for a CPU-only build, or use -DGGML_METAL=ON etc.cmake --build build -j --target llama-server llama-clihf download HuggingFaceBio/Carbon-X-GGUF carbon-x-q8_0.gguf --local-dir .llama-server -hf HuggingFaceBio/Carbon-X-GGUF:Q8_0 \ -ngl 99 -c 40960 --jinja \ --chat-template-kwargs '{"enable_thinking": false}' \ --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0 \ --port 8080curl -s http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{ "messages": [{"role": "user", "content": "Is this sequence more likely coding or non-coding? ATGGCGCTGTGGATGCGCCTCCTGCCCCTGCTGGCGCTGCTGGCCCTCTGGGGACCTGACCCAGCCGCAGCC"}]}' | jq -r '.choices[0].message.content'from openai import OpenAI
client = OpenAI(base_url="http://localhost:8080/v1", api_key="none")response = client.chat.completions.create( model="carbon-x", # llama-server serves a single model; the name is not checked messages=[{"role": "user", "content": "Is this sequence more likely coding or non-coding? ATGGCGCTGTGGATGCGCCTCCTGCCCCTGCTGGCGCTGCTGGCCCTCTGGGGACCTGACCCAGCCGCAGCC"}],)print(response.choices[0].message.content)llama-cli -hf HuggingFaceBio/Carbon-X-GGUF:Q8_0 -ngl 99 -c 40960 --jinja \ --chat-template-kwargs '{"enable_thinking": false}' \ --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0