No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from unsloth/diffusiongemma-26B-A4B-it
All models by unsloth →See Unsloth Dynamic 2.0 GGUFs for our quantization benchmarks.
Hugging Face |
GitHub |
Launch Blog |
Documentation
License: Apache 2.0 | Authors: Google DeepMind
These GGUFs run with the DiffusionGemma build of llama.cpp (the DiffusionGemma PR
ggml-org/llama.cpp#24423). DiffusionGemma is a block-diffusion
architecture, so it needs that branch plus the dedicated llama-diffusion-cli runner - the standard
llama-cli / llama-server cannot generate from it yet.
-n / --n-predict is the only length knob you need: it auto-derives the diffusion block count and
grows the batch/context to fit. On a single GPU the prompt prefix KV cache turns on automatically.
--diffusion-visual)Add --diffusion-visual to see each 256-token canvas being denoised in place (flicker-free, keeps
your scrollback):
-n, --n-predict N - target tokens; derives --diffusion-blocks and grows -ub / -b / -c.-ngl 99 - offload all layers to the GPU (-ngl 0 for CPU-only).-cnv - multi-turn conversation mode.--diffusion-visual - live canvas denoising view.--diffusion-eb auto). Tune it with
--diffusion-eb-max-steps (default 48), --diffusion-eb-t-max / --diffusion-eb-t-min
(0.8 -> 0.4), --diffusion-eb-entropy-bound (0.1), and --diffusion-eb-confidence (0.005).--diffusion-kv-cache {auto,on,off} - prompt prefix KV cache (auto = on for single GPU).DiffusionGemma is a generative model built by Google DeepMind. Based on the 26B A4B Mixture-of-Experts (MoE) Gemma 4 architecture, DiffusionGemma generates tokens using discrete diffusion. This open-weights model is multimodal, handling text, image, and video inputs to generate text output.
Built on a MoE foundation, DiffusionGemma is designed to improve generation speed (tokens per second) while remaining deployable across various hardware environments. DiffusionGemma builds upon the architectural and capability advancements of Gemma 4, introducing several core features:
system role, enabling more structured and controllable conversations.DiffusionGemma is engineered to reduce the sequential bottlenecks of standard causal language models. It employs an encoder-decoder architecture specifically optimized for inference speed.
The encoder operates in a prefill capacity, processing the initial prompt and generating the KV cache. The decoder then utilizes bidirectional attention to process an input block (a 'canvas') of tokens, accessing the cached context via cross-attention.
During inference, DiffusionGemma leverages multi-canvas sampling. Rather than generating one token at a time, the model iteratively denoises a full block of tokens using a diffusion sampler. Once a canvas is fully denoised, it is processed by the encoder and appended to the KV cache, after which the model generates the next canvas. This block-autoregressive approach facilitates text generation at higher speeds.
These models were evaluated against a large collection of different datasets and metrics to cover different aspects of text generation. Evaluation results marked in the table are for instruction-tuned models, with the recommended Entropy Bound (EB) sampler (see Best Practices below).
DiffusionGemma handles a broad range of tasks across text and vision. Key capabilities include:
You can use all Gemma 4 models with the latest version of Transformers. To get started, install the necessary dependencies in your environment:
pip install -U transformers torch accelerate
Once you have everything installed, you can proceed to load the model with the code below:
Once the model is loaded, you can start generating output:
For the best performance, use these configurations and best practices:
Use the following standardized sampling configuration across all use cases:
Similar to Gemma 4 models, we use standard system, assistant, and user roles. To properly manage the thinking process, use the following control tokens:
<|think|> token at the start of the system prompt. To disable thinking, remove the token (note that an empty thinking channel might still be emitted).<|channel>thought\n[Internal reasoning]<channel|>.<|channel>thought\n<channel|>[Final answer].[!Note] Note that many libraries like transformers handle the complexities of the chat template for you.
Aside from variable aspect ratios, DiffusionGemma supports variable image resolution through a configurable visual token budget, which controls how many tokens are used to represent an image. A higher token budget preserves more visual detail at the cost of additional compute, while a lower budget enables faster inference for tasks that don't require fine-grained understanding.
All models support image inputs and can process videos as frames. Video supports a maximum of 60 seconds assuming the images are processed at one frame per second.
Our pre-training dataset is a large-scale, diverse collection of data encompassing a wide range of domains and modalities, which includes web documents, code, images, audio, with a cutoff date of January 2025. Here are the key components:
The combination of these diverse data sources is crucial for training a powerful multimodal model that can handle a wide variety of different tasks and data formats.
Here are the key data cleaning and filtering methods applied to the training data:
DiffusionGemma was developed in partnership with internal safety and responsible AI teams. A range of automated as well as human evaluations were conducted to help improve model safety. These evaluations align with Google’s AI principles, as well as safety policies, which aim to prevent our generative AI models from generating harmful content, including:
For all areas of safety testing, we saw major improvements in all categories of content safety relative to previous generations of Gemma models. Overall, DiffusionGemma, like Gemma 4 models, significantly outperforms Gemma 3 and 3n models in improving safety, while keeping unjustified refusals low. All testing was intentionally conducted without safety filters to evaluate the model’s raw capabilities and baseline behaviors. For both text-to-text and image-to-text, and across all model sizes, the model produced minimal policy violations, and showed significant improvements over previous Gemma models.
These models have certain limitations that users should be aware of.
Multimodal models (capable of processing vision, language, and/or audio) have a wide range of applications across various industries and domains. The following list of potential uses is not comprehensive. The purpose of this list is to provide contextual information about the possible use-cases that the model creators considered as part of model training and development.
In creating an open, vision-language model, we have carefully considered the following:
Risks identified and mitigations:
At the time of release, this is a low-latency, high-performance open vision-language model that provides a compelling option for developers and those interested in researching diffusion language models. The model is designed from the ground up for responsible AI development compared to similarly sized models.
git clone https://github.com/ggml-org/llama.cppcd llama.cpp
# check out the DiffusionGemma PR (#24423)gh pr checkout 24423# or, without the gh CLI:# git fetch origin pull/24423/head:diffusiongemma && git checkout diffusiongemma
# build with CUDA (drop -DGGML_CUDA=ON for a CPU-only build)cmake -B build -DGGML_CUDA=ONcmake --build build -j --config Release --target llama-diffusion-clipip install -U "huggingface_hub[cli]"hf download unsloth/diffusiongemma-26B-A4B-it-GGUF \ --local-dir unsloth/diffusiongemma-26B-A4B-it-GGUF \ --include "*Q8_0*" # Use "*Q4_K_M*" for a smaller 16 GB download./build/bin/llama-diffusion-cli \ -m unsloth/diffusiongemma-26B-A4B-it-GGUF/diffusiongemma-26B-A4B-it-Q8_0.gguf \ -ngl 99 -cnv -n 2048./build/bin/llama-diffusion-cli \ -m unsloth/diffusiongemma-26B-A4B-it-GGUF/diffusiongemma-26B-A4B-it-Q8_0.gguf \ -ngl 99 -cnv -n 2048 --diffusion-visualfrom transformers import DiffusionGemmaForBlockDiffusion, AutoProcessor
MODEL_ID = "google/diffusiongemma-26B-A4B-it"
# Load modelprocessor = AutoProcessor.from_pretrained(MODEL_ID)model = DiffusionGemmaForBlockDiffusion.from_pretrained( MODEL_ID, dtype="auto", device_map="auto",)# Promptmessage = [ {"role": "user", "content": "Why is the sky blue?"}]
# Process inputinput_ids = processor.apply_chat_template( message, tokenize=True, add_generation_prompt=True, return_dict=True, return_tensors="pt").to(model.device)output = model.generate(**input_ids, max_new_tokens=512)
# Parse outputtext = processor.decode(output[0], skip_special_tokens=False)