Mirrored from speridlabs/iris-3b
All models by speridlabs →Pixel-space generation & general vision learner.
A 3-billion-parameter model that paints every pixel directly — no VAE, no latent space — and, fine-tuned, estimates depth and restores and upscales images.
Generative priors are a promising foundation for downstream vision tasks. In this project we explore pixel-space generative models as an alternative to vision foundation models such as DINOv2.
Most image generators work in a compressed "latent" space and rely on a separate decoder to turn that into an image. Iris-3B skips that step: the network itself outputs every pixel, so nothing is lost to a lossy, texture-biased latent. You type a prompt, and you get a 1024-pixel image.
Iris-3B is not only an image generator. The same pixel-space prior, fine-tuned with no architectural change, is a general vision learner for dense tasks where detail matters: monocular depth estimation and image restoration and upscaling ship in this repo (see A general vision learner below).
Iris-3B Turbo, in the turbo/ folder, is a 4-step distilled version of Iris-3B: a
1024×1024 image in about 0.7 s on one H100.
Generated at native aspect ratios of about one megapixel with the default settings (CFG 3, 100 steps).
1. Install the code
2. Download the weights (about 12 GB for text-to-image; the depth/ and upscaler/ folders add about 12 GB each, turbo/ about 6 GB)
3. Generate an image
The text encoder (Qwen3-VL-4B-Instruct) downloads automatically on the first run. You need an NVIDIA GPU with CUDA.
Default output is 1024×1024. Write prompts as plain descriptive English sentences.
[Image blocked: Iris-3B Turbo sample: a shiba inu in aviator sunglasses driving a supercar under the words IRIS TURBO]
Iris-3B Turbo, in the turbo/ folder of this repo, is a 4-step distilled version of Iris-3B. Classifier-free guidance
is baked into the weights, so each step is a single forward pass; a 1024×1024 image takes about 0.7 s on one H100
(batch 4). It is a full fine-tune of all 3B parameters, not a LoRA. Try it in the Turbo demo.
With the code installed (step 1 above):
The script reads the 4-step schedule from turbo/distill.yaml (σ = 1, 0.938, 0.833, 0.625, deterministic flow-Euler
steps) and samples without CFG. The schedule is part of the model: --steps, --cfg-scale and --negative-prompt
do not apply, while the prompt, --height, --width and --seed do. Turbo was distilled at about one megapixel in 7
aspect buckets: 1024×1024, 752×1344, 896×1344, 896×1200 and their transposes. The image above was generated on an H100 with --seed 6
and the prompt Colorful animated cartoon poster: a smiling shiba inu in aviator sunglasses behind the wheel of a low futuristic
supercar, dynamic angle, neon sunset sky, with big bold vector lettering "IRIS TURBO" in the sky.
All models at 1024×1024 with the official scorers and the same prompts and seeds; Iris-3B with its default sampling settings. The community adapter was run with its recommended settings through its own code. Turbo is slightly ahead of the base model on prompt following (GenEval, DPG-Bench) and behind it on long rendered text and on OneIG-Bench, where the gap is mostly text and diversity.
Turbo is distilled from Iris-3B with Trajectory Distribution Matching (TDM),
data-free, as a full fine-tune of all 3B parameters in two stages. The trainer and both configs are in the
code repository (scripts/distill.py).
The first few-step Iris came from the community: pentacoxian-dev built iris-3b-turbo-lora (code, demo), a 4-step DMD2 distillation packaged as a rank-64 LoRA. It is a lightweight 460 MB adapter trained on a single consumer GPU, far easier to download, share and build on than a full 6 GB checkpoint. Thank you, it is a joy to see Iris picked up by the community this quickly.
The same backbone, fine-tuned per task: Iris-3B estimates monocular depth and restores and upscales images. Both run the whole model in a single forward pass with the empty prompt, so no text encoder is needed. The weights download on first use.
Monocular depth — photo → relative depth:
Image restoration and upscaling — degraded input → restored image:
Try both interactively in the demo, or from the command line:
Depth is affine-invariant, not metric: fit a scale and shift in log space to compare with ground truth. Restoration tiles large outputs into overlapping 1024×1024 windows.
The full training recipe is in the code repository.
Released under the Apache License 2.0. The text encoder, Qwen3-VL-4B-Instruct, is downloaded from its publisher and stays under its own license (Apache 2.0).
Made with ❤️ by speridlabs.com
git clone https://github.com/speridlabs/iris-3b.gitcd iris-3bpip install -e . # Python 3.11+, PyTorch 2.7.1+hf download speridlabs/iris-3b --local-dir iris-3b --exclude "depth/*" "upscaler/*" "turbo/*"python scripts/sample.py --checkpoint iris-3b \ --prompt "a red fox sleeping in fresh snow, golden hour"hf download speridlabs/iris-3b --include "turbo/*" --local-dir iris-3b # about 6 GBpython scripts/sample.py --checkpoint iris-3b/turbo \ --prompt "a red fox sleeping in fresh snow, golden hour"python scripts/depth.py photo.jpg --out depth_out # <name>.npy + colorized <name>.pngpython scripts/upscale.py photo.jpg --out upscaled # <name>_x4.png@article{licai2026iris, title = {Iris-3B: Going Beyond the Latent with Pixel-Space Diffusion Training, Conversion and Fine-Tuning}, author = {Li Cai, Hanqiu and Garabito, Chema}, journal = {arXiv preprint arXiv:2610.09450}, year = {2026}, url = {https://arxiv.org/abs/2610.09450}}