No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from furiosa-ai/gemma-4-31B-it-FP8-block
All models by furiosa-ai →This repository contains RedHatAI/gemma-4-31B-it-FP8-block together with a Furiosa Executable Bundle (FXB) for running it on FuriosaAI RNGD with Furiosa-LLM. The same model also runs on other frameworks (such as vLLM, SGLang, and Transformers); for usage with those, see the upstream RedHatAI/gemma-4-31B-it-FP8-block model card.
Gemma 4 31B is a dense multimodal model from Google. It combines a vision encoder with a transformer decoder that interleaves sliding-window and full-attention layers. Its chat template supports optional reasoning and tool (function) calling. This Red Hat AI build applies FP8 block quantization to the text transformer's linear layers. The current Furiosa FXB supports text-to-text inference; image input is not yet supported. Its intended use is the same as the upstream RedHatAI/gemma-4-31B-it-FP8-block, released under the Apache 2.0 License.
The upstream Red Hat AI build statically quantizes the weights of linear operators in the text transformer to FP8 with 128×128 blocks. Their input activations are quantized dynamically to FP8 at runtime in groups of 128 values, without offline calibration. The vision tower, embeddings, output head, and KV cache stay in 16-bit precision.
enable_thinking and defaults it to false. The current Furiosa-LLM release does not provide a Gemma 4 reasoning parser, so the usage below keeps thinking disabled.On RNGD, gemma-4-31B-it-FP8-block runs with a tensor-parallel size of 32 PEs, which maps to four RNGD cards (8 PEs per card).
To run this model with Furiosa-LLM, follow the example commands below after installing Furiosa-LLM and its prerequisites.
Serve the model by passing its furiosa-ai/<repo> identifier:
Gemma 4's chat template can generate reasoning and tool calls, but Furiosa-LLM does not currently register parsers for the Gemma 4 output format. Keep the default non-thinking chat mode and do not enable automatic tool choice with this FXB.
When the server is ready, you will see:
The server exposes an OpenAI-compatible API. You can send a text request with
curl:
Current feature scope. The upstream model accepts image input and its chat
template supports optional reasoning through enable_thinking as well as native
tool calls. The current Furiosa FXB supports text input only, and Furiosa-LLM
does not yet provide the Gemma 4 reasoning and tool-call parsers required to
return those outputs through the corresponding OpenAI-compatible response
fields. Multimodal serving flags and automatic tool choice therefore do not
enable these upstream capabilities for this FXB.
See the Vision-Language Models guide and Tool Calling guide for the APIs that will apply when those capabilities are supported.
furiosa-llm serve) — full OpenAI-compatible API reference and serving options# Launch the server, listening on port 8000 by defaultfuriosa-llm serve furiosa-ai/gemma-4-31B-it-FP8-blockINFO: Started server process [27507]INFO: Waiting for application startup.INFO: Application startup complete.INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)curl http://localhost:8000/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "furiosa-ai/gemma-4-31B-it-FP8-block", "messages": [{"role": "user", "content": "What is the capital of France?"}] }' \ | python -m json.tool