No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from RedHatAI/Ministral-3-3B-Instruct-2512
All models by RedHatAI →
The smallest model in the Ministral 3 family, Ministral 3 3B is a powerful, efficient tiny language model with vision capabilities.
This model is the instruct post-trained version in FP8, fine-tuned for instruction tasks, making it ideal for chat and instruction based use cases.
The Ministral 3 family is designed for edge deployment, capable of running on a wide range of hardware. Ministral 3 3B can even be deployed locally, capable of fitting in 8GB of VRAM in FP8, and less if further quantized.
Learn more in our blog post and paper.
ModelCar Storage URI: oci://registry.redhat.io/rhai/modelcar-ministral-3-3b-instruct-2512:3.0
Validated on vLLM: 0.14.1
Validated on RHAIIS: 3.4 EA1
Validated on RHOAI: 3.4 EA1
Ministral 3 3B consists of two main architectural components:
The Ministral 3 3B Instruct model offers the following capabilities:
Ideal for lightweight, real-time applications on edge or low-resource devices, such as:
Bringing advanced AI capabilities to edge and distributed environments for embedded systems.
We recommend deploying with the following best practices:
Other formats available here.
We compare Ministral 3 to similar sized models.
The model can be used with the following frameworks;
vllm: See heretransformers: See hereWe recommend using this model with vLLM.
Make sure to install vllm >= 0.12.0:
Doing so should automatically install mistral_common >= 1.8.6.
To check:
You can also make use of a ready-to-go docker image or on the docker hub.
Due to their size and the FP8 format of their weights Ministral-3-3B-Instruct-2512, Ministral-3-8B-Instruct-2512 and Ministral-3-14B-Instruct-2512 can run on a single 1xH200 GPU.
A simple launch command is:
Key parameter notes:
Additional flags:
--max-model-len to preserve memory. By default it is set to 262144 which is quite large but not necessary for most scenarios.--max-num-batched-tokens to balance throughput and latency, higher means higher throughput but higher latency.Here we assume that the model mistralai/Ministral-3-3B-Instruct-2512 is served and you can ping it to the domain localhost with the port 8000 which is the default for vLLM.
Let's see if the Ministral 3 knows when to pick a fight !
Let's solve some equations thanks to our simple Python calculator tool.
Ministral 3 can follow your instructions to the letter.
You can also use Ministral 3 3B Instruct 2512 with Transformers !
Transformers recently added support for FP8, so make sure to install from main:
To make the best use of our model with Transformers make sure to have installed mistral-common >= 1.8.6 to use our tokenizer.
Try it out by running the following snippet.
[!Tip] On latest main as of 05/12/2025, by default a FP8 triton kernel for fast accelerated matmuls (
w8a8_block_fp8_matmul_triton) will be used without any degradation in accuracy. However, if you want to run your model in BF16 see (here)
Transformers allows you to automatically convert the checkpoint to Bfloat16. To do so, simply load the model as follows:
This model is licensed under the Apache 2.0 License.
You must not use this model in a manner that infringes, misappropriates, or otherwise violates any third party’s rights, including intellectual property rights.
pip install vllm --upgradepython -c "import mistral_common; print(mistral_common.__version__)"vllm serve mistralai/Ministral-3-3B-Instruct-2512 \ --tokenizer_mode mistral --config_format mistral --load_format mistral \ --enable-auto-tool-choice --tool-call-parser mistralfrom datetime import datetime, timedelta
from openai import OpenAIfrom huggingface_hub import hf_hub_download
# Modify OpenAI's API key and API base to use vLLM's API server.openai_api_key = "EMPTY"openai_api_base = "http://localhost:8000/v1"
TEMP = 0.15MAX_TOK = 262144
client = OpenAI( api_key=openai_api_key, base_url=openai_api_base,)
models = client.models.list()model = models.data[0].id
def load_system_prompt(repo_id: str, filename: str) -> str: file_path = hf_hub_download(repo_id=repo_id, filename=filename) with open(file_path, "r") as file: system_prompt = file.read() today = datetime.today().strftime("%Y-%m-%d") yesterday = (datetime.today() - timedelta(days=1)).strftime("%Y-%m-%d") model_name = repo_id.split("/")[-1] return system_prompt.format(name=model_name, today=today, yesterday=yesterday)
SYSTEM_PROMPT = load_system_prompt(model, "SYSTEM_PROMPT.txt")image_url = "https://static.wikia.nocookie.net/essentialsdocs/images/7/70/Battle.png/revision/latest?cb=20220523172438"
messages = [ {"role": "system", "content": SYSTEM_PROMPT}, { "role": "user", "content": [ { "type": "text", "text": "What action do you think I should take in this situation? List all the possible actions and explain why you think they are good or bad.", }, {"type": "image_url", "image_url": {"url": image_url}}, ], },]
response = client.chat.completions.create( model=model, messages=messages, temperature=TEMP, max_tokens=MAX_TOK,)
print(response.choices[0].message.content)import jsonfrom openai import OpenAIfrom huggingface_hub import hf_hub_download
# Modify OpenAI's API key and API base to use vLLM's API server.openai_api_key = "EMPTY"openai_api_base = "http://localhost:8000/v1"
TEMP = 0.15MAX_TOK = 262144
client = OpenAI( api_key=openai_api_key, base_url=openai_api_base,)
models = client.models.list()model = models.data[0].id
def load_system_prompt(repo_id: str, filename: str) -> str: file_path = hf_hub_download(repo_id=repo_id, filename=filename) with open(file_path, "r") as file: system_prompt = file.read() return system_prompt
SYSTEM_PROMPT = load_system_prompt(model, "SYSTEM_PROMPT.txt")
image_url = "https://math-coaching.com/img/fiche/46/expressions-mathematiques.jpg"
def my_calculator(expression: str) -> str: return str(eval(expression))
tools = [ { "type": "function", "function": { "name": "my_calculator", "description": "A calculator that can evaluate a mathematical expression.", "parameters": { "type": "object", "properties": { "expression": { "type": "string", "description": "The mathematical expression to evaluate.", }, }, "required": ["expression"], }, }, }, { "type": "function", "function": { "name": "rewrite", "description": "Rewrite a given text for improved clarity", "parameters": { "type": "object", "properties": { "text": { "type": "string", "description": "The input text to rewrite", } }, }, }, },]
messages = [ {"role": "system", "content": SYSTEM_PROMPT}, { "role": "user", "content": [ { "type": "text", "text": "Thanks to your calculator, compute the results for the equations that involve numbers displayed in the image.", }, { "type": "image_url", "image_url": { "url": image_url, }, }, ], },]
response = client.chat.completions.create( model=model, messages=messages, temperature=TEMP, max_tokens=MAX_TOK, tools=tools, tool_choice="auto",)
tool_calls = response.choices[0].message.tool_calls
results = []for tool_call in tool_calls: function_name = tool_call.function.name function_args = tool_call.function.arguments if function_name == "my_calculator": result = my_calculator(**json.loads(function_args)) results.append(result)
messages.append({"role": "assistant", "tool_calls": tool_calls})for tool_call, result in zip(tool_calls, results): messages.append( { "role": "tool", "tool_call_id": tool_call.id, "name": tool_call.function.name, "content": result, } )
response = client.chat.completions.create( model=model, messages=messages, temperature=TEMP, max_tokens=MAX_TOK,)
print(response.choices[0].message.content)from openai import OpenAIfrom huggingface_hub import hf_hub_download
# Modify OpenAI's API key and API base to use vLLM's API server.openai_api_key = "EMPTY"openai_api_base = "http://localhost:8000/v1"
TEMP = 0.15MAX_TOK = 262144
client = OpenAI( api_key=openai_api_key, base_url=openai_api_base,)
models = client.models.list()model = models.data[0].id
def load_system_prompt(repo_id: str, filename: str) -> str: file_path = hf_hub_download(repo_id=repo_id, filename=filename) with open(file_path, "r") as file: system_prompt = file.read() return system_prompt
SYSTEM_PROMPT = load_system_prompt(model, "SYSTEM_PROMPT.txt")
messages = [ {"role": "system", "content": SYSTEM_PROMPT}, { "role": "user", "content": "Write me a sentence where every word starts with the next letter in the alphabet - start with 'a' and end with 'z'.", },]
response = client.chat.completions.create( model=model, messages=messages, temperature=TEMP, max_tokens=MAX_TOK,)
assistant_message = response.choices[0].message.contentprint(assistant_message)uv pip install git+https://github.com/huggingface/transformerspip install mistral-common --upgradeimport torchfrom transformers import Mistral3ForConditionalGeneration, MistralCommonBackend
model_id = "mistralai/Ministral-3-3B-Instruct-2512"
tokenizer = MistralCommonBackend.from_pretrained(model_id)model = Mistral3ForConditionalGeneration.from_pretrained(model_id, device_map="auto")
image_url = "https://static.wikia.nocookie.net/essentialsdocs/images/7/70/Battle.png/revision/latest?cb=20220523172438"
messages = [ { "role": "user", "content": [ { "type": "text", "text": "What action do you think I should take in this situation? List all the possible actions and explain why you think they are good or bad.", }, {"type": "image_url", "image_url": {"url": image_url}}, ], },]
tokenized = tokenizer.apply_chat_template(messages, return_tensors="pt", return_dict=True)
tokenized["input_ids"] = tokenized["input_ids"].to(device="cuda")tokenized["pixel_values"] = tokenized["pixel_values"].to(dtype=torch.bfloat16, device="cuda")image_sizes = [tokenized["pixel_values"].shape[-2:]]
output = model.generate( **tokenized, image_sizes=image_sizes, max_new_tokens=512,)[0]
decoded_output = tokenizer.decode(output[len(tokenized["input_ids"][0]):])print(decoded_output)from transformers import Mistral3ForConditionalGeneration, FineGrainedFP8Config
model_id = "mistralai/Ministral-3-3B-Instruct-2512"model = Mistral3ForConditionalGeneration.from_pretrained( model_id, device_map="auto", quantization_config=FineGrainedFP8Config(dequantize=True))