No official torrent yet β requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from litert-community/Ministral-3-3B-Instruct-2512
All models by litert-community βmistralai/Ministral-3-3B-Instruct-2512
converted to the LiteRT-LM (.litertlm) format for on-device inference with
Google's LiteRT-LM runtime (the
engine behind the official litert-community/* models).
Text-only conversion (the Ministral-3 text decoder; the Pixtral vision tower is dropped).
Run with the LiteRT-LM runtime:
The .litertlm bundle carries the tokenizer and the prompt template (Ministral's
native Mistral [INST] β¦ [/INST] format, stop token </s>), so no separate
tokenizer files are needed. This is a direct-answering instruct model (no <think>
block) and terminates cleanly at </s>.
The easiest way to try this model on a phone is the official
Google AI Edge Gallery app β it
runs .litertlm models fully on-device and can import your own:
com.google.ai.edge.gallery, APK from the repo's
releases β 1.0.15+ supports
.litertlm). Older 1.0.x builds (package com.google.aiedge.gallery) only accept the
legacy MediaPipe .task format and reject .litertlm.Ministral-3-3B-Instruct-2512_q4_block32_ekv4096.litertlm from this repo and push it to the device:
.litertlm bundle already carries the
tokenizer and prompt template, so the model uses its native Mistral [INST]
chat format automatically.Device RAM (important β this is a ~2.7 GB / 3B model): GPU on Android needs roughly
2Γ the model size (the weights plus the ML Drift GPU weight cache), so GPU is only
offered on ~12 GB+ devices. On an 8 GB phone (e.g. Pixel 8a) only CPU is selectable,
and you must free RAM first (close apps / reboot β ~4 GB free) or the app is OOM-killed on
load. For smaller phones, prefer a 1β2B model (e.g. a Qwen3-1.7B .litertlm), which runs
comfortably and can use the GPU.
What the runtime can do, measured (added 2026-08-17): the paragraph above is about what the Gallery app offers β its accelerator choice is fixed at import time from device RAM. The model itself is not the limit: driving litert_lm_main directly on a Pixel 8a (Tensor G3, Mali-G715, 8 GB), this exact file runs entirely on the OpenCL delegate β 1187/1187 nodes in the 128-token prefill graph and 1087/1087 in decode, zero rejected ops β and answers correctly. So on an 8 GB phone the GPU is out of reach through the Gallery, not out of reach for the model.
See the Gallery
Importing Local Models
guide for details. To embed the model in your own Android app instead, use the
LiteRT-LM Kotlin API (Gradle artifact com.google.ai.edge.litertlm:litertlm-android,
getting started).
litert-lm benchmark (litert-lm 0.15.0) on an Apple M4 Max, -p 256 -d 256 --runs 3 (the tool averages three iterations), max-num-tokens 4096, warm-up run discarded, otherwise idle machine.
Reproducibility: the GPU rows repeat to within about 1% across invocations; the CPU rows are noisier β re-running the 1B control six times spread its CPU decode over 29.0β33.3 tok/s, so treat the CPU column as accurate to roughly Β±7%.
The iPhone row is three runs through the LiteRTDemo harness on iOS 27.0 (Metal GPU backend, prompt "Explain on-device AI in one short sentence.", 512-token budget, no warm-up turn). Decode is given as a range because it genuinely varies: 17.6 tok/s on the run that also loaded fastest (12 s) and 14.2 tok/s on the two runs that took about a minute to load. Prefill and TTFT were steady (39.7β42.3 tok/s, 0.40β0.42 s) and the post-generation footprint sat at 1.44β1.46 GB. The prefill figure is measured on that short prompt, so it reflects fixed per-turn overhead rather than prefill throughput and is not comparable to the 256-token desktop column.
Measured on GSM8K (n=100, greedy, 0-shot chain-of-thought asking for #### <n>,
identical prompt and answer-extraction for both rows so the only variable is the
on-device quantization:
LiteRT int4 is at parity: β4 pt vs bf16 with no reasoning collapse. The model also
passes the local quality gate 8/8 (non-degenerate, clean stop at </s>). 85% is a
strong on-device GSM8K for a 3B and far above a naive min-max int4 of the same model
(blockwise-32 + OCTAV optimal-clipping is what preserves the accuracy).
The published bundle runs on the Android GPU backend and generates.
Measured on a Samsung Galaxy S26 (SM-S942Q / SM8850, Android 16) with litert_lm_advanced_main from litert-lm 0.16.0, --backend=gpu --sampler_backend=cpu, prompt What is the capital of France?. Peak is the process high-water mark (VmHWM) sampled during that same run. Gated 2026-08-24.
The op counts above are the LiteRT GPU partitions. XNNPACK additionally takes 1 of the 4 nodes in decode_embedder and 1 of the 4 nodes in prefill_embedder_128; the runtime accepts that split.
No speed rows, on purpose. On this handset the GPU backend wins prefill and does not win decode, so a GPU throughput figure only means something beside a CPU row from the same handset, and no S26 CPU row exists for this model yet.
GPU wiring, including the Gallery import toggle: GPU guide.
Converted with litert-torch using a
blockwise int4 recipe (INT4 weights, block size 32, symmetric, OCTAV
optimal-clipping) with the tied embedding/lm_head kept at INT8, KV cache 4096, and
Ministral's native Mistral chat template. Ministral-3 is a standard dense decoder
(Ministral3ForCausalLM, YaRN RoPE), so it rides the existing converter and runtime
with no custom graph code; only the text decoder is exported (the vision tower is
dropped first).
externalize_embedder=True (required for iPhone). This 3B's weights would otherwise
be a single ~2.55 GiB TFLite section, which exceeds the ~2 GiB single-section mmap limit
on iOS β engine creation fails with "Failed to map section: Cannot allocate memory".
Externalizing the (tied) embedding into its own section drops the main weights section to
~1.8 GiB (and dedups the tied matrix, ~2.74 GB β 2.34 GB total), so the model loads on
iPhone (Metal GPU) as well as Android/desktop. Same weights, so GSM8K parity is
unchanged. Verified on-device: iPhone 17 Pro decodes at 14β18 tok/s on the
Metal GPU (prefill ~41 tok/s, TTFT 0.40 s, ~1.45 GB footprint) β the previously-failing
">2 GiB section / Cannot allocate memory" mmap error no longer occurs.
Template note (important for any Mistral/Ministral): the model must be exported
with its native Mistral [INST] β¦ [/INST] template and real EOS </s> β not
ChatML. Mistral's tekken tokenizer has no <|im_end|> token, so under a ChatML
template the int4 model never hits a registered stop token and runs away after the
correct answer. With the Mistral template it stops cleanly.
Built with litert-torch and a blockwise-32 + OCTAV int4 recipe, forcing the simple
Mistral [INST] chat template (the model's full jinja template doesn't render in the
runtime's minimal jinja engine, so the structured [INST] prefixes are extracted
instead):
The equivalent ai_edge_quantizer recipe is included as
ministral3_int4_block32_octav.json. The text decoder is extracted from the
multimodal checkpoint with scripts/extract_ministral3_text.py (drops the vision
tower; loads with missing=0/unexpected=0).
This is a weights-exact format conversion of mistralai/Ministral-3-3B-Instruct-2512; no new training was performed. Ministral was trained by Mistral AI on a large-scale, undisclosed proprietary mixture of web, code, and multilingual text. As the corpus is web-derived it may incidentally contain personal data/PII; none was deliberately collected and this format conversion adds none. Apply your own content/PII filtering before deployment. See the base model card for available details.
The upstream chat template emits a default system turn whenever the caller sends no system message β for this model: You are Ministral-3-3B-Instruct-2512, a Large Language Model (LLM) created by Mistral AI, a French startup headquartered in Paris. You power an AI assistant called Le Chat. Your knowledge base was last updated on 2023-10β¦. The converter's template probe renders the template with a system message already present, so that block was never seen and never reached the bundle: with no system message the model was running without the default system turn it was tuned with. The chat template in Ministral-3-3B-Instruct-2512_q4_block32_ekv4096.litertlm now emits the block exactly once when no system message is given. In Ministral-3-3B-Instruct-2512_q4_block32_ekv4096.litertlm, the block is not emitted when you pass a system message. The restored block adds 523 prefill tokens to a conversation that sends no system message, so time-to-first-token grows by that much; per-token speed is unchanged.
Metadata-only change: every section of the bundle except the LlmMetadata block is byte-identical to the previous file (verified by sha256 per section), so the weights, the graph and the tokenizer are unchanged and the speed and memory numbers on this card still describe exactly this file per token β only the file's own sha256 differs. What changed is the input: with no system message, the prompt now renders byte-identical to the upstream chat template's output, verified on the LiteRT-LM runtime. A system message you pass yourself renders as before. Greedy decoding can turn on a single token, so an individual answer can differ from the previous file. Unless a row says otherwise, the accuracy figures on this card were measured on the previous file and have not been re-measured on this one. If you downloaded before 2026-08-29, re-download.
Apache-2.0, inherited from the base model mistralai/Ministral-3-3B-Instruct-2512.
# build litert-lm from https://github.com/google-ai-edge/litert-lm, then:litert_lm_main \ --model_path Ministral-3-3B-Instruct-2512_q4_block32_ekv4096.litertlm \ --backend gpu \ --input_prompt "Explain on-device AI in one sentence."adb push Ministral-3-3B-Instruct-2512_q4_block32_ekv4096.litertlm /sdcard/Download/EXTERNALIZE_EMBEDDER=1 CACHE=4096 python scripts/export_simple_template.py \ src_models/ministral3-3b-text \ out/ministral3-3b-boctav4 \ templates/mistral_simple.jinja \ BOCTAV4 # blockwise-32 int4 + OCTAV, int8 embeddings