No official torrent yet β requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from litert-community/Qwen3-4B-Thinking-2507
All models by litert-community βQwen/Qwen3-4B-Thinking-2507 converted to
the LiteRT-LM (.litertlm) format for on-device inference with Google's
LiteRT-LM runtime (the engine behind the official
litert-community/* models).
Qwen3-4B-Thinking-2507 is a dense 4B reasoning model (Qwen3ForCausalLM, 36 layers) that
operates exclusively in thinking mode β it emits a <think>β¦</think> chain before its answer β
so it rides the existing Qwen3 converter and runtime directly.
Qwen3_4b_thinking_dynamic_wi4b32_afp32.litertlm is a dynamic INT4 variant (block-32 weights, FP32 activations). It was converted through the LiteRT Torch (litert-torch) path and quantized with AI Edge Quantizer. This artifact incorporates LiteRT-LM GPU graph optimizations, including composite ops for RoPE, fused QKV, and fused Gate/Up projections, and is configured with static prefill memory allocation.
Update (2026-08-31) β thought channel and think pre-fill repaired in place. Both bundles received two metadata-only repairs today (weights byte-identical each time, verified section by section). First, the
thoughtchannel (<think>\n/\n</think>) was declared in the bundle metadata β without it the runtime streams raw reasoning inline into the answer and silently ignores any thinking budget. Second, the generation prompt now pre-opens the think block exactly as the upstream Qwen3-4B-Thinking-2507 template does (<|im_start|>assistant\n<think>\n):model.litertlm's structured assistant prefix and the block-32 file's embedded template both previously ended atassistant\n, leaving the model to emit<think>on its own β a discipline quantized reasoning models lose first. After the repairmodel.litertlmscores 8/8 on the 8-question sanity gate on both macOS backends (CPU and Metal GPU, max-tokens 2048), and the block-32 file scores 8/8 on CPU.
This model generates a <think>β¦</think> reasoning chain, then the answer. Run it with
max_tokens β₯ 2048 β at a short limit it gets cut off mid-thought and never reaches the answer.
(All quality numbers below were measured at 2048.)
litert-lm benchmark (litert-lm 0.15.0) on an Apple M4 Max, -p 256 -d 256 --runs 3 (the tool averages three iterations), max-num-tokens 4096, warm-up run discarded, otherwise idle machine.
Reproducibility: the GPU rows repeat to within about 1% across invocations; the CPU rows are noisier β re-running the 1B control six times spread its CPU decode over 29.0β33.3 tok/s, so treat the CPU column as accurate to roughly Β±7%.
The iPhone row is carried over from this repository's own earlier on-device note (LiteRT-LM Swift runtime); its run log is not retained here, so the run count and prompt are not known.
Measured on GSM8K (n=100, greedy, 0-shot chain-of-thought, max_tokens 2048, identical prompt and answer-extraction for every row).
int4 is at parity (β4 pt). Note: evaluating a reasoning model at a short token budget badly
understates int4 β the longer int4 reasoning chains get truncated before the answer; benchmark
reasoning models with max_tokens β₯ 2048.
Why block 128 (and not block 32)? For this reasoning model the block-32 build degraded more
(β9 pt) and produced corrupted output under the iPhone GPU delegate, while block 128 is robust
on every backend, ~40 % faster to decode (ΒΌ the dequant scales β which matters when generating
long <think> chains), and stays at β4 pt parity. So block 128 is the build we recommend; the block-32 file is also published (see Artifact (Block 32) above), and both bundles run fully delegated on the Galaxy S26 GPU.
The block-32 file's GPU corruption is not iPhone-specific: on macOS Metal it degenerates too (question-echo loops, truncated reasoning; measured 2026-08-31 on the file both before and after the metadata repair, so it is pre-existing and unrelated to the repair). Use the block-32 file on CPU; on Android GPU it delegated and generated in the Galaxy S26 gate above.
Both published bundles run on the Android GPU backend and generate.
Measured on a Samsung Galaxy S26 (SM-S942Q / SM8850, Android 16) with litert_lm_advanced_main from litert-lm 0.16.0, --backend=gpu --sampler_backend=cpu, prompt What is the capital of France?. Peak is the process high-water mark (VmHWM) sampled during that same run. Gated 2026-08-24.
The op counts above are the LiteRT GPU partitions. In model.litertlm, XNNPACK additionally takes 1 of the 4 nodes in decode_embedder and 1 of the 4 nodes in prefill_embedder_128; the runtime accepts that split.
No speed rows, on purpose. On this handset the GPU backend wins prefill and does not win decode, so a GPU throughput figure only means something beside a CPU row from the same handset, and no S26 CPU row exists for this model yet.
GPU wiring, including the Gallery import toggle: GPU guide.
The .litertlm bundle carries the tokenizer and prompt template (Qwen3 ChatML β
<|im_start|>role\nβ¦<|im_end|>, stop token <|im_end|>), so no separate tokenizer files are
needed. The model produces a <think>β¦</think> block followed by its answer.
Update (July 2026): Google AI Edge Gallery v1.0.16+ can import litert-lm models directly from Hugging Face inside the app (tap +) β no computer or
adbneeded. The manual steps below are only required on older builds or for sideloading a local file.
The official Google AI Edge Gallery app runs
.litertlm models on-device:
com.google.ai.edge.gallery, 1.0.15+ supports .litertlm).model.litertlm and push it: adb push model.litertlm /sdcard/Download/A 4B int4 build needs ~2.5 GB free RAM; reboot the phone first if memory is tight.
The same .litertlm bundle runs on macOS / Linux / Windows with the official
LiteRT-LM CLI β including as a
local OpenAI-compatible API server:
Verified on iPhone 17 Pro (LiteRT-LM Swift runtime): loads and generates at ~14 tok/s.
Converted with the official litert-torch
converter β a standard Qwen3ForCausalLM, so it uses the existing Qwen3 path with no custom graph
code. Recipe: blockwise-128 int4 + OCTAV (INT4 weights, block 128, symmetric, OCTAV
optimal-clipping), embeddings INT8, KV cache 4096.
Apache-2.0, inherited from the base model Qwen/Qwen3-4B-Thinking-2507.
# build litert-lm from https://github.com/google-ai-edge/litert-lm, then:litert_lm_main \ --model_path model.litertlm \ --backend gpu \ --input_prompt "A bat and a ball cost \$1.10. The bat costs \$1.00 more than the ball. How much is the ball?"pip install litert-lmlitert-lm import --from-huggingface-repo litert-community/Qwen3-4B-Thinking-2507 model.litertlm qwen3-4b-thinking-2507litert-lm run qwen3-4b-thinking-2507 # interactive chat in the terminallitert-lm serve # local OpenAI-compatible API serverfrom litert_torch.generative.export_hf.export import exportexport( model="Qwen/Qwen3-4B-Thinking-2507", output_dir="out", quantization_recipe="qwen3_int4_block128_octav.json", # blockwise-128 int4 + OCTAV, int8 embeddings cache_length=4096, externalize_embedder=True,)