litert-community/Qwen2.5-Coder-1.5B-Instruct

text-generationlitert-lmUpstream checksumslitert-lmlitertlitertlmon-deviceedgecode

License: apache-2.0
Size: not mirrored yet
Downloads in the last 30 days: 0 / 30 days

No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.

Cataloged from litert-community/Qwen2.5-Coder-1.5B-Instruct

Qwen2.5-Coder-1.5B-Instruct — LiteRT-LM

Qwen/Qwen2.5-Coder-1.5B-Instruct converted to the LiteRT-LM (.litertlm) format for on-device inference with Google's LiteRT-LM runtime. Requires litert-lm ≥ 0.16.

A 1.54B code model at 1.12 GB — small enough that a phone-class device runs it at conversational speed rather than merely running it. Code assistance is the use case that most needs that: it returns short outputs constantly, so decode rate is the experience.

File	Recipe	Size
`Qwen2.5-Coder-1.5B-Instruct_int4.litertlm`	int4 blockwise-32 + OCTAV on linears, int8 embedding (externalised)	1.12 GB

A 3B sibling exists in this org — litert-community/Qwen2.5-Coder-3B-Instruct, 3.43 GB, converted by another contributor. Pick by what you are optimising: that one for capability, this one for speed and footprint. The two have not been benchmarked against each other on the same machine, so this card makes no claim about their relative quality.

Correctness

Two gates, because a general-knowledge check certifies nothing for a code model:

8-question sanity gate: 8/8 on the GPU backend (Mac, litert-lm 0.16.0).
Code gate: 6/6. Six small functions — fib, reverse_words, is_prime, largest contiguous sublist sum, count_vowels, flatten — where the generated code is executed against assertions, not read. A task counts only if the code imports and every assertion holds.

Usage

bash

litert-lm run ./Qwen2.5-Coder-1.5B-Instruct_int4.litertlm \  --prompt "Write a Python function that returns the nth Fibonacci number. Code only."
# GPUlitert-lm run ./Qwen2.5-Coder-1.5B-Instruct_int4.litertlm --backend gpu --cache no --prompt "..."

The bundle carries the tokenizer and Qwen's ChatML markers including the vendor's default system prompt (You are Qwen, created by Alibaba Cloud. You are a helpful assistant.), which the upstream template inserts whenever no system message is supplied — rendering byte-identical to upstream for a single-turn request. A 4096-token KV budget and six prefill signatures (1024, 256, 64, 16, 4, 1).

Performance

litert-lm benchmark (litert-lm 0.16.0), Apple M4 Max, -p 256 -d 256 --cache no, quiet machine, serialized:

Backend	Prefill (256)	Decode	TTFT	Init
GPU (Metal)	3037 tok/s	137.8 tok/s	0.099 s	2.38 s
CPU	292 tok/s	47.1 tok/s	1.06 s	2.87 s

For scale, a 3B-class int4 bundle converted on the same day and measured on the same machine and protocol runs 1241 tok/s prefill / 86.3 decode / 0.233 s TTFT. Decode on these runtimes is memory-bandwidth-bound, so halving the weights is the most direct speed lever there is.

Honest notes:

A Mac's GPU-vs-CPU ratio does not transfer to a phone. Here the GPU decodes 2.9× the CPU; on a Pixel 8a a 3B model's GPU and CPU decode land within 1% of each other, because a phone's CPU and GPU share the same LPDDR. On phone hardware the GPU's win is prefill and time-to-first-token.
The embedding table is stored in its own bundle section. That is what keeps this file at 1.12 GB — with the table quantized inline it duplicates per prefill signature and the same weights come out at 2.53 GB.

Galaxy S26 — GPU backend

The published bundle runs on the Android GPU backend and generates.

file	GPU backend	delegation	peak
`Qwen2.5-Coder-1.5B-Instruct_int4.litertlm`	runs	8451 / 8451 ops across 7 subgraphs on `LiteRT GPU`	885 MB

Measured on a Samsung Galaxy S26 (SM-S942Q / SM8850, Android 16) with litert_lm_advanced_main from litert-lm 0.16.0, --backend=gpu --sampler_backend=cpu, prompt What is the capital of France?. Peak is the process high-water mark (VmHWM) sampled during that same run. Gated 2026-08-24.

The op counts above are the LiteRT GPU partitions. XNNPACK additionally takes 1 of the 4 nodes in decode_embedder and 1 of the 4 nodes in prefill_embedder_1024; the runtime accepts that split.

No speed rows, on purpose. On this handset the GPU backend wins prefill and does not win decode, so a GPU throughput figure only means something beside a CPU row from the same handset, and no S26 CPU row exists for this model yet.

GPU wiring, including the Gallery import toggle: GPU guide.

Conversion notes

Converted with litert-torch 0.9.3 / litert-converter 0.3.1 / ai-edge-quantizer 0.8.0 / litert-lm-builder 0.16.0 — a pristine released stack, no patched checkout.

EXTERNALIZE_EMBEDDER=1 is required, not cosmetic. Qwen2.5-Coder ties its embedding and lm_head, so a recipe asking for int4 linears and an int8 embedder describes one tensor two ways; the quantizer resolves it by copying the 151936×1536 table once per signature. Seven copies, 1.63 GB, 65% of the file — and no error anywhere. Externalising the table removes the conflict.
The default system prompt is baked into the template. Upstream inserts it when the caller sends no system message; a plain ChatML template would drop it and put the model in a state it was not tuned in on every default request.
Quantization: int4 blockwise-32 with OCTAV clipping on the linears, int8 on the embedding.

License and changes

Distributed under Apache-2.0 (inherited from the base model). Changes from the original work: weights converted from safetensors bf16 to LiteRT flatbuffers and quantized as described above; tokenizer and chat template repackaged into the .litertlm bundle. This repository is a community conversion and is not affiliated with Alibaba Cloud or the Qwen team.