Qwen2.5-Coder-3B-Instruct LiteRT-LM Model
This repository contains LiteRT-LM variant of Qwen/Qwen2.5-Coder-3B-Instruct optimized for on-device text generation.
Available Artifact
Integration
Ready to integrate this into your product? Get started in the LiteRT-LM documentation.
Performance (measured)
Apple M4 Max
Measured with the LiteRT-LM CLI: litert-lm benchmark -p 256 -d 256 --runs 3 --cache no
(litert-lm 0.15.0) on an idle Apple M4 Max (macOS); 256 prefill / 256 decode tokens, 3 iterations
averaged by the tool. A desktop reference point β phone-side figures vary by SoC and backend.
Galaxy S26 β GPU vs CPU (litert-lm 0.16.0)
Measured on a physical Samsung Galaxy S26 (SM-S942Q, Snapdragon 8 Elite Gen 5 / SM8850, Android 16) with litert_lm_advanced_main from the litert-lm v0.16.0 release; the GPU backend is OpenCL (LITERT_CL). One fixed 205-token prompt text (223 tokens under this tokenizer), --benchmark. Two runs per backend taken back-to-back β cells show the range. Peak RSS is the process VmHWM. Before quoting, the same file was run on each backend with a real prompt: both backends produced a correct text answer.
The GPU takes the whole graph β decode 1603/1603 ops and prefill 1452/1452 on LITERT_CL. It wins prefill 2.2β2.7Γ and decode 1.2β1.4Γ, and peaks 4.6Γ lower (864 against 3934 MB) β for a 3.4 GB int8 bundle the RSS difference is the practical headline.