litert-community/Qwen2-0.5B-Instruct

text-generationlitert-lmUpstream checksumslitert-lmlitertlmqwenQwen2text-generationen

License: apache-2.0
Size: not mirrored yet
Downloads in the last 30 days: 0 / 30 days

No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.

Cataloged from litert-community/Qwen2-0.5B-Instruct

All models by litert-community →

Qwen2-0.5B-Instruct LiteRT-LM Model

This repository contains LiteRT-LM variants of Qwen/Qwen2-0.5B-Instruct optimized for on-device text generation.

Available Artifact

File	Quantization Recipe	Context	Size
`Qwen2_0.5B_Instruct.litertlm`	dynamic_wi8_afp32	-	647.4 MB

Performance (on device, measured)

Community measurement on a physical Samsung Galaxy S26 (SM-S942Q, Snapdragon 8 Elite Gen 5 / SM8850, Android 16): litert_lm_advanced_main from the litert-lm v0.16.0 release, GPU backend OpenCL (LITERT_CL) against CPU (XNNPACK), one fixed 205-token prompt text (213 tokens under this tokenizer), --benchmark. Two runs per backend taken back-to-back — cells show the range. Peak RSS is the process VmHWM. Before quoting, the file was run on each backend with a real prompt and both produced a correct text answer; the GPU rows are full delegation of the transformer graphs (decode 1063/1063 and prefill 972/972 ops on LITERT_CL; only the tiny embedding-lookup helper graphs stay on the CPU).

Backend	Prefill (213 tok)	Decode	Time-to-first-token	Init	Peak RSS
GPU (OpenCL)	1259–1452 tok/s	61.9–62.9 tok/s	0.16–0.19 s	0.8 s	559–561 MB
CPU (XNNPACK)	405–528 tok/s	49.1–54.5 tok/s	0.42–0.55 s	0.7–1.0 s	1005–1044 MB

What the table says:

The GPU leads everywhere on this bundle: prefill 2.7–3.6×, decode ~1.2×, and peak RSS 1.8× lower (560 against ~1020 MB) — for a 0.5B int8 model that decode margin is unusual; most small bundles tie the CPU on decode.
GPU engine init is 0.8 s — unusually cheap for the GPU path (most bundles pay several seconds), so the GPU backend costs almost nothing extra even for short-lived processes.

Integration

Ready to integrate this into your product? Get started in the LiteRT-LM documentation.