Ternary-Bonsai-1.7B β LiteRT-LM (INT2 / ternary, mobile GPU)
Ternary (INT2) .litertlm bundles of
prism-ml/Ternary-Bonsai-1.7B-unpacked,
packaged for the LiteRT-LM runtime and
Google AI Edge Gallery.
These run on the mobile GPU accelerator, not the NPU.
Recommended: bonsai-1.7b-int2pc-4k-gpu.litertlm. The plain, unmodified build with
no optimizations applied. Everything else in this repo is experimental β longer
context or lighter activations, but whether a given one loads depends on your app's
LiteRT / LiteRT-LM version and dependencies. Start with the recommended build; reach for
an experimental one only if you specifically need what it offers.
Builds
32768 is the model's full context β max_position_embeddings for Ternary-Bonsai-1.7B is 32768,
so the long-context builds here sit at the ceiling rather than truncating it.
All bundles carry Bonsai's own chat template (Qwen3 ChatML with the reasoning block intact).
Why several variants
The GPU accelerator runs a float graph β INT2 is a storage format, and compute happens
in fp16/fp32. Two consequences shape this list:
- Per-channel ternary dequantizes coherently. Block-quantized weights mix scales inside a
single GEMM, which is why the per-channel builds are the conservative choice.
- fp16 activations are lighter but less widely supported. The
fp16 bundles are the most
likely to meet a runtime that won't take them.
Support ranges by app, so the full set is published rather than a single "best" build.
Sampling defaults
Every bundle ships these in its LlmMetadata, so a LiteRT-LM host picks them up without
any configuration:
These are the values the bundles were built with. Override them in your host if you want
different behaviour.
Usage
Any LiteRT-LM host β the AI Edge Gallery app, or litert_lm_main β with the GPU backend selected.
Provenance and licensing
Created using Bonsai by Prism ML.
Training data: None was used here. These are post-training quantizations and repackagings of
the released Bonsai checkpoint; no additional training, fine-tuning, or calibration data was
involved. For the base model's training data, see the upstream Prism ML and Qwen3 model cards.
PII: No dataset was collected, processed, or shipped as part of this conversion, so no
personally identifiable information is present in these artifacts beyond whatever the upstream
released weights already encode.