Prism ML Website |
White Paper |
Demo & Examples |
Discord
Ternary-Bonsai-1.7B-mlx-2bit
Ternary (1.58-bit) language model for Apple Silicon
7.2x smaller than FP16 | 3.8x faster on M4 Pro | 103 tok/s on iPhone | runs on Mac, iPhone, iPad
Highlights
- 0.45 GiB (0.48 GB) packed 2-bit size (down from 3.44 GB FP16) — fits anywhere
- Ternary weights {-1, 0, +1} across embeddings, attention projections, MLP projections, and LM head
- 57.5 avg benchmark score across 6 categories
- 103 tok/s on iPhone 17 Pro Max
- MLX-native format with group size 128 and FP16 scaling
Resources
- White Paper
- Demo repo — examples for serving, benchmarking, and integrating Bonsai
- Discord — community support and updates
- Kernels: MLX (Apple Silicon) · mlx-swift (iOS/macOS) — 2-bit format is supported out of the box
Model Overview
Quantization Format: Ternary g128
Each weight takes a value from {-1, 0, +1}, with one shared FP16 scale per group of 128 weights:
The information-theoretic cost is log2(3) ≈ 1.585 bits per weight, plus FP16 group scales (16 bits per 128 weights), for a theoretical minimum of ~1.71 bits/weight. This release uses the MLX 2-bit format, which stores each ternary value in 2 bits plus group scales, for an effective ~2.125 bits/weight.
Memory
Quickstart
MLX (Python)
Throughput (MLX / Apple Silicon)
iPhone 17 Pro Max (MLX Swift)
Benchmarks
Evaluated with EvalScope v1.4.2 + vLLM 0.15.1 on NVIDIA H100. Full benchmark suite (10 benchmarks):
Intelligence Density
Citation
Contact
For questions, feedback, or collaboration inquiries: contact@prismml.com