No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from prism-ml/Bonsai-8B-gguf
All models by prism-ml →
Prism ML Website | Whitepaper | Demo & Examples | Colab Notebook | Discord
End-to-end 1-bit language model for llama.cpp (CUDA, Metal, CPU)
14.1x smaller than FP16 | 6.2x faster on RTX 4090 | 4-5x lower energy/token
Each weight is a single bit: 0 maps to −scale, 1 maps to +scale. Every group of 128 weights shares one FP16 scale factor.
Effective bits per weight: 1.125 (1 sign bit + 16-bit scale amortized over 128 weights).
Parameter memory only (weights and scales loaded into memory):
The GGUF file on disk is 1.16 GB (~6.6 MB larger) because the format embeds the tokenizer, chat template, and model metadata alongside the weights.
You can use a simple system prompt such as:
Open the web UI at http://127.0.0.1:8080, or see our llama.cpp fork for more examples.
¹ FP16 only fits partially on GPU's 6 GB VRAM; 1-bit fits entirely in VRAM.
Evaluated with EvalScope v1.4.2 + vLLM 0.15.1 on NVIDIA H100 under identical infrastructure, generation parameters, and scoring. All models are in the 6B–9B parameter range.
Despite being 1/14th the size, 1-bit Bonsai 8B is competitive with leading full-precision 8B instruct models.
Intelligence density captures the ratio of a model's capability to its deployed size:
Bonsai 8B achieves 10.8x higher intelligence density than full-precision Qwen 3 8B.
If you use 1-bit Bonsai 8B, please cite:
For questions, feedback, or collaboration inquiries: contact@prismml.com
You are a helpful assistant# Clone the PrismML fork of llama.cpp (includes Q1_0 kernels)git clone https://github.com/PrismML-Eng/llama.cppcd llama.cpp
# Build with CUDA supportcmake -B build -DGGML_CUDA=ON && cmake --build build -j
# Run inference./build/bin/llama-cli \ -m Bonsai-8B-Q1_0.gguf \ -p "Explain quantum computing in simple terms." \ -n 256 \ --temp 0.5 \ --top-p 0.85 \ --top-k 20 \ -ngl 99# Clone the PrismML fork of llama.cpp (includes Q1_0 kernels)git clone https://github.com/PrismML-Eng/llama.cppcd llama.cpp
# Build with Metal support (default on macOS)cmake -B build && cmake --build build -j
# Run inference./build/bin/llama-cli \ -m Bonsai-8B-Q1_0.gguf \ -p "Explain quantum computing in simple terms." \ -n 256 \ --temp 0.5 \ --top-p 0.85 \ --top-k 20 \ -ngl 99./build/bin/llama-server \ -m Bonsai-8B-Q1_0.gguf \ --host 0.0.0.0 \ --port 8080 \ -ngl 99alpha = -ln(1 - score/100) / size_GB@techreport{bonsai8b, title = {1-bit Bonsai 8B: End-to-End 1-bit Language Model Deployment Across Apple, GPU, and Mobile Runtimes}, author = {Prism ML}, year = {2026}, month = {March}, url = {https://prismml.com}}