No official torrent yet β requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from prism-ml/Ternary-Bonsai-27B-AWQ-4bit
All models by prism-ml βpip install sglang
python -m sglang.launch_server \ --model prism-ml/Ternary-Bonsai-27B-AWQ-4bit \ --port 8000 \ --dtype bfloat16curl http://localhost:8000/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{"model":"prism-ml/Ternary-Bonsai-27B-AWQ-4bit","messages":[{"role":"user","content":"Who are you?"}],"max_tokens":100}'# TP=2 DP=4 β 4 replicas, each split across 2 GPUs (throughput sweet spot)python -m sglang.launch_server \ --model prism-ml/Ternary-Bonsai-27B-AWQ-4bit \ --tp-size 2 --dp-size 4 \ --load-balance-method total_tokens \ --port 8000 --dtype bfloat16
# TP=2 β single replica split across 2 GPUspython -m sglang.launch_server \ --model prism-ml/Ternary-Bonsai-27B-AWQ-4bit \ --tp-size 2 \ --port 8000 --dtype bfloat16