orcarouter/Qwen3.8-27B-MLX

image-text-to-textmlxUpstream checksumsmlxsafetensorsqwen3_5qwenqwen3.8apple-silicon

License: apache-2.0
Size: not mirrored yet
Downloads in the last 30 days: 0 / 30 days

No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.

Cataloged from orcarouter/Qwen3.8-27B-MLX

All models by orcarouter →

Qwen3.8-27B-MLX

An MLX build of the official Qwen3.8-27B — 4 / 6 / 8-bit + MTP drafter for Apple Silicon

One Gateway. Every Model. — Route Smarter · Ship Safer · Spend Less.

Website · Model Catalog · Model API · GitHub · Discord · X

An MLX build of the official Qwen/Qwen3.8-27B — a 27B dense hybrid-attention (Gated DeltaNet linear + full attention) native vision-language model with thinking control, tool-calling and an MTP head — quantized to MLX for Apple Silicon. Three precisions (4 / 6 / 8-bit, affine) as subfolders, the 4-bit build also mirrored at the repo root (so orcarouter/Qwen3.8-27B-MLX loads directly in LM Studio), plus a separate mtp/ drafter for speculative decoding. The vision tower, norms and conv layers stay in BF16; only language-model linear weights (incl. embed_tokens / lm_head) are quantized. Browse all models in the OrcaRouter Model Catalog; deployed as API here.

Available quantizations

Folder	Bits	Group size	Size	Min Mac RAM	Quality vs BF16
`8-bit/`	8	32	~31 GB	32–64 GB	Near-lossless — best quality
`6-bit/`	6	32	~24 GB	32 GB	Excellent balance
`4-bit/`	4	64	~16 GB	24 GB	Very good — recommended default
`mtp/`	bf16	—	~0.9 GB	—	MTP drafter (speculative decoding)

Repo root = 4-bit/. --model orcarouter/Qwen3.8-27B-MLX (no subfolder) resolves to 4-bit.

Quality — KL-divergence vs BF16

Measured on this build: KL divergence and Top-1 token agreement of each quantized precision against the full-precision BF16 reference (wikitext-2, 1024 tokens, MLX CUDA backend). Lower KLD / higher Top-1 = closer to full precision.

Precision	Mean KLD	KLD p95	Top-1 agreement
8-bit	0.00068	0.00233	98.34 %
6-bit	0.00216	0.00795	96.97 %
4-bit	0.02824	0.10857	92.68 %

8-bit / 6-bit are near-lossless (Top-1 ≈ 97–98 %); 4-bit stays very close (Top-1 ≈ 93 %) — the recommended default for everyday use.

Multi-Token Prediction (MTP) — speculative decoding

Qwen3.8-27B has a native MTP head. In MLX it is loaded as a separate drafter: the main model is loaded with MTP stripped, and the drafter is passed explicitly. The drafter lives in mtp/ (model_type: qwen3_5_mtp, block_size 3) and works with any main precision.

bash

hf download orcarouter/Qwen3.8-27B-MLX --include "6-bit/*" "mtp/*" --local-dir ./Qwen3.8-27B-MLX
python -m mlx_vlm generate \    --model       ./Qwen3.8-27B-MLX/6-bit \    --draft-model ./Qwen3.8-27B-MLX/mtp \    --draft-kind mtp --draft-block-size 4 \    --prompt "Explain quantum entanglement in one sentence." --max-tokens 256

Requirements: an mlx-vlm build with the qwen3_5_mtp drafter and --draft-kind mtp (mlx-vlm main). MTP acceptance is lossless — greedy output is identical, just fewer forward passes.

Usage (mlx-vlm, Apple Silicon)

bash

pip install -U mlx-vlm    # needs mlx-vlm >= 0.6.13, mlx >= 0.32
hf download orcarouter/Qwen3.8-27B-MLX --include "4-bit/*" --local-dir ./Qwen3.8-27B-MLX
# textpython -m mlx_vlm generate --model ./Qwen3.8-27B-MLX/4-bit \    --prompt "Explain quantum entanglement in one sentence." --max-tokens 256# vision (image + text)python -m mlx_vlm generate --model ./Qwen3.8-27B-MLX/4-bit \    --image path/to/image.png --prompt "Describe this image." --max-tokens 256# OpenAI-compatible serverpython -m mlx_vlm server --model ./Qwen3.8-27B-MLX/4-bit --port 8080

On Apple Silicon the Metal backend is used automatically. (On a Linux CUDA backend, vision requires MLX_CUDA_USE_CUDNN_SDPA=0; not needed on macOS.)

LM Studio

Repo root is the 4-bit build. Two things: turn off KV-cache quantization (unsupported on this vision arch — load fails otherwise), and update the MLX runtime (Settings → Runtime) — qwen3_5 support landed in mlx-vlm 0.6.x.

Qwen3.8-27B

Following the widespread community adoption of the Qwen3.5 and Qwen3.6 series, Qwen3.8 is the most capable generation in the Qwen open-model family to date. Built on the architectural foundation of Qwen3.5, Qwen3.8-27B is a compact, deployment-friendly dense model: a native vision-language model that understands images and videos, with flexible thinking control, designed to carry complex, multi-step tasks through to completion with greater reliability.

Qwen3.8 Highlights

Core Capabilities: comprehensive improvements across coding, professional work, research, and long-horizon agentic tasks.
Agent Execution: stronger autonomous planning and better handling of environment feedback.
Downstream Compatibility: broader support for popular harnesses and development tools.
Flexible Thinking Control: thinking on by default, disable per request; tune depth with reasoning_effort, retain reasoning context via preserve_thinking.
Vision-Language Understanding: native image and video understanding, from STEM diagrams and documents to hour-scale videos.

Model Overview

Type: Causal Language Model with Vision Encoder
Parameters: 27B · Hidden: 5120 · Layers: 64 · Vocab: 248,320 (padded)
Hidden layout: 16 × (3 × (Gated DeltaNet → FFN) → 1 × (Gated Attention → FFN))
- Gated DeltaNet: 48 V heads / 16 QK heads, head dim 128
- Gated Attention: 24 Q heads / 4 KV heads, head dim 256, RoPE dim 64
- FFN intermediate: 17,408
MTP (Multi-Token Prediction): trained with multiple steps (shipped as the mtp/ drafter)
Context length: 262,144 native, extensible to 1,000,000 tokens

Best Practices

Sampling — thinking mode: temp=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0
Sampling — non-thinking: temp=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5
Output length: allow generous max-tokens; give ≥ 2048 headroom so the final answer isn't truncated by the thinking budget.
Ultra-long context: for total length beyond 262,144, use RoPE scaling (e.g. YaRN).

Citation

License

Apache 2.0, inherited from Qwen/Qwen3.8-27B. Quantization does not change the underlying license obligations.