webbrain-one/Ling-3.0-tiny-ONNX

text-generationtransformers.jsUpstream checksumstransformers.jsonnxqwen3_5_texttext-generationwebgpuonnxruntime

License: mit
Size: not mirrored yet
Downloads in the last 30 days: 0 / 30 days

No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.

Cataloged from webbrain-one/Ling-3.0-tiny-ONNX

Ling-3.0-tiny ONNX — Q4 WebGPU + CUDA

Community ONNX conversion of inclusionAI/Ling-3.0-tiny, a 7.9B-total / 1.3B-active-parameter hybrid reasoning MoE model.

[!IMPORTANT] This is an independent community conversion, not an official inclusionAI release. Read the original model card for training, evaluation, intended use, and base-model limitations.

Target	Files	Runtime	Download
Browser WebGPU	`onnx/model_q4f16.onnx` + 3 data shards	Transformers.js 4.2+	4.849 GB / 4.516 GiB
NVIDIA CUDA	`model.onnx` + `model.onnx.data`	ONNX Runtime GenAI	4.849 GB / 4.516 GiB

The WebGPU layout follows the standard Transformers.js q4f16 contract used by browser-oriented ONNX repositories: the graph is under onnx/, external tensor data is split into three sub-2 GB files, activations and cache are FP16, and hybrid recurrent-cache names use the Qwen3.5-compatible convention already supported by Transformers.js.

Run in the browser with WebGPU

Install Transformers.js 4.2 or newer:

javascript

import { pipeline, TextStreamer } from '@huggingface/transformers';
const generator = await pipeline(  'text-generation',  'webbrain-one/Ling-3.0-tiny-ONNX',  {    device: 'webgpu',    dtype: 'q4f16',  },);
const messages = [  { role: 'user', content: 'Explain why the sky is blue in two sentences.' },];
const output = await generator(messages, {  max_new_tokens: 128,  do_sample: false,  tokenizer_encode_kwargs: { enable_thinking: false },  streamer: new TextStreamer(generator.tokenizer, {    skip_prompt: true,    skip_special_tokens: true,  }),});
console.log(output[0].generated_text.at(-1)?.content);

For thinking mode, set enable_thinking: true. The original model card recommends temperature: 1.0, top_p: 0.95, and top_k: 20 when sampling in thinking mode.

Browser requirements

A current desktop browser with WebGPU enabled; Chrome or Edge is recommended.
Approximately 4.85 GB of model downloads on first load, plus browser cache.
Enough GPU memory for all weights, runtime buffers, state, and the requested context. The 1.3B active-parameter figure reduces compute, but all 7.9B model parameters still need to be stored.
Start with a short prompt and modest max_new_tokens, then increase context after confirming memory use on the target device.

The validation machine had two discrete NVIDIA GPUs, and Chromium selected the display-connected adapter. On multi-GPU systems, check the adapter selected by the browser rather than assuming it will match a CUDA compute workload.

This graph depends on WebGPU implementations of MatMulNBits, QMoE, LinearAttention, CausalConvWithState, and GroupQueryAttention. It is not a WASM/CPU fallback model.

Use from WebBrain

Use the same repository ID and standard Transformers.js settings:

WebBrain should select onnx/model_q4f16.onnx and fetch the three external-data files declared by config.json.

Run the CUDA variant with Python

The repository also retains the separately validated CUDA-targeted ONNX Runtime GenAI graph.

python

import numpy as npimport onnxruntime_genai as ogfrom transformers import AutoTokenizer
model_dir = "Ling-3.0-tiny-ONNX"model = og.Model(model_dir)tokenizer = AutoTokenizer.from_pretrained(model_dir, trust_remote_code=True)
input_ids = tokenizer.apply_chat_template(    [{"role": "user", "content": "Explain why the sky is blue."}],    add_generation_prompt=True,    tokenize=True,    return_tensors="np",    enable_thinking=False,)
params = og.GeneratorParams(model)params.set_search_options(    max_length=int(input_ids.shape[-1]) + 128,    do_sample=False,)
generator = og.Generator(model, params)generator.append_tokens(np.asarray(input_ids[0], dtype=np.int32))prompt_length = int(input_ids.shape[-1])
while not generator.is_done():    generator.generate_next_token()
print(tokenizer.decode(generator.get_sequence(0)[prompt_length:], skip_special_tokens=True))

Architecture and quantization

18 Kimi Delta Attention layers with recurrent and convolution state
6 Multi-Latent Attention layers with KV cache
23 sparse MoE layers with 128 routed experts, top-8 group-limited routing, expert bias, and one shared expert
one dense MLP layer at the start of the decoder
symmetric Q4/block-32 dense and routed-expert weights
FP16 embeddings, activations, recurrent state, and KV cache
FP32 MoE router weights and routing math

The ONNX graph contains 235 MatMulNBits, 23 QMoE, 18 LinearAttention, 18 CausalConvWithState, and 6 GroupQueryAttention nodes.

Validation

The complete CUDA artifact generated successfully on an NVIDIA GeForce RTX 5090 with ONNX Runtime GenAI 0.15.2 and ONNX Runtime GPU 1.28.0.
The WebGPU repack was verified tensor-for-tensor against the CUDA graph; all 4,835,749,912 external tensor bytes are identical.
Transformers.js 4.2.0 loads the config, dispatches Qwen3_5ForCausalLM, finds all 18 recurrent/conv caches and 6 attention caches, and applies the original Ling tokenizer/chat template.
A full remote-repository test passed in Chrome 150 with Transformers.js 4.2.0 on an NVIDIA T400 4GB WebGPU adapter using Windows shared-memory oversubscription. First load/session creation took 925.87 seconds; a deterministic 16-token generation took 90.18 seconds and produced a coherent answer. This is a compatibility smoke test, not a performance benchmark.

Limitations

Q4 quantization can change outputs and quality relative to the original BF16 checkpoint. No benchmark parity claim is made here.
Browser support, GPU limits, shader compilation time, and memory behavior vary by operating system, browser version, and GPU driver.
The advertised 131,072-token context is architectural; practical browser context is limited by available GPU memory.
The WebGPU config uses Transformers.js's existing qwen3_5_text hybrid-cache adapter solely as a runtime compatibility layer. The underlying graph and weights remain Ling/Bailing Hybrid, preserved in config_bailing_original.json.

Attribution and license

The model architecture, checkpoint, tokenizer, and chat template are by inclusionAI. This conversion retains the base model's MIT license. Please cite and credit the original Ling-3.0-tiny release when using or redistributing this artifact.