inclusionAI/Ming-3.0-audio-flash

text-to-speechtransformersUpstream checksumstransformersonnxsafetensorsbailing_moe_v3_audiofeature-extractionaudio

License: mit
Size: not mirrored yet
Downloads in the last 30 days: 0 / 30 days

No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.

Cataloged from inclusionAI/Ming-3.0-audio-flash

[Image blocked: Ming logo]

Ming-3.0-audio-flash

Hear clearly. Understand deeply. Act intelligently.

Ming-3.0-audio-flash is a native audio-language model from inclusionAI. It accepts audio together with text instructions and supports speech recognition, spoken question answering, audio understanding, audio reasoning, and optional synthesized-speech responses.

Model Summary

Property	Description
Developer	inclusionAI
Model type	Native audio-language Mixture-of-Experts model
Parameters	126B total parameters, 6.5B activated per inference step
Inputs	Audio and text instructions
Outputs	Text, or text with optional synthesized speech
Primary languages	Chinese and English, including varied accents and dialects
License	MIT

Highlights

Complex speech recognition: Handles Chinese and English speech, dialects, accents, noisy environments, and multi-speaker recordings.
Long-audio understanding: Comprehends and analyzes long-form recordings.
Multidimensional perception: Understands speech, non-speech sounds, acoustic scenes, and music.
Audio reasoning: Answers questions and performs reasoning over audio content.
Audio interaction: Produces text responses and can optionally synthesize speech with built-in voices or a reference voice.

Architecture

Ming-3.0-audio-flash combines an audio encoder and projector with a Mixture-of-Experts language backbone. An optional Talker generates speech responses from the model's output.

[Image blocked: Ming-3.0-audio-flash architecture]

Quick Start with Transformers

The model uses custom Transformers code. Review the repository code before enabling trust_remote_code=True.

bash

git clone https://github.com/inclusionAI/Ming-3.0-audio-flash.gitcd Ming-3.0-audio-flash
python -m venv .venv-transformerssource .venv-transformers/bin/activatepip install -r requirements-transformers.txt

Audio Understanding

Speech Recognition

The validated ASR prompt asks the model to identify the spoken language and place a tab between the language label and transcription.

Direct Python Usage

python

import torchfrom transformers import AutoModelForCausalLM, AutoProcessor
model_id = "inclusionAI/Ming-3.0-audio-flash"audio_path = "examples/audio/introduce_ant_group.wav"
processor = AutoProcessor.from_pretrained(    model_id,    trust_remote_code=True,    use_fast=False,)model = AutoModelForCausalLM.from_pretrained(    model_id,    dtype=torch.bfloat16,    device_map="auto",    trust_remote_code=True,).eval()
messages = [    {        "role": "user",        "content": [            {"type": "audio", "audio": audio_path},            {"type": "text", "text": "Please answer the question in the audio."},        ],    }]
inputs = processor(messages).to(model.device)inputs["audio_feats"] = inputs["audio_feats"].to(model.dtype)
with torch.inference_mode():    output_ids = model.generate(        **inputs,        max_new_tokens=512,        do_sample=False,    )
generated_ids = output_ids[0, inputs["input_ids"].shape[1] :]response = processor.decode(generated_ids, skip_special_tokens=True).strip()print(response)

Speech Output

The Talker example supports built-in voices and reference-audio voice prompting:

See the voice guide for available voice names and reference-audio usage.

Serving

vLLM-Omni

The reference 128K serving profile uses tensor parallelism across four 80 GB H800 GPUs. Select the deployment YAML that matches text-only or Talker output. On particularly long audio inputs, the four-GPU profiles may OOM; in that case, switch to the corresponding eight-GPU fallback profile (deploy/*_8gpu.yaml), which sets tensor_parallel_size: 8 across GPUs 0–7 (and the Talker reuses GPU 7). See the deployment guide for details.

For speech output, start the audio profile and select an IP or registered voice with the vLLM-Omni client's --voice option:

bash

MING_OUTPUT_MODE=audio ./scripts/serve_vllm_omni.sh "$MODEL_PATH"python examples/vllm_omni_audio_chat.py examples/audio/introduce_ant_group.wav \  --audio-response \  --voice US_Stella \  --output-audio outputs/answer.wav

The client sends this selection as Chat Completions audio.voice: {"id": "..."}. vLLM-Omni uses the built-in IP voice 灵小柔 when no voice is specified. The model directory also provides 灵楷 and US_James as preset reference voices under voice_presets/voice_audio/, with their matching transcripts in voice_presets/voice_prompts.json. Both names are also available as built-in IP voices.

SGLang

The validated SGLang profile uses four GPUs and provides text output. See the SGLang example for client usage.

SGLang-Omni

SGLang-Omni serves transcription, audio chat, and speech synthesis from one server. Serving uses two GPUs: --thinker.tp_size 2 places the thinker on both visible devices, the audio encoder uses GPU 0, and speech synthesis uses GPU 1.

The server exposes OpenAI-compatible /v1/audio/transcriptions, /v1/audio/speech, and /v1/chat/completions endpoints. See the SGLang-Omni cookbook for client examples, including voice cloning with ref_audio and ref_text.

Evaluation

Speech Recognition

[Image blocked: Speech recognition benchmarks]

Audio Intelligence

[Image blocked: Audio intelligence benchmarks]

Text Benchmarks

[Image blocked: Text benchmarks]

All benchmark values are percentages. Higher is better unless the metric is marked with a downward arrow. Results marked with an asterisk are self-evaluated. Refer to the GitHub README for the latest notes and evaluation details.

License

This project is released under the MIT License. See LEGAL.md for the legal disclaimer.