Mirrored from Edge0/Audio8-ASR-Infinite
All models by Edge0 βAudio8 ASR Infinite is a native streaming speech recognition model built to be as responsive as possible. It offers a selectable audio clock (80/120/160 ms) and a transcription delay (240β560 ms). With our adapted vLLM build it transcribes unlimited-length audio 24/7 without drifting.
The checkpoint has a native context of 30 seconds. But with Rolling KV Cache, it can transcribe 24/7 nonstop.
The following combinations of frame length and delay are post-trained. Other combinations can be used but performance may not be optimum.
target_delay_ms must be an integer multiple of the selected clock, so longer
delays stay available at every clock even when they are not listed above.
Inherits the Voxtral realtime audio architecture and DSM-style streaming.
Checkpoint specification:
This is the preview release: it delivers the transcription base. Realtime semantic perception is being built on the same frame grid and the same acoustic forward pass.
Greedy decode with EOS suppressed, at the 80 ms audio clock with
target_delay_ms = 480 (6 delay tokens). Error rates in percent. No repetition
loops and no dropped trailing words.
Programmatic simulated-streaming decode with the embedded remote code:
Only a full merged weight directory is supported (this repository as-is); adapter-style or partially converted weights are not.
Docker compose is the canonical deployment path; it also serves the web demo:
Verify with the web client shipped in the same stack:
The same socket can be driven from a terminal:
18191 is the host port published by docker/docker-compose.yml; the service
itself listens on 18190 inside the compose network. The rolling KV window is
30 s with exact RoPE re-basing, which is what keeps memory and latency bounded
over 24/7 operation.
import numpy as npimport torchfrom transformers import AutoFeatureExtractor, AutoTokenizer
from audio8_asr_infinite.modeling.modeling_audio8_asr_infinite import ( Audio8ASRInfiniteForConditionalGeneration, resolve_qwen_language_token_id, resolve_qwen_streaming_special_token_ids,)from audio8_asr_infinite.streaming_inference import simulated_streaming_greedy_decode_batch
checkpoint = "Edge0/Audio8-ASR-Infinite"tokenizer = AutoTokenizer.from_pretrained(checkpoint, trust_remote_code=True)feature_extractor = AutoFeatureExtractor.from_pretrained(checkpoint, trust_remote_code=True)model = Audio8ASRInfiniteForConditionalGeneration.from_pretrained( checkpoint, trust_remote_code=True, torch_dtype=torch.bfloat16).eval().cuda()
class AudioConfig: # duck-typed: raw_audio_samples_per_token / streaming_n_left_pad_tokens / sampling_rate raw_audio_samples_per_token = 1280 # 80 ms @ 16 kHz streaming_n_left_pad_tokens = 18 sampling_rate = 16000
waveform = np.load("sample.npy", allow_pickle=False).astype(np.float32) # [-1, 1], 16 kHz monoresults = simulated_streaming_greedy_decode_batch( model=model, tokenizer=tokenizer, feature_extractor=feature_extractor, waveforms=[waveform], language_token_ids=[resolve_qwen_language_token_id(tokenizer, "zh")], special_ids=resolve_qwen_streaming_special_token_ids(tokenizer), audio_config=AudioConfig(), num_delay_tokens=[480 // 80], right_pad_text_tokens=10, dtype=torch.bfloat16, device=next(model.parameters()).device, max_new_tokens=512,)print(results[0]["final_text"])cd dockerAUDIO8_MODEL_DIR=/path/to/checkpoint docker compose up -dhttp://localhost:8080/ # plain HTTPhttps://localhost:8443/ # TLS proxy; accept the self-signed certificatepython -m audio8_asr_infinite.examples.vllm_realtime_client \ --ws-url ws://127.0.0.1:18191/v1/realtime \ --audio sample.wav --language zh --target-delay-ms 480 --pacepython -m audio8_asr_infinite.examples.torch_streaming_decode \ --checkpoint /path/to/checkpoint \ --audio sample.wav --language zh --transcription-delay-ms 480