No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from OpenMOSS-Team/MOSS-VL-Realtime-FP8
All models by OpenMOSS-Team →[Image blocked: MOSS-VL]
English | 中文
MOSS-VL is an open vision-language model family from OpenMOSS, supporting image understanding, long-video understanding, and realtime streaming interaction. This repository provides the FP8-quantized checkpoint of MOSS-VL-Realtime.
Technical Report: https://arxiv.org/pdf/2608.15045
This is the Transformers FP8 release of MOSS-VL-Realtime. It preserves the timestamp-aware streaming interface for frame-by-frame video inference. This checkpoint is not an SGLang release.
[Image blocked: MOSS-VL architecture]
generation_config.json enables HQQ KV8 automatically. Load the checkpoint
directly and do not pass a second quantization configuration or replace its
generation config with the BF16 source file.
The final evaluation compares the original BF16 model with all four release profiles on their corresponding benchmark suites. For this streaming FP8 checkpoint, the scores are 70.66 on OVOBench Avg, 62.93 on StreamingBench Avg, and 65.50 on OmniMMI PA, compared with 70.86, 62.42, and 66.00 for BF16.
[Image blocked: MOSS-VL quantization benchmark comparison]
The validated 30-frame streaming test peaked at 25,522 MiB of process VRAM and
26,249 MiB total GPU memory, including a 727 MiB baseline. Use an NVIDIA GPU
with more than 26 GiB available memory, or allow Transformers to shard the
model across multiple GPUs with device_map="auto".
Validated core versions:
Video decoding also requires FFmpeg in PATH.
Supply PIL-compatible frames with non-decreasing timestamps. One model instance supports one active realtime session.
The model may emit control tokens such as <|silence|>, <|round_start|>,
and <|round_end|>; applications should filter or render them according to
their protocol.
The fixed validation used a Xinjiang aerial video at 1 FPS with 30 timestamped frames. It completed 30/30 frames and produced a relevant Chinese tour-guide description.
Full inputs, commands and raw logs:
config.json: model and FP8 weight/activation configuration.generation_config.json: Transformers HQQ KV8 configuration.recipe.yaml: compressed-tensors quantization recipe.modeling_moss_vl.py: checkpoint-local streaming and quantized-cache code.git clone https://github.com/OpenMOSS/MOSS-VL.gitcd MOSS-VL
conda create -n moss_vl_quant python=3.12 pip -yconda activate moss_vl_quantpip install -i https://pypi.org/simple --no-build-isolation -r requirements.txtpip install -i https://pypi.org/simple \ compressed-tensors==0.14.0.1 \ hqq==0.2.8.post1python -m pip checkimport torchfrom transformers import AutoModelForCausalLM, AutoProcessor
checkpoint = "OpenMOSS-Team/MOSS-VL-Realtime-FP8"
processor = AutoProcessor.from_pretrained( checkpoint, trust_remote_code=True, frame_extract_num_threads=1,)model = AutoModelForCausalLM.from_pretrained( checkpoint, trust_remote_code=True, device_map="auto", torch_dtype=torch.bfloat16, attn_implementation="flash_attention_2",)model.eval()import timefrom PIL import Image
session = model.create_realtime_session( processor, initial_prompt=( "Describe important changes in the video as they happen. " "Stay silent when there is no meaningful update." ), frame_queue_size=1, max_tokens_per_turn=12, max_new_tokens=4096, do_sample=False,)
frame_paths = [ "data/frame_0001.jpg", "data/frame_0002.jpg", "data/frame_0003.jpg",]
try: session.start() for index, frame_path in enumerate(frame_paths): image = Image.open(frame_path).convert("RGB") session.push_frame(image, timestamp=float(index)) while True: chunk = session.poll_output(timeout=0.0) if chunk is None: break print(chunk, end="", flush=True) time.sleep(1.0)finally: session.close()source /inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/quant/venv/bin/activate
/inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/quant/venv/bin/python \ /inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/quant/benchmark_mossvl_quant.py \ --label streaming_fp8_reproduce \ --checkpoint /inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/final_release/quant/MOSS-VL-0708-Streaming-FP8-Dynamic-KV8-HQQ \ --gpu 0 \ --frames 30 \ --attention-backend flash_attention_2 \ --timeout 300/inspire/qb-ilm/project/video-understanding/public/train/moss_vl_streaming/8B/quant/transformers_validation_20260811@misc{mossvl, title = {MOSS-VL Technical Report}, author = {Wang, Pengyu and Tan, Chenkun and Zhou, Shaojun and Zhou, Qirui and Chen, Yanxin and He, Xingyang and Zeng, Huazheng and Cheng, Jijun and Wang, Chenghao and Qian, Xiaomeng and Wang, Pengfei and Huang, Zhan and Gao, Shanqing and Huang, Wei and Cao, Longjun and Ran, Wu and Liu, Jie and Zhu, Changtai and Wang, Hongkai and Tian, Yixian and Liu, Chenghao and Ye, Zhen and Wang, Xinghao and Jiang, Botian and Feng, Guoguo and Fei, Zhaoye and Li, Ruixiao and Chen, Mingshu and Gao, Yang and Cheng, Qinyuan and Li, Shimin and Qiu, Xipeng}, year = {2026}, eprint = {2608.15045}, archivePrefix = {arXiv}, primaryClass = {cs.CV}, url = {https://arxiv.org/abs/2608.15045}}
@misc{mossvideopreview, title = {{MOSS-Video-Preview: Toward Real-Time Video Understanding via Cross-Attention}}, author = {Pengyu Wang and Chenkun Tan and Shaojun Zhou and Wei Huang and Qirui Zhou and Zhan Huang and Zhen Ye and Jijun Cheng and Xiaomeng Qian and Yanxin Chen and Xingyang He and Huazheng Zeng and Chenghao Wang and Pengfei Wang and Hongkai Wang and Shanqing Gao and Yixian Tian and Chenghao Liu and Xinghao Wang and Botian Jiang and Xipeng Qiu}, year = {2026}, eprint = {2606.07639}, archivePrefix = {arXiv}, primaryClass = {cs.CV}, url = {https://arxiv.org/abs/2606.07639}}