Qwen-AgentWorld-35B-A3B
[!Note]
This repository contains the model weights and configuration files for Qwen-AgentWorld-35B-A3B, a native language world model trained for agentic environment simulation.
These artifacts are compatible with Hugging Face Transformers, vLLM, SGLang, etc.
Qwen-AgentWorld is the first language world model to cover seven agent interaction domains within a single model. It simulates agentic environments via long chain-of-thought reasoning, predicting the next environment state given an agent's action and interaction history. Trained through a three-stage pipeline β CPT injects environment knowledge, SFT activates next-state-prediction reasoning, RL sharpens simulation fidelity β Qwen-AgentWorld is a native world model: environment modeling is the training objective from the CPT stage onward, not a post-hoc add-on.
Highlights
- Seven Unified Domains. A single model covers MCP (tool calling), Search, Terminal, SWE (software engineering), Android, Web, and OS β spanning both text and GUI interaction environments.
- Native World Model. Environment modeling from CPT onward, not post-hoc adaptation on a general-purpose LLM.
- Generalizable, Scalable & Controllable Simulator. Zero-shot generalization to OOD environments (e.g., OpenClaw); controllable perturbations and fictional-world construction surpass real-environment training.
- Agent Foundation Model. LWM RL warm-up on single-turn, non-agentic trajectories transfers to multi-turn, tool-calling agentic tasks across 7 benchmarks, including 3 entirely out-of-domain.
Model Overview
- Type: Causal Language Model (Language World Model)
- Base Model: Qwen3.5-35B-A3B-Base
- Training Stage: Continual Pre-Training (CPT) β Supervised Fine-Tuning (SFT) β Reinforcement Learning (RL, GSPO)
- Number of Parameters: 35B in total and 3B activated
- Hidden Dimension: 2048
- Token Embedding: 248320 (Padded)
- Number of Layers: 40
- Hidden Layout: 10 Γ (3 Γ (Gated DeltaNet β MoE) β 1 Γ (Gated Attention β MoE))
- Gated DeltaNet:
- Number of Linear Attention Heads: 32 for V and 16 for QK
- Head Dimension: 128
- Gated Attention:
- Number of Attention Heads: 16 for Q and 2 for KV
- Head Dimension: 256
- Rotary Position Embedding Dimension: 64
- Mixture Of Experts
- Number of Experts: 256
- Number of Activated Experts: 8 Routed + 1 Shared
- Expert Intermediate Dimension: 512
- Context Length: 262,144 tokens
- Disclaimer: No outputs from external API services are included in the training pipeline.
Performance
AgentWorldBench (Open-Ended Evaluation)
Five-dimensional rubric mean per domain, normalized to 0-100 scale.
Quickstart
Deployment
Qwen-AgentWorld-35B-A3B can be served via APIs with popular inference frameworks. In the following, we show example commands to launch OpenAI-compatible API servers.
[!Important]
The model has a default context length of 262,144 tokens.
If you encounter out-of-memory (OOM) errors, consider reducing the context window.
However, because Qwen-AgentWorld leverages extended context for multi-turn environment simulation, we advise maintaining a context length of at least 128K tokens.
SGLang
SGLang is a fast serving framework for large language models.
An OpenAI-compatible API will be available at http://localhost:8000/v1.
vLLM
vLLM is a high-throughput and memory-efficient inference engine for LLMs.
An OpenAI-compatible API will be available at http://localhost:8000/v1.
Inference with Transformers
Using via the Chat Completions API
[!Note]
We provide domain-specific world model system prompt templates in prompts/ of the GitHub repository for all 7 domains. These serve as general-purpose system prompts when using Qwen-AgentWorld as an environment simulator. Each domain folder contains a system_prompt.txt (world model system prompt) and a judge_system_prompt.txt (evaluation prompt).
Evaluate on AgentWorldBench
AgentWorldBench evaluates language world models by scoring each predicted environment observation on 5 dimensions: Format, Factuality, Consistency, Realism, and Quality.
Setup
Run Evaluation
The evaluation follows a three-step pipeline:
Best Practices
-
Sampling Parameters: We recommend temperature=0.6, top_p=0.95, top_k=20 for world model inference. The model uses thinking mode by default (<think>...</think>) to reason about environment state transitions before producing the predicted observation.
-
Adequate Output Length: We recommend an output length of 32,768 tokens for most queries. For long, multi-step trajectories, you may increase the max output length to accommodate detailed environment observations.
-
Domain-Specific System Prompts: For optimal simulation fidelity, use the domain-specific system prompts provided in the prompts/ directory of the GitHub repository.
Citation
If you find our work helpful, feel free to give us a cite.