Mirrored from autotrust/JEV-35B-NVFP4
All models by autotrust →
[!IMPORTANT] Try AutoTrust/GuruSearch live (new, 11 October 2026). A recommendation and search demo in which every ranking is a calibrated System 1 decision, made in about half a second:
- Recommendation: the latest headlines from 6 news feeds, ranked by importance.
- Search: results from Google News, Bing News and Yahoo News re-ranked by relevance, side by side with the search engines' own order, plus a short answer with citations.
This repository is autotrust/JEV-35B with its 10,240 routed expert MLPs (40 layers × 256 experts) quantized to NVFP4. Everything else is the bf16 model unchanged: attention, linear attention, the shared expert, the routers, the vision tower, the MTP layer, the lm_head, the System 1 adapter, the decision head and the temperatures. Below the NVFP4 comparison, the rest of this card is the JEV-35B card; its tables are the bf16 results unless marked otherwise.
The two videos below were recorded with this NVFP4 checkpoint; per-episode NVFP4 results are in
reports/demos-nvfp4/ [blocked], the bf16 ones in reports/demos/ [blocked].
What is quantized and how. NVFP4 is NVIDIA's 4-bit floating-point format: FP4 E2M1 values in groups of 16 with an
FP8 E4M3 scale per group and an FP32 scale per tensor. Per expert, the gate, up and down projections are quantized (gate
and up share one tensor scale, as vLLM fuses them); the expert activations use static scales calibrated on 3,072 System 1
prompts. The checkpoint uses the ModelOpt NVFP4 layout (quant_method: modelopt), so vLLM loads it without any flag.
The System 1 adapter has no LoRA on the routed experts, so adapter_vllm/ is the same as for the bf16 model.
serve.sh and the API are exactly those of JEV-35B (see Quick start); vLLM reads the
quantization from config.json / hf_quant_config.json.
Decision Index 0.3, public suite: 59.75 (our scoring with the kit), against 53.64 for autotrust/JEV-27B-VL on the board. Approximate public vision score 72.08 (our rebuild of the vision benchmarks; JEV-27B-VL 71.67 on the same rebuild, a tie within noise). Median single-request latency 241 ms on one B200 (JEV-27B-VL 271 ms).
Two models, two organisations. TypeSafe Jev 1.13 is the hosted, closed model made by TypeSafe AI. autotrust/JEV-35B-NVFP4 is an independent open-weights model built by AutoTrust AI; it is not affiliated with, endorsed by, or a product of TypeSafe AI.
JEV-35B: all 140,178 scoreable requests of the 0.3 public suite answered (0 errors), System 1 only (thinking off),
every choice read in one pass (up to 256 options), scored with the kit's score --edition 0.3. Our scoring, not a board
entry: the board's Full score also counts private tests (80 %), which only its maintainers run. JEV-27B-VL: the board's
public numbers for JEV-27B, whose text decisions JEV-27B-VL reproduces (see the JEV-27B-VL card).
JEV-35B is ahead in all five areas and on 24 of 37 benchmarks. Largest gains (skill points): PhishNChips +42.3, HoVer +28.3, Habermas Machine +28.1, VAST +22.8, WinoGrande +18.9, BANKING77 +18.3, iSarcasmEval +17.2, When2Call +14.9, GPQA +9.5. JEV-27B-VL is ahead on POP909-CL (+25.7), GSM8K (+18.3), ANLI (+11.4), BPoMP (+10.8), NLI4CT (+6.7) and BBH (+4.8).
Skill rescales the benchmark's own metric so that chance is 0 (below chance counts as 0); ★ = gold benchmark (weight 1.2); the higher score is in bold.
Knowledge & Reasoning (area skill 0.426; JEV-27B-VL 0.414)
Language Understanding (area skill 0.632; JEV-27B-VL 0.564)
Retrieval & Classification (area skill 0.710; JEV-27B-VL 0.537)
Tools & Automation (area skill 0.765; JEV-27B-VL 0.735)
Arts & Human Taste (area skill 0.422; JEV-27B-VL 0.416)
The vision benchmarks of the Decision Index are not published; this is our rebuild of their public datasets (CV-Bench, BLINK, RealWorldQA, CharXiv, InfographicVQA, Mind2Web, CORD + FUNSD, Hateful Memes, R-Bench-M, MMMU-Pro vision; Winoground not included), scored with the board's chance correction and weights. The same rebuild gives JEV-27B-VL 71.67 against its board score of 71.53 on the same benchmarks.
Overall the two models are level: the difference (+0.4) is inside the noise (paired bootstrap 95% interval −0.7 to +1.6). Only two per-benchmark differences are statistically significant (paired McNemar test, p < 0.01), both in favour of JEV-35B: moderation (+9.6) and MMMU-Pro (+4.3). The small deficits on RealWorldQA, CV-Bench, BLINK, InfographicVQA and R-Bench-M (−1 to −3) are not significant. The vision tower is Qwen3.5-35B-A3B's, unchanged; image decisions are zero-shot.
One sequential client, the same 587 rows (387 text rows sampled across the Decision Index, 200 image rows), POST
/v1/decide on serve.sh (vLLM, System 1 as a LoRA), one B200:
Same demo code, seeds, scenes and opponents as the JEV-27B-VL card; every
step is one System 1 decision (POST /v1/decide, thinking off), one model on one B200.
Computer use: screenshot → which element to click. A real browser (headless Chromium). Every clickable element gets a numbered box; System 1 picks the next click (or "the task is complete"), the browser clicks it, and the loop repeats.
Robot arm: pick and place from a camera image (MuJoCo). At every step System 1 answers two questions from the top camera: is the target left or right of the gripper, and above or below it? The arm halves its step whenever an answer flips.
Games (seed 0 for both models; board as image + text):
For game play, use JEV-27B-VL. Over five seeds JEV-35B's largest 2048 tile averages 64 (random play: 102), it passes
no Flappy Bird pipe and eats 3–18 pieces of food in Snake (mean 10). Per-episode results: reports/demos/ [blocked].
serve.sh runs serve_decide.py: the standard vLLM OpenAI server (same flags as vllm serve) with a POST /v1/decide
route. Plain requests are System 2; requests for the LoRA module jev-decision (adapter_vllm/: backbone LoRA + the
decision head as an lm_head LoRA) are System 1. Tested with a vLLM development build from September 2026.
The adapter has no LoRA on the routed experts. serve.sh sets JEV_DECIDE_NO_MOE_LORA=1 and --lora-target-modules
so that the routed experts run on vLLM's normal MoE kernels (the decision head needs --max-lora-rank 320; vLLM's
MoE-LoRA kernel supports at most rank 128).
POST /v1/decideThe response has options, probabilities, choice, choice_index and usage. GET /v1/decide/info lists the
defaults.
If you call /v1/completions for System 1 yourself, pass top_k: 0 and top_p: 1.0 so that the returned
probabilities are not truncated.
System 1 reads the hidden state at the last token of a bare-text prompt:
A 264-slot linear head (fp32) gives one logit per slot; the active slots of the question's kind are soft-maxed with a
per-kind temperature (calibration.json). In vLLM the head is expressed as an lm_head LoRA (adapter_vllm/, rank
320) and the head bias is added client-side (adapter_vllm/decision_head.json).
Apache-2.0. This repository contains the weights of Qwen/Qwen3.5-35B-A3B (Apache-2.0), with the routed experts quantized to NVFP4, plus the System 1 adapter, decision head and calibration of autotrust/JEV-35B.
hf download autotrust/JEV-35B-NVFP4 --local-dir JEV-35B-NVFP4bash JEV-35B-NVFP4/serve.sh # vLLM on :8000hf download autotrust/JEV-35B-NVFP4 --local-dir JEV-35B-NVFP4bash JEV-35B-NVFP4/serve.sh # vLLM on :8000; one GPU with 32 GB or morecurl localhost:8000/v1/decide -H 'Content-Type: application/json' -d '{ "kind": "choice", "state": "Customer message: my card was charged twice for the same order.", "question": "Which team should handle this ticket?", "options": ["billing", "shipping", "technical support", "account security"]}'import requestsrequests.post("http://localhost:8000/v1/chat/completions", json={ "model": "autotrust/JEV-35B-NVFP4", "messages": [{"role": "user", "content": "In one sentence, what is safety stock?"}], "max_tokens": 200})[kind] choice[state] ...[question] ...[options]A) ...B) ...[decision]:model-*.safetensors Qwen/Qwen3.5-35B-A3B with NVFP4 routed experts (ModelOpt layout), everything else bf16config.json · hf_quant_config.json · nvfp4_activation_amax.json quantization config and calibrated expert activation rangestokenizer* · chat_template.jinja · generation_config.json · *_config.jsonadapter/ System 1 LoRA (peft), rank 32head.safetensors 264-slot decision head (fp32)judge_config.json slot layout, verbalizer ids, read-outcalibration.json per-kind temperaturesadapter_vllm/ System 1 for vLLM: backbone LoRA + the head as an lm_head LoRA (rank 320), plus decision_head.jsonserve_decide.py · serve.sh vLLM server with POST /v1/decide next to the OpenAI endpointsvideos/ computer-use and robot-arm episodesreports/demos/ per-episode computer-use, robot-arm and game results (bf16)reports/demos-nvfp4/ the same for this NVFP4 checkpoint