No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from RedHatAI/GLM-5.2-speculator.dspark-preview
All models by RedHatAI →A DSpark speculator model for the zai-org/GLM-5.2-FP8 base model, enabling faster
inference through speculative decoding. DSpark extends the DFlash parallel draft
backbone with two lightweight heads: a Markov logit-bias head (low-rank
intra-block token dependency) and a per-position confidence head (accept-rate
prediction). Trained with the speculators
library.
main is the final epoch-3 checkpoint (best validation).
/chat/completions)block_size=8, full vocabulary (154,880), aux layers [8, 23, 39, 55, 70]Per-epoch checkpoints of a single 3-epoch run. main = the final (epoch-3) checkpoint;
each epoch is also a permanent revision.
Validation metrics after epoch 3 (held-out split):
Per-position acceptance (positions 1-7):
0.829 / 0.723 / 0.646 / 0.587 / 0.539 / 0.500 / 0.464
Epoch-over-epoch mean accepted length (train-set proxy for epochs 1-2, val for epoch 3): 3.376 → 3.819 → 3.967 (val).
The model was trained using the Speculators library on prompts from
Magpie-Align/Magpie-Llama-3.1-Pro-300K-Filtered and HuggingFaceH4/ultrachat_200k,
with responses regenerated by GLM-5.2-FP8 itself (published as
mgoin/GLM-5.2-FP8-magpie-ultrachat).
Training is online: the draft consumes hidden states streamed on-the-fly from a live GLM-5.2-FP8 vLLM server, with the trainer running FSDP data-parallel on separate GPUs. The three commands below (data prep → server → trainer) reproduce the run. Install speculators and vLLM from main. GPU indices/parallelism are examples — adjust to your hardware.
--assistant-patternis currently needed for GLM-5.2's inline-reasoning chat format (the<think>...</think>trace is kept inside the assistant turn); it may be auto-detected by future speculators versions.
Notes:
--draft-vocab-size trains on the full vocabulary; pass
--draft-vocab-size 32000 for a reduced draft vocab.--markov-rank, --enable-confidence-head,
--confidence-head-with-markov, --confidence-head-alpha. Dropping them (and
using --speculator-type dflash) recovers a plain DFlash draft.DSpark inference support in vLLM is landing; once available, deploy with speculative decoding:
AI assistance was used to build the training pipeline and run these experiments.
from transformers import AutoModelmodel = AutoModel.from_pretrained( "RedHatAI/GLM-5.2-speculator.dspark", trust_remote_code=True # or revision="epoch-3")python scripts/prepare_data.py \ --model zai-org/GLM-5.2-FP8 \ --trust-remote-code \ --data ./regenerated_data.jsonl \ --output ./output \ --seq-length 8192 \ --assistant-pattern '<\|assistant\|>((?:(?!<\|user\|>|<\|assistant\|>).)*)'CUDA_VISIBLE_DEVICES=0,1,2,3 python scripts/launch_vllm.py \ zai-org/GLM-5.2-FP8 \ --target-layer-ids 8 23 39 55 70 \ -- --port 8000 \ --tensor-parallel-size 4 \ --gpu-memory-utilization 0.9 \ --max-model-len 8192 \ --trust-remote-codeCUDA_VISIBLE_DEVICES=4,5,6,7 torchrun \ --standalone \ --nproc_per_node 4 \ scripts/train.py \ --verifier-name-or-path zai-org/GLM-5.2-FP8 \ --speculator-type dspark \ --num-layers 5 \ --block-size 8 \ --data-path ./output \ --vllm-endpoint http://localhost:8000/v1 \ --save-path ./output/checkpoints \ --epochs 3 \ --lr 0.0006 \ --scheduler-type cosine \ --total-seq-len 4096 \ --draft-arch qwen3 \ --draft-hidden-act silu \ --target-layer-ids 8 23 39 55 70 \ --max-anchors 1024 \ --markov-rank 256 \ --enable-confidence-head \ --confidence-head-with-markov \ --loss-fn '{"ce": 0.1, "tv": 0.9}' \ --confidence-head-alpha 1.0 \ --checkpoint-freq 0.2 \ --on-missing generate \ --on-generate delete \ --seed 42 \ --log-freq 100 \ --prefetch-factor 2 \ --num-workers 8 \ --trust-remote-codevllm serve zai-org/GLM-5.2-FP8 \ --tensor-parallel-size 4 \ --max-model-len 16384 \ --trust-remote-code \ --speculative-config '{ "model": "RedHatAI/GLM-5.2-speculator.dspark", "num_speculative_tokens": 7, "method": "dspark" }'