Contrastive-LM/CLM-v0.1-8B
text-rankingcontrastive-lmVerified checksumscontrastive-learningverifierrerankeragentsclmcontrastive-lm
- License
- apache-2.0
- Size
- 72 MB
- Revision
- main
Mirrored from Contrastive-LM/CLM-v0.1-8B
All models by Contrastive-LM β
CLM-v0.1-8B
| π Blog | π» Code | π£οΈ Discord |
Contrastive Language Model (CLM) is a new class of System One model trained with a contrastive learning objective that connects states and actions. CLM-8B consists of two small projection heads (a state head and an action head) on top of a frozen Qwen3-8B encoder trained with a bidirectional InfoNCE loss.
- Training: pre-trained on ~60M Nemotron Q&A pairs, mid-trained on ~30M synthetic hard negatives, post-trained on ~1M agentic trajectories.
- Zero-shot: on par with Jev on computer-use, gaming and tool-calling tasks, with up to 9Γ lower latency.
- Fine-tuned as a verifier: SOTA on DeepSWE (81.6%) and Terminal-Bench 2.1 (87.6%), 4β6Γ faster than Jev.
- State & Action Caching: states and actions are encoded separately, so action embeddings can be reused. With ~1k candidates, CLM is 13Γ faster than Jev.
Usage
With the contrastive-lm package
Ask typed questions about a state:
Or rank free-form candidates (best-of-N solutions, tool names, next moves):
Fine-tuning
Only the heads are trained, so fine-tuning is cheap. This checkpoint is the starting point for the DeepSWE and Terminal-Bench heads.
See the fine-tuning guide.
Playground
clm-serve also serves a web playground at http://localhost:8700/.
Limitations
- Encoder-locked: the heads require Qwen3-8B last-token-pooled embeddings.
- No generation: CLM only scores the candidates you give it, and its probabilities are relative to that set.
- Verifier results need fine-tuning: the SOTA agentic-benchmark numbers come from fine-tuned heads, not this checkpoint zero-shot.
- Generalization: CLM-8B is one rung of our scaling ladder. A multimodal CLM-35B, trained with more data, compute and parameters for stronger generalization, is coming in early October.
Citation
License
The CLM-8B weights are released under the Apache 2.0 License [blocked]. The base encoder Qwen3-8B is also Apache 2.0.