No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from vllm-sr/mmbert-embed-32k-2d-matryoshka
All models by vllm-sr →A multilingual embedding model with 32K context window and 2D Matryoshka support for flexible efficiency-quality tradeoffs.
This model supports two dimensions of flexibility:
For latency-critical applications, you can extract embeddings from intermediate layers:
mmBERT-Embed is significantly faster due to:
Key insight: The FA2 advantage grows with sequence length and batch size:
Benchmarked on AMD MI300X, bf16 precision.
Trained on BAAI/bge-m3-data (73GB, 279 JSONL files) with:
Pre-exported ONNX models are available for production deployment with ONNX Runtime. Each layer model enables true early-exit speedup.
Benchmarked on AMD MI300X with ROCm, fp16 precision, batch=1, dynamic sequence length.
Unlike PyTorch where you can use output_hidden_states=True for runtime layer selection, ONNX graphs are static DAGs - all nodes execute regardless of which output you read. Separate model files are required for true early-exit speedup.
Apache 2.0
from sentence_transformers import SentenceTransformer
# Load modelmodel = SentenceTransformer("vllm-sr/mmbert-embed-32k-2d-matryoshka")
# Encode sentencessentences = [ "This is a test sentence.", "这是一个测试句子。", "Dies ist ein Testsatz.",]embeddings = model.encode(sentences)print(embeddings.shape) # (3, 768)import torch.nn.functional as F
# Encode with full dimensionsembeddings = model.encode(sentences, convert_to_tensor=True)
# Truncate to smaller dimension (e.g., 256)embeddings_256d = embeddings[:, :256]embeddings_256d = F.normalize(embeddings_256d, p=2, dim=1)
# Or truncate to 64 dimensions for maximum compressionembeddings_64d = embeddings[:, :64]embeddings_64d = F.normalize(embeddings_64d, p=2, dim=1)# For long documents, set max_seq_lengthmodel.max_seq_length = 8192 # or up to 32768
long_document = "..." * 10000 # Very long textembedding = model.encode(long_document)from transformers import AutoModel, AutoTokenizerimport torchimport torch.nn.functional as F
model = AutoModel.from_pretrained( "vllm-sr/mmbert-embed-32k-2d-matryoshka", trust_remote_code=True, output_hidden_states=True)tokenizer = AutoTokenizer.from_pretrained("vllm-sr/mmbert-embed-32k-2d-matryoshka")
# Encodeinputs = tokenizer(sentences, padding=True, truncation=True, return_tensors="pt")with torch.no_grad(): outputs = model(**inputs) # Use layer 6 for 3.3× speedup (56% quality) hidden = outputs.hidden_states[6] hidden = model.final_norm(hidden) # Mean pooling mask = inputs["attention_mask"].unsqueeze(-1).float() pooled = (hidden * mask).sum(1) / mask.sum(1) embeddings = F.normalize(pooled, p=2, dim=1)@misc{mmbert-embed-2d-matryoshka, title={mmBERT-Embed: Multilingual Embedding Model with 2D Matryoshka Training}, author={vLLM Semantic Router Team}, year={2025}, url={https://huggingface.co/vllm-sr/mmbert-embed-32k-2d-matryoshka}}from huggingface_hub import hf_hub_download
# Download layer-6 for fast inferencemodel_path = hf_hub_download( "vllm-sr/mmbert-embed-32k-2d-matryoshka", "onnx/layer-6/model.onnx")data_path = hf_hub_download( "vllm-sr/mmbert-embed-32k-2d-matryoshka", "onnx/layer-6/model.onnx.data")import onnxruntime as ortfrom transformers import AutoTokenizer
# Load tokenizertokenizer = AutoTokenizer.from_pretrained( "vllm-sr/mmbert-embed-32k-2d-matryoshka")
# Load ONNX model (use ROCMExecutionProvider for AMD GPU)session = ort.InferenceSession( "onnx/layer-6/model.onnx", providers=["ROCMExecutionProvider", "CUDAExecutionProvider", "CPUExecutionProvider"])
# Inferenceinputs = tokenizer("Hello world", return_tensors="np", padding=True)outputs = session.run(None, { "input_ids": inputs["input_ids"], "attention_mask": inputs["attention_mask"],})hidden_state = outputs[0] # Shape: (batch, seq_len, 768)
# Mean poolingimport numpy as npmask = inputs["attention_mask"][..., np.newaxis]embeddings = (hidden_state * mask).sum(1) / mask.sum(1)embeddings = embeddings / np.linalg.norm(embeddings, axis=1, keepdims=True)use ort::{Session, execution_providers::ROCmExecutionProvider};
// Load models at startuplet fast_model = Session::builder()? .with_execution_providers([ROCmExecutionProvider::default().build()])? .commit_from_file("onnx/layer-6/model.onnx")?;
let full_model = Session::builder()? .with_execution_providers([ROCmExecutionProvider::default().build()])? .commit_from_file("onnx/layer-22/model.onnx")?;
// Runtime selection based on latency/quality needslet embedding = if need_fast_response { fast_model.run(inputs)? // ~2.6ms} else { full_model.run(inputs)? // ~11ms};