No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from SupraLabs/SupraBrain-50M
All models by SupraLabs →SupraBrain 50M v0.1 is an experimental 50-million parameter hybrid language model engineered by SupraLabs. It combines Gated DeltaNet linear recurrence with Sliding-Window Attention and Surprise-Gated update mechanisms, optimized using a custom Muon + AdamW hybrid optimizer schedule.
SurpriseBeta) that dynamically scales learning updates based on local sequence surprise/prediction error.individual_digits=True) to dramatically boost arithmetic and numerical reasoning performance in sub-100M parameter models.unembed_rank=32) and low-rank output gating (gdn_gate_rank=32) to conserve parameter count while maintaining model capacity in the core layers.Since SupraBrain uses a custom architecture without standard Hugging Face native integration, you must register the model class locally before loading it with AutoModelForCausalLM.
First, download the modeling script:
Then, load the model:
Dataset: 5 Billion Tokens Total
Stable Phase (3.6B Tokens): FineWeb-Edu (sample-100BT)
Anneal Phase (1.4B Tokens): 65% FineWeb-Edu (Score $\ge 4.2$) + 35% Cosmopedia-v2
Schedule: Warmup-Stable-Decay (WSD) with square-root decay during the annealing phase.
Batch Size: Micro-batch size 16 with Gradient Accumulation 8 ($\approx 262,144$ tokens/step over sequence length 1024).
flash-linear-attention (fla) for Gated DeltaNet kernels and PyTorch flex_attention for masked sliding-window operations.wget https://huggingface.co/SupraLabs/SupraBrain-50M/resolve/main/modeling_suprabrain.pyimport importlibimport torchfrom transformers import AutoConfig, AutoModelForCausalLM, AutoTokenizer
# Import custom model and config classes from the model script (modeling_suprabrain.py)sb_module = importlib.import_module("modeling_suprabrain")SupraBrainConfig = sb_module.SupraBrainConfigSupraBrainForCausalLM = sb_module.SupraBrainForCausalLM
# Register custom architecture with Hugging Face AutoClassesAutoConfig.register("suprabrain", SupraBrainConfig)AutoModelForCausalLM.register(SupraBrainConfig, SupraBrainForCausalLM)
model_id = "SupraLabs/SupraBrain-50M"
print("[*] Loading tokenizer and model...")tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)model = AutoModelForCausalLM.from_pretrained( model_id, trust_remote_code=True, torch_dtype=torch.bfloat16).to("cuda")
# Prompt setupprompt = "The mitochondrion produces"inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
# Generation with repetition penalty controlprint("[*] Generating text...")with torch.no_grad(): outputs = model.generate( **inputs, max_new_tokens=100, temperature=0.7, top_p=0.9, do_sample=True, no_repeat_ngram_size=3, # Prevents 3-gram repetitions pad_token_id=tokenizer.pad_token_id, eos_token_id=tokenizer.eos_token_id )
generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)print("\n--- Output ---")print(generated_text)