No official torrent yet β requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from SupraLabs/Supra-Mini-v6-1M
All models by SupraLabs βSupra Mini v6 1M is a very small model tand it's the sixth version of our Supra Mini series.
This model reached a final CrossEntropy loss (on the train set) of 3.79.
All benchmarks were executed using lm_eval.
For further benchmarks, see benchmarks.md in this repo's files list.
To use our model, just run this code:
We trained Supra Mini v6 1M on a single NVIDIA RTX 5060 Ti 16GB in ~3 hours for 1 epoch.
The full training code can be found in this repo as train_tokenizer.py (train costum BPE tokenizer with vocab size of 16384) and train_model.py (train the model).
The model was trained on the first 5 billion tokens of 70% Sample-10BT from Fineweb-Edu and 30% Cosmopedia-v2.
from transformers import pipelineimport torch
print("Loading Supra Mini v6 1M model from Hugging Face...")pipe = pipeline( "text-generation", model="SupraLabs/Supra-Mini-v6-1M", device_map="auto", torch_dtype=torch.float16 if torch.cuda.is_available() else torch.float32)
def generate_text(prompt, max_length=150): result = pipe( prompt, max_new_tokens=max_length, do_sample=True, temperature=0.5, top_k=25, top_p=0.9, repetition_penalty=1.2, pad_token_id=pipe.tokenizer.pad_token_id, eos_token_id=pipe.tokenizer.eos_token_id ) return result[0]['generated_text']
test_prompt = "The importance of education is"print(f"\nPrompt: {test_prompt}")print("-" * 30)print("\nOutput:\n" + generate_text(test_prompt))