No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from ibm-granite/granite-embedding-278m-multilingual
All models by ibm-granite →Model Summary: Granite-Embedding-278M-Multilingual is a 278M parameter model from the Granite Embeddings suite that can be used to generate high quality text embeddings. This model produces embedding vectors of size 768 and is trained using a combination of open source relevance-pair datasets with permissive, enterprise-friendly license, and IBM collected and generated datasets. This model is developed using contrastive finetuning, knowledge distillation and model merging for improved performance.
Supported Languages: English, German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, and Chinese. Users may finetune Granite-Embedding-278M-Multilingual for languages beyond these 12 languages.
Intended use: The model is designed to produce fixed length vector representations for a given text, which can be used for text similarity, retrieval, and search applications.
Usage with Sentence Transformers: The model is compatible with SentenceTransformer library and is very easy to use:
First, install the sentence transformers library
The model can then be used to encode pairs of text and find the similarity between their representations
Usage with Huggingface Transformers: This is a simple example of how to use the Granite-Embedding-278m-Multilingual model with the Transformers library and PyTorch.
First, install the required libraries
The model can then be used to encode pairs of text
Evaluation: The average performance of the Granite-Embedding-278M-Multilingual on Multilingual Miracl (across 18 langauges), Mintaka Retrieval (across 8 languages) and MTEB Retrieval for English (across 15 tasks), German (across 4 tasks), Spanish (across 2 tasks), Frenc (across 5 tasks), Japanese (across 2 tasks), Arabic (1 task), Korean (1 task) and Chinese (across 8 tasks) is reported below.
Model Architecture: Granite-Embedding-278m-Multilingual is based on an encoder-only XLM-RoBERTa like transformer architecture, trained internally at IBM Research.
Training Data: Overall, the training data consists of four key sources: (1) unsupervised title-body paired data scraped from the web, (2) publicly available paired with permissive, enterprise-friendly license, (3) IBM-internal paired data targetting specific technical domains, and (4) IBM-generated synthetic data. The data is listed below:
Notably, we do not use the popular MS-MARCO retrieval dataset in our training corpus due to its non-commercial license, while other open-source models train on this dataset due to its high quality.
Infrastructure: We train Granite Embedding Models using IBM's computing cluster, Cognitive Compute Cluster, which is outfitted with NVIDIA A100 80gb GPUs. This cluster provides a scalable and efficient infrastructure for training our models over multiple GPUs.
Ethical Considerations and Limitations: The data used to train the base language model was filtered to remove text containing hate, abuse, and profanity. Granite-Embedding-278m-Multilingual is finetuned on 12 languages, and has a context length of 512 tokens (longer texts will be truncated to this size).
Resources
pip install sentence_transformersfrom sentence_transformers import SentenceTransformer, util
model_path = "ibm-granite/granite-embedding-278m-multilingual"# Load the Sentence Transformer modelmodel = SentenceTransformer(model_path)
input_queries = [ ' Who made the song My achy breaky heart? ', 'summit define' ]
input_passages = [ "Achy Breaky Heart is a country song written by Don Von Tress. Originally titled Don't Tell My Heart and performed by The Marcy Brothers in 1991. ", "Definition of summit for English Language Learners. : 1 the highest point of a mountain : the top of a mountain. : 2 the highest level. : 3 a meeting or series of meetings between the leaders of two or more governments." ]
# encode queries and passagesquery_embeddings = model.encode(input_queries)passage_embeddings = model.encode(input_passages)
# calculate cosine similarityprint(util.cos_sim(query_embeddings, passage_embeddings))pip install transformers torchimport torchfrom transformers import AutoModel, AutoTokenizer
model_path = "ibm-granite/granite-embedding-278m-multilingual"
# Load the model and tokenizermodel = AutoModel.from_pretrained(model_path)tokenizer = AutoTokenizer.from_pretrained(model_path)model.eval()
input_queries = [ ' Who made the song My achy breaky heart? ', 'summit define' ]
# tokenize inputstokenized_queries = tokenizer(input_queries, padding=True, truncation=True, return_tensors='pt')
# encode querieswith torch.no_grad(): # Queries model_output = model(**tokenized_queries) # Perform pooling. granite-embedding-278m-multilingual uses CLS Pooling query_embeddings = model_output[0][:, 0]
# normalize the embeddingsquery_embeddings = torch.nn.functional.normalize(query_embeddings, dim=1)