No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from sentence-transformers/msmarco-distilbert-base-tas-b
All models by sentence-transformers →This is a port of the DistilBert TAS-B Model to sentence-transformers model: It maps sentences & paragraphs to a 768 dimensional dense vector space and is optimized for the task of semantic search.
Using this model becomes easy when you have sentence-transformers installed:
Then you can use the model like this:
Without sentence-transformers, you can use the model like this: First, you pass your input through the transformer model, then you have to apply the right pooling-operation on-top of the contextualized word embeddings.
Have a look at: DistilBert TAS-B Model
pip install -U sentence-transformersfrom sentence_transformers import SentenceTransformer, util
query = "How many people live in London?"docs = ["Around 9 Million people live in London", "London is known for its financial district"]
#Load the modelmodel = SentenceTransformer('sentence-transformers/msmarco-distilbert-base-tas-b')
#Encode query and documentsquery_emb = model.encode(query)doc_emb = model.encode(docs)
#Compute dot score between query and all document embeddingsscores = util.dot_score(query_emb, doc_emb)[0].cpu().tolist()
#Combine docs & scoresdoc_score_pairs = list(zip(docs, scores))
#Sort by decreasing scoredoc_score_pairs = sorted(doc_score_pairs, key=lambda x: x[1], reverse=True)
#Output passages & scoresfor doc, score in doc_score_pairs: print(score, doc)from transformers import AutoTokenizer, AutoModelimport torch
#CLS Pooling - Take output from first tokendef cls_pooling(model_output): return model_output.last_hidden_state[:,0]
#Encode textdef encode(texts): # Tokenize sentences encoded_input = tokenizer(texts, padding=True, truncation=True, return_tensors='pt')
# Compute token embeddings with torch.no_grad(): model_output = model(**encoded_input, return_dict=True)
# Perform pooling embeddings = cls_pooling(model_output)
return embeddings
# Sentences we want sentence embeddings forquery = "How many people live in London?"docs = ["Around 9 Million people live in London", "London is known for its financial district"]
# Load model from HuggingFace Hubtokenizer = AutoTokenizer.from_pretrained("sentence-transformers/msmarco-distilbert-base-tas-b")model = AutoModel.from_pretrained("sentence-transformers/msmarco-distilbert-base-tas-b")
#Encode query and docsquery_emb = encode(query)doc_emb = encode(docs)
#Compute dot score between query and all document embeddingsscores = torch.mm(query_emb, doc_emb.transpose(0, 1))[0].cpu().tolist()
#Combine docs & scoresdoc_score_pairs = list(zip(docs, scores))
#Sort by decreasing scoredoc_score_pairs = sorted(doc_score_pairs, key=lambda x: x[1], reverse=True)
#Output passages & scoresfor doc, score in doc_score_pairs: print(score, doc)SentenceTransformer( (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: DistilBertModel (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False}))