dell-research-harvard/AmericanStories
Updated • 12.5k • 167
How to use RepresentLM/RepresentLM-v1 with sentence-transformers:
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("RepresentLM/RepresentLM-v1")
sentences = [
"That is a happy person",
"That is a happy dog",
"That is a very happy person",
"Today is a sunny day"
]
embeddings = model.encode(sentences)
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [4, 4]How to use RepresentLM/RepresentLM-v1 with Transformers:
# Load model directly
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("RepresentLM/RepresentLM-v1")
model = AutoModel.from_pretrained("RepresentLM/RepresentLM-v1")This is a sentence-transformers model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.
The model is trained on the HEADLINES semantic similarity dataset, using the StoriesLM-v1-1963 model as a base.
First install the sentence-transformers package:
pip install -U sentence-transformers
The model can then be used to encode language sequences:
from sentence_transformers import SentenceTransformer
sequences = ["This is an example sequence", "Each sequence is embedded"]
model = SentenceTransformer('RepresentLM/RepresentLM-v1')
embeddings = model.encode(sequences)
print(embeddings)
Base model
StoriesLM/StoriesLM-v1-1963