Downloads · 30 days
6
27% of all-time downloads
armvectores/wikipedia_arlis_tokens_fasttextskipgram_300_5
wikipedia_arlis_tokens_fasttextskipgram_300_5 is a feature extraction model from armvectores. Use it when you need embeddings to search or compare text. It is set up for fasttext.
414M tokens 1) 73M hy wikipedia 2) 341M arlis database
Downloads · 30 days
6
27% of all-time downloads
All-time downloads
22
Public
Repo size
2.6 GB
Likes
0
Public
Click a slice to open those files.
.bin2.6 GB · 100%
From the Hugging Face model README
414M tokens
74951 unique words
3-5 ngrams
5 window length
300 embedding dim
skipgram
minimum number of words 150
100 epochs, 0.05 start lr
26 hours on 20 xeon gold cores
How to use
pip install fasttext-wheel
import fasttext
from huggingface_hub import hf_hub_download
model_path = hf_hub_download(local_dir=".",
repo_id="armvectores/wikipedia_arlis_tokens_fasttextskipgram_300_5",
filename="model.bin")
model = fasttext.load_model(model_path)
word = 'զենքեր'
print(model.get_nearest_neighbors(word))
print(model.get_sentence_vector(word))