Downloads · 30 days
0
bangla-fasttext
sagorsarker/bangla-fasttext
bangla-fasttext is a machine learning model from sagorsarker. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
This is a FastText pre-trained model for the Bengali language.
Access
Public
Updated Nov 12, 2022
Repo size
1.6 GB
Likes
2
Public
Repo makeup
Click a slice to open those files.
.zip1.6 GB · 100%
At a glance
- License
- mit
- Access
- Public
- Created
- Jun 11, 2022
- Updated
- Nov 12, 2022
- SHA
- 8fb1c05b
Model card
From the Hugging Face model README
Bangla FastText Model
This is a FastText pre-trained model for the Bengali language.
This model is build for bnlp package.
Datasets
Training Details
- Fasttext trained with total words = 20M, vocab size = 1171011, epoch=50, embedding dimension = 300
Evaluation Details
- training loss = 0.318668
Usage
pip install -U bnlp_toolkitpip install fasttext==0.9.2- Generate Vector Using Pretrained Model
from bnlp.embedding.fasttext import BengaliFasttext
bft = BengaliFasttext()
word = "গ্রাম"
model_path = "bengali_fasttext_wiki.bin"
word_vector = bft.generate_word_vector(model_path, word)
print(word_vector.shape)
print(word_vector)
- Train Bengali FastText Model
from bnlp.embedding.fasttext import BengaliFasttext
bft = BengaliFasttext()
data = "raw_text.txt"
model_name = "saved_model.bin"
epoch = 50
bft.train(data, model_name, epoch)
- Generate Vector File from Fasttext Binary Model
from bnlp.embedding.fasttext import BengaliFasttext
bft = BengaliFasttext()
model_path = "mymodel.bin"
out_vector_name = "myvector.txt"
bft.bin2vec(model_path, out_vector_name)
- License
- mit
- Created
- Jun 11, 2022
- Updated
- Nov 12, 2022