Downloads · 30 days
19
3% of all-time downloads
koen-t5-small-v0
nlp-with-deeplearning/koen-t5-small-v0
koen-t5-small-v0 is a text generation model from nlp-with-deeplearning. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as cc-by-nc-sa-4.0.
This model is for Korean to English Machine Translator, which is based on T5-small architecture, but trained from scratch.
All-time downloads
695
Public
Parameters
74.8M
299 MB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors299 MB · 99%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- cc-by-nc-sa-4.0
- Model type
- t5
- Access
- Public
- Created
- Dec 31, 2023
- Updated
- Dec 31, 2023
- SHA
- 525940c7
Model card
From the Hugging Face model README
KOEN-T5-SMALL-V0
This model is for Korean to English Machine Translator, which is based on T5-small architecture, but trained from scratch.
Code
The training code is from my lecture(LLM을 위한 김기현의 NLP EXPRESS), which is published on FastCampus. You can check the training code in this github repo.
Dataset
The training dataset for this model is mainly from AI-Hub. The dataset consists of 11M parallel samples.
Tokenizer
I use Byte-level BPE tokenizer for both source and target language. Since it covers both languages, tokenizer vocab size is 60k.
Architecture
The model architecture is based on T5-small, which is popular encoder-decoder model architecture. Please, note that this model is trained from-scratch, not fine-tuned.
Evaluation
I conducted the evaluation with 5 different test sets. Following figure shows BLEU scores on each test set.
DEEPCL model is private version of this model, which is trained on much more data.
Contact
Kim Ki Hyun ([email protected])
- Task
- Text Generation
- Library
- transformers
- Type
- t5
- License
- cc-by-nc-sa-4.0
- Languages
- ko, en
- Created
- Dec 31, 2023
- Updated
- Dec 31, 2023