Downloads · 30 days
21
3% of all-time downloads
llama2-7b-chatvi
infCapital/llama2-7b-chatvi
llama2-7b-chatvi is a text generation model from infCapital. Use it when you need the model to write or continue text. It is set up for transformers.
+ Extend vocab to 44,800 for better Vietnamese understanding + Continual Pre-Train with 2B tokens Vietnamese + Trainning profile: LoRa (rank=32, alpha=128, 16fp), 1 epoch, block size = 512. Takes 300GPU Hours x RXT409…
All-time downloads
708
Public
Repo size
27.4 GB
Likes
2
Public
Repo makeup
Click a slice to open those files.
.bin13.7 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- llama
- Access
- Public
- Created
- Sep 27, 2023
- Updated
- Oct 2, 2023
- SHA
- e83db0e2
Datasets
Model card
From the Hugging Face model README
Base Model: LLaMa2 7B Chat HF
- Extend vocab to 44,800 for better Vietnamese understanding
- Continual Pre-Train with >2B tokens Vietnamese
- Trainning profile: LoRa (rank=32, alpha=128, 16fp), 1 epoch, block size = 512. Takes 300GPU Hours x RXT4090 24GB
Can be better use for
- Futher training / Fine-tuning for Vietnamese tasks
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- Languages
- vi
- Created
- Sep 27, 2023
- Updated
- Oct 2, 2023