Downloads · 30 days
13
1% of all-time downloads
KoLLaVA-LLaMA-v2-7b-qlora
tabtoyou/KoLLaVA-LLaMA-v2-7b-qlora
KoLLaVA-LLaMA-v2-7b-qlora is a text generation model from tabtoyou. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as cc-by-nc-4.0.
This model is a large multimodal model (LMM) that combines the LLM(LLaMA-2-7b-ko) with visual encoder of CLIP(ViT-14), trained on Korean visual-instruction dataset using QLoRA.
All-time downloads
944
Public
Repo size
28.7 GB
Likes
2
Public
Repo makeup
Click a slice to open those files.
.bin14.3 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- cc-by-nc-4.0
- Model type
- llava
- Access
- Public
- Created
- Aug 4, 2023
- Updated
- Aug 6, 2023
- SHA
- e709fcf0
Try a prompt
Model card
From the Hugging Face model README
KoLLaVA : Korean Large Language and Vision Assistant (feat. LLaVA)
This model is a large multimodal model (LMM) that combines the LLM(LLaMA-2-7b-ko) with visual encoder of CLIP(ViT-14), trained on Korean visual-instruction dataset using QLoRA.
Detail codes are available at KoLLaVA github repository
- Training hyperparameters
- learning rate : 2e-4
- train_batch_size: 16
- distributed_type: multi-GPU (RTX3090 24G)
- num_devices: 4
- gradient_accumulation_steps: 2
- total_train_batch_size: 128
- total_eval_batch_size: 4
- lr_scheduler_type: cosine
- num_epochs: 1
- lora_enable: True
- bits: 4
Model License: cc-by-nc-4.0
- Task
- Text Generation
- Library
- transformers
- Type
- llava
- License
- cc-by-nc-4.0
- Created
- Aug 4, 2023
- Updated
- Aug 6, 2023