Downloads · 30 days
0
0% of all-time downloads
gorani-lora-3b
aripos1/gorani-lora-3b
gorani-lora-3b is a text generation model from aripos1. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
- Base Model: unsloth/Llama-3.2-3B-Instruct-bnb-4bit - LoRA Adapter: QLoRA 적용 (bnb-4bit) - Fine-tuned on: Custom parallel corpus (Korean-English) - Quantization: 4-bit (bnb-4bit) - Language: Korean & English - Trainin…
All-time downloads
3
Public
Repo size
212 MB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors195 MB · 92%
At a glance
- Task
- Text Generation
- License
- apache-2.0
- Access
- Public
- Created
- Feb 13, 2025
- Updated
- Feb 14, 2025
- SHA
- 695b065e
Base models
Datasets
Model card
From the Hugging Face model README
🦙 Gorani LoRA 3B (Llama 3.2-3B 기반)
🔹 Model Details
- Base Model:
unsloth/Llama-3.2-3B-Instruct-bnb-4bit - LoRA Adapter: QLoRA 적용 (bnb-4bit)
- Fine-tuned on: Custom parallel corpus (Korean-English)
- Quantization: 4-bit (bnb-4bit)
- Language: Korean & English
- Training Method: Supervised Fine-tuning (SFT) + LoRA
- Optimizer: AdamW (betas=(0.9, 0.95), weight_decay=0.01)
🔹 Hyperparameters
| Hyperparameter | Value |
|---|---|
| Learning Rate | 2e-4 |
| Batch Size | 16 |
| Epochs | 3 |
| Warmup Steps | 500 |
| Gradient Accumulation | 4 |
🔹 Evaluation Results
모델 평가를 위해 Comet Score 및 BERT Score를 사용했음.
| Model Version | Comet Score ↑ | BERT Score ↑ |
|---|---|---|
gorani-lora-v1 | 0.78 | 0.85 |
gorani-lora-v2 | 0.82 | 0.88 |
gorani-lora-v3 | 0.85 | 0.90 |
🔹 How to Use
from transformers import AutoModel, AutoTokenizer
from peft import PeftModel
base_model = AutoModel.from_pretrained("unsloth/Llama-3.2-3B-Instruct-bnb-4bit")
adapter_model = PeftModel.from_pretrained(base_model, "aripos1/gorani-lora-3b")
tokenizer = AutoTokenizer.from_pretrained("unsloth/Llama-3.2-3B-Instruct-bnb-4bit")
text = "안녕하세요, 오늘의 날씨는?"
inputs = tokenizer(text, return_tensors="pt")
outputs = adapter_model.generate(**inputs)
print(tokenizer.decode(outputs[0]))
- Task
- Text Generation
- License
- apache-2.0
- Languages
- ko, en
- Created
- Feb 13, 2025
- Updated
- Feb 14, 2025