Downloads · 30 days
0
iqbalahdagita/Qwen2.5-7B-Indonesian-LoRA
Qwen2.5-7B-Indonesian-LoRA is a machine learning model from iqbalahdagita. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Fine-tuning Qwen2.5-7B-Instruct with LoRA on Indonesian instruction dataset for better Bahasa Indonesia conversational fluency.
Downloads · 30 days
0
Access
Public
Updated May 9, 2026
Repo size
—
Likes
0
Public
Click a slice to open those files.
.py4.1 KB · 50%
From the Hugging Face model README
Fine-tuning Qwen2.5-7B-Instruct with LoRA on Indonesian instruction dataset for better Bahasa Indonesia conversational fluency.
| Component | Value |
|---|---|
| Base Model | Qwen/Qwen2.5-7B-Instruct (7.6B params) |
| Dataset | cahya/instructions_indonesian (~52K Indonesian instruction-response pairs) |
| Method | LoRA SFT (Supervised Fine-Tuning) |
| LoRA Rank | r=32, alpha=16 |
| Trainable Params | ~0.1% of full model |
| Learning Rate | 2.0e-4 (10x base for LoRA) |
| Epochs | 2 |
| Max Seq Length | 2048 |
| Effective Batch Size | 16 (batch=2 × grad_accum=8) |
The training follows the approach from SeaLLMs-v3 (arXiv:2407.19672) which demonstrated that Qwen2.5 tokenizer has superior Indonesian/SEA language coverage versus Llama/Mistral. Key hyperparameters from published results:
The dataset is preprocessed from the original text/label columns into conversational messages format:
{
"messages": [
{"role": "user", "content": "Bagaimana cara meningkatkan koneksi pikiran-tubuh saya?"},
{"role": "assistant", "content": "1. Mulailah dengan mengembangkan latihan kesadaran yang teratur..."}
]
}
hf_jobs or trl CLI:python -m hf_jobs.run \
--script train.py \
--hardware a10g-large \
--timeout 7h \
--dependencies transformers,trl,torch,datasets,peft,trackio,accelerate
Requires GPU with ≥24GB VRAM (e.g., A10G, A100, RTX 3090/4090):
pip install transformers trl torch datasets peft trackio accelerate
python train.py
Training metrics are logged to Trackio dashboard. Set TRACKIO_SPACE_ID environment variable to enable live tracking.