fine-tuned-model-indonesian-GRPO

fine-tuned-model-indonesian-GRPO

fine-tuned-model-indonesian-GRPO is an Indonesian-focused Language Model based on Llama-3.1-8B, fine-tuned using Group Relative Policy Optimization (GRPO) via Unsloth and Hugging Face TRL. The model is specially tailored for Legal AI RAG (Retrieval-Augmented Generation) workflows, producing structured reasoning (Chain-of-Thought) enclosed in <think>...</think> and <answer>...</answer> tags in Indonesian.

---

📌 Model Details

---

💡 Key Features

• Structured Reasoning (CoT): Generates logical step-by-step reasoning within <think> tags before delivering the final answer inside <answer> tags.
• Indonesian Legal Optimization: Fine-tuned to analyze Indonesian statutory regulations, government regulations (PP), laws (UU), and legal clauses without suffering from mode collapse.
• VRAM & Memory Efficient: Employs 4-bit quantization (NF4) for fast inference and low resource consumption during execution or continued training.

---

⚙️ Recommended Inference Parameters

To prevent language degradation, typos, or infinite sentence looping, use the following generation parameters during inference:

⚠️ Important Note: Avoid using small values for no_repeat_ngram_size (e.g., 4), as it can trigger token corruption (garbled text or typos) when the model needs to repeat legal terms or clause names.

---

🚀 How to Use

1. Using Hugging Face transformers

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "hendrimardani/fine-tuned-model-indonesian-GRPO"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype=torch.bfloat16,
    load_in_4bit=True
)

system_prompt = """Anda adalah asisten hukum AI yang analitis dan presisi.

ATURAN WAJIB GENERASI:
1. Di dalam tag <think>...</think>: Tuliskan HANYA 2-3 kalimat penalaran analitis ringkas. Hubungkan profil/pertanyaan user dengan pasal spesifik.
2. Di dalam tag <answer>...</answer>: Berikan jawaban langsung, solutif, dan jelas kepada user berbasis konteks."""

messages = [
    {"role": "system", "content": system_prompt},
    {
        "role": "user",
        "content": (
            "PERTANYAAN USER:\nSaya staf admin kerja lembur 3 jam, apakah berhak dapat uang lembur?\n\n"
            "DOKUMEN KONTEKS:\nPP No. 35 Tahun 2021 Pasal 27: Pengusaha wajib membayar upah lembur kepada pekerja/buruh yang dipekerjakan melebihi waktu kerja normal."
        )
    }
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
prompt += "<think>\n"  # Prefill reasoning tag

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=1024,
    do_sample=True,
    temperature=0.3,
    repetition_penalty=1.18,
    eos_token_id=tokenizer.eos_token_id,
    pad_token_id=tokenizer.pad_token_id if tokenizer.pad_token_id is not None else tokenizer.eos_token_id
)

response = tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True)
print("<think>\n" + response)

---

2. Integration with LangChain RAG Pipeline

from transformers import pipeline
from langchain_community.llms import HuggingFacePipeline

text_generation_pipeline = pipeline(
    task="text-generation",
    model=model,
    tokenizer=tokenizer,
    max_new_tokens=1024,
    do_sample=True,
    temperature=0.3,
    repetition_penalty=1.18,
    return_full_text=False,
    eos_token_id=tokenizer.eos_token_id,
    pad_token_id=tokenizer.pad_token_id if tokenizer.pad_token_id is not None else tokenizer.eos_token_id
)

llm = HuggingFacePipeline(pipeline=text_generation_pipeline)

---

📄 Model Output Format

The model is designed to consistently output structured responses in Indonesian:

<think>
User menanyakan hak upah lembur untuk posisi staf admin (3 jam lembur). Berdasarkan PP No. 35 Tahun 2021 Pasal 27, kerja melebihi waktu normal wajib diberi upah lembur. Maka staf admin berhak atas upah 3 jam tersebut.
</think>
<answer>
Ya, Anda berhak mendapatkan upah lembur. Berdasarkan PP No. 35 Tahun 2021 Pasal 27, pengusaha wajib membayar upah lembur untuk pekerja yang bekerja melebihi waktu kerja normal (8 jam sehari).
</answer>

---

⚠️ Disclaimer

This model is built to assist with legal research and document retrieval (RAG). Output generated by this model does not constitute legal advice. Users are advised to verify all generated information against official Indonesian statutory sources.