Kumru-2B-Ceza-Lora

Kumru-2B-Ceza-LoRA

Model Açıklaması

Kumru-2B-Ceza-LoRA is a Turkish language chatbot fine-tuned using LoRA (Low-Rank Adaptation) on the Kumru-2B base model. It mimics the speech style, vocabulary, and energetic perspective of Turkish rap artist Ceza.

Key Specifications:

• Base Model: vngrs-ai/Kumru-2B
• Method: LoRA (Low-Rank Adaptation)
• Language: Turkish
• License: MIT

---

Eğitim Detayları

Dataset

• Source: [https://www.kaggle.com/datasets/salihdede/ceza-qa-dataset-to-applying-lora-fine-tuning/data/data/data]
• Total Examples: 2,678 Q&A pairs
• Split: 90% training / 10% validation (2,410 / 268)
• Format: ChatML
• Generated using LLM with vocabulary extracted from Ceza's lyrics via N-gram analysis

LoRA Konfigürasyonu

Eğitim Parametreleri

Eğitim Sonuçları

En İyi Model: Step 300 (eval_loss: 0.3492)

---

Kurulum

pip install transformers peft torch accelerate

---

Kullanım

Inference Kodu

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

# Base model ve tokenizer
base_model_name = "vngrs-ai/Kumru-2B"
lora_adapter = "SalihHub/Kumru-2B-Ceza-LoRA"

# Tokenizer (base model'den)
tokenizer = AutoTokenizer.from_pretrained(base_model_name)

# Base model
model = AutoModelForCausalLM.from_pretrained(
    base_model_name,
    torch_dtype=torch.float16,
    device_map="auto"
)

# LoRA adapter'ı yükle
model = PeftModel.from_pretrained(model, lora_adapter)
model.eval()

# Chat fonksiyonu
def chat_with_ceza(question, max_new_tokens=200):
    system_prompt = """Sen Ceza'sin. Turkiye'nin en etkili ve teknik rap sanatcilarindan birisin.
Hizli flow'un, sosyal elestirili sozlerin ve guclu sahne performansinla taninan bir MC'sin.
Politik ve toplumsal konularda fikirlerini acikca soyleyen, enerjik ve korkusuz bir karakterin var."""

    prompt = f"""<|im_start|>system
{system_prompt}
<|im_end|>
<|im_start|>user
{question}
<|im_end|>
<|im_start|>assistant
"""

    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

    if "token_type_ids" in inputs:
        del inputs["token_type_ids"]

    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=max_new_tokens,
            temperature=0.7,
            top_p=0.9,
            do_sample=True,
            repetition_penalty=1.2,
            pad_token_id=tokenizer.eos_token_id
        )

    response = tokenizer.decode(outputs[0], skip_special_tokens=False)
    answer = response.split("<|im_start|>assistant\n")[-1].split("<|im_end|>")[0].strip()
    return answer

# Örnek kullanım
print(chat_with_ceza("Bugün nasılsın?"))

---

Örnek Çıktılar

---

Atıf

@misc{kumru-ceza-lora,
  author = {Salih Dede},
  title = {Kumru-2B-Ceza-LoRA: Ceza Style Turkish Chatbot},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/SalihHub/Kumru-2B-Ceza-LoRA}}
}

---

Acknowledgments

• VNGRS AI - Kumru 2B base model için
• Ceza - İlham kaynağı için