darmm-chat-kazakh-8b-lora

darmm-chat-kazakh-8b-lora

LoRA adapter only (~170MB) for Darmm/darmm-chat-kazakh-8b — the Kazakh conversational fine-tune of Qwen3-8B.

Use this if you already have the base model or want to apply the tune to your own (possibly quantized) copy of Qwen3-8B. If you just want the ready-to-use model, download the merged repo instead: Darmm/darmm-chat-kazakh-8b — same weights, no assembly.

Benchmarks (identical to the merged model)

Usage

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B", dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(base, "Darmm/darmm-chat-kazakh-8b-lora")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")

messages = [{"role": "user", "content": "Наурыз мейрамы туралы айтып бер."}]
text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)
inputs = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512,
                     do_sample=True, temperature=0.7, top_p=0.8, top_k=20, repetition_penalty=1.1)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Always use the sampling settings above (enable_thinking=False, temperature 0.7, top_p 0.8, repetition_penalty 1.1) — greedy decoding degrades long-form Kazakh. Also compatible with vLLM multi-LoRA serving (--enable-lora --lora-modules kazakh=Darmm/darmm-chat-kazakh-8b-lora).

Details

LoRA r=32, α=64, dropout 0.05 on all attention + MLP projections; trained with QLoRA (4-bit NF4 base) for 1 epoch on ~105k instructions (65k Kazakh / 25k Russian / 15k English). Full training details, data list, and limitations: see the merged model card.

Built by Darmm · 2026-09 · Apache 2.0