kanitakorn-qwen3-8b-final

kanitakorn-qwen3-8b-final

2-way task-arithmetic merge of R8 + R12. Best balanced after R12.

Base model: Qwen/Qwen3-8B Adapter type: LoRA (r=16, α=32, target_modules = q,k,v,o,gate,up,down_proj) Training recipe: 0.7 × R8 + 0.3 × R12

Benchmark scores

Peer Thai LLM benchmarks (their reported numbers)

Numbers in bold are at-or-above the best peer score available.

Comparison vs Typhoon-2-Llama3-8B (their card numbers)

Comparison vs Pathumma-1.0-7B (their card numbers, NLU suite)

Methodology

This adapter was produced via LoRA SFT on Qwen3-8B with a multi-round corpus mix:

• Kanitakorn original Thai academic corpus (~4,215 records)
• WangchanThaiInstruct sample
• Codex-generated Thai exam synth (multiple rounds, ~3,500 records total)
• 171 real Thai exam Q&A pairs extracted from 313 official PDFs (TGAT, A-Level, ONET)
• Round-specific data mixes

The final adapter is a task-arithmetic merge of multiple SFT rounds — directly averaging LoRA adapter weights with chosen ratios. See the full experiment log in the [project repo].

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

base_id = "Qwen/Qwen3-8B"
adapter_id = "Jnx03/kanitakorn-qwen3-8b-final"

tok = AutoTokenizer.from_pretrained(base_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True)
model = PeftModel.from_pretrained(model, adapter_id)
model = model.merge_and_unload()
model.eval()

messages = [
    {"role": "system", "content": "คุณเป็นผู้ช่วยภาษาไทย"},
    {"role": "user", "content": "อธิบายหลักการของอัตราการเร่งในวิชาฟิสิกส์"}
]
text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Limitations

• ThaiExam is 2pp behind Pathumma-1.0-7B (which uses continued-pretrained OpenThaiLLM-Prebuilt-7B as its base). Closing this gap likely requires continued pretraining on Thai web/book corpora, not just SFT.
• wisesight (Thai sentiment) is below Pathumma by 24pp — sentiment-specific training data is underrepresented in our mix.
• AIME (competition math) is weak (~0-3%) as expected for 8B models without competition-math RL.
• MT-Bench-TH is not scored — would require an LLM judge.

Citation

@misc{kanitakorn2026,
  title = {Kanitakorn: Thai-Optimized Qwen3-8B via Multi-Round LoRA SFT + Task-Arithmetic Merging},
  author = {Jnx03},
  year = {2026},
  publisher = {HuggingFace},
  url = {https://huggingface.co/Jnx03/kanitakorn-qwen3-8b-final}
}