shahnameh-qwen3-4b-fa-lora

Shahnameh Qwen3-4B — Persian LoRA

A LoRA adapter that makes Qwen3-4B-Instruct-2507 answer questions in Persian masnavi verse, in the style of Ferdowsi's Shahnameh.

This is the best Persian of the three models in this project.

Measured quality

Persian masnavi rhymes in couplets (AA, BB, CC), which makes quality partly measurable: compare the ends of each line pair and count.

Measured on 10 held-out Persian prompts never seen in training.

Trade-off you should know about

This model writes better Persian than its multilingual sibling, but it is less stable. It was trained 12 epochs at LoRA r=64 — deliberately past the lowest-validation-loss point, because for style transfer that point yields an underfit model. Validation loss bottomed at epoch 3 (2.43) and rose to 3.71 by epoch 12 while training loss fell to 0.84.

Rhyme improved anyway (33% → 65%), but overfitting cost something loss never measures: the model occasionally emits characters from other scripts inside Persian verse — Chinese, Latin, Cyrillic. Qwen3 is Chinese-developed, so those tokens carry high prior probability, and overfitting degraded the model's ability to suppress them. Roughly 6 stray characters across 10 answers.

If you need clean script in three languages, use SaeedAngiz1/shahnameh-qwen3-4b-multilingual-lora instead.

Usage

from unsloth import FastLanguageModel
import torch

model, tokenizer = FastLanguageModel.from_pretrained(
    "SaeedAngiz1/shahnameh-qwen3-4b-fa-lora",
    max_seq_length=1024, load_in_4bit=True, dtype=None)
FastLanguageModel.for_inference(model)

messages = [
    {"role": "system", "content": "تو شاعری هستی که به شیوه‌ی شاهنامه‌ی فردوسی، به شعر فارسی پاسخ می‌دهی."},
    {"role": "user", "content": "رستم که بود؟"},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
ids = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to("cuda")
out = model.generate(**ids, max_new_tokens=256, do_sample=True,
                     temperature=0.7, top_p=0.8, top_k=20,
                     pad_token_id=tokenizer.eos_token_id)
print(tokenizer.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=True))

Do not add repetition_penalty. Masnavi rhyme depends on recurring sounds and formulaic function words (به, ز, که, چو); penalising repetition breaks the form. Measured: 39% rhyme with repetition_penalty=1.1 versus 53% with none.

Training

Qwen3-4B-Instruct-2507 (4-bit) · LoRA r=64, lora_alpha=64 · lr 3e-4 · 12 epochs · 90 Persian examples · response-only loss · 2× Tesla T4 · 724 seconds.

Honest limitations

• Facts are frequently wrong. The dataset teaches voice, not Shahnameh genealogy. Expect invented names and inverted relationships.
• Persian has never been reviewed by a native speaker. Rhyme and script purity are measured mechanically; meter (عروض), grammar and classical register are unverified.
• Output varies run to run at temperature=0.7. Judge on averages, not favourites.
• Trained on only 270 (or 90) examples. That is enough for style, not for prosody.

How it was built

Trained on Kaggle's free tier: 2× NVIDIA Tesla T4. Total compute for the whole project is under three hours. Full method, measurements, charts, and a documented record of four separate failures: https://github.com/SaeedAngiz1/shahnameh-slm

Credits

Ferdowsi's Shahnameh (completed 1010 CE). The training verse is newly written in his style — it is not his text.