Slava-Qwen3-14B-Serbian

Slava: a Serbian Multi-Task LLM (Qwen3-14B)

Slava is a Qwen3-14B model full-fine-tuned for Serbian multi-task NLP. It is the model produced and evaluated in the master's thesis "A Hybrid System for Serbian Natural Language Processing with Large Language Models" (University of Vienna, 2026), and powers the generative backend of the accompanying web application.

• Base model: OpenPipe/Qwen3-14B-Instruct (a fine-tune-friendly instruct variant of Qwen3-14B; Apache-2.0)
• Adaptation: full fine-tuning (all weights), single epoch
• Language: Serbian (Cyrillic and Latin script)
• Tasks: grammar correction · Serbian→English translation · summarization · sentiment analysis · aspect-based sentiment analysis (ABSA)
• License: CC BY-NC-SA 4.0 (non-commercial, research use — see Licensing)
• Built with Qwen.

Intended use

Research and non-commercial use for Serbian NLP: instruction-style grammar correction, sr→en translation, abstractive summarization, sentence- and aspect-level sentiment analysis. The model expects a Serbian system instruction with the item in the user turn (see Prompting).

Not intended for: commercial deployment (the training data includes a NonCommercial source, see Licensing); high-stakes or safety-critical use; languages other than Serbian.

Prompting

Deterministic decoding (temperature 0, greedy) was used throughout. Each task uses a Serbian system instruction, e.g.:

Training data

A single instruction-formatted mixture of 24,601 examples across five generative tasks (system/user/assistant turns, with paraphrased instructions in both Cyrillic and Latin script):

Synthetic ABSA reviews and parts of the augmentation were generated with the Qwen2.5-72B-Instruct teacher model. The exact build scripts and per-source licenses are in data/ and data/SOURCES.md. The dataset is not redistributed as a bundle (incompatible source licenses); regenerate it from the scripts + original sources.

Training configuration (deployed 14B)

The training script is in training/train_14b_v3.py.

Evaluation (fine-tuned vs. base Qwen3-14B-Instruct)

Fine-tuning did not change general knowledge. On the full benchmark suite (28,986 questions, chat-based scoring), the base model scored 75.6% and Slava 74.8%, essentially identical. Slava was trained only on the five generative tasks, never on benchmark / multiple-choice data, so the substantive effect of fine-tuning is on the generative and sentiment tasks above.

A key methodological finding of the thesis: the same base model scores ≈34% under log-probability scoring but ≈75.6% under chat-based generation on the identical benchmark — so benchmark scores for instruction-tuned models depend heavily on the scoring protocol. See the thesis for the full per-category breakdown and decontamination details.

Limitations

• Summarization: the fine-tuned model produces noticeably shorter, more compressed summaries that can omit content or be unfaithful. In a native-speaker study, raters strongly preferred the base model for summarization (111 vs. 5). Automatic metrics (ROUGE-L) did not capture this.
• Reasoning-heavy categories (e.g. moral scenarios, mathematics) remained the weakest.
• Single run / single family: one full fine-tuning run of one model family (Qwen3); no run-to-run variance estimate.
• Teacher bias: synthetic portions inherit the Qwen2.5-72B teacher's biases.
• Decontamination was applied to the generative/sentiment evaluation sets but full coverage across the 24,601-example training set cannot be guaranteed.

Licensing & attribution

The released weights are licensed CC BY-NC-SA 4.0 (non-commercial, share-alike), because the training data includes the NonCommercial source SentiComments.SR. The code/scripts in data/ and training/ are released under the MIT License (see NOTICE).

This model is "Built with Qwen":

• Base model: Qwen3-14B (via OpenPipe/Qwen3-14B-Instruct), Apache-2.0 — modified by fine-tuning.
• Teacher (data generation): Qwen2.5-72B-Instruct, under the Qwen LICENSE AGREEMENT.

Data-source licenses and required citations are listed in data/SOURCES.md and CITATIONS.bib. If you use this model or data, please cite the thesis and the underlying datasets (notably Batanović et al., 2020 for SentiComments.SR and Mochtak et al. for ParlaSent).

Citation

@mastersthesis{cvetanovic2026serbian,
  title  = {A Hybrid System for Serbian Natural Language Processing with Large Language Models},
  author = {Cvetanovi\'{c}, Ivan},
  school = {University of Vienna},
  year   = {2026}
}