Kumru-2B-Thinking-v1.0

<p align="center"> <img src="./assets/kumru_header.png" alt="Kumru-2B-Thinking-v1.0 header" width="100%"> </p> <p align="center"> <img src="https://img.shields.io/badge/Language-Turkish-red?style=for-the-badge" /> <img src="https://img.shields.io/badge/Model-2.3B-blue?style=for-the-badge" /> <img src="https://img.shields.io/badge/Fine--tune-LoRA%20r%3D128-purple?style=for-the-badge" /> <img src="https://img.shields.io/badge/Context-8192-orange?style=for-the-badge" /> <img src="https://img.shields.io/badge/License-Apache--2.0-green?style=for-the-badge" /> </p> <p align="center"> <b>A compact Turkish reasoning model fine-tuned to generate structured <code>&lt;think&gt;...&lt;/think&gt;</code> reasoning traces.</b> </p> <p align="center"> <img src="./assets/model_specs_strip.png" alt="Model specs strip" width="100%"> </p>

Kumru-2B-Thinking-v1.0

Kumru-2B-Thinking-v1.0 is a Turkish reasoning-oriented language model built on top of vngrs-ai/Kumru-2B.

It was fine-tuned with rank-128 LoRA on a curated Turkish reasoning dataset to make the model produce structured reasoning traces before giving its final answer.

The model usually answers in this format:

<think>
Türkçe akıl yürütme süreci burada yer alır.
</think>

Final cevap burada yer alır.

This release is intended for Turkish reasoning experiments, educational demos, benchmark studies, and further alignment or fine-tuning research. It is a compact 2.3B model, so it should be used with realistic expectations and external verification for tasks that require high reliability.

---

Table of contents

• Model summary
• Overview
• Türkçe açıklama
• Quick start
• Recommended generation settings
• Reasoning format
• Benchmarks
• Training recipe
• Training data
• Example output
• Intended use
• Limitations
• Citation
• Acknowledgments
• License

---

Model summary

---

Overview

Kumru-2B-Thinking-v1.0 is designed to make a small Turkish language model better at producing structured and inspectable reasoning-style outputs.

The goal is not only to provide a short answer, but also to separate the model’s reasoning trace from the final answer. This makes the model useful for experiments where the reasoning process needs to be evaluated, parsed, compared, or used as synthetic training data.

The model is especially useful for:

• Turkish reasoning-format experiments
• Educational question answering demos
• Turkish benchmark evaluation pipelines
• Chain-of-thought style data generation experiments
• Further LoRA/full fine-tuning experiments
• GRPO/RL-style research starting from a Turkish reasoning model
• Small-model reasoning behavior analysis

Compared with the original Kumru-2B-Instruct baseline, this model shows better benchmark performance under the evaluation setup used in this release and more consistently follows a Turkish thinking-style response format.

---

Türkçe açıklama

Kumru-2B-Thinking-v1.0, Türkçe akıl yürütme formatı üretmesi için fine-tune edilmiş kompakt bir dil modelidir.

Model, cevap vermeden önce genellikle <think>...</think> bloğu içinde Türkçe bir düşünme/akıl yürütme bölümü üretir. Ardından final cevabı ayrı şekilde verir. Bu yapı, özellikle Türkçe reasoning çalışmaları, eğitim amaçlı demolar, benchmark denemeleri ve daha sonraki fine-tuning/RL çalışmalarında kullanışlıdır.

Bu modelin amacı, Türkçe görevlerde daha okunabilir ve ayrıştırılabilir reasoning çıktıları üretmektir. Model küçük ölçekli olduğu için matematik, kodlama, güncel bilgi, uzun bağlamlı karmaşık çıkarım ve yüksek riskli karar verme görevlerinde tek başına güvenilir bir sistem olarak kullanılmamalıdır.

---

Quick start

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "dogukanvzr/Kumru-2B-Thinking-v1.0"

tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

# Recommended for faster generation during inference
model.config.use_cache = True

messages = [
    {
        "role": "user",
        "content": (
            "Bir trende 96 yolcu var. İlk durakta yolcuların dörtte biri iniyor "
            "ve 18 yolcu biniyor. İkinci durakta mevcut yolcuların üçte biri iniyor. "
            "Son durumda trende kaç yolcu kalır?"
        ),
    }
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(
    inputs,
    max_new_tokens=1024,
    temperature=0.7,
    do_sample=True,
    top_p=0.95,
    repetition_penalty=1.05,
)

generated = tokenizer.decode(
    outputs[0][inputs.shape[-1]:],
    skip_special_tokens=True,
)

print(generated)

---

Recommended generation settings

For reasoning-style interactive use:

generation_config = {
    "max_new_tokens": 1024,
    "temperature": 0.6,
    "top_p": 0.95,
    "do_sample": True,
    "repetition_penalty": 1.05,
}

For more deterministic benchmark-style evaluation:

generation_config = {
    "max_new_tokens": 1024,
    "do_sample": False,
}

Practical notes:

• Use Turkish prompts for best results.
• Use max_new_tokens=768 or 1024 for reasoning-heavy tasks.
• Use shorter token limits for simple Q&A if you do not need long reasoning.
• For multiple-choice evaluation, ask the model to produce a strict final answer format.
• If generation is slow, make sure model.config.use_cache = True is enabled.

---

Reasoning format

The model was trained to produce a visible reasoning block followed by the final answer:

<think>
Burada model problemi Türkçe olarak adım adım değerlendirir.
</think>

Burada final cevap yer alır.

For tasks where you need reliable extraction, explicitly request a final answer marker.

Example for multiple-choice tasks:

Aşağıdaki soruyu çöz. Seçenekleri dikkatlice değerlendir.
Cevabı düşünüp son satırda yalnızca şu formatta yaz:

FINAL_ANSWER: <A/B/C/D>

Example for numerical tasks:

Problemi adım adım çöz.
Son satırda yalnızca şu formatta yaz:

FINAL_ANSWER: <sayı>

---

Splitting reasoning and final answer

A simple parser can separate the <think> block from the final answer:

import re

def split_thinking(text: str):
    match = re.search(r"<think>(.*?)</think>", text, flags=re.DOTALL)

    if not match:
        return None, text.strip()

    reasoning = match.group(1).strip()
    final_answer = text[match.end():].strip()

    return reasoning, final_answer

---

Benchmarks

The model was evaluated on 7 Turkish benchmark tasks:

• MMLU-TR
• ARC-TR
• TruthfulQA-TR
• Winogrande-TR
• HellaSwag-TR
• Belebele-TR
• GSM8K-TR

Evaluation setup:

7 tasks × 250 samples × 3 seeds × 2 decoding strategies

Reported metric:

Accuracy

For multiple-choice tasks, the answer was extracted from the final answer letter.
For GSM8K-TR, the final numerical answer was extracted.

<p align="center"> <img src="./assets/benchmark_average.png" alt="Average benchmark accuracy comparison" width="92%"> </p> <p align="center"> <img src="./assets/benchmark_tasks_sampling.png" alt="Per-task benchmark results with sampling" width="92%"> </p>

Results with max_new_tokens=1024

max_new_tokens=1024 is recommended for this model because reasoning traces can be longer than normal instruction-tuned outputs.

Sampling, temperature 0.7

Greedy decoding

<p align="center"> <img src="./assets/benchmark_tasks_greedy.png" alt="Per-task benchmark results with greedy decoding" width="92%"> </p>

Interpretation

Kumru-2B-Thinking-v1.0 improves over the Kumru-2B-Instruct baseline in the tested setup, especially on several Turkish multiple-choice reasoning benchmarks.

The main value of the model is not only the benchmark improvement, but also its ability to produce a consistent, inspectable Turkish reasoning format.

At the same time, this is still a compact 2.3B model. Absolute scores remain limited, especially for mathematical reasoning. For math, factual questions, or high-reliability use cases, answers should be verified externally.

<details> <summary>Additional token-budget visualizations</summary> <p align="center"> <img src="./assets/token_budget_delta_sampling.png" alt="Token budget delta sampling" width="92%"> </p> <p align="center"> <img src="./assets/token_budget_delta_greedy.png" alt="Token budget delta greedy" width="92%"> </p> </details>

---

Training recipe

<p align="center"> <img src="./assets/training_pipeline.png" alt="Training pipeline" width="96%"> </p>

Kumru-2B-Thinking-v1.0 was fine-tuned from vngrs-ai/Kumru-2B using LoRA.

The fine-tuning objective was to teach the model to generate structured Turkish reasoning traces while preserving general instruction-following behavior from the base model.

---

Training data

The model was trained on a curated Turkish reasoning dataset.

Dataset used:

dogukanvzr/kumru-v5-dataset

The dataset format encourages the model to produce:

1. A Turkish reasoning section inside <think>...</think>
2. A concise final answer after the reasoning block

The training data focuses on Turkish reasoning, instruction following, and answer formatting. The model is therefore expected to perform best on Turkish prompts.

---

Example output

Prompt:

Bir trende 96 yolcu var. İlk durakta yolcuların dörtte biri iniyor ve 18 yolcu biniyor. İkinci durakta mevcut yolcuların üçte biri iniyor. Şimdi trende kaç yolcu vardır?

Example style of output:

<think>
Başlangıçta trende 96 yolcu var.

İlk durakta yolcuların dörtte biri iner:
96 / 4 = 24

Bu durumda kalan yolcu sayısı:
96 - 24 = 72

Sonra 18 yolcu biner:
72 + 18 = 90

İkinci durakta mevcut yolcuların üçte biri iner:
90 / 3 = 30

Kalan yolcu sayısı:
90 - 30 = 60
</think>

Trende son durumda 60 yolcu vardır.

---

Intended use

<p align="center"> <img src="./assets/positioning_cards.png" alt="Model positioning cards" width="96%"> </p>

This model is suitable for:

• Turkish reasoning experiments
• Turkish educational assistant prototypes
• Chain-of-thought style output experiments
• Turkish benchmark and evaluation pipelines
• Synthetic reasoning data experiments
• Further fine-tuning and alignment research
• Small-model reasoning behavior analysis

---

Out-of-scope use

This model should not be used as the sole source of truth for:

• Medical, legal, financial, or safety-critical decisions
• High-stakes educational grading
• Fully automated expert systems
• Sensitive personal data processing
• Production systems that require strong factual reliability
• Mathematical solving without external verification
• Code generation without testing

---

Limitations

Kumru-2B-Thinking-v1.0 has several known limitations.

Mathematical reasoning

The model can produce step-by-step reasoning, but numerical accuracy is not guaranteed. Mathematical answers should be checked with a calculator, symbolic tool, or a stronger verifier model.

Factual accuracy

The model may hallucinate facts, dates, names, sources, or explanations. It does not have live internet access and should not be treated as a current knowledge system.

Long reasoning traces

The model may sometimes overthink simple questions or produce longer reasoning than necessary. Use a lower max_new_tokens value for short responses and higher values for benchmark-style reasoning.

English and code-mixed prompts

The model is Turkish-first. It may respond to English or mixed Turkish-English prompts, but quality is expected to be better on Turkish prompts.

Formatting

The model is trained to use <think>...</think>, but downstream applications should still validate the output format if strict parsing is required.

Safety

The model has not been extensively safety-aligned beyond the behavior inherited from the base model and the fine-tuning data. Additional safety filtering is recommended for public-facing applications.

---

Practical recommendations

For best results:

1. Use Turkish prompts.
2. Ask for a strict final answer format when evaluating.
3. Use max_new_tokens=768 or 1024 for reasoning-heavy tasks.
4. Use temperature=0.6–0.7 for reasoning-style sampling.
5. Use greedy decoding for deterministic benchmark runs.
6. Verify math and factual claims externally.
7. Parse the final answer separately from the <think> block.
8. Avoid using the model as the only decision-maker in high-stakes settings.

---

Architecture

This model inherits the architecture and tokenizer from vngrs-ai/Kumru-2B.

Fine-tuning was performed with LoRA adapters applied to the main attention and MLP projection layers, with additional trainable components where needed for reasoning-format adaptation.

The released checkpoint is provided in safetensors format for loading with the Hugging Face Transformers ecosystem.

---

Chat template

The model uses a chat template compatible with Hugging Face apply_chat_template.

Recommended usage:

messages = [
    {"role": "user", "content": "Sorunu Türkçe olarak buraya yaz."}
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt",
)

The model was trained primarily for direct Turkish user instructions. For best results, place task instructions clearly in the user message.

---

Loading with lower precision

Depending on hardware, the model can be loaded in bf16 or fp16.

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "dogukanvzr/Kumru-2B-Thinking-v1.0"

tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

model.config.use_cache = True

For limited VRAM environments, users may apply external quantization workflows such as bitsandbytes or GGUF conversion. Quantized releases are not guaranteed to match the benchmark results shown above unless evaluated separately.

---

Citation

If you use this model, please cite it as:

@misc{veziroglu2026kumru2bthinkingv1,
  title        = {Kumru-2B-Thinking-v1.0: A Turkish Reasoning-Oriented Fine-Tuned Language Model},
  author       = {Dogukan Veziroglu},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/dogukanvzr/Kumru-2B-Thinking-v1.0}}
}

---

Acknowledgments

This model is based on vngrs-ai/Kumru-2B.

Thanks to the VNGRS-AI team for releasing Turkish language models that make further Turkish NLP and reasoning experiments possible.

---

License

This model is released under the Apache-2.0 license.

Please also check the license and usage terms of the base model and any datasets used in your downstream application.

---

Contact

For questions, issues, or feedback, please use the Hugging Face community tab or contact the model author through the Hugging Face profile:

dogukanvzr