AhiskaAI-308m-Instruct-v0.2

AhiskaAI-308m-IT-v0.2

AhiskaAI-308m-IT-v0.2 is the instruction-tuned version of our 308M parameter Small Language Model. Fine-tuned on 16,000+ curated Turkish instruction-response pairs, it is designed to provide stronger conversational ability and improved instruction following while remaining efficient enough to run on consumer hardware.

Base Model: AhiskaAI-308m-Base-v0.2

---

Model Details

• Architecture: Llama-based architecture.
• Fine-tuning: Supervised Fine-Tuning (SFT).
• Format: ChatML.
• Parameters: 308M.
• Context Window: 1024 tokens.
• Tokenizer: Custom BPE Tokenizer (Vocabulary Size: 32,000).
• Training Framework: PyTorch & Transformers.
• Hardware: NVIDIA RTX 4050 6GB Laptop GPU.

---

Fine-tuning Dataset

The model was fine-tuned using more than 16,000 carefully curated Turkish instruction-response pairs.

The dataset includes tasks such as:

• Question answering
• General conversation
• Summarization
• Text generation
• Instruction following
• Basic reasoning

---

Design Goal

The 308M-IT model serves as the flagship conversational model of the AhiskaAI v0.2 family.

Its primary objectives are:

• Improved Turkish instruction following.
• Better contextual understanding.
• More natural conversational responses.
• A strong research foundation for future preference alignment methods such as DPO.

---

Training Logs

The graph above demonstrates the supervised fine-tuning convergence of AhiskaAI-308m-IT-v0.2.

---

Usage (ChatML Format)

Recommended System Prompt

Sen kibar, sorulan soruları tam cümlelerle yanıtlayan Türkçe bir asistansın.

Example Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("AhiskaAI/AhiskaAI-308m-IT-v0.2")
tokenizer = AutoTokenizer.from_pretrained("AhiskaAI/AhiskaAI-308m-IT-v0.2")

SYSTEM_PROMPT = "Sen kibar, sorulan soruları tam cümlelerle yanıtlayan Türkçe bir asistansın."

user_query = "Ahıska Türkleri hakkında bilgi verir misin?"

prompt = (
    f"<|im_start|>system\n{SYSTEM_PROMPT}<|im_end|>\n"
    f"<|im_start|>user\n{user_query}<|im_end|>\n"
    f"<|im_start|>assistant\n"
)

inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(**inputs, max_new_tokens=200)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---

Known Limitations

• Optimized primarily for Turkish.
• Context window is limited to 1024 tokens.
• Factual accuracy is still limited by model size and pretraining data.
• May generate incorrect or incomplete responses on complex reasoning tasks.

---

Future Plans

• Preference alignment using DPO.
• Larger, higher-quality Turkish datasets.
• Expanded evaluation benchmarks.
• Future AhiskaAI v0.3 model family.

---

About AhiskaAI

AhiskaAI is an independent open-source initiative dedicated to developing efficient Turkish Small Language Models trained completely from scratch.

Follow us on Hugging Face for updates and future releases.