Downloads · 30 days
95
16% of all-time downloads
mehmettozlu/Turkish-Qwen-2.5-7B-Instruct-GGUF
Turkish-Qwen-2.5-7B-Instruct-GGUF is a machine learning model from mehmettozlu. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for unsloth. The card lists the license as apache-2.0.
This repository contains the GGUF formatted version of the Qwen2.5-7B-Instruct model, which has been fine-tuned on a Turkish instruction dataset using Unsloth.
Downloads · 30 days
95
16% of all-time downloads
All-time downloads
577
Public
Repo size
4.7 GB
Likes
0
Public
Click a slice to open those files.
.gguf4.7 GB · 100%
From the Hugging Face model README
This repository contains the GGUF formatted version of the Qwen2.5-7B-Instruct model, which has been fine-tuned on a Turkish instruction dataset using Unsloth.
The model is optimized to act as a highly capable Turkish AI assistant running locally and 100% offline via llama.cpp.
Currently, the most optimal and highly requested quantization version is available in this repository:
| File Name | Size | Recommended RAM | Description |
|---|---|---|---|
*q4_k_m.gguf | ~4.3 GB | 8 GB | 🔥 The Golden Standard. Offers the best balance between inference speed and model intelligence. Ideal for running on standard consumer hardware. |
This model has been tested under identical conditions alongside other popular Turkish GGUF models and evaluated via an LLM-as-a-Judge benchmark to measure Turkish language proficiency, instruction-following capabilities, and coding performance.
| Model | Parameters | Geography (10) | Email Formatting (10) | Coding (10) | Total | Performance Summary |
|---|---|---|---|---|---|---|
| Qwen-2.5-Instruct | 7B | 4.0 | 8.5 | 10.0 | 22.5 / 30 | Most Balanced: Flawless Python code, fluent Turkish, and high instruction adherence. Minor hallucination tendencies on local geographical data. |
| Llama-3.1-Instruct | 8B | 0.5 | 0.0 | 5.0 | 5.5 / 30 | Partial Success: Strong algorithmic background (generates working code), but suffers from severe token repetition and looping on text tasks. |
| Mistral-NeMo-Instruct | 12B | 2.0 | 2.0 | 1.0 | 5.0 / 30 | Weak Instruction Following: While grammar is readable, it lacks task orientation (generates a list instead of code, fails to formalize casual tone). |
| Gemma-2-IT | 9B | 0.0 | 0.0 | 0.0 | 0.0 / 30 | Format Incompatibility: Due to special token structures and quantization sensitivity, it fails to produce meaningful output and enters a repetition loop. |
The Qwen 2.5 architecture uses the ChatML format. To prevent hallucinations and get the best performance, you must use the following structure when interacting with the model via API or code:
<|im_start|>system
Sen yardımsever bir Türkçe asistansın.<|im_end|>
<|im_start|>user
[Write your prompt here]<|im_end|>
<|im_start|>assistant
You can run this model locally with complete privacy and zero internet connection required.
llama-cpp-python)If you want to integrate the model into your own Python applications or RAG pipelines, you can easily use the llama-cpp-python library.
pip install llama-cpp-python
(Note: If you want GPU acceleration, refer to the llama-cpp-python documentation for cuDNN/Metal installation commands).
wget -O Turkish-Qwen-2.5-7B-Instruct-q4_k_m.gguf https://huggingface.co/mehmettozlu/Turkish-Qwen-2.5-7B-Instruct-GGUF/resolve/main/Qwen2.5-7B-Instruct.Q4_K_M.gguf
from llama_cpp import Llama
# 1. Load the model
llm = Llama(
model_path="./Turkish-Qwen-2.5-7B-Instruct-q4_k_m.gguf", # Path to your downloaded GGUF file
n_ctx=2048, # Context window size
n_gpu_layers=-1 # Offload all layers to GPU (set to 0 if using only CPU)
)
# 2. Chat with the model
response = llm.create_chat_completion(
messages=[
{"role": "system", "content": "Sen yardımsever bir Türkçe asistansın."},
{"role": "user", "content": "Yapay zeka modellerinin nasıl eğitildiğini kısaca açıklar mısın?"}
]
)
# 3. Print the assistant's response
print(response["choices"][0]["message"]["content"])
If you have Ollama installed, you can call the model directly from your terminal or integrate it into your local projects (e.g., LangChain, RAG pipelines) by creating a custom Modelfile.
Create a text file named Modelfile (no extension) on your computer and paste the following content:
# Note: Replace the filename below if yours is slightly different in the repo
FROM hf.co/mehmettozlu/Turkish-Qwen-2.5-7B-Instruct-GGUF/Turkish-Qwen-2.5-7B-Instruct-q4_k_m.gguf
TEMPLATE """<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""
SYSTEM """Sen yardımsever bir Türkçe asistansın."""
PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"
ollama create turkish-qwen -f Modelfile
ollama run turkish-qwen
If you prefer the raw performance and lightweight nature of the terminal, you can run the model directly using llama.cpp:
llama.cpp repository to your machine and build it (run make on Linux/macOS, or follow CMake instructions for Windows).*q4_k_m.gguf file from this repository and place it in your llama.cpp folder../llama-cli -m Turkish-Qwen-2.5-7B-Instruct-q4_k_m.gguf -c 2048 --conversation --color
(Note: If you are using an older version of llama.cpp, the executable might be named ./main instead of ./llama-cli. The --conversation flag automatically detects the ChatML prompt format embedded in the GGUF file.)