Downloads · 30 days
127
16% of all-time downloads
mehmettozlu/Turkish-Llama-3.1-8B-Instruct-GGUF
Turkish-Llama-3.1-8B-Instruct-GGUF is a machine learning model from mehmettozlu. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for unsloth. The card lists the license as llama3.1.
This repository contains the GGUF formatted version of the Meta-Llama-3.1-8B-Instruct model, which has been fine-tuned on a Turkish instruction dataset using Unsloth.
Downloads · 30 days
127
16% of all-time downloads
All-time downloads
792
Public
Repo size
35.3 GB
Likes
0
Public
Click a slice to open those files.
.gguf35.3 GB · 100%
From the Hugging Face model README
This repository contains the GGUF formatted version of the Meta-Llama-3.1-8B-Instruct model, which has been fine-tuned on a Turkish instruction dataset using Unsloth.
The model is optimized to act as a highly capable Turkish AI assistant running locally and 100% offline via llama.cpp, Ollama, and LM Studio.
We provide multiple quantization options. The q4_k_m version is highly recommended for consumer hardware:
| File Name | Size | Recommended RAM | Description |
|---|---|---|---|
*q4_k_m.gguf | ~4.9 GB | 8 GB | 🔥 The Golden Standard. Offers the best balance between inference speed and model intelligence. Ideal for running on standard consumer hardware. |
*q5_k_m.gguf | ~5.7 GB | 10 GB | Higher quality with a slight trade-off in generation speed. |
*q8_0.gguf | ~8.5 GB | 12 GB | Near-lossless original quality. Requires more RAM/VRAM. |
This model has been tested under identical conditions alongside other popular Turkish GGUF models and evaluated via an LLM-as-a-Judge benchmark to measure Turkish language proficiency, instruction-following capabilities, and coding performance.
| Model | Parameters | Geography (10) | Email Formatting (10) | Coding (10) | Total | Performance Summary |
|---|---|---|---|---|---|---|
| Qwen-2.5-Instruct | 7B | 4.0 | 8.5 | 10.0 | 22.5 / 30 | Most Balanced: Flawless Python code, fluent Turkish, and high instruction adherence. Minor hallucination tendencies on local geographical data. |
| Llama-3.1-Instruct | 8B | 0.5 | 0.0 | 5.0 | 5.5 / 30 | Partial Success: Strong algorithmic background (generates working code), but suffers from severe token repetition and looping on text tasks. |
| Mistral-NeMo-Instruct | 12B | 2.0 | 2.0 | 1.0 | 5.0 / 30 | Weak Instruction Following: While grammar is readable, it lacks task orientation (generates a list instead of code, fails to formalize casual tone). |
| Gemma-2-IT | 9B | 0.0 | 0.0 | 0.0 | 0.0 / 30 | Format Incompatibility: Due to special token structures and quantization sensitivity, it fails to produce meaningful output and enters a repetition loop. |
Llama 3.1 uses a specific token structure for chat completions (<|start_header_id|> and <|eot_id|>). To prevent hallucinations and get the best performance, you must use the following standard structure:
<|begin_of_text|><|start_header_id|>system<|end_header_id|>
Sen yardımsever bir Türkçe asistansın.<|eot_id|><|start_header_id|>user<|end_header_id|>
[Write your prompt here]<|eot_id|><|start_header_id|>assistant<|end_header_id|>
You can run this model locally with complete privacy and zero internet connection required.
pip install llama-cpp-python
wget -O Turkish-Llama-3.1-8B-Instruct-q4_k_m.gguf [https://huggingface.co/mehmettozlu/Turkish-Llama-3.1-8B-Instruct-GGUF/resolve/main/Turkish-Llama-3.1-8B-Instruct-q4_k_m.gguf](https://huggingface.co/mehmettozlu/Turkish-Llama-3.1-8B-Instruct-GGUF/resolve/main/Turkish-Llama-3.1-8B-Instruct-q4_k_m.gguf)
from llama_cpp import Llama
llm = Llama(
model_path="./Turkish-Llama-3.1-8B-Instruct-q4_k_m.gguf",
n_ctx=4096, # Context window size
n_gpu_layers=-1 # Offload all layers to GPU
)
response = llm.create_chat_completion(
messages=[
{"role": "system", "content": "Sen yardımsever bir Türkçe asistansın."},
{"role": "user", "content": "Yapay zeka modellerinin nasıl eğitildiğini kısaca açıklar mısın?"}
]
)
print(response["choices"][0]["message"]["content"])
Create a text file named Modelfile (no extension) and paste the following content:
FROM hf.co/mehmettozlu/Turkish-Llama-3.1-8B-Instruct-GGUF/Turkish-Llama-3.1-8B-Instruct-q4_k_m.gguf
TEMPLATE """{{ if .System }}<|start_header_id|>system<|end_header_id|>
{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>
{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>
"""
SYSTEM """Sen yardımsever bir Türkçe asistansın."""
PARAMETER stop "<|start_header_id|>"
PARAMETER stop "<|end_header_id|>"
PARAMETER stop "<|eot_id|>"
PARAMETER stop "<|eom_id|>"
Build and run the model:
ollama create turkish-llama-8b -f Modelfile
ollama run turkish-llama-8b