Cozum-4B-GGUF

Yigit-Karaman/Cozum-4B-GGUF

Cozum-4B-GGUF is a text generation model from Yigit-Karaman. Use it when you need the model to write or continue text.

This repository contains a comprehensive collection of quantized GGUF weights for Yigit-Karaman/Cozum-4B, optimized for efficient local execution and offline inference across various hardware setups.

Downloads · 30 days

646

16% of all-time downloads

All-time downloads

4.1K

Public

Repo size

48.1 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.gguf27.2 GB · 100%

At a glance

Task: Text Generation
Access: Public
Created: Jul 20, 2026
Updated: Jul 22, 2026
SHA: 3605597e

Try a prompt

Base models

Yigit-Karaman/Cozum-4B

Model card

From the Hugging Face model README

Cozum-4B-GGUF

This repository contains a comprehensive collection of quantized GGUF weights for Yigit-Karaman/Cozum-4B, optimized for efficient local execution and offline inference across various hardware setups.

Available Files & Quantizations

We provide a "ladder" of quantization options so you can choose the best fit for your machine's hardware capabilities. Compatibility notes are tailored for an 8GB VRAM profile (e.g., RTX 4060 Laptop):

File Name	Size	Recommended Use
`Cozum-4B_BF16.gguf`	8.42 GB	Unquantized 16-bit: Maximum precision, but exceeds standard 8GB VRAM limits. Will require system RAM offloading and run slower.
`Cozum-4B_Q8_0.gguf`	4.48 GB	Maximum Quality (8-bit): Near-unquantized performance. Ideal for dedicated GPUs with 8GB+ VRAM, fitting perfectly with room for context.
`Cozum-4B_Q6_K.gguf`	3.46 GB	High Quality (6-bit): Very minimal degradation. Leaves excellent VRAM headroom for large system prompts or extended conversations.
`Cozum-4B_Q5_K_M.gguf`	3.07 GB	The Sweet Spot (5-bit): Excellent balance of quality and size. Highly recommended for general use and coding.
`Cozum-4B_Q4_K_M.gguf`	2.71 GB	Mainstream Standard (4-bit): The best balance of speed and acceptable quality for mid-range hardware.
`Cozum-4B_Q3_K_M.gguf`	2.26 GB	Low-End Hardware (3-bit): Maximum compression for severe memory constraints.
`Cozum-4B_Q2_K_L.gguf`	2.07 GB	Extreme Compression (2-bit): Only use if absolutely necessary; expects noticeable degradation in complex logic and generation quality.
`Cozum-4B_BF16-mmproj.gguf`	676 MB	Multimodal Projection: Required file if utilizing the model's vision/image processing capabilities.

Model Overview

Base Model: Yigit-Karaman/Cozum-4B
Architecture: Qwen (4B Parameters)
Format: GGUF (Various Quantization Levels)
Primary Use Case: Local text generation, code assistance, and lightweight AI deployment.

Prompt Format (ChatML)

This model natively uses the ChatML format. Ensure your local runner's prompt template matches the structure below to prevent repetitive or unexpected generation output:

<|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
Hello!<|im_end|>
<|im_start|>assistant

Task: Text Generation
Created: Jul 20, 2026
Updated: Jul 22, 2026