Olive-4B-Thinking-GGUF

Olive-4B-Thinking-GGUF

GGUF conversion of Quitcool/Olive-4B-Enhanced-Fixed-T4

Model Information

• Original Model: Quitcool/Olive-4B-Enhanced-Fixed-T4
• Format: GGUF (F16 precision)
• File Size: 7.49 GB
• Architecture: Qwen3/Llama compatible
• Compatible with: Ollama, llama.cpp, text-generation-webui

Quick Start

Ollama Usage

# Download the model
wget https://huggingface.co/Quitcool/Olive-4B-Thinking-GGUF/resolve/main/Olive-4B-Thinking-GGUF.gguf

# Create Modelfile
cat > Modelfile << 'EOF'
FROM ./Olive-4B-Thinking-GGUF.gguf

TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
<|thinking|>
Let me think about this step by step...
</|thinking|>

{{ .Response }}<|im_end|>
"""

PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"
PARAMETER stop "<|thinking|>"
PARAMETER stop "</|thinking|>"
PARAMETER temperature 0.7
PARAMETER top_p 0.9
PARAMETER repeat_penalty 1.1
EOF

# Create and run the model
ollama create olive_4b_thinking_gguf -f Modelfile
ollama run olive_4b_thinking_gguf

Test the model

ollama run olive_4b_thinking_gguf "Explain quantum computing in simple terms"

Files

• Olive-4B-Thinking-GGUF.gguf - Main GGUF model file (7.49 GB)

Conversion Details

• Method: Core parameter GGUF conversion with RMS norm fix
• Fixed: Missing hyperparameters for llama.cpp compatibility
• Date: 2026-02-25

---

Converted with essential parameters for maximum compatibility