Fara-7B-GGUF

Microsoft Fara-7B GGUF — Quantized by BatiAI

<p align="center"> <a href="https://flow.bati.ai"><img src="https://img.shields.io/badge/BatiFlow-on--device%20AI-blue?style=for-the-badge&logo=apple"></a> <a href="https://ollama.com/batiai/fara-7b"><img src="https://img.shields.io/badge/Ollama-batiai%2Ffara--7b-green?style=for-the-badge"></a> </p>

imatrix-calibrated GGUF quantizations of microsoft/Fara-7B (Qwen 2.5 VL based multimodal, 7B). Quantized directly from official Microsoft BF16 weights by BatiAI.

Why Fara-7B?

• Microsoft-built agentic multimodal model (Qwen 2.5 VL backbone)
• 7B parameters — runs on Mac mini 16GB
• Multimodal native: text + vision (image-text-to-text)
• 128K context window
• MIT license — fully commercial-friendly
• arxiv:2511.19663 — research paper
• Released 2026-05-19 by Microsoft

Quick Start

# Q4_K_M (recommended for most users, ~5GB)
ollama pull batiai/fara-7b:q4

# IQ3_XXS (smallest, ~3GB, Mac mini 16GB)
ollama pull batiai/fara-7b:iq3

# Q8_0 (highest quality, ~8GB)
ollama pull batiai/fara-7b:q8

Available Quantizations

Multimodal: download mmproj-*-Q6_K.gguf and use with llama-mtmd-cli / llama-server --mmproj.

How to run

Ollama (text-only)

ollama run batiai/fara-7b:q4

llama.cpp (text + vision)

hf download batiai/Fara-7B-GGUF --include "*Q4_K_M*" --include "mmproj-*-Q6_K.gguf" --local-dir ./fara-7b

llama-mtmd-cli \
    -m ./fara-7b/microsoft-Fara-7B-Q4_K_M.gguf \
    --mmproj ./fara-7b/mmproj-microsoft-Fara-7B-Q6_K.gguf \
    --image input.jpg -p "Describe this image."

Model details

• Source: microsoft/Fara-7B
• Architecture: Qwen2_5_VLForConditionalGeneration — Qwen 2.5 VL backbone, Microsoft fine-tuned
• Context: 128K
• License: MIT

BatiAI signing

All GGUFs carry:

• general.author = BatiAI
• general.url = https://flow.bati.ai

License

Inherits source: MIT.

About BatiFlow

BatiFlow — free on-device AI automation for Mac.

<!-- BENCH-START -->

Benchmarks coming once Mac measurements complete.

<!-- BENCH-END -->