Llama-3.2-3B-Instruct-f32-GGUF

Llama-3.2-3B-Instruct-f32-GGUF

Llama 3.2 3B Instruct by Meta is a lightweight, instruction-tuned large language model with 3.21 billion parameters, designed for efficient text-only, multilingual dialogue applications such as agentic retrieval, summarization, and instruction following. It employs an optimized transformer architecture with supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF) to align with human-like helpfulness and safety. Boasting an impressive context length capacity of up to 128,000 tokens, it supports multiple languages including English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai.

The model is tailored for on-device and edge deployment, enabling fast, private, and low-latency inference while maintaining high-quality performance on benchmarks, making it suitable for mobile and resource-constrained environments. Llama 3.2 3B Instruct excels in text generation, rewriting, and dialogue tasks, and represents a robust solution for building advanced AI assistants with strong multilingual and instruction-following capabilities.

Model Files

Quants Usage

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):