Security-Llama3.2-3B-GGUF

Security-Llama3.2-3B-GGUF

security-llama3.2-3b is a dense, decoder-only Transformer model with approximately 3 billion parameters. It is optimized for generating text, particularly in response to prompts in a chat format, with a context length of up to 4,000 tokens. The model is specialized toward cybersecurity content, drawing on a mixture of publicly available blogs, papers, reference datasets (e.g. from the PEASEC cybersecurity repository), synthetic “textbook-style” data, and academic Q&A sources to enhance performance in security-themed tasks. For usage, the model accepts chat-style inputs (e.g. alternating “user” / “assistant” messages) and can be deployed via the Hugging Face transformers library (e.g. via pipeline("text-generation", model="viettelsecurity-ai/security-llama3.2-3b")). The model weights are stored in safetensors format, configured with fp16 (half precision), and no inference provider currently hosts it by default.

Model Files

Quants Usage

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):