Downloads · 30 days
749
26% of all-time downloads
Vikhr-Llama-3.2-1B-Instruct-GGUF
tinyopsec/Vikhr-Llama-3.2-1B-Instruct-GGUF
Vikhr-Llama-3.2-1B-Instruct-GGUF is a text generation model from tinyopsec. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as llama3.2.
GGUF quantizations of Vikhrmodels/Vikhr-Llama-3.2-1B-Instruct.
All-time downloads
2.9K
Public
Repo size
10.9 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf10.9 GB · 100%
At a glance
- Task
- Text Generation
- Library
- gguf
- License
- llama3.2
- Access
- Public
- Created
- Sep 14, 2026
- Updated
- Sep 14, 2026
- SHA
- c9811003
Base models
Model card
From the Hugging Face model README
Vikhr-Llama-3.2-1B-Instruct GGUF
GGUF quantizations of Vikhrmodels/Vikhr-Llama-3.2-1B-Instruct.
Quantization Table
| File | Bits | Size | Use Case |
|---|---|---|---|
model_f16.gguf | 16 | ~2.5 GB | Maximum quality, reference |
model_q8_0.gguf | 8 | ~1.3 GB | Best quality, near-lossless |
model_q6_k.gguf | 6 | ~1.0 GB | Great quality |
model_q5_k_m.gguf | 5 | ~0.9 GB | Balanced quality/size |
model_q5_k_s.gguf | 5 | ~0.85 GB | Slightly smaller Q5 |
model_q4_k_m.gguf | 4 | ~0.77 GB | Recommended, good balance |
model_q4_k_s.gguf | 4 | ~0.72 GB | Smaller Q4 |
model_q3_k_l.gguf | 3 | ~0.65 GB | Low VRAM, decent quality |
model_q3_k_m.gguf | 3 | ~0.60 GB | Low VRAM |
model_q3_k_s.gguf | 3 | ~0.55 GB | Very low VRAM |
model_q2_k.gguf | 2 | ~0.45 GB | Minimum size, lowest quality |
VRAM Requirements
| Quant | VRAM |
|---|---|
| F16 | ~3 GB |
| Q8_0 | ~1.8 GB |
| Q6_K | ~1.5 GB |
| Q5_K_M | ~1.3 GB |
| Q4_K_M | ~1.1 GB |
| Q3_K_M | ~0.8 GB |
| Q2_K | ~0.6 GB |
Usage
llama.cpp
./llama-cli -m model_q4_k_m.gguf -p "Your prompt here" -n 512
llama-cpp-python
from llama_cpp import Llama
llm = Llama(
model_path="model_q4_k_m.gguf",
n_ctx=2048,
n_gpu_layers=-1
)
response = llm(
"Your prompt here",
max_tokens=512,
stop=["<|eot_id|>"]
)
print(response["choices"][0]["text"])
LM Studio
Download any .gguf file and load it directly in LM Studio.
Ollama
ollama run hf.co/tinyopsec/Vikhr-Llama-3.2-1B-Instruct-GGUF
Original Model
Vikhrmodels/Vikhr-Llama-3.2-1B-Instruct
Quantized using llama.cpp.
- Task
- Text Generation
- Library
- gguf
- License
- llama3.2
- Languages
- ru, en
- Created
- Sep 14, 2026
- Updated
- Sep 14, 2026