Downloads · 30 days
402
3% of all-time downloads
Nanbeige4.2-3B-GGUF
Abiray/Nanbeige4.2-3B-GGUF
Nanbeige4.2-3B-GGUF is a text generation model from Abiray. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as apache-2.0.
This repository contains GGUF quantizations for Nanbeige/Nanbeige4.2-3B.
All-time downloads
14.9K
Public
Repo size
18.1 GB
Likes
16
Public
Repo makeup
Click a slice to open those files.
.gguf18.1 GB · 100%
At a glance
- Task
- Text Generation
- Library
- gguf
- License
- apache-2.0
- Access
- Public
- Created
- Jul 22, 2026
- Updated
- Jul 22, 2026
- SHA
- 774a61f8
Try a prompt
Base models
Model card
From the Hugging Face model README
Nanbeige4.2-3B - GGUF Quants
This repository contains GGUF quantizations for Nanbeige/Nanbeige4.2-3B.
- Original Model: Nanbeige/Nanbeige4.2-3B
- Base Architecture: Looped Transformer (3B non-embedding parameters)
- Quantization Format: GGUF (
Q4_K_M,Q4_K_S,Q5_K_M,Q6_K,Q8_0)
Available Files & Quantization Details
| File Name | Size | Quant Method | Description |
|---|---|---|---|
Nanbeige4.2-3B-Q4_K_M.gguf | ~2.57 GB | Q4_K_M | 4-bit medium. Recommended balance of speed, memory usage, and quality. |
Nanbeige4.2-3B-Q4_K_S.gguf | ~2.50 GB | Q4_K_S | 4-bit small. Slightly lower memory footprint. |
Nanbeige4.2-3B-Q5_K_M.gguf | ~2.99 GB | Q5_K_M | 5-bit medium. Higher precision with slight increase in size. |
Nanbeige4.2-3B-Q6_K.gguf | ~3.42 GB | Q6_K | 6-bit quantization. Very close to FP16 performance. |
Nanbeige4.2-3B-Q8_0.gguf | ~4.43 GB | Q8_0 | 8-bit quantization. Maximum quality for GGUF. |
Usage Guide
1. Running with llama.cpp
For full support, clone the official or nanbeige42 fork of llama.cpp:
# Clone the repository with Nanbeige support
git clone -b nanbeige42 [https://github.com/Nanbeige/llama.cpp.git](https://github.com/Nanbeige/llama.cpp.git)
cd llama.cpp
# Build with CUDA support
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release -j
# Download a model from this repository
huggingface-cli download Abiray/Nanbeige4.2-3B-GGUF Nanbeige4.2-3B-Q4_K_M.gguf --local-dir .
# Run CLI inference
./build/bin/llama-cli -m Nanbeige4.2-3B-Q4_K_M.gguf -ngl 99 -p "Which number is bigger, 9.11 or 9.8?"
- Task
- Text Generation
- Library
- gguf
- License
- apache-2.0
- Languages
- en, zh
- Created
- Jul 22, 2026
- Updated
- Jul 22, 2026