Downloads · 30 days
24
6% of all-time downloads
Nanbeige4.2-3B-AWQ-W4A16-ASYM
LostGentoo/Nanbeige4.2-3B-AWQ-W4A16-ASYM
Nanbeige4.2-3B-AWQ-W4A16-ASYM is a text generation model from LostGentoo. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
Activation-aware 4-bit weight quantization of Nanbeige/Nanbeige4.2-3B produced with llm-compressor.
All-time downloads
435
Public
Parameters
4.2B
3.7 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors3.7 GB · 99%
Parameter types
How the weights are stored.
I323.2B · 75%
Try a prompt
Base models
Model card
From the Hugging Face model README
Nanbeige4.2-3B AWQ W4A16_ASYM
Activation-aware 4-bit weight quantization of Nanbeige/Nanbeige4.2-3B produced with llm-compressor.
Details
| Scheme | W4A16_ASYM (compressed-tensors / pack-quantized) |
| Calibration | 256 samples × 1024 tokens, HuggingFaceH4/ultrachat_200k |
| Quant host | NVIDIA RTX 5060 Ti 16GB |
| Fork commit | 01af9909913c38fe9b77fb4be0f019573e4cfb6f (nanbeige42) |
| Size | ~3.5GB safetensors |
Requires Nanbeige/vllm branch nanbeige42 — stock vLLM cannot load NanbeigeForCausalLM.
Serve
vllm serve LostGentoo/Nanbeige4.2-3B-AWQ-W4A16-ASYM \
--host 0.0.0.0 --port 8000 \
--tensor-parallel-size 1 \
--gpu-memory-utilization 0.90 \
--max-model-len 32768 \
--enable-auto-tool-choice \
--tool-call-parser nanbeige \
--reasoning-parser nanbeige \
--trust-remote-code
- Task
- Text Generation
- Library
- transformers
- Type
- nanbeige
- License
- apache-2.0
- Created
- Jul 22, 2026
- Updated
- Jul 22, 2026