Downloads · 30 days
5
3% of all-time downloads
TheDrummer-Behemoth-X-123B-v2-NVFP4
ealexeev/TheDrummer-Behemoth-X-123B-v2-NVFP4
TheDrummer-Behemoth-X-123B-v2-NVFP4 is a machine learning model from ealexeev. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
This is an NVFP4 quantization of TheDrummer/Behemoth-X-123B-v2.
All-time downloads
155
Public
Parameters
69.3B
70.1 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors70.1 GB · 100%
Parameter types
How the weights are stored.
U860.9B · 88%
Base models
Model card
From the Hugging Face model README
Model Card for ealexeev/TheDrummer-Behemoth-X-123B-v2-NVFP4
This is an NVFP4 quantization of TheDrummer/Behemoth-X-123B-v2.
Quantization Details
Used https://github.com/ealexeev/llm-quantization script.
Calibration dataset size: 1024 Calibration data:
- HuggingFaceH4/ultrachat_200k
- allenai/c4_en
- mrcedric98/fiction_books_v8
These were shuffled and mixed at a ratio of 3:2:3
Procedure
python ./quantize_nvfp4.py --model TheDrummer/Behemoth-X-123B-v2 --output ./TheDrummer/Behemoth-X-123B-v2 --size 1024 --seed 42 --ultra_chat 3 --c4_en 2 --fiction_v8 3
I had read in VLLM docs that NVFP4 quantization needs very few samples. In my experience 1024 is a good number for larger models.
Quantization Evals
| Metric | NVFP4 |
|---|---|
| ARC Challenge (Logic/Reasoning) | 0.6391 |
| IFEval (Strict Instruction Following) | 0.7431 |
| HellaSwag (Flow/Common Sense) | 0.6961 |
| Wikitext (Word Perplexity) | 4.5030 |
| Lambada (Perplexity) | 2.4814 |
| Winogrande | 0.8089 |
Bias, Risks, and Limitations
This is already a creative fine-tune. It was quantized with that usecase in mind. Probably not gonna pass any leet-coder challenges with this one.
How To Use
bash
vllm serve ealexeev/TheDrummer-Behemoth-X-123B-v2-NVFP4 \
--tensor-parallel-size 1 \ # 1 GPU
--gpu-memory-utilization 0.8 \ # Else it will take it all for KV
- Type
- mistral
- Created
- Mar 9, 2026
- Updated
- Mar 11, 2026