Downloads · 30 days
0
yapwithai/thedrummer-magidonia-24B-v4.3-trt-awq
thedrummer-magidonia-24B-v4.3-trt-awq is a text generation model from yapwithai. Use it when you need the model to write or continue text. The card lists the license as other.
This is a INT4-AWQ quantized version of https://huggingface.co/TheDrummer/Magidonia-24B-v4.3, optimized for TensorRT-LLM inference.
Downloads · 30 days
0
Access
Public
Updated Jan 3, 2026
Repo size
28.7 GB
Likes
0
Public
Click a slice to open those files.
.safetensors14.5 GB · 50%
From the Hugging Face model README
This is a INT4-AWQ quantized version of https://huggingface.co/TheDrummer/Magidonia-24B-v4.3, optimized for TensorRT-LLM inference.
Key Features:
| Specification | Details |
|---|---|
| Source Model | https://huggingface.co/TheDrummer/Magidonia-24B-v4.3 |
| Quantization Method | INT4-AWQ |
| Precision | 4-bit weights |
| KV Cache | int8 |
| KV Cache Type | paged |
| KV Reuse | enabled |
| Block/Group Size | 128 |
| TensorRT-LLM Version | 1.2.0rc5 (used for quantization) |
| Max Batch Size | 64 |
| Max Input Length | 5525 |
| Max Output Length | 150 |
| SM Architecture | sm90 |
| GPU | NVIDIA H100 NVL |
| CUDA Toolkit | 13.0 |
| Generated | 2026-01-03 18:54:59 UTC |
trt-llm/
checkpoints/
*.safetensors
config.json
engines/sm90_trt-llm-1.2.0rc5_cuda13.0/
rank*.engine
config.json
| Parameter | Value |
|---|---|
| Method | INT4-AWQ |
| Calibration Size | 64 samples |
| Calibration Seq Length | 5675 |
| AWQ Block Size | 128 |
| Calibration Batch Size | 16 |
1.2.0rc5engines/ subdirectoriesThis quantized model inherits the license from the original base model: other
See the original model's license for full terms.