Downloads · 30 days
32
3% of all-time downloads
Cydonia-24B-v4.3-W4A16-GPTQ
tacodevs/Cydonia-24B-v4.3-W4A16-GPTQ
Cydonia-24B-v4.3-W4A16-GPTQ is a text generation model from tacodevs. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
W4A16 GPTQ quantization of TheDrummer/Cydonia-24B-v4.3, made with llm-compressor.
All-time downloads
1K
Public
Parameters
4.3B
14.2 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors14.1 GB · 100%
Parameter types
How the weights are stored.
I3222.2B · 94%
Try a prompt
Base models
Model card
From the Hugging Face model README
Cydonia-24B-v4.3-W4A16-GPTQ
W4A16 GPTQ quantization of TheDrummer/Cydonia-24B-v4.3, made with llm-compressor.
Quantization Details
- Method: GPTQ (W4A16 — 4-bit weights, 16-bit activations)
- Tool: llm-compressor (vLLM official)
- Group size: 128
- Calibration: 512 samples from ultrachat_200k, 2048 seq length
- Model size: ~14 GB (vs ~48 GB FP16)
- Marlin kernel compatible: Yes (auto-detected by vLLM)
Quality Metrics
| Metric | Value | Threshold |
|---|---|---|
| Perplexity | 3.78 | < 8.0 |
| Repetition rate | 0/50 (0%) | < 10% |
| Unique token ratio | 0.794 | > 0.6 |
Usage
# vLLM (recommended — uses Marlin kernels for fast inference)
vllm serve Irvollo/Cydonia-24B-v4.3-W4A16-GPTQ --dtype auto --max-model-len 65536 --gpu-memory-utilization 0.95
# With priority scheduling
vllm serve Irvollo/Cydonia-24B-v4.3-W4A16-GPTQ --dtype auto --max-model-len 65536 --scheduling-policy priority --enable-prefix-caching --enable-chunked-prefill --kv-cache-dtype fp8_e4m3
Original Model
Cydonia v4.3 by TheDrummer — a Mistral Small 3.1 24B fine-tune optimized for roleplay and creative writing.
Hardware Requirements
- Minimum: 16 GB VRAM (with short context)
- Recommended: 24 GB VRAM (RTX 4090, A100) for 64k context support
- Task
- Text Generation
- Type
- mistral
- License
- apache-2.0
- Created
- Mar 19, 2026
- Updated
- Mar 19, 2026