Downloads · 30 days
94
9% of all-time downloads
Huihui-Qwen35-A35B-Ablit-Small-TQ3_4S
YTan2000/Huihui-Qwen35-A35B-Ablit-Small-TQ3_4S
Huihui-Qwen35-A35B-Ablit-Small-TQ3_4S is a image-text-to-text model from YTan2000. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
All-time downloads
1K
Public
Repo size
14.2 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.gguf14.2 GB · 100%
At a glance
- Task
- Image-Text-to-Text
- License
- apache-2.0
- Access
- Public
- Created
- Apr 16, 2026
- Updated
- Apr 16, 2026
- SHA
- ffd6167f
Try a prompt
Base models
Model card
From the Hugging Face model README
Huihui-Qwen35-A35B-Ablit-Small-TQ3_4S
GGUF quantization of huihui-ai/Huihui-Qwen3.5-35B-A3B-abliterated using TQ3_4S with mixed-precision MoE compression — 2-bit experts, 4-bit attention. Abliterated (uncensored) model.
Files
| File | Description |
|---|---|
Huihui-Qwen35-A35B-Ablit-Small-TQ3_4S.gguf | Main model (12.4 GiB, 3.07 BPW) |
mmproj-Qwen35-A35B-f16.gguf | Multimodal projector (F16) |
Quantization
| Component | Quant | Rationale |
|---|---|---|
| Expert MLP gate/up | Q2_K | 98% of params, MoE-tolerant |
| Expert MLP down | Q3_K | Write-back sensitivity |
| Attention Q/K/V/O | TQ3_4S | WHT-protected |
| Embeddings + output | Q6_K | Quality anchor |
Runtime Requirement
This model requires the public TurboQuant runtime fork:
Recommended Settings (16GB VRAM)
./build/bin/llama-server \
-m Huihui-Qwen35-A35B-Ablit-Small-TQ3_4S.gguf \
-ngl 99 -c 16384 -np 1 \
-ctk q4_0 -ctv tq3_0 -fa on \
--jinja \
--reasoning off --reasoning-budget 0 --reasoning-format deepseek
Performance (RTX 5060 Ti 16GB)
| Metric | Value |
|---|---|
| PP512 | 1882 tok/s |
| TG128 | 98.7 tok/s |
| Size | 12.4 GiB |
| BPW | 3.07 |
| ngl | 99 (full GPU) |
Quality
10/10 correct on standard QA benchmark.
Safety Notice
This is an abliterated/uncensored model. Use responsibly.
Base Model
License
Apache 2.0 — same as the base model.
- Task
- Image-Text-to-Text
- License
- apache-2.0
- Languages
- en
- Created
- Apr 16, 2026
- Updated
- Apr 16, 2026