Downloads · 30 days
105
4% of all-time downloads
sanbanfu/Qwen3.8-27B-ShortThink-NVFP4-GGUF
Qwen3.8-27B-ShortThink-NVFP4-GGUF is a text generation model from sanbanfu. Use it when you need the model to write or continue text. It is set up for llama.cpp. The card lists the license as apache-2.0.
短思考版 Qwen3.8-27B —— 保留推理质量,压缩思考长度 Short-thinking Qwen3.8-27B — keep reasoning quality, compress thinking length
Downloads · 30 days
105
4% of all-time downloads
All-time downloads
2.4K
Public
Repo size
74.2 GB
Likes
4
Public
Click a slice to open those files.
.gguf74.2 GB · 100%
From the Hugging Face model README
短思考版 Qwen3.8-27B —— 保留推理质量,压缩思考长度 Short-thinking Qwen3.8-27B — keep reasoning quality, compress thinking length
ShortThink 是一个基于 Qwen3.8-27B 的短思考后训练版本(post-training)。
目标是解决 Qwen3.8 的一个痛点:xhigh 模式思考质量高但太长,medium 模式质量下降明显。
ShortThink is a short-thinking post-trained version of Qwen3.8-27B.
It addresses a pain point of Qwen3.8: xhigh reasoning is high-quality but verbose; medium drops quality noticeably.
MLP → NVFP4(4-bit,MSE 校准 128 条,W4A16,Blackwell FP4 张量核心加速) Attention → Q4_K / Q5_K / Q6_K / Q8_0(多档可选) lm_head / embeddings / MTP 末层 → Q8_0(保两头质量) GDN 核心(ssm_alpha/beta)→ F16(注意力质量关键)
| 文件 | 大小 | Attention | 定位 |
|---|---|---|---|
Qwen3.8-27B-ShortThink-NVFP4-MLP-Q4K-attn.gguf | 15.9 GB | Q4_K | 🚀 最小最快 |
Qwen3.8-27B-ShortThink-NVFP4-MLP-Q5K-attn.gguf | 16.6 GB | Q5_K | 速度优先 |
Qwen3.8-27B-ShortThink-NVFP4-MLP-Q6K-attn.gguf | 17.4 GB | Q6_K | ⚖️ 平衡推荐 |
Qwen3.8-27B-ShortThink-NVFP4-MLP-Q8_0-attn.gguf | 19.1 GB | Q8_0 | 质量优先 |
mmproj-F16.gguf | 0.86 GB | — | 视觉投影(多模态用) |
MLP → NVFP4 (4-bit, MSE-calibrated 128 samples, W4A16, Blackwell FP4 tensor core) Attention → Q4_K / Q5_K / Q6_K / Q8_0 (multiple tiers) lm_head / embeddings / MTP last layer → Q8_0 (preserve quality at both ends) GDN core (ssm_alpha/beta) → F16 (critical for attention quality)
# 文本推理
llama-server -m Qwen3.8-27B-ShortThink-NVFP4-MLP-Q6K-attn.gguf \
--jinja -c 16384 -ngl 99 -fa on
# 多模态推理(加视觉投影)
llama-server -m Qwen3.8-27B-ShortThink-NVFP4-MLP-Q6K-attn.gguf \
--mmproj mmproj-F16.gguf --jinja -c 16384 -ngl 99 -fa on
支持 reasoning_effort 档位(low/medium/high/xhigh),与 Qwen3.8 官方行为一致。
本项目基于 Apache-2.0 许可发布。使用 Qwen3.8-27B 请遵守其 Apache-2.0 许可。 Published under Apache-2.0. Base model Qwen3.8-27B is Apache-2.0 licensed.