nerkyor-Qwen3.8-27B-EfficientThink-SimPO-Ninfer

nerkyor-Qwen3.8-27B-EfficientThink-SimPO · NInfer Artifacts

中文说明见下方 · Converted .ninfer v3 engine artifacts of EfficientThink by Lynn (nerkyor)

Credits: All weights and model capabilities belong to Lynn (nerkyor), who open-sourced the EfficientThink post-training series (Qwen3.8-27B · SFT → SimPO) together with the BF16 / W4A4-NVFP4 / DFlash2 / MTP components used here. This repository is purely a packaging conversion for the NInfer inference engine. Base model: Qwen3.8-27B (Apache-2.0).

What is this

Three files, each a self-contained .ninfer v3 container (NInfer's native single-file format — not GGUF / not safetensors; it does not work with llama.cpp, vLLM or SGLang):

All three include the vision tower (27-layer, run with --vision) and support the native 262,144-token context. Source files were SHA256-verified against the author's published checksums before conversion; conversion was performed with NInfer's official converter (Python, CPU).

⚠️ An apology, and a known limitation. A miscalculation on our side during weight conversion quantized the DFlash2 selector head of qwen3.8-27b-lynn-nf4.ninfer to NVFP4 as well. NInfer's linear_topk does not accept that head format, so DFlash2 acceleration is unavailable on Blackwell with this artifact: --spec dflash2 is refused at startup — it never silently computes a wrong selector. Fortunately, MTP itself is in practice faster than DFlash2 on high-SM GPUs such as the RTX 5090: MTP is the model's own native head, its draft-side overhead is far below DFlash2's separate draft layers plus selector, and its acceptance rate is both higher and steadier. We therefore do not plan to re-convert and re-release for this one tensor. Use --spec mtp with this artifact. If you want DFlash2, take the df2 artifact above — it is a different (groupwise Q4/Q5) encoding and runs on both Ada and Blackwell.

How to pick

• RTX 40 series (Ada) → df2 (author's draft head, ~66–67% acceptance reported) or mtp (smaller file; tends to scale better with SM count).
• RTX 50 series (Blackwell) → any of the three; nf4 uses the FP4 fast path, so run it with --spec mtp — its DFlash2 head is rejected on Blackwell (see the note above).

Run

Engine: upstream NInfer (sm_120a), or our community build QilinWan/Ninfer-Blackwell-Ada, which adds sm_89 (RTX 40 series) support on top of the official v3 codebase — all three artifacts in this repository have passed loading and functional tests on it. Please follow that repository's README for build instructions.

ninfer run qwen3.8-27b-lynn-df2.ninfer --spec dflash2 --vision
ninfer run qwen3.8-27b-lynn-mtp.ninfer --spec mtp --vision
ninfer run qwen3.8-27b-lynn-nf4.ninfer --spec mtp --vision      # DFlash2 is rejected on Blackwell here

Download

# grab only what you need:
hf download QilinWan/nerkyor-Qwen3.8-27B-EfficientThink-SimPO-Ninfer \
  --include "qwen3.8-27b-lynn-df2.ninfer" --local-dir ./ninfer-models

Mainland-China users may prefer the ModelScope mirrors (identical files, verified): DF2 · MTP · NF4

Checksums

qwen3.8-27b-lynn-df2.ninfer  19,986,249,216 B
  sha256 d6d1425c23ca3ea2f93d898d247c4df523199543c3eb6462176c0b03f62754f4
qwen3.8-27b-lynn-mtp.ninfer  18,210,703,360 B
  sha256 7970ac0f6263f666ff21bc0ec891ec78976a2fbbc33281aefce6b147ebe73958
qwen3.8-27b-lynn-nf4.ninfer  19,345,900,804 B
  sha256 faf551e2c782022606e0ffd382e8d015cec574914a9476ca6007f346c4c53268

Reproduce

Conversion commands (official converter + two small bridge overrides: tensor-level-FP8 DFlash2 head → Q8 requant; ModelOpt NVFP4 naming → compressed-tensors conventions) are documented in the Chinese section's ModelScope repos. Published under Apache-2.0, same as upstream.

---

中文说明

NInfer 引擎专用格式（.ninfer v3），权重来自 Lynn 老师（HF: nerkyor）的 EfficientThink 后训练模型（Qwen3.8-27B · SFT → SimPO · Uncensored），由我本地用 NInfer 官方转换工具转出， 源权重全部通过作者仓库 SHA256SUMS 校验后转换。

致敬与感谢：感谢 Lynn（nerkyor）老师开放高质量的 EfficientThink 权重、完整的 BF16/NVFP4/DFlash2/MTP 组件与详尽的评测记录。本仓库仅是"给 NInfer 引擎换一种封装格式"的 搬运转换，模型能力与贡献全部归属原作者。底座为 Qwen3.8-27B（Apache-2.0）。

三个文件怎么选

三个容器均内含 27 层视觉编码器（--vision 启用）与 262K 原生上下文；未选中的组件不占显存。

⚠️ 需要致歉的一处问题。 因为我们在转换权重时的计算疏忽，qwen3.8-27b-lynn-nf4.ninfer 的 DFlash2 selector 头被一并量化成了 NVFP4；引擎的 linear_topk 不接受这种头格式，因此该产物在 Blackwell 架构下 DFlash2 加速不可用——--spec dflash2 会在启动时被直接拒绝，不会静默算错。 好在实测表明，在 5090 这类算力充裕的 GPU 上 MTP 本身就比 DFlash2 更快：MTP 是模型自带的原生头， 草稿侧开销远小于 DFlash2 的独立草稿层 + selector，接受率也更高更稳。因此我们不打算为这一个张量 重转权重、重发产物，请在该产物上使用 --spec mtp。要用 DFlash2 请取上面的 df2 版—— 那是另一套 groupwise Q4/Q5 编码，Ada 与 Blackwell 都能跑。

运行引擎

官方 NInfer 引擎仅支持 Blackwell（sm_120a）。RTX 40 系（sm_89）可使用我们维护的社区构建 QilinWan/Ninfer-Blackwell-Ada—— 在官方 v3 代码基础上增加了 sm_89 支持，本仓库三个文件均已通过其加载与功能测试； 构建方法见该仓库 README。

适用与不适用

• ✅ RTX 4080 SUPER 32GB / 4090 / 4090D（sm_89，用上述社区构建）；RTX 5090 / RTX PRO 6000（sm_120a，官方或社区构建均可）
• ❌ 不用于 llama.cpp / vLLM / SGLang——这是 NInfer 引擎专有格式
• 速度参考（同代引擎、官方底模的社区实测）：4080S DFlash2 档 ~157–160 t/s；4090D MTP 档 ~195–203 t/s。微调后模型的实际接受率与速度可能略有出入。

国内用户可从魔搭镜像下载（同一批文件，服务端哈希已验证）： DF2 · MTP · NF4

许可

遵循上游：EfficientThink 权重与 Qwen3.8-27B 均为 Apache-2.0；本转换产物同样以 Apache-2.0 发布。