Downloads · 30 days
377
2% of all-time downloads
nerkyor/Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF
Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF is a text generation model from nerkyor. Use it when you need the model to write or continue text. It is set up for llama.cpp. The card lists the license as apache-2.0.
Downloads · 30 days
377
2% of all-time downloads
All-time downloads
20.6K
Public
Repo size
45.2 GB
Likes
20
Public
Click a slice to open those files.
.gguf45.2 GB · 100%
From the Hugging Face model README

LCB note: LCB is the standard abbreviation for LiveCodeBench. It selects competitive-programming problems published after the evaluated model's training-data cutoff date and judges code correctness by executing test cases, helping reduce training-data contamination. The LCB100 reported on this card is a fixed 100-problem coding evaluation under the same protocol.
This repository contains the 44G LynnStyle layer-wise mixed-precision GGUF of Qwen3.5-122B-A10B. The main GGUF already includes this training branch's step64 LoRA update, so no separate LoRA adapter is required. An optional multimodal mmproj is included.
The repository does not ship an MTP sidecar for two practical reasons: the main model, KV cache, and runtime overhead need to fit reliably on one 48GB GPU, and controlled testing found that quantizing the MTP sidecar reduced output quality. All reported scores and recommended commands therefore use NO MTP.
At a glance
- Main GGUF: 44.61 GB (41.55 GiB); main model plus mmproj: 45.23 GB (42.13 GiB).
- This 44G mixed-precision scheme keeps critical pathways at high precision while letting the largest ordinary expert layers absorb most of the compression.
- Put plainly: components that strongly affect understanding, reasoning, and output convergence are compressed conservatively; the numerous, space-heavy ordinary routed-expert weights carry the aggressive compression.
- Final NO-MTP evaluation: MMLU 91.6% (458/500), GPQA 86.4% (171/198), and LCB 82.0% (82/100).
- Verified runtime:
C4 = 4 × 32K, 131072 total context, Q4 K/V cache, full GPU offload,--fit off; peak VRAM was 45,551 MiB.
| File | Role | Size | SHA256 |
|---|---|---|---|
Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF.gguf | Main model with the step64 LoRA update merged | 44,612,547,968 bytes | 6a25dfc683de6754ffd632141a9a454f5f1daa0ed34fe62c5192725fdb0b4910 |
Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF-mmproj-Q8_0.gguf | Optional vision projector | 619,212,000 bytes | e77a1be433e92791e9ee85f99963dad6421aa8188314d9a6670bb9ab196034eb |
Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF.SHA256SUMS | File-integrity checks | — | — |
Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF.manifest.json | Build-time manifest | — | — |
The mmproj filename keeps the Q8_0 tier label, while its actual conversion contains Q8_0 weights with F16 fallbacks and small F32 structures. The main model can be used on its own for text inference; add --mmproj only when processing visual input.
qwen3next; the main GGUF contains 807 tensors.GPT5.6Sol-SFT in the repository name identifies this model's training branch.--lora.--model-draft.| GGUF type | Count |
|---|---|
| F32 | 313 |
| IQ2_XS | 126 |
| Q5_K | 20 |
| Q6_K | 108 |
| Q8_0 | 240 |
| Total | 807 |
| Protection group | Layers / tensor family | Actual type | Count | Purpose |
|---|---|---|---|---|
| Input/output anchors | token_embd.weight, output.weight | Q5_K | 2 | Preserve vocabulary entry and final token selection |
| Shared-expert main matrices | All ffn_*_shexp.weight tensors | Q8_0 | 192 | Preserve the shared path traversed by every token |
| Full attention | Every present attn_q/k/v/output.weight tensor | Q8_0 | 48 | Preserve full-attention reads, writes, and long-range integration |
| SSM main matrices | All ssm_ba/in/out.weight tensors | Q6_K | 108 | Preserve the linear-attention/state-space pathway |
| Routed experts: high protection | Gate/up/down on L26 and L43–L47 | Q5_K | 18 | Protect mid-to-late hot layers and output convergence |
| Routed experts: compression group | Gate/up/down on the remaining 42 layers | IQ2_XS | 126 | Carry most of the size reduction |
| Router, norms, biases, and other small structures | Remaining structures | F32 | 313 | Avoid low-bit damage to small, high-leverage controls |
The protection unit is layer × path × tensor family. Standard GGUF stores the routed experts of one layer in a stacked tensor, so this build protects entire layer-level expert tensors rather than assigning a separate precision to each expert inside one tensor.
REAP/router observations identify task pathways, while the imatrix constrains quantization error. Neither replaces final evaluation. L26 and L43–L47 were preregistered as the high-protection set during the build; this does not claim that any single layer independently determines model quality. Release acceptance comes from the complete MMLU500, GPQA198, LCB100, and real-load gates.
q4_0 K/V cache. This is runtime context storage and does not change the model-file weights.The main GGUF is 65.65% smaller than the 129,871,496,512-byte original Q8 and 82.17% smaller than the 250,196,479,366-byte BF16 source.
The fixed evaluation set contains MMLU500, GPQA198, and LCB100. Generation used temperature 0.6, top_p=0.95, and at most 32768 output tokens per slot. The 44G evaluation used C4, 32K per slot, Q4 K/V cache, and NO MTP; MMLU was 5-shot.
| Version | MMLU500 | GPQA198 | LCB100 | Notes |
|---|---|---|---|---|
| Original Q8 baseline | 93.4% (467/500) | 89.4% (177/198) | 78.0% (78/100) | Q8 GPQA was rejudged from the saved 198 rows using the final standalone answer; Q8 LCB is natural 32K |
| 44G, natural 32K | 91.6% (458/500) | 85.4% (169/198) | 81.0% (81/100) | GPQA had 2 length IDs; LCB had 10 length IDs |
| 44G, final corrected gate | 91.6% (458/500) | 86.4% (171/198) | 82.0% (82/100) | Each natural-32K length ID was independently retested once at 36K and replaced by its actual result |
| 44G final vs. Q8 | -1.8 pp | -3.0 pp | +4.0 pp* | This is not a quantization-only ablation |
* The Q8 LCB baseline did not receive a symmetric 36K length-only retest. Under the same natural-32K condition, 44G scored 81.0% (81/100) versus Q8's 78.0% (78/100), a +3.0 pp difference.
This comparison is not a pure “change only the quantization bits” A/B test. The Q8 artifact is the original quality baseline, while the 44G main model also includes the step64 LoRA update. The measured differences therefore combine the training update and mixed quantization.
reasoning_content to the final answer before parsing. Rejudging the same saved 198 rows by the final standalone A/B/C/D choice gives 89.4% (177/198); all 198 rows ended with stop, and parse/error/timeout counts were zero.stop and were correct, producing 86.4% (171/198).The final corrected gate passed. Peak VRAM was 45,551 MiB, below the 49,152 MiB limit.
from huggingface_hub import snapshot_download
model_dir = snapshot_download(
repo_id="nerkyor/Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF"
)
print(model_dir)
Or:
git lfs install
git clone https://huggingface.co/nerkyor/Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF
cd Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF
sha256sum -c Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF.SHA256SUMS
Use a recent llama.cpp build with qwen3next and the corresponding multimodal projector support.
llama-server \
-m Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF.gguf \
-ngl all --fit off \
-c 32768 -np 1 -n 32768 \
--cache-type-k q4_0 --cache-type-v q4_0 \
--host 0.0.0.0 --port 8080 \
--metrics --no-webui --cache-ram 0
llama-server \
-m Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF.gguf \
--mmproj Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF-mmproj-Q8_0.gguf \
-ngl all --fit off \
-c 131072 -np 4 -n 32768 \
--cache-type-k q4_0 --cache-type-v q4_0 \
--host 0.0.0.0 --port 8080 \
--metrics --no-webui --cache-ram 0
Do not add --model-draft: this repository does not publish MTP, and its quality scores were not produced with MTP.

LCB 说明: LCB 是 LiveCodeBench 的标准缩写。它按照被测模型训练数据的截止日期,选取该日期之后发布的竞赛编程题,并通过执行测试用例判定代码正确性,以降低训练数据污染。本卡的 LCB100 为固定 100 题同口径编码测试。
这是 Qwen3.5-122B-A10B 的 44G LynnStyle MoE 分层混合精度 GGUF。主模型已合入本训练分支的 step64 LoRA 更新,下载后不需要再挂载独立 LoRA;仓库另附可选的视觉 mmproj。为了让主模型、运行时缓存和必要开销稳定装进单张 48GB 显卡,本仓不附带 MTP sidecar;同时,受控测试发现量化后的 MTP 会降低输出质量,因此公开质量分数与推荐启动方式全部采用 NO MTP。
先看结论
- 主 GGUF:44.61 GB(41.55 GiB);主模型加 mmproj:45.23 GB(42.13 GiB)。
- 这份44G的混合精度方案:关键通路保留高精度,体积最大的普通专家层承担主要压缩。
- 换句话说,容易影响理解、推理和输出收口的部分尽量少压;数量最多、最占空间的普通专家权重压得更狠。模型推理本身仍使用 FP16 / FP32,Q4 只用于运行时的上下文缓存。
- NO-MTP 全量终局:MMLU 91.6%(458/500)、GPQA 86.4%(171/198)、LCB 82.0%(82/100)。
- 已验证
C4 = 4 × 32K、总上下文 131072、Q4 KV、全 GPU、--fit off,峰值 45,551 MiB。- ModelScope 侧栏可能把该文件标成
unknown / IMATRIX;真实类型以本卡、manifest 和 GGUF 解析为准。
| 文件 | 作用 | 大小 | SHA256 |
|---|---|---|---|
Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF.gguf | 主模型,已合入 step64 LoRA | 44,612,547,968 bytes | 6a25dfc683de6754ffd632141a9a454f5f1daa0ed34fe62c5192725fdb0b4910 |
Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF-mmproj-Q8_0.gguf | 可选视觉投影 | 619,212,000 bytes | e77a1be433e92791e9ee85f99963dad6421aa8188314d9a6670bb9ab196034eb |
Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF.SHA256SUMS | 文件校验 | — | — |
Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF.manifest.json | 构建时 manifest | — | — |
mmproj 文件名沿用 Q8_0 档名;实际转换结果为 Q8_0 + F16 fallback + F32 小结构,不是所有 tensor 都统一 Q8。主模型可独立用于文本推理;视觉输入时再添加 --mmproj。
qwen3next,主 GGUF 共 807 tensors。GPT5.6Sol-SFT 对应该模型训练分支。--lora。--model-draft。| GGUF 类型 | 数量 |
|---|---|
| F32 | 313 |
| IQ2_XS | 126 |
| Q5_K | 20 |
| Q6_K | 108 |
| Q8_0 | 240 |
| 合计 | 807 |
| 保护桶 | 层 / tensor family | 实际类型 | 数量 | 目的 |
|---|---|---|---|---|
| 输入输出锚点 | token_embd.weight、output.weight | Q5_K | 2 | 保护词表进入与最终输出 |
| shared-expert 主矩阵 | 全层 ffn_*_shexp.weight | Q8_0 | 192 | 保护每个 token 都会经过的共享路径 |
| full-attention | 全部实际存在的 attn_q/k/v/output.weight | Q8_0 | 48 | 保护全注意力读写与长程整合 |
| SSM 主矩阵 | 全部 ssm_ba/in/out.weight | Q6_K | 108 | 保护线性注意力 / 状态空间主通路 |
| routed experts:高保护 | L26、L43–L47 的 gate/up/down | Q5_K | 18 | 保护中后段热点与输出收口 |
| routed experts:压缩桶 | 其余 42 层的 gate/up/down | IQ2_XS | 126 | 承担主要体积压缩 |
| 路由、norm、bias 等小结构 | 其余未降档结构 | F32 | 313 | 避免小而高杠杆的控制结构被低比特化 |
这里的保护单位是 层 × 路径 × tensor family。标准 GGUF 会把同层 routed experts 堆叠到同一张大 tensor 中,因此本档保护的是整层专家张量,而不是在一张 tensor 内给单个 expert 指定不同精度。
REAP / 路由观测用于识别任务路径,imatrix 用于约束量化误差;二者都不能替代最终测评。L26、L43–L47 是构建时预注册的高保护集合,并不等于声称某一层单独“决定”全部能力。发布结论只由完整 MMLU500、GPQA198、LCB100 和真实加载门禁给出。
q4_0 K/V cache。它只影响运行时上下文缓存,不会改变模型文件里的权重。主 GGUF 相比 129,871,496,512-byte 原始 Q8 缩小 65.65%,相比 250,196,479,366-byte BF16 缩小 82.17%。
主评测使用固定 MMLU500 / GPQA198 / LCB100 资产;生成温度 0.6、top_p=0.95、每路最大输出 32768 tokens。44G 使用 C4、每路 32K、Q4 KV、NO MTP;MMLU 为 5-shot。所有百分比均在前,绝对值附后。
| 版本 | MMLU500 | GPQA198 | LCB100 | 说明 |
|---|---|---|---|---|
| 原始 Q8 基线 | 93.4%(467/500) | 89.4%(177/198) | 78.0%(78/100) | Q8 GPQA 对同一 198 条保存结果做尾部最终选项重判;LCB 为自然 32K |
| 44G,自然 32K | 91.6%(458/500) | 85.4%(169/198) | 81.0%(81/100) | GPQA 有 2 个 length ID;LCB 有 10 个 length ID |
| 44G,最终修正门禁 | 91.6%(458/500) | 86.4%(171/198) | 82.0%(82/100) | 只对自然 32K 的 length ID 各做一次独立 36K 补测并替换原题 |
| 44G 最终 vs Q8 | -1.8pp | -3.0pp | +4.0pp* | 不是纯量化单变量 A/B,见下方说明 |
* Q8 LCB 没有做对称的 36K length-only 补测。完全同为自然 32K 时,44G 是 81.0%(81/100),相对 Q8 的 78.0%(78/100) 为 +3.0pp。
这张表不是“只改量化位宽”的纯 A/B:Q8 是原始质量基线,而 44G 主模型已经包含 step64 LoRA 更新。因此差值同时反映训练更新与混合量化,不能全部归因于量化。
reasoning_content 拼在最终答案前而错误统计为 50.0%(99/198)。同一份 198 条 JSONL 采用“最后一个独立 A/B/C/D”规则重判后为 89.4%(177/198);198/198 均为 stop,parse/error/timeout 为 0。最终 corrected gate 为 PASS;运行峰值 45,551 MiB,低于 49,152 MiB 门槛。
from huggingface_hub import snapshot_download
model_dir = snapshot_download(
repo_id="nerkyor/Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF"
)
print(model_dir)
或:
git lfs install
git clone https://huggingface.co/nerkyor/Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF
cd Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF
sha256sum -c Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF.SHA256SUMS
llama-server \
-m Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF.gguf \
-ngl all --fit off \
-c 32768 -np 1 -n 32768 \
--cache-type-k q4_0 --cache-type-v q4_0 \
--host 0.0.0.0 --port 8080 \
--metrics --no-webui --cache-ram 0
llama-server \
-m Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF.gguf \
--mmproj Qwen3.5-122B-A10B-44GB-GPT5.6Sol-SFT-LynnStyle-GGUF-mmproj-Q8_0.gguf \
-ngl all --fit off \
-c 131072 -np 4 -n 32768 \
--cache-type-k q4_0 --cache-type-v q4_0 \
--host 0.0.0.0 --port 8080 \
--metrics --no-webui --cache-ram 0
请使用支持 qwen3next 与相应 multimodal projector 的较新 llama.cpp。不要添加 --model-draft:本仓没有发布 MTP,质量分数也不是 MTP 成绩。
mmproj 已通过 GGUF 结构与文件审计;本卡没有发布独立视觉质量基准,不应把文本分数解释成视觉分数。unknown / IMATRIX 当成损坏。