Downloads · 30 days
286
48% of all-time downloads
lkjiop8/Yuanl-27B-v5-7-GGUF
Yuanl-27B-v5-7-GGUF is a machine learning model from lkjiop8. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
MTP-aware GGUF builds of lkjiop8/Yuanl-27B-v5-7 applied on unsloth/Qwen3.6-27B, converted with llama.cpp b9180 converthftogguf.py. The blk.64.nextn. MTP speculative-decode head is present and verified.
Downloads · 30 days
286
48% of all-time downloads
All-time downloads
596
Public
Repo size
45.9 GB
Likes
0
Public
Click a slice to open those files.
.gguf45.9 GB · 100%
From the Hugging Face model README
MTP-aware GGUF builds of lkjiop8/Yuanl-27B-v5-7 applied on unsloth/Qwen3.6-27B, converted with llama.cpp b9180 convert_hf_to_gguf.py. The blk.64.nextn.* MTP speculative-decode head is present and verified.
| File | Size | Notes |
|---|---|---|
Yuanl-27B-v5-7-MTP-Q8_0.gguf | ~29 GB | Near-lossless (8.50 BPW) |
Yuanl-27B-v5-7-MTP-Q4_K_M.gguf | ~17 GB | 4.92 BPW; fits 24 GB VRAM w/ Q8 KV cache @ ~32K |
The unsloth training/merge path strips the MTP block (unsloth_fixed_mtp=True), so the trained adapter and its merged model contain trunk weights only. To produce a usable --spec-type draft-mtp build, the 15 mtp.* tensors were re-injected from the original unsloth/Qwen3.6-27B before GGUF conversion, then bundled into blk.64.nextn.*.
Consequences:
--metrics. No acceptance-rate claim is made.b9180+ (Qwen 3.6 MTP support)../llama-server \
-m Yuanl-27B-v5-7-MTP-Q8_0.gguf \
--alias Yuanl-27B-v5-7 \
--host 0.0.0.0 --port 8080 \
-c 122880 --parallel 1 \
-ngl 99 -sm layer -ts 23,25 -fa on \
-b 4096 -ub 2048 \
-t 8 -tb 16 --threads-http 8 \
-ctk q8_0 -ctv q8_0 \
--spec-type draft-mtp --spec-draft-n-max 2 \
--cache-reuse 256 --kv-unified \
--jinja --reasoning auto --reasoning-format deepseek \
--reasoning-budget 256 \
--temp 0.3 --top-p 0.85 --top-k 20 --min-p 0.05 \
--repeat-penalty 1.05 --repeat-last-n 256 --presence-penalty 0.10 \
--no-mmproj --no-webui --metrics \
--slot-save-path ./slots
Designed for authorized red-team / research / academic use. The operator carries the legal and ethical authorization.