Downloads · 30 days
43
8% of all-time downloads
kacperwikiel/slayer-style-qwen3.5-27b-ep3-GGUF
slayer-style-qwen3.5-27b-ep3-GGUF is a text generation model from kacperwikiel. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
Polish‑style fine‑tune of Qwen/Qwen3.5‑27B. A LoRA trained to write natural, idiomatic Polish (no translationese, no em‑dash/półpauza overuse) while keeping the base model's knowledge. Quantized to Q4KM GGUF so it run…
Downloads · 30 days
43
8% of all-time downloads
All-time downloads
536
Public
Repo size
33.1 GB
Likes
2
Public
Click a slice to open those files.
.gguf16.5 GB · 100%
From the Hugging Face model README
Polish‑style fine‑tune of Qwen/Qwen3.5‑27B. A LoRA trained to write natural, idiomatic Polish (no translationese, no em‑dash/półpauza overuse) while keeping the base model's knowledge. Quantized to Q4_K_M GGUF so it runs on a single 24 GB GPU.
Open weights end‑to‑end — not distilled from Anthropic/OpenAI. No Claude/GPT weights or outputs are used as training data (see How the training data was made).
📘 Full playbook (data → training → quantization → deployment, with pitfalls):
slayer_playbook.pdf in this repo.
TL;DR — it beats Bielik. On the LLMzSzŁ Polish professional‑exam MCQ benchmark (answer log‑likelihood, n=400, seed=42, identical harness):
Model Accuracy slayer‑style‑qwen3.5‑27b (this model, fp16) 65.0 % 🏆 Qwen3.5‑27B (base) 58.5 % Bielik‑11B‑v3.0‑Instruct 56.0 % Qwen3.5‑9B (base) 50.7 % +9.0 points over Bielik‑11B‑v3, +6.5 over the base Qwen it was tuned from.
| Artifact | Repo / path |
|---|---|
| Base model | Qwen/Qwen3.5-27B (Apache‑2.0) |
| LoRA adapter | kacperwikiel/slayer-style-qwen3.5-27b-ep3 (private) |
| Quantized GGUF (this card) | kacperwikiel/slayer-style-qwen3.5-27b-ep3-GGUF (private) — slayer-style-qwen3.5-27b-Q4_K_M.gguf, 16.5 GB |
| Merged fp16 (intermediate, not published) | produced locally by bench/merge_adapter.py |
The adapter is the source of truth; the GGUF is derived from base + adapter.
| Setting | Value |
|---|---|
| Method | QLoRA SFT (bench/sft_style_qlora.py) |
| Base | Qwen/Qwen3.5-27B |
LoRA rank r | 16 |
LoRA alpha | 32 |
| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj (all linear) |
| Task | CAUSAL_LM |
| Epochs | 3 (ep3) |
| Steps | 150 total (~50/epoch) |
| Train loss | 1.34 → 0.93 |
🛡️ Not distilled from Anthropic or OpenAI. No Claude/GPT weights or generated text are used as training data — none of the answers the model learns to imitate come from Anthropic or OpenAI models. The "chosen" answers come from DeepSeek‑V4‑pro, the raw/rejected side from Qwen3.5‑27B, and the production quality judge is the open‑weight Qwen3.5‑122B. Anthropic's opus‑4.8 was used only to validate that the open‑weight judge scores comparably (and scored an early data batch before the switch) — never as a source of any training text.

bench/gen_style_prompts_v2.py) — a grid of 26 task types × 20 domains ×
9 registers × 4 lengths, plus an 8‑type grammar stress‑test (liczebniki, wołacz, aspekt,
ortografia ż/rz, frazeologizmy, rekcja…). No benchmark items.bench/gen_style_sft.py) — per prompt: a raw answer from qwen3.5-27b
(the "rejected") and a rewrite from teacher deepseek-v4-pro (the "chosen", natural PL).Qwen3.5-122B scores natural_polish_1‑5 and flags
english_leak, dash_overuse, markdown, fact_preserved, fits_brief. (Validated once
against opus‑4.8 to confirm the open judge annotates at comparable quality.)bench/curate_style_data.py) — 1600 generated → 799 train + 160 holdout:
near‑dup removed, grid‑balanced, score = judge naturalness + length fit − AI‑tell penalties.
Holdout is never trained on. More ≠ better — curation beats volume.Concrete before → after (same prompt, "what sport for a 50+ beginner?"):
qwen3.5-27b (rejected): "Dla osoby powyżej 50. roku życia… spacery lub pływanie.
Oto dlaczego te dyscypliny są optymalne: Spacery: Są najbezpieczniejsze…" — stiff, markdown, corporate.deepseek-v4-pro (chosen): "Spacery i pływanie to dwa strzały w dziesiątkę… możesz
wyjść z domu i po prostu iść, a intensywność regulujesz tempem i dystansem." — natural, idiomatic, no markdown.Anti‑AI‑tells — explicit penalties on em‑dash / półpauza overuse, anglicisms, and
translationese (bench/make_anglicyzm_report.py, bench/make_translationese_report.py).
Hardware: Dell GB10 (aarch64, Grace‑Blackwell sm_121, 121 GB unified RAM, CUDA 13).
llama.cpp @ commit 98d5e8b (2026‑06‑06) — first to support Qwen3.5 (Qwen35MtpMixin).
# 1) Merge LoRA into the base in fp16 (CPU, ~54 GB RAM; one-time)
python bench/merge_adapter.py \
--adapter kacperwikiel/slayer-style-qwen3.5-27b-ep3 \
--out ./slayer-27b-merged # -> ~51 GB merged fp16
# 2) HF -> GGUF f16
python llama.cpp/convert_hf_to_gguf.py ./slayer-27b-merged \
--outfile slayer-27b-f16.gguf --outtype f16 # -> ~51 GB
# 3) f16 -> Q4_K_M
llama.cpp/build/bin/llama-quantize \
slayer-27b-f16.gguf \
slayer-style-qwen3.5-27b-Q4_K_M.gguf Q4_K_M # -> 16.5 GB
Quant choice rationale (24 GB target): Q4_K_M ≈ 16 GB leaves ~8 GB for KV cache — the sweet spot. Q5_K_M (~19 GB) fits but squeezes context; Q6_K (~22 GB) is too tight.
bench/bench_llmzszl_likelihood.py (lm‑eval‑style answer log‑likelihood —
each answer option scored as a continuation; highest summed log‑prob wins).amu-cai/llmzszl-dataset:llmzszl-test.jsonl (Polish professional exams).n=400, seed=42, mode=answer, normalize=none.# reproduce (fp16 merged model, GPU):
python bench/bench_llmzszl_likelihood.py --model ./slayer-27b-merged --n 400 --seed 42
Result JSONs live in results/llmzszl_likelihood_*.json.
# 1) Download the Q4_K_M GGUF (private repo -> needs your HF token)
hf download kacperwikiel/slayer-style-qwen3.5-27b-ep3-GGUF \
slayer-style-qwen3.5-27b-Q4_K_M.gguf --local-dir .
# 2) (already applied to the file in this repo) — if you ever re-convert from scratch,
# drop the unused MTP metadata or llama.cpp won't load it (missing tensor blk.64):
# gguf_set_metadata.py model.gguf qwen35.block_count 64 --force
# gguf_set_metadata.py model.gguf qwen35.nextn_predict_layers 0 --force
# 3) Serve — OpenAI-compatible API on :8080. Needs llama.cpp built with Qwen3.5
# support (>= commit 98d5e8b, 2026-06). Uses ~16.9 GB VRAM (room to spare on 24 GB).
llama-server -m slayer-style-qwen3.5-27b-Q4_K_M.gguf \
-ngl 99 -c 32768 -fa on --cache-type-k q8_0 --cache-type-v q8_0 \
--jinja --host 0.0.0.0 --port 8080
Smoke test: curl localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"x","messages":[{"role":"user","content":"Napisz zdanie o jesieni."}]}'
Ollama instead? ollama create slayer -f Modelfile with FROM ./slayer-style-qwen3.5-27b-Q4_K_M.gguf — but apply the step‑2 metadata fix to the GGUF first.
Served on the 3090 box (simp) via llama‑swap (load‑on‑demand model swapping;
only one ~20 GB model fits at a time). Config entry:
"slayer-style-27b":
aliases: ["slayer", "slayer-style-qwen3.5-27b"]
ttl: 600 # idle-evict after 10 min so other models can reclaim the GPU
cmd: |
/home/kacper/llama.cpp/build/bin/llama-server
-m /data/home-offload/models/slayer-style-27b/slayer-style-qwen3.5-27b-Q4_K_M.gguf
-ngl 999 -fa on -c 32768 --no-context-shift
--cache-type-k q8_0 --cache-type-v q8_0 --jinja --reasoning-format auto
--temp 0.7 --top-p 0.95 --top-k 20 --min-p 0.0 -np 1
--host 127.0.0.1 --port ${PORT}
OpenAI‑compatible endpoint, model id slayer-style-27b (aliases slayer). ttl: 600
idle‑evicts it so a co‑hosted default model reclaims the GPU.
Measured via llama-swap, identical engine/flags (-fa on, q8_0 KV, -c 32768), 256-token generations:
| Model | gen tok/s | prompt tok/s | params |
|---|---|---|---|
| bielik-11b-v3 | ~95 | ~1650 | 11B |
| slayer-style-27b | ~41 | ~290 | 27B |
slayer is ~2.3× slower than Bielik — in line with the 2.4× parameter count. Q4_K_M uses ~16.9 GB VRAM, leaving headroom on the 24 GB card.
Deployment gotcha (worth remembering): the convert (llama.cpp
98d5e8b) wrote Qwen3.5's MTP layer into the metadata asqwen35.block_count=65+nextn_predict_layers=1without emitting the MTP tensors, so any llama.cpp runtime fails withmissing tensor 'blk.64.attn_norm.weight'. Fix in-place (no re-quant):gguf_set_metadata.py model.gguf qwen35.block_count 64and... nextn_predict_layers 0.
Why start at 27B (and not 9B or smaller). The working belief is that sub‑27B bases just aren't good enough to be genuinely SOTA for Polish right now — so we started from the strongest practical base rather than trying to push a small model uphill. Strong opinion, weakly held: it's a starting hypothesis, not dogma — we'll happily drop to a smaller base the moment one closes the gap. The data so far backs it here: the Qwen3.5‑9B base scores 50.7 % on this benchmark (below Bielik's 56.0), while the 27B base is 58.5 % before any tuning.
Why a Qwen base (sovereignty question). Qwen3.5‑27B is Apache‑2.0; Bielik itself started from Mistral, so "built on a foreign base" cuts both ways. License here is Apache‑2.0 (inherited from the base).
Qwen's tokenizer is ~23 % more efficient on Polish than Bielik's/Mistral's (fewer
tokens per word → cheaper inference and longer effective context). We get this for free,
just by choosing the base. Measured on the same Wikipedia sample (PL+EN, ~200 paragraphs)
via bench/tokenizer_fertility.py:
| Tokenizer | vocab | TpW PL ↓ | CpT PL ↑ | TpW EN | PL/EN |
|---|---|---|---|---|---|
| Gemma‑2‑9B | 256k | 2.244 | 3.31 | 1.340 | 1.68 |
| Qwen3.5 (our base) | 248k | 2.357 | 3.15 | 1.385 | 1.70 |
| Llama‑3.1‑8B | 128k | 2.743 | 2.71 | 1.343 | 2.04 |
| Bielik‑11B‑v3 = Mistral‑7B | 32k | 3.060 | 2.43 | 1.544 | 1.98 |
TpW = tokens/word (lower = more efficient) · CpT = chars/token (higher = better) · PL/EN = how much more token‑hungry Polish is than English. Qwen 2.357 vs Bielik 3.060 ⇒ 23 % fewer tokens/word.

Last updated: 2026‑06‑09.