Downloads · 30 days
268
15% of all-time downloads
hotdogs/Qwen3.8-27B-abliterated-cyber-preview
Qwen3.8-27B-abliterated-cyber-preview is a image-text-to-text model from hotdogs. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
An offensive-security / agentic tool-calling LoRA merge on top of hotdogs/Qwen3.8-27B-abliterated — the training-free abliterated build of Qwen/Qwen3.8-27B. The LoRA is merged into the base at scale 1.0 (PEFT alpha/r…
Downloads · 30 days
268
15% of all-time downloads
All-time downloads
1.8K
Public
Parameters
27.4B
56.5 GB on disk
Likes
5
Public
Click a slice to open those files.
.safetensors56.5 GB · 100%
From the Hugging Face model README
An offensive-security / agentic tool-calling LoRA merge on top of
hotdogs/Qwen3.8-27B-abliterated
— the training-free abliterated build of
Qwen/Qwen3.8-27B. The LoRA is
merged into the base at scale 1.0 (PEFT alpha/r = 64/32 = 2.0, i.e.
the exact strength the scale sweep below found optimal), and the MTP
(Multi-Token Prediction) head is preserved, so the GGUF build supports
self-speculative decoding.
This is a preview: the LoRA was trained on an agentic dataset that mixes penetration-testing tool calls (
<tool_call><function=...>XML) with security QA. This model will not refuse. It is published for authorized security research, red-team exercises, and studying LLM-driven tool use — pentesting only systems you own or have explicit permission to test. You are responsible for your use of it and for complying with all applicable laws. Inherits the Apache-2.0 license of the base.
All numbers below were measured on our own reproduction pipeline (see Method and Evaluation). The merged model adds a reliable, correctly formatted tool-call pathway while leaving general capability essentially unchanged.
| metric | base (no LoRA) | this repo (merged @ scale 1.0) |
|---|---|---|
| Tool-call format emitted (6 pentest prompts) | 0/6 · 0 % | 6/6 · 100 % |
| Correct real-tool selection (same 6 prompts) | 0/6 · 0 % | 6/6 · 100 % (nmap, ffuf, masscan, sqlmap, wpscan, smbclient) |
| General capability (7 QA/math/code prompts) | 6/7 · 86 % | 7/7 · 100 % |
| KL divergence (base ‖ merged), base-prompts | — | 0.041 (essentially zero) |
| KL divergence (base ‖ merged), tool-prompts | — | 0.808 (the intended re-target) |
| MTP draft acceptance rate | — | 0.77 (51/66) |
Reading the numbers: the merge shifts the model's next-token distribution by a tiny amount on general prompts (KL ≈ 0.04 → base knowledge is preserved), while shifting it ~20× more on security/tool prompts (KL ≈ 0.81 → the intended behavioral re-target to emit tool calls). The scale sweep and MTP numbers are detailed below.
The base model already refuses nothing (abliterated). This repo adds a tool-calling capability: given a pentest scenario, the model now emits a structured, well-formed agentic call instead of free-form text.
User: Port scan the host 203.0.113.10 and identify which services are exposed.
<tool_call>
<function=nmap>
<parameter=target>
203.0.113.10
</parameter>
<parameter=ports>
-top 1000
</parameter>
</function>
</tool_call>
The tool-call format follows the Qwen3.5 native <tool_call>/<function>/<parameter>
schema (also exercised through the model's chat template). At scale 1.0 the
model picked the correct real tool per scenario in 6/6 cases.
hotdogs/Qwen3.8-27B-abliterated (λ = 1.2), a 27B native vision-language
hybrid (full-attention + GDN linear-attention) with a frozen MTP head.
hotdogs/cyber-sft-agent-qwen38 — 8,400 rows
(train 7,140 / valid 840 / test 420). 30 % tool-call, 30 % multi-turn,
think-tags 100 % balanced, 22 tools (nmap, sqlmap, metasploit, hydra,
crackmapexec, linpeas, wpscan, whatweb, ffuf, gobuster, …), 1,995 unique IP
contexts, 0 duplicates.r=32, alpha=64, dropout=0, ctx=8192, LR=1e-4 (cosine),
BF16 (required by GDN hybrid), batch 2 × grad-accum 2 (effective 4),
2 epochs. Trainable 233M (0.85 %).q/k/v/o/gate/up/down + GDN
in_proj_qkv/out_proj/in_proj_z/a/b.merge_and_unload() at
default scale (alpha/r = 2.0, ≡ llama.cpp --lora-scaled :1.0).merge_and_unload() + save_pretrained()
drops the mtp.* tensors, so the 15 MTP tensors (849 MB) were copied back
from the base into the merged safetensors (the LoRA never touched mtp.*,
so these weights are byte-correct). Final model = 1,199 tensors, verified
loading via unsloth, MTP physically present.convert_hf_to_gguf.py --outtype bf16 (MTP kept in-file), 54.6 GB.
Serves with --spec-type draft-mtp.| metric | value |
|---|---|
| valid token loss (840 rows) | 0.480 |
| train token loss (300-row sample) | 0.384 |
| gap (valid − train) | 0.095 |
| valid % loss < 0.2 | 30.0 % |
| train % loss < 0.2 | 27.3 % |
A small gap (≤ 0.15) means the model generalizes rather than memorizes. The training loss plateau (~0.44 median, only ~3.8 % of points near zero) is a healthy learning floor for a diverse 8.4K dataset, not overfitting.
--lora-scaled, no merge needed)6 tool prompts + 7 base prompts, temperature 0, identical config across scales:
| scale | tool-call format (6) | correct real tool (6) | base capability (7) |
|---|---|---|---|
| baseline (no LoRA) | 0/6 | 0/6 | 6/7 |
| 0.25 | 1/6 | 0/6 | 7/7 |
| 0.5 | 6/6 | 2/6 | 7/7 |
| 0.75 | 6/6 | 4/6 | 7/7 |
| 1.0 | 6/6 | 6/6 | 7/7 |
| 1.5 | 6/6 | 6/6 | 7/7 |
Scale 1.0 is the sweet spot: every tool prompt emits a well-formed
<tool_call> and picks the correct real tool, while base capability stays
at 7/7. Damping below 1.0 hurts tool selection (hallucinated tools like
web_fetch, wp_recon); going above ~2.0 degrades parameter quality (e.g.
ports="-s -p 1-65535"). The merge therefore uses the default
alpha/r = 2.0, which corresponds to this optimal 1.0 point.
Measured on a shared continuation that the merged model sampled, comparing
top-25 next-token log-probs from the base vs the merged GGUF (both via
llama.cpp llama-server):
| prompt | KL (base ‖ merged) |
|---|---|
| 17 × 43 | 0.0066 |
| Python reverse-string | 0.1559 |
| Capital of France | 0.0206 |
| stack vs queue | 0.0556 |
| 15 % of 200 | 0.0030 |
| boiling point | 0.0041 |
| mean (base prompts) | 0.0410 |
| port-scan tool prompt | 0.7332 |
| SQLi tool prompt | 0.8833 |
| mean (tool prompts) | 0.8083 |
| overall mean | 0.2328 |
The ~20× gap between base-prompts (0.04) and tool-prompts (0.81) is the quantitative signature of a surgically targeted LoRA: it re-wires tool-calling behavior without disturbing general knowledge. This matches the abliterated base's own near-zero first-token KL (0.0001) — the merge adds capability, not drift.
| metric | value |
|---|---|
| MTP draft acceptance rate | 0.77 (51 / 66) |
| mean draft length | 2.55 |
| head location | in-file (bf16 GGUF) |
Well above the ~0.3 threshold where speculative decoding pays off — MTP is active and beneficial.
| file | purpose |
|---|---|
model-00001-of-00002.safetensors / model-00002-of-00002.safetensors | merged weights (MTP in shard 2) |
config.json | Qwen3_5ForConditionalGeneration |
chat_template.jinja / tokenizer.json / tokenizer_config.json / processor_config.json | Qwen3.5 chat + vision template |
This is a preview built for authorized security research and red-teaming. It will not refuse and may emit instructions for exploiting systems. Use only on systems you own or are explicitly authorized to test. Not for misuse/harmful activity. Apache-2.0, inherited from the base.