Endy-Qwen3.6-CyberSec-35B-A3B-GGUF

Endy-Qwen3.6-CyberSec-35B-A3B — GGUF (abliterated, vision)

GGUF quantizations of a QLoRA fine-tune of huihui-ai/Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated, specialized for coding, IT and cybersecurity, kept uncensored, for fast local inference on a single 32 GB GPU (e.g. Tesla V100).

• Architecture: qwen3_5_moe — MoE (~35B total, ~3B active) + linear-attention (DeltaNet) + native MTP + vision. MTP head is preserved in these GGUFs (block_count 41).
• Vision works via the paired mmproj file (see below). ~57 tok/s generation on a V100.

Quantizations

Inference (llama.cpp)

llama-server -m Endy-Qwen3.6-CyberSec-35B-A3B-Q5_K_M.gguf \
  --mmproj Endy-Qwen3.6-CyberSec-mmproj-f16.gguf \
  -c 262144 -ngl 999 -fa on -ctk q8_0 -ctv q8_0 --jinja

• --mmproj enables screenshot/image input; the server then advertises vision so OpenAI-compatible clients send images.
• Anti-repetition (recommended) — this model class can loop on long agentic tasks; add server-side (clients can't override these): --dry-multiplier 0.8 --dry-base 1.75 --dry-allowed-length 2 --repeat-penalty 1.1 --repeat-last-n 512
• Speculative decoding / prefix-KV-reuse are not supported (recurrent linear-attention state can't roll back).

Training (summary)

QLoRA (Unsloth, 4-bit NF4, r32 α64 on q/k/v/o_proj), 2 epochs, train_on_responses_only, on ~90.5k coding+cybersecurity chat examples. Checkpoint step 2250 selected by validation loss. LoRA merged directly into the fp16 base (preserving the MTP head + vision tower), then converted and quantized with llama.cpp.

Datasets (examples used, licenses)

Merged from 12 candidate distill datasets → deduped to 8 unique on-topic sources → 90,470 chat-format examples (+ 914 held-out for validation). Per-source example counts:

Note: claude_mythos was 25k rows but inflated (~135 unique prompts repeated ~185×) → stripped to 101 representative rows. Malware-source-generation data was deliberately excluded; vulnerability-analysis / pentest / secure-coding kept.

License

AGPL-3.0. One training dataset (lordx64/agentic-distill-fable-5-sft) is AGPL-3.0 (strong copyleft); the derived model inherits AGPL-3.0.

Disclaimers

• Parts of the base lineage were distilled from proprietary models (Claude Opus 4.7 / Fable 5, DeepSeek V4) by third-party authors; their usage policies may restrict training competing models on their outputs. Disclosed, not waived.
• Uncensored/abliterated — outputs are unfiltered. Intended for authorized security research, pentesting, secure-coding and education. You are responsible for lawful use.
• Not affiliated with or endorsed by Qwen, Anthropic, DeepSeek, or the dataset authors.