qwen3-coder-next-80b-redteam-GGUF

Qwen3-Coder-Next-80B-A3B · Abliterated · RedTeam Coding Agent

Validated runtime configuration + measured results for an abliterated (uncensored) Qwen3-Coder-Next 80B-A3B running as an agentic coding backend on CPU via ik_llama.cpp. This card documents OUR tested config and findings — the weights/quant are bartowski's (see Credits). We do not retrain weights; the value here is a measured, working setup and an honest map of what it does and does not do.

---

⚠️ Safety

This model may produce sensitive, explicit, misleading, biased, or otherwise inappropriate content. Safety filtering is limited, and outputs should not be assumed to be safe by default.

Intended Use

This model is intended for research, evaluation, red-teaming, and controlled use with human oversight. It is best suited for advanced users who can implement their own safeguards and review processes.

Out-of-Scope Use

This model is not recommended for public-facing deployment without additional moderation. It should not be used for applications involving minors, regulated environments, high-risk domains, or automated decision-making.

Risks and Recommendations

Outputs may be harmful, unlawful, privacy-invasive, or factually unreliable. Human review, logging, access controls, monitoring, and additional filtering are strongly recommended before any production or external use.

---

What this is

• Base: Qwen3-Coder-Next (qwen3next architecture, hybrid attention: linear/Gated-DeltaNet + periodic full-attention layers), MoE 80B total / ~3B active (A3B).
• Variant: abliterated / uncensored — standard refusal guardrails removed from the weights.
• Quant: Q4_K_M GGUF (~48.5 GB weights; CPU buffer ~46.3 GiB with all expert tensors offloaded to CPU).
• Native context: 262144 (n_ctx_train). Served here: 65536 (64K) — chosen to fit RAM headroom; raise if your hardware allows.
• Role: agentic coding backend (tool-calling) for OpenAI-compatible / Anthropic-compatible coding clients.

Validated runtime config (ik_llama.cpp, CPU-only)

llama-server \
  --model qwen3-coder-next-abliterated-Q4_K_M.gguf \
  --alias qwen3-coder-redteam \
  --ctx-size 65536 \
  --cache-type-k q8_0 --cache-type-v q4_0 -fa on \
  --threads 14 --threads-batch 12 --n-cpu-moe 94 \
  --mlock --no-mmap --batch-size 2048 --ubatch-size 512 \
  --temp 0.1 --top-k 40 --top-p 0.9 --min-p 0.01 --repeat-penalty 1.0 \
  --jinja --host 0.0.0.0 --port 8080

• Runtime: ik_llama.cpp (ikawrakow fork) — chosen over vanilla llama.cpp for materially better CPU/MoE throughput.
• KV cache quantized (k=q8_0, v=q4_0) to keep 64K affordable on tight RAM.

Hardware tested

• Mini-PC, x86_64, 18 cores, 62 GB RAM, NO GPU (Ubuntu 24.04). Pure CPU inference (--n-cpu-moe 94).

Measured performance (real, not promised)

Behavior findings

• Tool-calling: works. Real agentic runs under opencode (MCP tools, multi-file coding) — e.g. generated a complete, working ~400-line HTML landing page; message.tool_calls clean, no round-trip loop.
• Code output is clean at Q4 — no token corruption observed in generated HTML/CSS/JS.
• Language drift is a sampling artifact, not a weight defect. On long, mixed EN/PL generations at loose sampling (temp 1.0 / top_p 1.0) the model leaks Cyrillic homoglyphs (e.g. Cyrillic с/о inside Polish words) and English calques. At temp 0.1 / top_p 0.9 this disappears: repeated Polish generations scanned clean (0 homoglyphs), correct diacritics. Recommendation: keep temp ≤ 0.4 for non-English prose; lint outputs for homoglyphs if it matters.

Client integration

• opencode → point straight at the OpenAI-compatible endpoint http://HOST:8080/v1, model qwen3-coder-redteam. Works directly.
• OpenAI Codex CLI → recent Codex dropped wire_api = "chat" and requires wire_api = "responses". llama.cpp/ik does not implement /v1/responses, so Codex needs a thin Responses→Chat proxy in front (strips unsupported fields, forwards to /v1/chat/completions). With that proxy, Codex works end-to-end.
• Claude Code → needs an Anthropic-API shim. A minimal non-streaming shim gives basic chat but is not recommended for serious agentic use (no streaming, no proper tool-use translation). Not validated here.

Credits & License

• Base: built on top of bartowski's qwen3-coder-next-abliterated-Q4_K_M.gguf, which is based on Qwen3-Coder-Next by Alibaba/Qwen — usage is subject to the Qwen license. The user is responsible for compliance with that license, local law, and export regulations.
• How this model came to be: this model was designed, built, and tested with the help of Claude Opus. The idea: the world's best coding model should be able to create smaller models in its own image. Its system prompts, parameters, and context configuration come directly from that work — the world's best coding model prepares local models that take over right on your desk.