OxCoder-9B-GGUF

OxCoder-9B GGUF

This repository provides GGUF quantizations for OxCoder-9B, a 9-billion parameter coding model optimized for long-horizon agentic software engineering and terminal tasks.

OxCoder-9B is built on the Qwen 3.5 9B foundation and trained on agentic trajectories from frontier coding agents (including Fable-5.1 and GLM-5.3 traces across Claude Code, OpenCode, and Codex). It incorporates read-before-write inspection, tool call error recovery, and minimal edit diff generation.

All GGUF files were converted from the original safetensors weights using llama.cpp at native BF16 precision, then quantized into standard k-quant variants.

---

Available Files and Quantizations

---

Architecture Requirements

OxCoder-9B uses the Qwen 3.5 architecture (Qwen3_5ForConditionalGeneration / qwen35 in llama.cpp), which incorporates hybrid linear attention layers interspersed with full self-attention and interleaved MRoPE rotary position embeddings.

Ensure your llama.cpp build or downstream runtime (such as Ollama, LM Studio, or Jan) includes Qwen 3.5 support.

---

Quickstart

1. Terminal Inference with llama.cpp

Run the model interactively using llama-cli:

llama-cli -m ./OxCoder-9B-Q4_K_M.gguf \
    -p "<|im_start|>user\nWrite a Python script to parse git commit history and summarize author stats.<|im_end|>\n<|im_start|>assistant\n" \
    -n 1024 \
    -c 8192 \
    --temp 0.6

To run a local OpenAI-compatible API server:

llama-server -m ./OxCoder-9B-Q4_K_M.gguf \
    --host 127.0.0.1 \
    --port 8080 \
    -c 16384

2. Ollama

Create a file named Modelfile in the same directory:

FROM ./OxCoder-9B-Q4_K_M.gguf

TEMPLATE """<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""

PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"
PARAMETER stop "<|endoftext|>"
PARAMETER temperature 0.6

Then register and start the model:

ollama create oxcoder-9b -f Modelfile
ollama run oxcoder-9b

3. LM Studio

1. Place any .gguf file from this folder into your LM Studio models directory.
2. Search for OxCoder-9B in your local models tab.
3. Select ChatML as the prompt format preset.

---

Benchmark Highlights

---

Model Details

• Base model: Qwen/Qwen3.5-9B
• Architecture: Qwen 3.5 (Qwen3_5ForConditionalGeneration)
• Parameters: ~9 Billion
• Native context length: 262,144 tokens
• Vocabulary size: 248,320 tokens
• License: Apache 2.0

---

Attribution & Credits

Base model architecture by Qwen. Training trajectories distilled from frontier coding agent traces.

GGUF conversions produced by Pink Pixel.

Made with 💖 by Pink Pixel