Qwable-v1-NVFP4-GGUF

Qwable-v1 NVFP4 GGUF

GGUF quantization of lordx64/Qwable-v1 — an agentic coding model built by layering Claude Fable-5 tool-use behavior on top of a Claude Opus 4.7 reasoning distill of Qwen3.6-35B-A3B.

Model Details

• Architecture: Qwen3.5 MoE, 41 blocks (40 layers + 1 MTP head), 256 experts (8 active/token)
• Active Parameters: ~3B
• Context: 262,144 tokens
• Vision: Yes (27-layer SigLIP ViT)
• License: AGPL-3.0
• Base: Qwen3.6-35B-A3B → Opus 4.7 reasoning distill → Fable-5 agentic SFT

What's Included

Quantization Details

NVFP4

• All weight tensors quantized to NVFP4 (4-bit floating point)
• F32/F16 bias and norm tensors kept at original precision
• Vision encoder and projector kept at F16 (not quantized)

Usage

# With llama.cpp (vision + text)
./llama-server -m qwable-v1-nvfp4.gguf --mmproj mmproj-qwable-v1-f16.gguf --host 0.0.0.0 --port 8080

# Text-only (no vision)
./llama-cli -m qwable-v1-nvfp4.gguf -p "Hello, how are you?"

Agentic Tool-Use

Qwable-v1 emits <tool_use> XML when prompted with an agent-style system prompt:

system: You are a coding agent. When you need to read, write, edit, or run code,
emit XML tool calls in this exact format:
<tool_use name="X" id="toolu_01abc">
{"...": "..."}
</tool_use>

Without the agent prompt, the model falls back to the Opus 4.7 reasoning prior (markdown code blocks).

Credits

• Original model: lordx64
• Quantization: FreedomAISVR