Downloads · 30 days
13
1% of all-time downloads
Grug-35B-A3B-NPU2
Atomic-Germ/Grug-35B-A3B-NPU2
Grug-35B-A3B-NPU2 is a text generation model from Atomic-Germ. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
Grug 35B A3B is a compact-reasoning fine-tune of Qwen/Qwen3.6-35B-A3B. It keeps the Qwen MoE/A3B architecture: 35B total parameters with roughly 3B activated per token, 40 text layers, 256 experts, and 8 routed expert…
All-time downloads
1.3K
Public
Repo size
23.2 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.q4nx23.2 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- qwen3_5_moe
- Access
- Public
- Created
- Aug 12, 2026
- Updated
- Aug 25, 2026
- SHA
- e2a2391b
Try a prompt
Base models
Datasets
- hotdogs/uka-glm-5.2
- Scale-or-Reason/general-reasoning-ift-pairs
- samcheng0/lumia-reasoning-sft-v1
- HSH-Intelligence/verified-math-reasoning-3k
- kd13/CodeDebug-Instruct-v2-Reasoning
- Madarabr/cortex-adaptive-thinking
- CL-From-Nothing/code_rose_initial_1_7B_SFT_10K_rollouts_Qwen3-4B-Thinking-2507_k12_t0.7_maxtok12288
Model card
From the Hugging Face model README
IF YOU USE COMMUNITY QWEN MODELS DO NOT UPGRADE TO FLM v1.0.2+
Grug-35B-A3B - Q4NX for FastFlowLM (AMD Ryzen AI XDNA2)
Grug 35B A3B is a compact-reasoning fine-tune of Qwen/Qwen3.6-35B-A3B.
It keeps the Qwen MoE/A3B architecture: 35B total parameters with roughly 3B
activated per token, 40 text layers, 256 experts, and 8 routed experts per
token plus the shared expert path. Converted to Q4NX for FastFlowLM. This variant ships vision Q4NX weights alongside the text weights.
What is Q4NX?
Q4NX is FastFlowLM's native packed-quantization format - a rearranged Q4_1 layout tuned for the NPU matrix engine's tile sizes and memory access patterns. It is not a GGUF file and it does not run on llama.cpp or Ollama; it is meant exclusively for the FastFlowLM engine on AMD Ryzen AI NPUs.
Requirements
- FastFlowLM >= 0.9.45 (
flmCLI) - AMD Ryzen AI processor with XDNA2 (NPU2) - Strix Point / Ryzen AI 300 series or later
- Linux with the XRT NPU stack installed
- ~51 GB of unified system memory (Q4NX weights + activations + KV cache)
Files
| File | Purpose |
|---|---|
| model.q4nx | Quantized Q4NX weights |
| config.json | FastFlowLM model configuration |
| tokenizer.json | Tokenizer |
| tokenizer_config.json | Special tokens and chat template |
| chat_template.jinja | Chat template (optional) |
| vision_weight.q4nx | Vision tower weights (multimodal input) |
| flm-add.py | Installer script - registers this model with FastFlowLM |
Install and run
This repository works with flm-add, a small installer that copies the model
into the FastFlowLM user directory and registers the tag. It never
modifies the system FastFlowLM install.
pip install flm-add or uv tool install flm-add
uv tool install flm-add
flm-add Atomic-Germ/Grug-35B-A3B-NPU2 --tag grug-moe:35b-a3b --family qwen3.6-moe
FLM_CONFIG_PATH="$HOME/.config/flm/model_list.json" FLM_XCLBIN_PATH="$HOME/.config/flm" flm run grub-moe:35b-a3b
Kernels
FastFlowLM's NPU kernels (xclbins) are closed source and are not shipped in
this repository. flm-add.py links the kernels of the official qwen3.6-moe:35b-a3b
model (Qwen3.6-35B-A3B-NPU2), because this model shares the same engine family
(qwen3.6-moe) and architecture.
Model
- Registry tag:
qwen3.6-a3b:35b - Engine family:
qwen3.6-moe - Kernel source: Qwen3.6-35B-A3B-NPU2
- Context length: 262,144 tokens (from config)
model.q4nxsize: 23.24 GB- Base model: kai-os/Grug-35B-A3B
- License: apache-2.0
Original model card
See the upstream model card for training details, benchmarks, and upstream usage. This repository only contains the Q4NX conversion for FastFlowLM.
- Upstream card: kai-os/Grug-35B-A3B
- Task
- Text Generation
- Library
- transformers
- Type
- qwen3_5_moe
- License
- apache-2.0
- Languages
- en
- Created
- Aug 12, 2026
- Updated
- Aug 25, 2026