Qwen3Guard-Gen-0.6B-ONNX

Overview

The Qwen3Guard‑Gen‑0.6B model is a GenAI guardrail model designed for content safety and moderation in text generation tasks. Built as part of the Qwen3Guard series, the 0.6B‑parameter variant provides a lightweight foundation for safety-aware generative AI workflows. This repository hosts the quantized and compiled ONNX model for deployment via the on-device CPU and eIQ Neutron NPU, created using SpinQuant rotation and 4‑bit block‑wise RTN quantization.

Model Description

This is a quantized version of Qwen/Qwen3Guard-Gen-0.6B prepared for NXP i.MX 95 processors.

• Base Model: Qwen/Qwen3Guard-Gen-0.6B
• Original Model Authors: Qwen Team, Alibaba Cloud
• Original License: Apache-2.0
• Modified by: NXP

Modifications

This model is a derivative work with the following changes from the original:

• Quantization: SpinQuant (Hadamard rotation, A8) + W4 block-wise RTN (symmetric, block_size=32) + dynamic quantization
• Runtime: CPU via ONNX Runtime (CPUExecutionProvider, NeutronExecutionProvider)
• Format: Converted to ONNX

Original model available at: Qwen/Qwen3Guard-Gen-0.6B.

Limitations and Biases

This model inherits all limitations from the original Qwen/Qwen3Guard-Gen-0.6B model. Additional limitations:

• Runtime-specific: Requires ONNX Runtime with CPUExecutionProvider or NeutronExecutionProvider
• Quantization effects: May have accuracy differences due to quantization

License

This model is released under the Apache License 2.0, the same license as the original Qwen/Qwen3Guard-Gen-0.6B model.

Citation

• Haiquan Zhao et al., "Qwen3Guard Technical Report", arxiv.org:2510.14276, 2025.

If you use this model, please cite both this work and the original model:

@misc{zhao2025qwen3guardtechnicalreport,
      title={Qwen3Guard Technical Report}, 
      author={Haiquan Zhao and Chenhan Yuan and Fei Huang and Xiaomeng Hu and Yichang Zhang and An Yang and Bowen Yu and Dayiheng Liu and Jingren Zhou and Junyang Lin and Baosong Yang and Chen Cheng and Jialong Tang and Jiandong Jiang and Jianwei Zhang and Jijie Xu and Ming Yan and Minmin Sun and Pei Zhang and Pengjun Xie and Qiaoyu Tang and Qin Zhu and Rong Zhang and Shibin Wu and Shuo Zhang and Tao He and Tianyi Tang and Tingyu Xia and Wei Liao and Weizhou Shen and Wenbiao Yin and Wenmeng Zhou and Wenyuan Yu and Xiaobin Wang and Xiaodong Deng and Xiaodong Xu and Xinyu Zhang and Yang Liu and Yeqiu Li and Yi Zhang and Yong Jiang and Yu Wan and Yuxin Zhou},
      year={2025},
      eprint={2510.14276},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2510.14276}, 
}