Qwopus3.6-27B-v1-preview-INT4-FOEM

🌟Qwopus3.6-27B-v1-preview-INT4-FOEM

⚠️ Notice

Special thanks to Neiko2002, who pointed out in the discussion that using the official chat_template.jinja from Qwen3.6-27B yields the best performance. With this template, Qwopus3.6-27B-v1-preview-INT4-FOEM achieved the highest HermesAgent score among 27B models, reaching 82, while Qwen/Qwen3.6-27B-FP8 achieved 82.5. Previously, we had been using the chat_template.jinja inherited from Qwopus3.6-27B-v1-preview.

<div align="left"> <a href=https://ojs.aaai.org/index.php/AAAI/article/view/40123 target="_blank"><img src=https://img.shields.io/badge/Official%20Site-333399.svg?logo=homepage height=22px></a> <a href=https://huggingface.co/Xingyu-Zheng/Qwopus3.6-27B-v1-preview-INT4-FOEM target="_blank"><img src=https://img.shields.io/badge/%F0%9F%A4%97%20Models-d96902.svg height=22px></a> <a href=https://github.com/Xingyu-Zheng/FOEM target="_blank"><img src= https://img.shields.io/badge/Page-bb8a2e.svg?logo=github height=22px></a> <a href=https://arxiv.org/abs/2507.11017 target="_blank"><img src=https://img.shields.io/badge/Report-b5212f.svg?logo=arxiv height=22px></a> </div>

This is an unofficial quantized version of Qwopus3.6-27B-v1-preview.

🧠 Quantization Framework

GPTQModel

🗺️ Quantization Method

FOEM (AAAI 2026)

FOEM is an improved quantization method over GPTQ. The resulting model preserves the same inference structure as GPTQ, ensuring compatibility with existing deployment pipelines while achieving better accuracy.

📚 Calibration Dataset

We randomly sampled 512 examples from nohurry/Opus-4.6-Reasoning-3000x-filtered.

📋 Usage Example

This model can be deployed using standard frameworks such as vLLM, just like other GPTQModel-quantized models.

Example evaluation command:

lm-eval --model vllm --model_args pretrained=models/gptqmodel/Qwopus3.6-27B-v1-preview-INT4-FOEM,tensor_parallel_size=1,gpu_memory_utilization=0.45 --tasks wikitext --batch_size 1

🙏 Acknowledgements

Special thanks to Jackrong for providing the original model: Qwopus3.6-27B-v1-preview.

Special thanks to Neiko2002, who pointed out in the discussion that using the official chat_template.jinja from Qwen3.6-27B yields the best performance. With this template, Qwopus3.6-27B-v1-preview-INT4-FOEM achieved the highest HermesAgent score among 27B models, reaching 82, while Qwen/Qwen3.6-27B-FP8 achieved 82.5.

📖 Citation

If you use this model in your research or projects, please cite:

@misc{jackrong_qwopus36_27b_v1_preview,
  title        = {Qwopus3.6-27B-v1-preview},
  author       = {Jackrong},
  year         = {2026},
  publisher    = {Hugging Face}
}

@misc{qubitium2024gptqmodel,
  author = {ModelCloud.ai and [email protected]},
  title = {GPT-QModel},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/modelcloud/gptqmodel}},
  note = {Contact: [email protected]},
  year = {2024},
}

@inproceedings{zheng2026first,
  title={First-order error matters: Accurate compensation for quantized large language models},
  author={Zheng, Xingyu and Qin, Haotong and Li, Yuye and Chu, Haoran and Wang, Jiakai and Guo, Jinyang and Magno, Michele and Liu, Xianglong},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
  volume={40},
  number={34},
  pages={28883--28891},
  year={2026}
}