Xing4.0-29B-A4B-GGUF

<h1 align="center">Xing4.0-29B-A4B-GGUF</h1> <p align="center"><b>Run the agentic flagship on your own machine —<br>official 2–8-bit GGUF builds from 11 to 34 GB, native in llama.cpp.</b></p> <p align="center"> 🦉 <a href="https://github.com/XingChen-AGI/Xing4.0-29B-A4B">GitHub</a> • 🤗 <a href="https://huggingface.co/XingChen-AGI">Hugging Face Org</a> • 🤖 <a href="https://modelscope.cn/models/XingChen-AGI/Xing4.0-29B-A4B-GGUF">ModelScope</a> • 🪐 <a href="https://modelers.cn/models/XingChen-AGI/Xing4.0-29B-A4B-GGUF">Modelers</a> • 📖 <a href="https://github.com/XingChen-AGI/Xing4.0-29B-A4B/blob/main/tutorial/llama.cpp/README_EN.md">Local Guide</a> </p> <p align="center"> <a href="https://github.com/XingChen-AGI/Xing4.0-29B-A4B"><img alt="Quantization" src="https://img.shields.io/badge/Quantization-GGUF%202--8--bit-8B5CF6"></a> <a href="https://github.com/XingChen-AGI/Xing4.0-29B-A4B"><img alt="Weights" src="https://img.shields.io/badge/Weights-11%E2%80%9334%20GB-22C55E"></a> <a href="https://github.com/XingChen-AGI/Xing4.0-29B-A4B"><img alt="Parameters" src="https://img.shields.io/badge/Params-29B%20%2F%204B%20active-7C3AED"></a> <a href="https://github.com/XingChen-AGI/Xing4.0-29B-A4B/blob/main/LICENSE"><img alt="License" src="https://img.shields.io/badge/License-Apache%202.0-blue"></a> </p> <p align="center"> <b>Pick a build:</b>&nbsp; <a href="https://huggingface.co/XingChen-AGI/Xing4.0-29B-A4B">BF16</a> • <a href="https://huggingface.co/XingChen-AGI/Xing4.0-29B-A4B-FP8">FP8</a> • <b>GGUF (this repo)</b> • <a href="https://huggingface.co/XingChen-AGI/Xing4.0-29B-A4B-w8a8">W8A8</a> </p>

---

Contents

• Run Locally with llama.cpp
• Introduction
• Model at a Glance
• Highlights
• Architecture
• Benchmarks
• Quickstart
  • Inference
  • Serving & Ecosystem
• Recommended Parameters
• Evaluation Details
• Citation

Run Locally with llama.cpp

This repository provides low-bit quantized weights and a quantization imatrix for Xing4.0-29B-A4B, covering a 2–8-bit quantization range and achieving better accuracy. Files range from 11 to 34 GB — pick a size that fits a single consumer-grade GPU.

[!TIP] llama.cpp support is in the mainline. Native support was merged into the llama.cpp mainline (#31589, 2026-11-05): a stock llama.cpp build b12174+ loads the GGUF files in this repository directly on CPU / CUDA — no patch build required.

# 1) download one quant that fits your GPU (2–8-bit, 11–34 GB) from the Files tab, e.g.
huggingface-cli download XingChen-AGI/Xing4.0-29B-A4B-GGUF <quant-file>.gguf --local-dir ./models

# 2) serve it with a stock llama.cpp build (b12174+)
llama-server -m ./models/<quant-file>.gguf -c 32768 -ngl 99

-ngl 99 offloads all layers to the GPU; drop it for CPU-only inference. For quant selection, launch flags, and troubleshooting, see the full guide: Running Xing4.0-29B-A4B locally.

Introduction

Xing4.0-29B-A4B is a next-generation large language model in the Xing series (formerly TeleChat), developed by China Telecom Artificial Intelligence Technology Co., Ltd. With 29B total parameters and only 4B activated per token, it natively supports a 256K context length, extensible to 512K. It was trained end-to-end on a fully in-house stack — from silicon to software — and is deeply optimized for complex engineering tasks: long-context planning, multi-step reasoning, tool calling, and structured output.

For full project documentation, deployment guides, and fine-tuning recipes, visit the GitHub repository or the BF16 repository.

Model at a Glance

Highlights

• Agent-oriented architecture — built on the mHC + MLA + MTP architecture, supporting multi-step planning, tool calling, and complex reasoning-chain execution, ensuring task coherence and execution stability under long contexts.
• Deep software–hardware co-optimization — adapted to domestic NPU clusters with a fully in-house training framework, including mHC feature adaptation and fused-operator development, enabling stable and efficient training at scale.
• ~96% training throughput gain — fine-grained MoE communication optimization, selective recomputation, automatic graph–operator fusion, and hand-tuned fused mHC kernels lifted overall training throughput by approximately 96% over out-of-the-box performance.
• Native in the open ecosystem — fine-tuning via ms-swift and LLaMA-Factory merged upstream; serving via SGLang mainline (#39793) and the official vLLM plugin; local GGUF inference via llama.cpp mainline (#31589, b12174+); format-aligned with agent frameworks such as OpenCode, Claude Code, OpenClaw, and Hermes for out-of-the-box workflows.
• Easy adaptation for domain scenarios — lightweight customization on proprietary data for intent classification, table understanding, contract auditing, and knowledge-based QA, for rapid and low-cost domain capability deployment.

Architecture

Benchmarks

Scores below are measured on the base model (BF16). Head-to-head with same-class open agentic models:

Bold marks the best score in each benchmark.

Quickstart

Inference

Xing4.0-29B-A4B can be accessed via an OpenAI-compatible API (served with any stack below).

from openai import OpenAI

client = OpenAI(
    base_url="your-base-url",
    api_key="your-api-key",
)

completion = client.chat.completions.create(
    model="Xing4.0-29B-A4B",
    messages=[{"role": "user", "content": "Briefly explain the basic principles of quantum computing."}],
    temperature=1.0,
    top_p=0.95,
    extra_body={
        "repetition_penalty": 1.05,
        "skip_special_tokens": False,
        "spaces_between_special_tokens": False,
        "chat_template_kwargs": {
            "enable_thinking": True,  # Set to False to disable thinking
        },
    },
)

print(completion.choices[0].message.content)

Serving & Ecosystem

Your local path is llama.cpp — the GGUF files in this repository run natively in it (as of 2026-11-10):

Prebuilt container images and step-by-step deployment guides are listed in the GitHub repository.

Recommended Parameters

Evaluation Details

• SWE-bench Verified: We evaluate Xing4.0-29B-A4B using the SWE-agent harness with temperature=1.0, top_p=0.95, repetition_penalty=1.05 and a 210K context window.
• Terminal-Bench 2.1: We evaluate in terminus-2 with temperature=0.8, top_p=0.95, repetition_penalty=1.05, max_tokens=64K and a 24-hour timeout. We report the average score over 3 runs.
• SWE-bench Multilingual: We evaluate Xing4.0-29B-A4B using the SWE-agent harness with temperature=1.0, top_p=0.95, repetition_penalty=1.05 and a 210K context window.
• Claw-Eval: We evaluate using the official harness with temperature=0.8, top_p=0.95, repetition_penalty=1.05, max_tokens=16384 and a 256K context window. We report the average score over 3 runs.
• Tau3-Bench: We evaluate using the official harness with temperature=0.8, top_p=0.95, repetition_penalty=1.05, max_tokens=16384. We report the average pass^1 over 4 runs.
• DeepresearchBII: We evaluate using the OpenCode harness with temperature=0.8, top_p=0.95, repetition_penalty=1.05, max_tokens=32768 and a 256K context window, with Exa MCP server enabled.
• AIME2026: We evaluate with temperature=1.0, top_p=0.95, repetition_penalty=1.05, max_tokens=131072. We report the average score over 5 independent runs.
• IFBench: We evaluate with temperature=0.8, top_p=0.95, repetition_penalty=1.05, max_tokens=81920.
• AA.LCR: We evaluate with temperature=1.0, top_p=0.95, repetition_penalty=1.05, max_tokens=65536. We report the average score over 3 runs.

Citation

@misc{liu2025trainingreporttelechat3moe,
      title={Training Report of TeleChat3-MoE}, 
      author={Xinzhang Liu and Chao Wang and Zhihao Yang and Zhuo Jiang and Xuncheng Zhao and Haoran Wang and Lei Li and Dongdong He and Luobin Liu and Kaizhe Yuan and Han Gao and Zihan Wang and Yitong Yao and Sishi Xiong and Wenmin Deng and Haowei He and Kaidong Yu and Yu Zhao and Ruiyu Fang and Yuhao Jiang and Yingyan Li and Xiaohui Hu and Xi Yu and Jingqi Li and Yanwei Liu and Qingli Li and Xinyu Shi and Junhao Niu and Chengnuo Huang and Yao Xiao and Ruiwen Wang and Fengkai Li and Luwen Pu and Kaipeng Jia and Fubei Yao and Yuyao Huang and Xuewei He and Zhuoru Jiang and Ruiting Song and Rui Xue and Qiyi Xie and Jie Zhang and Zilu Huang and Zhaoxi Zhang and Zhilong Lu and Yanhan Zhang and Yin Zhang and Yanlei Xue and Zhu Yuan and Teng Su and Xin Jiang and Shuangyong Song and Yongxiang Li and Xuelong Li},
      year={2025},
      eprint={2512.24157},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2512.24157}, 
}

@misc{wang2025technicalreporttelechat2telechat25,
      title={Technical Report of TeleChat2, TeleChat2.5 and T1}, 
      author={Zihan Wang and Xinzhang Liu and Yitong Yao and Chao Wang and Yu Zhao and Zhihao Yang and Wenmin Deng and Kaipeng Jia and Jiaxin Peng and Yuyao Huang and Sishi Xiong and Zhuo Jiang and Kaidong Yu and Xiaohui Hu and Fubei Yao and Ruiyu Fang and Zhuoru Jiang and Ruiting Song and Qiyi Xie and Rui Xue and Xuewei He and Yanlei Xue and Zhu Yuan and Zhaoxi Zhang and Zilu Huang and Shiquan Wang and Xin Wang and Hanming Wu and Mingyuan Wang and Xufeng Zhan and Yuhan Sun and Zhaohu Xing and Yuhao Jiang and Bingkai Yang and Shuangyong Song and Yongxiang Li and Zhongjiang He and Xuelong Li},
      year={2025},
      eprint={2507.18013},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2507.18013}, 
}

---

<p align="center"> <b>China Telecom Artificial Intelligence Technology Co., Ltd.</b><br> <a href="https://huggingface.co/XingChen-AGI">More open models from XingChen-AGI</a> </p>