Downloads · 30 days
246
4% of all-time downloads
Abiray/Agents-A1-Q4_K_M-GGUF
Agents-A1-Q4_K_M-GGUF is a text generation model from Abiray. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as apache-2.0.
This repository contains the GGUF quantized version of Agents-A1, a 35B Mixture-of-Experts (MoE) agentic model developed by InternScience. Quantization has been performed using the Q4KM method to optimize performance…
Downloads · 30 days
246
4% of all-time downloads
All-time downloads
7K
Public
Repo size
21.2 GB
Likes
8
Public
Click a slice to open those files.
.gguf21.2 GB · 100%
From the Hugging Face model README
This repository contains the GGUF quantized version of Agents-A1, a 35B Mixture-of-Experts (MoE) agentic model developed by InternScience. Quantization has been performed using the Q4_K_M method to optimize performance and reduce memory consumption while preserving agentic reasoning capabilities.
| File Name | Size | Quantization Method | Description |
|---|---|---|---|
Agents-A1-Q4_K_M.gguf | 21.2 GB | Q4_K_M | Medium 4-bit quantization. Recommended balance between size, speed, and accuracy. |
Agents‑A1 is a 35B Mixture‑of‑Experts agentic model built to scale heterogeneous agentic abilities across multiple domains including Long‑horizon Search, Engineering, Scientific Research, Instruction Following, and Tool-calling.
Despite operating in the ~35B model class, Agents-A1 delivers highly competitive performance against frontier-scale systems such as GPT-5.5, DeepSeek-V4-pro, and Kimi-K2.6. It achieves overall SOTA results on several challenging benchmarks, including Seal-0 (56.4), HiPhO (46.4), FrontierScience-Olympiad (79.0), FrontierScience-Research (40.0), IFBench (80.6), and IFEval (94.8).
You can run this GGUF model using popular inference engines like llama.cpp or Ollama.
To run inference via the command line with llama.cpp, download the file and use the following command structure:
./llama-cli \
--model Agents-A1-Q4_K_M.gguf \
--prompt "<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n<|im_start|>user\nHello, who are you?<|im_end|>\n<|im_start|>assistant\n" \
--ctx-size 8192 \
--threads 8