Downloads · 30 days
44
1% of all-time downloads
groxaxo/Qwen3.6-35B-A3B-GPTQ-Pro-FOEM-4bit-g128
Qwen3.6-35B-A3B-GPTQ-Pro-FOEM-4bit-g128 is a machine learning model from groxaxo. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as other.
<p align="center" <img src="qwen.png" alt="Qwen3.6-35B-A3B GPTQ-Pro FOEM banner" width="100%" </p
Downloads · 30 days
44
1% of all-time downloads
All-time downloads
7.2K
Public
Parameters
6.9B
22.4 GB on disk
Likes
4
Public
Click a slice to open those files.
.safetensors22.3 GB · 100%
How the weights are stored.
I324.3B · 62%
From the Hugging Face model README
Qwen3.6-35B-A3B-GPTQ-Pro-FOEM-4bit-g128 is a GPTQ-quantized checkpoint intended for efficient GPU inference, published by groxaxo.
It is intended for open-source evaluation, reproducible experimentation, and compatible local or
hosted inference workflows. The wording below is deliberately limited to what can be verified
from this repository's metadata and artifacts.
| Field | Details |
|---|---|
| Format | GPTQ |
| Source / base | Qwen/Qwen3.6-35B-A3B |
| Intended task | image-text-to-text |
| License | other |
*.safetensors (6 files)config.jsongeneration_config.jsontokenizer.jsontokenizer_config.jsonprocessor_config.jsonchat_template.jinjaquantize_config.jsonvllm serve groxaxo/Qwen3.6-35B-A3B-GPTQ-Pro-FOEM-4bit-g128 \
--quantization gptq_marlin \
--dtype float16 \
--trust-remote-code
This command is taken from the repository documentation. Adjust tensor parallelism, context length, and cache settings to match your hardware and vLLM version.
Quantization or conversion changes numerical behavior, memory use, and throughput relative to the source checkpoint; validate quality on your own workload.
Generated outputs may be inaccurate or unsuitable for a given use case. Users are responsible for testing behavior, applying appropriate safeguards, and complying with applicable licenses and laws.
<!-- polished-overview:end -->A fast, production-minded GPTQ-Pro quantization of Qwen3.6-35B-A3B, built for local agentic coding, reasoning, tool use, and high-throughput vLLM serving on dual 24GB GPUs.
This checkpoint takes the official Qwen3.6-35B-A3B model — a sparse MoE with 35B total parameters and 3B active parameters per token — and packages it into a practical 4-bit GPTQ-Pro deployment profile for builders who want Qwen3.6-class capability without needing enterprise GPU memory.
The goal is simple: strong Qwen3.6 reasoning, much lower VRAM pressure, clean vLLM deployment, and real benchmark numbers instead of vibes.
Official Qwen3.6-35B-A3B is designed for real-world developer workflows: agentic coding, frontend tasks, repository-level reasoning, tool use, and iterative problem solving. This quantized release keeps that use case in focus while making the model easier to run on accessible local hardware.
This release is tuned for:
| Feature | Details |
|---|---|
| Base model | Qwen/Qwen3.6-35B-A3B |
| Architecture | Sparse Mixture-of-Experts |
| Parameters | 35B total / 3B active |
| Quantization | GPTQ-Pro FOEM 4-bit |
| Group size | 128 |
| vLLM backend | gptq_marlin |
| Tested serving profile | TP2 across 2x RTX 3090 |
| Tested context | 150K max sequence length |
| API style | OpenAI-compatible /v1/chat/completions |
| Reasoning parser | qwen3 |
| Tool parser | qwen3_coder |
This quantization is aimed at the ugly real world: limited VRAM, large prompts, coding-agent loops, and production serving where speed and stability matter.
GPTQ-Pro FOEM 4-bit gives this model a practical deployment shape:
gptq_marlinThis is not a toy quant for screenshots. It is built to serve.
Validated on a TP2 vLLM deployment with 2x RTX 3090 GPUs.
| Metric | Value |
|---|---|
| Total tests | 109 |
| Latency stdev | 171.63 ms |
| Mean throughput | 170.90 tokens/sec |
| Median throughput | 171.50 tokens/sec |
| Min throughput | 150.54 tokens/sec |
| Max throughput | 174.13 tokens/sec |
Benchmark mix: reasoning, coding, math, general QA, and practical chat prompts.
These numbers are provided to give builders an actual deployment baseline. Your exact results will depend on GPUs, PCIe topology, CPU, RAM, CUDA stack, vLLM version, context length, batch size, and sampling settings.
Validated with:
2gptq_marlinqwen3qwen3_coder150000The official Qwen3.6-35B-A3B model supports a larger native context window, but this quantized release was specifically validated at 150K in the tested setup. Treat 150K as the known-good production profile.
hf download groxaxo/Qwen3.6-35B-A3B-GPTQ-Pro-FOEM-4bit-g128 \
--local-dir ./qwen36-gptqpro