qwen36-35b-a3b-2p02bit

Qwen3.6-35B-A3B — 2.02-bit routed experts

A weight-compressed build of Qwen/Qwen3.6-35B-A3B. The routed experts average 2.02 bits; every other weight (attention, shared expert, router, embeddings, LM head) is unchanged BF16. Weights ship dequantized into BF16 tensors, so the checkpoint loads with stock transformers / vLLM and needs no custom kernel.

Usage

from vllm import LLM, SamplingParams

llm = LLM(model="minjaechoi/qwen36-35b-a3b-2p02bit", tensor_parallel_size=2, max_model_len=72048)
out = llm.chat(
    [{"role": "user", "content": "Prove that there are infinitely many primes."}],
    SamplingParams(temperature=0.0, max_tokens=32768),
)
print(out[0].outputs[0].text)

This is a reasoning model: it emits a thinking block terminated by </think> before its answer. Give it a generous max_tokens budget — long problems routinely use tens of thousands of tokens.

Data

Calibration and validation

128 documents (235,725 tokens), sampled source-stratified from:

Held-out validation splits are drawn from the same pool, split by original document so no document appears in both. Sequences are capped at 2,048 tokens for calibration and 4,096 for validation.

GPQA is excluded from every split (0 records), so the evaluation below is uncontaminated by the data used to build this checkpoint.

Evaluation

Results

Greedy decoding, vLLM. GPQA uses a 70,000-token generation limit and flexible answer extraction; AIME 2026 uses a 100,000-token limit and math_verify.

License

Inherits the license of the base model, Qwen/Qwen3.6-35B-A3B.