Qwen3.6-35B-A3B-MixFP4

Qwen3.6-35B-A3B MixFP4

MixFP4-quantized checkpoint of Qwen/Qwen3.6-35B-A3B.

MLP linear tensors are stored as packed 4-bit INT4/E2M1 payloads with E4M3 block scales. Attention, router, embedding, norm, and LM-head weights remain unquantized.

Benchmarks

Notes

• MMLU-Pro accuracy is measured on all 12032 problems using instruct mode (reasoning/thinking disabled) for all checkpoints.
• KL divergence is measured on 100 conversations of 256 tokens from Aeala/ShareGPT_Vicuna_unfiltered.
• WikiText-2 perplexity is measured on the full Salesforce/wikitext wikitext-2-raw-v1 test split with Qwen tokenization, non-empty rows joined by blank-line separators, sequence length 2048, and stride 2048.

Usage

Requires a current Transformers build with native qwen3_5_moe support and trust_remote_code=True.

Example Code

from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "makora-ai/Qwen3.6-35B-A3B-MixFP4"

tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    repo_id,
    trust_remote_code=True,
    device_map="auto",
)

messages = [{"role": "user", "content": "What does int4 mean in the context of quantization? Respond in under 10 words."}]

inputs = tokenizer.apply_chat_template(messages, enable_thinking=False, add_generation_prompt=True, return_tensors="pt").input_ids.to(model.device)

outputs = model.generate(inputs, max_new_tokens=16, do_sample=False)

print(tokenizer.decode(outputs[0][inputs[0].shape[0]:], skip_special_tokens=True))
# Output:
# It represents 4-bit integer quantization.