Qwen3.6-35B-A3B-RFT

Qwen3.6-35B-A3B-RFT

A fine-tuned version of Qwen/Qwen3.6-35B-A3B using Rejection Fine-Tuning (RFT) on self-generated data, inspired by the Simple Self-Distillation (SSD) paper. The LoRA adapter has been merged into the base weights -- this is a standard bf16 model ready for direct use or quantization.

Method (RFT, Not Pure SSD)

Our method is inspired by the SSD paper ("Embarrassingly Simple Self-Distillation Improves Code Generation", arxiv 2604.01193) but differs in a critical way:

• SSD (the paper): Generates samples from the model and trains on ALL of them -- correct and incorrect -- with NO filtering. That is the paper's key insight: unfiltered self-generated data still improves pass@k.
• Our method: We generated samples at high temperature, then filtered for correctness using execution-based verification (2,000 generated, 1,796 passed tests). We trained only on correct outputs.

This makes our method Rejection Fine-Tuning (RFT) -- also known as rejection sampling + SFT or on-policy distillation. RFT is a well-established technique. The difference matters: SSD's claim is that filtering is unnecessary; we used filtering, so we cannot validate or invalidate that claim.

Model Details

Architecture note: The HuggingFace config reports model_type: qwen3_5_moe -- Qwen3.6 is built on the Qwen3.5 MoE architecture with the addition of Gated DeltaNet linear attention layers.

Training Details

Method

1. Generated 2,000 coding solutions from the base model at temp=1.6, top_k=20, top_p=0.8
2. Filtered for correctness (execution + test pass) -- 1,796 samples survived
3. Split into 1,616 train / 180 validation
4. Fine-tuned with LoRA, then merged adapter into base weights

LoRA Configuration

The target modules include both standard transformer attention/MLP layers and Qwen3.6's DeltaNet linear attention layers (in_proj_qkv, in_proj_z, out_proj).

Training Hyperparameters

Training Results

Merge

Adapter merged into base weights using PeftModel.merge_and_unload() from PEFT 0.19.1. The result is a standard HuggingFace model -- no adapter loading required at inference time.

Evaluation

Tested as a 6-bit MLX quantization on Mac Studio M4 Max (128GB) against the base model (unsloth 4-bit quantization). 13 coding problems, 10 samples each at temp=0.7:

Biggest improvement on the hardest problem (expression evaluator with operator precedence and parentheses): base 7/10 -> merged 9/10.

Important caveats:

• Quantization confound: The base model was tested at 4-bit quantization while the merged model was tested at 6-bit. Higher quantization preserves more model information. Some or all of the quality difference (128/130 vs 126/130) may be attributable to quantization level rather than the RFT training. A controlled comparison at matched quantization has not been run.
• Statistical significance: The difference of 2/130 samples is not statistically significant (p ~= 0.28, Fisher's exact test). These results are within noise at this sample size.
• Temp=0 behavior: At temp=0, the merged model is expected to behave very similarly to the base model, though weights differ due to the LoRA merge. We have not formally tested temp=0 equivalence.

How to Use

With Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "shaneMattner/Qwen3.6-35B-A3B-RFT",
    torch_dtype=torch.bfloat16,
    device_map="auto",
    attn_implementation="eager",
)
tokenizer = AutoTokenizer.from_pretrained("shaneMattner/Qwen3.6-35B-A3B-RFT")

messages = [
    {"role": "user", "content": "Write a Python function to merge two sorted lists into one sorted list."}
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

With MLX (Apple Silicon)

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("shaneMattner/Qwen3.6-35B-A3B-RFT")
response = generate(
    model,
    tokenizer,
    prompt="Write a Python function to merge two sorted lists.",
    max_tokens=512,
)
print(response)

Or quantize first for faster inference:

# Convert to 6-bit MLX format
python -m mlx_lm.convert \
    --hf-path shaneMattner/Qwen3.6-35B-A3B-RFT \
    --mlx-path Qwen3.6-35B-A3B-RFT-6bit \
    -q --q-bits 6

Note: If you encounter errors related to model_type, you may need to change "model_type": "qwen3_5_moe_text" to "model_type": "qwen3_5_moe" in config.json for mlx-lm compatibility.

With llama.cpp / GGUF

Convert to GGUF for use with llama.cpp, Ollama, or other GGUF-compatible tools:

# Clone llama.cpp and convert
python convert_hf_to_gguf.py shaneMattner/Qwen3.6-35B-A3B-RFT --outtype bf16

# Quantize to desired format
./llama-quantize Qwen3.6-35B-A3B-RFT-bf16.gguf Qwen3.6-35B-A3B-RFT-Q4_K_M.gguf Q4_K_M

Limitations

• Coding-focused: Fine-tuned exclusively on Python coding tasks. General instruction following may not improve (or may slightly regress) compared to the base model.
• Bounded by base model: Self-distillation cannot exceed the base model's capability ceiling -- it improves sampling consistency, not peak ability.
• Small training set: 1,616 samples is a proof-of-concept. Larger datasets with more diverse problems would likely yield stronger results.
• Eval coverage: Tested on 13 coding problems only. Broader benchmarks (HumanEval, MBPP, etc.) have not been run. Results are not statistically significant at this sample size.
• Quantization confound: Base and merged models were evaluated at different quantization levels (4-bit vs 6-bit), confounding the quality comparison.
• DeltaNet targeting: The in_proj_a and in_proj_b DeltaNet gating layers were not included in LoRA targets -- adding them may improve results in future iterations.

Architecture Notes

Qwen3.6-35B-A3B uses a hybrid architecture:

• Mixture of Experts (MoE): 256 experts with 8 active per token, keeping active compute at ~3B parameters despite 34.66B total
• Gated DeltaNet linear attention: 30 of 40 layers use linear attention (every 4th layer uses full attention), enabling efficient long-context processing
• 262K context window: Supports up to 262,144 tokens

Citation

If you use this model, please cite:

@misc{mattner2026qwen36rft,
  title={Qwen3.6-35B-A3B-RFT: Rejection Fine-Tuned Qwen3.6 for Coding},
  author={Shane Mattner},
  year={2026},
  url={https://huggingface.co/shaneMattner/Qwen3.6-35B-A3B-RFT}
}

Related Work

• Qwen3.6-35B-A3B -- Base model by Qwen team
• LoRA: Low-Rank Adaptation of Large Language Models -- Hu et al., 2021
• Embarrassingly Simple Self-Distillation Improves Code Generation -- The SSD paper that inspired this work. Our method deviates from SSD by adding execution-based correctness filtering (making it RFT rather than pure SSD).

License

Apache 2.0 (same as the base model Qwen/Qwen3.6-35B-A3B)