Downloads · 30 days
94
14% of all-time downloads
Qwopus-MoE-35B-A3B
samuelcardillo/Qwopus-MoE-35B-A3B
Qwopus-MoE-35B-A3B is a machine learning model from samuelcardillo. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
QLoRA fine-tune of Qwen3.5-35B-A3B (MoE, 3B active parameters) with Claude Opus 4.6 reasoning distillation. Training recipe adapted from Jackrong's Qwopus3.5-27B-v3.
All-time downloads
677
Public
Parameters
36B
71.9 GB on disk
Likes
11
Public
Repo makeup
Click a slice to open those files.
.safetensors71.9 GB · 100%
Parameter types
How the weights are stored.
BF1636B · 100%
Base models
Model card
From the Hugging Face model README
Qwopus MoE 35B-A3B — Claude Opus 4.6 Reasoning Distilled
QLoRA fine-tune of Qwen3.5-35B-A3B (MoE, 3B active parameters) with Claude Opus 4.6 reasoning distillation. Training recipe adapted from Jackrong's Qwopus3.5-27B-v3.
This is the full BF16 safetensors model. For GGUF quantizations (Q4, Q5, Q6, Q8), see samuelcardillo/Qwopus-MoE-35B-A3B-GGUF.
Credits
This model is based on the work of Jackrong and his Qwopus3.5-27B-v3 training methodology — same datasets, same philosophy, adapted for the MoE architecture. See his complete training guide.
Model Details
| Property | Value |
|---|---|
| Base Model | Qwen/Qwen3.5-35B-A3B |
| Architecture | Mixture of Experts (MoE) |
| Total Parameters | ~35B |
| Active Parameters | ~3B per token |
| Precision | BF16 |
Training Details
| Parameter | Value |
|---|---|
| Method | QLoRA (4-bit base + LoRA in BF16) |
| Framework | Unsloth 2026.4.2 + TRL |
| LoRA Rank | 32 |
| LoRA Alpha | 32 |
| LoRA Targets | q_proj, k_proj, v_proj, o_proj |
| Trainable Parameters | 6,881,280 (0.02%) |
| Epochs | 2 |
| Final Loss | 0.5517 |
| GPU | NVIDIA RTX PRO 6000 Blackwell (96GB) |
| Training Time | ~3.5 hours |
Datasets (3,209 examples)
| Dataset | Examples |
|---|---|
| nohurry/Opus-4.6-Reasoning-3000x-filtered | 2,326 |
| Jackrong/Qwen3.5-reasoning-700x | 633 |
| Roman1111111/claude-opus-4.6-10000x | ~250 |
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
"samuelcardillo/Qwopus-MoE-35B-A3B",
torch_dtype="bfloat16",
trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(
"samuelcardillo/Qwopus-MoE-35B-A3B",
trust_remote_code=True,
)
Acknowledgements
- Type
- qwen3_5_moe
- License
- apache-2.0
- Languages
- en, zh
- Created
- Apr 6, 2026
- Updated
- Apr 6, 2026