Downloads · 30 days
40
2% of all-time downloads
Qwen3.5-4B-MoLE
hotdogs/Qwen3.5-4B-MoLE
Qwen3.5-4B-MoLE is a machine learning model from hotdogs. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
8 LoRA experts trained on Qwen3.5-4B base for the MoLE Research project.
All-time downloads
2.1K
Public
Repo size
232 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf95.8 GB · 57%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- May 16, 2026
- Updated
- May 18, 2026
- SHA
- bfb2eca4
Base models
Model card
From the Hugging Face model README
Qwen3.5-4B MoLE (Mixture of LoRA Experts)
8 LoRA experts trained on Qwen3.5-4B base for the MoLE Research project.
Experts
| Expert | Dataset | eval_loss | Status |
|---|---|---|---|
codex | 50K coding | 0.497 | ✅ PASSED |
logos | 50K reasoning | 0.409 | ✅ PASSED |
scientia | smoltalk 20K | 0.865 | ✅ PASSED |
dialogos | ultrachat 30K | 1.256 | ✅ PASSED |
poiesis | gpt4all-j 50K | 1.645 | usable |
psyche | hh-rlhf 50K | 1.766 | usable |
aegis | dolphin 10K | 1.504 | usable |
polyglot | opus-100 20K | 1.521 | usable |
Usage
from peft import PeftModel
from transformers import AutoModelForCausalLM
base = "huihui-ai/Huihui-Qwen3.5-4B-Claude-4.6-Opus-abliterated"
model = AutoModelForCausalLM.from_pretrained(base)
model = PeftModel.from_pretrained(model, "hotdogs/Qwen3.5-4B-MoLE/experts/codex")
Training
- Framework: Unsloth QLoRA (4-bit)
- LoRA: r=32, alpha=64
- Max steps: 300 per expert
- Quality gate: eval_loss < 1.5
- Hardware: NVIDIA RTX 4060 Ti 16GB
- License
- apache-2.0
- Languages
- en, th, multilingual
- Created
- May 16, 2026
- Updated
- May 18, 2026