Downloads · 30 days
5
13% of all-time downloads
Qwen3.5-397B-A17B-LoRA-SFT-v2
JinnP/Qwen3.5-397B-A17B-LoRA-SFT-v2
Qwen3.5-397B-A17B-LoRA-SFT-v2 is a machine learning model from JinnP. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.
LoRA adapter fine-tuned on AMD GPU kernel optimization agent trajectories using LLaMA-Factory.
All-time downloads
39
Public
Repo size
149 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors257 MB · 87%
At a glance
- Library
- peft
- License
- apache-2.0
- Access
- Public
- Created
- Mar 20, 2026
- Updated
- Mar 20, 2026
- SHA
- 29570b37
Base models
Model card
From the Hugging Face model README
Qwen3.5-397B-A17B LoRA SFT Adapter (v2 - 3-view, 32K)
LoRA adapter fine-tuned on AMD GPU kernel optimization agent trajectories using LLaMA-Factory.
Training Details
| Metric | Value |
|---|---|
| Base model | Qwen/Qwen3.5-397B-A17B (397B MoE, 17B active) |
| Final train loss | 0.0848 |
| LoRA rank / alpha | 16 / 32 |
| Target modules | all (13 types: q/k/v/o_proj, gate/up/down_proj, shared_expert_gate, in_proj) |
| Trainable params | 64.2M / 396.9B (0.016%) |
| cutoff_len | 32,768 |
| Training examples | 296 (3-view extraction from 102 raw trajectories) |
| Epochs | 3 |
| Runtime | 1h 32 min |
| Hardware | 8x AMD Instinct MI355X, DeepSpeed ZeRO-3 |
| wandb | amdpilot/amdpilot-sft/runs/8hh4pmqx |
Usage
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base_model = AutoModelForCausalLM.from_pretrained(
"Qwen/Qwen3.5-397B-A17B", device_map="auto", torch_dtype="bfloat16"
)
model = PeftModel.from_pretrained(base_model, "JinnP/Qwen3.5-397B-A17B-LoRA-SFT-v2")
- Library
- peft
- License
- apache-2.0
- Created
- Mar 20, 2026
- Updated
- Mar 20, 2026