Downloads · 30 days
44
46% of all-time downloads
qwen3.8-27b-gsm8k-lora
Balab2021/qwen3.8-27b-gsm8k-lora
qwen3.8-27b-gsm8k-lora is a text generation model from Balab2021. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.
LoRA adapter for grade-school math word problems.
All-time downloads
95
Public
Repo size
1.9 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors638 MB · 97%
At a glance
- Task
- Text Generation
- Library
- peft
- License
- apache-2.0
- Access
- Public
- Created
- Sep 17, 2026
- Updated
- Sep 18, 2026
- SHA
- 414be928
Try a prompt
Base models
Datasets
Model card
From the Hugging Face model README
qwen3.8-27b-gsm8k-lora
LoRA adapter for grade-school math word problems.
Fine-tuned from Qwen/Qwen3.8-27B on openai/gsm8k with QLORA.
Results
| Model | GSM8K exact match |
|---|---|
| Qwen3.8-27B (base) | 83.6% (209/250) |
| this model | 93.6% (234/250) |
Training setup
| Method | QLORA |
| LoRA rank / alpha | 32 / 64 |
| Adapted modules | attention q,k,v,o on the full-attention layers; FFN gate,up,down on all 64 layers |
| Gated DeltaNet projections | left frozen |
| Vision tower and MTP head | left frozen, carried over from the base checkpoint |
| Precision | bf16 (fp16 produces NaN gradients in the Gated DeltaNet blocks) |
| Max sequence length | 1024 |
| Hardware | 1 node, 8 GPUs, Slurm partition batch-xdr |
Training data is GSM8K rationales as visible (non-thinking) content, so evaluate thinking-mode behaviour separately if you depend on it.
Usage
from transformers import AutoModelForImageTextToText, AutoTokenizer
from peft import PeftModel
import torch
base = AutoModelForImageTextToText.from_pretrained(
"Qwen/Qwen3.8-27B", dtype=torch.bfloat16, device_map="auto"
)
model = PeftModel.from_pretrained(base, "Balab2021/qwen3.8-27b-gsm8k-lora")
tokenizer = AutoTokenizer.from_pretrained("Balab2021/qwen3.8-27b-gsm8k-lora")
Reproducing
Scripts: cluster-activities/qwen38-gsm8k-sft in the source repo.
Requires transformers>=5.2.0 and flash-linear-attention for the
qwen3_5 architecture.
- Task
- Text Generation
- Library
- peft
- License
- apache-2.0
- Languages
- en
- Created
- Sep 17, 2026
- Updated
- Sep 18, 2026