Downloads · 30 days
10
11% of all-time downloads
shaunak1234/qwen3-32b-lora-finetome
qwen3-32b-lora-finetome is a text generation model from shaunak1234. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.
A general-purpose instruction-following model fine-tuned from Qwen3-32B using LoRA on the FineTome-100k dataset. Enhances the base model's ability to follow complex instructions, reason step-by-step, and produce well-…
Downloads · 30 days
10
11% of all-time downloads
All-time downloads
88
Public
Repo size
36.5 GB
Likes
0
Public
Click a slice to open those files.
.pt34.4 GB · 61%
From the Hugging Face model README
A general-purpose instruction-following model fine-tuned from Qwen3-32B using LoRA on the FineTome-100k dataset. Enhances the base model's ability to follow complex instructions, reason step-by-step, and produce well-structured responses.
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
base_model = AutoModelForCausalLM.from_pretrained(
"Qwen/Qwen3-32B",
torch_dtype=torch.bfloat16,
device_map="auto",
trust_remote_code=True,
)
model = PeftModel.from_pretrained(base_model, "shaunak1234/qwen3-32b-lora-finetome")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-32B", trust_remote_code=True)
messages = [
{"role": "system", "content": "You are a helpful, accurate, and thoughtful assistant."},
{"role": "user", "content": "Explain the difference between TCP and UDP, and when you'd choose each for a real-world application."}
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=1024, temperature=0.7, top_p=0.9)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
from peft import PeftModel
from transformers import AutoModelForCausalLM
import torch
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-32B", torch_dtype=torch.bfloat16)
model = PeftModel.from_pretrained(base, "shaunak1234/qwen3-32b-lora-finetome")
merged = model.merge_and_unload()
merged.save_pretrained("qwen3-32b-finetome-merged")
# Serve with vLLM, TGI, or any inference framework
| Parameter | Value |
|---|---|
| LoRA rank (r) | 64 |
| LoRA alpha | 128 |
| LoRA dropout | 0.05 |
| Target modules | all-linear |
| Batch size | 8 |
| Gradient accumulation | 4 |
| Effective batch size | 32 |
| Learning rate | 2e-4 |
| LR scheduler | Cosine |
| Warmup ratio | 0.05 |
| Weight decay | 0.01 |
| Max sequence length | 4096 |
| Epochs | 1 |
| Total steps | 765 |
| Precision | bfloat16 |
| Gradient checkpointing | Yes |
| Component | Details |
|---|---|
| GPU | AMD Instinct MI300X (192GB HBM3) |
| Platform | AMD DevCloud |
| Software | PyTorch 2.5.1 + ROCm 6.2 |
| Framework | Transformers 4.51.3, PEFT 0.15+ |
| Training speed | ~33-35 seconds/step |
| Total training time | ~7 hours |
| Cost | ~$14 (at $2/hr) |
Apache 2.0 (following Qwen3-32B base model license)