Downloads · 30 days
117
25% of all-time downloads
Luck-Qwen3-4b-Code-FineTune
ahmetggg/Luck-Qwen3-4b-Code-FineTune
Luck-Qwen3-4b-Code-FineTune is a text generation model from ahmetggg. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.
A QLoRA fine-tune of unsloth/Qwen3-4B-Instruct-2507 specialized in agentic coding, multi-step reasoning, tool use, and calibrated uncertainty.
All-time downloads
464
Public
Parameters
4B
46.6 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors8 GB · 100%
At a glance
- Task
- Text Generation
- Library
- peft
- License
- apache-2.0
- Model type
- qwen3
- Access
- Public
- Created
- Aug 31, 2026
- Updated
- Aug 31, 2026
- SHA
- ae050b5b
Try a prompt
Base models
Model card
From the Hugging Face model README
Luck-Qwen3-4b-Code-FineTune
A QLoRA fine-tune of unsloth/Qwen3-4B-Instruct-2507 specialized in agentic coding, multi-step reasoning, tool use, and calibrated uncertainty.
TL;DR: This model improves output formatting consistency, tool interaction, and knowledge boundary awareness ("I don't know" responses) without increasing the underlying capability ceiling of the base model.
<p align="center"> <img src="training_loss.png" alt="Training Loss Curve" width="100%"> </p> <p align="center"><em>Training loss convergence from ~1.7 to ~0.55 across 1,267 steps.</em></p>Model Summary
| Parameter / Metric | Value |
|---|---|
| Base Model | unsloth/Qwen3-4B-Instruct-2507 (4.05B params, dense) |
| Method | QLoRA (4-bit NF4) |
| LoRA Rank / Alpha | r=16, alpha=16, dropout=0 |
| Target Modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Trainable Parameters | 33,030,144 / 4,055,498,240 (0.81%) |
| Context Length | 4,096 tokens |
| Dataset Size | 10,129 examples |
| Training Steps / Epochs | 1,267 steps / 1 epoch |
| Effective Batch Size | 8 (2 per-device × 4 gradient accumulation steps) |
| Optimizer / Schedule | adamw_8bit, learning rate 2e-4, cosine schedule |
| Hardware | Single T4 GPU (Google Colab / Kaggle) |
| Final Loss | ~0.55 (started at ~1.7) |
| Frameworks | Unsloth, TRL (SFTTrainer), PEFT, Transformers |
In This Repo
This repository contains:
- QLoRA LoRA Adapter weights
- Merged FP16 safetensors weights
Note: Quantized GGUF files are hosted separately at ahmetggg/Luck-Qwen3-4b-Code-FineTune-GGUF.
Intended Use
- Agentic Coding: Repository-level navigation, issue resolution, and code generation.
- Multi-Step Reasoning: Step-by-step problem-solving in science and math domains.
- Tool Use: Structured multi-turn function and tool calling.
- Calibrated Uncertainty: Refusing to answer ("I don't know") when encountering actual knowledge gaps.
Training Data
| Source | Focus Area | Details / Filtering |
|---|---|---|
nvidia/Open-SWE-Traces | Agentic coding | Filtered exclusively to resolved == 1 trajectories |
open-thoughts/OpenThoughts3-1.2M | Math & science reasoning | Code domain excluded (handled by Open-SWE-Traces) |
Agent-Ark/Toucan-1.5M | Tool calling | Real multi-turn tool interaction trajectories |
| Custom R-Tuning Set | Calibration | ~150–300 MMLU items queried on base model; correct outputs retained as normal QA, incorrect outputs relabeled as "I don't know" |
Usage
Option 1: Unsloth (LoRA Adapter)
from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
model_name = "ahmetggg/Luck-Qwen3-4b-Code-FineTune",
max_seq_length = 4096,
load_in_4bit = True,
)
FastLanguageModel.for_inference(model)
messages = [{"role": "user", "content": "Write a Python function to reverse a linked list."}]
inputs = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt").to("cuda")
outputs = model.generate(inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))
- Task
- Text Generation
- Library
- peft
- Type
- qwen3
- License
- apache-2.0
- Languages
- en
- Created
- Aug 31, 2026
- Updated
- Aug 31, 2026