Downloads · 30 days
7
10% of all-time downloads
qwen3-4b-grpo-v1
Kaito-F/qwen3-4b-grpo-v1
qwen3-4b-grpo-v1 is a text generation model from Kaito-F. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
This model is fine-tuned from Kaito-F/qwen3-4b-instruct-lora-v2 using GRPO (Group Relative Policy Optimization) with Unsloth.
All-time downloads
73
Public
Parameters
4B
8.1 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors8 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- qwen3
- Access
- Public
- Created
- Feb 20, 2026
- Updated
- Feb 20, 2026
- SHA
- fe15721b
Try a prompt
Base models
Model card
From the Hugging Face model README
GRPO-tuned Agent Model
This model is fine-tuned from Kaito-F/qwen3-4b-instruct-lora-v2 using GRPO (Group Relative Policy Optimization) with Unsloth.
Training Details
- Method: GRPO with 4-bit quantized training + LoRA
- Base model: Kaito-F/qwen3-4b-instruct-lora-v2 (SFT-tuned)
- Learning rate: 5e-6
- LoRA: r=32, alpha=64
- KL coefficient (beta): 0.04
- Group size: 8
- Max steps: 200
Reward Functions
- Format Reward: ReAct format (Thought/Action) compliance
- ALFWorld Reward: Task completion (+2.0), invalid action (-0.5)
- DBBench Reward: SQL syntax (+0.5), result match (+1.5)
- Conciseness Reward: Fewer steps = higher reward
Dataset Mix
- DBBench: 70% (aggregation, counting, comparison prioritized)
- ALFWorld: 30%
Sources & Terms
- ALFWorld dataset: u-10bei/sft_alfworld_trajectory_dataset_v5
- DBBench dataset: u-10bei/dbbench_sft_dataset_react_v4
- License: Apache 2.0
- Task
- Text Generation
- Library
- transformers
- Type
- qwen3
- License
- apache-2.0
- Languages
- en
- Created
- Feb 20, 2026
- Updated
- Feb 20, 2026