Downloads · 30 days
0
Kaito-F/qwen3-4b-sft-v13-mixed-rlvmr
qwen3-4b-sft-v13-mixed-rlvmr is a text generation model from Kaito-F. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.
This repository provides a LoRA adapter fine-tuned from Qwen/Qwen3-4B-Instruct-2507 using LoRA + Unsloth.
Downloads · 30 days
0
Access
Public
Updated Feb 27, 2026
Parameters
4B
8.1 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors8 GB · 100%
From the Hugging Face model README
This repository provides a LoRA adapter fine-tuned from Qwen/Qwen3-4B-Instruct-2507 using LoRA + Unsloth.
This repository contains LoRA adapter weights only. The base model must be loaded separately.
This adapter is trained to improve multi-turn agent task performance on ALFWorld (household tasks) and DBBench (database operations).
All training data is tagged with RLVMR meta-reasoning tags (planning, explore, reflection, monitor) to enable subsequent reinforcement learning with verifiable meta-reasoning rewards.
Loss is applied to all assistant turns in the multi-turn trajectory, enabling the model to learn environment observation, action selection, tool use, and recovery from errors.
| Source | Samples | Description |
|---|---|---|
| DBBench v1-v4 (RLVMR tagged) | ~3,060 | Multi-turn SQL agent trajectories |
| DBBench v4 (upsampled) | ~1,000 | Additional v4 samples |
| LogicCat (RLVMR tagged) | ~4,030 | Single-turn MySQL CoT (MIT license) |
| ALFWorld v5 (RLVMR tagged) | ~2,500 | Household task trajectories |
| ALFWorld v4 (RLVMR tagged) | ~2,500 | Household task trajectories |
| ALFWorld v3 (RLVMR tagged) | ~1,000 | Household task trajectories |
| Total | ~14,090 | DBBench ~57% / ALFWorld ~43% |
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch
base = "Qwen/Qwen3-4B-Instruct-2507"
adapter = "Kaito-F/qwen3-4b-sft-v13-mixed-rlvmr"
tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(
base,
torch_dtype=torch.float16,
device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter)
Training data:
This repository does NOT redistribute the dataset. Users must comply with the dataset license and base model terms.