qwen3-4B-shell

qwen3-4B-shell (RFT Adapter)

A LoRA adapter for Qwen3-4B that teaches tool-calling, bad command correction, and diagnostic thinking for shell/systems tasks.

Training Data

Synthetic dataset generated with xiaomi mimo-v2.5 — a reasoning model used to create multi-turn conversations with tool-call patterns.

The dataset teaches a try → fail → research → correct pattern:

User: "Count files in a directory"
Assistant: Let me try.
Assistant: [bash: ls | wc -l]
Tool: Attempted: ls | wc -l
Assistant: That wasn't right. Let me research and correct.
Assistant: [research: "count all files linux"]
Tool: Use find with -type f
Assistant: Use `find . -type f | wc -l`. `ls | wc -l` fails for files with newlines.

Behavior Trained

1. Tool calls — correct, specific commands (uv over pip, find over ls, etc.)
2. Bad command correction — recognize wrong commands, research the fix, explain why
3. Diagnostic thinking — check logs, profile processes, inspect permissions before acting
4. Safety — suggest backups, Docker/VMs for destructive operations
5. Platform awareness — pfctl on macOS, Get-Content on PowerShell, etc.

Benchmark Results

Tested on 20 cases across 4 categories:

RFT scored highest overall, excelling at bad command fixes (80.0 vs SFT's 65.0) and diagnostics (95.0 vs SFT's 85.0).

Adapter Details

• Base model: unsloth/qwen3-4b-unsloth-bnb-4bit (4-bit NF4 quantized)
• LoRA rank: 16, alpha: 32, dropout: 0.05
• Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Trainable params: 33M (0.81% of total)
• PEFT version: 0.19.1

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel, LoraConfig

# Load base model
bnb_config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4")
model = AutoModelForCausalLM.from_pretrained(
    "unsloth/qwen3-4b-unsloth-bnb-4bit",
    quantization_config=bnb_config,
    device_map="auto",
)

# Apply LoRA and load adapter
lora_config = LoraConfig(
    r=16, lora_alpha=32, lora_dropout=0.05,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
                    "gate_proj", "up_proj", "down_proj"],
    task_type="CAUSAL_LM",
)
model = PeftModel(model, lora_config)
model.load_adapter("whos-carmen/qwen3-4B-shell", adapter_name="rft")
model.set_adapter("rft")

# Chat
tokenizer = AutoTokenizer.from_pretrained("whos-carmen/qwen3-4B-shell")
messages = [
    {"role": "system", "content": "You are a shell assistant. Prefer uv over pip. Research before guessing."},
    {"role": "user", "content": "I ran `ls | wc -l` but it only counts visible files. How do I count all files?"},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:]))

Files

whos-carmen/qwen3-4B-shell/
├── README.md              # This file
├── rft/
│   ├── adapter_config.json
│   ├── adapter_model.safetensors
│   ├── chat_template.jinja
│   ├── tokenizer.json
│   └── tokenizer_config.json
└── data/
    ├── rft_train.jsonl    # Training data (13 samples)
    └── rft_valid.jsonl    # Validation data (3 samples)