qwen3-4b-gdpo-length-sft-l2

Qwen3-4B GRPO — Level 2 (Concise)

Qwen3-4B-Instruct fine-tuned with SFT → GRPO + new reward for compressed chain-of-thought reasoning at Level 2 (Concise).

Training Pipeline

Qwen/Qwen3-4B-Instruct-2507
  → SFT LoRA (ssurface/qwen3-4b-cot-compress-l2)
  → Merged
  → GRPO with new reward
  → Merged (this model)

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("ssurface/qwen3-4b-gdpo-length-sft-l2")
tokenizer = AutoTokenizer.from_pretrained("ssurface/qwen3-4b-gdpo-length-sft-l2")

messages = [{"role": "user", "content": "Solve this using Level 2 (Concise).\nProblem: ..."}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=False))