qwen35_4b_sft_VE_GRPO_RL

Qwen3.5-4B · Video-Evaluation GRPO

This directory contains the Hugging Face conversion of the final checkpoint from the video-evaluation GRPO run initialized from JianhuiWei/qwen35_4b_sft_VE.

Training summary

• Initial model: JianhuiWei/qwen35_4b_sft_VE
• RL algorithm: GRPO
• Agent harness: OpenCode
• Final checkpoint: global step 294
• Samples per prompt: 16
• Learning rate: 1e-6 with a constant schedule
• KL regularization: low-variance KL with coefficient 0.001
• Advantage normalization: enabled
• Error Tool-Call Mask in the initial SFT model: disabled

The full effective configuration, artifact provenance, and Step-294 evaluation snapshot are recorded in RL_TRAINING_METADATA.md.

Custom chat template

The bundled chat_template.jinja is the custom Qwen3.5 template used by the training and inference stack. It preserves assistant reasoning content in <think> blocks and renders tool calls in Qwen's XML-style tool-call format.

When serving with SGLang, pass the template explicitly:

python -m sglang.launch_server \
  --model-path /path/to/model \
  --chat-template /path/to/model/chat_template.jinja \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder

Transformers loading

import torch
from transformers import AutoProcessor, Qwen3_5ForConditionalGeneration

model_id = "/path/to/model"
processor = AutoProcessor.from_pretrained(model_id)
model = Qwen3_5ForConditionalGeneration.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

The checkpoint is stored as sharded Hugging Face safetensors. The language weights come from RL step 294; the frozen visual encoder is preserved from the initial SFT model. Optimizer, scheduler, trainer, and RNG states are intentionally excluded.