Qwen3.5-9B-PereGRM-Harmonic0830-Step600

Qwen3.5-9B PereGRM Harmonic0830 — Step 300

This repository contains a full-parameter GRPO checkpoint derived from Qwen/Qwen3.5-9B. The original four-way FSDP model shards were merged into standard Hugging Face Transformers safetensors format.

Training snapshot

• Checkpoint: global_step_300
• Run: qwen35_9b_full_grpo_song2_4plus4_len2048_nothink_save300_modelonly_harmonic0830_gptoss_original_strictbuf_val100_compactlog_20260830_102251
• Reward implementation: reward_0830.py
• Training method: full-parameter GRPO (lora_rank=0)
• Actor GPUs: 4; rollout tensor parallel size: 4
• Train batch size: 4; rollout samples per prompt: 8
• Maximum prompt/response lengths: 512 / 2048 tokens
• Thinking mode: disabled by the chat-template argument used during training
• Actor learning rate: 1e-6
• KL loss: enabled, coefficient 0.001, low_var_kl
• Checkpoint policy: model-only, every 300 steps
• Validation frequency: every 100 steps, using the 100-sample reward validation set

Reward design

The Judge evaluated the original PereGRM Resonation, Expression, and Reception dimensions. Each dimension used four Judge trials during training, and the three scores were combined with a harmonic mean. That PereGRM score was then combined by another harmonic mean with three fixed-rubric rewards:

1. Task response quality
2. Memory use
3. Over-personalization restraint

Both training and validation used the configured gpt-oss-20b Judge endpoints. Although the internal run identifier contains strictbuf, the actual launch command did not explicitly set trainer.reward_buffer_size=1024.

Usage

This model requires a recent Transformers version with Qwen3.5 support.

import torch
from transformers import pipeline

pipe = pipeline(
    "image-text-to-text",
    model="KiloHelios/Qwen3.5-9B-PereGRM-Harmonic0830-Step300",
    dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {
        "role": "user",
        "content": [{"type": "text", "text": "How can I plan a focused week?"}],
    }
]
result = pipe(text=messages, max_new_tokens=512)
print(result[0]["generated_text"][-1]["content"])

The repository includes the tokenizer, processor configuration, and the chat template saved with the checkpoint.

Limitations

This is a research checkpoint optimized for a personalized-response reward. It has not been established as a general capability or safety improvement. Outputs should be independently evaluated for the intended deployment domain.