Qwen3.5-4B-RL-MATH

Qwen3.5-4B-RL-MATH

Qwen3.5-4B-RL-MATH is a mathematical reasoning reinforcement-learning model trained from Qwen/Qwen3.5-4B. It was post-trained on wflying/math-rlvr-15k, a 15,000-example English mathematical reasoning dataset prepared for reinforcement learning with verifiable rewards (RLVR).

The model uses a GRPO-style RLVR objective with rule-verifiable mathematical correctness signals. This post-training stage focuses on improving competition mathematics and transferable reasoning ability in non-thinking mode.

Model details

• Base model: Qwen/Qwen3.5-4B
• Training dataset: wflying/math-rlvr-15k
• Training method: GRPO-style reinforcement learning with verifiable mathematical rewards
• Primary use: Mathematical reasoning and competition-style problem solving
• Architecture: Qwen3.5 multimodal conditional-generation architecture, approximately 4.66B parameters, BF16 weights
• Training mode: Non-thinking (enable_thinking=False)

Evaluation results

All results below were obtained in non-thinking mode. The table reports the evaluation scores for the released model and two Qwen3.5 baselines.

Under this evaluation setup, Qwen3.5-4B-RL-MATH improves over the Qwen3.5-4B baseline by 16.05 points on AIME24, 20.21 on AIME25, 20.42 on HMMT25, 5.68 on GPQA-Diamond, 7.20 on zebralogicbench, and 10.52 on LiveCodeBench-v6.

Usage example

Use a recent version of transformers with Qwen3.5 support. The example below follows the non-thinking configuration used for evaluation.

import torch
from transformers import AutoProcessor, Qwen3_5ForConditionalGeneration

model_id = "wflying/Qwen3.5-4B-RL-MATH"

model = Qwen3_5ForConditionalGeneration.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
)
processor = AutoProcessor.from_pretrained(model_id)

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "text",
                "text": "Solve the problem and put the final answer in \\boxed{}: What is the sum of the first 20 positive integers?",
            }
        ],
    }
]

inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    enable_thinking=False,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

with torch.inference_mode():
    generated_ids = model.generate(**inputs, max_new_tokens=512)

generated_ids = generated_ids[:, inputs.input_ids.shape[1]:]
print(processor.batch_decode(generated_ids, skip_special_tokens=True)[0])

Limitations

• This checkpoint is specialized for mathematical reasoning; performance may vary on unrelated tasks.
• The reported scores use non-thinking mode and may not transfer directly to other prompts, decoding settings, or evaluation implementations.
• The RLVR stage used text-only mathematical data. The model retains the multimodal architecture of its base model, but multimodal behavior was not separately optimized or evaluated in this post-training stage.
• Generated answers can still be incorrect. Verify outputs before using them in high-stakes settings.

License

This model inherits the Apache License 2.0 from the base model. Users must also comply with the applicable terms of the training dataset and dependent components.