Gwimi-4-12B-IT-BF16

Gwimi-4-12B-IT-BF16

Gwimi-4-12B-IT-BF16 is a merged BF16 release of a Gemma 4 12B instruction model that underwent two consecutive post-training stages:

1. Supervised Fine-Tuning (SFT) on a 20,000-example curated reasoning mixture.
2. Group Sequence Policy Optimization (GSPO) on 12,000 frozen reinforcement-learning prompts with programmatic correctness, formatting, and anomaly rewards.

The final release contains the original BF16 base weights with the cumulative SFT + GSPO LoRA updates merged into them. No separate adapter is required for inference.

Model repository: trjxter/Gwimi-4-12B-IT-BF16

---

Model summary

The GSPO phase continued training the same cumulative adapter produced by SFT. This was not a second independent adapter stacked on top of the SFT adapter.

The final merge can be summarized as:

Exact Gemma 4 12B BF16 base
        +
Cumulative SFT + GSPO LoRA update
        =
Standalone Gwimi-4-12B-IT-BF16 model

---

Training pipeline

unsloth/gemma-4-12b-it
        ↓
20,000-row reasoning dataset
        ↓
Supervised Fine-Tuning
        ↓
SFT adapter
        ↓
12,000-prompt frozen RL dataset
        ↓
GSPO reinforcement learning
        ↓
Checkpoint 2250
        ↓
Merge into exact original BF16 base
        ↓
Gwimi-4-12B-IT-BF16

---

Stage 1: Supervised Fine-Tuning

SFT objective

The SFT stage trained the model to imitate high-quality reasoning responses across mathematics, science, coding, debugging, technical reasoning, general reasoning, and multilingual STEM material.

The model was trained for one epoch using a large rank-128 rsLoRA adapter while the base model was loaded in 8-bit precision.

SFT dataset composition

The final SFT corpus contained exactly 20,000 examples.

Curated sources

Kimi K2.5 top-up source

The remaining 7,640 examples were selected from:

Jackrong/Kimi-K2.5-Reasoning-1M-Cleaned

Final SFT dataset arithmetic

12,360 curated rows
+ 7,640 Kimi K2.5 top-up rows
= 20,000 total rows

The dataset-building process used normalized-prompt deduplication. Curated rows were given priority over top-up rows when a duplicate was found.

The final top-up selection had zero normalized-prompt overlap with the curated set.

SFT train/evaluation split

SFT hyperparameters

SFT evaluation design

The 2,000 held-out examples were divided into:

• 100 fixed anchor examples
• 1,900 examples in a rotating evaluation pool

During training, evaluation was scheduled every 250 optimizer steps using:

100 fixed anchor examples
+ 100 rotating examples
= 200 examples per scheduled evaluation

The fixed anchor set made checkpoint-to-checkpoint comparisons more consistent. The rotating slices increased coverage without requiring a full 2,000-example evaluation every 250 steps.

A final evaluation was then run over all 2,000 held-out examples.

Recorded SFT evaluation results

Checkpoint-1000 retrospective evaluation

These checkpoint-1000 results should not be confused with the final model evaluation.

Final full held-out evaluation

These are teacher-forced SFT evaluation metrics. They measure prediction quality on held-out target responses and are not equivalent to free-generation benchmark accuracy.

---

Stage 2: GSPO reinforcement learning

What GSPO means in this run

GSPO stands for Group Sequence Policy Optimization.

For each prompt, the policy generated multiple candidate completions. Programmatic reward functions scored those completions, and the trainer compared their reward values within the prompt group.

The defining GSPO behavior in this run was:

importance_sampling_level = "sequence"

This uses one importance ratio per generated response sequence rather than a separate ratio for every token.

The implementation used TRL's GRPOTrainer and GRPOConfig, but sequence-level importance sampling made the optimization GSPO-style.

Frozen GSPO dataset suite

The RL dataset was frozen before training and stored as reproducible artifacts.

Training split

Anchor evaluation split

Held-out evaluation split

Protected SciQ test split

The frozen dataset suite had zero cross-split prompt overlap.

The GSPO training worker used only the 12,000-row training split. The anchor, held-out, and protected SciQ splits were reserved for evaluation and contamination control.

Reward functions

The training run used three frozen reward functions:

correctness_reward_func
format_reward_func
anomaly_reward_func

They were loaded from the frozen reward module:

gemma_native_rewards_v2_mathverify.py

Correctness reward

The correctness reward evaluated whether the generated answer matched the expected solution criteria. Mathematical answer comparison used a parser-oriented verification approach rather than relying only on literal string equality.

Format reward

The format reward encouraged the response structure expected by the training pipeline, including a valid final-answer structure and properly formed output.

Anomaly reward

The anomaly reward was designed to detect suspicious, malformed, repetitive, degenerate, or parser-exploiting output patterns.

The reward functions were frozen before the official run so that the optimization target could not silently change during training.

GSPO hyperparameters

The KL penalty was intentionally disabled. A zero KL value in this configuration does not mean that the policy was identical to a reference model; it means that KL regularization was not part of the objective.

GSPO stopping point

The official run stopped at:

global_step = 2250

The run had originally resumed from checkpoint 450, so the successful continuation added approximately 1,800 optimizer updates.

By the stopping point, the run had processed approximately 15.55 million tokens according to the training telemetry.

The decision to stop at 2250 was based on:

• stable entropy rather than continued collapse;
• controlled completion lengths;
• a learned and saturated format reward;
• active but non-saturated sequence clipping;
• continued reward variation;
• complete resumable checkpoints;
• and diminishing expected benefit relative to additional compute cost.

Near-terminal GSPO telemetry

The values below are a near-terminal W&B snapshot around global step 2250. They describe the final observed training batch or local window, not an external benchmark score.

How to interpret these metrics

Reward: Higher is generally better only when the reward implementation correctly reflects the desired behavior. Reward is not equivalent to standardized benchmark accuracy.

Reward standard deviation: Some variation is necessary because GSPO learns from differences among completions in the same prompt group.

Fraction of zero-variance groups: A high value means some prompt groups produced no relative reward signal. The metric was noisy because each rollout contained only three unique prompts, so possible values naturally moved in increments of one third.

Entropy: Entropy is not simply “higher is better.” In this run it declined and then stabilized around 0.09 rather than continuing toward zero. That was treated as evidence against an obvious late-stage policy collapse.

Sequence clip ratio: Some clipping is expected in GSPO. The near-terminal value showed that the trust region was active without clipping the entire rollout.

Completion clipped ratio: This is different from policy-ratio clipping. It measures completions that reached the maximum-length condition. Because truncated completions were masked from policy loss, a high value would reduce training efficiency.

Gradient norm: The logged value was above 1.0 before clipping. The configured maximum gradient norm constrained the actual optimizer update.

Important GSPO evaluation note

The online GSPO worker used:

eval_strategy = "no"

The frozen anchor, held-out, and protected SciQ splits were preserved, but they were not used as periodic online evaluations during the expensive generation loop.

Therefore:

• the SFT loss and perplexity results above are official recorded evaluation results;
• the GSPO values above are training telemetry;
• no claim is made here that the final model has already beaten the base model or SFT-only checkpoint on a standardized external benchmark;
• a proper comparison should evaluate the base, SFT-only checkpoint, and final SFT+GSPO model under the same prompts, decoding settings, and scoring implementation.

This distinction is intentional. Reward curves are useful for diagnosing RL training, but they are not substitutes for independent benchmark evaluation.

---

Validation and artifact integrity

Checkpoint validation

The final checkpoint contained:

adapter_config.json
adapter_model.safetensors
trainer_state.json
optimizer.pt
scheduler.pt
rng_state.pth
training_args.bin
tokenizer.json
tokenizer_config.json
processor_config.json
chat_template.jinja

trainer_state.json reported:

global_step = 2250

The final adapter had:

LoRA rank:       128
LoRA alpha:      256
rsLoRA:          true
Adapter size:    1.954 GiB
Trainable params: 524,550,144

Exact-base merge validation

The final adapter was merged into the exact untouched BF16 base revision:

0f03bc8c5a5d620f219daea1ef93e05775ae9b74

The original base and merged model had the same byte size because the merge changed existing BF16 tensor values rather than adding a second set of parameters:

Base size:   23,919,549,408 bytes
Merged size: 23,919,549,408 bytes

The SHA-256 hashes were different:

Base SHA-256:
5a84cb313260ac447237b890387116dfa8682e49a6b44bc585ae8353abbff18d

Merged SHA-256:
1e024792bf994c200fc7757621d202eb2bb2ba11593afcf6a6a98ab6bb9c4845

This verifies that the uploaded BF16 model contains modified cumulative SFT + GSPO weights and is not an unchanged copy of the original base model.

---

Intended uses

This model is intended for experimentation with:

• mathematical and scientific reasoning;
• technical question answering;
• coding and debugging assistance;
• long-form structured reasoning;
• instruction following;
• local inference research;
• comparison of SFT-only and SFT+RL post-training pipelines.

The model is best treated as a research and development release rather than a guaranteed production system.

---

Limitations

1. No independent final benchmark suite is reported yet.
   The README separates SFT evaluation loss, GSPO training telemetry, and future independent evaluation.

2. Reward optimization can inherit reward-function blind spots.
   Correctness, formatting, and anomaly rewards cannot measure every aspect of answer quality.

3. The GSPO stage focused heavily on verifiable mathematics, science, and structured reasoning prompts.
   Performance may vary outside those distributions.

4. Long-context behavior was not independently benchmarked after GSPO.
   SFT used a maximum sequence length of 32,768, while GSPO used a 4,096-token runtime window with a 1,280-token completion cap.

5. Text behavior was the focus of this post-training project.
   Any other capabilities inherited from the base model were not independently evaluated here.

6. The model can still hallucinate, make calculation errors, produce unsafe advice, or follow incorrect premises.
   Outputs should be verified for high-stakes use.

7. Training reward should not be interpreted as benchmark accuracy.
   The terminal reward values describe the model under the training reward system and rollout distribution.

---

Loading the BF16 model

This model was exported through Unsloth's Gemma 4 unified implementation.

At export time, the working stack included:

Unsloth:       2026.6.1
Unsloth Zoo:   2026.6.5
Transformers:  5.10.2 compatibility build
PyTorch:       2.10.0 + CUDA 12.8

A Transformers installation that does not recognize the gemma4_unified architecture may fail to load the model. Use a current compatible Unsloth and Transformers environment.

Example with Unsloth

import torch
from unsloth import FastLanguageModel

model_id = "trjxter/Gwimi-4-12B-IT-BF16"

model, processor = FastLanguageModel.from_pretrained(
    model_name=model_id,
    max_seq_length=4096,
    dtype=torch.bfloat16,
    load_in_4bit=False,
)

FastLanguageModel.for_inference(model)

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "text",
                "text": "Solve the problem carefully and provide a clear final answer: What is 17% of 240?"
            }
        ],
    }
]

inputs = processor.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

with torch.inference_mode():
    output_ids = model.generate(
        **inputs,
        max_new_tokens=512,
        do_sample=True,
        temperature=0.7,
        top_p=0.95,
    )

generated_ids = output_ids[0, inputs["input_ids"].shape[-1]:]

print(
    processor.decode(
        generated_ids,
        skip_special_tokens=True,
    )
)

For deterministic evaluation, use greedy decoding or a low-temperature configuration and keep decoding parameters identical across compared models.

---

Recommended evaluation approach

For a meaningful comparison, evaluate all three checkpoints under identical conditions:

1. Original Gemma 4 12B instruction base
2. Gwimi SFT-only checkpoint
3. Final Gwimi SFT + GSPO model

Keep the following fixed:

• prompt set;
• prompt formatting;
• chat template;
• maximum generated tokens;
• temperature and top-p;
• random seeds;
• answer extraction;
• reward and benchmark scoring;
• hardware and inference backend where practical.

Useful evaluation categories include:

• exact-answer mathematics;
• scientific multiple choice;
• code generation and debugging;
• instruction-format compliance;
• long-response termination behavior;
• repetition and anomaly rate;
• response length;
• pass@1 and sampled pass@k;
• qualitative reasoning review.

---

Training and release hardware

---

Acknowledgements

This release builds on:

• the Gemma model family;
• Unsloth;
• Hugging Face Transformers, PEFT, TRL, and Hub;
• Math-Verify;
• the dataset authors and teacher-model outputs represented in the SFT and GSPO data mixtures.

---

License

This model is a derivative of Gemma and remains subject to the applicable Gemma license and terms of use.

Users are responsible for reviewing the upstream license and ensuring that their intended use complies with it.