stride-qwen3-4b-stabilized-2048-single_step-20260916

Qwen/Qwen3-4B-Instruct-2507: single_step, lower learning rate, warmup and KL experiment

STRIDE ablation using one pooled reasoning embedding per eligible response.

Training explicitly renders enable_thinking=False, recorded as thinking_mode: false in the scientific contract. The pinned tokenizer and chat template remain unchanged. Use the same explicit keyword at inference, particularly for Qwen3-1.7B, whose default template enables thinking. Every checkpoint in this repository requires explicit nonthinking provenance.

This public repository retains every published optimizer-update adapter, including update zero (the initial, untrained adapter). Training is planned for 4 epochs on the same 2,048-question training split as the earlier STRIDE runs. The global prompt batch is 64 questions with eight rollouts each (512 responses per update), giving 32 updates per epoch and 128 planned updates. Prompt plus response context is capped at 8,192 tokens; the random seed is 42. Completion is reported by actual entries in checkpoint_index.json; planned epochs do not imply that training has finished. No evaluation or superiority claim is made. Correct final answers do not verify every intermediate proof step.

This is a separate experiment initialized from the pinned base model with a fresh LoRA adapter, not a continuation of an earlier trained adapter. The configured STRIDE alpha is 1, separate from LoRA alpha 32; GRPO does not use the STRIDE diversity bonus. The peak learning rate is 2e-5, with 10 optimizer updates of linear warmup, followed by a constant learning rate. Update 1 uses 2e-6 and update 10 reaches 2e-5. Warmup is indexed by absolute completed optimizer updates, so exact resume does not restart it. A 0.01 KL coefficient penalizes drift from the frozen initial base policy using the sampled-token k3 estimator expm1(log_p_ref - log_p_policy) - (log_p_ref - log_p_policy). The KL penalty is aggregated over the same global generated-token denominator as the policy loss. This is the original GRPO k3 implementation, without the importance-ratio correction; it does not claim an unbiased exact reverse-KL gradient. These settings are intended to investigate stability; their effectiveness has not been established by this repository's existence.

Each immutable checkpoint-NNNNNN/ folder contains PEFT safetensors weights, adapter configuration, the tokenizer and chat template, training metadata, and a SHA256 manifest. The index records the optimizer step and completed epoch fraction. Every checkpoint has a separate Hub commit; local upload receipts record the exact commit to pin when downloading.

Reload a particular adapter

import torch
from huggingface_hub import snapshot_download
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

repo_id = "sandeep123/stride-qwen3-4b-stabilized-2048-single_step-20260916"
checkpoint = "checkpoint-000128"  # choose a published folder from checkpoint_index.json
commit = "main"  # replace with the checkpoint's immutable Hub commit SHA
root = snapshot_download(repo_id, revision=commit,
                         allow_patterns=[checkpoint + "/*"])
adapter_dir = root + "/" + checkpoint
tokenizer = AutoTokenizer.from_pretrained(adapter_dir)
base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-4B-Instruct-2507", revision="cdbee75f17c01a7cc42f958dc650907174af0554",
    torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(base, adapter_dir, is_trainable=False)
model.eval()
messages = [{"role": "user", "content": "Solve the supplied mathematical problem."}]
prompt = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)

The base model is pinned to cdbee75f17c01a7cc42f958dc650907174af0554 and is not included here. LoRA uses rank 16, alpha 32, dropout 0, no bias, and the q/k/v/o plus gate/up/down projection modules. Per-checkpoint metadata records the exact learning rate, warmup schedule, KL settings, rollout group size, prompt batch, epoch, seed and dataset hash. The public adapter config replaces the machine-local base path with the pinned Hub model ID; original local checkpoint files remain byte-for-byte unchanged.

Inference versus training resume

These are portable inference adapters. is_trainable=True also permits further adapter training with a newly initialized optimizer. Exact continuation of the original run additionally requires its local state_NNN optimizer/RNG files, matching adapter_NNN, manifest, training contract and learner topology. The latest required full resume pair is public under latest-resume/, including Adam optimizer state, per-rank RNG, the matching adapter, original scientific contract and frozen-source hash inventory. latest_resume.json identifies its step. The folder is replaced atomically at each required backup; older states may remain in Git history. Follow latest-resume/RESUME.md to download regular files, verify hashes and resume with the same environment, base model, data and four-learner topology. Extending the schedule beyond four epochs requires --allow-epoch-extension; all other scientific contract fields must remain identical. The matching training code is retained separately and is not published here. Training questions, rollouts, credentials and arbitrary files are excluded.

Uploads occur only after the local state completion marker and adapter hashes validate. A checkpoint's files, manifest and index entry are committed atomically. The uploader independently verifies remote file sizes and hashes at that commit.

Verified server storage policy

Every optimizer-update LoRA adapter remains public. During normal training, full optimizer/RNG resume backups are uploaded at completed epoch boundaries (updates 32, 64, 96, 128). Initial recovery and controlled stop/final flush also back up the exact latest committed update. The newest two local full states are retained. Older optimizer states are removed as obsolete only after verifying the newest local full state, every adapter backup, and the required public full backup; this does not claim every intermediate optimizer state was uploaded. checkpoint_index.json and latest_resume.json deliberately report separate adapter and full-resume watermarks. See retention_policy.json.