Qwen3-8B-SDFT-Math-LoRA-new

Qwen3-8B-SDFT-Math-LoRA-new

Qwen3-8B (dense, 36 layers, 8.19B params) tuned for mathematical reasoning. This repo holds fully merged bf16 weights — the LoRA adapter has been folded into the base matrices, so it is a drop-in replacement for Qwen/Qwen3-8B-Base; no PEFT adapter loading is required.

Training pipeline

1. Base: Qwen/Qwen3-8B-Base.
2. SFT: supervised fine-tuning on a balanced oracle mixture (math / retrieval-augmented QA / tool-use dialogue).
3. Math RL, stage 1: GRPO on math problems with a verifiable answer-matching reward, training LoRA adapters (rank 16, alpha 32, scaling 2.0) on q/k/v, o_proj, gate/up_proj and down_proj of every layer. The stage-1 adapter was merged into the SFT weights.
4. Math RL, stage 2 (this checkpoint): RL continued from the stage-1 merged model for 60 more optimizer steps, again with a fresh LoRA adapter of the same shape, which is merged here.

Trained with slime on Megatron-LM (TP=4, CP=2, bf16).

Checkpoint details

Non-finite check: 0 NaN/Inf tensors across all shards.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "willamazon1/Qwen3-8B-SDFT-Math-LoRA-new"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16).to("cuda")

prompt = (
    "Question: Natalia sold clips to 48 friends in April, and then she sold "
    "half as many clips in May. How many clips did Natalia sell altogether "
    "in April and May?\nAnswer:"
)
ids = tok(prompt, return_tensors="pt").input_ids.cuda()
out = model.generate(ids, max_new_tokens=128, do_sample=False)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

This is a base-style (completion) model, not an instruction-tuned chat model: prompt it with Question: ... \nAnswer: style completions rather than a chat template.

Limitations

Stage 2 is a short RL continuation (60 steps), so the weight delta over the stage-1 model is small. The model inherits the biases and knowledge cutoff of Qwen3-8B-Base, and its math answers are not guaranteed correct — verify outputs before relying on them.