d24-sft-v5-nogsm8k

d24-sft-v5-nogsm8k

d24-sft-v5-nogsm8k is an experimental 756,819,456-parameter English chat model intended as a clean SFT-exposure control for GSM8K reinforcement-learning experiments. Its supervised fine-tuning mixture omits the complete 7,473-example GSM8K training component.

All 756,819,456 parameters in the published safetensors file are BF16.

Scope of the no-GSM8K claim: the explicit openai/gsm8k component was removed from supervised fine-tuning. This is not a claim that the much larger pretraining and midtraining corpora contain no GSM8K overlap, paraphrases, or similar grade-school mathematics. Other non-GSM8K SFT sources can also contain mathematical material.

The standard-mixture sibling, which did include GSM8K during SFT, is sfanm/d24-sft-v5-simplechat.

Lineage

1. ftajwar/d24-climbmix-100b: pretrained from scratch on about 100B tokens of ClimbMix.
2. ftajwar/d24-climbmix-dolmino-midtrain-100b: continued-pretrained for another approximately 100B tokens on 80% OLMo-3 Dolmino plus 20% ClimbMix replay.
3. This checkpoint: one epoch of SFT on the d24 conversation mixture after removing the GSM8K component, formatted with the simple ChatML template.

The model has seen approximately 200B pretraining/midtraining tokens before SFT. The v5 label identifies this training lineage; it is not a parameter-count label.

What was removed

The no-GSM8K corpus was derived from the standard sfanm/d24-sft-mixture source mixture. The standard and filtered simple-ChatML preparation summaries differ by exactly 7,473 source conversations, the size of the GSM8K training split used by the standard mixture:

Both preparations used sequence length 2,048, first_fit_shuffle, seed 42, the GPT-2 tokenizer, and the same simple ChatML template. Removing source conversations changes packing boundaries, so packed-sequence differences are not expected to equal the source-conversation difference.

Architecture

The Hugging Face architecture is LlamaForCausalLM, but the model uses the GPT-2 tokenizer and the d24 architecture above. It is not a Llama-family pretrained checkpoint.

SFT training

The final SFT validation language-model loss was 0.797625 (perplexity 2.220262). This is an in-distribution SFT validation metric, not a downstream capability or safety evaluation.

Chat format and stopping

The template is:

<|im_start|>user
...<|im_end|>
<|im_start|>assistant
...

<|im_end|> is a literal string represented by ordinary GPT-2 BPE tokens. It is not eos_token_id=50256 and is not a registered special token. Generation must stop on the string <|im_end|>; otherwise the model can continue into another turn until max_new_tokens.

Transformers usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "sfanm/d24-sft-v5-nogsm8k"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
).eval()

messages = [
    {
        "role": "user",
        "content": "Natalia sold clips to 48 friends in April and half as many in May. How many total?",
    }
]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(
    **inputs,
    max_new_tokens=512,
    do_sample=False,
    stop_strings=["<|im_end|>"],
    tokenizer=tokenizer,
)
response = tokenizer.decode(
    output[0, inputs.input_ids.shape[1]:],
    skip_special_tokens=False,
)
print(response)

For vLLM, use SamplingParams(stop=["<|im_end|>"]).

Intended use

• controlled studies of learning GSM8K through RLVR without explicit GSM8K examples during SFT;
• comparisons against the GSM8K-exposed v5 simplechat sibling;
• research on GRPO, RLOO, maxRL, SFT, and small-model behavior.

Limitations

• The no-GSM8K designation applies only to the explicit SFT mixture component; it does not prove global benchmark decontamination across pretraining and midtraining.
• This is a small experimental research model and has not undergone production safety alignment or a comprehensive capability/safety evaluation.
• It can produce incorrect, incoherent, biased, or unsafe text. Verify outputs independently and do not use it for high-stakes decisions.
• The context window is 2,048 tokens.
• Chat generation is incorrect unless the caller stops on the literal <|im_end|> string.
• The model and SFT mixture derive from multiple upstream datasets. The Hub metadata therefore uses license: other; review source licenses and terms before redistribution or downstream use.

Reproducibility

• SFT checkpoint: sft-d24-v5-nogsm8k/iter_0001773
• HF export: d24-sft-v5-nogsm8k-hf
• Packed corpus: sft-pack-d24-nogsm8k
• Chat template: src/nemotron/data_prep/templates/simple_chatml.jinja
• Training repository: yudasong/unified-training