d24-sft-v5-simplechat

d24-sft-v5-simplechat

d24-sft-v5-simplechat is an experimental 756,819,456-parameter English chat model for research on supervised fine-tuning and reinforcement learning. It is the canonical simple-ChatML SFT policy from the d24 v5 lineage.

All 756,819,456 parameters in the published safetensors file are BF16.

Lineage

1. ftajwar/d24-climbmix-100b: pretrained from scratch on about 100B tokens of ClimbMix.
2. ftajwar/d24-climbmix-dolmino-midtrain-100b: continued-pretrained for another approximately 100B tokens on 80% OLMo-3 Dolmino plus 20% ClimbMix replay.
3. This checkpoint: one epoch of SFT on the standard d24 conversation mixture, formatted with the simple ChatML template.

The model has seen approximately 200B pretraining/midtraining tokens before SFT. The v5 label identifies this training lineage; it is not a parameter-count label.

Architecture

The Hugging Face architecture is LlamaForCausalLM, but the model uses the GPT-2 tokenizer and the d24 architecture above. It is not a Llama-family pretrained checkpoint.

SFT data and training

The source conversations are the sfanm/d24-sft-mixture research mixture: SmolTalk, MMLU auxiliary data, GSM8K, spelling tasks, and identity examples. The local simple-ChatML preparation produced 227,443 packed training sequences and 7,381 packed validation sequences at length 2,048, with about 476M tokens across both splits.

Training used:

The final SFT validation language-model loss was 0.79675 (perplexity 2.2183). This is an in-distribution SFT validation metric, not a downstream capability or safety evaluation.

Chat format and stopping

The template is:

<|im_start|>user
...<|im_end|>
<|im_start|>assistant
...

<|im_end|> is a literal string represented by seven GPT-2 BPE tokens. It is not eos_token_id=50256 and is not a registered special token. Generation must stop on the string <|im_end|>; otherwise the model can continue into another turn until max_new_tokens.

Transformers usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "sfanm/d24-sft-v5-simplechat"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
).eval()

messages = [
    {
        "role": "user",
        "content": "Natalia sold clips to 48 friends in April and half as many in May. How many total?",
    }
]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(
    **inputs,
    max_new_tokens=512,
    do_sample=False,
    stop_strings=["<|im_end|>"],
    tokenizer=tokenizer,
)
response = tokenizer.decode(
    output[0, inputs.input_ids.shape[1]:],
    skip_special_tokens=False,
)
print(response)

For vLLM, use SamplingParams(stop=["<|im_end|>"]).

Intended use

• research on SFT, RLVR, GRPO/RLOO/maxRL, and small-model behavior;
• a warm-start chat policy for the d24 regular-RL recipes;
• controlled comparisons with other d24 SFT checkpoints.

The accompanying unified-training repository contains the reproducible cluster setup, current RL defaults, and algorithm baselines in docs/REGULAR_RL_TRAINING.md.

Limitations and evaluation caveats

• This is a small experimental research model and has not undergone production safety alignment or a comprehensive capability/safety evaluation.
• It can produce incorrect, incoherent, biased, or unsafe text. Verify outputs independently and do not use it for high-stakes decisions.
• The context window is 2,048 tokens.
• The SFT mixture includes GSM8K training examples. Do not treat GSM8K results from this checkpoint as a clean no-exposure benchmark. The separate local d24-sft-v5-nogsm8k-hf checkpoint exists for that control but is not part of this repository.
• Chat generation is incorrect unless the caller stops on the literal <|im_end|> string.
• The model and SFT mixture derive from multiple upstream datasets. The Hub metadata therefore uses license: other; review the source dataset licenses and terms before redistribution or downstream use.

Reproducibility

• SFT checkpoint: sft-d24-v5-simplechat/iter_0001777
• HF export: d24-sft-v5-simplechat-hf
• Chat template: src/nemotron/data_prep/templates/simple_chatml.jinja
• Training repository: yudasong/unified-training